Film Industry 4.0 and the Digital Motion Picture Factories. 2017

ENG
Analytical Reports
Film Industry 4.0 and the Digital Motion Picture Factories. 2017

Film Industry 4.0 and the Digital Motion Picture Factories. 2017

Abstract.

Today, the audiovisual industry as the all world is on the verge of a new Fourth Industrial Revolution.

The production units of this revolution, Film Industry 4.0, will be digital motion picture factories—the systems of integrated technological solutions that provide for digital models of audiovisual components (objects) of the work and the combination and assembly of these models into an audiovisual work in accordance with an artist's general intention.

If the current legislation defines a film as "an audiovisual work... consisting of images fixed on film or other types of carriers combining them into a thematic whole of interconnected consecutive frames...", then the production of a digital motion picture factory is a film as an aggregate and combination of digital models of audiovisual objects and scenarios of action and interaction that are united by the author’s artistic concept.

Keywords. Fourth Industrial Revolution, Film Industry 4.0, digital motion picture factory, digital model, technological platform, multi-modal audiovisual content

Introduction

In 2016, Klaus Schwab, President of the World Economic Forum in Davos, wrote in the foreword to his book, The Fourth Industrial Revolution, that "we are at the beginning of a revolution that is fundamentally changing the way we live, our work and relate to one another. In its scale, scope and complexity, what I consider to be the fourth industrial revolution is unlike anything humankind has experienced before."^[1]

The concept of the Fourth Industrial Revolution has already been the subject of numerous papers.^[2]^[3]^[4]In many countries, state programs for the development of national industry have been adopted in the paradigm of a new industrial structure. In Germany in 2011, during the Hanover exhibition, a national program for technological development, which was named "Industry 4.0", was launched and became synonymous with the Fourth Industrial Revolution. In the United States, the American Manufacturing Partnership project, in which industry, universities and the US federal government collaborated in the development and implementation of breakthrough technologies, was launched. In the UK, the transition to a new industrial revolution is underway within the framework of programs such as The Knowledge Transfer Network and High Value Manufacturing Catapult. In the Russian Federation, the National Technological Initiative, which is the government program of measures to support the development of promising industries over the next twenty years, can become the foundation of the world economy.

Economists distinguish three "industrial revolutions" in the history of mankind. At the end of the 18th century, the transition from manual labor to mechanized machine tools and the energy of water and steam marked the First Industrial Revolution. The advent of the first production lines—conveyors (for example, G. Ford’s conveyor)—and the widespread use of electricity prompted the Second Industrial Revolution at the end of the 19th century. The development of electronics and information technology, programmable controllers and automation in the early 1970s laid the foundation for the Third Industrial Revolution.

Industrial production, which is based on the total digitization of all processes, digital modeling and the creation of virtual "digital twins" of physical objects of production, enables us to discuss the new revolution, the Fourth Industrial Revolution. The production of the Fourth Industrial Revolution is based on cyber-physical systems that consist of various natural objects, artificial subsystems and controllers, which enable it to represent the new combination as a single whole. In cyber-physical systems, a close relationship and coordination between computing and physical resources exists.^[5]

The components of the Fourth Industrial Revolution are included in the development concepts of the automobile, shipbuilding, aerospace and aviation industries, nuclear power engineering, and the industry of building houses and other infrastructure facilities.

Despite the rather short lifetime of the industry—120 years since the first show by the Lumiere brothers—how can we discuss the emerging new concept of the film industry as the Film Industry 4.0?

The production of motion pictures and all audiovisual content is inextricably linked to technology. The emergence of photography, the first optical theaters and attractions, and cinematography—the first film cameras and projection cameras (for the Lumiere brothers, for example, they comprised the same device) based on a manual drive—the simplest optical-mechanical mechanisms and lighting systems can be described within the logic of the First Industrial Revolution (mid-18th century - early 20th century).

The emergence of the electric drive, the "separation" of the shooting and projection devices into two branches of design and technological development, complication of the production processes of the cinematic art, and the widespread use of conveyor production directly impacted not only picture-taking, projection and other film technology but also the introduction of the principles of conveyor production in the technological processes (production of film, for example), the separation of technological functions inside the gigantic motion picture studios by the first specialized enterprises—laboratories for film processing and printing and research units—and the emergence of national cinema chains became the basis of the Second Industrial Revolution in the film industry, which is characteristic of^[6] the industrial cycle of the film industry development from the late 1910s to the early 1970s in the United States and during the mid-1990s in Russia.

The Third Industrial Revolution in the film industry was marked by the introduction of the first electronic computing systems in film production. Special effects in pictures, electronic editing of movies, the use of television technologies in production (from video control systems to tape recording), and further digitalization of all processes from filmmaking to the management of the motion picture production process. During the Third Industrial Revolution in the film industry, the system of multiplexes and transnational networks of cinemas that currently dominate was formed. A characteristic feature of the Third Industrial Revolution in the film industry was the splitting of gigantic motion picture studios into isolated small-service companies that specialized in particular fields of film production—sound studios, special effects studios, and rental agencies for equipment. In practice, these independent companies, which were united by a common technological process, collaborated according to the cluster principle and geographically cooperated in the national traditional centers of cinematography—Hollywood, New York, London and Moscow. The Third Industrial Revolution became the basis for a new cycle of differentiation in the film industry, beginning in the USA in the 1970s and in Russia in the mid-1990s, to be completed by 2020-2025.

Today, the film industry is on the verge of a new Industrial Revolution. The first harbingers of this revolution are visible—digital modeling in the production of computer games and computer animation, and digitalization of the processes of capturing (include the light-field capture technologies), processing, distribution, storage, and projection of movies and audiovisual works—from digital cameras and digital film production systems to digital cinema projection and digital cinemas (and even the Field-of-Light Displays technologies). Digital technologies are becoming elements of the production process of a motion picture that pervade the whole life cycle of a film. And this is not only about production technologies, but about new technologies of storytelling – so called the hybrid content, where the traditional audiovisual essences are combined with software-based solutions as the applications.

However, the total introduction of digital technologies into the cinema process is not a new industrial paradigm. The concept of the Fourth Industrial Revolution in the film industry is more profound. This concept is not concerned with the emergence of virtual, augmented and mixed reality technologies, although they will find their place in the new technological paradigm. The technologies of virtual and augmented reality can generate new technological solutions for the film industry; however, they cannot be a revolutionary change in the paradigm of production and perception of audiovisual content.

Is the transition from film—from the analog image and sound to digital images and phonograms—a revolution? I do not think so. If we follow the chain of transitions from ancient petroglyphs—rock carvings—to drawings on paper to photographs and the first attempts at mass replication of an image to the sequence of photographs that fix the phases of movements to film, even in digital form, film will remain a fixed sequence of image frames. The transition from film to digital is a revolution in form but not a revolution in content. The transition to the paradigm of a new industrial revolution must be radical and significant that fundamentally changes the essence of film production. The main revolution is just beginning. We are on the verge of moving from the fixed images on a carrier to digital models of objects, changing film's paradigm to film as a combination of models of digital objects and a combination of digital scenarios. The models of the interactions among these objects and between the objects and the viewer, united by a common artistic, semantic composition of the author's intention. The visualization of the combinations of these digital models will be directly performed at the time of playback, depending on the type of playback device and considering the scenarios of emotional perception by various social groups of viewers. Researchers from BBC R&D and MIT call this paradigm as the object-based media.^[7] I consider this realization to be a revolution!

If the current legislation defines a film as "an audiovisual work... consisting of images fixed on film or other types of carriers combining them into a thematic whole of interconnected consecutive frames...",^[8] then the production of a digital motion picture factory is a film as an aggregate and combination of digital models of audiovisual objects and scenarios of action and interaction that are united by the author’s artistic concept.

Digital Motion Picture Factories

How the classic components of current motion picture studios for the production of motion pictures can be presented as components of digital motion picture factories in the Film Industry 4.0 paradigm?

At the heart of digital filmmaking—the digital motion picture factory—is an ideology of creating sets of static and dynamic digital models of the objects as film components; the film is a combination and assembly of these digital models and scenarios. The digital modeling of the components of films is a logical development of current technologies—the technologies for the production of computer games, computer animation, the capture and scanning of objects and images, digital post production, and visual effects (VFX). Many of these technologies are effectively employed in modern traditional filmmaking. To produce a new platform and a new paradigm of the film industry, these technologies should be assembled into a single "package of technologies", which will be a key factor in the transition to a new industrial paradigm.

Modern technologies make it possible to employ digital actors and digital twins of actors with a high cinematic reliability. The technology of motion capture enables the creation of digital masks that convey all nuances of facial expressions and body movements of not only man but also animals, both real and imaginary. The use of these technologies is widespread in the film industry. For example, these technologies were used by D. Cameron for "Avatar", and by director Ridley Scott, who had to replace actor Oliver Reed, who died during the filming of “Gladiator” in 1999, with his digital counterpart that was created using a double and computer-generated imagery. In 2004, Sir Lawrence Olivier appeared in the movie "Sky Captain and the World of Tomorrow". In the movie "Fast and Furious 7", the lead actor Paul Walker, who perished in a car crash during the filming, was replaced with a digital copy made with the participation of the actor’s brothers Caleb and Cody, whose faces were altered by a digital mask created by scanning images of Paul Walker.^[9] In "Rogue One: A Star Wars Story", the digital counterpart of actor Peter Cushing, who died 23 years earlier, appeared in the role of Tarkin.^[10]

The use of digital copies of actors is prevalent in advertising. In 2013, young Audrey Hepburn "starred" in a chocolate commercial. Two actresses were employed for the digital copy—the first actress had an identical figure, whereas the second actress had a lookalike face, which according to journalists was a blank canvas for the digital copy of the young Audrey Hepburn. In China, a controversy was caused by a commercial in which a realistic digital copy of the actor Bruce Lee was employed. In the process of filming "The First Avenger. Another war", all principal actors were scanned for the use of "digital doubles" in complex stunts. The controversy included the use of "digital stuntmen" in film production.

"Digital actors" may have the principal task of actors’ casting tests for the role. The actor's tests will involve the selection of a digital model of the actor fitting the director's intention. Digital models of actors address the challenge of shooting and producing mass scenes; the technology of "multiplication" of extras is prevalent in current film production.

It is important to emphasize that it is not about complete replacement of "live" actors with their digital models, although these creative techniques will be common in filmmaking. It is about the use of digital scanning technologies and digital actor modeling to expand the artistic capabilities of the authors of audiovisual content, primarily those associated with adaptive, multimodal moving picture playback.

The extensive use of technologies for creating digital copies of actors produces significant problems in the field of copyright law. Lawyers have not regulated the relationship between the actor who serves as a prototype for a digital model, the artists and programmers who create the digital model of the actor, and the director who is responsible for the acting of the digital double.

The digital modeling of actors will lead to change of the long-established "stardom" as an institution and a review of marketing strategies for promoting audiovisual content.

The "digital actor" technology will fundamentally change the work of make-up artists in cinema; however, training machine intelligence to match the talent of make-up artists will remain challenging although algorithms will substantially simplify work regarding make-up for digital actors.

The development of digital modeling will facilitate the creation of any digital model of clothing of any epoch and any culture. This issue relates to not only the creation of static digital clothing but also the algorithms of the clothes’ "behavior" on the body of a digital actor, which will require the development of numerous algorithms and the consideration of all factors that affect this "behavior."

Similar to a digital wardrobe, the digital models of props—any physical objects used in the shooting of a film: furniture, various utensils, machines, and mechanisms will be necessary for making the film. New tasks and opportunities for creating uncountable digital models of props are created, which will be not only the outcome of creative approaches but also the objects of a new digital economy of film production. Even if you need to shoot physical objects (props and decorations), the use of digital prototyping and additive 3D-printing technologies enable almost any digital model to be materialized.

The effects of explosions, firing, and smoke in modern digital filmmaking is not a new concept. Within the framework of the digital motion factories, new digital models and new algorithms that can achieve realism, indistinguishable from reality, will be developed.

In the field of digital film sets, we can expect colossal changes in the paradigm of the digital motion picture factory. Even today, the technological level of computer modeling enables directors to shoot large-scale pictures without leaving a small room. The action of the entire "Life of Pi" film directed by Ang Lee takes place in the open ocean, and all filming was studio work. The nature in "The Jungle Book" is fully "painted" on a computer screen. As the guru of special effects, Rob Legato, a VFX supervisor and cameraman in D. Cameron’s company Digital Domain, mentioned the concept of photorealistic cinematography in his speech at the Next Generation Cinema NABShow-2017 conference. When we see a synthesized digital image of an object that does not exist in reality, we understand that this image is not real, because these objects do not exist in our minds, for example, the planet Pandora in D. Cameron's film "Avatar." For real objects—the ocean, the forest, the jungle and even a whole part of Moscow in the film "Attraction" by F. Bondarchuk, we are not sure whether an object is a real object or its digital model. The boundary between the representation of a real object and a virtual object is not visible today. According to Rob Legato, modern VFX technologies blur the line between special effects and traditional production. Computer modeling is becoming very realistic; the differences between special effects and reality are not easily detected.

Digital models of scenery actively penetrate domestic film production. In the previously mentioned F. Bondarchuk's film "Attraction", professionals from the Moscow company Main Road|Post created by scanning a digital 3D model of the real Moscow Chertanovo neighborhood, where the entire film unfolds. The created image was so realistic that even the inhabitants of the Chertanovo district did not notice that the real image was substituted with a digital model. In the concept of the Film Industry 4.0, any location, regardless of the geographic remoteness of the location, can be recreated in the form of a multidimensional digital model. The issue extends beyond modeling modern buildings and avenues to digital models of all paraphernalia of urban environments of different (!) epochs, from signboards and posters to clothes, cars, personal effects and other artifacts. Some national libraries of digital models of different regions around the world may be created. These libraries due to a joint effort of various teams will require additional digital standardization and certification of digital models and the development of the semantics of digital scenarios and processes.^[11] These projects are already under way in industry, for example, digital models of large metropolises are being created. These models prevent the need for film crews to travel to remote regions for filming on location. Some of the funds currently earmarked for "rebate programs" can be allocated for the support of regional film production. In Film Industry 4.0, geographic remoteness will not be an obstacle for joint filmmaking!

Digital technologies in sound recording and sound design of films have been employed for more than 35 years. Modern multi-dimensional surround sound systems, such as Dolby Atmos™ and Barco IOSONO™, are already based on the concept of virtual sound objects and the synthesis of the sound field. However, the concept of Film Industry 4.0, which is based on total digital modeling, will introduce new opportunities for sound work in the movies, from the technology of digital synthesis of sound effects, the automatic adaptation of sound solutions based on the analysis of big data and the imagery of the film, to the new technologies of sound field synthesis during the playback of a film. For example, the unique noise of the closing door of an ordinary car in the movies still requires recording the sound of a real car door (or its imitation) in a sound recording studio. The use of digital image recognition will enable the generation (or selection from an existing library) of sound that is ideally suited for this frame.

The development of machine translation from one language to another language will create radically new opportunities for localizing the content for different language groups of the viewers and interactive communication among these language groups either directly with the content or each other.

Digital image modeling, which is extensively employed in 3D-animation, enables the application of certain properties of objects, for example, texture types and surface reflection parameters, various positions of light sources and virtual cameras, can be employed in modeling the sound palette of a movie. The wave nature of light in its physical laws is close to the nature of sound waves— the sound properties of surrounding objects (sound-absorption and sound reflection) can be set in the digital model of an episode to be filmed. Sound sources instead of lighting sources, and virtual micro phones instead of virtual cameras can be installed to obtain a new system for digital modeling of the sound solutions of a film. This area comprises an extensive field of activity for the developers of libraries of standard digital models of sound objects, scenarios and processes.

The extensive use of digital technologies renders special effects a pervasive concept of the entire film production cycle. In a world where a film is a combination of digital models, the film is the result of special effects shooting! Even modern technologies for creating visual effects and color correction enable a high cinematographic realism. The application of these technologies in the concept of digital modeling of a film will be extensively developed in the framework of Film Industry 4.0.

If I refer to completely digital models as components of digital motion factories, new filmmaking equipment is a vivid example of a real cyber-physical system that connects the virtual world and the physical world based on digital scanning of filmed objects, Internet communications and digital networking in multi-camera shooting. The basis for the filming technology of Film Industry 4.0 may be light field cameras, which are emerging on the market, and cameras based on recording the quantum radiation of physical objects. These cameras allow the transformation of real filmed scenes with the participation of real actors, real objects and scenery into digital scenes and digital "replicas".

The mass-scale development of unmanned aircraft and vehicles—drones and robots—creates new opportunities for digital modeling of a film. The tasks of scanning the real objects of photography to the full automation of all parameters of real camera motion are needed to implement the creative solution of the director of a film and its director of photography.

The development of the light field capturing technologies creates new possibilities for modeling the lighting in the digital model of a motion picture. The first algorithms for image processing workstations, which were created by the German Fraunhofer Institute, demonstrate great potential for filmmakers in control of the lighting, re-focusing the image, changing the shot angle, and creating the depth manuscripts for 3D images in the digital space of a filmed or simulated movie scene.^[12]

Thereby Digital modeling technologies for films and audiovisual content as a combination of digital objects and scenarios—have become the backbone, all-pervasive and all-encompassing technologies of Film Industry 4.0.

Digital motion picture factories are structural resource units, production media for the digital models of objects, scenarios and processes, and the creation as a combination of these digital models based on audiovisual content. Some digital motion picture factories produce an extensive range of digital models of various types and combine them into films. Other digital factories provide digital services, specialize in the production of a certain type of digital model (objects, scenarios or processes), create libraries of standard digital models, combine ready-made models into films, or provide digital services to other digital film factories—data storage services, computational services, and infrastructural services for film viewing (e.g., movie theaters and other facilities of audiovisual communication). Some digital film factories specialize in developing process management technologies (Fig. 1).

Figure 1. Sample structure of the digital motion picture factory of Film Industry 4.0.

Total digitalization of the entire production process will require significant computing power and new technologies for distributed computing, for example, fog calculations. The entire life cycle of the audiovisual content will be based on the continuous analysis of large data sets—big Data—generated for the production of the content during its production and the process of its consumption. Computing power and big data will become the "oil and gas" of the new industrial revolution.

The processes of combining and assembling digital models of objects, scenarios and processes into an integral audiovisual product—a film—will be arranged according to the nesting doll principle: separate static and dynamic models of objects will be combined into composite digital models that conform to the artistic presentation of a movie scene and unified by certain scenarios and processes as an integrated form of the film (Fig. 2). The nesting doll topology of these combinations in an audiovisual product—embedding of digital models into other digital models—may take the form of both a single representation of an integral digital manifold model unified by the scenario and a separate non-manifold representation but will be correlated with the scenarios and process models.

Figure 2. “Matryoshka” architecture of an audiovisual content in Film Industry 4.0.

What will be the basis for these digital motion picture factories? Both the technologies and their "assembled packages" directly determine the transition to the new industrial level. The basis on which these technologies develop is also important—the organizational structure of the productive forces. The combination of these two components—the technologies and organizational structures of production—with the energy of entrepreneurship triggers each new industrial revolution.

Organizational Structures

In his research on the origins of industrial revolutions, P. Shchedrovitsky notes that a special "cell" corresponds to each industrial revolution, namely, the organizational structure of cooperation, which serves as the basis of the current production system. For the First Industrial Revolution, for which a typical example was a Dutch shipyard, such a cell was a "cluster"—a geographically integrated group of small manufacturing workshops that specialized in one or more stages of commodity production, usually of the goods to be used in the manufacturing of larger-scale products (for example, production of oakum for ropes to equip sea vessels). For the Second Industrial Revolution, which is exemplified by Arkwright’s spinning factory, the "factory"—a mechanized production facility that consists of numerous workshops that specializes in certain production operations—becomes such a cell. In the Third Industrial Revolution, "transnational companies" constitute the basis of the production system. "Technological platforms", according to P. Shchedrovitsky, will serve as the basis for the Fourth Industrial Revolution.^[13]

From the standpoint of the evolution of the organizational forms of the film industry development, P. Shchedrovitsky's theory of "cells" fully applies to the sphere of cinematography. Despite their isolation, the first practically handcrafted photo to current film studios followed the logic of cluster development—being established around economic centers and metropolitan cities (access to financial and human capital) and in geographically favorable locations (access to cheap energy and sunlight)—which explains how the first movie clusters were formed in Hollywood, USA, and in Russia, Moscow.

The second industrial revolution in the film industry introduced a new organization of production—gigantic motion picture studios (studio complexes). Similar to the largest Hollywood studios, which occupied entire studios for their vast filming stages and production workshops, entire “movie cities” existed in the Soviet Union, such as Mosfilm, Lenfilm, Gorky film studio, and Sverdlovsk film studio.

A gigantic motion picture studio comprised the entire technological chain of the film production. These studios were essentially film-producing factories and were referred to as "dream factories" because the entire technological cycle of film production was concentrated in one place (with the exception of filming on location during expeditions).

With the Third Industrial Revolution (the early 1970s in the United States and the mid-1990s in Russia), the process of mass-scale formation of small specialized service companies began. This process initially occurred in gigantic film studios and then occurred elsewhere. For example, in the US at this time, in San Francisco but not in Hollywood, a famous special effects company was founded by J. Lucas—Industrial Light & Magic. In Russia, based on certain production workshops of film studios, private companies also began to emerge.

Despite the fact that they work in interconnected film production chains, these companies do not enjoy full-fledged digital interaction with one another. Their current interaction is rather informational. Main production operations are performed with fixed images and sequentially transferred from one company to another company as the production cycle runs. Part of the technological processes in modern film production can be performed in parallel, for instance, the processing and generation of images during the process of filming. However, the modern production process is linear: work on the sound design is usually conducted after a picture is edited. The need to make any changes in previously filmed material, for example, due to a new artistic solution of the director, causes a return to the film-making phase. The release of different versions of a film, which are adapted for viewing on the big screen of a movie theater or the screen of a TV or tablet, usually prompts the creation of different physical versions of the film.

In the new economic cycle of the film industry development, the integrated digital production of audiovisual content will be realized based on the full-scale digital networking of all participants in the production process at the level of the life cycle of digital models, whose combination comprises the film. Developers of algorithms, creators of digital models (objects), suppliers of computing power, and authors of artistic solutions that translate the whole idea of the film into an integral combination of these digital models will find themselves in a single network of interactions, whether geographically distributed or not (here the boundaries lose their original meaning). The main idea is that all participants in the process create not only digital models of objects of a film but also digital models that can interact with each other in a single space of the film. A significant role will be assigned to both digital certification technologies and interfaces for compatibility of all digital components of the film.

Production technologies, which are united by a single logic, philosophy, common standards and interfaces, represent a unique technological platform on which a specific audiovisual ecosystem is formed. In the Film Industry 4.0 paradigm, many platforms and ecosystems may be interconnected (open) and isolated (closed). In principle, the digital technology platform underlies the entire life cycle of the audiovisual content—from its conception to the perception by the spectator.

The new Fourth Industrial Revolution will be built on the foundation of a radically new organizational form of productive forces—on digital technological platforms.

Digital Technological Platforms

The technological platform is the basis of the digital ecosystem of technologies, which underlies the logic of Industry 4.0. Based on digital technological platforms, digital models of specific objects and scenarios and processes that underpin the life cycle of either individual digital objects or combinations of these digital objects, which representing the "upper" layer—the end product of digital factories of the Fourth Industrial Revolution—are developed. As P. Shchedrovitsky noted, the technological platforms democratize communication and cooperation and reduce the transaction costs for these processes which in the current stage comprise a more effective form of creation, accumulation, development and dissemination of knowledge than TNCs.^[13]

Companies such as Apple, Alphabet (Google), Microsoft, Amazon and Facebook, which have created digital technology platforms, are among the ten largest companies in the world by capitalization,^[14] because they have outperformed many companies in the oil and gas and financial sectors of the economy. These companies are the leaders of the Third Industrial Revolution! Note that all five companies realize global ambitions in the field of creation, distribution and storage of audiovisual content and have the opportunity to become the locomotives of Film Industry 4.0 while eliminating all old organizational forms of cooperation in traditional filmmaking.

Digital technological platforms of Film Industry 4.0 will also define the business models for the production, distribution and consumption of audiovisual content. The technologies for ensuring the security of the content (both the content and protection against its illegal use), models of financing and return on investments, and copyright management, including rights to digital virtual objects, will be based on distributed ledger technologies, i.e., blockchain technologies.

The transition from the concept of a film as a fixed sequence of images and sounds to the model of a film as a combination of digital models of objects, scenarios and processes, united by the author's artistic concept, creates new possibilities in the development of technologies for the perception (consumption) of audiovisual content.

Technologies for the Perception of Content and Film Industry 4.0

The diversity of the modern media of audiovisual communication, i.e., the means of presenting the content, has prompted the need for the adaptation of the content to each type of media. The author distinguishes three types of audiovisual communication media—screens on which the viewer directly consumes the audiovisual content:

- large-screen systems for collective viewing, including screens in movie theaters, video walls and street displays (stadium, etc.);

- screens intended for family/group viewing: TVs, computer monitors, and home theater systems;

- screens for personal/individual viewing: tablets, laptops, smartphones, and eyeglasses with built-in displays, including head-mounted displays of virtual, augmented and mixed reality.

These screen types require special versions of the audiovisual content. Playing a movie that is made for a home screen on a screen in the movie theater, or vice versa, playing a movie made for a widescreen theater on a smartphone screen usually causes a loss of interest by the viewer due to the inconsistency between the scale of the movie scene and the screen size.

As a combination of digital models of objects that do not exist in the form of fixed "ready-made" images but exist in the form of scalable, multidimensional digital images, which are formed in the process of visualization (rendering) at the moment of demonstration on a screen (in modern computer games) in combinations that depend on the viewing conditions, from the points chosen by the viewer for following the story, the development of film production technologies creates new opportunities for the development of, for example, 3D playback technologies. These technologies include the development of virtual and augmented reality technologies and technologies of digital holography and Field-of-Light Displays, but also represents a model of adaptive playback of the audiovisual content, such as a film with feedback from the film display devices and the moviegoer’s sensors. Unlike modern technologies for the playback of audiovisual content, the adaptive technologies are not based on scaling, framing or panning of the film components—images and sound. They are based on the recombination of digital models of film objects according to the preset playback scenarios determined by communication devices and the conditions of content presentation to the viewer to enhance the perception of the meaning of the film.

These adaptive technologies create new artistic opportunities for the creators of audiovisual content—from films where scenarios can be altered and versions of films by authoritative people to targeting different groups of consumers by changing the audiovisual content on the basis of the analysis of dramatic and emotional structures of perception. A film can be made the way some of the great moviemakers would have presented it, for example, after analyzing the artistic techniques of a certain director.

Based on the concept of a new technological paradigm, we can discuss the multimodality of the audiovisual content not only as a combination of modes—the means of conveying the meaning (image, sound, and text) in the context of the practice of audiovisual communication—but also from the point of view of the theory of content distribution—multivariate playback of a movie on various communication devices (Fig. 3).

Figure 3. Adaptive multi-modal playback of AV content on different types of AV devices.

Film Industry 4.0 is broader than the digital modeling of a film. The total digitization of all processes includes the digitization of all aspects of the infrastructure of the new film industry. In Film Industry 4.0, the viewer can be an observer and act inside the content in the virtual world of the events in the audiovisual product, and the new methods of film perception will radically change not only the technology, for example, of public screenings, but also the architecture of the movie theater and the concept of the traditional screen as a means of reproducing a motion picture. Digital modeling of movie theaters, which may become one of the technologies for the formation of Film Industry 4.0, will model the acoustic properties of theaters and public cinemas, soundproofing and sound absorption materials, and the visualization properties of a film during playback, which will create new possibilities not only for the creators of these software solutions but also for architects and designers of the digital cinema environment.

Adaptive technologies of the perception of the digital content, in combination with the technologies for analysis of the big data about a particular content consumer, including technologies based on artificial intelligence and neural interfaces, enable predictive and participative scenarios for the playback of digital content—a film—based on the analysis of personal behavioral characteristics, assimilated libraries of archetypes and images, reactions of a particular moviegoer or a group of moviegoers, and changing the rate of the transfer of meanings, determined by the artistic decision of the director.

The new paradigm of Film Industry 4.0 will enable the creation of certain models of management of the audience's attention based on, for example, the standard creative patterns of content perception using certain devices.

Thus, we can highlight another property of the emerging new cycle of the economic development of the film industry—the cycle of convergence—as classified by the author. In the paradigm of Film Industry 4.0, the issue will be not only the convergence of the audio-visual communication devices—the three screens—and the convergence of channels for audio-visual content distribution but also the convergence of the technologies for producing audiovisual content, from computer animation and games production technologies to digital modeling of audiovisual content.

The analysis of potential technologies that will become the basis of Film Industry 4.0 is inseparable from the realization of the increasing role of human creativity in the world of digital models.

Human Creativity in Film Industry 4.0

Despite the emergence of fundamentally new digital cyber-physical technologies and production paradigms, distribution and perception of audiovisual content within the framework of the Fourth Industrial Revolution in the film industry, the role of the human being, the creator, and the author of a film's concept will increase. Technologies cannot replace the creative process: the domination of the technological approach should not downplay the work of the artist, determine the scope for implementing an idea and limit artistic perception. The idea and its artistic manifestation and composition remain the essence of the artist, performer, and actor. Although a machine can be taught to write music and create scripts and visual images and hone their meanings, the human brain is behind this "machine creativity". The discussion of theories of machine creativity should remain outside the framework of the upcoming new cycle of film industry development.

Full digitalization of all film production processes will enable fundamental changes in the essence of many professions in the film industry. Some of these processes will disappear, some processes will change beyond recognition, and some processes will retain their basic features. Professions associated with accounting and dispatching, such as financial accounting or the management of the production cycle; professions that involve manual labor—stage hands and seamstresses responsible for the costumes—and other auxiliary staff of film crews and film studios will likely disappear. Some professions—set decorator, pyrotechnician, costume designer, sound designer, and cameraman—will undergo significant changes with respect to technologies not work and creativity. All problems will be solved at the level of digital modeling of certain film objects, and the human creative work—the writing of scripts and the artistic concept of the film design—will remain the defining factor. I emphasize that the transition to Film Industry 4.0 does not imply the rejection of the involvement of actors, artists, designers, directors, cameramen, and sound engineers in filmmaking. Creativity and its realization remain the essence of human talent, performing and acting skills. The new processes of digital modeling will only become a new technology of film production.

The film industry can become the prime locomotive of change in modern education models that are based on the dominant logic of science, technology, engineering and mathematics (STEM). The demand for the humanitarian component in STEM education is steadily increasing. Educational institutions are increasingly adding another component to STEM education: the component A for "Art" changes STEM to STEAM.^[15] The demand for teaching modern engineers disciplines such as psychology, art, philosophy, and neuropsychology is increasing. Without this "Art" component, the future of the film industry is inconceivable in the era of the Fourth Industrial Revolution, because it is an issue of a new type of knowledge, of interpenetrating creative and technological competencies, of the emergence of new professions, such as screenwriter-cum process engineer, and combining the analog nature of the concept with digital technologies for its implementation. As noted by Rob Legato, who is one of the authors of the concept of photorealistic cinematography, even in the digital world, all ideas are born in an analog form—in the form of musical phrases recorded as sheet music, as sketches and drafts of visual solutions, and as notes in notepads or even on napkins.

Technological Concept of Film Industry 4.0

The technological model of Film Industry 4.0 is an assemblage of different audiovisual systems that consists of three levels in the paradigm of the Fourth Industrial Revolution in cinematography (Fig. 4):

Level 1 (Resources)

o Technological content platforms - sets and hierarchies of procedures, algorithms, structures, constants, libraries, and operating systems that are assembled by a single semantic, logic, and philosophy; standardized, autonomous, open or closed systems of mutually compatible basic technologies that underlie the life cycle of audiovisual content and are intended for the production of the means of production; end-user software, programmable digital models and environments for creation, storage, distribution and perception by the audience of digital audiovisual content.

o The technological platform is the basis of a certain audio-visual ecosystem and determines the total functionality of this ecosystem.

o Technological platforms may have external interfaces (open platforms) or be isolated (closed platforms).

Level 2 (Services)

o Digital motion picture factories are systems of integrated technological solutions that provide digital modeling of components of an audiovisual product (objects, scenarios and processes) and a combination of these models in audiovisual work initiated by the general artistic concept of an author based on technologies and standards of technological platforms of audiovisual ecosystems.

o Network-based (distributed) production, storage, distribution and delivery of audiovisual content by audiovisual communication to the audiovisual content viewer, based on digital interaction and new models of labor division between all participants in the production process.

o Cloud and fog digital platforms for storing and distributing films in the digital environment, including platforms based on memory modules with DNA architecture.

o Technologies of adaptive, predictive, participatory playback and perception of a film by the viewer, based on sensors and neurotechnologies, technologies for analyzing large data sets and new technological solutions for an audiovisual product during the process of playback, including multidimensional scalable playback and digital holography.

Level 3 (Processes)

o Technologies for managing the life cycle of an audiovisual product from its conception to its implementation in the form of a combination of digital models and the viewer’s perception.

o Technologies for managing the processes of big data analysis.

o Technologies for managing the business cycles of audiovisual content.

o Technologies for managing the copyright for the design, and digital models and their combinations, including the process of the viewer’s perception.

Figure 4. Audiovisual ecosystem of Film Industry 4.0. (inspired by Borovkov A., TechNet road map. hUp://fea.ru/news/6554).

Conclusions

The imminent Fourth Industrial Revolution offers the film industry a unique chance to advance its development of the new concept of production, distribution and consumption of movies and audio-visual content. The foundation of the Fourth Industrial Revolution is knowledge. The main investments must be channeled into knowledge, education, competence, research, and the formation of new technological principles and concepts.

From financing research and development in the field of architecture of the content platforms, pilot projects of digital motion factories, technologies for creating and combining digital models based on mathematical algorithms, statistical models, materials science, bionics and solid state and materials physics, technologies for copyright management and the security of the content, and protection of users from malicious content to the development of technologies for the perception of audiovisual content and new professional and educational standards for the digital motion picture industry, a new program of humanitarian and engineering education is proposed.

The new industrial revolution will create not only new technologies but also new tasks for the study of humanitarian and social aspects of Film Industry 4.0—in the fields of art history, humanitarian technologies of conveying and compressing meanings, psychology and neuropsychology, and for understanding the human-machine interface.

The development of any modern industry is impossible without the development of labor division systems within the framework of international cooperation. This development includes education, the accumulation of a person's knowledge and competencies, and the formation of a model of an open Russian economy.

The trends of industry development should be evaluated to address the engineering and technological challenges that are "beyond the understanding". The creation of a new space, a new "puzzle", and a combination of new platforms and technologies in the industry is needed.

Acknowledgements

The author expresses gratitude to P. Schedrovitsky, A. Borovkov, the MediaNet working group of the National Technological Initiative, the teams of the NTI Foresight Fleet 2016 and the regional foresight fleet North-West 2017 for their significant influence on the author's research.

References

^[1] K. Schwab, The Fourth Industrial Revolution, New York: Crown Business, 2017.

^[2] P. Marsh, The New Industrial Revolution: Consumers, Globalization and the End of Mass Production, Yale University Press: New Haven, CT and London, UK, 2012.

^[3] A. Dolgin, Manifesto of the New Economy. The Second Invisible Hand of the Market, AST: Moscow, Russia, 2010.

^[4] J. Rifkin, The Third Industrial Revolution: How Lateral Power is Transforming Energy, the Economy, and the World, St. Martin's Press: New York, 2011.

^[5] “How to Start a Digital Enterprise. 6 Stages on the Way to Industry 4.0”, Aug. 22, 2017, tadviser.ru/a/376310

^[6]O. Berezin, Betting on Box Office: Major Cycles and Market Conjuncture in Film Exhibition, Amazon Digital Services LLC, 2015.

^[7] Tony Churnside, “Object-Based Broadcasting”, BBC R&D, June 20, 2017 www.bbc.co.uk/rd/blog/2013-05-object-based-approach-to-broadcasting

^[8] Russian Federal Law No. 126-FZ, "On State Support for the Cinematography of the Russian Federation." July 29, 2017, http://www.consultant.ru/cons/cgi/online.cgi?req=doc&base=LAW&n=221298&fld=134&dst=1000000001,0&rnd=0.56870477754415#0

^[9] Filmpro, "How the Authors of ‘Fast and Furious 7’ Made the Digital Paul Walker", March 26, 2015, www.filmpro.ru/materials/35371

^[10] D. Itzkoff, “How ‘Rogue One’ Brought Back Familiar Faces,” The New York Times, Dec. 23, 2016, www.nytimes.com/2016/12/27/movies/how-rogue-one-brought-back-grand-moff-tarkin.html

^[11] GOST R 52440-2005, "Digital Terrain Models. General Requirements”

^[12] Light Field Forum, "Fraunhofer DCA Presents Light-Field Plugin for Avid Media Composer", Sept. 26, 2014, lightfield-forum.com/2014/09/fraunhofer-dca-presents-light-field-plugin-for-avid-media-composer/

^[13] P. Shchedrovitsky, "How the Businesses from the Regions can Benefit from the New Industrial Revolution," Lecture Materials: Tomsk, Russia, 2017.

^[14] "The Top 10 US Companies by Market Capitalization", CNBC, March, 7, 2017, www.cnbc.com/2017/03/08/the-top-10-us-companies-by-market-capitalization.html

^[15] High-Tech Internet Portal, "IT Companies Need Humanitarians, Not STEM-specialists”, Aug. 9, 2017, https://hightech.fm/2017/08/09/focusing-on-steam?utm_source=telegram&utm_campaign=daily_channel

02.04.2018