The art of mixing motion pictures

For the motion-picture industry, big has always been beautiful – large, projected images accompanied by fully enveloping soundtracks. But with the rapid success of the new wave of 3D, major film post-production centres are focusing on how to increase the experience of sound-conscious audiences. So how are audio playback formats changing as a result?

‘Our primary rerecording format remains 5.1-channel soundtracks,’ considers ‘Doc’ Goldstein, VP of post-production engineering at Universal Studios Sound. ‘But we can accommodate other multichannel formats and always have our eye on the future requirements of filmmakers.’ The ubiquitous 5.1-channel format involves three screen channels (left, centre, right) plus separate surround channels beside and behind the audience (labelled left-surround and right-surround) in addition to a low-frequency extension/LFE channel that carries reduced-bandwidth material (hence the ‘0.1’ label).

Such material is carried to audiences on analogue film using one of three data-compressed formats: Dolby Digital, which optically prints the digitised audio between the sprocket holes; DTS, which uses a time code track on the film to synchronise a companion CD-ROM that carries the multichannel audio; and SDDS – Sony Dynamic Digital Sound – which uses a similar technique to Dolby but, as we shall see, can accommodate additional screen channels.

Dolby Digital premiered in 1992 with Batman Returns, while DTS launched a year later with Jurassic Park – the film’s director, Steven Spielberg, invested in development of the format, reportedly because of a concern that he couldn’t achieve the large dinosaur roars he wanted for the film’s soundtrack.

Meanwhile, reacting to a need for a more immersive soundtrack experience and to provide additional panning options for sound effects and similar elements, Dolby co-developed with Lucasfilm-THX the Digital Surround EX format, which added an extra centre-surround channel. (DTS-ES, the ES standing for Extended Surround, offers similar capabilities.)

The first Digital Surround EX release, in May 1999, was for Star Wars Episode I: The Phantom Menace. Earlier this year Dolby unveiled what many filmmakers regard as a logical progression: Dolby Surround 7.1, with four surround-channel zones covering left-side, right-side, left-rear and right-rear. First used on the soundtrack for Disney/Pixar’s Toy Story 3, further recent Dolby Surround 7.1 releases include Offspring Entertainment’s Step Up 3D, Megamind from DreamWorks, and Tangled from Disney Pictures. Approximately 1,000 cinemas worldwide are currently equipped to handle the new format.

There is however another 7.1-channel format that offers extra behind-the-screen loudspeakers. Developed in the early 1990s by Sony Pictures Entertainment, Sony Dynamic Digital Sound (SDDS) specified five up-front channels – left, inner-left, centre, inner-right and right – to provide additional panning locations, as well as enabling, for example, music tracks to use separate screen channels from dialogue and effects. SDDS premiered in 1993 with Last Action Hero. The format also supports conventional 5.1-channel mixes; the data-compression algorithm is often said to offer sonic advantages over other release formats.

According to Gary Johns, SVP of Sony’s Digital Cinema Solutions, there are roughly 7,000 screens worldwide equipped for SDDS 5.1 playback, with fewer than 1,000 screens outfitted for SDDS 7.1. ‘In its heyday there were probably a dozen SDDS eight-channel movies released in any one year,’ he explains. Measured against the maybe 400 films released within North America per year – with as many as 5,000 worldwide - we can begin to appreciate why SDDS 7.1 is often considered a minority format. Although manufacture of SDDS decoders and related playback hardware has ceased, the company continues to support and service existing units, including optical cameras, and to license the SDDS format to all major film distributors.

‘Of the formats beyond 5.1, we have seen some 7.1-channel mixes,’ Universal’s Mr Goldstein offers. The new Dolby Surround 7.1-channel format ‘offers the ability to provide more discrete sound locations, which means that we can place audio more accurately. In addition to targeting one particular zone for hard sound effects, we can use point sources to provide a greater sense of ambience around the audience.’

All current analogue film releases also carry a two-channel optical Dolby Pro Logic soundtrack that contains matrix-encoded signals for the three screen and surround channels; in mono and stereo cinemas, this track is always available for older projection systems. It also provides a back-up soundtrack if the digital channels fail for any reason.

In addition to the IMAX presentation format that uses a 70mm film or digital file to project onto screens that are at least 22m by 16.5m with a separate multichannel analogue/digital soundtrack synchronised to the visuals, also under development are two other multichannel formats that offer exciting possibilities.

Tomlinson Holman, formerly with Lucasfilm’s Skywalker Ranch and now president of TMH Corporation, has been advocating several playback formats, including a 10.2 configuration. Co-developed with Chris Kyriakakis of the USC Viterbi School of Engineering, and first demonstrated in 2001, the 10.2 Channel Surround format augments the left- and right-surround channels with two point-surround channels which, Mr Holman says, ‘can more finely manipulate sound, allowing the mixer to shift sounds in a distinct 360-degree circle around the movie watcher.’ 10.2 is also used to refer to 12.2, which uses five front and five surround channels, where 10.2 uses five front and three surround channels.

‘The difference is not the placement of the speakers,’ Mr Holman stresses, ‘but rather the type of speakers and the information sent to them. 12.2 would use both surround-diffuse and surround-direct channels.’ Diffuse channels would feed dipole speakers and be used for ambient effects. Surround-direct would feed standard monopole speakers and carry sound directly to the listener. The full 14 discrete channels comprise left-wide, left, centre, right and right-wide as front speaker channels, and left-surround-diffuse, left-surround-direct, back-surround, right-surround-diffuse and right-surround-direct as surrounds. The ‘0.2’ designates a pair of subwoofer channels: LFE-left and LFE-right.

Multichannel Mixes for digital cinema

The advent of digital projection with playback from hard-disk servers rather than analogue film has created a number of creative opportunities for studios and exhibitors alike. Thanks to the work done by Digital Cinema Initiatives/DCI, a joint venture of SMPTE and Disney, Fox, Paramount, Sony Pictures Entertainment, Universal and Warner Bros, we now have a well-documented format that enables up to 16 uncompressed channels of 24-bit 48/96kHz audio to be carried to audiences. Currently, the format carries a six-, seven- or eight-track digital clone of the film’s print master soundtrack (dependent upon the format to which it was mixed) plus ancillary tracks.

‘Beginning in April 2011,’ points out Charles Flynn from the DCinemaCompliance Group, a consulting company that specialises in digital cinema, ‘all facilities will be able to accept SMPTE-compliant Digital Cinema Packages/DCP, after years of accepting what has been called the InterOp format. Right now, the package is delivered with up to eight channels of uncompressed audio. Over the last year media player vendors have been converting software and client hardware to 16 channels – the last two for the hearing- and visually-impaired audience equipment – with cinemas purchasing audio upgrades to handle more channels within their existing processors.’

The ability to carry uncompressed audio to audiences at enhanced bit rates and sample rates will extend filmmakers’ creative options, in addition to offering more delivery capabilities than current eight-channel configurations. According to Universal’s Mr Goldstein, ‘our three feature dub theatres – Studios 3, 4 and 6 – are all equipped with Harrison MPC4 Digital Production Consoles that can handle pretty much any multi-track format currently being used by filmmakers.’ Like the other two stages, Studio 4 – also known as the Hitchcock Theatre, with a 208-seat area - features a 528-input Harrison MPC4 console feeding JBL Cinema 5000 Series behind-the-screen channels powered by Bryston 4BST amplifiers, plus Meyer Sound X-800 subwoofers.

Sony Pictures Studios’ postproduction complex features five state-of-the art dubbing stages that are ‘capable of handling mixes in all release formats,’ states Richard Branca, the facility’s EVP Director of Post Stages, ‘including both flavours of 7.1: Dolby Surround 7.1 and our SDDS format.’ The Burt Lancaster Theatre features a 320-channel Harrison MPC3-D Digital Console and a custom JBL/Sony theatrical system supplemented with three Turbosound subwoofers; while the Anthony Quinn, William Holden and Kim Novak Theatres feature virtually identical equipment. ‘We consider all five stages as being fully compatible with one another,’ says Mr Branca. ‘We freely move multi-track mixes between stages as the schedules demand. To offer full forward compatibility with digital cinema, we are upgrading all our Pro Tools workstations to 96kHz sample rates to provide the enhanced resolution available from new digital theatrical formats, be they in 5.1, 6.1 EX or full 7.1-channel mix formats.’

Iosono Surround Sound - a perfect companion to 3D releases?

Developed by Germany’s Fraunhofer Institute, Iosono Surround Sound is a new encoding and delivery format for digital cinema based on research into wave field synthesis.

Since post facilities need to use existing rerecording consoles that feature limited output busses and panning, the Iosono format currently uses a Nuendo add-on which accepts up to 32 individual sources or groups of sources that can be level-balanced and panned anywhere within a horizontal soundfield using a customised user interface. The resultant 32 data tracks carry a representation of the encoded sound sources to the final venue, and currently are carried on the last eight tracks of the DCI Specification, using 4:1 HD-AAC lossless compression.

Having been programmed with the actual dimensions of the replay space, and number of available playback sources, the Iosono decoder outputs audio signals tailored for the installation-specific loudspeaker channels. The result has been described as an ‘acoustic hologram.’

‘While it is possible to create a fairly realistic sense of acoustic space with conventional surround-sound technology,’ concedes Brian Slack, Iosono’s SVP of Studio Technologies, ‘there is one major drawback – the mix requires that the listener sits in the so-called ‘sweet spot.’ Outside of that, any sound will be perceived to originate only from a very general direction.’

Wave field synthesis is based on the well-founded principle of superimposing many smaller sound waves that emanate from points in space through which the original wave front has passed. The smaller waves required to synthesise the targeted wave front are reproduced by a large number of loudspeakers fixed at equidistant positions along the perimeter of the listening space. The Iosono computer-based decoder actuates each loudspeaker at the moment the desired wave front would pass through it.

To synthesise a spherical wave front originating from a point beyond the speakers, for example, the speaker closest to the virtual source is actuated first, followed by the speakers to the right and left of it. This result is a wave front with a relatively large radius and a virtual source point outside the listening space. Reversing the order of actuation results in a wave field corresponding to that of a source within the listening space. ‘The result is a stable wave field in which the listener can localise the virtual sound sources as if they were emanating from actual objects,’ Mr Slack concludes.