Having celebrated its opening earlier this year, Spyscape in New York, NY is a new museum dedicated to the world of espionage. Over the course of a couple of hours, visitors are taken on a journey of the shadowy underworld of spooks both real and fictitious, and they’re even invited to test their skills. Think you’re a good liar? Do you vaunt your ability to keep a secret? Do you fancy yourself to be observant? At Spyscape, you’ll find out if you have what it takes to be the next James Bond or Lana Kane.

The Spyscape experience is an immersive one, complete with carefully crafted visuals and sound. At the heart of the museum’s audio system is L-ISA, L-Acoustics’ immersive audio system comprised of processing, mixing, and playback to deliver three-dimensional sound. Guillaume Le Nost, L-ISA research and development director at the loudspeaker manufacturer, headquartered in Marcoussis, France, explained that this is now possible thanks to a major shift in how audio is mixed. “One of the main technological developments [in immersive audio] is the move from channel-based mixing (mixing to a bus) to object-based mixing (mixing to a position),” he said. “[This] means that a mix can now be independent from a speaker reproduction layout.” He added that digital processing through FPGA (field-programmable gate array) or CPU (central processing unit) cores also contribute by allowing for “massive” multichannel sound processing in real time, and that audio-over-IP technologies facilitate the deployment of immersive speakers.

For performance venues, Le Nost pointed out that VR and AR are driving the demand for immersive audio. “Virtual reality and augmented reality use binaural technology to bring 3D sound to the experience, [and] live show expectations are fed by these other experiences,” he said. He also noted that 3D audio lets engineers “focus” sound, citing two main advantages this offers: “The first one is what L-ISA calls ‘hyperrealism’—the sensation of localizing the sound where the visual action takes place, to a point where one totally forgets the sound is reinforced,” he explained. “This is achieved by replacing the traditional L/R systems with a frontal array of at least five speaker points.” The second main advantage is what L-ISA refers to as “immersion”: it’s what gives audience members the feeling of being “in” the sound. “This can be achieved by extending the frontal array to surround or height speakers.”

In museums, this technology has the potential to contribute to the creation of what Le Nost dubbed “immersive realism” in interactive expositions, or for playback. “For playback, L-ISA is promoting a new reproduction format called Blu 23.1 that highlights all the benefits of 3D audio, and contains all other consumer formats,” he said. “This format is the result of years of research and is optimized to match human abilities to perceive sounds in space.” He added that he believes Blu 23.1 should garner interest among sound artists seeking a “no-compromise platform.”

Martin Audio, a loudspeaker manufacturer headquartered in High Wycombe, U.K., offers Sound Adventures, an object-based 3D immersive audio system that was created in partnership with Astro Spatial Audio, a sound technology developer based in Odiliapeel, the Netherlands. At the system’s core is the SARA II Premium Rendering Engine, and the technology has the capacity to move audio sources in real time, or via time-code, cue-based, or tracking systems. Again, the goal behind this is to enable audience members to hear the sound “objects,” rather than the loudspeakers themselves.

d&b Soundscape, developed by d&b audiotechnik, a loudspeaker manufacturer headquartered in Backnang, Germany, is an immersive audio system based on delay-based localization. The system combines three main components: d&b’s DS100 Signal Engine with Dante networking, the d&b En-Scene software module for sound object positioning, and the d&b En-Space, a software module that acts as a room emulator, enabling the addition or modification of a room’s reverberation signatures. Soundscape can be used in both fixed installations as well as on tour, and has been put to the test by a number of productions, including Sting’s Last Ship and Andrew Lloyd Webber’s updated Starlight Express.

“[Our point of view] is that with a channel-based system, it’s not possible to get a consistent sound image for the entire audience, [and that] everybody should hear the sound correctly—not just the people seated in the [center seating],” said Georg Stummer, product manager for Soundscape and the DS100 at d&b. “The second thing is, with an object-based system, the resolution of the many sound events that are taking place on stage is much higher than in a channel-based system.”

For tech managers who find themselves delving into the world of immersive audio, Le Nost listed a number of questions to help guide the design process: Is the project about localization, immersion, or both? Is there a main “visual anchor,” such as a stage, involved? Will the sound be static, or will there be many trajectories? Will the experience involve interaction with the public? He also pointed out that 3D audio designs call for more speakers than traditional sound reinforcement designs, which heightens the level of required planning and coordination with those working in lighting, video, and rigging. To ensure as smooth a process as possible, he urges tech managers to plan ahead. “[My] main advice would be to think about sound as early as possible in a project,” he said.

Carolyn Heinze is a freelance writer/editor.

Info

ASTRO SPATIAL AUDIO
www.astroaudio.eu

D&B AUDIOTECHNIK GMBH
www.dbaudio.com

L-ACOUSTICS
www.l-acoustics.com
www.l-isa-immersive.com

SENNHEISER
www.sennheiser.com

Heady Stuff for Creatives

Sennheiser’s AMBEO Smart Headset for iOS is designed for content creators that want to capture 3D audio with their mobile devices. The headset, which creates binaural recordings, is equipped with two omni-directional mics in the earpieces, as well as an A/D converter, a mic preamp, and SoftLimit technology. The two microphones serve to capture the small differences in timbre, timing, and volume that surface as sound coming from difference directions arrives at each ear. Sennheiser says that when this captured audio is played back on stereo headphones, the sound is “lifelike.”

I press play, and I’m back on the frenzied streets of Marrakesh. Mopeds dart past in all directions, a souk vendor beckons from behind me in Arabic, and Gnawa music jingles off in the distance. Throw in the smell of some sizzling tagine, and one might almost believe he had been mysteriously transported within the walls of the old-world city.  

I recently traveled around Morocco with my wife on our honeymoon, a fascinating adventure in a delightfully foreign land. With me, I brought a Sennheiser AMBEO Smart Headset—which features microphones on each earpiece for 3D audio recording—to help capture the experience. In various locations around Marrakesh, I shot scenes in 4K60 video with binaural sound, and while the footage looks fantastic, the audio is the real souvenir. 

There’s a couple reasons for this. For one, video has a long way to go to reach audio’s power of verisimilitude. To achieve full visual immersion, we’ll need virtual reality headsets capable of playing back video at 8K per eye to resolve the intrusive presence of pixel grids. Meanwhile, today’s audio equipment can readily replicate the richness of musical instruments and the subtleties of speech, and as 3D recording and playback techniques advance, recreate full environments. But even when video finally catches up to the immersive fidelity of audio, it still won’t compete with the intimate connection humans have with sound.

We have physiological and emotional ties to the things we hear, bonds that are perhaps stronger than those of any other sense. Combinations of sounds can put us physically on edge, at ease, or make us want to dance; certain songs immediately call up moments in life for which they served as a soundtrack. Many years from now, when I stumble upon my Morocco videos and listen, I’ll be immersed not only in the action of the Marrakesh Medina, but also in the memory of one of the happiest periods of my life.

Ok, so I’m a big fan of audio—and writing for an AV audience, I’m surely preaching to the choir. But, since we’re at our best when we combine our passions with practice and study, we’ve dedicated our November issue of AV Technology to audio, to help bring you up to speed on the latest in pro audio for meetings, conferencing, concerts, and more. In the Tech Manager’s Guide to Audio for All-Hands Meetings, we provide expert insight and case studies that illustrate the challenges of pulling off these crucial gatherings without a hitch; in our Drill Down and AV in Action pieces, we examine how networked audio protocols are making life easier for a range of applications; and in our features, we take a look at how spatial audio is taking places like museums and performance venues to a new level.

Because, although “audio” is just half of our industry’s name, it’s really so much more. It’s the difference between a productive meeting and a frustrating failure, between a successful videoconference and a useless silent film—and between a moment’s simple recollection and its visceral reliving. 

To hear the video, check out my review of the Sennheiser AMBEO Smart Headset on TWICE.com.

The Audio Engineering Society will hold the second International Conference on Audio for Virtual and Augmented Reality on August 20–22, 2018, at the DigiPen Institute of Technology, Redmond, WA. The conference and exhibition will bring together a community of influential research scientists, engineers, VR and AR developers, and content creators.

The conference’s esteemed keynote presenters—Jean-Marc Jot, distinguished fellow, Magic Leap; Ivan Tashev, partner architect, Microsoft Research Labs Redmond; and Ravish Mehra, lead research scientist, Facebook Reality Labs—are at the forefront of innovation for VR and AR.

“Audio and video are integral components of AR/VR devices,” Tashev said. “Spatial audio is an area still under development; we need better capture, representation, and rendering technologies. To make it mainstream, we have to catch up with authoring and editing tools as well.”

The three-day conference and expo will focus on the dissemination of top-level research in the field of spatial audio for virtual and augmented reality, with demonstrations and discussions focused on technical solutions and recommended practices. Leading researchers, practitioners, and industry luminaries will offer panel discussions, tutorials, and workshops on new and forthcoming technologies.

The conference promises to be as interactive and cross-disciplinary as the field of VR and AR itself, and will include an exhibition of relevant software and product demos. The conference will bring together talented researchers and developers who are creating the next generation of audio tools for immersive content, with presentation topics including:

• The state of AR/VR audio content development tools and workflows, including personalized versus generic HRTFs
• Realistic sound propagation for immersing the user in virtual environments
• Auditory distance discrimination in cinematic virtual reality
• Spatial audio capture, rendering, and synthesis over headphones and speakers
• Binaural, ambisonics, and wave field synthesis techniques
• 3D sound field navigation
• HRTF modeling and derivation from optical and/or acoustic measurements
• Reverb and room acoustics synthesis
• 3D audio mixing and content production
• Music and sound design for VR/AR
• Higher education and immersive media
• Sound for 360 VR
• XR [extended reality] audio in diverse industries
• Dialogue for VR/AR and MPEG-H