ST Engineering Antycip has partnered with Renault Group to design and integrate a powerful virtual reality suite for one of the world’s largest automotive manufacturers.

Building on the companies’ previous collaborations in the VR world, and after a competitive tender process, ST Engineering Antycip was selected by Renault to develop a collaborative, 4K, powerwall-based solution which could be installed at Renault’s facilities in South Korea, India and Brazil.

“Renault France contacted ST Engineering Antycip as they are a historical client of ours in the automotive industry,” explained Johan Besnainou, ST Engineering Antycip's regional director for France and Spain, recalling the genesis of the project. “We got involved in the request for a quotation and won the tender thanks to our VR expertise, experience of working to budgets and track record of delivering high-performance AV equipment, as well as our international network of strong local partners, including in South Korea, India and Brazil.”

[How Analog Way and VISCON GmbH Enable VR Automotive Design for Hyundai]

The trio of installations, in Busan (South Korea), Chennai (India) and São José dos Pinhais (Brazil), utilize three identical systems comprising one powerwall, one high-end 5x2.5-meter screen and one plinth-mounted Christie 4K10-HS laser projector. A PC cluster, monitor, desk, cabling, 5.1 audio system and wireless presentation hardware (Barco ClickShare) complete the solution, which was entirely sourced, designed and implemented by ST Engineering Antycip.

ST Engineering Antycip additionally sourced most of the AV equipment locally to help with the logistics and keeping within budget, with the company also providing training on the new system through its local partners in each country. “We used local suppliers to avoid delays—we didn’t want the products to get stuck at customs,” continued Besnainou. “We also found people to facilitate the installations locally, while offering our international expertise.”

The three powerwall systems, which run Renault’s own 4K-native software and content, will be used by the carmaker’s local teams for product design and review, and are also designed to collaborate with each other, as well as with Renault Group’s headquarters in Paris.

[AV Network's top stories, product news, and expert insights] 

The success of the project, across multiple countries and continents and involving several manufacturers, is testament to ST Engineering Antycip’s global reach and technological expertise, as well as its Gallic savoir-faire, according to Besnainou. “We are proud to have delivered this project alongside Renault France in three countries, working in partnership with local Renault teams to create prestigious spaces for automotive design using 4K resolution with VR capabilities,” he concluded.

There is growing interest in immersive reality experiences by students and adults. According to the 2022 GroupM Annual Survey on Consumer Attitudes Toward Technology, nearly 1 in 3 consumers own an augmented reality/virtual reality device, and 15 percent will buy such a device in 2022.

In an effort to help students understand the science behind extended reality and its explosive potential to impact every aspect of consumers’ daily lives, Brave Williams, MFA, the director of the iEX Center and an associate professor in Husson University’s School of Technology and Innovation, will share his thoughts on immersive experiences at the Maine Science Festival on Thursday, March 17, 2022 from 10 a.m. – 1 p.m. from the main stage at the Cross Insurance Center in Bangor.

Husson’s iEX Center has the largest variety of extended reality (XR) equipment in one place in Maine. Williams will provide Maine Science Festival attendees with an overview of the iEX Center, a demonstration of some of the equipment, and an overview of what XR is and why it’s important.

Extended reality (XR) is an umbrella term for three-dimensional computing environments. It uses computing technologies like mobile devices, various specialized software, virtual reality (VR) goggles, augmented reality (AR) glasses, website XR (WebXR), and AR environments to create immersive, integrated, and interactive experiences. These new technologies represent enormous potential. Everyone from “artists to the zoologists” will be able to utilize three-dimensional presentations in the future.

According to Williams, “Six-hundred and fifty-six (656) students have already signed up for this seminar and there are more on the waiting list. Students have a lot of interest in learning more about immersive technologies. I’m hoping that this Husson University presentation will encourage the next generation of programmers and artists to learn more about the potential of extended reality and available professional career opportunities associated with this growing industry.”

[ Augmented Reality Market to Hit $175B by 2026: ABI Research ]

[ Extended Reality (xR) Experiences Enable New Storytelling Possibilities ]

The Science Behind Bad Concert Sound

Later in the week, on Sunday, March 20, 2022, from 1 – 2 p.m. at Knapp’s Music Center, 51 Main Street in Bangor, another Husson University faculty member will share his insights. New England School of Communications Assistant Professor Eric Ferguson, MM, be talking about the science associated with live performances as part of a presentation called, “Don’t Blame the Sound Person: The Science Behind Bad Concert Sound.”

“Anyone who goes to concerts regularly has, at one time or another, been disappointed by the sound quality,” said Ferguson. “My goal is to help people understand why it’s almost impossible for the sound person to please everybody. This presentation will look at the different variables that affect how music is reproduced and perceived in a venue. These will include psychoacoustics, venue acoustics, and phase interaction between loudspeakers.”

Ferguson has over 30 years of professional audio experience. As a staff engineer at the legendary A&M Studios, he worked with numerous platinum recording artists such as Chicago, Ringo Starr, Macy Gray, and Billy Corgan. As a freelance engineer, Ferguson has been involved in over a hundred recording, mixing, and post-production projects including work for such luminaries as Phil Collins, Aretha Franklin, Kool & The Gang, Fourplay, and Dave Grusin. Accomplished in sound reinforcement, Ferguson has toured in over 17 countries for artists such as Lee Ritenour, James Ingram, and Dave Koz.

“One of the great things about the Maine Science Festival is that it always has interesting presentations from a variety of disciplines,” said Dr. Marie Hansen, dean of the College of Business at Husson University. “Students often don’t stop to think about how science is making their virtual reality goggles or audio devices possible. But thanks to the Maine Science Festival and the educational contributions of Husson University faculty members, students now have the opportunity to understand and appreciate the scientific principles that makes their technology work and concert experiences enjoyable.”

[ Cal Poly's Spanos Theatre Comes Back to Life with L-Acoustics Kiva II ]

Husson University Senior Vice President for Academic Affairs and Provost Lynne Coy-Ogan, EdD, observed, “The Maine Science Festival is also an outstanding way to help showcase the variety of science-related expertise available at Husson University. Our faculty members have both in-depth scientific knowledge and practical experience in applying science to real-life situations. We’re proud to support this educational effort and help make the festival a great experience for students of all ages.”

For more than 120 years, Husson University has shown its adaptability and strength in delivering educational programs that prepare future leaders to handle the challenges of tomorrow through innovative undergraduate and graduate degrees. With a commitment to delivering affordable classroom, online and experiential learning opportunities, Husson University has come to represent a superior value in higher education. The hallmarks of a Husson education include advanced knowledge delivered through quality educational programs in business; health and education; pharmacy studies; science and humanities; as well as communication. According to a recent analysis of tuition and fees by U.S. News & World Report, Husson University is one of the most affordable private colleges in New England. For more information about educational opportunities that can lead to personal and professional success, visit Husson.edu.

Immersive media enables people to interact with virtual content on a realistic level. Rather than just viewing a static video, it creates an environment that allows people to step into the video and interact with the content. Immersive media eliminates traditional greenscreens and creates an environment that can be interactive, almost as if it’s a physical environment. Over the last year, immersive technology has gained popularity in entertainment and broadcast production through mixed reality. 

Drexel University Westphal College of Media Arts and Design launched one of the first accredited virtual reality and immersive media programs. In 2018, Drexel University’s Animation, Capture and Effects Lab, known as the ACE-Lab, introduced immersive media, including augmented reality (AR), virtual reality (VR), and other interactive experiences. Today, Drexel’s virtual reality and immersive media programs continue to lead, teaching and engaging students with the latest processes and technologies. 

[With Scalable Display Technologies Solutions, the University of Bern Takes Sports Science Research Into a New Frontier]

In early 2021, Drexel University introduced new immersive media technology that provides students with in-depth and hands-on learning experiences. Drexel University collaborated with Scalable Display Technologies to create a cost-effective, realistic solution for the traditional LED displays utilized in immersive environments. 

“The typical immersive media environment is incredibly expensive,” said Nicholas Jushchyshyn, program director of VR and immersive media at Drexel University Westphal College of Media Arts and Design. “It requires multiple high-quality displays and it can be challenging for an academic institution, small organization, or independent producer to justify the investment. We wanted to utilize projection, rather than LED walls, to create a cost-effective solution."

The ACE-Lab is using an existing screen that is 10 feet wide and 12 feet high for the background. "The talent has traditional studio lights on them, but they’re not overexposed and the projection background can be pure white." Jushchyshyn added. "This is the result of two laser projectors that are synchronized, calibrated, warped, and overlapped. It creates the same intensity from a traditional LED video wall, but it’s from two projectors. Scalable enabled us to take two laser projectors and stack the projections with a perfect overlay.”

[LG's Transparent OLED and Augmented Reality Combine to Elevate the Retail Experience]

The ACE-Lab is not only utilized for immersive media programs but also by many other graduate and undergraduate degree programs, including animation, visual effects, and digital design. Drexel University needed to implement a system that would accommodate the other programs utilizing the space. 

“For us, there were many advantages to utilizing projection instead of LED displays,” noted Jushchyshyn. “Not only is it far less expensive, but it’s incredibly flexible if we want to utilize it in a different location. The space we’re using for production is a multipurpose space. We have designed the entire system on a mobile cart. We can retain the versatility of the room by being able to relocate the projectors. 

“A secondary advantage of this particular approach is the future-proof design. We are currently utilizing a 12-foot wall, but we can add another wall in the future and create a full projection surface across a larger space. Utilizing Scalable’s software, we can project side-by-side and have an overlap blend between the images.”

Scalable’s software automatically warps and blends multiple projectors into one seamless display. It allows Drexel University to streamline recalibration if the system has been relocated. 

“Scalable’s one-touch calibration is incredibly intuitive,” said Jushchyshyn. “Their automatic calibration outputs text-based files with the calibration result. Once the calibration is completed, it’s effectively saved onto the GPU. It’s fully transparent to any software that’s running on the operating systems. By allowing the data to be saved internally to the GPU, it eliminates nearly all latency and any complexity on higher levels of software.”

The result was successful at all levels. The immersive media system provided Drexel University with a high-quality, cost-effective solution. “We can recreate the traditional immersive media experience at nearly a tenth of the cost of a traditional solution,” said Jushchyshyn. “We also have the added benefit of being able to readily move so we can maintain the flexibility of the space. Scalable provides opportunities, in particular for schools and independent productions, to work with immersive media technology. They can incorporate the software implications and toolsets identical to feature film-grade LED walls, motion capture and camera tracking. It’s the same technology as a large production but at a more accessible price point.”

Twitch is more than a video game streaming service—it’s a vibrant community and global phenomenon. With more than 15 million active daily users, Twitch foregrounds a different kind of connectedness in gaming. It’s become a premier locus for video gamers, esports and music broadcasts, and “in real life” streams.

Rounding the corner of his two-year Twitch anniversary, Kevin Little, the company’s Collaboration Services manager, is ideally stationed to report on the fast-changing realities of the modern workplace. Little knows how the right technology can improve workflows, but tech is just one piece of the puzzle. For Twitch’s 1,600 employees it’s the vital culture of teamwork, equitable meetings, and frictionless collaboration that help everyone thrive.

Leveling Up

To claim COVID was an enterprise variable is a colossal understatement. For nearly two years, corporations large and small have navigated logistical changes in staffing, human resources, and supply chains. Many contend that the “office” will never look the same. Nor should it.

Accelerating an existing trend, but spurred by the pandemic, remote workers will represent “32 percent of all employees worldwide by the end of 2021,” according to Gartner.

The Twitch brain trust utilized time during the pandemic to level up its collaboration methodologies—for onsite, hybrid, and fully remote meetings. Through surveys and focus groups, Little pinpointed collaboration pain points and changing needs, and the strategies to address them.

“It was interesting to see,” Little reflected, “how quickly all the platforms tried to raise their bar—for remote workers, especially.”

Little also tracks the underlying tensions that can emerge for remote workers and ways the collaboration leaders can create bridges within existing infrastructures. 

One of the ways to make communications as seamless as possible is to create a bridge for various platforms and touch points.  

Twitch has long been a “Google shop,” said Little, and, because it is an Amazon subsidiary, “we live between Amazon Chime and Google Meet.”

Technology leaders like Kevin Little focused on designing and integrating ways of communicating between the codecs. “We’ve surveyed a lot of employees, both on the Amazon side, and on the Twitch side, and we’ve worked to bridge that gap,” he said. 

Planning for the Unknown

2020 saw Twitch stakeholders back at the drawing board: white boarding and workshopping “return to work” plans. But, as rapidly developing COVID-19 variant news keeps reminding the world, even surefire plans need to adjust.  

Little says the key to keeping the workforce agile and connected is listening to users. 

“We’ve read enough surveys about how Twitch wants to collaborate, and we plan in line with that [data],” he said. “Survey, survey, survey.”

As far as a Unified Collaboration and Communication (UCC) solution to pull disparate pieces together, Little said that Twitch chose Cisco to “bridge the gap between all the platforms—allowing internal meetings in the ways people want to meet, so Twitch can be inclusive and equitable at the same time.”

Invest in Community & Culture

With a headquarters in San Francisco, and employees around the world, Little helps “Twitch employees work the way they need to with Amazon employees.”  

With its deep knowledge of Twitch’s audience, “Our team always has a very good lens on what’s coming next,” Little enthused. “They’ve done a fantastic job of weathering the [COVID-19] storm. Personally, I’ve taken the initiative to work with our Workplace Team, as well, to understand future builds, to understand how we will move forward.” 

Mission matters more than ever, Little added. 

Twitch has “doubled down on the community and the mission of Twitch, which is our culture,” Little said. “There will always be a balance with remote work, where and when people want to work, but at Twitch, it’s important that everybody sees each other, plus or minus three or four days a week.”

Remote Management

To build systems “with very few failure points,” Little noted, “Twitch is becoming more vendor agnostic.”

Driving that functionality-first approach is the focus on remote management, remote access, and managed services. 

“All of this is done in real time now, so we can see and we can monitor, through a cloud service, or remote management,” he said.

Conferencing is Changing

Another notable trend at Twitch: Conference rooms are getting smaller and more multipurpose, trending away from the gilded, siloed executive suites of yore. 

“From what I’ve seen,” Little shared, “conference rooms are becoming more multipurpose. In some cases, it means smaller, where you can have one-, two-, and three-person meetings, but also have more remote people in the meeting, in real time.”

Today’s conference spaces must also be versatile enough to support bigger events—with the flexible tech backbone to instantly shift from a huddle into a training room or an all-hands meeting space.

No matter the size or scope of the room, “It is very important to build to our standard,” Little said. 

Tools Must Be Inclusive 

There’s no shortage of UCC and videoconferencing platforms, soft codecs, and hardware. But there was a perennial challenge in facilitating collaborations so remote workers didn’t feel “lost,” Little said. That’s where an investment in cameras and audio—endpoints that could track people moving around the conference room—was beneficial. 

There should be, ultimately, one platform. “Wherever you are, onsite or remote, you should feel included,” Little pressed. It’s critical to keeping people engaged and team morale high.

Little pointed to Twitch leadership as setting the tone for inclusion. “Lenke Taylor, our Chief People Officer at Twitch, is really dialed in and passionate about inclusivity and equitable meetings,” he said. “We want the technology to follow that vision.”

More AV/IT Team Articles

[ AV/IT Team: Building Blocks of an Immersive Experience ]

[ AV/IT Team: 5 Large-scale Displays for Immersive Visualization ]

[ AV/IT Team: Out of this World Visualization ]

[ AV/IT Team: The Digital Canvas of Fame ]

[ AV/IT Team: Purpose-built Displays ] 

Don’t Chase Technology

Technology is always in flux, but one core truth remains: Don’t chase the shiny new gadget. Kevin Little confides that he has “scrapped ideas” due to changing product lines. He believes tech managers should “stop chasing technology.”

He takes a broader view: “Manufacturers and vendors will give you a roadmap as fast as they can, but adoption and systems take time.’ 

The media is also quick to leap on the latest product or trend. “Don’t put the cart before the horse,” he said. “Do lots of focus groups. Do a lot of use-case studies. Find out what is or isn’t important to employees. Maybe there’s one group for which a new tool could be useful. Build your decisions for the company on how they want to meet.”

The takeaway: Technology decisions should always be centered around people. 

“Put people first,” is Little’s motto.

Next in Tech

Though Little envisions a time when Twitch might explore virtual reality (VR) as a “proof of concept in its lab,” he is in no rush. “I see us waiting a couple more years until adoption.” 

What he’s focusing on now is video. “Because we’re such a heavy video streaming company. It is interesting to see where cameras are going—a lot of them are using AR and they’re trying to compose in different ways.”

The evolution of video transport and camera technology will come with growing pains for users, “both remote and [onsite],” Little said. What he sees in the support channels illustrates this, including the latency of the home network. “A lot of the codecs are going into NDI and IP. I’m interested to see what this will look like over one big network, as we move away from more like video cabling. It’s all over Cat-6 now.”

If Twitch’s staggering growth is any measure, the game streaming company is poised to influence the network video game profoundly.

When COVID-19 shuttered sporting and events arenas, many events were cancelled. That wasn’t good enough for Worre Studios founder and CEO, Marina Worre. With plans to host and interact with tens of thousands of attendees during multiple events scheduled to take place at the MGM Garden Arena in Las Vegas, Worre quickly pivoted to creating a virtual studio. With just three months until the first event was to take place, the Worre Studios team pulled out all the stops to ideate, design and install a 25,000-square-foot production studio and event space. 

The space required broadcast-quality audio, video equipment and displays, capable of hosting live events with real-time interactivity between hosts and attendees. “The most important aspect in designing the studio was to create a space that gave every attendee a front row seat, as well as the ability to interact with the event hosts,” Worre said.

Overcoming Obstacles 

The project began in mid-September 2020, with Advanced LED Displays sourcing LED, trusses, and lighting for the space. “Most LED takes 60 to 90 days to manufacture, so that was a big obstacle right out of the gate. And then determining how long it was going to take us to build it, and install all the other pieces,” said Art Martinez, vice president of Business Development, at Advanced LED Displays.

Martinez and his team designed free-standing curved LED walls. “The walls grew in size, and tightened in radius,” Martinez said of the installation. “At that point, we knew we needed some serious processing to handle it—and most processors can’t handle a wall of this size. It was not an option to do anything other than the Spyder X80.” 

The Christie Spyder X80 multi-screen windowing processor can control up to 80 million pixels with 24 inputs and 16 outputs, and with image processing in 4K@60Hz. One Spyder X80 was installed to drive content to the wall, and one as a back-up.

Two of the video walls are 73 feet wide by 14 feet tall, and curved, and the other two are 25 feet wide by 14 feet tall. “After the walls were built, Marina asked if I could move one of the large walls to the other end of the room to accommodate space for an audience and prepare for hybrid events in the future. I accepted the challenge and designed a set of tracks recessed into the concrete floor for one of the large walls to move across the room,” Martinez said. 

More AV/IT Team Articles

[ AV/IT Team: Building Blocks of an Immersive Experience ]

[ AV/IT Team: 5 Large-scale Displays for Immersive Visualization ]

[ AV/IT Team: Out of this World Visualization ]

[ AV/IT Team: The Digital Canvas of Fame ]

[ AV/IT Team: Purpose-built Displays ] 

Seamless Interaction

The result is a studio capable of in-person, hybrid and virtual events, where presenters can seamlessly interact with hundreds of thousands of participants virtually, reach millions through pretty much any social media channel, stream through platforms including Twitter and YouTube, as well as host a live audience.

“Events are fully interactive, and we host guests from all around the world – and with our real-time translation platform, we’ve removed the language barrier so everyone can have the same live experience,” Worre added. “The studio was built for our needs originally but given its success and the demand for a pliable space with this technology, it’s now available for any event. We have hosted seminars, done product launches using the Spyder that reached more than 10 million viewers, and we even recorded a music video.” 

As a result of these needs, selecting a processor that was not only reliable, but had space to handle additional inputs and outputs as needed, was a requirement. “At Agility AVL, we design and build a lot of sports books, and the Spyder is always our go-to because of the number of layers, up to 24, and the options we have on inputs and outputs,” Martinez said. “It’s something only the Spyder has, and it can handle 80 million pixels.”

“There’s nothing that has been built like this in the world,” Martinez said of Worre Studios. “For us, it was always Christie Spyder. It’s the only product that I know will work flawlessly and continue to run day in and day out without problems.”

Inspired and encouraged by a creative father, Brian Allen, executive vice president of Technology and Content, Illuminarium Experiences, grew up building haunted houses in the family garage. But his creations took the experience to a level most wouldn’t have dreamed. Skeleton masks sitting on compressed air actuators were triggered as unsuspecting trick-or-treaters walked past. “I live and breathe immersive,” Allen said. “Since childhood my mind has been molded and framed to think about experience.”  

For Allen, the idea spans across many modes of life and can be applied to any exercise of curating, manipulating, or framing the way a person sees, hears, touches, smells, and tastes a world you are trying to create. “These are the building blocks of experience—whether you are designing a restaurant or a rock concert—all the same rules apply,” he said. This principle applies beyond one’s physiological sense. “I would argue that when designing a truly immersive experience you need to take into account one’s emotional and cognitive events. What does the audience feel emotionally? Do they come away with questions? Do they come out inspired? These are all important factors in a great experience.”

Transportation

Allen and his team at Illuminarium Experiences use projection as the main method to transport audiences. “With a pixel space in the hundreds of millions, there is no shortage of canvas to work with. No matter how big or small the installation or high or low the budget is—in the world of projection, content is king,” Allen explained. But visual content is only one element, and he urges creators to broaden their ideas about what content brings to the audience experience. “Too often I see installations heavily rely on video content to carry the audience experience. This tends to be one-dimensional in that that experience is only stimulating one out of the five senses.”

More AV/IT Team Articles

[ AV/IT Team: 5 Large-scale Displays for Immersive Visualization ]

[ AV/IT Team: Out of this World Visualization ]

[ AV/IT Team: The Digital Canvas of Fame ]

[ AV/IT Team: Purpose-built Displays ] 

A Landscape of Sound

In theater, the illusion of depth is created with layers of set pieces and props given the fixed audience position. “You can take that same idea of layering to create depth where there is none. You can layer the mise en scène (the arrangement of scenery and stage properties, usually in a play) of a given shot in your video content or you can take it a step further by breaking through the fourth wall and “spill” your content into the physical world,” Allen said. 

Now, what does it sound like? Audio content is often underappreciated and overlooked but adding a curated layer of audio to your space can make all the difference. “Audio technology nowadays can offer many methods of delivering sound to your audience. It can be localized, big, small, far away, vocal, musical, or completely ambient. The word ‘soundscape’ gets thrown around a lot and I love that for describing sound across an experience. A landscape of sound,” Allen explained. Experience designers and creators imagine what an experience sounds like, not just at any given point in time but also at any given vantage point in physical space. “An extra dimension in sound design is another tool in the toolbox for creation. Not every experience requires every tool to be used to be great but remembering that you have them at your disposal is a key tactic when designing and engineering a new experience,” he added.

Dream-on

There is a wide array of special effects available when designing immersive spaces. “The dreaming phase when you have no constraints to fit into is the time for you to craft your experience and think outside the box,” Allen said. “There are no limits whatsoever and the world is truly your oyster.” Then comes the seemingly difficult part. Designers need to work within budgets, physical constraints, power requirements and structural load requirements, which for some, puts the dream back into a box. But not for Allen, “I think this is where the magic happens. As a creative, giving yourself rails to work within is often one of the most freeing things you can do. Not burdened by choice, you now have the opportunity to shape your idea into something that will function. It's good to have one foot in the dream world without boundaries and one firmly grounded in what is actually possible. This will make you dangerous in this industry: A realistic dreamer.”

When creating a dream within the bounds of reality, Allen suggests that less is often more. “Can this experience be boiled down to a few core elements? Stripping away the fluff will often times bring the foundational elements back into focus.”  

360-degree Experiential Entertainment

Illuminarium Experiences is a global experiential entertainment company created by worldwide leaders in cinematic and interactive content, architectural and theatrical design, and venue operations. The immersive entertainment spectacles presented in custom-designed venues are called "Illuminariums." These reprogrammable immersive theaters surround visitors in a sensory space of sight, sound, and scale, and provide access to exotic and often out-of-reach places, people, and experiences.

Panasonic is the exclusive visual solution provider of native 4K projectors, 4K professional displays, and 4K professional camera solutions for Illuminarium’s 360-degree immersive experiential entertainment centers.

In July 2021, the first illuminarium opened in Atlanta, with its debut spectacle, WILD: A Safari Experience. The 26,000 square-foot entertainment complex showcases the splendor of Africa's most exotic animals in their natural habitats. Created by Alan Greenberg, RadicalMedia and Rockwell Group, and operated by Legends, Illuminarium Experiences will open in Las Vegas at AREA15 in January 2022, and in Miami, at Mana in the Wynwood Arts District, in fall 2022. Other North American locations are under consideration with plans to create a global footprint in the world's leading international destinations. 

[ Panasonic Expands Its Immersive Entertainment Collaboration with AREA15 ]

Personalized

Personalization and real-time content are going to change the way experiences are made. “With the advent of the game engine and the golden age of real-time graphics, more and more experiences will take advantage of this technology,” Allen said. “So much technology around us can be manipulated and customized to fit your specific needs.” He likens this to personal experiences such as frequenting a restaurant or bar and being greeted by name, or the bartender knows your favorite drink. “When personalization and real time manipulation intersect, you get something truly immersive. Not just in scale or in beauty, but real interaction. A chance to leave your mark and contribute to a living experience,” he added. 

Create Your Own Playbook

In the world of immersive and experience there is no playbook, which means you can find inspiration from anything. “Allowing yourself to become inspired by anything in life will take you much further. That incredible dinner you had last night where the wait staff remembered your name and never let your water glass go empty, or that botanical garden greenhouse you visited and smelled roses and oranges in the dead of winter—these are the tidbits that begin to craft legendary experiences,” Allen said. These are things that people remember for the rest of their lives. “Great artists and designers can look at the most mundane things in life and pick out the tiny spark that allows others to change the way they look at the world. Anything can be captivating and there are endless opportunities to wow, awe and inspire,” Allen concluded. 

The HIVE (Hub for Immersive Visualization and eResearch) at Curtin University in Perth, Western Australia, is one of the most advanced visualization facilities of its kind, and the only one in the world that contains five different large-scale visualization displays. Its work with technologies such as virtual reality (VR), augmented reality (AR), data and scientific visualization, photogrammetric 3D reconstruction, and volumetric imaging has been used to, among other things, help patients with spinal cord injuries, investigate consumer behavior and create virtual experiences of shipwreck sites.

“Essentially, it is a sandbox offering university research staff the opportunity to explore a wide range of visualization technology options in research projects across all expertise areas,” explained associate professor Andrew Woods, manager of the Curtin HIVE, who also served as the client lead for the upgrade project. “The facility was originally established in 2013, so it was definitely time to update the HIVE AV systems with the latest projection technology,” he added.

Immersive Visualization

The display facilities at the HIVE, located in a 49 foot-square former gallery space, comprise the Dome display, a 13 foot-diameter half-dome which fills a user’s primary and peripheral field of view; the Cylinder display, a wrap-around 3D display providing a VR experience for up to 50 users; the Wedge display, a pair of 12.5-foot diagonal 3D displays used mainly for scientific visualization; the Tiled display, which with 24MP resolution is used for super high-resolution visuals; and the Hologram Table, which provides two users with 3D visuals that project out of its tabletop display surface. The upgrade project focused on the Dome and the Cylinder displays.

“The HIVE Cylinder display is the flagship system in the facility,” Woods said. “The Cylinder display uses three projectors to light the 9.9-foot high, 26.25-foot diameter, 180-degree cylindrical projection screen. In upgrading the HIVE Cylinder display, in particular, we were keen to increase the display resolution to use native 4K projectors, which would provide a fourfold increase in displayed pixels, and to increase the brightness, all while maintaining the existing 120Hz stereoscopic 3D capability of the display.” 

After talking to Digital Projection, the team at InDesign Technologies, the project’s consultant, realized quickly that Digital Projection’s Satellite MLS was the only projection system that met the HIVE’s requirements (full 4K with 120Hz compatibility, and not subject to the exclusion zones that would be necessary with a class-3 laser product) – to the extent that Curtin University was willing to wait for it to come to market rather than compromise by specifying a different system.

“The Digital Projection Satellite Modular Laser System (MLS) was still in development as a technology when discussions commenced,” recalled Tanya Hall, business development manager, integrated solutions, for Amber Technology, Digital Projection’s distributor in Australia and New Zealand. Hall worked closely with Woods, InDesign, integrator Vizcom Technologies, and Emilijo Mihatov, regional sales manager for Digital Projection, to discuss and scope the technology requirements of the project, and VIOSO to develop a solution for the unique requirements of the HIVE Cylinder install.

More AV/IT Team Articles

[ AV/IT Team: Out of this World Visualization ]

[ AV/IT Team: The Digital Canvas of Fame ]

[ AV/IT Team: Purpose-built Displays ] 

All-Around Support

Amber Technology worked with, and were supported by, Digital Projection at a very high level throughout the project timeline, said the Amber team. There were many hours of communication with the Digital Projection team to tap into their expertise on the specification, solution scope and system commissioning. In particular, the Digital Projection team’s insight into how the system would interact with the HIVE’s existing infrastructure was invaluable.

Working with such a new technology was bound to bring certain challenges. The projectors required specialist lenses that did not, at that time, have any local distribution representation, so with the help of Emilijo Mihatov from Digital Projection, Amber Technology were able to contact the lens manufacturer and establish a trading arrangement to bring the required lenses into Australia for this project.

Similarly, with the sophisticated blending requirements for the Cylinder component of the project, Amber was able to make the connection with VIOSO through Digital Projection and source the blending workstations.

Physical mounting positions also proved to be an issue, recalled Wrightson, with a full structural reinforcement required to have the mounting beams in the correct positions for both the Cylinder and Dome displays.

“With this mounting solution also came some environmental vibrations from the building’s HVAC systems,” Wrightson continued, “which was particularly noticeable when using the Jenoptik fisheye lens in the Dome display area, so we had to design and construct a custom lens collar to negate this.”

Richard Neale, Amber Technology’s general manager, integrated solutions, commented, “As the distributor for Digital Projection in Australia and New Zealand, we were excited to be involved in the first installation in the world of this new, cutting-edge technology. For Amber Technology, the HIVE is an important first as we introduce the capabilities of the new Digital Projection Satellite MLS system to the broader market in Australia and New Zealand. The HIVE perfectly showcases the possibilities for projection in new and exciting environments.”

“It has truly been a pleasure in seeing this project come to life,” Wrightston said, “and I hope I get the chance to work closely with the Digital Projection and Amber team again in the future.”

Satellite Modular Laser Technology

At the HIVE, three Satellite Modular Laser System (MLS) systems are installed in the cylinder display, with a further system deployed in the dome display. Taking advantage of the Satellite MLS’s modular nature, which allows up to 100m separation between the projection head and light source, InDesign was able to hang the unobtrusive satellite projector heads at the display sites while locating the light sources around 49 feet away in a server room containing the other AV and computer equipment.

The unique Satellite MLS separates the light source from the small, lightweight projector ‘head’, giving integrators greater freedom, particularly where space and access are restricted, while reducing noise and heat from the audience location. This technology makes high-end RGB laser illumination accessible at a cost-effective price point.

“There was a requirement for the projectors to make as little noise impact to the space as possible,” said Scott Wrightson, project manager for Vizcom Technologies. “This made the Digital Projection Satellite MLS systems invaluable. By having the main laser processor housed in the central comms room it made the operating noise level of the projection heads very low, which in turn suited this project perfectly.”

“The Digital Projection satellite technology allowed us to meet our needs and more, with a small head size, minimized audible noise from the head—because the light source is in the server room—rich laser-light engine visuals, and cinema-quality three-chip DLP images, at a reasonable budget point,” Woods added.

Invisible Technology

For the HIVE, the improved AV system is an important step forward for a facility whose work revolves around high-impact visuals. “Ultimately our aim is that the displays in the HIVE are essentially invisible to the user, just providing a window into a different world,” Woods explained. “Our aim is to put amazing visuals on the blank canvas of these displays—visuals that support the applications and research projects that our users come to us with.” The Satellite MLS more than delivers on that aim, he added, “Digital Projection worked incredibly hard to deliver a new-to-market product for installation into the HIVE, half-way across the world. Despite all the challenges, the systems are now in and working wonderfully.

“The Cylinder and Dome displays also have much more punch than they previously did, creating high-quality experiences for the HIVE’s users and visitors overall. We are very impressed with the result and already wowing people with the quality of the visuals and the university’s leading-edge research,” Woods concluded.

Church on the Rock in St. Peters, MO, considers its online community a distinct campus, with distinct needs that go beyond the church’s physical space. With its emphasis on high-quality streamed services, the church places a high priority on having the right digital tools. The team there chose the Waves eMotion LV1 Live Mixer to mix and stream the church’s services

Mixer, producer, and audio consultant Yamil Martinez consulted Church on the Rock on this project. His recommendation to adopt the eMotion LV1 was based on the LV1’s ability to deliver precision sound for live events and streams: “LV1 is based on integrated Waves plug-ins, so it helps me tailor the sound to any requirement in ways that most regular consoles can’t,” he said. “Because Waves’ precision plug-ins are so accessible from within the mixer, eMotion LV1 enables us to achieve an extremely polished studio sound in a live situation. We’ve been able to accomplish processing chains that make miracles happen, especially when dealing with unwanted noise. To me, the LV1 is a chameleon-like console that I can personalize to achieve any coloring palette needed.

“Our decision to adopt the LV1 was driven, first of all, by our need to improve the church’s online experience,” Martinez continued. “Before the LV1, the church’s broadcast system was quite convoluted. It was based on a combination of groups and direct outputs from the Avid Venue D-Show console at FOH, plus a few direct analog splits that fed an analog Midas Heritage mixer. Drums were fed from a premix, as were the brass and half of the vocals. Lead vocals were received individually from a processed direct-out feed. Secondary sources, or sources that aren’t used every day, derived from a split of a split, out of the main splitter. But the biggest drawback prior to adopting the LV1 was that not all the decisions made for the FOH mix translated properly in broadcast. This was both because of the physical sound of instruments in the auditorium, and because each of the church’s mix engineers or mix volunteers had different criteria. Just imagine all the inconsistencies that the former system had. This is why our big LV1 project was born.”

One Mixer Turns to Three

“Our initial idea was to extract unprocessed individual audio channels from the FOH console to a dedicated TV mix in a single eMotion LV1 mixer, but we quickly discovered that interfacing with the existing consoles just wasn’t cost-effective, nor did it provide the flexibility needed or the necessary channel counts. Neither did we want to replace the analog splitter with another one with a third output, or to invest in connectivity for outdated consoles. These were not viable solutions,” Martinez explained.

[ Streaming Audio in Houses of Worship ]

“In the end, we simply replaced our Midas Pro digital console at monitors with an eMotion LV1 mixer and shared its inputs with broadcast. But that was just the start! Eventually, three high-end consoles got replaced with eMotion LV1 mixers: two Midas consoles, one analog and one digital, which reached their limits; and a Yamaha PM1D that was stuck in outdated technology. Those consoles had to be replaced because they couldn’t adapt to our new circumstances and needs.”

Freedom of Movement

“With the Waves eMotion LV1, we are able to have full console control from different locations at the same time,” Martinez said. “This has been absolutely crucial to the church’s TV broadcast, 2D streaming and 360° VR operations. To provide a better online experience, Church on the Rock has content variations that include online hosts who interact with the online campus community and inform them of COTR’s wide variety of activities. Those hosts move around the facilities, depending on where the activities are being held. No matter where you are within Church on the Rock’s facilities, if there is a network connection, all you have to do is assign the specific port to the proper subnet to send and receive audio.”

“One huge advantage of the Waves LV1 is that it’s based on Waves SoundGrid technology, which is an Ethernet network-based system with minimum operator interaction. It does away with all the messy networking challenges, and it eliminates the hassle out of the crew hands,” Martinez explained. “You just assign the network ports and then select, update and enable the devices, assign the clock master, and you’re done. Your network is ready. Church on the Rock already had in place a solid Ethernet network infrastructure that allowed a dedicated network for SoundGrid. This made it easy to implement SoundGrid as the central hub for all the audio connections from anywhere in the buildings. All this has been a big plus for COTR’s Creative Arts team. Now they can generate content without boundaries within the buildings, or anywhere an Ethernet cable can reach. Our network covers two physical campuses across a street, with ten audio working areas. Any corner can be a creative spot!”

Modular Setup

Martinez continued, “Our designed system consists of an audio network that covers two different buildings across a street. It includes three LV1s, one Avid VENUE D-Show, one Pro Tools HDX, two Behringer X32 mixers, one Zilia 3D ambience microphone, and 14 Waves SoundGrid devices (excluding various DAW computers), including four DSPRO StageGrid 4000 stageboxes, three DiGiGrid IOC audio interfaces, one DiGiGrid IOX high-count interface, two DiGiGrid DLI interfaces, two DN32-WSG cards for Behringer, one Waves DMI interface and one DiGiGrid D portable desktop interface. The compact DiGiGrid D interface moves all over both buildings, depending on where the TV hosts are. The latest addition to the system is a 3D immersive audio mixing system that complements a 360° video content destined to COTR VR campus. Virtual reality is the best way to remotely be part of the COTR experience.”

[ How Audio Helps Houses of Worship Regather with Confidence ]

“This setup is wonderfully modular and expandable,” Martinez said. “As your audio needs evolve, the system can grow, too. Our project is proof: we started with a single LV1 as a simple broadcast mixer, and evolved to an entire audio network that covers two multistory buildings.”

Communication

About the facility’s monitor system, Martinez said, “At Church on the Rock, we don’t have a dedicated person for monitor mixing. Each musician controls their own mix wirelessly with an iPad running the Waves MyMon personal mixing app, which ‘communicates’ with the LV1. Having the MyMon app was indispensable in the selection of the LV1. The monitor LV1 can be fully mixed and cued from the stage, FOH and broadcast positions, with the latter two having absolute control over the monitor console. The more experienced FOH and broadcast engineers can assist musicians from their own position.”

He added, “The LV1/SoundGrid system allows us to have an elaborate internal communication system through a dedicated matrixing system that receives 15 talkback/communication inputs and distributes to seven destinations throughout both buildings. Before the LV1, there was no way of getting completely rid of the sinful mono IEM mix. Now, every IEM mix is stereo. Any instrument can have its own stereo reverb without complicated routing or mixing procedures. Having the reverb as part of the channel signal flow keeps the proper relation between dry and wet at every monitor mix.”

Waves Plug-Ins

About using Waves plug-ins, Martinez said, “Church of the Rock has always invested in top-notch gear since the analog days, but the results we’ve achieved with the LV1 are unprecedented. The wide selection of plug-ins from the Waves Mercury, Abbey Road, SSL, Dugan Automixer and DTS Neural Surround bundles gives us precise solutions for every need. We use many plug-ins; however, there are what I like to call ‘MVPs’—most valuable plug-ins—that make an especially huge impact on the final result. First, the Waves Playlist Rider plug-in is great for content that alternates between music and speech and has a broad dynamic range. We don’t want people constantly adjusting their volume and getting bothered by level inconsistency. Playlist Rider stabilizes all pre-recorded material, speech microphones and broadcast master output, providing smooth transitions without distraction.”

He continued, “Another situation where Waves plug-ins are life-savers is stage mic bleed. Having many instruments and numerous vocal mics open at the same time can result in a lot of stage bleed that clutters the mix and compromises intelligibility. Waves’ Primary Source Expander (PSE) plug-in eliminates stage bleed, especially when using automatic pitch correction for broadcast. With its sidechaining capabilities, PSE can be placed after a few layers of EQ, saturation and compression and still track properly. We replaced all the filters and EQ that we used to use with the extremely flexible and low-latency F6 Floating-Band Dynamic EQ. With its different slopes, the F6 filters are more adaptable. The dynamic low- and high-shelf bands allow you to expand those frequencies as needed instead of using fixed gain, reducing bleed when a direct source is not present enough. The narrow parametric bands, with a Q of 60, make the F6 perfect for surgical needs. It’s like a crossbreed between a Swiss Army knife and a scalpel.”

He added, “When it comes to reverbs, the OneKnob Wetter and GTR Reverb plug-ins are great for mixing with simplicity and consistency. We need good-sounding, low-latency and CPU-friendly reverbs to insert on a great deal of individual channels. With these two plug-ins, we cover all our bases for channel-inserted reverbs on instruments, and even on vocals in monitoring duties. Another plug-in that deserves special mention is the eMo D5 Dynamics , which is part of the eMotion LV1’s channel strip. It’s a wonderful-sounding dynamics plug-in with extensive side chain capacity—the only plug-in inserted on every single input and output of all three LV1 consoles. It handles beautifully all the heavy lifting in terms of dynamics. The church’s latest plug-in addition has been the Waves Dugan Automixer for auto-balancing multiple talkback mics. We throw at it constantly changing combinations of 15 possible inputs, and it keeps them all totally balanced, automatically. Talkback screamers are welcome since the Dugan will take care of them—easy.”

[ The Next Wave of Virtual Events ]

“The consistently great sound quality we’ve achieved with the LV1 exceeded our expectations,” Martinez noted. “With Waves’ eMotion LV1 mixer, Waves plug-ins and Waves SoundGrid real-time processing and networking technology, we’ve been able to achieve more polished results, faster and more easily, delivering better broadcast services to Church on the Rock’s online campus community.”

More on Audio in Houses of Worship

Waves eMotion LV1 Live Mixer Enhances Michigan Church Streaming Services • In order to keep parishioners connected while in quarantine, Mount Hope Church in Lansing, MI, has been presenting its services via livestream to the community, with the help of a Waves eMotion LV1 mixer.

Streaming Audio in Houses of Worship • There's no question that the ways people come together to worship have changed dramatically this year. That has also meant a lot of changes to the way services sound—and what they may sound like in the future.

A New Way to Worship • During the pandemic, houses of worship often held virtual service, which in turn has meant an increase in requests to systems contractors and integrators to facilitate streaming services.

Getting Started in House of Worship Integration • Just Add Power’s Ed Qualls shares best practices to limit project hiccups and help integrators deliver the perfect solution for house of worship customers.

How Audio Helps Houses of Worship Regather with Confidence • The new normal has proven to be particularly challenging for church leaders who are tasked with bringing parishioners together across a variety of locations, whether in-person or attending virtually from home.

More on Livestreaming

Going Live: Planning an Engaging Livestream • Your organization has asked you to livestream an upcoming event. Now what? Industry experts provide their advice on how to plan and execute a livestream that looks great and boosts engagement.

The Next Wave of Virtual Events • When the COVID-19 pandemic sidelined live events, out-of-the-box thinkers came to the table with solutions to keep the events industry alive by connecting people virtually.

What’s Next for Streaming? • During the pandemic, streaming services kept businesses running, friends connected and entertainment rolling out to fans. Now that streaming has become indispensable, what else can it do?

The What: Modus VR has upgraded its Virtual Reality software with the addition of new Conference Technology objects, features and partners that specifically address the space planning needs of  enterprise technology integrators, designers, architects and customers.

The What Else: Newly added commercial objects include several products from the ever-expanding Modus VR partners such as Logitech (including the Rally series of communications components) and Logitech MeetUp, Salamander Designs Unifi Conference Tables, Bose VB1, Barco Clickshare and XT Series LED Wall, Avocor E Series Touchscreens, and generic conference cameras, conference microphones, and pendants.

[The Integration Guide to Conferencing & Collaboration]

The list of new features includes a Camera Field Of View (FOV) Design Assistant. Leveraging the power of VR, this new field-of-view tool allows proper design and deployment of cameras based on room size, table shapes and other challenging aspects of precision camera placement. Also new: the Modus VR Pixel Pitch Design Assistant, a tool that allows users to quickly verify the screen pitch, the possible positions of viewers and thereby decide how best to set up the conference room or space. Using a color-coded system, Modus VR software indicates what will be the ideal viewing experience based on the viewer’s location, so the designer can quickly and easily find the best balance of performance and cost for all project stakeholders. 

This package also includes Speaker Output Visualization, a means for users to plan the location of conference room speakers and display the entire speaker coverage pattern visually.

The Bottom Line: According to the company, the latest additions make it faster and easier for all stakeholders to create conference rooms that precisely match the real layout of a given space in 3D. Designs can be built in real time and complete with AV equipment, furniture,  microphones, cameras, and more.

Colleges and universities have been making use of augmented and virtual reality technologies for a while now. As the pandemic has forced institutions to reach more students remotely, AR and VR have the potential to contribute to a more engaging distance learning experience. Recently, AV Technology spoke with several leaders in the field to gain a better understanding of what this tech can do for higher ed.

Augmenting Anatomy

The Health Education Campus at Case Western Reserve University in Cleveland officially opened its doors during the summer of 2019. Built in partnership with the Cleveland Clinic, the facility gathers students from Case Western’s medical, nursing, and dental programs under one roof. It also houses a number of spaces that were specifically designed to deliver anatomy education via augmented reality. 

Related: Volvo Tests New Car Designs by Driving With an XR Headset

Developed by what Case Western has dubbed the Interactive Commons—a group of anatomy faculty and software engineers—the university’s HoloAnatomy software in combination with Microsoft HoloLens AR headsets provides first- and second-year medical students with 3D perspectives of the human body, without the need for wet labs and cadavers. Miro Humer, associate vice president of client experience at Case Western, explained that his team is responsible for supporting the HoloLens devices as well as the technology that runs them. 

Humer said that during HoloAnatomy’s planning and design, there were some logistics to consider: how long would students need to wear these devices (and would they become too heavy to wear after a certain amount of time)? Would the devices cause headaches? And what about battery life? If a session was to last 45 minutes, and classes were scheduled back to back, would there be time to recharge the devices? “It drives how many devices you have to have on hand,” he said. “Our battery life is typically about two hours, but with age, that recedes.” He added that there was also significant discussion around how to apply the HoloLens to student testing: “When we have testing sessions and we’re using the HoloLens for the test, do we have enough devices? How do we proctor those tests? Some tests last an hour and a half, so the devices have to be fully charged and the batteries have to be in good condition to manage that.” And, if a device fails during the test, it’s necessary to have a backup on hand.

As of press time, only part of Case Western’s student body was back on campus, while the rest continued with remote learning. Humer relayed that when the university went fully remote in the spring of 2020, HoloLens devices were shipped to HoloAnatomy students and classes were able to continue. “Everybody saw how well this can work in a distributed world, where students can be anywhere and keep going with their education,” he said. During remote classes, students examining the same holographic image can also see avatars of each other, providing a level of interaction that Humer said is well beyond Zoom. “We’re going to continue our anatomy programs remotely. That would have been impossible in the old world, and that would have had significant financial impact to the university if we weren’t able to continue those classes.” 

Humer acknowledged that different people have different expectations of what AR and VR can deliver, and in order for an initiative like this to be successful, genuine collaboration between faculty and tech staff is a must. “There has to be a very strong partnership between the technology organization and the academic organization within the university to develop an effective solution—more so than, say, if you’re just deploying computers in a room,” he said. “It’s very closely aligned with the academic goals and students’ goals as well, [in terms of] what they can achieve in class. The technology organization has to be closely involved in the development of it.”

Professors Without Borders

In June of last year, Almo Professional A/V partnered with ARHT Media Inc., a holographic solutions developer based in Toronto.

Brian Rhatigan, director of business development at Almo, recounted that his colleague Sam Taylor, executive vice president and COO, was one of the driving forces behind the partnership. “Sam recognized this as a pioneering technology that he felt we wanted to be a part of, and that it could be a mutually beneficial relationship for both companies,” he said. “We’ve been working cohesively to raise awareness of ARHT Media, which is not necessarily a brand-new company, but new to our customer base. We’re getting the word out about the possibilities of the things you can do that, really, are outside of what the traditional pro AV integrator typically thinks of from a presentation and communications perspective.”

At the core of ARHT Media’s technology is the ARHT Engine, software that enables high-quality, low-latency AV streaming with end-to-end encryption. HoloPresence is a 3D display that doesn’t require viewers to wear 3D glasses, and which allows remote lecturers to appear in front of audiences as live holograms. Released in November 2020, HoloPod is a more portable version of HoloPresence (which does require an hour and a half to deploy). Designed for lecture halls, corporate training centers, and large corporate boardrooms, HoloPod is a plug-and-play cabinet on wheels that can be fully secured when not in use. Virtual Global Stage (VGS), the ARHT Media’s online, low-latency presentation platform, rounds out the company’s current lineup. 

ARHT Media CEO Larry O’Reilly explained that in using these solutions, universities become truly borderless. “What it allows higher education institutions to do is bring in core curriculum instructors or guest lecturers from anywhere in the world and have them present live or pre-recorded, lifelike, life-size, and fully interactive,” he said.

Last September, global healthcare company Novartis enlisted ARHT Media for a sponsored online presentation at the European Respiratory Society (ERS). Addressing the topic “Asthma Trends and Digital Health Solves,” three peer academic faculty based in Australia, Greece, and Germany were able to appear together, life-size, on the same Virtual Global Stage and interact with each other.  

“They wanted their presentation to look better than a Zoom call, which is what everybody else was doing … and we brought them together so that they could do their presentations individually, but more importantly, then have a panel discussion,” O’Reilly said. “It looked like they were all together in the exact same room because there was no latency. It was way more engaging.” According to ARHT Media, the presentation received over 2,000 views, and the company has continued to work with Novartis on other events since.

More Than Just a Laptop

The platform developed by zSpace Inc., an interactive technology company headquartered in San Jose, CA, integrates AR/VR technology into a laptop. The system also includes lightweight eyeglasses (enabling users to see each other while interacting with the technology), as well as a stylus. The laptop displays are equipped with tracking; as users tilt their heads to look around objects, the software automatically updates so that the image is presented in the correct perspective in full high definition. Users hold the stylus like they would a pen, with built-in buttons for additional functionality.

zSpace also offers a learning content catalog developed both in-house and by third-party providers. 

Michael Carbenia, executive director of career technical education (CTE) at zSpace, actually started out as a customer. Having grown up in the AV industry, Carbenia eventually pursued a career in education; prior to joining zSpace, he was the director of CTE for St. Lucie Public Schools in St. Lucie County, FL.

“The idea of AR and VR is that it allows somebody to experience something that they’ve never experienced before, or perhaps they’re scared to [experience],” Carbenia said. “It’s never going to replace actually working on a car, or actually welding. That’s not the intent, and I think that’s a misconception that people often think about. This is really about accelerating learning and getting someone to have the option to be exposed to something.”

Last fall, Renton Technical College in Renton, WA supplied students in the school’s automotive technology program with zSpace laptops pre-loaded with course curriculum to accommodate remote learning. According to Warren Takata, automotive technology instructor at the school, the technology actually improved student engagement. 

“Usually when you transition to remote learning, you worry about students completing their work,” Takata said in an announcement. “With zSpace, a lot of them went beyond the regular lessons. It’s really engaging content, and they dove in even further than their assignments called for, learning on their own about motors and transmissions.”

Takata believes that in order to maintain this level of engagement, schools need to be open to incorporating new technologies into their teaching methods. “We’ve taught automotive technology in the same way for decades,” he said. “Students gravitate toward learning with high-tech tools like zSpace. Programs that don’t adopt new technologies are going to be left behind.” 

Carolyn Heinze is a freelance writer/editor.

Frost & Sullivan’s recent analysis, Future of User Interfaces Shaping New Consumer Experiences, finds that user interface (UI) technologies have moved beyond the concept of simply representing machines to their users to enabling sophisticated and personalized interaction. The COVID-19 pandemic has accelerated their use in healthcare, manufacturing, education, retail, and banking to simplify interactivity and improve engagement. 

The global augmented/virtual reality (AR/VR) market is expected to reach $661.40 billion by 2025, at a compound annual growth rate (CAGR) of 86.3 percent from 2019 to 2025, driven by contactless commerce. Beyond 2030, AR and VR will merge, allowing users access to the total reality-virtuality continuum. Meanwhile, the global biometrics market revenue is forecast to reach $54.97 billion in 2025, with next-generation identification, palm vein and behavioral biometrics experiencing significant demand.

Related: Future Vision: Volvo Tests New Car Designs by Driving With an XR Headset

“The need for adoption of new user interface (UI) technologies to alleviate challenges posed by the global pandemic is immediate but constrained by infrastructure issues such as a lack of 5G networks and capable devices,” said Murali Krishnan, visionary innovation group senior industry analyst at Frost & Sullivan. “For futuristic UIs to become ubiquitous, security and privacy will be critical. Organizations must prioritize cybersecurity, business continuity plans, and risk assessments. Further, the transition to the fifth generation of wireless technology will be critical to the success of UI devices. Its ability to connect 1 million devices for every square kilometer will make 5G the backbone for UI devices and overall connected ecosystems.”

Krishnan added: “UI technologies will be an important component in the shift from the office to a virtual workplace that is mobile and more flexible, while interactive interfaces have expanded the scope of businesses to engage with their customers and build deep relationships. Technological advances in mixed reality, holographic displays, computer vision, and wearables will create new application opportunities.”

New technologies will drive growth opportunities in the short term as the adoption rate for user interfaces rises and organizations realize the productivity gains created by UI technologies:

Biometric technologies: With vendors integrating biometric technologies with AI and machine learning, there is a strong market for high-end biometric interfaces in the public and commercial sectors. A combination of iris, vein, fingerprint, facial recognition, and voiceprint interfaces for verification will offer additional security without compromising the user experience.

Digital twins: An estimated four out of five IoT platforms will employ digital twins over the next five years, propelling the global digital twin market to expand at a strong CAGR of 47.0 percent between 2019 and 2025. Retail, automotive, healthcare, manufacturing, energy, and smart cities will benefit. Successful implementations will require high-speed communication networks and no service disruptions.

Brain-machine interface: Neural interfaces could have a significant impact on the restoration of vision and hearing, treatment of mental health disorders, and alleviating pain through electrical stimuli by focusing on the appropriate areas of the brain. In addition, the gaming community is pioneering new gaming control mechanisms based on brain signals utilizing electroencephalography (EEG) headsets.

AR/VR: By 2025, China is expected to lead the VR market, with investments pouring into patents, standards, and products. AR, VR, and robotics will merge to develop VR robots, which will have a huge impact on manufacturing, retail, security, healthcare, and defense. In particular, AR/VR market revenue for manufacturing, robotics, and the engineering supply chain is expected to reach $161.29 billion by 2025.

Future of User Interfaces Shaping New Consumer Experiences is the latest addition to Frost & Sullivan’s Visionary Innovation Group research and analyses available through the Frost & Sullivan Leadership Council, which helps organizations identify a continuous flow of growth opportunities to succeed in an unpredictable future.

Through deep analysis of both key players and technologies, IDTechEx’s most recent report—Augmented, Mixed and Virtual Reality 2020-2030: Forecasts, Markets and Technologies—can provide readers with the trends this market will face in the coming decade.

The report covers one of the key markets of the future: the AR/VR/MR market. VR, MR, and AR products are used in many different settings, for example, for day-to-day workflow management, and on production lines.  This market, which IDTechEx forecasts predict to be over $30Bn by 2030, will impact many different industries, and future innovations will continue its growth in the wearables market. 

The report reviews and analyses over 100 products and details of over 80 companies to create succinct and detailed conclusions about the future of this market. This market research report includes market forecasts, player profiles, investments, and comprehensive company lists are all provided.

The company has also created a second report—Optics and Displays in AR, VR, and MR 2020-2030: Technologies, Players and Market—that offers a concise overview of the markets of optics and displays in both the AR/MR and VR technologies. It includes the forecasts for both the optical and display markets in AR/MR and VR markets. The current state of these markets is discussed and analyzed with long-range forecasts from 2020-2030 for revenue and volume by technology in AR/MR Optics, VR Optics, AR/MR micro-displays, and VR micro-displays.

For more information on both these market research reports, email research@IDTechEx.com or visit IDTechEx.com/Research/WT.

For generations, NYC’s Times Square has been the heart of New Year’s Eve celebrations around the globe, with millions watching and counting down the seconds until the clock strikes midnight. However, this year’s celebrations, like many things, will look a little different. Events will be virtual and socially-distanced from home, putting Samsung’s LED displays at One Times Square on center stage as the world welcomes 2021.

To help everyone safely celebrate the new year, Jamestown—a global real estate firm and owner of One Times Square, home of the New Year’s Eve Ball Drop Celebration—is partnering with several high-profile brands, including Samsung, to create a memorable and immersive virtual experience.

“As a company, Samsung is proud that our company and display products are being used for such a creative and exciting experience,” said Mark Quiroz, vice president, marketing, Display Division, Samsung Electronics America. “To see Samsung’s signage front and center in bringing friends, family, and strangers together for a new virtual New Year’s Eve is something we are excited about.”

[Best Practices for Creating an Immersive Outdoor Holiday Spectacular]

To bring the historic celebration into homes around the globe, Times Square will host the first ever multi-media, experiential technology platform that incorporates and controls live broadcast, augmented reality, virtual reality, entertainment, and gaming experiences through a dedicated App, website, and streaming services.

“Samsung is an integral partner for One Times Square’s New Year’s Eve Ball Drop Celebration, providing the technology for the building’s iconic LED displays,” said Michael Phillips, president at Jamestown. “This year, our partnership extends to our new VNYE experience. Samsung Odyssey continues to help us share engaging content with our viewers not only in the physical world, but now in the virtual world of Times Square as well.”

The NYE App will allow holiday revelers to create a personalized avatar to enter and explore the virtual world of NYC’s Times Square in a year when guests cannot be there live. Guests can walk around the plaza before heading over to One Times Square to access game rooms, and to the observation deck where they can take in the sights of the city along with Times Square’s iconic billboards powered by Samsung LED technology.

Samsung will have a special featured game in the app. The virtual One Times Square building will invite Avatars in to take the elevator to the Samsung Zero G game where they can get lost in a virtual world and win points along the way.  Avatars can also use the Samsung Odyssey Speed Gate to jump through and travel at light speed to reach other locations and hidden spots.

Join in the virtual celebration at  VNYE.com or download the app NYE available in all app stores.

The Audio Engineering Society has released preliminary details on the upcoming International Conference on Audio for Virtual and Augmented Reality (AVAR), set to take place in its own online virtual venue, August 17–19. The event will serve as a hub of innovation for the latest research in the fields of spatial audio and technology for virtual, augmented, and mixed reality, with a focus on technical solutions and recommended practices in the field. 

Registration for this event, sponsored by Facebook Reality Labs and Valve, and hosted by DigiPen Institute of Technology, starts at just $25 for AES members ($150 for non-members) and will be using AltSpaceVR as the platform to share keynotes, panel discussions, workshops, and more. The use of AltSpaceVR offers the option of attending in standard 2D mode or as an immersive experience, utilizing AR/VR devices such as the HTC Vive, Oculus Quest, Oculus Go, Oculus Rift, Samsung GearVR, or Windows Mixed Reality.   

“The necessity to take AVAR 2020 fully online is ideally suited to the conference content,” said Matt Klassen, chair. “Our attendees can plug in entirely, create an avatar, and participate and interact inside a virtual environment. One of the primary advantages of holding the conference in VR is the very compelling nature of the 3D audio for small group conversations—a big step up from flat-screen teleconferencing!” 

“Not only will AVAR 2020 explore practical and effective production workflow for audio production for immersive experiences, it will itself be such an experience and demonstrate an application of the technology that couldn’t be more timely,” said co-chair and treasurer Lawrence Schwedler.    

The 2020 AES Conference on Audio for Virtual and Augmented Reality builds upon the AES’s two previous AVAR conferences and will offer a fresh look at audio production for these rapidly growing market segments. This year’s topics will include spatial audio capture, rendering, synthesis over headphones and loudspeakers, binaural applications, Ambisonics, 3D audio mixing and content production, sound design for VR/AR, and more, plus a featured panel discussion with Emily Ridgway of the Valve audio team on August 18 titled “When Worlds Collide: Audio Development of the Valve Index and Half:Life Alyx.” Additionally, two keynote addresses will take place during the conference: “New Audio Realities at Scale” with Varun Nair of Facebook on August 17 and “Empathetic Technology and Embodied User Experience” with Poppy Crum of Dolby on August 19. 

The AES AVAR 2020 Conference is organized by committee members from a variety of disciplines and experiences in the industry, including Matt Klassen, chair; Lawrence Schwedler, co-chair, treasurer; Pablo F. Hoffmann, papers chair; Sally Kellaway, workshops chair; Greg Dixon, audiovisual coordinator; Randi Ganulin, graphic designer/photographer; Phillip Klassen, social media coordinator; Anna Lantz, webmaster; and Dustin Williams, volunteer coordinator.  

For further information and registration links, click here. 

Disguise has enhanced its software with the aim of helping its community meet production challenges while away from their studios. With its latest release, r17.2, disguise introduces improvements to help users optimize their workflow at home, such as Application Mode, and devise new concepts to connect with audiences online with augmented reality and 360 video.

A key addition in r17.2 is the ability to run disguise’s Designer software in Application Mode. This enables Designer to run alongside multiple applications so users can perform tasks concurrently and switch between its software and other apps. With Application Mode, users can pull content from the web into a project to work up concepts and share works in progress with others to communicate progress, gather feedback, and collaborate better.

Related: Glue Launches Next-Gen VR Collaboration Platform for Dispersed Teams

r17.2 introduces support for HTC VIVE tracking accessories, which make it possible to develop AR experiences at home without high-end tracking equipment. HTC VIVE accessories can be used to emulate camera or object tracking systems, so users can work up AR concepts from lockdown in their home office setup, living room, bedroom, bathroom or kitchen—wherever they choose to augment their world.

r17.2 also allows users to create simple AR screens without the need for other tools, so they can quickly test out ideas for AR experiences. By making it possible to render layers separately, users can also experience better overall project performance and a cleaner workflow.

With the new Spherical Camera in r17.2, users can now render 360 video content in disguise, engaging content for online audiences. 360 video is also a valuable addition to users’ remote production toolkit, enabling them to test out arrangements, gather feedback, and gain confidence before sharing content. Users with a VR headset can also enter the 360 experience or share with others to view using a VR player.

Available now, r17.2 software is designed to help disguise users reach new creative heights at home during lockdown. The software update can be downloaded here. 

In just a few short months, coronavirus, aka COVID-19, has gone from an isolated outbreak in China to a global pandemic. While personal and family health has remained a top priority, we are also modifying the way we do business and educate our children.

If necessity is the mother of invention, adaptation is the father of survival. Learning to leverage virtual reality in our lives is a natural transition. Our modern-day activities that once required an in-person presence can now be accomplished virtually.

Related: Glue Launches Next-Gen VR Collaboration Platform for Dispersed Teams

Virtual reality (VR) has come a long way with capabilities that stretch far beyond gaming. VR can help make the workday feel more professional and education more engaging, all while staying home as you keep yourself and your family safe.

Business

Video calls and conferences are seeing historic adoption as we work to stay connected in a wide range of business situations, but in many cases, they are proving to be insufficient. There’s no substitute for being in the same room as your team members or coworkers. To compensate, VR technology is gaining traction in keeping the lines of communication open in both B2B and B2C activities.

Utilizing VR in businesses allows you to gather with your colleagues without possible exposure to the virus and without the need for travel. Products from Oculus have upped the quality of VR experiences and made virtual meetings and training sessions a reality. It’s now possible for participants to feel as though they’re interacting in the same virtual space. This 3D environment facilitates remote communication and collaboration in unprecedented ways.

There are a number of software solutions emerging to facilitate VR meetings. A PC application called SPACES allows users to join Zoom meetings and other video calls through VR; other more sophisticated software from companies like Glue Collaboration allow for VR-based collaboration.

If you are taking a more individualized approach, Oculus VirtualSpeech gives your employees the opportunity to hone in on their public speaking skills. You can upload slides for presentations and receive real-time feedback on delivery. You can even receive an eye-contact rating.

Education

The coronavirus has led to the temporary closure of schools worldwide. According to UNESCO, these closures have affected over 87 percent of the world’s student population. In a matter of weeks, students across the globe found themselves in temporary homeschooling situations.

New educational approaches could result in innovations to narrow the gap caused by the digital divide, and there are companies that have already been focusing on providing new educational options via the VR learning environment. Teachers around the globe are finding new ways to use VR technology to connect with students, bridging the gap created by social distancing.

Tools like the Expeditions app, originally intended to bring virtual reality and augmented reality into the classroom, could prove just as effective at bringing the classroom into students’ homes.

Apple has created tools such as headsets to combine virtual reality with augmented reality. It is currently developing the ability to record AR and VR experiences to be watched again later or transmitted to others. While it’s already possible to record live gameplay, this application would enable you to capture both real-world and computer-generated elements.

What’s Next

These are unprecedented times and we have to continue looking for ways to interact and stimulate our wellbeing. Through virtual reality, social interaction seems possible and can reduce our levels of anxiety and isolation, along with improving our quality of life.

Justin Schwartz is the founder and CEO of Sage AVT, a service-oriented custom integration company in New York.

To stay up to date with the impact of coronavirus on pro AV, subscribe to our enews. 

You’re behind the wheel, cruising along at highway speed. But you’re not looking at the road—at least not directly. No, you’re piloting your car from behind a head-mounted display, and the world that meets your eyes is a curious mixture of the real and the virtual.

It sounds outrageous, but the practice of test driving vehicles while wearing a mixed reality headset has proven supremely practical for Volvo Cars, and the results have been nothing short of groundbreaking. Spearheaded by Casper Wickman, the company’s senior technical leader of human-centric methods and tools, these technological trials have dramatically simplified the Swedish marque’s process of evaluating new safety systems and design concepts. 

But what’s it like from the driver’s seat? 

“You would think that this would be a crazy experience, like you’re in a video game or something,” Wickman said. “But the thing is, as soon as you put the glasses on and you sit behind the steering wheel and start to drive, then your primary task is to drive the car, basically. Then you really forget that you’re inside a headset. It feels super natural to do it. It becomes your world, basically.”

Of course, you can’t do that with just any headset. To arrive at this destination required a tight-knit development partnership with Finnish XR innovator Varjo—and a whole lot of clever engineering.

Starting Up

Wickman has been helping Volvo improve its processes for two decades. He joined the company in 2000 as a student in its industrial PhD program, which employs a group of 50 to 70 students at its Gothenburg headquarters in tasks like variation simulation and visualization, and the analysis of quantitative data with the goal of eliminating manufacturing defects.

After five years in this capacity, he joined a new department formed to study “perceived quality”—to predict how things will look and feel in the completed vehicle. It’s within this general area that he has remained, working his way up through various leadership tiers to his current senior technical role focused on novel methodologies. 

“We want to develop tools with new emerging technologies to be able to put the user in the experience in the very early stages of design, in order to be able to evaluate whether we are doing the correct things before we do anything,” he said. “I’m working a lot within user experience, but really in any area where you need to put the user in the experience.” 

Getting in Gear

Wickman began considering the prospect of harnessing virtual and augmented reality in his work several years ago, but the technology wasn’t ready. “When it comes to detailing and understanding the look of things in a car context, you need to be able to see the stitching, see the textures, see how things are lined up against each other,” he said. “You need to see more than the complete design.”

Then, a couple years back, he participated in a project that involved using Microsoft HoloLens AR glasses for manufacturing and packaging activities. “But (in my view) they were using augmented reality as if it were an ordinary desktop application in some way,” Wickman said. “They were not utilizing the full capacity of AR.”

After some brainstorming over a better use for the technology, Wickman arrived at the wild idea of driving with the headset. “They answered ‘No, that wouldn’t be possible for several reasons: you wouldn’t be able to track stuff, you wouldn’t be safe,’” he said. 

It was around this time, however, that some other people at Volvo tipped him off to the work that Varjo was doing—specifically, the development of a head-worn display with a pixel density so high the company markets it as “human eye resolution.” Feeding this display with actual footage of the environment from cameras on the headset, Varjo’s XR-1 enables the merging of virtual and physical realities with a clarity never before possible. 

In January of 2018, Wickman set up a meeting with Varjo’s founder and chief product officer Urho Konttori and chief marketing officer Jussi Mäkinen to go over his vision. “We presented that in terms of different steps that we’d like to go through, with the first step just being able to drive with a pair of glasses, using the cameras with video pass-through, to evaluate just how you would manage motion sickness and whether it would be a safe thing to do,” he said.

If that worked, he wanted to begin overlaying data onto the real world; then, he wanted to be able to interact with the virtual content. He also proposed the desire to analyze “gaze data” to see exactly where people looked as they completed tasks. Lastly, if everything else was possible, he wanted to add interactive virtual objects outside of the car. 

“It was pretty cool,” Wickman said. “[Urho and Jussi] said directly, ‘This is the most challenging use case we’ve ever heard of in any industry. Let’s do this together. We’re in.’” 

Picking Up Speed

Over the next six months, as the details of the contract were being hammered out, both companies began preparing on their own: Varjo worked to boost the capabilities of its XR-1 headset, and Volvo went about formulating a way to extract all of the necessary signals from the car. They then established a schedule over the following eight months in which they would conduct their tests. “At each trial, Varjo came over to Volvo and they brought a prototype XR-1 headset, and at each drive they had enhanced a lot of things,” Wickman said.

The setup begins with a computer in the back seat running Unity, the software in which everything operates. A local area network is used to handle all of the signal and data transmission. Mounted in front of the driver is a tracking system called 5C, from German company ART, which follows a set of little spherical sensors on the driver’s headset for spatial positioning. Also running through the LAN to the computer are the signals extracted from the vehicle, which consist of the flex ray and local interconnect network (LIN). These communicate a whole host of information to the system in Unity. 

With this setup, Wickman and his team can then overlay a conceptual virtual cockpit over the actual one and interact with it. “You can read out signals like the steering wheel angle and connect that to the virtual model, so the physical steering wheel and virtual one are synchronized while driving,” Wickman said. Or, “if you press a switch in the car that’s usually used to activate the radio, we can use that signal instead to activate something else on the virtual model. So, maybe we’ve decided that in the next generation of the car to use a stalk or a switch for something else, then we can simulate that.”

Navigating Challenges

According to Wickman, the toughest challenge on Volvo’s end was getting the tracking system right. “A car is really a super complex environment,” he said. “You have magnetic fields, you have different light properties—sometimes it’s low sunshine, sometimes it’s bright, sometimes it’s dark—so it needs to be able to track in different light conditions.” On top of that, you have to deal with the vibrations and bumps from the road.

He said that Volvo and Varjo both tried out a number of tracking systems independently before arriving at the 5C system from ART, which is mounted very rigidly to the car. “That works super well,” he said. “Of course, some of the content might move a little bit, but it’s really impossible to realize that because you’re moving with the car yourself. So you will actually experience it like it’s super solid and fixed to the physical world.”

The other major challenge was latency. The earliest versions of the XR-1 headset had a latency of around 30 milliseconds—not great, but not terrible. “It was barely possible to drive a car with it, because it’s super sensitive,” Wickman said. “If you just look away to the side, then back to the road again, you might end up in the middle of the road.” Still, they managed to complete the test as scheduled, and eventually Varjo managed to shrink the response time down to just 12 milliseconds. 

Then, of course, there’s the occasional hiccup in the system. “We’ve had a number of blackouts,” he said with a chuckle. “Actually, one time on the test track, the computer stopped working, so the headset became black. But we have a procedure for that: the driver says ‘Can’t see anything, take over the steering wheel,’ and they take off the headset. There’s never been a dangerous situation at all.”

The Road Ahead

As of this writing, Volvo had just received its own production version Varjo XR-1 headset, allowing Wickman and his team to begin formulating workflows around it as a bona fide development tool for the company. “We’re setting up a plan for all the future evaluations we’re going to do throughout this year,” he said. “We have a whole stack of things we want to do.”

Before new Volvo models ever hit the road, the company will know exactly what the visual experience will be like for the driver. This starts with visibility, or “how much you see from the car to be able to maneuver it,” Wickman said. “That’s one big application: to be able to bring the new design proposal into the car and overlay that, and then you can drive with the car outside and evaluate visibility.”

Wickman also plans to use the XR-1 to evaluate active safety technologies. Currently, when Volvo wants to test, for example, an automatic braking feature, they have to drive a car toward a physical object. “And if the system fails, you will actually hit the thing you’re trying to avoid,” he said. “Although we’re using soft targets, it’s still damaging to the sensors on the car.” Instead, they can place a virtual obstacle up the road and experiment with new safety systems in the digital realm first.

The next step from there is the development of technology for automated vehicles, or AVs. Wickman said that Volvo is already working on research projects in this field with the help of funding from Swedish authorities. “We’re trying to understand things like how AV cars should behave in terms of overtaking bicycles, for example,” he said. In order to test this the traditional way, you’d need to actually put someone on a bike ahead of the car; now they can do that in Unity. “We just add a virtual object as a bicycle, and a virtual object as an approaching car,” he said. “And you can optimize the algorithms for how the AV car should move without taking any risks for other people while trying it.”

Ultimately, what makes the integration of the Varjo XR-1 into Wickman’s work so successful is that it so directly fulfils the mission of his department: understanding and improving upon the experience people have with Volvo cars. And there’s much more to come.

“We have the perspective of the human being as the starting point for everything we’re doing,” he said. “We also scout a lot of different new technologies. A lot of start-ups all over the world are doing amazing things that we might want to incorporate in our cars in the future. So, being able in a very early stage to evaluate things without the need of actually implementing them, that is a huge advantage.”

The What: Industrial-grade VR/XR developer Varjo has introduced real-time chroma keying and marker tracking as early access features for its XR-1 Developer Edition headset. An industry-standard technique known as “green-screening” used in broadcasting and film, Varjo is the first company to deliver chroma keying in real-time for mixed reality devices. With marker tracking, professional users can instantly anchor any virtual objects to the real world using printable visual markers.

[Varjo and Lenovo Partner on Advanced XR Computing for Enterprises]

The What Else: Chroma keying is particularly beneficial for professional workflows where aligning virtual content accurately with the physical world is crucial. Users can now easily define parts of reality, identify them with color, and replace them with virtual models or scenery without heavy development costs. With chroma key, virtual content also occludes with real-world objects or hands, allowing intuitive interactions. Using Varjo’s object tracking with visual markers, professionals can make virtual objects appear exactly where they want them in their surroundings. Example use cases include:

• Conduct training and simulation: A pilot can sit in a replica of a plane or helicopter cockpit and be able to look outside and see oneself flying in an ultra-immersive visual scenery, while operating physical cockpit controllers for realistic training. Chroma keying also enables multi-user training scenarios.
• Design the products of tomorrow: An automotive designer can sit in a car and replace parts of the interior with designs that are not yet built in reality with the click of a button. Designers can also collaborate in an immersive mixed reality space, interacting with virtual models and making changes to them in real-time, or virtually “dress” 3D prints to look like material-finished products.
• Run academic, clinical, and commercial research: Researchers can conduct studies inside life-like mixed reality, simultaneously combining virtual and real world elements into the research environment. The subjects can hold virtual products or instruments in their hands and interact with them.

“Since its commercial launch in December 2019, Varjo’s XR-1 Developer Edition has quickly become the most demanded mixed reality product for professional users, transforming the way companies train, design, and conduct research in immersive environments,” said Urho Konttori, chief product officer and co-founder of Varjo. “When our customers asked us to create a seamless solution for blending the real and virtual worlds, we immediately jumped to the challenge. We’re excited to introduce real-time chroma keying and object tracking to our customers just three months after the first deliveries of the XR-1, enabling absolute immersion inside mixed reality.”

The Bottom Line: Together, chroma keying and marker tracking allow enterprise customers to seamlessly integrate virtual and real worlds, interact with photorealistic virtual content as they would in real life, and achieve pixel-perfect accuracy and occlusion inside mixed reality. Demo videos showcasing the power of chroma keying and marker tracking are available here.

Both chroma keying and marker tracking are available in early access to all users of the XR-1 Developer Edition headset. 

The What: Lenovo introduced VR Classroom 2, a solution that empowers teachers and administrators to integrate virtual reality lessons into their curriculum. With new classroom management tools for teachers, Lenovo VR Classroom 2 aims to make it easier for schools to get started with implementing immersive education.

[Varjo and Lenovo Partner on Advanced XR Computing for Enterprises]

The What Else: The solution comes with a new headset and content management system, as well as diverse curriculum-mapped experiences, including STEM, virtual tours, and career exploration. Lenovo VR Classroom 2 is powered by the ThinkReality platform, which is designed to make it easier for IT staff to deploy a fleet of headsets. Lenovo VR Classroom 2 also uses LanSchool to improve the deployment of VR.

“Preparing young people for the careers of tomorrow requires smarter technology and a reliable partner to support deployment. Teachers value the unbounded possibilities VR experiences bring to students when it is to easy-to-use and fully supported,” said Rich Henderson, director of global education solutions at Lenovo. “Whether it’s taking a virtual field trip inside DNA strands to teach a unit on genetic engineering, or a guided tour of the Colosseum in Ancient Rome in history class—Lenovo’s VR Classroom 2 offers a seamless journey for educators and students.”

The Bottom Line: Lenovo VR Classroom 2 is a complete solution for teaching with VR, providing fully integrated hardware, content, training, and support designed specifically for middle schools and high schools.

The Lenovo VR Classroom 2 will be available in spring 2020.

The door opened, and I ducked into the cockpit of an Airbus A350. Easing myself into the pilot’s seat, I studied the minute details in the controls and flight map. I reached for the throttle—but my hand passed through it like an apparition. 

Of course, I wasn’t at the helm of an aircraft, but rather in a hotel room in Midtown Manhattan, trying out what is perhaps the world’s most sophisticated VR/AR headset: the XR-1 Developer Edition from Finland-based Varjo Technologies. With a pixel density so high that Varjo calls it “human eye resolution,” it’s easy to get lost in virtual environments. And while you can tell that you’re looking at a digitized version of reality (a very fine graining of pixels is still perceptible), it doesn’t interfere with the sense of immersion or the ability to critically discern details like small print on virtual displays. 

[Varjo and Lenovo Partner on Advanced XR Computing for Enterprises]

All of what I’ve described so far, however, has been available since the company’s first product, the VR-1, was released early in 2019. Organizations around the world are employing it and its successors, the VR-2 and VR-2 Pro, for a variety of applications, including training for airline pilots.

What makes Varjo’s new XR-1 special is its augmented reality capabilities, which, when paired with the company’s just-launched 2D/3D immersive user interface called Varjo Workspace, open a plethora of opportunities for design-related fields. This “Dimensional Interface” is compatible with a number of Microsoft Windows applications and software tools, and enables designers to switch between real, virtual, and mixed-reality modes and modify their creations while experiencing them in 3D.

In my demonstration, the Varjo team of Urho Konttori, founder and chief product officer and Jussi Mäkinen, chief marketing officer, guided me through a few applications that are possible with the XR-1 and Varjo Workspace. At first, I was just sitting in a digitized version of the hotel room—reality captured through the XR-1’s imaging technology. Then, a giant floating screen appeared in front of the room’s actual TV, and Mäkinen played the trailer for the upcoming Star Wars movie on it, proving how smoothly the system is able to render content, even at such high resolutions. 

Next, Mäkinen—who was operating the PC that was driving my headset with a wireless keyboard and mouse—switched me into a virtual environment, an automotive showroom of one of Varjo’s customers, Volvo. At first, the room was empty except for a blue SUV. Then, he inserted a floating display above the vehicle, essentially a Windows computer screen running CAD software. Using the software, he gradually began adding different pieces of furniture to the room around a blue SUV—demonstrating the ease with which one can design an environment from within that environment. 

With Varjo’s video pass-through-based mixed reality, designers can just as easily implement virtual elements into an existing physical environment, greatly enhancing their creative potential. In fact, the XR-1 is so capable that engineers at Volvo are actually driving prototype vehicles while wearing it, to evaluate interior designs and active safety technologies.

According to Mäkinen and Konttori, the company’s software is compatible with CAD and visualization tools including Autodesk VRED, Unity, and Unreal, so designers can simply upgrade their current workflows into the realm of virtual and augmented reality.

“Varjo Workspace shows how professionals will use and interact with computers in the future,” Konttori said. “Varjo is unique in being the only product maker capable of photorealistic visual fidelity in both VR and XR, with perfect color accuracy and control. This is a key enabler for the next computing paradigm.” 

Varjo Workspace is currently shipping to customers and partners as part of the software delivered with the XR-1 Developer Edition. The XR-1 Developer Edition headset is available for purchase immediately at $9,995 and is sold together with Varjo’s Software and Support service at $1,995. Varjo plans to implement customer and partner feedback in 2020 to further deepen its integration with professional design, engineering, and simulation tools.

Varjo Technologies, provider of industrial-grade VR/XR headsets, has announced a cooperation agreement for technical and business collaboration with Lenovo. The new partnership will create Certified for Varjo pairings on Lenovo desktop and mobile workstations for all Varjo head-mounted displays (HMDs). This combines world-leading professional VR/XR products with powerful, high-end computers, enabling enterprise organizations to adopt and implement VR use cases more quickly. 

[Behind Varjo's ‘Human Eye Resolution’ VR Headset]

“Our partnership with Lenovo helps drive a monumental shift in enterprise usage of immersive technologies,” said Timo Toikkanen, president and COO at Varjo. “Combining our products with Lenovo’s pioneering enterprise portfolio products into readymade bundles eases adoption hurdles and quickly gives organizations the tools they need to start taking advantage of the many benefits of virtual reality.”

Professional VR applications including research, training and simulation, industrial design, and more require the most powerful and sophisticated technologies in order to create a valuable user experience. Varjo’s human-eye resolution VR devices, matched with the processing and compute power found in Lenovo’s workstation portfolio products, push the limits of spatial computing forward and deliver the most natural visual experience for exploring and interacting with high-resolution models and simulations.

“When you’re looking for the ultimate VR and XR viewing experience, you look for the only human-eye resolution display available,” said Rob Herman, general manager, Workstation and Client AI at Lenovo. “The technology Varjo brings to the table is extremely sophisticated and requires professional workstation power to bring that clarity to life. That’s why I’m excited to see the partnership between Varjo and Lenovo come to life after months of development and collaboration—certifying several desktop and mobile workstation configurations across our ThinkStation and ThinkPad P Series portfolio. This certification ensures the highest-quality experience for professional applications—spanning across automotive, simulation, training, architectural visualization, and more.”

This global partnership announcement comes on the heels of Varjo’s recent launch of two VR headsets: the VR-2 and the VR-2 Pro.

As the grade school teacher explains to her class the colossal proportions of a blue whale, the students stare in awe as the world’s largest creature glides gracefully beneath their desks. In lessons about Mesopotamia and ancient Egypt, they watch as civilization springs forth from the Fertile Crescent and witness the pyramids rise from the Giza Plateau.

How much more powerful would your imagination be if you had all of this imagery installed in your mind at such an impressionable age? How much more curious would the average student be if he could engage at a visceral level with the material being discussed? And how much more compassionate might we be if we could observe the world from points of view most different from our own?

As I made my way through elementary school in the 1990s, my education was abetted by largely the same tools as my parents’ generation in the 50s and 60s: textbooks, chalkboards, pull-down maps and charts, and overhead projectors. Occasionally, the teacher would wheel in a CRT TV strapped to an AV cart, and we’d all gather around to watch a video. Sure, we had computer labs—but the primitive educational games and software on these machines were hardly memorable, let alone transformative. 

The decades since have brought a deluge of innovation to instruction at all levels of education. Where I had to rely upon photographs and illustrations in text and library books to help visualize the things I read, grade school lessons can now be brought vividly to life with all manner of imagery. With digital whiteboards and short-throw projection, multimedia content is a mainstay of the learning experience from the lowest grades onward. And with interactive devices and applications, students can touch and connect with today’s classroom media, rather than just sitting passively around it. 

But things are really getting interesting as we introduce the ability for students to actually get inside their lessons, by way of augmented and virtual reality. A child goes from learning about tropical rainforests to being immersed in their flora, from hearing about foreign cultures to taking a seat at the center of their traditions. Thanks to technology, future generations will matriculate into society with far more perspective, enthusiasm, and empathy. 

In AV Technology, we take a look at the ways audiovisual technology is enhancing education now, and where it’s poised to go in the next few years. From shaping young minds with AR and VR, to elevating the value of higher education with active learning and AI-enabled lecture capture, AV is a principal force behind a building wave of pedagogical transformation. 

And as students engage with all of these novel technologies, you can be sure that some of them will emerge not only curious about what they learned, but about the tech itself—and wonder how they might one day engineer a way to make it even better.

Parallux, a new immersive technology start-up born from NYU’s Future Reality Lab, will unveil a new shared XR technology with the U.S. premiere of its VR short-film, CAVE, at the Tribeca Immersive programming at the 2019 Tribeca Film Festival.

Parallux’s core technology, known as the Mass Audience Synchronization Solution (MASS), enables large-scale AR/VR experiences for audiences of all sizes. It allows up to 50 co-located audience members to see and hear immersive content—as well as one another—from their unique point of view within shared virtual or augmented environments.

“Unlike traditional 360-degree VR, our solution provides viewers with a sense of shared holographic immersion,” said Sebastian Herscher, co-founder and CEO of Parallux. “Every audience member feels embodied and physically present in a common story world, just as they would when attending live theater or a concert.”

Created in partnership with NYU’s Future Reality Lab, CAVE was designed from the ground up to challenge the status quo of how audiences collectively experience immersive arts and entertainment. During the experience, up to 30 people at a time are transported back to 10,000 BC, when stories were told around a campfire and the history of our ancestors was written on the walls of caves. There, a young woman named Ayara struggles to decide whether to accept her role as Shaman, her tribe’s only link to the spirit world.

In CAVE, Parallux addresses the realities of delivering immersive content to mass audiences today, and gives a glimpse into the future. By maximizing the number of people that can share an experience, the piece avoids the low throughput and long waits often associated with location-based VR installations. It also provides a passive, familiar, and less game-like way for its viewers to experience the virtual world.

“Inspired by the human tendency to gather as a group and tell stories, CAVEattempts to solve the inaccessibility and isolation of AR/VR,” said Kris Layng, Parallux’s chief creative officer. “We believe this is the future of shared cinematic experiences.”

Integrated Systems Europe has announced that Ambarish Mitra, the co-founder and CEO of augmented reality technology company Blippar, will present the ISE Opening Address on February 4, 2019.

Blippar’s mission is to enhance everyday life with AR and computer vision, a field of AI. Since 2011, Blippar has been pushing the boundaries of what is possible in AR. It has helped brands, retailers, and agencies use AR/AI across the consumer journey to deepen customer engagement, drive footfall, and increase sales. 

The company gives access to an AR ecosystem with products and professional services to meet specific business needs—from consultancy to AR creation tools that publish AR content within a client’s own app, or rich media banners with no app required.

Blippar’s technology has been used by leading brands such as PepsiCo, Porsche, Nestlé, L’Oréal, GSK, and Procter & Gamble to create exciting experiences.

Blippar has been ranked three years running within CNBC’s global Disruptor 50 list, alongside the likes of Uber, Airbnb, Snapchat, and Spotify in 2017. The company has also been named a Top Business Innovator by Bloomberg, Most Innovative AR/VR Company by FastCompany in February 2018, and won Best AR App at Mobile World Congress in 2017.

In the opening address at ISE 2019, Ambarish Mitra will discuss how AR and AI are disrupting key vertical sectors served by the professional AV industry, and what future developments are likely to bring.

“Augmented reality is a horizontal disruptor and will have a positive impact on every industry," Mitra said. "What we are seeing today is just a glimpse of the potential of this technology. Blippar is at the forefront of this and I look forward to sharing my vision with ISE attendees.”

Mitra is a serial entrepreneur who started his tech career in 1997 as a high school dropout, creating his first tech company around women’s empowerment in India. After moving to London a few years later, he is today an active private investor in the fields of AR, AI, food tech, and genomics.

His pioneering work in the field of AR has received wide acclaim. He was Ernst & Young UK Entrepreneur of the Year for 2016, and was appointed as an ambassador for the GREAT Britain campaign, which promotes British innovation around the world. He was also recognized as a Young Global Leader by the World Economic Forum in 2017.

“We’re greatly looking forward to welcoming Ambarish Mitra to ISE 2019," said Mike Blackman, Integrated Systems Europe managing director. "Augmented reality and artificial intelligence are fast-growing areas with enormous potential. I can’t wait to hear what he has to say about the opportunities these technologies offer to the world of AV systems integration.” 

The opening address will take place in the forum of the RAI Amsterdam, shortly after the conclusion of the Smart Building Conference. It will be followed by the ISE opening reception. Both events are free to attend.

According to ABI Research, a market-foresight advisory firm providing strategic guidance on transformative technologies, a reevaluation of Augmented Reality (AR) and the surrounding market is necessary to maximize influence and potential. In the most recent quantitative market database, the total AR market value is expected to reach US $116 billion by 2023, spread across devices, software, content, platforms, licensing, connectivity, and more. This growth represents an 89 percent CAGR between 2018 and 2023.

“While the potential of the technology remains significant and revolutionary—with applications from enterprise to consumer, mobile to head-worn—the trajectory the augmented reality market is following has shifted over the past year,” said Eric Abbruzzese, principal analyst at ABI Research. “What was once driven by hands-free information access and strong visualization capability, is now split between maintaining this hands-free capability or leveraging mobile devices for the AR visualization component of the story, which alters not only the primary device type being used for AR but also the content, use cases, and applications being leveraged.”

The growth rate of the overall market hints at a very important trend: growth rate is down when compared to the same window one year earlier, showcasing the beginning of a much-discussed inflection point in the market. With so many waiting to see notable AR adoption, this trend is especially important to highlight. This inflection point carries through 2019, as greater and greater adoption will be seen across markets. Healthcare, retail, education, and media and entertainment are poised for the strongest growth outside of the “traditional” AR power areas in industry and enterprise, all with over 100 percent CAGR through 2023.

While enterprise adoption has dominated the story, the consumer market sees increasing mindshare through the forecast. Apple’s recent hardware event supplied a concrete example of this, with the tech behemoth continuing their mobile AR story with increasing capability of their ARKit SDK with new and interesting applications for their immense user base. Expect the other major smartphone players to follow suit, especially Google and their parallel ARCore SDK. According to ABI Research’s latest data, mobile devices will still make up over 60 percent of total augmented reality usage across verticals and use cases in 2023, which is notably larger than previously anticipated. Revenue continues to flow through the entire value chain, however, with needs for content creation and distribution, platform support, implementation, and more whatever the end user device may be.

“The impact of increasingly capable mobile devices is beginning to be felt, and the effects of this will be realized over the next five years,” Abbruzzese said. “The traditionally strong, ROI-driven verticals will continue to adopt AR at a strong clip, with mobile devices offering another potential adoption path alongside hands-free smart glasses. Markets that may have been held back by being limited to smart glasses—due to the high price, disjointed performance needs, or general use case applicability—now have a well-understood and prolific device platform to push AR applications and experiences. The combination of these factors makes for a feasible and universal augmented reality market.”

These findings are from ABI Research’s Augmented and Mixed Reality Devices and Enterprise Verticals report. This report is part of the company’s Augmented and Mixed Reality research service, which includes research, data, and Executive Foresights.

The Audio Engineering Society will hold the second International Conference on Audio for Virtual and Augmented Reality on August 20–22, 2018, at the DigiPen Institute of Technology, Redmond, WA. The conference and exhibition will bring together a community of influential research scientists, engineers, VR and AR developers, and content creators.

The conference’s esteemed keynote presenters—Jean-Marc Jot, distinguished fellow, Magic Leap; Ivan Tashev, partner architect, Microsoft Research Labs Redmond; and Ravish Mehra, lead research scientist, Facebook Reality Labs—are at the forefront of innovation for VR and AR.

“Audio and video are integral components of AR/VR devices,” Tashev said. “Spatial audio is an area still under development; we need better capture, representation, and rendering technologies. To make it mainstream, we have to catch up with authoring and editing tools as well.”

The three-day conference and expo will focus on the dissemination of top-level research in the field of spatial audio for virtual and augmented reality, with demonstrations and discussions focused on technical solutions and recommended practices. Leading researchers, practitioners, and industry luminaries will offer panel discussions, tutorials, and workshops on new and forthcoming technologies.

The conference promises to be as interactive and cross-disciplinary as the field of VR and AR itself, and will include an exhibition of relevant software and product demos. The conference will bring together talented researchers and developers who are creating the next generation of audio tools for immersive content, with presentation topics including:

• The state of AR/VR audio content development tools and workflows, including personalized versus generic HRTFs
• Realistic sound propagation for immersing the user in virtual environments
• Auditory distance discrimination in cinematic virtual reality
• Spatial audio capture, rendering, and synthesis over headphones and speakers
• Binaural, ambisonics, and wave field synthesis techniques
• 3D sound field navigation
• HRTF modeling and derivation from optical and/or acoustic measurements
• Reverb and room acoustics synthesis
• 3D audio mixing and content production
• Music and sound design for VR/AR
• Higher education and immersive media
• Sound for 360 VR
• XR [extended reality] audio in diverse industries
• Dialogue for VR/AR and MPEG-H

Toronto’s CN Tower, Canada’s National Tower, has released the new CN Tower Viewfinder app, which is designed and developed by Fuzz Productions in Brooklyn, NY. Created to enrich the experience of on-site visitors, the app represents the first and only attraction in Canada to offer a virtual reality observation experience at 1,136 feet.

The app celebrates the City of Toronto’s skyline from the perspective of the newly renovated observation level at CN Tower and features spectacular 360-degree day and night panoramic images. Guests with the app can identify various points of interest across the city and can now answer the question “What am I looking at?” from their own iPhone or Android mobile device.

“As a Canadian icon, we are always looking to expand our digital engagement with our guests from all corners of the world,” said Joanne Pawlicki, director, marketing and communications for CN Tower.

“Through leveraging emerging 3D and virtual reality technologies, the new Viewfinder app enhances guest experience and greater interaction with Toronto’s skyline,” said Kevin McManus, CN Tower‘s director of information technology. “It is also complementary to the CN Tower Experience app, launched in June 2016.”

As part of CN Tower’s unveiling of the newly renovated observation experience, the new Viewfinder app further launches it into a new echelon of digital tourism. Designed and developed by Fuzz Productions, the app calibrates a user’s location on the observation level using the phone’s internal sensors that behave like a gyroscope. This pinpointed calibration allows guests to scan the Toronto skyline and discover over 90 landmarks and points of interest, including architecture, entertainment, and geography. Reflecting the diversity of its guests, the app supports five languages: English, French, Spanish, Korean, and Mandarin. The app is available as a free download at the App Store and Google Play.

“Fuzz couldn’t be more excited to work with CN Tower on a 3D immersive experience in the new Viewfinder app,” said Nat Trienans, CEO and co-founder of Fuzz. “It both enhances the CN Tower visitor’s experience as a digital tour guide to the Toronto skyline, and allows everyone around the world to enjoy the incredible views from CN Tower right on their mobile.”

Future considerations include exploring the addition of Augmented Reality (AR) capabilities by harnessing the soon to be released iOS 12, which includes Apple’s new AR development toolkit. CN Tower plans to also add additional features such as detailed descriptors for each landmark and evaluate capabilities to share postings to social media sites.

A platoon of camouflaged soldiers steps from a Blackhawk helicopter and eases quietly through the yellow-green night-vision glow toward its target. Once the troops assemble against an exterior wall, an explosion blasts through an adjacent door. The midnight raid is on.

It’s not real for visitors to the First Division Museum at Cantigny Park in Wheaton, IL, but a new virtual reality (VR) exhibit with vivid audiovisual elements gets them as close to the action as possible.

“The museum wanted to show the experience of a modern soldier and [what it’s like] to be in their shoes,” said Kevin Snow, creative director at Luci Creative, one of the firms behind the VR experience. “We wanted to create some sense of empathy for how hard it is, through their eyes. The natural progression was to start looking at what we might do through the VR world.”

The First Division Museum tells the history of the first permanent division of the regular U.S. Army, formed in 1917, to serve on the Western Front of World War I, through tanks, artifacts, and more from every theater of war where the division has served.

Part of a major exhibit redesign completed in August 2017, the museum’s VR experience transports 14 visitors at a time to a compound raid from the cabin of a replica Blackhawk. Video content captured from a four-camera rig, including 360-degree footage, gives visitors the soldiers’ point of view, while augmented stereo sound effects blend with human voices for a realistic immersion into a modern military campaign.

“We felt that allowing people to get on board a Blackhawk helicopter, put on a VR headset and be immediately transported to the moment of being on the helicopter with a squad—and then go with those guys through the entire operation, and really see what it would be like—was something unique,” Snow said.

Museums are moving outside the traditional exhibit space by exploring creative ways of getting visitors inside their stories and subjects. Audiovisual technologies are playing a lead role in these immersive and interactive spaces.

At the Museum of Science and Industry in Chicago, a 400,000-square-foot facility that welcomes more than a million visitors every year, a reimagined submarine exhibit combines video, audio, and physical media and effects to put visitors in a simulated World War II battle.

The U-505 Dive Simulator draws visitors deeper into the story of the collection’s 251-foot German U-505 submarine, captured off the coast of Africa in 1944, with an immersive, videogame-based sensory experience.

The simulation begins with a tutorial that weaves in historical context, showing how submarine crews reacted to threats from other vessels and aircraft in a combat setting. Then, the user is propelled into the heat of battle, with a challenging dive order to avoid a torpedo that leads to one of four outcomes based on the user’s skill level.

“The museum wanted to make it really difficult, actually, but approachable at the same time,” said Austin Mayer, interactive director at Leviathan, a creative agency specializing in digital media and interaction. “That [led to], ‘How do we make it fun, how do we make it immersive?’”

Each simulation station faces three HD LCD screens with a 5760x1080 composite resolution, arranged to give the user a wide-angle visual. Audio of the simulated mission—the sounds of the engines, a commanding officer barking orders—hits the user in 4.1 surround delivered by four loudspeakers in the headrest. Two ButtKickers, one under the seat and one on the steering column, take the place of subwoofers and give the sensation of movement and impact.

Creative uses of audio are also part of the Mississippi Civil Rights Museum in Jackson, MS, which tells unvarnished stories of the civil rights movement in the state. Visitors walking through the first two of eight galleries randomly trigger flash points and hear barking dogs or harassment from disembodied voices. The effect addresses the culture of fear that African-Americans experienced in the Jim Crow era in the Southern U.S.

The museum’s centerpiece is a towering, reactive light sculpture that fills the central area connecting all seven other galleries, named “This Little Light of Mine” for one of three civil rights-era rallying songs used in the exhibit. Sensors placed in four different zones in the room detect movement, so as more people gather in the gallery, more voices are heard singing. Addressable LED strips adhered within the sculpture also react with increased intensity as the crowd grows, creating a brilliant, unifying collage of light and color.

“The initial story of civil rights is such a dark story, and it's such a strong story [that] evokes such strong emotions in people, that we needed a space for people to stop and take a breath before they continued their journey,” said Richard Woollcott, vice president of museum planning and exhibit design at Hilferty and Associates.

“The sculpture was designed to remind everybody that it was the combined forces of everyone working together, all of those lights coming together, to create the civil rights movement,” he added.

Ninety miles east in Meridian, MS., the Mississippi Arts and Entertainment Experience incorporates RFID-triggered projection and curved LED screens to educate visitors on the state’s native creative luminaries, including William Faulkner, Oprah Winfrey, and Jim Henson.

Design firm Gallagher and Associates, which counts exhibits at the International Spy Museum and the National Archives in Washington, DC among its clients, drew from its team members’ experience in theatrical art and design for the museum project. The group’s extensive projection mapping experience was key to a series of overhead-projection pieces, including an exhibit on literary figures where 4K video animates a blank writing desk and typewriter with details on the writer’s life and writing style.

“The projection mapping technique we used really connects in a unique way a physical object [with the] cinematic storytelling environment,” said Ariel Effron, creative director at Gallagher and Associates. “That involves the very manual labor of mapping every key of the typewriter to be able to project them pixel accurate. We gain this sort of theatrical magic of the small-scale set.”

The museum’s culinary experience takes this concept a step further. When visitors place RFID-tagged plates at specific locations on a dining room table, RFID readers trigger tailored video content from the overhead projection system. Visitors learn about regional cuisine and the favorite foods of famous Mississippians, like Elvis Presley’s peanut butter and banana sandwich.

The museum also features a video experience that takes visitors on a simulated boat ride through the state’s natural history, using a series of staggered, curved LED screens that engulf viewers for a three-dimensional effect.

“I think [that arrangement of screens] creates a much more immersive environment,” said Effron. “It puts visitors in a very realistic envelope.

“When you take it from two dimensional to the third dimension, the peripheral vision of each visitor is surrounded. You are now inside this environment.”

Augmented Reality (AR) is growing in presence in industrial applications such as smart manufacturing and remote operation of industrial machinery. To serve workers that use devices in a wider range of locations or on the move, cellular connectivity is the better option. The 5G network, with extreme throughput, ultra-low latency, and uniform experience, will be the ideal solution for connected AR/VR experiences. ABI Research, a market-foresight advisory firm providing strategic guidance on transformative technologies, forecasts that almost 10 percent of industrial smart glasses and standalone virtual reality (VR) devices will have a 5G connection by 2026.

“Wearing smart glasses, rather than using AR on handheld screens, empowers the worker to use both hands and look directly at the work that needs doing,” said Marina Lu, senior analyst at ABI Research. “AR will enable shop-floor workers to see a digital twin overlaid on a physical object with assembly or repair instructions according to customized needs. Remote applications that connect field engineers to a remote expert require high-accuracy interaction and low end-to-end latency for time-sensitive applications, and thus continuous connectivity is vital. When users in field service and maintenance are in remote locations where Wi-Fi is nonexistent, devices can leverage 4G and eventually 5G networks to keep these workers connected and safe.”

Connectivity vendors, such as Qualcomm, Huawei, Ericsson, and Nokia, as well as telcos such as Verizon, SK Telekom, and Orange, view AR and VR as one of the prime use cases for the 5G network. Ericsson has recently used augmented reality troubleshooting (ART) at its own production sites in Tallinn, Estonia, and is expanding its use to other Ericsson sites in China. By using ART, the engineers can solve tricky issues with just-in-time fault finding data and immediate information sharing, which can boost productivity by 50 percent. Xerox Israel has deployed AR in the field to improve first-time fix rates, remote resolution rates, and mean time to repair.

Cellular connectivity could expand the possible working area of AR/VR. Low-Power Wide-Area Networks (LPWAN) can efficiently support simple remote devices that do not communicate frequently while remaining energy efficient. The combination of IoT and AR/VR improves the entire value chain for use in manufacturing. Some manufacturers have already started to adopt LPWA, as shown by Huawei and Toshiba’s NB-IoT solution for smart factory monitoring. Flowserve, a manufacturer and aftermarket service provider of flow control products and services, uses real-time sensors with AR to predict pump failure, show the exact steps for making the fix, and share management analytics.

“Mobility is the key to enhance user AR/VR experiences and industry market penetration, which poses new requirements on operator’s network structure and services, but also create new opportunities because only operators can create value in connecting the supply chain, connecting the factory and the product, and understanding the end customers,” said Eric Abbruzzese, principal analyst at ABI Research. “Ubiquitous connectivity is necessary for users to interact with the surrounding environment and receive on-demand information anytime and anywhere. New business models that can leverage connectivity capabilities and bring value to end users wherever they are operating need to be developed.”

These findings are from ABI Research’s Augmented and Virtual Reality Device Connectivity. This report is part of the company’s AR & Mixed Reality research service, which includes research, data, and executive foresights.

Drones may fly high but Kevin Kelly, president/COO, Stampede, has a firm grip on their presence in the AV industry.

“History has shown time and again that the adoption of new technologies is driven by enthusiasts, passionate early adopters who see the potential before anyone else does,” Kelly noted. “This is true in both the consumer and commercial marketplaces.”

New facets of commercial AV can begin as an idea in search of commercial validation, driven by a core group of companies evangelizing the benefits of new technology. “It's the visionaries who open the eyes of the mainstream systems integrators. This has always been the case,” said Kelly.

That gradual adoption through specialists, followed by core adoption by traditional systems integrators is evident with drones. “The adoption rate for any new technology is always due, in part, to the appetite a traditional business has for taking on any new category,” he added. “Drone technology is evangelized by those with a personal interest in it and is driven by folks who get excited by it.”

Although systems integrators are not getting a lot of requests for drones at the moment, as education and government accounts start including them in their quote requests, integrators will incorporate them as well, according to Tim McPherson, sales manager, VSA.

“Integrators and installers will have to pursue the drone market and include drones in their repertoires to remain competitive,” he explained. “Right now, many systems integrators prefer to go with what they know. Sometimes it takes a customer to request a product for it to really take off. So, I don’t think it will be long before integrators start to include drones in their product lines. Especially as Amazon, UPS, and Fedex look to drone delivery for their customers.”

McPherson’s initial drone experience was for a sale to a government contractor. “They used a drone to record a spot for a new Humvee; it was a phenomenal video. Since then, I’ve seen a lot of cool videos taken by drones. Most of the major universities—including Penn State and the University of Nebraska’s HuskerVision—are using them.”

In addition to the education market, McPherson currently works with drone customers ranging from mom and pop videographers to the New York Mets, and houses of worship to communications firms and many amateur video artists. Most clients are interested in how drones can help them stand out from the competition.

Recently, VSA had a top-of-the-line drone project for a parks department that wanted to gain insight as to the health of its various parks without having its employees walk around the parks for hours, or even days. Using drones, the staff was able to see what upgrades and additions were needed in a matter of minutes or hours, and the collected data was readily available.

“From the AV integrator’s point of view, drone technology is something they are probably preparing for,” McPherson said. “Most Fortune 500 companies will partner with integrators when these types of projects come up for maintenance and other needs. Those who integrate and install production equipment, though, already know how drones can benefit the end user.”

So, what's next from manufacturers in drone technology? Intel, for one, is taking drone technology to the next level and generation of products and solutions.

“We see drones as a valuable tool to capture data safely, accurately, and efficiently,” said Cindy Ng, head of marketing, Intel Drone Group. “Our focus for the next generation of drone technologies is increased automation of both the data capture, and, more importantly, the data management and analysis. Managing and inferring insights from these large data sets is driving the digital transformation and will aid in the business decisions of enterprises across industries such as oil and gas, utilities, construction, and agriculture. We can expect to see emerging applications for this technology that help businesses automate everyday tasks and identify, analyze and predict issues in a way that could not be done before.”

Let’s Get Physical

If all new technologies engender a sense of anticipation and excitement, drones do so times ten. “Whenever you can interact with a product because of physical movement, it likens that product to playing video games with a lot of personal movement and direction,” Kelly said. “There’s a visceral feeling of linkage between the hand and eye control and movement. This visceral linkage, coupled with the growing awareness of economic and life-saving benefits drones deliver, will ultimately prevail in moving drones from the enthusiast era into the mainstream business of systems integrators.”

There are, of course, obstacles integrators face when adopting drone technology, he cautioned. “The challenges are some of the same ones that have challenged the development and advancement of every major technological innovation since the dawn of time; they include human resistance, human inertia, legitimate regulatory and safety concerns, licensing, and, yes, security concerns. Slowly, but inevitably, these challenges are being addressed in ways that are ensuring public safety and safety of operation. As they are being addressed, the concrete human and business benefits of drones are being discovered, tested, and proven in real world applications every day. It's this reality that is making drones an inevitable business component for systems integrators and this is becoming more apparent each day.”

The learning curve represents another obstacle for some integrators, McPherson said. “I haven’t seen a lot of marketing to integrators from manufacturers. If you’re like me, you have to have your hands on the product to get real world experience and learn how it applies to the industry and to your customers. As distributors or wholesalers we have to learn as we go. With so many manufacturers and with technology changing every day, I have to know a little about a lot of products.”

Typically, McPherson is on the phone with sixty or more clients per day, many of whom are communications professionals or videographers. “I bring up the advantages and disadvantages of drones in about 25 percent of these conversations.

Public opinion has been another factor challenging the industry, Kelly pointed out. “For the first year in which drones were a consumer product, it was all about privacy. Today, people talk about the economics of drones; they make money and save money. I know of an integrator who put together a drone system for Miami Beach lifeguards. The drones are equipped to fly a life preserver to someone in trouble and can reach the swimmer faster than a lifeguard can get to them.”

Real or Virtually Real?

As the use of various mixed reality technologies grows within the pro AV community, drones will take part in more and more solutions. “If you can program a drone to do something then that drone can be part of an even larger solution that includes VR (virtual reality), AI (artificial intelligence), and smart tech in general,” Kelly said. “In fact, this is a great example of how one radical new idea—drones—suddenly becomes a more mainstream component of even newer, more radical ideas, VR and AI enabled systems.”

And it’s the human imagination that creates the big new breakthroughs, then limits them through fear and inertia, finally liberating them through the creation of the next big idea, he noted. “This is the history of technology. It always has been and, I suppose, it always will be.”

As an AV techie, McPherson is excited to see what experiences VR will provide by drone footage. “We’ll see an increase in drones for AV as costs go down and more people obtain licenses to operate them. The increase will be in the corporate market and in real estate as realtors use them more and more to showcase video projects from various, difficult to photograph, angles.”

In an effort to give back to his alma mater, Middle Tennessee State University (MTSU) in Murfreesboro, Tenn., Nic Dugger, president of Nashville-based TNDV: Television, has donated equipment to MTSU’s Department of Media Arts, one of several Bachelor's of Science programs offered by the MTSU College of Media and Entertainment.

The equipment package included: two DJI Inspire Series quadcopter drone systems—capable of transmitting live, 4K/HD video feeds from close to 1,000 feet in the air—and an Oculus Rift virtual reality system.

As one of the most extensive creative media production programs in the region, MTSU invested $1.7 million in 2016/2017 to build a new 40-foot Mobile Production Lab that students regularly use to learn about producing live TV shows, such as MTSU athletics, music concerts, and other special events.

Dugger donated the drones specifically to expand the mobile unit’s live production capabilities, including the capture and transmission of exciting live aerials and beauty shots, which can be combined with conventional tripod- and shoulder-mounted camera shots, and other live sources.

“As the owner of a mobile production company, we are frequently working with drone video feeds,” Dugger said. “I recognized that having this equipment on MTSU’s truck would give the students an opportunity to acquire the knowledge and hands-on experience they need to be more attractive candidates for entry-level broadcast production positions.”

While a student at MTSU, Dugger worked on the school’s previous HD truck, and has since built a successful career running a mobile production truck business of his own. TNDV has amassed a fleet of eight HD and 4K trucks, including MTSU’s original mobile unit now dubbed “TNDV: Inspiration”, and their latest 4K-capable 53-foot flagship unit, “TNDV: Exclamation”.

“I observed that MTSU did not have drone capability on their HD truck. So, to really give back to the program that benefited me, I wanted to give them not one, but two drone systems to future-proof their curriculum,” Dugger said. “And since the department is also developing a new virtual reality program, I included the Oculus Rift VR gear as a little icing on the cake.”

The DJI Inspire drone system integrates a live HD camera, multiple remote controllers, battery charging devices and video transmission. With a flight time of about 25 minutes, the quadcopter drone flies at speeds of about 90 to 100 kilometers per hour, while relaying 4K/HD 30fps (H.265) encoded video in real-time. The Oculus Rift Virtual Reality Platform includes all the hardware and software necessary to create an immersive VR experience.

Dugger presented the equipment to MTSU Chief Engineer Marc Parrish and MTSU Assistant Chief Engineer Mike Forbes at the end of the spring semester, in mid-May 2018, to give the faculty members ample time to become familiar with their operation before the students returned for their fall semester.

When discussing VR, thoughts often conjure up images of people wearing VR headsets or perhaps, for the more knowledgeable on the subject – a VR CAVE (Cave Automatic Virtual Environment).

But hidden away in an unassuming and unremarkable looking former TV studio on the outskirts of Lille, France – VR takes a new form.

“We believe this is the most immersive example of VR ever built,” proudly proclaimed Yann Coello, director of the SCALab laboratory (Science Cognitive Et Sciences Affectives) at the University, and coordinator of the Equipex IrDIVE investment program.

Designed, built and installed by Antycip Simulation – a French integrator of virtual reality solutions and 3D immersive rooms – The Open Reality Experience (or TORE as it’s more commonly known) marks a technological leap forward to existing flat-sided, walled VR CAVEs.

Capable of housing up to 20 people, TORE – which cost €2.5 million to build – uses a half sphere shaped painted acrylic screen (creating a room within a room) to provide a 180-degree view display with no visible walls or edges, thus removing any sense of depth and height once inside.

“The opportunities are enormous and the possibilities incredibly exciting”

Measuring four metres high by eight metres wide, the screen is eight metres deep. The projection surface is made of eight 30mm thick curved acrylic elements and two flat ones. These had to be delivered on site before being assembled using liquid acrylic; once the material cooled down, the structure was sanded to obtain an even projection surface. The team at Antycip Simulation had to design specific tools in order to assemble the elements. Different coatings were then tested before opting for the one that would offer the best visual performance. The painting process took two weeks to complete.

World without boundaries

Powered by 20 Christie Tri-DLP projectors (placed out of view behind), and using custom-made mapping/warping software (called myIG), the sphere is designed to create an immersive realistic 3D environment to walk around in.

In fact, Antycip claims it’s so realistic, they’ve had to put measures in place to ensure people don’t physically walk into the walls, with the image blurring or disappearing if they get too close.

“TORE represents a major scientific breakthrough for both scientists and technologists, enabling them to benefit from a totally innovative visualisation space that goes beyond the capacities of an immersive CAVE,” gushed Coello who’s been part of TORE since funding was granted in 2012.

“It really does set itself apart from any other CAVE, annihilating the visual disruption caused by cubic shapes. A traditional VR cave would have a front, sides and sometimes a top and bottom. In these environments, the content is not always fluid because you have different sharp angles that break the feeling of immersion. With TORE there are no break in the vision and there is no other structure like this in the world.”

Seeing is believing

To get a clearer understanding, AVTE was invited to test it out for ourselves, with the firm offering three different user cases by example.

But before entering the half sphere – there were two things required by way of preparation.

Firstly, due to the importance of keeping the acrylic CAVE in tip-top shape and the VR illusion unhindered, all those that step onto its surface must wear a set of slippers.

Secondly, and most crucially, users must don a pair of 3D glasses – comparable to those you wear when viewing a 3D movie at the cinema or at home. But, as you can see from the images, these are no ordinary 3D cinema glasses. Equipped with infrared sensors (see picture), these communicate directly with TORE’s tracking system (hidden), which follows the individual’s every movement, including where they happen to be looking, adjusting the image viewed accordingly.

"The glasses decrease visual fatigue, and more readily reinforces the immersion in the virtual world being projected"

VR without the headache

The glasses, so says Yann, provide significant benefits over VR headsets, removing eyestrain, allowing for longer usage, and improving balance due to the wearer being able to see their own body.

“The glasses decrease visual fatigue, and more readily reinforces the immersion in the virtual world being projected,” Yann explained. “After 10 minutes wearing a VR helmet or goggles, it can cause you some problems.

“The main difference with TORE is that you are physically embedded into the scene. So, it’s your body in the scene. When you wear a helmet you cannot see yourself, only the visual scene. Here it all feels real. There’s no feeling of claustrophobia and no feeling that you could be about to trip over something.”

Down to a fine art

With my slippers (forcing me to shuffle around like a moody school child) and glasses on, I stepped into TORE and was immediately transported back in time to 18th century Paris.

The location was Pont Notre-Dame, the most famous formely inhabited bridge in the history of Paris.

Today, the bridge is an empty open space providing pedestrians and vehicles with scenic 360-degree views of the river Seine and surrounding areas – including Notre-Dame cathedral. Back then however, the bridge (think Florence today) was lined with 60 or more buildings and a hive of activity. One of those buildings was (is) engrained in Parisian history – a tiny art boutique called Au Grand Monarque (later A la Pagode), occupied by Parisian art trader Edmé-François Gersaint.

“Careful where you walk,” someone shouted from behind me. “They didn’t have toilets back then”

Despite being demolished by 1788, a multidisciplinary team of scientists specialised in computer and historical sciences and 3D computer graphics professionals were able to bring the boutique back to life digitally for people (such as historians) to view, walk around and experience for themselves today.

“It was rebuilt using a 3D scanner,” explained Yann. “We have taken a lot of different examples from various different pieces of art and literature. It’s exactly how it was at the time, with real images of the pictures on the wall and the size and design of the building.”

Mixed reality

Being a digital recreation, the experience of walking around the boutique was a surreal one – like being absorbed into a computer game or animated film (think Mary Poppins or Tron). But that’s not a criticism.

The objective of being truly immersed was achieved instantly – with no visual boundaries to be seen and a real sense of space and freedom.

The tracking system worked with minimal (if any) latency, ensuring that views will always remain clear and adjusted accordingly.

From there, the setting was changed by a watching technician, sending me out into the street to give me a full view of the boutique from the bridge, whilst also taking in the sights of the surrounding areas.

“Careful where you walk,” someone shouted from behind me. “They didn’t have toilets back then.” I checked, but it was all clear – perhaps something to add in the future? Perhaps not.

A room with a view

The second of the three-user case demonstrations transported me to a room inside the Château de Selles in Cambrai, which was once used as a prison and not accessible to the public.

Unlike the boutique, the visuals creating the room were put together using a 3D camera scanner. The small room provides the opportunity to walk around as if you were there; crouching down to view different carvings and marks created by prisoners during the era it was used.

“It’s all real content, so you are seeing what’s actually there,” said Yann. “For many this would be the only opportunity to experience, view and study this room.”

The third and final room, reverting back to digital again, was a garage holding a Ferrari. This scenario provides the ability to explore the car with great detail, viewing from every angle possible. The 3D effects were so real you could almost feel the car. I even repeatedly moved my head to avoid banging it on an overhead light.

Business case

All three examples are designed to demonstrate just some of the business cases the project can provide.

In total, the project has received €17m in funding as part of a ten-year plan (see box out). Antycip is aiming to have the business commercially operational (and viable) within the next couple of years and is already in discussions with prospective clients.

Its main area of focus includes academic research (computer science, historical science and social science), but the company believes there are some big opportunities for businesses and governments when it comes to design.

This is an expensive toy and needs to be monetised

“This is an expensive toy and needs to be monetised,” said Olivier Colot, CRIStAL laboratory director.

“The idea is to try and emulate some of the ideas with brands. It could be architecture, it could be automotive, oil and gas, telecoms, energy – you name it. The idea would be to bring some funding from projects.

He continued: “If you’re an architect, you can use TORE to make sure the design, the space the lighting – everything is perfect. Perhaps you’re a car manufacturer and you want to test out a new design, so you allow consumers to take a look and sit inside first. It’s basically a decision-making tool. If Audi for example wants to have a new door and they want to make sure it slams correctly, they can test it in a VR setting first.

“On a bigger scale, if you’re the mayor of a city, you could look at and test out different examples of designs – bringing in people involved with the building of cities, politicians and people within public transportation.

“The opportunities are enormous and the possibilities incredibly exciting.”

What’s next?

With audio already possible, adding to the experiences (although not demonstrated on our visit) the next phase of the project will be in the form of interaction during the VR experience. This would provide feedback and the sensation of touching something physically.

“We’re not there yet, but that’s the goal going forward,” said Yann, concluding the tour.

With more than 100 researchers and companies all occupying its head office, all focused on advancing gaming, AV, VR and AR technology, it may not be too long before VR again takes another important step forward.

Underscoring its commitment to expand its reach to more commercial AV resellers than ever before, ZEISS has appointed Stampede to serve as a United States distribution partner for its ZEISS line of virtual reality headsets.

“ZEISS has been committed to contributing to technological progress for 170 years, developing the most innovative solutions in the market,” ZEISS business representative Dave Hodgson said. “Our partnership with Stampede will help us to bring our unique line of virtual reality headsets to more end users in more commercial markets than ever before. Stampede’s experienced sales network and their commitment to success will help us to exceed our growth objectives for the year.”

Offered within the ZEISS line of virtual reality products are the VR ONE and VR ONE Plus headsets. Designed for comfort and simplicity of use, the newer of the two—the VR ONE Plus—is a device that combines premium optics and appealing design. Compatible with select iPhone, Android, and Samsung Galaxy smartphone models, the lightweight design and 100 percent portability make it the perfect companion for videos, games, and augmented reality, according to the company.

Stampede president and COO Kevin Kelly added: “For years, ZEISS has consistently provided solutions that feature the most innovative technology and fit the highest quality standards for performance. We’ve seen growth in virtual reality solutions over the years and with the addition of the ZEISS line of virtual reality headsets to our portfolio of vendor partners, we look forward to furthering this course by offering new and unique solutions that will exceed customer expectations.”

Advanced Education has been named the Canadian distributor for Veative’s educational virtual reality (VR) solutions.

“Veative is a VR-focused education company that creates innovative VR curriculums at an affordable price point to transform how students are learning,” said Advanced executive vice president, Mark McPherson. “Veative is the first content provider in the space to develop an affordable and practical virtual reality solution for schools. It combines immersive and interactive education modules with plug-and-play virtual reality headsets and controllers. We are proud to bring their innovative technology to more K-12 schools in Canada than ever before!”

Veative is a global provider of education technology and learning simulations for schools and industries, using immersive technologies such as 3D, virtual reality, augmented reality and mixed reality. Curating an ever-growing STEM library—currently with over 450 modules—Veative works to engage students with concepts.

“Everyone is talking about VR right now, and many are worried that this is a fad. But we strive to show the importance of having this tool in the classroom, which allows focused attention on topics, and is a valuable part of the learning process. With learning modules mapped to the curriculum, we can deliver a powerful VR solution for K-12 schools that will engage students with concepts, closing the gap between knowledge and understanding,” Dave Dolan, Veative’s director of product management, noted. “Advanced Education is the perfect partner for us to bring our solutions, with modules in both English and French, to Canada’s classrooms. We know that our partnership will be successful based on their impressive track record in introducing new edtech solutions to K-12 classrooms across Canada.”

The What: At InfoComm 2018, IHSE will demonstrate an all-new solution to extend and switch immersive reality signals over long distances using Draco KVM systems.

The What Else: Extending the distance between source computers and users, and switching between VR/AR applications are impediments to more widespread adoption of these technologies. IHSE's Draco KVM solutions allow high-performance computers for VR/AR systems, CAVEs, and large LED walls to be located at a remote distance of up to 80 kilometers via a fiber connection. This approach eliminates the need to locate expensive PC hardware equipment close to users or to purchase multiple high-performance computers. Redundancy options for mission-critical systems are also available.

The Bottom Line: The new solution addresses growing interest in virtual and augmented reality (VR/AR) systems in a wide variety of pro AV market sectors: in corporate and education applications such as product design and prototyping; in situational training and advanced control environments; and in retail and leisure applications including cinemas, shops, and museums.

Immersive, 3D, or spatial audio systems—label them as you wish—are hardly new. But 2018 is beginning to feel like the year that immersive audio will go mainstream, with more than a handful of hardware and software manufacturers rolling out technologies that enable audiences to experience spatialized sound at locations outside the cinema, the home, or a pair of headphones.

One recent example, the ISM Hexadome, a travelling audiovisual installation incorporating video projection and a spatial audio system, featured sound and visual artists including musicians Brian Eno, Ben Frost, and Thom Yorke at a museum in Germany throughout the month of April. The mobile structure, created by Germany’s ZKM | Institute for Music and Acoustics, incorporates 52 Meyer Sound speakers driven by a choice of control software (one by ZKM and two from France’s IRCAM research institute). Intended by the Institute for Sound & Music (ISM) as the first step toward establishing a permanent museum recognizing immersive arts, sound, and electronic music culture, the installation is scheduled to visit locations across Europe and the U.S. throughout 2018.

Art installations, theme park attractions, and live performances are an obvious application for spatial sound technologies, but Sennheiser’s Brian Glasscock, a user-experience researcher who also works on the company’s AMBEO VR mic project and is helping guide the future of immersive audio for broadcast, believes there are everyday applications, too.

“There are a lot of problems in traditional corporate AV installations where 3D audio could improve audio experiences,” he said. “I’m thinking, for example, of conferencing. If we were to enable conference experiences with 3D audio, we could improve intelligibility, improve efficiency, reduce fatigue, and improve productivity.”

Earlier this year, Sennheiser acquired Sonic Emotion, based in Switzerland, which has developed software and hardware tools that render spatial sound and can control virtual acoustics using wave field synthesis (WFS). “We are very interested in how we can present 3D audio experiences outside of headphones,” said Glasscock, who also noted that Sennheiser introduced the AMBEO Soundbar at CES 2018. According to published reports, the soundbar unit houses 13 drivers (six woofers, five tweeters, and two up-firing speakers) to virtualize a 360-degree immersive sound environment.

Kevin O’Connor, in charge of sales and marketing for Sony’s new Sonic Surf VR platform, can also see uses for the technology beyond location-based entertainment experiences. The system, comprising multichannel active loudspeaker modules, a processor, and control software, also uses WFS to position static or moving audio objects within 3D space and will be officially introduced at InfoComm 2018.

Commercial and corporate applications might include foreign language directions in a public space or descriptions in, say, a museum, said O’Connor, who reports that Sony is already building systems for various clients, including an airline. “It’s very directional, so you could have it such that in this three-foot area you’re hearing Spanish. Right next to it, you’re hearing English. And there’s no crossover.”

A demonstration of Sonic Surf VR developed by the Advanced Technology Interactive Group at Universal Parks & Resorts was featured at SXSW 2018 in Austin, TX. “Ghostly Whisper” placed participants around a table at a Victorian-era séance, said O’Connor. “Your head is constantly turning because you feel like there’s somebody talking to you right on your shoulder.”

A second SXSW demo of Sony’s tech involved a 576-speaker set-up that enabled attendees to follow sounds as they move around them in the Acoustic Vessel Odyssey.

“Wave field synthesis loudspeaker systems are also used, for example, in live sound,” noted Veronique Larcher, Ph.D, director of Sennheiser’s AMBEO program. “When a singer or performer is moving across the stage, the loudspeaker system can deliver a congruence between what you see on stage and the sound being emitted.”

Indeed, several concert sound production technologies offering various spatial audio formats have started to attract attention this year. For example, the L-Acoustics L-ISA system, which generates what the company describes as “immersive hyperrealism,” uses software-driven, MADI-interfaced hardware processing to audibly match the positioning of performers on the stage with individual control of panning, width, depth, and elevation for 64 audio objects. Artists have been performing using L-ISA in Europe for a couple of years, but a high-profile tour by Lorde and shows by Odesza and Deadmau5 have only recently introduced it to US audiences.

The manufacturer has just announced partnerships with Avid, BlackTrax, DiGiCo, and KLANG:technologies that integrates L-ISA control into their respective products. Open Sound Control (OSC) implementation allows other external controllers and automation systems to also be used.

At ISE in Amsterdam in February, d&b audiotechnik demonstrated its d&b Soundscape system interoperating via the OSC protocol with QLab and TTA Stagetracker II. At the core of the system is the Dante-enabled DS100 Signal Engine controlled by two software modules, d&b En-Scene, handling up to 64 audio objects, and d&b En-Space, managing the acoustic environment.

Amadeus, a manufacturer of sound reinforcement systems and custom studio monitors, launched its new live sound spatialization hardware processor system, Holophonix, at Prolight + Sound 2018. The processor is based on IRCAM’s Spat engine and, similarly to IRCAM’s Panoramix software, supports a broad range of spatialization options. Holophonix’s algorithms include 2D or 3D VBAP (vector-base amplitude panning), 2D DBAP (distance-based amplitude panning), 2D and 3D Higher-Order Ambisonics, and wave field synthesis. The Holophonix system, which will reportedly start shipping before year’s end, can be custom-configured for MADI, RAVENNA, or AES67 connectivity and works with show control software and DAWs supporting OSC.

The Astro Spatial Audio system, manufactured in the Netherlands, is also object-based. At Prolight + Sound 2018, the company demonstrated its SARA II Premium Rendering Engine, which is MADI- or Dante-enabled and integrated with TTA Stagetracker II performer tracking and QLab show control software. The processor typically supports 32 audio objects, expandable to 64. Current installations include several planetariums and an art gallery.

“We are now at a tipping point where everyone is understanding and promoting the added value of immersive audio for fixed installs,” said Astro Spatial Audio director Bjorn Van Munster. “Immersive audio seems to have been the missing creative link, now bringing light, video, and audio onto an equal creative level, and offering producers and designers an entire new range of possibilities to convey messages to, attract, and blow away the audience.”

For Richard F. Morse, the process is as interesting—and valuable—as the final product. Currently serving as the media technical specialist at Franklin Pierce University in Rindge, NH, Morse is encouraging students to collaborate and push the boundaries of creativity. New technologies are proving to be useful tools for educational inquiry.

INTERDISCIPLINARY ENGAGEMENT

Morse knows firsthand the value of interdisciplinary approaches in university classrooms: he holds a BA in literary studies from Middlebury College in Vermont and an MFA in film and media arts from Temple University in Philadelphia. In his post at Franklin Pierce University, he is enhancing pedagogy with audiovisual equipment and cutting-edge technologies. As new media tools like AR, VR, and 360 rigs hit the market, Morse sees exciting new opportunities for next-level storytelling.

As many higher education stakeholders know, every new semester ushers in different technological requirements, from projector adapters to BYOD support, to software updates. That often translates into needs assessments and refining departmental missions. A new endeavor in 2017 for The Depot—the division that provides the university with professional-grade audio and video field equipment, including six non-linear editing suites with Adobe Creative Suite software—meant splitting from the information technology’s media services function. Last year, The Depot moved into the Marlin Fitzwater Center for Communication (known as “the Fitz” and “Fitzwater”), a high-tech hub that houses innovative departments, vibrant academic degree programs like the communications department, social media and emergent technologies (SMET), and extracurricular programming. “The Marlin Fitzwater Center for Communication is all about the student experience,” said Dr. Kristen Nevious, director of The Marlin Fitzwater Center for Communication.

SOCIAL MEDIA AND EMERGENT TECHNOLOGIES

An interdisciplinary team of faculty passionate about technology—professors Paul Bush, Heather Tullio, Gary Heald, Dr. Raysa Erramilli, and Dr. Kristen Nevious—worked under dean Kerry McKeever’s guidance for several years to develop and create the social media and emergent technologies major, which was among the first of its ilk in the country. While the program was going through the curriculum committee, the university supported the GoPro technology at a time when it was cutting edge.

“There were few training courses to attend,” Nevious said. “We all—students and faculty alike—learned through immersive learning experiences. Our student media have been out in the field using the technology since the summer of 2016, covering the two national political conventions and exploring its storytelling capacity on campus.”

Professor Heather Tullio has used those experiences to develop the 360 Storytelling course, and this past semester was the first in which students were eligible to enroll. “At the same time, I worked with the student media to explore augmented reality applications in the newsroom, and I developed the course, which will be offered for the first time in the fall,” Dr. Nevious said. “We look forward to graduating our first SMET students in 2020, and we have no doubt they will be leaders in the field.”

Also new to SMET is professor Christopher Previte, who is the first full-time member of the program, and he is already contributing his expertise. He and Tullio are both leaders in the campus digital literacy movement.

NEW CHALLENGES, NEW LEARNING EXPERIENCES

With the transition from IT to Fitzwater came Richard Morse. His responsibilities now include teaching new media courses, managing studio operations, and supporting student and media productions.

Morse’s first months working in Fitzwater were busy with the transition from IT and related tasks such as streamlining audiovisual equipment inventory management. On any given day, you can find him teaching in one of the studios or classrooms, or supporting student editing or the recording of podcasts in the acoustically treated Internet Radio Studio. He also leads webcasts using the NewTek TriCaster and manages campus productions for live events. One such production included an ambitious six-camera shoot of the Presidential Installation, all with a student crew.

The cutting-edge technologies that Morse uses in the field are woven into his course syllabi, including his popular new class, 360-Degree Storytelling.

“We’re using our GoPro 10-camera array and VideoStitch, the stitching software,” Morse said. “We’re editing in Adobe Premiere, in the new 360 VR interface. The students are learning how to use 360-degree film technologies, and we’re exploring techniques to tell different kinds of stories.”

One example of innovative storytelling is a music video that the students are directing, filming, and editing. The project tells a story through dance and movement, calling upon the diverse interests of the Franklin Pierce students. From student choreographers and dancers to videographers, editors, and lighting experts, this music video is a case study in collaboration.

THE (VERY) BIG PICTURE

Other fascinating projects in which audiovisual technologies play a pivotal role are a new 360 video of the student-run volunteer fire department and a video of a gallery exhibit. “We’re using 360 in three different locations for the gallery video,” Morse said. “The professor, who is the artist, is recording a voiceover in studio B, and we’ll use some of his voiceover. We’ll be talking about his art, methods, inspiration, and all the other elements of his process.”

Morse explained how the students plan to cut the audio track and create a cohesive narrative using visuals from 360 to support it. “So if [the artist] is talking about what’s on the other side of the room, you can use the 360 technology to spin around and look at the painting that he’s talking about.”

ACADEMIC AND SYSTEMATIC

While the 360 rig was on campus before Morse arrived, he is a key part of the initiative to bring 360 video into the academic arena by teaching the inaugural course in 360 Storytelling. He is also slated to teach an augmented reality course next semester. A recent grant received by the school will help to upgrade the campus TV studio as well as its augmented reality and virtual reality tools. Oculus Rift virtual reality systems are one of the technologies that the Fitzwater Center received, as well as new 360-degree cameras that will allow students to shoot in and create stereoscopic content. “That is something that I’ve been a part of, as far as bringing to the school more ways to use our virtual reality in an academic context,” Morse said. The next logical step is how to use content and techniques honed in the academic setting for “telling stories in the real world.”

There is another valuable lesson to having emerging technology on campus: training students to be respectful of the equipment. “We know that we are on the forefront and technology is not easily replaced. It is expensive, and we want to get the most out of it before we move on to the next.”

ENHANCING CREATIVITY AND CAREER READINESS

While interfaces and operating systems change, the one constant is that there will be a new platform or new tool faster than you can say “hashtag.” Morse observes that his students are mostly comfortable with AV technology usage, even the latest iterations.

“They take to new technology really well,” Morse said, and shared that one of the attributes he appreciates about Franklin Pierce students is their “can-do attitude” and intrepidity in pursing goals. “In a way, they are fearless,” he said. “I tell them that they can make mistakes, so they’re willing to work, try new things, and experiment with the technology. And I think that they’re naturally inclined to some of the VR work because a lot of the students are gamers. They have headsets at home.”

Morse believes that the deployment of 360, VR, and AR technologies is happening across higher education, though some institutions are making it more systematic.

“The demand is increasing,” he said. Though many schools are just getting starting with teaching VR and augmented reality, “there is a desire to consume this kind of immediate content.” He recently attended the Intercollegiate Broadcasting Conference in New York where he met other media technical specialists and teachers. “We were exchanging ideas and sharing course materials.”

Morse’s advice for technologists and professors who want to make forays into emerging technologies, or pivot from traditional broadcasting into AR, VR, and 360, is to attend conferences with specific AR and VR tracks. He also encourages media technicians and AV departments to help faculty explore the possibilities. “I will be conducting faculty development to show someone in the science field how they could use an Oculus Rift. If they’re teaching anatomy, they could use some of the scientific programming that’s out there already and use that to enhance and augment their classroom experience. It can support the ways they teach and help reinforce core concepts.”

Ultimately, Morse believes what we now consider “new technologies” will become integral parts of our daily life. People are already interacting in their homes with smart speakers—voice commands are bridging the algorithmic and augmented reality worlds, he said. The fact that Franklin Pierce is at the nexus of emerging technology is significant.

“It’s unique at our school, being a campus of 1,200 people, that we are able to implement these technologies and have students on some of the latest equipment, so when they graduate and move into the workforce they can say, ‘Oh, I’ve used this. I know how to stitch. I was on the first version of it and I’ve watched it mature.’ I think that’s impressive that we’re able to give students these kinds of opportunities.”

Margot Douaihy is the editor-at-large of AV Technology.

Welcome to The Fitz

The Marlin Fitzwater Center for Communication—affectionately known as “The Fitz”—at Franklin Pierce University is dedicated to educating leaders of conscience in public communication. In addition to housing the University’s Department of Communication, the Fitz is the home for the University’s nationally recognized polling operation and the Pierce Media Group, which consists of the student-run newspaper, TV studio, Internet Radio Studio, and Raven Sports Broadcasting Network. http://www.franklinpierce.edu

Snapshot of Tech in The Fitz

• * Adobe Creative Cloud
• * Backbone Internet Radio Production Suite
• * Canon DSLR Cameras
• * GoPro Cameras and 360 Rig
• * Insta360 Pro Cameras
• * Marantz Recorders for Location Sound Recording
• * NewTek Tricaster and Tricaster Mini
• * Oculus Rift
• * Shure, Sennheiser, and Sony Professional Microphones
• * Microsoft HoloLens
• * Television Studio – FPTV

Go Figure

As reported in NewBay’s Installation, figures from analyst IDC forecast that global revenues for the AR/VR market will increase 100 percent or more over each of the next four years, with spending on AR/VR products and services expected to soar from $11.4 billion in 2017 to nearly $215 billion 2021. Take a VR Tour of Franklin Pierce University campus: https://www.youvisit.com/fpuravens/101873

The What: Following its initial introduction to the mobile market at MWC 2018, Dirac Research will showcase its second-generation Dirac VR 3D audio solution to the U.S. market at VRLA 2018, from May 4-5 at the Los Angeles Convention Center at booth 332.

The What Else: Dirac’s VR 3D solution possesses a sophisticated 3D audio engine that produces low-latency audio, smooth sound renderings, and highly accurate audio localization in the virtual X-Y-X plane. The second-generation Dirac VR has increased its overall processing efficiency through the further optimization of CPU and memory usage, thereby making it a better solution for power-constrained devices.

“Enormous potential across nearly all industries will be unlocked with the advent of truly immersive virtual reality; however, this potential and promise will forever ring empty without a perfected 3D audio solution that produces soundscapes that are 100 percent true to life,” said Nadeem Firasta, Dirac’s VP of product management and business development, North America.

“After over a decade pioneering digital audio technologies for the automotive, headphone, mobile, and home theater markets, we now possess this 3D audio solution and we look forward to demonstrating to manufacturers, content creators, and end users the worlds that can now be created and experienced with Dirac VR,” Firasta continued.

According to the company, the VR 3D solution’s underlying technology is more than 10 years in the making. Dirac’s 3D reverb engine, for example, is built upon a Dirac Unison solution used in high-end automobiles, such as the Volvo XC90, to reproduce in-vehicle acoustics that mirror places like the Gothenburg Concert Hall. In addition, Dirac VR’s ultra-low CPU and memory usage is achieved by leveraging the company’s experience creating digital audio solutions for other low-bandwidth devices such as smartphones and smart speakers.

According to Lars Isaksson, Dirac’s general manager and business director of AR/ VR, Dirac VR offers Unity developers a tool to produce increasingly sophisticated virtual reality experiences—ones that involve dynamically moving audio sources positioned around the user on the X-Y-Z plane, separated from one another by as little as one degree.

“Dirac VR equips developers with the 3D audio platform to create the acoustics of any world that exists within their imagination—regardless of complexity and sophistication,” Isaksson said. “Users can walk through a jungle teaming with wildlife and be immersed in the aural experience of monkeys howling as they swing above, frogs croaking as they hop below, and toucans squawking as they fly in the distance. We look forward to watching, and listening, to what the world will create with Dirac VR!”

The Bottom Line: Dirac VR’s 3D audio platform is rooted in a proprietary method of dynamic head-related transfer function (HRTF) measurement, which no longer requires individualized measurements of one’s head size, ear shape, and more, in order to create realistic soundscapes tailored to each user’s cranial dimensions.

Companies including FoxNext, Zero Latency, and Disney's ILMxLab with Star Wars are demonstrating that location-based virtual reality (LBVR) is a viable business for investment, which is enticing mainstream consumers to satisfy their VR curiosity. This is demonstrated by the high levels of growth, with consumer spend expected to reach $809 million by 2022, according to the latest VR Tracker Report from Futuresource Consulting.

"Q1 2018 saw significant growth and rising consumer spend, which is expected to lead to full-year spend of $299 million in 2018 across the four types of LBVR categorised as arcades, multiplayer experiences, VR cinemas, and VR theme parks," said Carl Hibbert, associate director of consumer media and technology at Futuresource Consulting. "Of all four categories, 'VRcades' has seen the fastest take off. The category was estimated at $79 million consumer spend in 2017, accounting for 40 percent of all LBVR revenues."

The VRcade category has the highest number of locations, with an estimated 4,000 worldwide, of which those in China account for 75-80 percent. However, the VRcades, particularly those in China, offer low-cost and consequently limited quality experiences, according to the Futuresource Consulting Global Virtual Reality Tracker Report.

However, at the other end of the spectrum are high-end, free-roam, and interactive multiplayer 'location-based' VR experiences. By 2022, multiplayer VR experiences will account for 41 percent of Global LBVR consumer spend.

"Multiplayer LBVR allows players to roam free, untethered, and with others in a VR environment,” Hibbert said. “This category is expected to have the highest CAGR rate across the forecast period, despite a lower number of installations. This is due to consumer desire to pay for sociable experiences with friends, especially in more developed markets. Furthermore, the majority of multiplayer installations offer original content restricted to franchise partners, thus remaining exclusive and adding to the attractiveness.”

Notable players in the market such as The Void and IMAX are banking on using well-known franchises to support their rollout of LBVR and support consumers' appeal of this new tech medium. But with the cost of popular franchises restrictive for many, start-ups including Zero Latency are building their own content and experience portfolio.

Location-based VR is seeing popularity as the entry point for mass consumer adoption of VR, as it serves well to increase consumer awareness and engagement with the technology. The at-home market has seen slower-than-expected uptake, whereas LBVR allows consumers to experience VR without the large investment costs involved in purchasing hardware.

Virtual reality is not new. Simulation spaces such as CAVE (cave automatic virtual environment), and other simulators have been around for nearly two decades. However, the collective technologies that have traditionally gone into such largescale environments were astronomically expensive. But times are changing, and technology that was once reserved for those with the highest of budgets is making its way into all facets of everyday life—including delivering amazing up-close-and-personal experiences to enrich the learning experiences of fertile young minds.

For most schools, integrating virtual reality into the classroom is in its infancy. “It starts with building awareness,” said Maya Georgieva, co-founder at Digital Bodies, an immersive VR/AR/MR consulting group. A small number of students might have experienced Samsung’s Gear VR or Sony’s PlayStation VR headset, but the classroom experience is different.

Georgieva recommended schools first build an awareness about VR and AR outside of the classroom in an extracurricular activity. Then, employees can “think of ways these emerging technologies can connect to what are taking in place in the classroom. Often it is during these events that faculty see the potential of how the technology may connect to their areas of study, research, or classroom.”

Most places only have a small number of high-end VR headsets Georgieva noted. In these cases, other students in the class will see what the headset wearer is viewing on a video wall. “For me it’s not really about wiring every student in the classroom to have a headset on their face,” she said. “Where it is useful now is to enhance the learning experience by having students step into faraway places or witness something up close, and in some cases, being able to walk or interact with objects within the space. Then use these new experiences as an opportunity to enhance the classroom discussion bringing a unique perspective from having been there or stepped into somebody else’s shoes.”

Mixed reality (MR) is taking augmented reality to the next level. “When we talk about mixed reality, it is not only a digital overlay of contextual information, but one where digital objects are integrated into—and responsive to—the natural world and you can actively interact with them,” Georgieva said. This is where products such as Microsoft’s HoloLens and Magic Leap come into play.

Visualization technology, encompassing VR and MR, was cited in EDUCAUSE Center for Analysis and Research’s Higher Education’s Top 10 Strategic Technologies for 2017 report. “What they have in common is that they tap the brain’s inherent ability to rapidly process visual information, identify patterns, and sense order in complex situations,” the report noted.

AR/VR/MR have the ability to enable educators to create a personalized learning experience. “At its core that means that with these new immersive technologies, we are able to give students the opportunity to practice and fail safely,” Georgieva said. “Nothing really breaks in the virtual world the way it does in the physical world. We can respond to students individually and provide feedback to them contextually. This is where I think virtual and immersive technologies will play a huge role in the future of education.”

The Audio Engineering Society will hold the second International Conference on Audio for Virtual and Augmented Reality on August 20-22, 2018, at the DigiPen Institute of Technology, Redmond, WA. The conference and exhibition will bring together a community of influential research scientists, engineers, VR and AR developers, and content creators to explore such topics as: spatial audio capture, rendering, and synthesis over headphones and speakers; binaural, ambisonics, and wave field synthesis techniques; 3D sound field navigation; HRTF modeling and derivation from optical and/or acoustic measurements; reverb and room acoustics synthesis; 3D audio mixing and content production; and sound design for VR/AR.

The conference’s keynote presenters—Jean-Marc Jot, Ivan Tashev, and Ravish Mehra—are at the forefront of innovation for VR and AR.

According to Tashev, “Audio and video are integral components of AR/VR devices. Spatial audio is an area still under development; we need better capture, representation, and rendering technologies. To make it mainstream, we have to catch up with authoring and editing tools as well.”

The three-day conference and expo will focus on the dissemination of top-level research in the field of spatial audio for virtual and augmented reality, with demonstrations and discussions focused on technical solutions and recommended practices. Leading researchers, practitioners, and industry leaders will offer panel discussions, tutorials, and workshops on new and forthcoming technologies.

The deadline for submission of papers and workshops is May 1, 2018. For more information content submissions, click here.

The conference aims to be as interactive and cross-disciplinary as the field of VR and AR itself, and will include an exhibition of relevant software and product demos. Organizers expect to attract talented researchers and developers who are creating the next generation of audio tools for immersive content.