Enhancing Live Streaming with Mixed Reality
Although there's quite a bit of hype for augmented and virtual reality (AR and VR, respectively), a closer look confirms that these technologies have valid applications, especially for live streaming events. One of the most promising technologies is mixed reality (MR).
MR is a combination of various levels of AR and VR that blends the real world with the virtual world, in effect creating a totally new environment. These tools can be directly integrated into a live streaming experience that offers a higher level of audience engagement than standard live streaming does.
With MR, companies can enhance their presentations and events by adding holographic, computer-generated graphics, animations, and 360° sound to live video. This allows viewers to visualize, manipulate, and act upon objects or events during the presentation. Viewers are able to better understand complex concepts, detailed contextual information, abstract data, and ideas beyond the capabilities of typical presentations.
Live streams can now take full advantage of these capabilities due to the rapid increase in processing power of mobile devices, smartphones, and PCs. In addition, consumers are now more aware and have become more familiar with the apabilities of AR due to the success of apps such as Pokémon GO. Even apps like Snapchat delve into AR with their data and geofilters.
Mobile devices, which are a great stepping stone to mass mixed reality adoption, can serve as the head-up display (HUD) or overlay when presenting graphics, data, analytics, and statistics. It's only going to get better with the release of high-definition headsets and wearables that will combine AR, VR, and MR technologies in an immersive and dynamic environment.
Real-Time Interactive Video, Graphics, Virtual Objects, and Data
Imagine if a presenter could show computer-generated representations of a product where viewers could manipulate the object on their mobile devices and even zoom in and out while observing additional details and information through graphic overlays. Or during training sessions, an instructor could show full processes and comprehensive simulations as if he were actually on-site with fully interactive 3D models and locations. A presenter could even be composited into a virtual environment, or a virtual object could be brought into the real world where physical movement could be tracked within the environment. To get an idea of these scenarios, check out "Fantastic Contraption Mixed-Reality Stream Sample."
I foresee that real-time video game engines such as Unity and Unreal will eventually be merged with live video.
Analytics and Metrics
Another great feature of AR/VR/MR is that companies can measure and evaluate user attention on a granular level and determine what they are most interested in through visual analytics with gaze/eye/gesture tracking and interaction analysis of each live stream. This will allow for custom experiences, since it can be modified in real time while adjusting to the behavior of the viewer on-the-fly.
Despite MR's initial high cost, you can now leverage existing software and services to take the initial steps in developing basic MR features in your current live streaming productions within a limited scope or budget. Content creators and live streaming producers should look at MR as the new paradigm since it simply allows a better and more effective method of presenting information. If you aren't considering how to incorporate MR into your productions, you are missing out on a great opportunity to communicate, captivate, and resonate with audiences in a whole new way.
[This column appears in the Septemer 2016 issue of Streaming Media magazine as "The Cutting Edge: Enhancing the Live Experience with Mixed Reality."
Related Articles
The rush is on to create mixed reality studios capable of creating immersive experiences that merge the real with the virtual. Here's how they operate.
28 Jun 2017
Trying to compete with more established platforms YouNow and Live.me, Busker lets video creators sell physical or digital goods directly to fans.
11 Oct 2016