Another company demoed a public speaking training app that used eye tracking to make sure you’re focusing on your audience instead of the teleprompter. It even places you in a depressing windowless DoubleTree conference room (tacky carpeting and everything)!
At CES, Lockheed Martin demoed a flight simulator , aimed at augmenting the user’s “physical flight stick and throttle controls with gaze tracking that can be used to activate the numerous subsystems within a fighter jet for a drone shoot-down mission.”
You probably haven’t seen too many (or even one) Hololenses in the wild. But behind the iron curtain that is the United States defense department, Microsoft struck a $480 million deal with the The Army.
Think about it: the United States Military can create a virtual environment where adversaries are attacking from dozens of angles, using a barrage of tactics. They can put a headset on thousands of First Lieutenants, and train them to look for snipers, identify explosives, maintain fighter jets, assemble weapons and mitigate hundreds of risks, all from a conference room at Fort Campbell.
The new Vive Pro Eye and Microsoft’s latest mixed reality headset, the Hololens 2 , does just this. Suddenly, eye tracking can be leveraged for foveated rendering, of course. But it also enables in-app controls (e.g. look down at a loot box just to open it) and analysis of user attention during training.
Foveated rendering. It’s a fancy term for how VR headsets are finding computational efficiencies. Our brains already do this, to an extent. If our eyes are focused on our iPhones, for example, then the world around it melts away. Our peripheries become blurred as our brains focus on only the thing in front of us. If VR headsets could track where you’re eyes are looking, they could lower the resolution of everything happening in your peripheral vision, effectively saving a ton of graphics rendering power.
The U.S. Government Loves VR. Both NASA and the U.S. military are investing in virtual reality. NASA uses the technology to try to connect engineers with the devices they send into space. Using the Oculus, and motion sensing equipment from the Xbox One gaming console, NASA engineers are developing ways to control a robotic arm with gestures made by the operator here on Earth. The military uses VR to recruit and to train soldiers before they are deployed. The simulated scenarios provide opportunities for teams to work together in immersive, realistic environments to better prepare them for the chaos of combat.
So far, these things are mostly in California malls. Assuming they see more commercial success than IMAX, they’ll start to expand. You may be wearing a headset and toting a computerized vest sooner than you think.
None of this is surprising when you consider the video game industry is outpacing movies and music . These companies, according to PWC , are always trying to diversify “beyond the dominating content categories of gaming and video to find new opportunities, especially in enterprise VR apps or venue-based VR experiences.”
They’ll be joined by similar ventures like SPACES , backed by Comcast, and The Void , backed by Disney.
Enter: Dreamscape Immersive . The company, backed by IMAX (lol), AMC, Fox and Warner Bros, is aiming to do what IMAX couldn’t. According to The Verge :
So wouldn’t it be better, then, if there was an experience where everything — the story arc, the props, the equipment, even the waiting room — is designed to transport you to an alien zoo, or an abandoned ship at the bottom of an ocean?
So how’d they screw up? One theory is the basis behind creating the venue in the first place; the centers felt more like promotional vehicles for their core product — that being IMAX theaters and movies — and less like an immersive storytelling experience. They lacked focus, sometimes acting as a show floor for new VR technology instead of an entertainment destination.
Creating a centralized space for consumers to experience a new technology so that over time — as the tech becomes more affordable and performant — they’ll bring it into their own home? Sounds like a recipe for success (see: arcades → PlayStation, drive-in theaters → Netflix, laser tag → MMOs etc). That, and the “location-based entertainment” market for VR experiences is expected to grow to $12 billion within the next five years.
Some of you may remember (but most won’t because BOY they flopped, hard), IMAX raised $50 million to open “VR Experience Centers” in major cities worldwide. They successfully opened in New York, L.A., Manchester, Bangkok, Shanghai, and Toronto, only to announce in December of 2018 that the company is shutting down its remaining locations.
Five years ago, Sixense debuted the STEM System controllers on Kickstarter, promising a more intuitive, full-tracking way to interact with virtual reality. The past year has been one of change for the company, as it's switched CEOs and acquired Personify's Teleport system, which is used to capture augmented reality footage of people or things.
The stark contrast between investment and how users actually use VR, MR, and AR begs the question: what should we expect in 2019? In short, I propose five things, or “need-to-knows”:
Yet there seems to be more AR news about Bhad Bhabie than how it’s altering our spending habits. And while VR is seeping into every industry to “increase empathy” and train employees, it’s far from occupying a space in every living room, conference room, and treatment facility.
Virtual, mixed, and augmented reality. The heads-up display and reality-altering technologies that were supposed to revolutionize everything from gaming to socializing to porn. Despite a lack of VR “killer apps” and AR app adoption, the industry is growing ; they’re expected to hit $209 billion by 2022, and an aggregate one billion users by 2020.
Virtual Reality Has ROI. While it might feel as if virtual reality has been around forever, it’s still a relatively new technology. This has caused some businesses to question whether virtual reality is actually beneficial. The truth: of course virtual reality has shown to have positive ROI. British travel group Thomas Cook reported a 190 percent increase in tours booked to New York City after offering a virtual reality experience of the city in their stores. Amnesty International reported a 16 percent increase in direct-debit donations brought on by its VR campaign.
Those are two real examples that actually already exist in the world. Imagine the possibilities: UX and CX (user and customer experience) professionals can test consumer attention and preference. Medical students can explore digital cadavers for passing grades before stepping into an OR. Hell, baristas can identify steam wands from the steam tips before risking burning themselves on real espresso machines.
In 2019, VR, due to eye tracking, will make great strides entering industries of all kinds as a B2B application; in healthcare, education, defense, aviation, advertising, e-commerce, retail, and more. Analytics platforms already exist. This is just the beginning.
Need-to-Know #3: Expect increased VR hardware development and adoption (Or: Oculus and Playstation will set your limbs free!)
There was a pivotal moment in the development of VR when individuals could strap on a headset and successfully (i.e. without throwing up) view an immersive experience.
In other words, the fields of view were wide enough and latencies were low enough to come close to the human brain’s preferences (which clock in at 180º for field of view and 7–15 milliseconds between the time a player moves their head and the time the player sees a new, corrected view of the scene).
Cool, but you’re still confined to a chair, or tethered to a computer, or forced to use mysterious space ladles to get around.
This year, we break free.
Developers and manufacturers are hard at work creating things that will encourage movement, mobility, and interaction in VR. Take the 3D Rudder for Playstation VR . Sure, you’re still confined to an armchair, but it’s a step in the right direction (away from teleporting and toward physical movement).
There’s the Oculus Quest , or what the company is calling the first “all-in-one gaming system built for virtual reality,” coming out this year. That means no PC connectivity for computing power, rendering graphics, or internet connectivity. Which also means no wires. Which also means more mobility!
You’ve even got fitness brands like NordicTrack (exercise bikes) jumping into the VR game to encourage more movement and the continued use of specialized hardware that does much more than just complement VR experiences.
And you can’t talk about tech in 2019 without mentioning 5G (apparently). To some, the infrastructure that supports 5G is — in a sense — XR hardware. Faster connectivity and lowered latencies means more freedom to move, richer experiences without hardwired connections, etc, etc. I won’t go too deep into the promises of and heavy hype over 5G here (as I’ve already done this extensively), but it’ll very likely lead to a spike in VR, MR, and AR usage.
Need-to-Know #4: Augmented reality ads are coming (Or: bringing the magic of QVC to your living room!)
The Military Is Using It. It turns out that the U.S. military is totally loving virtual reality. The Army, the Navy, and the Air Force have all used virtual reality in the past few years to train their soldiers. Keep in mind that this is not a game but a real training for some intense military action, including flying, medical training, fighting in the battlefield, and driving as well. The military is also reportedly using virtual reality in getting new recruits.
Note: if you’re interested in learning more about AR, check out my deep dive into the state of AR .
So there you are, wearing your PJs, watching a Netflix original, sipping tea, scrolling through Twitter. Suddenly you see an ad for a coffee table. Twitter knows you’ve just moved to a new apartment, so you’ve been served ads like this since you created a Zillow account. Normally, it’d be easy to ignore. After all, you’re not about to check the table’s specs, bust out a tape measure, and use your precious brain power to imagine how that table looks next to your new couch.
But times they are a changin’. Snapchat’s AR ads used to cost half a million dollars in 2017. Now, just about anyone can make an AR lens with Lens Studio (seriously, I’ve made several), and brands can advertise them for as little as $50 .
Facebook is testing AR ads in the news feed and in Messenger (e.g. Ray Ban advertises glasses to you; you allow access to your camera and voilà), and brands like the NBA, Sephora, and Wayfair are already buying in . And as AR ads become cheaper to produce and success metrics continue to pour out ( one that is particularly impressive: “what is stored, or encoded into memory is 70% higher for AR experiences”), consumers will start seeing coffee table ads accompanied by “check it out in AR,” which is certainly more compelling than, “learn more.”
Need-to-Know #5: Enhancements to the AR/MR/VR creator’s toolbelt. (Or: don’t watch porn while taking tests!)
In 2018, I created a breakdown of every major AR development platforms , from Facebook’s AR Studio and Amazon Sumerian to Apple’s ARKit and Google’s ARCore. To make things simple (and to prevent myself from going down a rabbit hole and rewriting the whole thing), know that these companies will continue pouring money into the advancement of these products. If they can encourage creators, developers, and brands to continue making AR content, they can find ways to monetize it (via app stores and advertising). And there’s the Unreal Engines and Unitys of the world. They will continue to lead the pack in creating high-powered, cross platform VR and AR experiences.
All of these companies will also continue to integrate other products into their AR development products. Take Amazon Sumerian, for example. On the surface, it’s an IDE for creating VR and AR experiences, designed mostly for training purposes. But with Amazon Lambda (their serverless computing platform), Amazon Lex (their chatbot platform), Amazon Polly (their voice AI), Amazon Rekognition (image and video analysis software) and other software platforms integrated, you can build a virtual assistant that is able to proctor an exam, and scold students when they attempt to watch porn . Finally?
On the hardware side, you’ve got companies like GoPro making the production of high-quality 360º video relatively easy and affordable .
One More Note
I try and cover the state of XR at least once a year. It’s ever-changing, and the buy-in from companies and people with deep pockets is real. That said, people — as in the public — don’t really care that much. At least, not yet.
I mentioned this at the top, but the killer apps that justify the investment are yet to be seen. In theory, technologies that transport you to new worlds or “augment” your current one feel like winners. And maybe it’s just a matter of time and tech before your dreams and fantasies are digitally overlayed before your eyes.
The VFX-1. We can’t do a list about the history of Virtual Reality and not include the VFX-1. Released in the middle of the 1990s, the VFX-1 system was one of the most capable virtual reality headsets released on the market at the time. With stereoscopic 3D, multi-axis head movement detection and rotation, and the ability to play games that were not truly supported by the system, the VFX-1 was the true Virtual Reality deal at the time. Furthermore, their price tag was relatively cheap compared to other products on the market, coming at a mere $600. However, the VFX-1 was too advanced of a technology and it didn’t really take off. Later on, the company Vuzix that made the glasses was bought by Forte Technologies, which released a more expensive VFX 3D version, but it also didn’t manage to achieve huge success.
Admittedly, I’m still skeptical. Maybe humans don’t actually want escapist virtual tech. Maybe the hype around augmented reality is just detracting from a more important conversation about digital wellbeing, accessible UI, and user privacy. Maybe the military regrets spending half a billion dollars on a headset that can only augment in a 52º field of view .
Skeptical or no, the companies that dominate every digital interaction you have — from the hardware and apps you use (Apple, Amazon, Microsoft, Samsung), to the ads you unwillingly see (Facebook, Snapchat, Google), to the infrastructure that connects us all using wireless (AT&T, Verizon, T-Mobile) — have an incentive to push XR to consumers, businesses, and the government.
So, let’s see how 2019 shakes out. I’m skeptical, yes, but I’m also excited to see what comes.
Serious thanks to ISL’s own tech director Scott Donaldson and XR guru Donell Ellis.
Image Credits :Freepik - Free Graphic resources for everyone
More than 3 millions free vectors, PSD, photos and free icons. Exclusive freebies and all graphic resources that you…www.freepik.com Creative Commons - Attribution 3.0 Unported - CC BY 3.0
Bahasa Indonesia Bahasa Malaysia Castellano (España) Català Dansk Deutsch English Español Esperanto Euskara français…creativecommons.org Dreamscape Immersive is bringing VR to the masses
The company's new installation in a Los Angeles shopping center lets audiences take a trip to an ancient temple or the…www.theverge.com Star Wars: Secrets of the Empire raises the bar for what a true VR experience can be
It's not every day you can say you've snuck into a top-secret Empire facility on the bubbling lava planet of Mustafar…www.alphr.com Microsoft Significantly Misrepresented HoloLens 2's Field of View at Reveal
To significant anticipation, Microsoft revealed HoloLens 2 earlier this week at MWC 2019. By all accounts it looks like…www.roadtovr.com Hands On: SMI Proves that Foveated Rendering is Here and it Really Works - Road to VR
SMI, a company working in the field of gaze detection for over 20 years, hit CES this year with an application of their…www.roadtovr.com The $399 Oculus Quest is a premium VR headset with no wires-and no PC required
Here we are, the fifth iteration of Oculus Connect-four years since the Facebook buyout, two and a half years after the…www.pcworld.com New PSVR Controller For Your Feet
3dRudder, a motion-control peripheral that's currently available for PC, is coming to PlayStation VR. Currently used…www.psu.com