• The Dance of Intelligence: Speakers agreed that training AI (artificial intelligence) to have knowledge of the physical world could push robotic intelligence from computational to contextual. “We think of technology as transactional, but the way we interact with one another socially is more like a dance,” said Randi Williams, a graduate student at the MIT Media Lab, who is researching therapeutic robots for children.

According to Jeff Hawkins, chief executive of Silicon Valley AI research institute Numenta, in order to design more intelligent technologies, computer engineers should take this notion of a dance literally and borrow inspiration from how the human brain learns. “The brain builds a model of everything we know. It’s a sensory motor system. You have to move to learn, which most AI today doesn’t do. AI needs to have an internal model of the world, not just know how to play Go.”

Williams cited therapy bots – a topic we discuss on our podcast Future Thinking – as one early example of socially savvy AI. “In order for [therapy robots] to work, you need a robot to engage you in that conversation… If it nods and responds, you’re going to give it a long answer.”

• Enhanced Consumer Capabilities: “We’re in a new era of computing at this point,” said Timoni West, vice president of XR tools at San Francisco software company Unity. “Our devices have more context. Your car helps you to park. Wearables have a semantic understanding of what they’re looking at.” Despite this progress, speakers noted that physical barriers prevent current tech – especially VR – from seamlessly interfacing between real and rendered worlds.

When it comes to VR engagement, touch is the roadblock to overcome. Aubrey Simonson, a graduate student at MIT Media Lab, shared his concept for gloves that enable natural hand movement in digital environments (consumer VR headsets currently rely on gaming-style controllers). Simonson argued that simplifying physical engagement with virtual worlds would improve VR’s practical applications by reflecting the physical-mental connection that drives human learning, as detailed above.

Offering an early example of real-world VR-led embodied learning, Amitai Zav, director of the extended reality hub at Israel’s Sheba Medical Centre, said that the hospital is currently experimenting with VR as a tool to help patients requiring physical and cognitive therapy with post-surgery rehab.

• Augmenting Professional Knowledge: Speakers agreed that technologies like VR and AI will become essential assistants to employees in high-stress, high-risk fields. Zav explained how Sheba Medical Centre is trialling AR and VR tools to help surgeons prepare, and in some cases perform, high-risk surgeries. In the case of AR, the hospital is exploring the possibility of surgeons performing operations while wearing a Hololens (a mixed reality headset). The Hololens could display a patient’s CT scan (or other relevant information), acting as an omnipresent guide to ensure safe, accurate procedures.

Responding to an audience question asking how safe practice environments translate to life-or-death situations, Gordon Cooke, director of research at American military university West Point, argued that failure-friendly practice spaces will become essential for military training, precisely because they replicate traditional learning situations. “A high-stakes environment isn’t good for learning. You’re so stressed you can’t remember.”

The quality of the VR simulation used for training also informs confidence and capabilities. The VR headset from Finnish company Varjo, which has human-eye resolution, was recently approved by the EU’s aviation safety authority as a tool for pilots to gain flight hours, making it the first VR simulation to sub in for in-person experience. Stay tuned for our report Virtual Work Worlds, publishing November 5, for more on this topic.

For more on AI innovation, see The Future of Intelligence: CogX 2021. For more on consumer VR, see 10 Tech Trends for 2021.