Newsletter
Smart home tech and virtual reality share a lot in common. Each type of technology requires several sensors, input methods, devices, and displays to adequately complete the experience. This creates some obvious pain points for consumers, who often have to invest significant amounts of time and money in order to get the full experience.
Researchers at the Facebook Reality Labs have been working overtime to come up with new ways of interacting with virtual worlds, and while these experiments could further complicate VR if they were all released as individual products, Facebook’s product team hasn’t historically done that. Rather, all of these projects should end up playing a role in developing the next-generation of controller input for VR and, ultimately, a single product that will give consumers a better experience in the end.
Take Tasbi, a new wrist-worn strap that provides haptic feedback when players interact with virtual content. This strap is designed to provide a combination of vibration and squeezing to trick your brain into thinking you’re actually touching or interacting with virtual objects. Touching a virtual button, for instance, will elicit a vibration response that makes your hand feel like it’s pressing a button down in physical space. Shaking someone’s hand in a social game like VRChat would create slight pressure on the wrist and a subsequent vibration that makes it feel like you’re physically shaking someone’s hand.
Surprisingly, these sorts of movements done on the wrist and via the carpals can elicit a near-identical response in the brain to what our fingers feel, limiting the need for haptics in the individual fingers themselves. This sort of add-on can be coupled with something like a touch glove, which would enable natural movements and interaction in the virtual world as well.
Facebook researchers have been coming up with more natural ways to interact with the virtual world for years now, including a touch glove that acts fits over your hand and responds to virtual objects in an identical manner to your real hand. A technical blog from late last week discusses findings from the research group, including how latency plays an even bigger role in our experience in the virtual world than was previously realized.
In this test, researchers placed a VR headset (HMD) with headphones onto participants and outfitted them with touch-sensitive gloves. Tests were performed on physical and virtual objects, and latency of the visual, audible, and touch responses were all measured and recorded, and the results were extremely interesting.
During the regular tests, audio latency was slower than the physical touch latency of the glove. Despite the fact that touch latency was low enough to be nearly indistinguishable from real-time, the added audio latency made it feel like the touch response was much slower.
Researchers found that the brain automatically fused these sensory inputs and created a single event in the minds of participants; something that’s fully automated in the human brain and cannot be simply ‘turned off’. Disabling the audio entirely created an experience that felt instantaneous in all participants, giving researchers new insight into how the human brain understand sensory input.
Researches then went on to create several different scenarios where they introduced lag in one of three areas: visual, audio, or touch (haptics). They then measured the difference in reaction from people and how closely they felt the experience matched up to reality. This helped researchers create a graph that showed how sensitive the average person was to latency introduced in each type of sensory input, and how the brain can account for differences in the fusion of the three.
The idea is to come up with guidelines to help developers create haptic experiences that feel as real as they look. We’re constantly seeing new gadgets and ideas for how to make VR more realistic as an overall experience. Wireless headsets, freedom of movement, touch input, haptic feedback, and quality sound all play an equally huge part in what makes the experience a good one in the end, and all work together to fool the brain into thinking that the virtual could actually be real.