Facebook AR Could Help You Hear Sounds Like A Superhero

The Facebook Reality Labs Research team wants to help people hear better via AR. And not just in the usual ways a hearing aid might help either. Or at least not based on the team’s recently-reported description of the latest-developed innovations. In fact, the technology may land in the company’s first AR glasses and could give wearers near-genuine hearing superpowers.

Dubbed “audio presence” and “perceptual superpowers,” the goal is to extend human hearing beyond amplifying overall audio. Instead, the team says it has focused on bringing specific sounds to the forefront. And pushing all other environmental noise to the background. All while ensuring that the audio experience itself remains natural, rather than sounding like it comes from a device attached at the ear.

What exactly do ‘audio presence’ and ‘perceptual superpowers’ do?

The apparent breakthrough stems from the hardware utilized on the Facebook-branded glasses in question, as detailed by the team. In effect, the system seems to work similarly to beamforming or to how some AI speakers do. Namely, those with far-field mics that listen for audio cues at a distance. By utilizing multiple microphones, the system can work out where sounds are coming from. That’s the “perceptual superpowers” part of the technology.

The glasses can then use other technology to determine where a user’ss head is pointing and where their gaze is being directed. It utilizes that information to determine which audio a user is trying to hear. And which sounds the user would rather hear less of. The audio that needs to be heard is amplified while background noise is dampened.

Simultaneously, the audio presence technology utilizes facts about how sound travels. That’s so that the Facebook-built AR device can present the amplified audio with directional clarity. Or, put more succinctly, so that it can present audio that differs ever-so-slightly based on ear shape and the direction the audio should arrive from.

That would, in effect, allow the audio to come through naturally. As though there was no intermediary device providing the audio boost. Audio Presence could also find its way into more traditional AR and VR experiences, allowing more natural sounds in games and other experiences.

Is this ever going to make a real-world consumer market appearance?

For the time being, Facebook’s proposed advances in AR audio are still under testing. There are no details regarding when or even if this technology will ever be made consumer-available. Although it seems unlikely Facebook would be touting the technology if it weren’t close to finalizing designs for future use.