The Meta Orion glasses (understandably) generated a lot of buzz at the 2024 Connect event. They promise unique EMG capabilities, phenomenal visuals, and state-of-the-art tracking technology. However, what these glasses will be able to do will depend on another, less-talked-about Meta initiative: Project Aria.

Initially announced in September 2020, Project Aria hasn’t received nearly as much attention as other Meta projects, like the latest Quest headsets and Meta AI solutions. However, it’s been quietly shaping Meta’s XR future for years.

Already, research conducted with Project Aria has helped Meta gather the data required to develop scene understanding software, new 3D modelling capabilities, and more. Now that Meta allows all kinds of researchers to request an Aria development kit, the company could begin innovating faster than ever.

We could see a version of the Orion glasses, packed with features we didn’t encounter at Connect, much sooner than expected. Here’s everything you need to know about Project Aria, and what it means for Meta’s XR roadmap.

What is Project Aria? The Basics

Project Aria, introduced by Meta in 2020, is a research project created to help Meta develop its first set of “wearable augmented reality devices.” On a broad scale, it was the first evidence that Meta wasn’t just interested in creating virtual reality headsets or software like Horizon Worlds.

According to Meta, Project Aria, which gives developers access to a pair of sensor-packed glasses (without AR displays), helps innovators capture real-world data from a first-person perspective. This ensures Meta can gather data to help it build AR devices that fully understand location, context, and intent.

Initially, Meta limited access to the Aria project to a handful of industry partners and researchers. For instance, in 2021, the tech giant partnered with BMW, giving the automotive firm access to their technology to help bring virtual content into cars.

Since then, Meta has been exploring a range of potential use cases for AR technology. Options range from live maps and wayfinding capabilities to solutions for training and education. Already, Project Aria research has helped drive the creation of:

• SceneScript: A new technology for representing and inferring scene geometry using auto-regressive AI structured language models.
• EgoBlur: An enhanced AI model created by Meta to help preserve privacy in XR. It automatically detecting and blurring PII from images.
• EFM3D: A set of novel foundation models rooted in 3D space understanding and a benchmark for future development.

Project Aria Accessibility: Bringing Aria to Everyone

Meta has long allowed a handful of people to get involved with Project Aria research. Various researchers and academic groups have contributed to open research “challenges,” leading to new datasets. Previous challenges include Aria Everyday Activities (built for computer vision researchers) and Aria Synthetic environments. Meta has even invited researchers to help them use Aria sensor technologies to create digital twins.

In 2024, however, Meta introduced the opportunity for all research community members to apply for an “Aria Research Kit” (ARK) for the first time. Participating parties still need to undergo an assessment from Meta to ensure they’re using the technology safely (and for relevant use cases).

However, anyone can apply to take part in the project, and gain access to:

• Meta Aria glasses: The Meta Aria glasses are a pair of sensor-packed specs with an integrated battery, privacy switch, cameras, and seven microphones. They can capture all types of data in real time, from a user’s eye movements and location to video and audio.
• Client SDK: The client SDK allows developers to interface directly with the Aria glasses, creating their own applications and features for the specs and gathering data.
• Cloud services: Machine perception services give approved research partners access to various cloud-based services, such as 6DoF tracking and 3D eye-gaze estimation. This ensures users can harness algorithms and tools used internally by Meta’s Reality Labs researchers and accelerate the development process.

What is Meta Doing with the Project Aria Research Kit?

From day one, Meta has been using Project Aria to rapidly gather data that will power its progress in the worlds of augmented reality and artificial intelligence. However, the project was initially intended for internal use. Now, anyone can get involved (if they’re approved by Meta) and contribute to the growth of the extended reality space and Meta’s future devices.

To inspire interested research centers and partners to take advantage, Meta published information about various research projects that could help to determine future features of Meta’s Orion glasses.

For instance, based on the latest research partnerships, Orion glasses may allow users to:

1. Learn Directly from Experts

Imagine a world where you could learn how to mimic the skills of professionals in augmented reality. You’d gain instant insights into how the pros move and interact with the real world to guide your education. The University of Bristol focuses on that with its Project Aria research.

The educational group, involved in the “Ego-Exo4D” project, are capturing “egocentric data” from highly skilled professionals, such as mechanics, athletes, musicians, and chefs. They’re capturing their movements, and exploring how they interact with real-world objects to develop more intuitive training experiences. With these insights, Meta could create highly immersive AR apps that walk users through guided training sessions.

For instance, you could see in real-time exactly how a chef measures the right amount of ingredients, adds them to a recipe, prepares certain foods, and so much more.

2. Drive More Safely

Meta has been interested in exploring the benefits of augmented reality content for drivers for a while now. As mentioned above, BMW was one of the first companies Meta chose to receive access to Project Aria for future product development. Now, researchers at IIIT Hyderabad are building on that research with a new solution.

Using Project Aria, these researchers are developing a new “Driver Intent Prediction project.” Essentially, they’re leveraging the outward-facing and inward-facing eye-tracking cameras on the Aria glasses, alongside other car sensors, to track where drivers are looking.

Thanks to the eye gaze data they capture, researchers can identify when drivers might need extra alerts to stay safe on the road. For instance, imagine a pedestrian steps into the street blindly when you’re checking your mirrors. The glasses can detect that you haven’t seen them yet and issue a warning. Overall, the project’s goal is to minimize traffic accidents and keep people alert on the road.

3. Enhance Accessibility

Two additional Project Aria initiatives by Carnegie Mellon University and the University of Iowa focus on accessibility. In Iowa, researchers use the Aria Research Kit to help people with hearing loss. They’re using the microphone array in the specs to determine where sound is coming from.

These insights could be paired with prompts on the displays or screens in augmented reality specs. Or, they could be used alongside hearing aids to improve experiences for people with hearing issues. At Carnegie Mellon University, researchers are using Project Aria to help people with visual issues.

Using external beacons, and the sensors built into the glasses, these researchers are working on a navigation app. This app would help those with visual impairments explore unfamiliar environments safely, with intelligent real-time guidance.

Building Meta’s AR Future with Project Aria

Meta’s ongoing investment in Project Aria, and the company’s decision to expand access to its technology to a broader range of researchers is good news for the XR community. As more people access Aria research kits, Meta will gain access to more data. That will help the company truly transform augmented reality glasses and software.

It’s difficult to know whether this research will accelerate the release of the Meta Orion glasses to the general public. After all, Meta still has some extra work to do if it wants to ensure these specs aren’t prohibitively expensive. Thanks to their ultra-advanced components, the Orion glasses currently cost around $10,000 to make (per unit).

However, continued research with Project Aria should shape Meta’s augmented reality glasses. Whether we end up with a unique version of the Orion specs or something completely different, Aria will pave the way.

Meta’s ongoing research could even mean that Meta develops the most advanced, user-friendly, and powerful specs we’ve seen on the market so far.