How will astronauts use augmented reality glasses?

Microsoft HoloLens glasses may be a pipe dream to all but the earliest adopters with a hefty price tag of $3,000, but that’s not the case for astronauts aboard the International Space Station. As part of Microsoft’s Sidekick partnership with NASA, two pairs of HoloLens have been shipped to the ISS.

HoloLens allows for augmented reality, meaning that what you see in your field of vision is visually edited with extra information. For astronauts, this could mean less training time for certain operations or easier access to help in crisis situations. “HoloLens and other virtual and mixed reality devices are cutting edge technologies that could help drive future exploration and provide new capabilities to the men and women conducting critical science on the International Space Station,” says Sam Scimemi, director for International Space Station at NASA headquarters, in a NASA news release. Looking ahead, this technology could be critical to manned missions to Mars.

Project Sidekick has two modes: Remote Expert Mode and Procedure Mode. Remote Expert Mode uses Skype to let ground crew see what the astronauts see. Then they can instantly annotate the visual that the astronaut is seeing with notes, instructions and illustrations. Procedure Mode allows astronauts to learn about certain devices. It uses holographic images to offer information about whichever device the astronaut is looking at. For longer missions when communications become delayed, having real-time access to information could be vital.

Prior to going into space, Project Sidekick was also tested underwater by astronauts at the 62-foot-deep Aquarius underwater research station off the coast of Florida during a NEEMO (NASA Extreme Environment Mission Operations) mission.

After testing Project Sidekick on the ISS, Scott Kelly and Tim Peake had a little fun with it, playing a game of “Space Invaders.” NASA edited the video below for effect.

Project Sidekick is not Microsoft’s only virtual reality partnership with NASA. Project OnSight is another initiative, one that will allow scientists to study Mars as though they are on the planet. The OnSight software will work with JPL’s rovers. It will interpret data gathered by the rovers and create a 3-D map, allowing researchers to explore the red planet from the comfort of their terrestrial offices. JPL’s OnSight project manager Jeff Norris says, “Previously, our Mars explorers have been stuck on one side of a computer screen. This tool gives them the ability to explore the rover’s surroundings much as an Earth geologist would do field work here on our planet.”

Scientists who have tested OnSight are impressed with the functionality. “It’s like teleportation,” says Fred Kalif, a geologist at JPL. “When you use it long enough, it’s like you’re not in the room anymore. You are on Mars.”

OnSight also increases the connection between scientists and the robots they’ve sent ahead to explore Mars. Norris, who works on both Project Sidekick and Project OnSight explains, “By building tools that make us feel more connected to those robots and the environments that they’re exploring, we can change our experience of exploration in a very fundamental and exciting way.”

These holographic computing tools will make space travel safer for astronauts and will also make data accessible for explorers stuck on Earth.

You don’t have to be a scientist to appreciate Project OnSight. Destination: Mars is Microsoft’s “mixed reality tour,” and it allows ordinary people to experience Mars with HoloLens. The virtual tour even includes an appearance by a holographic Buzz Aldrin.

This summer, a permanent Destination: Mars exhibition will open at the Kennedy Space Center in Florida. So, if you’re not chosen for a mission to Mars, investigating the red planet in virtual reality might be the next best thing.