The technical challenges in streaming a 360 video is huge.
Facebook is preparing for a world in which video is no longer flat.
The social network’s push into virtual reality with its Oculus Rift along with similar initiatives by Microsoft and Google is expected to create a flood of video for the devices. Meanwhile, companies like GoPro FB 4.01% are developing cameras that people can use to film themselves in 360 degrees while skydiving and snowboarding.
Sourced through Scoop.it from: fortune.com
To deal with the likely influx of huge video files, Facebook has built its own streaming technology that it says will work quickly and without straining its data centers. The social network discussed some of that innovation at its video technology conference on Thursday, joined by prominent streaming services like Netflix NFLX -1.59% and YouTube.
Facebook executives spent a significant amount of time discussing 360 video, which lets viewers put themselves into the scene and gives them a view in any direction they choose. This 3-D view is considered to be an improvement over the two-dimensional video widely used today.
Three-D videos are closely related to virtual reality, albeit with slight differences, explained Jay Parikh in an interview with Fortune. Virtual reality is more immersive than 360 video in that people can interact with the content itself, he said. For example, in a virtual reality game like the upcoming Minecraft VR game, users can pick up objects and build towers in a virtual environment that seems to surround them.
Get Data Sheet, Fortune’s technology newsletter.
In anticipation of its users uploading 360 video to its servers, Facebook FB 4.01% has developed video processing and encoding techniques to help efficiently deliver the images. Because 360 video files are much bigger than for regular video, Facebook had to figure out a way to shrink them to reduce the load on its servers, said Facebook software engineer David Pio.
Part of the reason the files are so big is because of the way the 360 videos are formatted to accommodate the screens of computers and mobile devices. Think of 360 video as being a traditional rectangular image that is stretched and wrapped around an imaginary sphere, so that people can view it at different angles, Pio explained.
However, this creates some problems with how the video looks because some parts appear distorted as the image stretches to accommodate the sphere. Pio likened this to how land near the North and South Poles appears stretched on a globe and doesn’t accurately reflect its true size.
Additionally, stretching makes file sizes larger than they need to be. An excess of pixels end up having to be streamed.
To shave them off, Facebook instead decided to wrap the video around an imaginary square rather than a sphere. Essentially, each side of the square displays a portion of the 360 video and represents a different viewing perspective. Facebook’s video system then stitches the chunks back together and streams it without the distortion and excess imagery.
Facebook has worked on similar techniques to reduce the size of 360 videos when displayed through virtual reality devices like the Oculus. In this case, the 360 video files must be even bigger because the devices require files to be displayed at higher resolutions than traditional flat screens. This bulk results in file sizes that can lead to buffering, or long loading times, for people with slow Internet connections.
To solve this problem, Facebook developed a way to stream the 360 videos to VR devices that involves cutting the video into chunks. Only images in the viewer’s line of sight are streamed.
While gawking at a 360 video of the Eiffel Tower, for example, you’d want the monument to appear clearly while you may not care about how the buildings to the left look because they only appear in your peripheral vision. However, when you move your head, those buildings would appear sharper.
Facebook essentially cuts the video into 30 sections that represent different viewing perspectives of the scene. Additionally, each section has five different resolutions that Facebook’s servers can stream to viewers depending on their Internet connection.
In effect, Facebook’s system automatically streams the right viewing perspective of the video at its highest quality each time a person turns his or her head with the rest of the video streaming in chunks at lower qualities.
See on Scoop.it – Augmented World