Introducing Parallax
Since cave paintings humans captured and preserved moments they found fascinating. Photographic film made such capture much easier, and more recently digital sensors and smartphone cameras democratized it even further. What about realism? Modern digital photographs look highly realistic, but they are flat (still a good match for the flat screens that we use to look at them).
​
These days, however, various types of 3D displays are steadily gaining popularity. Tens of millions of VR headsets have been sold, while AR glasses and 3D lenticular monitors are also approaching mainstream adoption. These displays can show the world as it truly is — three-dimensional.
Today, we present Parallax — a VR app containing volumetric photographs that capture the world in 3D. The app supports Meta Quest 2/3/3s and contains almost two thousand photographs, featuring bustling Indian cities, Iceland ice caves and waterfalls, paradise Zanzibar beaches, cobbled Edinburgh streets, and many more. The preview video above is composed from several Parallax photos, but watching the same scenes in VR adds another dimension in every sense of this word!​ So give it a try!
​Technical details
Parallax photos have several distinctive properties. First, they are three-dimensional (aka "six-degrees-of-freedom") providing correct parallax in response to head motion in VR. This is in contrast to more traditional stereo-photographs (including "spatial photos") that contain two fixed viewpoints for the two eyes, and therefore do not handle parallax correctly. Stereo-photographs often lead to a nagging discomfort known as cybersickness or VR nausea, especially when they are not confined to a narrow window. This almost never happens with Parallax photos, even when they are viewed immersively at natural scale.
Secondly, Parallax photos are taken with a single shot capturing a particular moment in time. This is in contrast to photogrammetric reconstruction that provides stunning (and more complete) scene models by processing images taken subsequently from multiple viewpoints. Photogrammetry typically fails to capture moving subjects and objects well. Being based on single shots, Parallax photos successfully capture things that are non-stationary but truly worth capturing: people, animals, vegetation, waves and waterfalls, etc.
The photos in Parallax are acquired with two types of devices. Some of the photos are taken with smartphones. Ultrawide cameras of flagship smartphones offer a large field of view, leading to a rather immersive VR experience. They have also made remarkable progress in terms of image quality, and Parallax fully capitalizes on that. In some phone models, simultaneously capturing a pair of views from the ultrawide and the main camera is possible, and we utilize this as well. Alongside smartphone captures, Parallax features a lot of 3D photos acquired with a more professional kit: Canon R5 camera with RF5.2 stereo-lens. This kit is a popular combination for stereo-photography among VR enthusiasts.
At Cinemersive Labs, we have developed AI pipelines that upgrade smartphone photos or stereo-photographs to immersive "six-degrees-of-freedom" 3D. This AI capability is based on three years of research and benefits from large training datasets that we have collected around the world.
Looking into the future
Going forward, we plan to grow the Parallax collection significantly. Some stunning locations and events are already in preparation! If you are seriously into travel photography, want to contribute an album about a great location or an interesting event, and have a recent model of an iPhone (model 12 or newer) or the Canon stereo kit, do get in touch with us at parallax@cinemersivelabs.com . You can also join the Discord server for the Parallax app.
We are also working on bringing 3D photography to every smartphone owner. As the interactive gallery below shows, portraits and family events can also be captured with our technology, so we anticipate lots of users willing to capture and preserve their valuable personal moments in 3D. Stay tuned!​
​(The animations above are interactive, you can stop them by clicking/tapping and then interact by dragging and pinching/wheeling. You can see more interactive demos in our web gallery.)
Finally, we also work on the 6DoF video reconstruction, and our early research results are available in the Cinemersive Videos app for Meta Quest.
​