We here produce stereo panorama image pairs (for left and right eyes) from a hand-held GoPro video. User can perceive 3D stereo by looking at the corresponding image for each eye.
Below is an introduction video of the project. We illustrate the pipeline, the resulting panorama images for left- and right-eye, and a re-cyan stereoscopy of them. For easily visualizing the scenes from different viewpoints, we generate a multi-view panorama animation.
The input is a hand-held video with circular motion. We sample out the frames using FFmpeg.
We use the video data provided by Megastereo (Megastereo: Constructing high-resolution stereo panoramas by Richardt, Christian, Yael Pritch, Henning Zimmer, and Alexander Sorkine-Hornung in CVPR 2013)
The camera lens distortion are removed by OpenCV with the GoPro parameters. The left image above is an input frame and the right image above is its result after undistortion. Bundle adjustment is then performed with the source codes provided by Jérémie Dumas
The undistorted frames are warped to a cylinder space and above left is the resulting world map. We crop out the frame from the world map, as showed at the right-side.
As described in Omnistereo (Omnistereo: Panoramic stereo imaging by Peleg, Shmuel, Moshe Ben-Ezra, and Yael Pritch in PAMI 2001), the strips for left- and right-eyes are extracted as the areas shifted an angle left and right from the center of a frame.
As the flow-based image blending method described in Megastereo, optical flows are first calculated from adjacent frames. We substract the motion between two frames and obtain the net flow:
F*_k→l(x) = F_k→l (A_0→k x) − (A_0→l x − A_0→k x)
The above images are (from left to right) two adjacent warped frames, the flow between them, and the net flow.
We blend the strip areas from two frames as:
S_k(x) = (1−η) · I_k(A_0→k x + η · F_k→l*(x)) + η · I_l(A_0→kx + (1−η) · F_l→k*(x))
The left-eye panorama (top) and the right-eye panorama (down).
The disparity map between them.
The red-cyan stereoscopy from the panorama pair.
The multi-viewpoint panorama animation.
