Seamless Video Stitching with Hand-held Camera Inputs

Kaimo Lin1, Shuaicheng Liu2, Loong-Fah Cheong1, Bing Zeng2

1. National University of Singapore   2. University of Electronic Science and Technology of China

Abstract

Images/videos captured by portable devices (e.g., cellphones, DV cameras) often have limited fields of view. Image stitching, also referred to as mosaics or panorama, can produce a wide angle image by compositing several photographs together. Although various methods have been developed for image stitching in recent years, few works address the video stitching problem. In this paper, we present the first system to stitch videos captured by hand-held cameras. We first recover the 3D camera paths and a sparse set of 3D scene points using CoSLAM system, and densely reconstruct the 3D scene in the overlapping regions. Then, we generate a smooth virtual camera path, which stays in the middle of the original paths. Finally, the stitched video is synthesized along the virtual path as if it was taken from this new trajectory. The warping required for the stitching is obtained by optimizing over both temporal stability and alignment quality, while leveraging on 3D information at our disposal. The experiments show that our method can produce high quality stitching results for various challenging scenarios.

Materials

Examples

BibTex

@article{lin2016seamless,
        author = {Lin, Kaimo and Liu, Shuaicheng and Cheong, Loong-Fah and Zeng, Bing},
        title = {Seamless Video Stitching with Hand-held Camera Inputs},
        journal = {Computer Graphics Forum},
        volume = {35},
        number = {2},
        year = {2016},
        month = May,
        pages = {479--487},
        organization = {Wiley Online Library}
}