Global Mosaic and Motion Segmentation

M. Aprile, A. Colombari, A. Fusiello, R. Marzotto, and V. Murino Department of Computer Science
University of Verona, Verona - Italy

Overview

This work present an application which takes a video shot as input and produces a compact representation composed by a background layer and segmented moving objects. We deal, in particular, with the problems of global registration, superesolution mosaicing, objects segmentation and tracking. Global registration is achieved with a graph-based technique that exploits situations when the frames have been captured using a 2D scanning of the scene rather then 1D scan as is commonly done. Objects segmentation is based on motion analysis using a robust statistical model of the background. Tracking is based on graph representation of motion and blob matching. Specific techniques are used to cope with crossing and occlusions. We also propose a new measure for mosaic quality estimation.

Videocoding results

Original sequence		Mosaic of background	Foreground sequence	encoded/decoded sequence	Video editing examples
Walking down S.Zeno						flip background
Walking down S.Zeno						on the moon
Children in P.za Dante						change color
Children in P.za Dante						remove object
Bicycle in P.za Dante						new scenarious

Mosaicing results

Original sequence		Mosaic before and after global alignment	Super resolution
Panoramic mosaic of Basilica di S. Zeno in Verona
Panoramic mosaic of Verona from Castel S. Pietro
Another panoramic mosaic of Verona from Castel S. Pietro
Planar mosaic of a paper
Panoramic mosaic of Torre dei Lamberti in Verona

[Older results]

Reference papers

M. Aprile, A. Colombari, A. Fusiello, and V. Murino. Segmentation and tracking of multiple objects in video sequences. In 5th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS 2004), Lisboa, Portugal, 21-23 April 2004. (PDF)
A. Fusiello, M. Aprile, R. Marzotto, and V. Murino. Mosaic of a video shot with multiple moving objects. In IEEE International Conference on Image Processing, volume II, pages 307-310, Barcelona, Spain, September 2003. IEEE. (PDF)