Visual Information Processing in Multi-Camera Systems

We consider a dense multi-camera system and assume that a large number of cameras is monitoring a certain scene from multiple viewpoints. The aim then is to fuse all this data acquired to perform segmentation, scene interpretation and classification, preferably in an automatic fashion.Traditional algorithms do not scale with the number of cameras and become impracticable when the number of images acquired is large.  The  data acquired by multiple cameras from multiple viewpoints can be parameterized with a single function called the plenoptic function.  The  aim therefore is to perform segmentation and  scene interpretation  directly  in the plenoptic domain.  Segmentation is achieved using the level set method  and, by segmenting the multi-view images jointly, we can deal with occlusions efficiently.  The extracted hypervolumes can be used for classification, interpolation or for augmented reality as shown below.

(a)	(b)	(c)	(d)

• J. Berent and P.L. Dragotti, Plenoptic Manifolds: Exploiting Structure and Coherence in Multiview Images, IEEE Signal Processing Magazine, vol. 24 (6), pp.34-44, November 2007.
• J. Berent and P.L. Dragotti, Unsupervised Extraction of Coherent Regions for Image Based Rendering, British Machine Vision Conference (BMVC), Warwick, UK, September 2007.
• J. Berent, P. L. Dragotti, "Segmentation of Epipolar-Plane Image Volumes with Occlusion and Disocclusion Competition," in Proceedings of IEEE International Workshop on Multimedia Signal Processing (MMSP'06), Victoria, Canada, October 3-6, 2006.