Kulbir SINGH, Tushar JADHAV, Aditya ABHYANKAR

Volumetric 3D reconstruction of real objects using voxel mapping approach in a multiple-camera environment

: Extracting 3D information from 2D images is an inverse estimation problem and a challenging task initself. The aim of 2D to 3D reconstruction is to generate either a volume or a surface representing the object frommultiple views. This paper presents a simple and accurate multiple-view volumetric 3D reconstruction method usingan integrated approach based on homography estimation and voxel mapping. The homography-based approaches giveaccurate estimates but do not provide system dynamics. The voxel-based volumetric reconstruction methods providesystem dynamics that are essential for system modeling. However, they face challenges while modeling the concavities.This paper presents a proposed 3D reconstruction method that combines homography estimation and the voxel mappingapproach for improving the accuracy of 3D reconstruction. Experimental results show that the method efficientlyreconstructs objects of known and unknown shape, fragile objects, and complex scenes with multiple objects. The useof homography along with voxel mapping in a multiple-camera environment brings out more details of the object forimproving the quality of reconstruction.

PDF

___

[1] Gallo A, Muzzupappa M, Bruno F. 3D reconstruction of small sized objects from a sequence of multi-focused images. J Cult Herit 2014; 2: 173-182.
[2] Seo Y, Choi S, Kim H, Hong K. Where are the ball and players? Soccer game analysis with color-based tracking and image mosaic. In: Image Analysis and Processing Conference; 17 September 1997; Berlin, Germany. pp. 196-203.
[3] Iwase S, Saito H. Tracking soccer players based on homography among multiple views. In: Visual Communications and Image Processing; June 2003; Lugano, Switzerland. pp. 283-292.
[4] Labayen M, Olaizola IG, Aginako N, Florez J. Accurate ball trajectory tracking and 3D visualization for computerassisted sports broadcast. Multimed Tools Appl 2014; 73: 1819-1842.
[5] Chang J, Lee K, Lee S. Multiview normal field integration using level set methods. In: IEEE 2007 Computer Vision and Pattern Recognition Conference; 18–23 June 2007; Minneapolis, MN, USA. New York, NY, USA: IEEE. pp. 1-8.
[6] Seitz S, Curless B, Diebel J, Scharstein D, Szeliski R. A comparison and evaluation of multi-view stereo reconstruction algorithms. In: IEEE 2006 Computer Vision and Pattern Recognition; 17–22 June 2006. New York, NY, USA: IEEE. pp. 519-528.
[7] Fuhrmann S, Langguth F, Moehrle N, Waechter M, Goesele M. MVE – An image-based reconstruction environment. Comput Graph 2015; 53: 44-53.
[8] Jang I, Cho J, Lee K. 3D human modeling from a single depth image dealing with self-occlusion. Multimed Tools and Appl 2012; 58: 267-288.
[9] Shen S. Accurate multiple view 3D reconstruction using patch-based stereo for large-scale scenes. IEEE T Image Process 2013; 22: 1901-1914.
[10] Bottino A, Laurentini A. Introducing a new problem: shape-from-silhouette when the relative positions of the viewpoints is unknown. IEEE T Pattern Anal 2003; 25: 1484-1492.
[11] Srinivasan P, Liang P, Hackwood S. Computational geometric methods in volumetric intersection for 3D reconstruction. Pattern Recogn 1990; 23: 843-857.
[12] Frank S, Sturm J, Cremers D. Volumetric 3D mapping in real-time on a CPU. In: IEEE Robotics and Automation Conference; 31 May–7 June 2014; Hong Kong. New York, NY, USA: IEEE. pp. 2021-2028.
[13] Tabb A. Shape from silhouette probability maps: reconstruction of thin objects in presence of silhouette extraction and calibration error. In: IEEE 2013 Computer Vision and Pattern Recognition; 8–10 June 2013; Boston, MA, USA. New York, NY, USA: IEEE. pp. 161-168.
[14] Chen J, Bautembach D, Izadi S. Scalable real-time volumetric surface reconstruction. ACM T Graphic 2013; 32: 113: 1-16.
[15] Wu C, Liu Y, Dai Q, Wilburn B. Fusing multiview and photometric stereo for 3D reconstruction under uncalibrated illumination. IEEE T Vis Comput Gr 2011; 17: 1082-1095.
[16] Wu Z, Song S, Khosla A, Yu F, Zhang L, Tang X, Xiao J. 3D shapenets: A deep representation for volumetric shapes. In: IEEE 2015 Conference on Computer Vision and Pattern Recognition; 7–12 June 2015; Boston, MA, USA. New York, NY, USA: IEEE. pp. 1912-1920.
[17] Yan X, Yang J, Yumer E, Guo Y, Lee H. Perspective transformer nets: Learning single-view 3d object reconstruction without 3d supervision. In: Advances in Neural Information Processing Systems; 5–10 December 2016; Spain. pp. 1696-1704.
[18] Hane C, Zach C, Cohen A, Angst R, Pollefeys M. Joint 3D scene reconstruction and class segmentation. In: IEEE 2013 Computer Vision and Pattern Recognition; 25–27 June 2013; Portland, OR, USA. New York, NY, USA: IEEE. pp. 97-104.
[19] Zhao C, Xiao W. A parallelizing and improving on voxel colouring technology for 3D reconstruction based on images. In: Proceedings of the IEEE International Conference on Mechatronics and Automation; 29 July–1 August 2005; Canada. New York, NY, USA: IEEE. pp. 2185-2189.
[20] Feldmann T, Brand K, Worner A. Enhancing voxel carving by capture volume calculations. In: Proceedings of the IEEE International Conference on Image Processing; 26–29 September 2010; Hong Kong. New York, NY, USA: IEEE. pp. 4053-4056.
[21] Tsai R. A versatile camera calibration technique for high accuracy 3D machine vision metrology using off-the-shelf TV cameras and lens. IEEE J Robot Autom 1981; 24: 381-395.
[22] Bouguet JY. Camera Calibration Toolbox for MATLAB. Natick, MA, USA: MathWorks, 2006.
[23] Zhang Z. A flexible new technique for camera calibration. IEEE T Pattern Anal 2000; 22: 1330-1334.
[24] Turner E, Zakhor A. Watertight planar surface meshing of indoor point-clouds with voxel carving. In: IEEE 3DTV Conference; 7–8 October 2013; United Kingdom. New York, NY, USA: IEEE. pp. 41-48.
[25] Jadhav T, Singh K, Abhyankar A. A review and comparison of multi-view 3D reconstruction methods. J Eng Res-Kuwait 2017; 5: 50-72.
[26] Wu C. Towards linear-time incremental structure from motion. In: IEEE 2013 3D Vison Conference; 29 June–1 July 2013; Seattle, WA, USA. New York, NY, USA: IEEE. pp. 127-134.
[27] Fremont V, Chellali R. Turntable-based 3D object reconstruction. In: IEEE 2004 Cybernetics and Intelligent Systems Conference; 1–3 December 2004; Singapore. New York, NY, USA: IEEE. pp. 1277-1282.
[28] Vogiatzis G, Torr P, Cipolla R. Multi-view stereo via volumetric graph-cuts. In: IEEE 2005 Computer Vision and Pattern Recognition; 20–26 June 2005; San Diego, CA, USA. New York, NY, USA: IEEE. pp. 391-398.
[29] Kolmogorov V, Zabih R. Multi-camera scene reconstruction via graph cuts. In: European Conference on Computer Vision; 28–31 May 2002; Copenhagen, Denmark. pp. 82-96.
[30] Hernandez C, Schmitt F. Silhouette and stereo fusion for 3D object modeling. Comput Vis Image Und 2004; 96: 367-392.
[31] Goesele M, Curless B, Seitz SM. Multi-view stereo revisited. In: IEEE Computer Vision and Pattern Recognition; 17–22 June 2006. New York, NY, USA: IEEE. pp. 2402-2409.
[32] Furukawa Y, Ponce J. Accurate, dense, and robust multiview stereopsis. IEEE T Pattern Anal 2010; 32: 1362-1376.
[33] Vu HH, Labatut P, Pons JP, Keriven R. High accuracy and visibility-consistent dense multiview stereo. IEEE T Pattern Anal 2012; 34: 889-901.