A novel solution in the simultaneous deep optimization of RGB-D camera calibration parameters using metaheuristic algorithms

This article presents a novel method for estimating 19 parameters of RGB and depth camera calibrationsimultaneously. The proposed algorithm is based on applying metaheuristic methods for deep optimization and estimatingall parameters of intrinsic, extrinsic, and lens distortions of cameras. This paper compares four metaheuristic algorithms,i.e. a genetic algorithm, particle swarm optimization, the colonial competitive algorithm, and the shuffled frog leapingalgorithm, with a numerical algorithm called singular value decomposition. The proposed method does not need theinitial estimation for optimization and it can avoid being trapped in local minima. By using nondirect estimation, weachieve middle computing matrices such as the homography matrix, which is used in the pinhole camera model. Bothversions of Kinect sensors are used for the experimental evaluation. The mean square of the reprojection error criteria isdefined as the objective function in the proposed algorithm. The experimental results show that the proposed methodis more efficient and accurate than traditional numerical solutions.

PDF

___

[1] Zhang Z. A flexible new technique for camera calibration. IEEE T Pattern Anal 2000; 22: 1330-1334.
[2] Kanatani K. Statistical Optimization for Geometric Computation: Theory and Practice. 1st ed. Amsterdam, the Netherlands: Elsevier, 1996.
[3] Kanatani K, Takeda S. 3-D motion analysis of a planar surface by renormalization. IEICE T Inf Syst 1995; E78-D: 1074-1079.
[4] Kanatani K. Optimal homography computation with a reliability measure. In: IAPR 1998 Workshop on Machine Vision Applications; 17–19 November 1998; Chiba, Japan. pp. 426-429.
[5] Hartley R, Zisserman A. Multiple View Geometry in Computer Vision. 2nd ed. New York, NY, USA: Cambridge University Press, 2004.
[6] Harker MJ, O’Leary PL. Computation of homographies. In: British Machine Vision Conference; 5–8 September 2005; Oxford, UK. pp. 30.1-30.10.
[7] Herrera CD, Kannala J, Heikkila J. Joint depth and color camera calibration with distortion correction. IEEE T Pattern Anal 2012; 34: 2058-2064.
[8] Raposo C, Barreto JP, Nunes U. Fast and accurate calibration of a Kinect sensor. In: IEEE 2013 International Conference on 3D Vision; 29 June–1 July 2013; Seattle, WA, USA. New York, NY, USA: IEEE. pp. 342-349.
[9] Ji Q, Zhang Y. Camera calibration with genetic algorithms. IEEE T Syst Man Cy A 2001; 31: 120-130.
[10] Hati S, Sengupta S. Robust camera parameter estimation using genetic algorithm. Pattern Recogn Lett 2001; 22: 289-298.
[11] Song X, Yang B, Feng Z, Xu T, Zhu D, Jiang Y, Camera calibration based on particle swarm optimization. In: IEEE 2009 2nd International Congress on Image and Signal Processing; 17–19 October 2009; Tianjin, China. New York, NY, USA: IEEE. pp. 1-5.
[12] Merras M, Akkad NE, Saaidi A, Nazih AG, Satori K. Camera self calibration with varying parameters by an unknown three dimensional scene using the improved genetic algorithm. 3D Research 2015; 6: 1-14.
[13] Tsai RY. An efficient and accurate camera calibration technique for 3D machine vision. In: IEEE 1986 Computer Vision and Pattern Recognition Conference; 22–26 June 1986; Miami, FL, USA. New York, NY, USA: IEEE. pp. 364-374.
[14] Yılmaz O, Karaku¸s F. Stereo and Kinect Fusion for continuous 3D reconstruction and visual odometry. Turk J Elec ¨ Eng & Comp Sci 2016 24: 2756-2770.
[15] Han J, Shao L, Xu D, Shotton J. Enhanced computer vision with Microsoft kinect sensor: a review. IEEE T Cybern 2013; 43: 1318-1334.
[16] Zhang Z. Camera calibration. In: Medioni G, Kang SB, editors. Emerging Topics in Computer Vision. Upper Saddle River, NJ, USA: Prentice Hall, 2005. pp. 5-44.
[17] Drap P, Lefevre J. An exact formula for calculating inverse radial lens distortions. Sensors 2016; 16: 1-18.
[18] Holland JH. Adaptation in Natural and Artificial Systems. 1st ed. Ann Arbor, MI, USA: University of Michigan Press, 1975.
[19] Kennedy J, Eberhart R. Particle swarm optimization. In: IEEE 1995 Neural Networks International Conference; 27 November–1 December 1995; Perth, Australia. New York, NY, USA: IEEE. pp. 1942-1948.
[20] Atashpaz-Gargari E, Lucas C. Imperialist competitive algorithm: An algorithm for optimization inspired by imperialistic competition. In: IEEE 2007 Evolutionary Computation Congress; 25–28 September 2007; Singapore. New York, NY, USA: IEEE. pp. 4661-4667.
[21] Eusuff M, Lansey K, Pasha F. Shuffled frog-leaping algorithm: a memetic meta-heuristic for discrete optimization. Eng Optimiz 2006; 38: 129-154.
[22] Karan B. Accuracy improvements of consumer-grade 3D sensors for robotic applications. In: IEEE 2013 Intelligent Systems and Informatics Symposium; 26–28 September 2013; Subotica, Serbia. New York, NY, USA: IEEE. pp. 141-146.
[23] Pagliari D, Pinto L. Calibration of Kinect for Xbox One and comparison between the two generations of Microsoft sensors. Sensors 2015; 15: 27569-27589.
[24] Butkiewicz T. Low-cost coastal mapping using Kinect v2 Time-of-flight cameras. In: IEEE 2014 Oceans Conference; 14–19 September 2014; St. John’s, Canada. New York, NY, USA: IEEE. pp. 1-9.