Alamgir HOSSAIN, Eui-Nam HUH, Tien-Dung NGUYEN

A novel accuracy assessment model for video stabilization approaches based on background motion

In this paper, we propose a new accuracy measurement model for the video stabilization method based onbackground motion that can accurately measure the performance of the video stabilization algorithm. Undesired residualmotion present in the video can quantitatively be measured by the pixel by pixel background motion displacementbetween two consecutive background frames. First of all, foregrounds are removed from a stabilized video, and thenwe find the two-dimensional flow vectors for each pixel separately between two consecutive background frames. Afterthat, we calculate a Euclidean distance between these two flow vectors for each pixel one by one, which is regarded as adisplacement of each pixel. Then a total Euclidean distance of each frame is averaged to get a mean displacement foreach pixel, which is called mean displacement error, and finally we calculate the average mean displacement error. Ourexperimental results show the effectiveness of our proposed method.

PDF

___

[1] Zhang C, Chockalingam P, Kumar A, Burt P, Lakshmikumar A. Qualitative assessment of video stabilization and mosaicking systems. IEEE Work App Comp 2008; 1-6.
[2] Tanakian MJ, Rezaei M, Mohanna F. Camera motion modeling for video stabilization performance assessment. 7th Iran Conf Mach 2011; 1-4.
[3] Karpenko A, Jacobs D, Baek J, Levoy M. Digital video stabilization and rolling shutter correction using gyroscopes. CSTR 2011; 2.
[4] Baker S, Bennett E, Kang SB, Szeliski, R. Removing rolling shutter wobble. Proc CVPR IEEE 2010; 2392-2399.
[5] Liu S, Wang Y, Yuan L, Bu J, Tan P, Sun, J. Video stabilization with a depth camera. Proc CVPR IEEE 2012; 89-95
[6] Liu S, Yuan L, Tan P, Sun J. Bundled camera paths for video stabilization. ACM T Graphic 2013; 78.
[7] Kopf J. 360 video stabilization. ACM T Graphic 2016; 195.
[8] Liu F, Gleicher M, Jin H, Agarwala, A. Content-preserving warps for 3D video stabilization. ACM T Graphic 2009; 44.
[9] Oshima M, Hayashi T, Fujioka S, Inaji T, Mitani H, Kajino J, Ikeda K, Komoda K. VHS camcorder with electronic image stabilizer. IEEE T Consum Electr 1989; 749-758.
[10] Winkler S. Issues in vision modeling for perceptual video quality assessment. Signal Processing 1999; 231-252.
[11] Wang Z, Sheikh HR, Bovik AC. Objective video quality assessment. The Handbook of Video Databases: Design and Applications 2003; 1041-1078.
[12] Wang Y, Ostermann J, Zhang YQ. Digital Video Processing and Communications. Hoboken, NJ, USA: Prentice Hall, 2001.
[13] Liu F, Gleicher M, Wang J, Jin H, Agarwala A. Subspace video stabilization. ACM T Graphic 2011; 4.
[14] Grundmann M, Kwatra V, Essa I. Auto-directed video stabilization with robust l1 optimal camera paths. Proc CVPR IEEE 2011; 225-232.
[15] Matsushita Y, Ofek E, Ge W, Tang X, Shum HY. Full-frame video stabilization with motion inpainting. IEEE T Pattern Anal 2006; 1150-1163.
[16] Liu S, Yuan L, Tan P, Sun J. Steadyflow: Spatially smooth optical flow for video stabilization. Proc CVPR IEEE 2014; 4209-4216.
[17] Safdarnejad SM, Atoum Y, Liu X. Temporally robust global motion compensation by keypoint-based congealing. European Conference on Computer Vision 2016; 101-119.
[18] Grundmann M, Kwatra V, Castro D, Essa I. Calibration-free rolling shutter removal. IEEE Conference on Computational Photography 2012; 1-8.
[19] Forssén PE, Ringaby E. Rectifying rolling shutter video from hand-held devices. Proc CVPR IEEE 2010; 507-514.
[20] Morimoto C, Chellappa R. Evaluation of image stabilization algorithms. Int Conf Acoust SPEE 1998; 2789-2792.
[21] Wang Z, Bovik AC, Sheikh HR, Simoncelli EP. Image quality assessment: from error visibility to structural similarity. IEEE T Image Process 2004; 600-612.
[22] Kinape RM, Amorim MF. A study of the most important image quality measures. International Conference of the Engineering in Medicine and Biology Society 2003; 934-936.
[23] Eskicioglu AM, Fisher PS. Image quality measures and their performance. IEEE T Commun 1995; 2959-2965.
[24] Niskanen M, Silvén O, Tico M. Video stabilization performance assessment. IEEE Int Con Multi 2006; 405-408.
[25] Qu H, Song L, Xue G. Shaking video synthesis for video stabilization performance assessment. P Soc Photo-Opt Ins 2013; 1-6.
[26] Zhai B, Zheng J, Wang Y, Zhang C. A multi-scale evaluation method for motion filtering in digital image stabilization. Proc Int C Tools Art 2015; 682-688.
[27] Zhang L, Chen XQ, Kong XY, Huang, H. Geodesic video stabilization in transformation space. IEEE T Image Process 2017; 2219-2229.
[28] St. Charles PL, Bilodeau GA, Bergevin R. Subsense: A universal change detection method with local adaptive sensitivity. IEEE T Image Process 2015; 359-373.
[29] Farnebäck G. Two-frame motion estimation based on polynomial expansion. Scandinavian Conference on Image Analysis 2003; 363-370.