A novel optical ow-based representation for temporal video segmentation

Temporal video segmentation is a eld of multimedia research enabling us to temporally split video data into semantically coherent scenes. In order to develop methods challenging temporal video segmentation, detecting scene boundaries is one of the more widely used approaches. As a result, representation of temporal information becomes important. We propose a new temporal video segment representation to formalize video scenes as a sequence of temporal motion change information. The idea here is that some sort of change in the optical ow character determines motion change and cuts between consecutive scenes. The problem is eventually reduced to an optical ow-based cut detection problem from which the average motion vector concept is put forward. This concept is used for proposing a pixel-based representation enriched with a novel motion-based approach. Temporal video segment points are classi ed as cuts and noncuts according to the proposed video segment representation. Consequently, the proposed method and representation is applied to benchmark data sets and the results are compared to other state-of-the art methods

PDF

___

[1] Gibson JJ. The Perception of the Visual World. Boston, MA, USA: Houghton Mifflin, 1950.
[2] Royden CS, Moore KD. Use of speed cues in the detection of moving objects by moving observers. Vision Res 2012; 59: 17-24.
[3] Vasileios TC, Aristidis CL, Nikolaos PG. Scene detection in videos using shot clustering and sequence alignment. IEEE T Multimedia 2009; 11: 89-100.
[4] Gianluigi C, Raimondo S. An innovative algorithm for key frame extraction in video summarization. Journal of Real-Time Image Processing 2006; 1: 69-88.
[5] Ballan L, Bertini M, Bimbo AD, Serra G. Video event classi cation using string kernels. Multimed Tools Appl 2010; 48: 69-87.
[6] Wang F, Jiang YG, Ngo CW. Video event detection using motion relativity and visual relatedness. In: ACM Multimedia; 27{31 October 2008; Vancouver, BC, Canada. New York, NY, USA: IEEE. pp. 239-248.
[7] Sand P, Teller S. Particle video: long-range motion estimation using point trajectories. Int J Comput Vision 2008; 80: 72-91.
[8] Chaudry R, Ravichandran A, Hager G, Vidal R. Histograms of oriented optical ow and Binet-Cauchy kernels on nonlinear dynamical systems for the recognition of human actions. In: Conference on Computer Vision and Pattern Recognition; 22{24 June 2009; Miami, FL, USA. New York, NY, USA: IEEE. pp. 1932-1939.
[9] Lertniphonphan K, Aramvith S. Human action recognition using direction histograms of optical ow. In: Interna- tional Symposium on Communications and Information Technologies; 12{14 October 2011; Hangzhou, China. New York, NY, USA: IEEE. pp. 574-579.
[10] Laptev I, Lindeberg T. Space-time interest points. In: International Conference on Computer Vision; 13{16 October 2003; France. New York, NY, USA: IEEE. pp. 432-439.
[11] Barbu T. Novel automatic video cut detection technique using Gabor ltering. Comput Electr Eng 2009; 35: 712{ 721.
[12] Whitehead A, Bose J, Laganiere R. Feature based cut detection with automatic threshold selection. In: International Conference on Image and Video Retrieval; 21{23 July 2004; Dublin, Ireland. New York, NY, USA: IEEE. pp. 410- 418.
[13] Pass G, Zabih R. Comparing images using joint histograms. Multimedia Syst 1999; 7: 234-240.
[14] Effelsberg W. The MoCA Project{Movie Content Analysis Research at the University of Mannheim. Mannheim, Germany: Jahrestagunug University of Mannheim, 1998.
[15] Xiong W, Chung J, Lee M. Efficient scene change detection and camera motion annotation for video classi cation. Comput Vis Image Und 1998; 71: 166-181.
[16] Kowdle A, Chen T. Learning to segment a video to clips based on scene and camera motion. In: European Conference on Computer Vision; 7{13 October 2012; Firenze, Italy. New York, NY, USA: IEEE. pp. 272-286.
[17] Fatemi O, Zhang S, Panchanathan S. Optical ow based model for scene cut detection. In: Canadian Conference on Electrical and Computer Engineering; 26{28 May 1986; Calgary, AB, Canada. New York, NY, USA: IEEE. pp. 470-473.
[18] Roghayeh D, Hamidreza R, Rashidy K. AVCD-FRA: A novel solution to automatic video cut detection using fuzzy-rule-based approach. Computer Vis Image Und 2013; 117: 807-817.
[19] Barron J, Fleet D, Beauchemin S. Performance of optical ?ow techniques. Int J Comput Vision 1994; 12: 43-77.
[20] Horn KP, Schunck BG. Determining optical ?ow. Artif Intell 1981; 17: 185-203.
[21] Lucas B, Kanade T. An iterative image registration technique with an application to stereo vision. In: Proceedings of the Imaging Understanding Workshop; 1981. New York, NY, USA: IEEE. pp. 121-130.
[22] Anandan P. A computational framework and algorithm for the measurement of visual motion. Int J Comput Vision 1989; 2: 283-310.
[23] Heeger DJ. Optical ow using spatiotemporal lters. Int J Comput Vision 1988; 1: 279-302.
[24] Buxton B, Buxton H. Computation of optical ow from the motion of edge features in image sequences. Image Vision Comput 1984; 2: 59-74.
[25] Fleet DJ, Jepson AD. Computation of component image velocity from local phase information. Int J Comput Vision 1990; 5: 77-104.
[26] Shi J, Tomasi C. Good features to track. In: Conference on Computer Vision and Pattern Recognition; 21{23 June 1994; Seattle, WA, USA. New York, NY, USA: IEEE. pp. 593-600.
[27] Harris C, Stephens M. A combined corner and edge detector. In: 4th Alvey Vision Conference; 31 August{2 September 1988; Manchester, UK. New York, NY, USA: IEEE. pp. 147-151.
[28] Laptev I, Marszalek M, Schmid C, Rozenfeld B. Learning realistic human actions from movies. In: Conference on Computer Vision and Pattern Recognition; 23{28 June 2008; Anchorage, AK, USA. New York, NY, USA: IEEE. pp. 1-8.
[29] Akpnar S, Alpaslan FN. Optical ow-based representation for video action detection. In: Deligiannidis L, Arabnia H, editors. Emerging Trends in Image Processing, Computer Vision and Pattern Recognition. Boston, MA, USA: Morgan Kaufmann, 2014. pp. 331-351