Muhammad Nasir KHAN, Muhammad UZAIR, Bilal A. KHAWAJA, Robert D. DONY, Mohsin JAMIL

A no-reference framework for evaluating video quality streamed through wireless network

In this work, a no-reference framework is proposed for the video quality estimation streamed through thewireless network. The work presents a comprehensive survey of the existing full reference (FR), reduced reference(RR), and no-reference (NR) algorithms. A comparison has been made among existing algorithms, i.e. in terms ofsubjective correlation and feasibility to use these algorithms in wireless architecture, to describe the necessity of theproposed framework to overcome the limitations of the existing algorithms. A brief summary of our previously publishedalgorithms, i.e. NR blockiness, NR blur, NR network, NR just noticeable distortion, and RR, has also been presented.These algorithms have also been used as function modules in the proposed framework. The proposed framework is ableto measure the video quality by taking into account major spatial, temporal, network impairments, and human visualsystem effects for a comprehensive quality evaluation. The proposed framework is able to measure the video qualitycompressed by different codecs, i.e. MPEG x / H.264x, Motion JPEG/Motion, and JPEG2000, etc. The frameworkis able to work with two different kinds of received data, i.e. bit streams and decoded pixels. The framework is anintegration of the RR and NR method, and can work in three different modes depending on the availability of the RRdata, i.e. 1) only RR measurement, 2) hybrid of RR and NR measurement, and 3) only NR estimation. In addition,any individual function block, i.e. blurring, can also be used independently for particular specific distortion. A newsubjective video quality database containing compressed and distorted videos (due to channel induced distortions) is alsodeveloped to test the proposed framework. The framework has also been tested on publicly available LIVE Video QualityDatabase. Overall test results show that our framework demonstrates a strong correlation with subjective evaluation ofthe two separate video databases as compared with other existing algorithms. The proposed framework also shows goodresults while working only in NR mode as compared with existing RR and FR algorithms. The proposed framework ismore scalable and feasible to use in any kind of available network bandwidth as compared with other algorithms, as itcan be used in different modes by using different function modules.

PDF

___

[1] Lajos H, Peter J, Jurgen S. Wireless Video Communication: Second to Third Generation Systems and Beyond. New York, NY, USA: Wiley, 2001.
[2] Theodore S. Wireless Communication: Principle and Practice. NJ, USA: Prentice Hall, 2002.
[3] Haohong W, Lisimachos P. 4G Wireless Video Communications. London, UK: Wiley, 2009.
[4] Otto R, Schd S. Optical and photoelectric analog of eye. Journal of Optical Society of America 1956; 46 (9): 721-738. doi: org/10.1364/JOSA.46.000721
[5] Mannos J, Sakrison D. The effects of a visual fidelity criterion of the encoding of images. IEEE Transactions on Information Theory 1974; 20 (4): 525-536. doi: 10.1109/TIT.1974.1055250
[6] Hekstra P, Beerends J, Ledermann D, Caluwe F, Kohler S. PVQM – a perceptual video quality measure. ELSEVIER Signal Processing: Image Communication 2002; 17 (10): 781-798. doi: org/10.1016/S0923-5965(02)00056-5
[7] Wang Z, Simoncelli P, Bovik A. Multiscale structural similarity for image quality assessment. In: IEEE 2003 Conference on Signals, Systems and Computers; Asilomar, California, USA; 2003. pp. 1398-1402.
[8] Shnayderman A, Gusev A, Eskicioglu M. An SVD-based grayscale image quality measure for local and global assessment. IEEE Transactions on Image Processing 2006; 15 (2): 422-429. doi: 10.1109/TIP.2005.860605
[9] Jianxin P, Rong Z, Lu L, Jinhui T, Zhengkai L. A projection‐based image quality measure. International Journal of Imaging Systems and Technology 2008; 18 (2): 94-100. doi: org/10.1002/ima.20156
[10] Jianxin P, Rong Z, Zhenkai L. Image quality assessment metrics with radon transform. In: IEEE 2008 International Conference on Systems, Man and Cybernetics; Singapore, Singapore; 2008. pp. 1-6.
[11] Arthur A, Coleen T, Stephen W. Objective video quality assessment system based on human perception. Proceedings of SPIE 1993; 1913: 15-26. doi: 10.1117/12.152700
[12] Ulrich E, Tubagus K, Jurgen Z. Perceptual quality assessment of wireless video applications. In: IEEE 2006 4th International Symposium on Turbo Codes and Related Topics; Munich, Germany; 2006. pp. 1-6.
[13] Stephen W, Margaret H. Spatial-temporal distortion metric for in-service quality monitoring of any digital video system. Proceedings of SPIE 1999; 3845: 266-277. doi: org/10.1117/12.371210
[14] Uzair M, Fayek D. Reduced reference image quality assessment using principal component analysis. In: IEEE 2011 International Symposium on Broadband Multimedia Systems and Broadcasting; Munich, Germany; 2011. pp. 1-6.
[15] Miyahara M, Kotani K, Algazi R. Objective picture quality scale (PQS) for image coding. IEEE Transactions on Communications 1998; 46 (9): 1215-1226. doi: 10.1109/26.718563
[16] Chan W, Goldsmith P. A psychovisually-based image quality evaluator for JPEG images. In: IEEE 2000 International Conference on Systems, Man, and Cybernetics; Nashville, TN, USA; 2000. pp. 1541-1546.
[17] Wu H. A generalized block-edge impairment metric for video coding. IEEE Signal Processing 1997; 4 (11): 317-320.
[18] Uzair M, Fayek D. An efficient no-reference blockiness metric for intra-coded video frames. In: IEEE 2011 14th International Symposium on Wireless Personal Multimedia Communications; Brest, France; 2011. pp. 1-6.
[19] Erasmus S, Smithi K. An automatic focusing and astigmatism correction system for the SEM and CTEM. Journal of Microscopy 1982; 127 (2): 185-199. doi: 10.1111/j.1365-2818.1982.tb00412.x
[20] Wang Z, Sheikh H, Bovik A. Handbook of Image and Video Processing. New York, NY, USA: Academic Press, 2000.
[21] Uzair M, Dony R. An efficient no-reference blurriness metric for images and video frames. In: IEEE 2016 Canadian Conference on Electrical and Computer Engineering; Vancouver, Canada; 2016. pp. 1-4.
[22] Amir M. A video streaming quality assessment scheme based on packet level measurement. In: IEEE 2015 International Conference on Communications and Signal Processing; Sharjah, UAE; 2015. pp. 1556-1562.
[23] Matteo N, Marco T, Stefano T. No-reference video quality monitoring for H.264/AVC coded video. IEEE Transactions on Multimedia 2015; 11 (5): 932-946. doi: 10.1109/TMM.2009.2021785
[24] Mario V, Denis V. Video transmission artifacts detection using no-reference approach. In: IEEE 2018 Zooming Innovation in Consumer Technologies Conference; Novi Sad, Serbia; 2018. pp. 1932-1937.
[25] Zhiguo H, Qiqiang Z. A new approach for packet loss measurement of video streaming and its application. Springer Multimedia Tools and Applications 2018; 77 (10): 1158–1168. doi: 10.1007/s11042-016-3566-0
[26] Uzair M, Dony R. No-reference transmission distortion modelling for H.264/AVC coded video. IEEE Transactions on Signal and Information Processing over Networks 2015; 1 (3): 209-221. doi: 10.1109/TSIPN.2015.2476695
[27] Chou C, Chen C. A perceptually optimized 3-D subband codec for video communication over wireless channels. IEEE Transactions on Circuits and Systems for Video Technology 1996; 6 (2): 143–156. doi: 10.1109/76.488822
[28] Anmin L, Fan Z. Just noticeable difference for images with decomposition model for separating edge and textured regions. IEEE Transactions on Circuits and Systems for Video Technology 2010; 20 (11): 1648-1652. doi: 10.1109/TCSVT.2010.2087432
[29] Ahumada A, Peterson H. Luminance-model-based DCT quantization for color image compression. Proceedings of SPIE 1992; 1666: 365-374. doi: org/10.1117/12.135982
[30] Chou C, Chen W. A perceptually optimized 3-D subband codec for video communication over wireless channels. IEEE Transactions on Circuits and Systems for Video Technology 1996; 6 (2): 143-156. doi: 10.1109/76.488822
[31] Kelly D. Motion and vision. II. stabilized spatio-temporal threshold surface. Journal of the Optical Society of America 1979; 69 (10): 1340-1349. doi: 10.1364/JOSA.69.001340
[32] Daly S. Engineering observations from spatiovelocity and spatiotemporal visual models. Proceedings of SPIE 1998; 3299: 180-191. doi: doi.org/10.1117/12.320110
[33] Uzair M, Dony R. Estimating just-noticeable distortion for images/videos in pixel domain. IET Journal in Image Processing 2017; 11 (8): 559-567. doi: 10.1049/iet-ipr.2016.1120
[34] Mannos J, Sakrison D. Video quality assessment accounting for temporal visual masking of local flicker. ELSEVIER Signal processing: Image communication 2018; 67: 182-198. doi: org/10.1016/j.image.2018.06.009
[35] Peng P. An efficient temporal distortion measure of videos based on space time texture. ELSEVIER Pattern Recognition 2017; 70: 1-11. doi: 10.1016/j.patcog.2017.04.031
[36] Bampis G. Enhancing temporal quality measurements in a globally deployed streaming video quality predictor. In: IEEE 2018 25th IEEE International Conference on Image Processing; Athens, Greece; 2018. pp. 614-618.
[37] Yuming L, Lai P, Fang Y. No-reference image quality assessment with deep convolutional neural networks. In: IEEE 2016 International Conference on Digital Signal Processing; Beijing, China; 2016. pp. 685-689.
[38] Maria T, Antonio L. Predictive no-reference assessment of video quality. ELSEVIER Signal Processing: Image Communication 2017; 52: 20-32. doi: 10.1016/j.image.2016.12.001
[39] Sewong A. No-reference video quality assessment based on convolutional neural network and human temporal behavior. In: Asia-Pacific Signal and Information Processing; Hawaii, USA; 2018. pp. 318-325.
[40] Sebastian B, Wojciech S. Deep neural networks for no-reference and full-reference image quality assessment. IEEE Transactions on Image Processing 2018; 27 (1): 206-219. doi: 10.1109/TIP.2017.2760518
[41] Yuming L, Chun C, Xuyuan Xu. No-reference video quality assessment with 3D shearlet transform and convolutional neural networks. IEEE Transactions on Circuits and Systems for Video Technology 2016; 26 (6): 575-589. doi: 10.1109/TCSVT.2015.2430711
[42] Varga A, Hornig R. An overview of the OMNeT++ simulation environment. In: SimuTools 2008 International Conference on Simulation Tools for Communications and Networks; Marseile, France; 2008. pp. 1-11.
[43] Alexis M, Karsten S, Gary S. H.264/14496-10 AVC Reference Software Manual. London, UK: Dolby Laboratories Inc., Fraunhofer Institute HHI, Microsoft Corporation, 2009.
[44] Wang Z, Simoncelli P, Bovik A. Multiscale structural similarity for image quality assessment. In: IEEE 2003 Conference on Signals, Systems and Computers; Asilomar, California, USA; 2003. pp. 1398-1402.
[45] Seshadrinathan K, Soundararajan R, Bovik A, Cormack L. Study of subjective and objective quality assessment of video. IEEE Transactions on Image Processing 2010; 19 (6): 1427-1441. doi: 10.1109/TIP.2010.2042111
[46] Moorthy A, Seshad K, Soundar R, Bovik A. Wireless video quality assessment: a study of subjective scores and objective algorithms. IEEE Transactions on Circuits and Systems for Video Technology 2010; 20 (4): 587-599. doi: 10.1109/TCSVT.2010.2041829
[47] Rajiv S, Alan C. Video quality assessment by reduced reference spatio-temporal entropic differencing. IEEE Transactions on Circuits and Systems for Video Technology 2013; 23 (4): 684-694. doi: 10.1109/TCSVT.2012.2214933
[48] Wang Z, Simoncelli P. Reduced-reference image quality assessment using a wavelet-domain natural image statistic model. Proceeding of SPIE 2005; 5666: 1578-1583. doi: org/10.1117/12.597306