Generative Networks and Royalty-Free Products

In recent years, with the increasing power of computers and Graphics Processing Units (GPUs), vast variety ofdeep neural networks architectures have been created and realized. One of the most interesting and generativetype of the networks are Generative Adversarial Networks (GANs). GANs are used to create things such asmusic, images or a film scenerio. GANs consist of two networks working simultaneously. Generative networkcaptures data distribution and discriminative network estimates the probability of the Generative Networkoutput, coming from training data of discriminative network. The objective is to both maximizing the generativenetwork products reality and minimize the discriminative network classification error. This procedure is aminimax two-player game. In this paper, it has been aimed to review the latest studies with GANs, to gather therecent studies in an article and to discuss the possible issues with royalty free products created by GANs. Withthis aim, from 2018 to today, the studies on GANs have been gathered to the citation numbers. As a result, therecent studies with GANs have been summarized and the potential issues related to GANs have been submitted.

Üretken Ağlar ve Telifsiz Ürünler

Son yıllarda, bilgisayarların ve Grafik İşlem Birimlerinin (GPU'ların) artan gücüyle, çok çeşitli derin sinir ağları mimarileri oluşturulmuş ve gerçekleştirilmiştir. En ilginç ve üretken ağ türlerinden biri de Üretken Çekişmeli Ağlardır (GANs). GAN ağları müzik, görüntü ve film senaryolarının üretiminde kullanılmaktadır. GAN ağları eş zamanlı çalışan iki ağ yapısından oluşmaktadır. Üretici ağ, veri dağıtımını üstlenmekte ayırıcı ağlar ise, ayrımcı ağın veya üretken ağ ürününün eğitim verilerinden gelen Üretken Ağ çıktısının olasılığını tahmin etmektedir. Amaç hem üretken ağın ürettiği verinin gerçekliğini maksimize ederken, ayırıcı ağın da hatasını minimize etmektir. Bu süreç iki oyunculu bir minimax problemidir. Bu çalışmada GAN ağları ile ilgili yapılan son çalışmaların gözden geçirilerek bir makale altında bir araya getirilmesi ve GAN ağları ile üretilen telifsiz ürünler ile ortaya çıkacak olası konuların tartışılması amaçlanmıştır. Bu amaç ile 2018’den günümüze bu konuda yapılmış olan çalışmaların atıf sayısı en yüksek olanları bu çalışmada bir araya getirilmiştir. Sonuç olarak GAN ağları ile yapılmış bu çalışmaların özet ve sonuçları bir tablo haline getirilerek sunulmaktadır. Bu şekilde GAN ağlarının mevcut uygulamaları bu çalışmada ortaya konulmaktadır. Yine GAN ağları ile ilgili olası sorunların ne olacağı da bu çalışmada sunulmaktadır.

PDF

___

[1] P. McCorduck, M. Minsky, Selfridge, O. G. & H. A. Simon, History of Artificial Intelligence. In IJCAI , pp. 951-954,1997.

[2] W. S. McCulloch and W. Pitts, “A logical calculus of the ideas immanent in neurons activity,” Bull. Math. Biophys., vol. 5, pp. 115–133, 1943.

[3] L. Zhang and B. Zhang, "A geometrical representation of McCulloch-Pitts neural model and its applications," in IEEE Transactions on Neural Networks, vol. 10, no. 4, pp. 925-929, 1999.

[4] F. Rosenblatt, The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, 65(6), pp. 386–408,1958.

[5] pace, “pace homepage,” 2019. [Online]. Available: http://csis.pace.edu/~ctappert/srd2011/rosenblatt-contributions.htm [Accessed: 20-Feb-2020].

[6] M. Minsky, & S. Papert, An introduction to computational geometry. Cambridge tiass., HIT, 1969.

[7] D. E. Rumelhart, E. H. Geoffrey, and R. J. Williams, Learning internal representations by error propagation. No. ICS-8506. California Univ San Diego La Jolla Inst for Cognitive Science, 1985.

[8] T. Kohonen, "The self-organizing map," in Proceedings of the IEEE, vol. 78, no. 9, pp. 1464- 1480, 1990.

[9] J. Elman, “Finding structure in time,” Cognitive Science, vol. 14, pp. 179-211, 1990.

[10] G. A.Carpenter, S. Grossberg, and D. B. Rosen. "Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system." Neural networks 4.6, pp.759-771,1991.

[11] Y. L. Cun and Y. Bengio, "Word-level training of a handwritten word recognizer based on convolutional neural networks," Proceedings of the 12th IAPR International Conference on Pattern Recognition, Conference C: Signal Processing (Cat. No.94CH3440-5), Jerusalem, Israel, vol.2, pp. 88-92, 1994.

[12] O. Russakovsky, J. Deng, , H. Su et al., ImageNet Large Scale Visual Recognition Challenge. Int J Comput Vis 115, pp. 211–252, 2015.

[13] J. Hu, J. Lu, Y. Tan and J. Zhou, "Deep Transfer Metric Learning," in IEEE Transactions on Image Processing, vol. 25, no. 12, pp. 5576-5588, 2016.

[14] I.Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, & Y. Bengio, Generative adversarial nets. In Advances in neural information processing systems, pp. 2672- 2680,2014.

[15] Hadjeres, Gaëtan, F. Pachet, and F. Nielsen, "Deepbach: a steerable model for bach chorales generation." Proceedings of the 34th International Conference on Machine Learning-Volume 70,2017.

[16] aiva, “aiva homepage,” 2019. [Online]. Available: https://www.aiva.ai/ [Accessed: 21-Feb2020].

[17] Dong, Hao-Wen et al., "MuseGAN: Symbolic-domain music generation and accompaniment with multi-track sequential generative adversarial networks.", 2017.

[18] L. Yang, S. Chou, and Y. Yang. MidiNet: A convolutional generative adversarial network for symbolic-domain music generation. In Proceedings of the 18th International Society for Music Information Retrieval Conference (ISMIR 2017), Suzhou, China, 2017.

[19] A. Oord, S.Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, and K. Kavukcuoglu. WaveNet: A generative model for raw audio, 2016.

[20] Hadjeres, Gaëtan, and F. Nielsen. "Anticipation-RNN: Enforcing unary constraints in sequence generation, with application to interactive music generation." Neural Computing and Applications, pp.1-11,2018.

[21] Briot, J. Pierre, G. Hadjeres, and F. Pachet. "Deep learning techniques for music generation--a survey.", 2017.

[22] Karras, Tero, S.Laine, and T. Aila. "A style-based generator architecture for generative adversarial networks." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019.

[23] Antipov, Grigory, M. Baccouche, and J. Dugelay. "Face aging with conditional generative adversarial networks." 2017 IEEE international conference on image processing (ICIP). IEEE, 2017.

[24] Oh, T.Hyun, et al.. "Speech2face: Learning the face behind a voice." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019.

[25] Heo, Hwan, and Y. Hwang. "Automatic Sketch Colorization using DCGAN." 2018 18th International Conference on Control, Automation and Systems (ICCAS). IEEE, 2018.

[26] thereforefilms, “thereforefilms homepage,” 2016. [Online]. Available: http://www.thereforefilms.com/sunspring.html [Accessed: 21-Feb-2020].

[27] I.Goodfellow, J. Pouget-Abadie, M.Mirza, B. Xu, D.Warde-Farley, S. Ozair, & Y. Bengio, Generative adversarial nets. In Advances in neural information processing systems ,pp. 2672- 2680, 2014.

[28] Dong, Hao-Wen, and Y. Yang, "Towards a deeper understanding of adversarial losses.", 2019.

[29] M. Babaee, Y. Zhu, O. Köpüklü, S. Hörmann and G. Rigoll, "Gait Energy Image Restoration Using Generative Adversarial Networks," 2019 IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan, pp. 2596-2600, 2019.

[30] C. He and Z. Zhang, "Restoration of Underwater Distorted Image Sequence Based on Generative Adversarial Network," 2019 IEEE 8th Joint International Information Technology and Artificial Intelligence Conference (ITAIC), Chongqing, China, pp. 866-870, 2019.

[31] Q. Wang, H. Fan, L. Zhu and Y. Tang, "Deeply Supervised Face Completion With MultiContext Generative Adversarial Network," in IEEE Signal Processing Letters, vol. 26, no. 3, pp. 400-404, 2019.

[32] Y. Shi, Q. Li and X. X. Zhu, "Building Footprint Generation Using Improved Generative Adversarial Networks," in IEEE Geoscience and Remote Sensing Letters, vol. 16, no. 4, pp. 603-607,2019.

[33] G. Gong and K. Zhang, "Local Blurred Natural Image Restoration Based on Self-Reference Deblurring Generative Adversarial Networks," 2019 IEEE International Conference on Signal and Image Processing Applications (ICSIPA), Kuala Lumpur, Malaysia, pp. 231-235, 2019.

[34] J. Zheng, W. Song, Y. Wu, R. Xu and F. Liu, "Feature Encoder Guided Generative Adversarial Network for Face Photo-Sketch Synthesis," in IEEE Access, vol. 7, pp. 154971-154985, 2019.

[35] L. He and J. Zhang, "Snowflakes Removal for Single Image Based on Model Pruning and Generative Adversarial Network," 2019 IEEE 4th International Conference on Image, Vision and Computing (ICIVC), Xiamen, China, pp. 172-176, 2019.

[36] P. Xiang, L. Wang, F. Wu, J. Cheng and M. Zhou, "Single-Image De-Raining With FeatureSupervised Generative Adversarial Network," in IEEE Signal Processing Letters, vol. 26, no. 5, pp. 650-654, 2019.

[37] R. Yin, "Multi-Resolution Generative Adversarial Networks for Tiny-Scale Pedestrian Detection," 2019 IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan, pp. 1665-1669, 2019.

[38] F. Gu, H. Zhang, C. Wang and F. Wu, "SAR Image Super-Resolution Based on Noise-Free Generative Adversarial Network," IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan, pp. 2575-2578, 2019.

[39] H. Huang, F. Zhang, Y. Zhou, Q. Yin and W. Hu, "High Resolution SAR Image Synthesis with Hierarchical Generative Adversarial Networks," IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan, pp. 2782-2785, 2019.

[40] R. Jiang et al., "Learning Spectral and Spatial Features Based on Generative Adversarial Network for Hyperspectral Image Super-Resolution," IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan, pp. 3161-3164, 2019.

[41] C. Zheng, X. Jiang, Y. Zhang, X. Liu, B. Yuan and Z. Li, "Self-Normalizing Generative Adversarial Network for Super-Resolution Reconstruction of SAR Images," IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan, pp. 1911-1914, 2019.

[42] H. Liu, F. Wang and L. Liu, "Image Super-resolution Reconstruction Based on an Improved Generative Adversarial Network," 2019 1st International Conference on Industrial Artificial Intelligence (IAI), Shenyang, China, pp. 1-6, 2019.

[43] H. Wang, W. Wu, Y. Su, Y. Duan and P. Wang, "Image Super-Resolution using a Improved Generative Adversarial Network," 2019 IEEE 9th International Conference on Electronics Information and Emergency Communication (ICEIEC), Beijing, China, pp. 312-315, 2019.

[44] L. Liu, S. Wang and L. Wan, "Component Semantic Prior Guided Generative Adversarial Network for Face Super-Resolution," in IEEE Access, vol. 7, pp. 77027-77036, 2019.

[45] I. Alnujaim, D. Oh and Y. Kim, "Generative Adversarial Networks to Augment Micro-Doppler Signatures for the Classification of Human Activity," IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan, pp. 9459-9461, 2019.

[46] Y. Tang, Y. Tang, M. Han, J. Xiao and R. M. Summers, "Abnormal Chest X-Ray Identification With Generative Adversarial One-Class Classifier," 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), Venice, Italy, pp. 1358-1361, 2019.

[47] Z. Pan, W. Yu, X. Yi, A. Khan, F. Yuan and Y. Zheng, "Recent Progress on Generative Adversarial Networks (GANs): A Survey," in IEEE Access, vol. 7, pp. 36322-36333, 2019.

[48] Ö. Ö. Karadağ and Ö. Erdaş Çiçek, "Experimental Assessment of the Performance of Data Augmentation with Generative Adversarial Networks in the Image Classification Problem," 2019 Innovations in Intelligent Systems and Applications Conference (ASYU), Izmir, Turkey, pp. 1-4, 2019.

[49] L. Li, C. Wang, H. Zhang and K. Zhang, "SAR Image Urban Scene Classification based on an Optimized Conditional Generative Adversarial Network," 2019 SAR in Big Data Era (BIGSARDATA), Beijing, China, pp. 1-4, 2019.

[50] W. Han, R. Feng, L. Wang and J. Chen, "Supervised Generative Adversarial Network Based Sample Generation for Scene Classification," IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan, pp. 3041-3044, 2019.

[51] J. Liu et al., "Semi-Supervised Change Detection Based on Graphs with Generative Adversarial Networks," IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan, pp. 74-77, 2019.

[52] J. Ma, Z. Zhou, B. Wang and Z. An, "Hard Ship Detection via Generative Adversarial Networks," 2019 Chinese Control And Decision Conference (CCDC), Nanchang, China, pp. 3961-3965, 2019.

[53] J. Shen, N. Liu, H. Sun and H. Zhou, "Vehicle Detection in Aerial Images Based on Lightweight Deep Convolutional Network and Generative Adversarial Network," in IEEE Access, vol. 7, pp. 148119-148130, 2019.

[54] P. Wang, B. Hou, S. Shao and R. Yan, "ECG Arrhythmias Detection Using Auxiliary Classifier Generative Adversarial Network and Residual Network," in IEEE Access, vol. 7, pp. 100910-100922, 2019.

[55] G. Mateo-García, V. Laparra and L. Gómez-Chova, "Domain Adaptation of Landsat-8 and Proba-V Data Using Generative Adversarial Networks for Cloud Detection," IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan, pp. 712-715, 2019.

[56] B. Mandal, N. B. Puhan and A. Verma, "Deep Convolutional Generative Adversarial NetworkBased Food Recognition Using Partially Labeled Data," in IEEE Sensors Letters, vol. 3, no. 2, pp. 1-4, 2019.

[57] D. Wadhwa, U. Maharana, D. Shah, V. Yadav and P. Pandey, "Human Sketch Recognition using Generative Adversarial Networks and One-Shot Learning," 2019 Twelfth International Conference on Contemporary Computing (IC3), Noida, India, pp. 1-6, 2019.

[58] J. Gu et al., "Aerial Image and Map Synthesis Using Generative Adversarial Networks," IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan, pp. 9803-9806, 2019.

[59] Z. Zhu, Y. Lu and C. Chiang, "Generating Adversarial Examples By Makeup Attacks on Face Recognition," 2019 IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan, pp. 2516-2520, 2019.

[60] F. Chen, F. Zhu, Q. Wu, Y. Hao, Y. Cui and E. Wang, "InfraRed Images Augmentation Based on Images Generation with Generative Adversarial Networks," 2019 IEEE International Conference on Unmanned Systems (ICUS), Beijing, China, pp. 62-66. 2019.

[61] Y. Chen, Y. Lv and F. Wang, "Traffic Flow Imputation Using Parallel Data and Generative Adversarial Networks," in IEEE Transactions on Intelligent Transportation Systems, vol. 21, no. 4, pp. 1624-1630, 2020.

[62] S. Chen, D. Zhang, L. Yang and P. Chen, "Age-invariant Face Recognition Based on Sample Enhancement of Generative Adversarial Networks," 2019 6th International Conference on Systems and Informatics (ICSAI), Shanghai, China, pp. 388-392, 2019.

[63] Y. Cui and W. Wang, "Colorless Video Rendering System via Generative Adversarial Networks," 2019 IEEE International Conference on Artificial Intelligence and Computer Applications (ICAICA), Dalian, China, pp. 464-467, 2019.

[64] Y. Pang, J. Xie and X. Li, "Visual Haze Removal by a Unified Generative Adversarial Network," in IEEE Transactions on Circuits and Systems for Video Technology, vol. 29, no. 11, pp. 3211-3221, 2019.

[65] C. Xiang, J. Xu, C. Yan, Q. Peng and X. Wu, "Generative Adversarial Networks Based Error Concealment for Low Resolution Video," ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, United Kingdom, pp. 1827- 1831,2019.

[66] X. Wang, G. Xu, Y. Wang, D. Lin, P. Li and X. Lin, "Thin and Thick Cloud Removal on Remote Sensing Image by Conditional Generative Adversarial Network," IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan, pp. 1426- 1429, 2019.

[67] K. Lata, M. Dave and K. N. Nishanth, "Image-to-Image Translation Using Generative Adversarial Network," 2019 3rd International conference on Electronics, Communication and Aerospace Technology (ICECA), Coimbatore, India, 2019, pp. 186-189.

[68] R. Yanagi, R. Togo, T. Ogawa and M. Haseyama, "Query is GAN: Scene Retrieval With Attentional Text-to-Image Generative Adversarial Network," in IEEE Access, vol. 7, pp. 153183-153193, 2019.

[69] Z. Gao, B. Peng, T. Li and C. Gou, "Generative Adversarial Networks for Road Crack Image Segmentation," 2019 International Joint Conference on Neural Networks (IJCNN), Budapest, Hungary, pp. 1-8, 2019.

[70] F. Jiang, F. Zhou, J. Qin, T. Wang and B. Lei, "Decision-Augmented Generative Adversarial Network for Skin Lesion Segmentation," 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), Venice, Italy, pp. 447-450, 2019.

[71] S. Liu, J. Zhang, Y. Chen, Y. Liu, Z. Qin and T. Wan, "Pixel Level Data Augmentation for Semantic Image Segmentation Using Generative Adversarial Networks," ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, United Kingdom, pp. 1902-1906, 2019.

[72] Y. Cao et al., "Recent Advances of Generative Adversarial Networks in Computer Vision," in IEEE Access, vol. 7, pp. 14985-15006, 2019.

[73] T. Li and D. P. K. Lun, "Image Reflection Removal Using the Wasserstein Generative Adversarial Network," ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, United Kingdom, pp. 1-5, 2019.

[74] Z. Zhou, Y. Wang, Y. Guo, Y. Qi and J. Yu, "Image Quality Improvement of Hand-Held Ultrasound Devices With a Two-Stage Generative Adversarial Network," in IEEE Transactions on Biomedical Engineering, vol. 67, pp. 298-311, 2020.

[75] A. Förster, J. Behley, J. Behmann and R. Roscher, "Hyperspectral Plant Disease Forecasting Using Generative Adversarial Networks," IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan, pp. 1793-1796, 2019.

[76] H. Wang, C. Tao, J. Qi, H. Li and Y. Tang, "Semi-Supervised Variational Generative Adversarial Networks for Hyperspectral Image Classification," IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan, pp. 9792-9794, 2019.

[77] copyright, “copyright homepage,” 2020. [Online]. Available: https://www.copyright.gov/ [Accessed: 21-Apr-2020].