LATİS TABANLI ANLAM ÇÖZÜMLENMESİ İLE TÜRKÇE İŞARET DİLİ TERCÜME SİSTEMİ

Günlük hayatta insanlar; fikirlerini, düşüncelerini ve yaşadıklarını çevrelerindeki insanlara iletmek için birbirleriyle etkileşirler. İşitme ve konuşma engelli insanlar ise çevreleriyle bu etkileşimi sağlayamazlar. Başkalarıyla iletişim kurmak için işaret dilini kullanırlar. İşaret dili ise, işitme ve konuşma engellilerin kendi aralarında el hareketleri ve yüz mimikleri ile iletişim kurmalarını sağlayan ülkeden ülkeye değişen evrensel olmayan bir dildir. Bu anlamda yapılan çalışmanın amacı, işitme ve konuşma bozukluğu olan kişilerle normal insanlar arasındaki iletişimi sağlayan Türkçe İşaret Dili tercüme sistemini geliştirmektir. Önerilen bu sistemle işaret dilini gösteren el hareketleri, Kinect aygıtı yardımıyla yakalanarak Kontur Analizinde kullanılan algoritmalarla çözümlenmiştir. Çözümlenen görüntülerden elde edilen kelimelerin gerçek anlamları, Biçimsel Kavram Analizi Kuramı çerçevesinde hazırlanan tematik rol latisleriyle bulunmuştur. Gerçek anlamları bulunan bu kelimeler, içinde geçtikleri cümlelerle birlikte bilgisayar ekranında görüntülenmiştir. Böylece bu sistemle, engelli bir kişinin kitlesel bir kalabalıkla iletişim kurabilmesi sağlanmıştır. Ayrıca geliştirilen bu sistemle Türkçe’nin anlamsal çözümlenmesine de katkı sağlanması hedeflenmiştir.

Anahtar Kelimeler:

Biçimsel kavram analizi, işaret dili tanımlama, kinect, kontur analizi, tematik rol latisleri

TURKISH SIGN LANGUAGE TRANSLATION SYSTEM WITH A LATTICE-BASED SEMANTIC ANALYSIS

In daily life people interact with each other to communicate their ideas, thoughts and experiences to the people around them. On the other hand, people with hearing and speech impairments are not capable of communicating with their environment in the given sense. Thus, they use sign language to communicate with others. Sign language is a non-universal language that changes depending on the country in which it's being used and enables people with hearing and speech impairments to communicate with other people through hand gestures and facial expressions. In this sense, the aim of the study is to develop Turkish Sign Language translation system that enables communication between people with speech and hearing impairments and healthy people. With this proposed system, the hand movements showing sign language are captured with the help of Kinect device and analyzed with the algorithms used in Contour Analysis. Actual meanings of the words obtained from the analyzed images were found thematic role lattices that are prepared in the frame of Formal Concept Analysis theory. The words of which actual meanings were found are displayed with the sentences in which they were used. Therefore, thanks to this system it is ensured that a disabled person can communicate with a mass crowd. The developed system also aims to contribute to the semantic analysis of the Turkish Language.

Keywords:

Formal concept analysis, sign language recognition, kinect, contour analysis, thematic role lattices,

PDF

___

[1] AARTHI, M., VIJAYALAKSH, P., “Sign Language to Speech Conversion”, Fifth International Conference on Recent Trends in Information Technolgy, 2016, Doi: 10.1109/ICRTIT.2016.7569545.
[2] AKMELIAWATIL, R., PO-LEEN OOI, M., KUANG, Y. C., “Real-Time Malaysian Sign Language Translation using Color Segmentation and Neural Network”, IEEE Instrumentation and Measurement Technology Conference, 1-6, Warsaw, Poland, 2007.
[3] ARSAN, T., ÜLGEN, O., “Sign Language Converter”, International Journal of Computer Science & Engineering Survey (IJCSES), 6(4), 39-51, 2015.
[4] BUI, T. D., NGUYEN, L. T., “Recognition of Vietnamese sign language using MEMS accelerometers” 1st International Conference on Sensing Technology, 118-122, Palmerston North, New Zealand, 2005.
[5] ELLIS, K., BARCA, J. C., “Exploring Sensor Gloves for Teaching Children Sign Language”, Advances in Human-Computer Interaction, 2012, 1-8, 2012.
[6] ERSHAED, H., AL-ALALI, I., KHASAWNEH, N., FRAIWAN, M., “An arabic sign language computer interface using the xbox Kinect”, In Annual Undergraduate Research Conf. on Applied Computing, May, 2011.
[7] FIRAT, Y., UÇAR, Ö., KILIÇASLAN, Y., “Semantic Analysis with a LatticeBased FrameNet”, Journal of International Scientific Publications: Language, Individual & Society, 8, 512-518, Bulgaria, 2014.
[8] FIRAT, Y., “The Semantic Inferences And Mappings Realized In Computer Through The Formal Concept Analysis”, Journal of the International Scientific Researches, 2(1), 86-107, 2017, Doi number:http://dx.doi.org/10.21733/ibad2099.
[9] FIRAT, Y., “Referans Parametreleri ile Biçimlendirilmiş Kavram Latislerinin Bilgisayarlı Gerçekleştirimi”, Erzincan Üniversitesi, Fen Bilimleri Enstitüsü Dergisi, 10(1), 90-111, 2017, DOI: 10.18185/erzifbed.285800. [10] FRAIWAN, M., KHASAWNEH, N., ERSHEDAT, H., AL-ALALI, I., AL-KOFAHI, H., ”A Kinect-based system for Arabic sign language to speech translation”, International Journal of Computer Applications in Technology, 52(2), 117-126, 2015.
[11] GANTER, B., WILLE, R., Formal Concept Analysis Mathematical Foundation, 5-23, Berlin: Springer, Verlag, 1999.
[12] GAO, W., FANGA, G., “A Chinese sign language recognition system based on SOFM/SRN/HMM”, Journal of Pattern Recognition, 37 (12), 2389-2402, 2004.
[13] GUNASEKARAN, K., MANIKANDAN, R., “Sign Language to Speech Translation System Using PIC Microcontroller”, International Journal of Engineering and Technology, 5(2), 1024-1028, 2013.
[14] HERNNDEZ-REBOLLAR, J.L., LINDEMAN, R.W., KYRIAKOPOULOS, N., “A multi-class pattern recognition system for practical finger spelling translation”, 4th IEEE International Conference on Multimodal Interfaces (ICMI’02), 185-190, Pittsburg, USA, 2002.
[15] JACKENDOFF, R., “On the Role of Conceptual Structure in Argument Selection: A Reply to Emonds”, Natural Language and Linguistic Theory 11, 11(2), 279-312, 1993.
[16] LI, Y., CHEN, X., ZHANG, X., WANG, K., WANG, ZJ., “Sign-Component-Based Framework for Chinese Sign Language Recognition Using Accelerometer and sEMG Data”, IEEE Transactions On Biomedical Engineering, 59(10), 2695-2704, 2012.
[17] ONG, EJ., BOWDEN, R., “A boosted classifier tree for hand shape detection”, In Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition FGR'04, IEEE Computer Society, 889-894, Washington, DC, USA, 2004.
[18] QUAN, Y., “Chinese Sign Language Recognition Based On Video Sequence Appearance Modeling”, 5th IEEE Conference on Industrial Electronics and Applications, 1537-1542, 2010.
[19] RAJAGANAPATHY, S., ARAVIND, B., KEERTHANA, B. , SIVAGAMI, M., “Conversation of Sign Language to Speech with Human Gestures”, 2nd International Symposium on Big Data and Cloud Computing (ISBCC’15), Procedia Computer Science 50, 10-15, 2015.
[20] STARNER, T., WEAVER, J., PENTLAND, A., “Real-time american sign language recognition using desk and wearable computer based video”, IEEE Transactions on Pattern Analysis and Machine Intelligence. 20(12), 1371-1375. 1998.
[21] https://www.codeproject.com/Articles/196168/Contour-Analysis-for-Image-Recognition-in-C (erişim tarihi 13.05.2018)
[22] WILLE, R., Restructuring lattice theory: An approach based on hierarchies on concepts, 445-470, ed. (I. Rival), in Ordered Sets, Dordrecht-Boston: D. Reidel Publishing Company, 1982.