El Hareketlerinden İşaret Dilini Algılayıp Yazıya Dönüştürme

Tasarlanacak işaret dili tanıma projesinin güncel teknolojiler kullanılarak optimize biçimde gerçeklenmesi amaçlanmıştır. Projenin makine öğrenmesi bölümü Keras ve Sklearn kullanılarak TensorFlow üzerinden yapılacaktır. TensorFlow, ilerleyen aşamalarda projeyi mobil bir ortama taşıma ihtimali göz önünde bulundurularak seçilmiştir. Kullanılacak nesne tanıma yöntemi MediaPipe Holistic olarak seçilmiştir.

Anahtar Kelimeler:

Gerçek Zamanlı İşaret Dili Tanıma, Makine Öğrenmesi, Derin Öğrenme, Nesne Tanıma, Bilgisayarla Görü

Detecting Sign Language from Hand Gestures and Translating it into Text

The sign language recognition project to be designed is aimed to be realized in an optimized way using up-to-date technologies. The machine learning part of the project will be done over TensorFlow using Keras and Sklearn. TensorFlow was chosen considering the possibility of moving the project to a mobile environment in the future. The object recognition method to be used was chosen as MediaPipe Holistic.

Keywords:

Real Time Sign Language Recognition, Machine Learning, Deep Learning, Object Recognition, Computer Vision,

PDF

___

Ko, S.-K., Kim, C. J., Jung, H., & Cho, C. (2019). Neural Sign Language Translation Based on Human Keypoint Estimation. Applied Sciences, 9(13), 2683. doi:10.3390/app9132683
RWTH-PHOENIX-2014-T veri seti, https://www-i6.informatik.rwth-aachen.de/~koller/RWTH-PHOENIX-2014-T/
Necati Cihan Camgoz, Simon Hadfield, Oscar Koller, Hermann Ney, and Richard Bowden. Neural Sign Language Translation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
Konstantinidis, D., Dimitropoulos, K., & Daras, P. (2018). Sıgn Language Recognıtıon Based On Hand And Body Skeletal Data. 2018- 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON). doi:10.1109/3dtv.2018.8478467
Hosain, A. A., Santhalingam, P. S., Pathak, P., Rangwala, H., & Kosecka, J. (2020). FineHand: Learning Hand Shapes for American Sign Language Recognition. 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020). doi:10.1109/fg47880.2020.00062
Zhang, Z., Pu, J., Zhuang, L., Zhou, W., & Li, H. (2019). Continuous Sign Language Recognition via Reinforcement Learning. 2019 IEEE International Conference on Image Processing (ICIP). doi:10.1109/icip.2019.8802972
Pytorch vs Tensorflow 2021, https://towardsdatascience.com/pytorch-vs-tensorflow-2021-d403504d7bc3