Controlling of Virtual Mirror with Voice and Hand Motion

Although online shopping has been very popular recently, customers have some disadvantages such as the impossibility to try the product before buying. Owing to the recent technological advances, people can try on products to be purchased on a virtual mirror. Inspired by this situation, it has been developed a system which users can try handbags on a monitor and see different poses of handbags as if they wear them. Also, users can see different colors of handbags by saying the color that they want to see to the system. In this application, it is taken advantage of Kinect Sensor to follow user motions and hand skeleton joints which is placed on the bag. The voice command recognition is used which is based on correlation method.

El Hareketleri ve Ses ile Sanal Ayna Kontrolü

Online alışveriş son zamanlarda popüler olmasına rağmen bu alışveriş türünde ürün deneyememe gibi bazı dezavantajlar mevcuttur. Son zamanlardaki teknolojik gelişmeler ile birlikte sanal aynalar aracılığı ile insanlar satın almak istedikleri ürünleri deneyebilmektedir. Bu yaklaşımdan esinlenilerek, kullanıcıların el çantalarını sanal olarak deneyebildikleri ve farklı pozlarını bir ekran üzerinde görebildikleri bir sistem geliştirilmiştir. Ayrıca, kullanıcılar ses komutları ile çantaların farklı renklerini de deneyebilmektedir. Bu çalışmada, kullanıcı hareketlerinin ve el iskelet bağantılarının takibi için Kinect Sensor kullanılmıştır. Ses komutlarını tanımak için korelasyon yönteminden faydalanılmıştır.

___

[1] Weiyuan Liu, “Natural user interface- next mainstream product user interface,” in 2010 IEEE 11th International Conference on Computer-Aided Industrial Design & Conceptual Design 1, 2010, pp. 203–205.

[2] K.-L. Kramer, User Experience in the Age of Sustainability. Science Direct, 2012.

[3] Y. El Miedany, Rheumatology Teaching. Springer.

[4] M. A. M. Echeverría, P. C. Santana-Mancilla, H. F. Q. Carrillo, and E. A. F. Enciso, “Natural User Interfaces to Teach Math on Higher Education,” Procedia - Soc. Behav. Sci., vol. 106, pp. 1883–1889, Dec. 2013.

[5] F. Cassola, L. Morgado, F. de Carvalho, H. Paredes, B. Fonseca, and P. Martins, “Online-Gym: A 3D Virtual Gymnasium Using Kinect Interaction,” Procedia Technol., vol. 13, pp. 130–138, 2014.

[6] H. Cho and N. Schwarz, “I like those glasses on you, but not in the mirror: Fluency, preference, and virtual mirrors,” J. Consum. Psychol., vol. 20, no. 4, pp. 471–475, Oct. 2010.

[7] P. Eisert, P. Fechteler, and J. Rurainsky, “3-D Tracking of shoes for Virtual Mirror applications,” in 2008 IEEE Conference on Computer Vision and Pattern Recognition, 2008, pp. 1–6.

[8] T. Blum, V. Kleeberger, C. Bichlmeier, and N. Navab, “mirracle: An augmented reality magic mirror system for anatomy education,” in 2012 IEEE Virtual Reality (VR), 2012, pp. 115–116.

[9] S. Casas, C. Portalés, L. Vera, and J. V. Riera, “Virtual and Augmented Reality Mirrors for Mental Health Treatment,” pp. 95–117.

[10] R. Nikam, C. Thete, S. Sonawane, S. Gadhave, and D. S. Thosar, “Virtual Shopping Using Kinect,” in National Level Conference On "Advanced Computing and Data Processing.

[11] J. Ishikiriyama and K. Suzuki, “An interactive virtual mirror to support makeup for visually impaired persons,” in 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2017, pp. 1393–1398.

[12] G. Yolcu, S. Kazan, and C. Oz, “Real Time Virtual Mirror Using Kinect,” , Balkan Journal vol. 2, no. 2, 2014.

[13] G. Yolcu, C. Oz, and T. Tasci, “Developing and Establishing a Painting Program Controlled by Hand Motions Using Kinect,” 2Nd Int. Symp. Innov. Technol. Eng. Sci., pp. 1–7, 2014.

[14] M. R. Alam, "Isolated Speech Recognition System Based on Cross-Correlation Technique", ResearchGate, 09 September 2016.

[15] A. Suzen, “Home automation for disabilities using kinect technology,” Suleyman Demirel, 2012.

[16] P. Cottone, G. Maida, and M. Morana, “User Activity Recognition via Kinect in an Ambient Intelligence Scenario,” IERI Procedia, vol. 7, pp. 49–54, 2014.

[17] D. Tuvshinjargal, B. Dorj, and D. J. Lee, “Hybrid Motion Planning Method for Autonomous Robots Using Kinect Based Sensor Fusion and Virtual Plane Approach in Dynamic Environments,” J. Sensors, vol. 2015, pp. 1–13, 2015.