Mehmet BERKMAN, Güven ÇATAK, Mıstık EREMEKTAR

Bir Bulmaca Oyununda VR ve Masaüstü Oyun Kullanıcı Deneyimi Kıyaslaması: “Keep Talking and Nobody Explodes”

Çağdaş oyun üretim süreci Entegre Geliştirme Ortamı (IDE) uygulamalarına dayandığından, geliştiricilerin oyunlarının hem VR hem de masaüstü platformları için birden çok sürümünü oluşturmaları daha kolaydır. Bu drum, araştırmacılara da, sanal gerçeklik uygulamalarının kullanıcı deneyimini karşılaştırmalı olarak değerlendirme fırsatı sağlamaktadır. Bu çalışmada, oyun başarısı ve oynanış süresi objektif ölçümlerinin yanında, oyun kullanıcısı deneyimini değerlendirmek için birçok öznel ölçümü de kullanarak bir bulmaca oyununu masaüstü ve VR'yi karşılaştırması yaparak değerlendirdik. GUESS ölçeğinin (Oyun Kullanıcı Deneyimi Memnuniyeti Ölçeği) seçilen boyutlarına ek olarak, mevcudiyeti ölçmek için MEC-SPQ (Ölçüm Etkileri Koşulları - Mekansal Mevcudiyet Anketi) kullanıldı. Ayrıca, VR ve masaüstü oyun ortamlarında yürütülen aynı görevin algılanan görev karmaşıklığını kıyaslamak için NASA-TLX (NASA Görev Yük Endeksi) ölçeğinden yararlanıldı. Sonuçlar, VR ve masaüstü ortamı karşılaştırıldığında, oyuncu performansının objektif ölçümlerinde önemli bir fark olmadığını ortaya koydu. Oyun Kullanıcı Deneyimi Memnuniyeti Ölçeği, VR ve masaüstü deneyimlerinin ortalama puanları arasında anlamlı bir fark görülmedi. MEC-SPQ'nun mekansal mevcudiyetle ilgili boyutları olan Olası Eylemler ve Öz-Konum boyutları için VR ortamında anlamlı biçimde daha yüksek skorlar gözlendi. NASA-TLX ağırlıklı skorlar fiziksel yük boyutunda VR ve hayal kırıklığı boyutunda masaüstü için anlamlı olarak daha yüksektir. Sonuçlar, VR'de deneyimlenen bulmaca tabanlı bir oyunun, oyun kullanıcı deneyimi açısından daha yüksek bir memnuniyet düzeyine yol açmadığını, ancak mekansal mevcudiyet duygusunu tetiklediğini göstermektedir. Farklı kontrol şemaları nedeniyle, oyuncular kafaya takılan ekran tabanlı oyunun daha fazla fiziksel görev yükü gerektirdiğini düşünmektedirler. Bununla birlikte, oyun süresi ve oyun başarı oranı önemli ölçüde farklı değildir. Masaüstü oyunlarındaki başarısızlık, deneyim oyunculara daha tanıdık geldiği için daha yüksek hayal kırıklığına yol açmıştır. Sonuçlar daha önceki çalışmalarla kısmen uyumlu olduğu için, farklı sürükleyici teknolojilerin oyun kullanıcı deneyimi üzerindeki etkisine ilişkin kesin bir sonuç çıkarmak mümkün değildir. Oyun türünden ziyade oyun tasarımı bileşenlerine odaklanan daha tutarlı bir metodoloji yoluyla daha fazla çalışmaya ihtiyaç vardır.

Comparison of VR and Desktop Game User Experience in a Puzzle Game: “Keep Talking and Nobody Explodes”

Since the contemporary game production process is based on the Integrated Development Environment (IDE) applications, it is easier for developers to create multiple versions of their game for both VR and desktop platforms. This provided a great opportunity for researchers to conduct comparative studies to explore the user experience of the relatively novel virtual reality applications In this study, we evaluated a puzzle game through a within-subjects experiment design using objective measures of game success and gameplay duration, as well as plenty of subjective measures in order to assess game user experience, comparing desktop and VR. In addition to selected dimensions of GUESS (Game User Experience Satisfaction Scale), we employed MEC-SPQ (Measurement Effects Conditions - Spatial Presence Questionnaire) to measure presence. Furthermore, we employed NASA-TLX (NASA Task Load Index) to compare the perceived task complexity of the same task executed in VR and desktop gaming environments. Results revealed that there is not a significant difference in objective measures of player performance, comparing the VR and desktop gameplay. The Game User Experience Satisfaction Scale did not reveal any significant difference between the mean scores of VR and desktop experiences. The spatial presence related dimensions of MEC-SPQ revealed significantly higher scores of VR, for Possible Actions and Self Location dimensions. NASA-TLX weighted scores were significantly higher for VR in physical load and for desktop in frustration. Our results show that a puzzle-based game experienced in VR does not lead to a higher level of satisfaction in terms of game user experience but triggers a sense of spatial presence. Due to the different control schemes, players perceive that HMD based gameplay demands more physical task load. However, the gameplay duration and game success rate are not significantly different. The failure in desktop gameplay might have led to higher frustration, since the experience seems more familiar to players. Since the results are partially concordant with previous studies, it is not possible to make a strict conclusion on the effect caused by different immersive technologies on game user experience. Further studies are required through a more consistent methodology with a focus on game design components rather than game genre.

Keywords:

Virtual Reality, Game User Experience, Puzzle Game Task Load, Presence, Video Games,

PDF

___

Abeele, V. Vanden, Spiel, K., Nacke, L., Johnson, D., & Gerling, K. (2020). Development and validation of the player experience inventory: A scale to measure player experiences at the level of functional and psychosocial consequences. International Journal of Human Computer Studies. https://doi.org/10.1016/j.ijhcs.2019.102370

Aksayim, A., & Berkman, M. İ. (2020). Effect of Physical Activity on VR Experience: An Experimental Study. In S. Richir (Ed.), Laval Virtual ConVRgence (VRIC) Virtual Reality International Conference - VRIC 2020. Retrieved from https://ijvr.eu/article/view/3316

Anton, D., Kurillo, G., & Bajcsy, R. (2018). User experience and interaction performance in 2D/3D telecollaboration. Future Generation Computer Systems. https://doi.org/10.1016/j.future.2017.12.055

Aoki, H., Oman, C. M., Buckland, D. A., & Natapoff, A. (2008). Desktop-VR system for preflight 3D navigation training. Acta Astronautica. https://doi.org/10.1016/j.actaastro.2007.11.001

Arns, L., Cook, D., & Cruz-Neira, C. (1999). Benefits of statistical visualization in an immersive environment. Proceedings - Virtual Reality Annual International Symposium.

Bacim, F., Ragan, E., Scerbo, S., Polys, N. F., Setareh, M., & Jones, B. D. (2013). The effects of display fidelity, visual complexity, and task scope on spatial understanding of 3D graphs. Proceedings - Graphics Interface.

Berkman, M. İ., Bostan, B., & Yalçın, B. (2020). Controllers in VR Game User Experience: Perceived User Performance on a VR Puzzle Game. In Contemporary Topics in Computer Graphics and Games: Selected Papers from the Eurasia Graphics Conference Series.

Berkman, M. I., & Akan, E. (2019). Presence and Immersion in Virtual Reality. In Encyclopedia of Computer Graphics and Games (pp. 1–10). https://doi.org/10.1007/978-3-319-08234-9_162-1

Brooke, J. (1996). SUS-A quick and dirty usability scale. Usability Evaluation in Industry.

Buttussi, F., & Chittaro, L. (2018). Effects of Different Types of Virtual Reality Display on Presence and Learning in a Safety Training Scenario. IEEE Transactions on Visualization and Computer Graphics. https://doi.org/10.1109/TVCG.2017.2653117

Carroll, M., Osborne, E., & Yildirim, C. (2019). Effects of VR gaming and game genre on player experience. 2019 IEEE Games, Entertainment, Media Conference, GEM 2019. https://doi.org/10.1109/GEM.2019.8811554 Chirico, A., Cipresso, P., Yaden, D. B., Biassoni, F., Riva, G., & Gaggioli, A. (2017). Effectiveness of Immersive Videos in Inducing Awe: An Experimental Study. Scientific Reports. https://doi.org/10.1038/s41598-017-01242-0

Chirico, A., Ferrise, F., Cordella, L., & Gaggioli, A. (2018). Designing awe in virtual reality: An experimental study. Frontiers in Psychology, 8(JAN), 2351. https://doi.org/10.3389/fpsyg.2017.02351

Christensen, J. V., Mathiesen, M., Poulsen, J. H., Ustrup, E. E., & Kraus, M. (2018). Player experience in a VR and non-VR multiplayer game. ACM International Conference Proceeding Series. https://doi.org/10.1145/3234253.3234297

Clarke, R. I., Lee, J. H., & Clark, N. (2017). Why Video Game Genres Fail: A Classificatory Analysis. Games and Culture. https://doi.org/10.1177/1555412015591900

Elmqvist, N., Tudoreanu, M. E., & Tsigas, P. (2008). Evaluating motion constraints for 3D wayfinding in immersive and desktop virtual environments. Proceeding of the Twenty-Sixth Annual CHI Conference on Human Factors in Computing Systems - CHI ’08, 1769. https://doi.org/10.1145/1357054.1357330

Hart, S. G. (2006). NASA-task Load Index (NASA-TLX). Proceedings of the Human Factors and Ergonomics Society.

Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. Advances in Psychology. https://doi.org/10.1016/S0166-4115(08)62386-9

Hartmann, T., Wirth, W., Schramm, H., Klimmt, C., Vorderer, P., Gysbers, A., … Sacau, A. M. (2015). The spatial presence experience scale (SPES): A short self-report measure for diverse media settings. Journal of Media Psychology, 28(1), 1–15. https://doi.org/10.1027/1864-1105/a000137

Hunicke, R., Leblanc, M., & Zubek, R. (2004). MDA: A formal approach to game design and game research. AAAI Workshop - Technical Report.

IJsselsteijn, W. A., de Kort, Y. A. W., & Poels, K. (2007). Game Experience Questionnaire: development of a self-report measure to assess the psychological impact of digital games — Eindhoven University of Technology research portal. PRESENCE 2007.

Johnsen, K., & Lok, B. (2008). An evaluation of immersive displays for virtual human experiences. Proceedings - IEEE Virtual Reality. https://doi.org/10.1109/VR.2008.4480764

Kennedy, R. S., Lane, N. E., Berbaum, K. S., & Lilienthal, M. G. (1993). Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness. The International Journal of Aviation Psychology, 3(3), 203–220. https://doi.org/10.1207/s15327108ijap0303_3 Kim, D., & Ko, Y. J. (2019). The impact of virtual reality (VR) technology on sport spectators’ flow experience and satisfaction. Computers in Human Behavior, 93, 346–356. https://doi.org/10.1016/j.chb.2018.12.040

Kim, K., Rosenthal, M. Z., Zielinski, D. J., & Brady, R. (2014). Effects of virtual environment platforms on emotional responses. Computer Methods and Programs in Biomedicine. https://doi.org/10.1016/j.cmpb.2013.12.024

Kober, S. E., Kurzmann, J., & Neuper, C. (2012). Cortical correlate of spatial presence in 2D and 3D interactive virtual reality: An EEG study. International Journal of Psychophysiology, 83(3), 365–374. https://doi.org/10.1016/j.ijpsycho.2011.12.003

Kulshreshth, A., Schild, J., & LaViola, J. J. (2012). Evaluating user performance in 3D stereo and motion enabled video games. Foundations of Digital Games 2012, FDG 2012 - Conference Program. https://doi.org/10.1145/2282338.2282350

Laha, B., Sensharma, K., Schiffbauer, J. D., & Bowman, D. A. (2012). Effects of immersion on visual analysis of volume data. IEEE Transactions on Visualization and Computer Graphics. https://doi.org/10.1109/TVCG.2012.42

Laugwitz, B., Held, T., & Schrepp, M. (2008). Construction and evaluation of a user experience questionnaire. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). https://doi.org/10.1007/978-3-540-89350-9-6

Lessiter, J., Freeman, J., Keogh, E., & Davidoff, J. (2001). A cross-media presence questionnaire: The ITC-sense of presence inventory. Presence: Teleoperators and Virtual Environments, 10(3), 282–297. https://doi.org/10.1162/105474601300343612

Li, H., & Giudice, N. A. (2013). The effects of immersion and body-based rotation on learning multi-level indoor virtual environments. Proceedings of the 5th ACM SIGSPATIAL International Workshop on Indoor Spatial Awareness, ISA 2013. https://doi.org/10.1145/2533810.2533811

Litwiller, T., & LaViola, J. J. (2011). Evaluating the benefits of 3d stereo in modern video games. Conference on Human Factors in Computing Systems - Proceedings. https://doi.org/10.1145/1978942.1979286

Lugrin, J.-L., Cavazza, M., Charles, F., Le Renard, M., Freeman, J., & Lessiter, J. (2013). Immersive FPS games. Proceedings of the 2013 ACM International Workshop on Immersive Media Experiences - ImmersiveMe ’13, 7–12. https://doi.org/10.1145/2512142.2512146

Mania, K., & Chalmers, A. (2001). The effects of levels of immersion on memory and presence in virtual environments: A reality centered approach. Cyberpsychology and Behavior. https://doi.org/10.1089/109493101300117938 Mania, K., & Chalmers, A. (2002). The Effects of Levels of Immersion on Memory and Presence in Virtual Environments: A Reality Centered Approach. CyberPsychology & Behavior, 4(2), 247–264. https://doi.org/10.1089/109493101300117938

Martel, E., Su, F., Gerroir, J., Hassan, A., Girouard, A., & Muldner, K. (2015). Diving Head-First into Virtual Reality—Evaluating HMD Control Schemes for VR Games. Proceedings of the 10th International Conference on the Foundations of Digital Games - FDG 2015. Retrieved from https://pdfs.semanticscholar.org/2c7d/b9069822f2599cdb466d51f17b28fd2baad6.pdf?_ga=2.187769836.550852407.1589226699-1224180204.1587306111

McMahan, R. P., Bowman, D. A., Zielinski, D. J., & Brady, R. B. (2012). Evaluating display fidelity and interaction fidelity in a virtual reality game. IEEE Transactions on Visualization and Computer Graphics. https://doi.org/10.1109/TVCG.2012.43

Mizell, D. W., Jones, S. P., Slater, M., & Spanlang, B. (2002). Comparing immersive virtual reality with other display modes for visualizing complex 3D geometry. In University College London, ….

Monteiro, D., Liang, H. N., Xu, W., Brucker, M., Nanjappan, V., & Yue, Y. (2018). Evaluating enjoyment, presence, and emulator sickness in VR games based on first- and third-person viewing perspectives. Computer Animation and Virtual Worlds. https://doi.org/10.1002/cav.1830

Moroney, W. F., Biers, D. W., Eggemeier, F. T., & Mitchell, J. A. (2003). A comparison of two scoring procedures with the NASA task load index in a simulated flight task. https://doi.org/10.1109/naecon.1992.220513

Pallavicini, F., Ferrari, A., Zini, A., Garcea, G., Zanacchi, A., Barone, G., & Mantovani, F. (2018). What distinguishes a traditional gaming experience from one in virtual reality? An exploratory study. Advances in Intelligent Systems and Computing. https://doi.org/10.1007/978-3-319-60639-2_23

Pallavicini, F., Pepe, A., & Minissi, M. E. (2019). Gaming in Virtual Reality: What Changes in Terms of Usability, Emotional Response and Sense of Presence Compared to Non-Immersive Video Games? Simulation & Gaming, 50(2), 136–159. https://doi.org/10.1177/1046878119831420

Patrick, E., Cosgrove, D., Slavkovic, A., Rode, J. A., Verratti, T., & Chiselko, G. (2000). Using a large projection screen as an alternative to head-mounted displays for virtual environments. Conference on Human Factors in Computing Systems - Proceedings. https://doi.org/10.1145/332040.332479

Pausch, R., Proffitt, D., & Williams, G. (1997). Quantifying immersion in virtual reality. Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 1997. https://doi.org/10.1145/258734.258744

Pestaluky, A., Kane, B., & Fetter, B. (2015). Keep Talking and Nobody Explodes. Retrieved from https://www.keeptalkinggame.com/

Phan, M. H., Keebler, J. R., & Chaparro, B. S. (2016). The Development and Validation of the Game User Experience Satisfaction Scale (GUESS). Human Factors. https://doi.org/10.1177/0018720816669646

Qi, W., Taylor, R. M., Healey, C. G., & Martens, J. B. (2006). A comparison of immersive HMD, fish tank VR and fish tank with haptics displays for volume visualization. Proceedings - APGV 2006: Symposium on Applied Perception in Graphics and Visualization. https://doi.org/10.1145/1140491.1140502

Ragan, E. D., Kopper, R., Schuchardt, P., & Bowman, D. A. (2013). Studying the effects of stereo, head tracking, and field of regard on a small-scale spatial judgment task. IEEE Transactions on Visualization and Computer Graphics. https://doi.org/10.1109/TVCG.2012.163

Riddle, K. (2010). Remembering Past Media Use: Toward the Development of a Lifetime Television Exposure Scale. Communication Methods and Measures, 4(3), 241–255. https://doi.org/10.1080/19312458.2010.505500

Roettl, J., & Terlutter, R. (2018). The same video game in 2D, 3D or virtual reality – How does technology impact game evaluation and brand placements? PLoS ONE. https://doi.org/10.1371/journal.pone.0200724

Ruddle, R. A., Payne, S. J., & Jones, D. M. (1999). Navigating large-scale virtual environments: What differences occur between helmet-mounted and desk-top displays? Presence: Teleoperators and Virtual Environments. https://doi.org/10.1162/105474699566143

Ruddle, R. A., & Péruch, P. (2004). Effects of proprioceptive feedback and environmental characteristics on spatial learning in virtual environments. International Journal of Human Computer Studies. https://doi.org/10.1016/j.ijhcs.2003.10.001

Sauro, J., & Lewis, J. R. (2012). Quantifying the User Experience. In Quantifying the User Experience. https://doi.org/10.1016/C2010-0-65192-3

Schild, J., LaViola, J. J., & Masuch, M. (2012). Understanding user experience in stereoscopic 3D games. Conference on Human Factors in Computing Systems - Proceedings. https://doi.org/10.1145/2207676.2207690

Shelstad, W. J., Smith, D. C., & Chaparro, B. S. (2017). Gaming on the rift: How virtual reality affects game user satisfaction. Proceedings of the Human Factors and Ergonomics Society. https://doi.org/10.1177/1541931213602001

Slater, M., Linakis, V., Usoh, M., & Kooper, R. (1995). Immersion , Presence , and Performance in Virtual Environments : An Experiment with Tri-Dimensional Chess. Virtual Reality. https://doi.org/10.1.1.34.6594

Slobounov, S. M., Ray, W., Johnson, B., Slobounov, E., & Newell, K. M. (2015). Modulation of cortical activity in 2D versus 3D virtual reality environments: An EEG study. International Journal of Psychophysiology. https://doi.org/10.1016/j.ijpsycho.2014.11.003

Sousa Santos, B., Dias, P., Pimentel, A., Baggerman, J. W., Ferreira, C., Silva, S., & Madeira, J. (2009). Head-mounted display versus desktop for 3D navigation in virtual reality: A user study. Multimedia Tools and Applications. https://doi.org/10.1007/s11042-008-0223-2

Swindells, C., Po, B. A., Hajshirmohammadi, I., Corrie, B., Dill, J. C., Fisher, B. D., & Booth, K. S. (2004). Comparing CAVE, wall, and desktop displays for navigation and wayfinding in complex 3D models. Proceedings of Computer Graphics International Conference, CGI. https://doi.org/10.1109/CGI.2004.1309243

Tan, C. T., Leong, T. W., Shen, S., Dubravs, C., & Si, C. (2015). Exploring gameplay experiences on the oculus rift. CHI PLAY 2015 - Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play. https://doi.org/10.1145/2793107.2793117

Usoh, M., Catena, E., Arman, S., & Slater, M. (2000). Using presence questionnaires in reality. Presence: Teleoperators and Virtual Environments. https://doi.org/10.1162/105474600566989

Ventura, S., Brivio, E., Riva, G., & Baños, R. M. (2019). Immersive Versus Non-immersive Experience: Exploring the Feasibility of Memory Assessment Through 360° Technology. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2019.02509

Vorderer, P., Wirth, W., Gouveia, F. R., Biocca, F., Saari, T., Jäncke, F., … Jäncke, P. (2004). MEC Spatial Presence Questionnaire (MECSPQ): Short Documentation and Instructions for Application. Retrieved from https://www.researchgate.net/profile/Feliz_Gouveia/publication/318531435_MEC_spatial_presence_questionnaire_MEC-SPQ_Short_documentation_and_instructions_for_application/links/598041b5458515687b4fa65d/MEC-spatial-presence-questionnaire-MEC-SPQ-Short-docume

Waller, D., Hunt, E., & Knapp, D. (1998). The transfer of spatial knowledge in virtual environment training. Presence: Teleoperators and Virtual Environments. https://doi.org/10.1162/105474698565631

Weidner, F., Hoesch, A., Poeschl, S., & Broll, W. (2017). Comparing VR and non-VR driving simulations: An experimental user study. Proceedings - IEEE Virtual Reality. https://doi.org/10.1109/VR.2017.7892286 Wilson, R., & Mayhorn, C. B. (2019). Examining the Role of Video in Sports Media Viewing. Proceedings of the Human Factors and Ergonomics Society Annual Meeting. https://doi.org/10.1177/1071181319631424

Winn, W., Windschitl, M., Fruland, R., & Lee, Y. (2002). When Does Immersion ina a virtual Environment Helps Students construct Understanding? ICLS 2002.

Wirth, W., Vorderer, P., Hartmann, T., Klimmt, C., & Schramm, H. (2003). Constructing Presence : Towards a two-level model of the formation of Spatial Presence experiences. In Communication (Vol. 2003). Retrieved from https://www.researchgate.net/publication/318531733_Constructing_Presence_Towards_a_two-level_model_of_the_formation_of_Spatial_Presence

Witmer, B. G., & Singer, M. J. (1998). Measuring presence in virtual environments: A presence questionnaire. Presence: Teleoperators and Virtual Environments. https://doi.org/10.1162/105474698565686

Yildirim, Ç., Bostan, B., & Berkman, M. İ. (2019). Impact of different immersive techniques on the perceived sense of presence measured via subjective scales. Entertainment Computing. https://doi.org/10.1016/j.entcom.2019.100308

Yildirim, C., Carroll, M., Hufnal, D., Johnson, T., & Pericles, S. (2018). Video Game User Experience: To VR, or Not to VR? 2018 IEEE Games, Entertainment, Media Conference (GEM), 1–9. https://doi.org/10.1109/GEM.2018.8516542

Zanbaka, C. A., Lok, B. C., Babu, S. V., Ulinski, A. C., & Hodges, L. F. (2005). Comparison of path visualizations and cognitive measures relative to travel technique in a virtual environment. IEEE Transactions on Visualization and Computer Graphics, 11(6), 694–705. https://doi.org/10.1109/TVCG.2005.92