Türkan KARAKUŞ YILMAZ, Umut DURAK, Kürşat ÇAGILTAY, Halit OĞUZTÜZÜN

Usability evaluation of a web-based ontology browser: the case of TSONT

As the use of ontologies expands, their visualization is becoming increasingly important. In this study, anontology browser for visualizing the Trajectory Simulation ONTology (TSONT) was evaluated in terms of usability byconsidering its subdimensions, which are effectiveness, efficiency, and user satisfaction. The methodology employed inthis study for evaluating an ontology browser is reported along with the results of the evaluation. The TSONT browseris a tree-type ontology browser created to allow developers to visualize TSONT. Six flight simulation programmers withat least one year of experience participated in the study. The participants were given usability tasks and their voicesand eye movements were recorded using a sound recorder and eye-tracker, respectively. The results not only showedthat guidance and terminology influence the efficiency, effectiveness, and user satisfaction of the ontology browser, butthey also revealed important insights into the requisites of the general usability of ontologies, even in simple text-basedinterfaces.

PDF

___

[1] Gruber TR. Towards principles for the design of ontologies used for knowledge sharing. Int J Hum-Comput St 1995; 43: 907-928.
[2] Uschold M, King M. Towards a methodology for building ontologies. In: Workshop on Basic Ontological Issues in Knowledge Sharing. Montreal, Quebec, Canada, 1995.
[3] Durak U, Schmidt A, Pawletta T. Ontology for objective flight simulator fidelity evaluation. Simulation Notes Europe, 2014; 24: 69-78.
[4] Mizoguchi R, Ikeda M. Towards ontology engineering. Journal-Japanese Society for Artificial Intelligence. 1998; 13: 9-10.
[5] Durak U, Oguztuzun H, Ider SK. An ontology for trajectory simulation. In: Proceedings of the 2006 Winter Simulation Conference. pp. 1160-1167.
[6] Durak U, Oguztuzun H, Ider SK. Ontology based domain engineering for trajectory simulation reuse. Int J Softw Eng Know 2009; 19.
[7] Tudorache T, Nyulas C, Noy NF, Musen MA. WebProt´eg´e: A collaborative ontology editor and knowledge acquisition tool for the web. Semant Web 2013; 4: 89-99.
[8] Lohmann S, Link V, Marbach E, Negru S. WebVOWL: Web-based visualization of ontologies. In: Lambrix P et al., editors. Knowledge Engineering and Knowledge Management. EKAW 2014. Lecture Notes in Computer Science, vol 8982. Cham, Switzerland: Springer, 2015. pp. 154-158.
[9] Peroni S, Shotton D, Vitali F. Tools for the automatic generation of ontology documentation: A task-based evaluation. Int J Semant Web Inform Syst (IJSWIS) 2013; 9: 21-44.
[10] Kalyanpur A, Parsia B, Sirin E, Grau BC, Hendler J. Swoop: A web ontology editing browser. J Web Semant 2006; 4: 144-153.
[11] Gangemi A, Presutti V, Blomqvist E. The computational ontology perspective: Design patterns for web ontologies. In: Sartor G, Casanovas P, Biasiotti M, Fernandez-Barrera M, editors. Approaches to Legal Ontologies: Theories, Domains, Methodologies. Law, Governance and Technology Series. Dordrecht, the Netherlands: Springer, 2011. pp. 201-217.
[12] Herv´as R, Bravo J, Fontecha J. A context model based on ontological languages: A proposal for information visualization. J Univers Comput Sci 2010; 16: 1539-1555.
[13] Uschold M, Gruninger M. Ontologies: Principles, methods and applications. Knowl Eng Rev 1996; 11: 93-136.
[14] Peroni S, Motta E, d’Aquin M. Identifying key concepts in an ontology, through the integration of cognitive principles with statistical and topological measures. In: Proceedings of ASWC 2008; 8–11 December 2008; Bangkok, Thailand: Berlin, Germany: Springer. pp. 242-256.
[15] Haghighi PD, Burstein F, Zaslavsky A, Arbon P. Development and evaluation of ontology for intelligent decision support in medical emergency management for mass gatherings. Decis Support Syst 2013; 54: 1192-1204.
[16] Baader F, Horrocks I, Sattler U. Description Logics. In: Staab S, Studer R, editors. Handbook on Ontologies. Berlin, Germany: Springer, 2013. pp. 3-28.
[17] Gomes-Perez A. Ontology evaluation. In: Staab S, Studer R, editors. Handbook on Ontologies. Berlin, Germany: Springer, 2013. pp. 251-274.
[18] Gomez-P´erez A. Evaluation of ontologies. Int J Intell Syst 2001; 16: 391-409.
[19] Giboin A, Gandon F, Corby O, Dieng R. Assessment of ontology-based tools: a step towards systemizing the scenario approach. In: EON2002: Evaluation of Ontology-Based Tools Workshop at the 13th International Conference on Knowledge Engineering and Knowledge Management EKAW 2002; 30 September 2002. pp. 63-73.
[20] ISO 9240-11 (1998). Ergonomic requirements for office work with visual display terminals (vdts) part 11: Guidance on usability.
[21] Albert W, Tullis T. Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics. Waltham, MA, USA: Elsevier, 2013.
[22] Lee Y, Kozar KA. Understanding of website usability: specifying and measuring constructs and their relationships. Decis Support Syst 2012; 52: 450-463.
[23] Shiffman RN, Liaw Y, Brandt CA, Corb GJ. Computer-based guideline implementation systems. J Am Med Inform Assn 1999; 6: 104-114.
[24] Gomez-Perez A, Juristo N, Pazos J. Evaluation and assessment of knowledge sharing technology. In Mars NJ, editor. Towards Very Large Knowledge Bases – Knowledge Building and Knowledge Sharing Technology. Amsterdam, the Netherlands: IOS Press, 1995. pp. 289-296.
[25] Yu J, Thom JA, Tam A. Ontology evaluation using Wikipedia categories for browsing. In Proceedings of CIKM 2007, 6–10 November 2007; Lisbon, Portugal: ACM. pp. 223-232.
[26] Shneiderman B. Designing the User Interface. Boston, MA, USA: Addison-Wesley, 1996.
[27] Forsell C, Cooper MC. An introduction and guide to evaluation of visualization techniques through user studies. In: Huang W, editor. Handbook of Human Centric Visualization. New York, NY, USA: Springer, 2012. pp. 285-314.
[28] Freitas CMDS, Pimenta MS, Scapin DL. User centered evaluation of information visualization techniques: Making the HCI InfoVis connection explicit. In: Huang W, editor. Handbook of Human Centric Visualization. New York, NY: Springer, 2012. pp. 315-336.
[29] Fu B, Noy NF, Storey MA. Eye tracking the user experience – an evaluation of ontology visualization techniques. Semant Web 2015; PrePrint, pp. 1-19.
[30] Stojanovic L, Motik B. Ontology evolution within ontology editors. In: Proceedings of the OntoWeb-SIG3 Workshop at the 13th International Conference on Knowledge Engineering and Knowledge Management, 2002. pp. 53-62.
[31] Motta E, Mulholland P, Peroni S, d’Aquin M, Gomez-Perez J M, Mendez V, Zablith F. A novel approach to visualizing and navigating ontologies. In: Proceedings of ISWC 2011, 23–27 October 2011; Bonn, Germany: Springer-Verlag. pp. 470-486.
[32] Katifori A, Halatsis C, Lepouras G, Vassilakis C, Giannopoulou E. Ontology visualization methods—a survey. ACM Computing Surveys 2007; 39: 10.
[33] Lewis JR. Sample sizes for usability studies: additional considerations. Hum Factors 1994; 36: 368-378.
[34] Zapata C, Pow-Sang JA. Sample size in a heuristic evaluation of usability. In: Zapata CM, Gonz´alez G, Manjarr´es R, Vargas FA, Ar´evalo W, editors. Software Engineering: Methods, Modeling, and Teaching. Lima, Peru: Pontificia Universidad Cat´olica del Per´u, 2012. pp. 37-36.
[35] Turner CW, Lewis JR, Nielsen J. Determining usability test sample size. International Encyclopedia of Ergonomics and Human Factors. 3, 2006. pp. 3084-3088.
[36] Pretorius MC, Calitz AP, van Greunen D. The added value of eye tracking in the usability evaluation of a network management tool. In: Proceedings of SAICSIT 2005, 20–22 September 2005. White River, South Africa: SAICSIT. pp. 1-10.
[37] Coltekin A, Heil B, Garlandini S, Fabrikant SI. Evaluating the effectiveness of interactive map interface designs: A case study integrating usability metrics with eye-movement analysis. Cartogr Geogr Inf Sc 2009; 36: 5-17.
[38] Conesa J, Storey VC, Sugumaran V. Usability of upper level ontologies: The case of ResearchCyc. Data Knowl Eng 2010; 69: 343-356.
[39] Braun S, Schmidt A, Walter A, Zacharias V. The ontology maturing approach for collaborative and work integrated ontology development. Evaluation, results and future directions. In: Proceedings of ESOE 2007, 12 November 2007; Busan, Republic of Korea. pp. 5-18.
[40] Carvalho RM, Berrios DC, Wolfe SR, Williams J. Ontology development and evolution in the accident investigation domain. In: Proceedings of the IEEE Aerospace Conference, Big Sky, MT, USA. Piscataway, NJ, USA: IEEE, 2005.