Immersive Virtual Reality Evaluation

In this work, the main objective is to understand how researchers and practitioners can perform, in VR, usability evaluation of prototypes of novel forms of interaction that are at the idea or early concept stage. There is evidence that Science Fiction can inspire the research design process [3]. Technologies that were first imagined in sci-fi media, such as video conferencing, multi-touch, gesture interaction, eventually found their way in the consumer market. However, once actual implementations of these technologies became available, usability problems were found. For example, “gorilla-arm” for gesture interaction [2]. It’s valuable to identify, at an early stage, the respective usability issues of devices based on new technologies. Virtual reality (VR) is actively being used in industries to support decision making and enable innovation [1]. For example, VR is used to investigate the visibility, ergonomics, packaging, aesthetic quality as well as applied in abstract data visualization. Meanwhile, VR also shows its suitability as a research methodology in empirical studies. Empirical studies are a cornerstone of HCI research. Technical development enables new study methods, and the study method significantly affects the study result [4]. Previous work [4] also shows VR and real-life in-situ studies provide similar high ratings for usability, attractiveness, pragmatic and hedonic qualities. Base on those results, in this project, we are going to develop a novel methodology that is especially suited for performing Usability Evaluation in VR. We focus on the HMD based evaluations and we shall investigate potential answers to the following questions: RQ1: What is the best way to perform a usability evaluation in VR? Various methodologies can be applied in conventional usability evaluation scenarios. We ask ourselves how different established usability evaluation methods might affect the results of VR evaluation. Which conventional methods can still be reliably used in the VR study? Do the usability issues found in the VR study also help improve the physical prototype and vice versa? RQ2: How does the fidelity of the VR prototype affect the number and the severity of the usability issues found? Which qualities of the implementation fidelity have the most impact? I.e. the graphic appearance through the used material and texture or different types of physical properties. Does the interaction capabilities of the prototype also affect results? Does the use of higher fidelity physical prototypes help to identify more usability problems than through the use of passive physical prototypes or controller-based approaches? To answer these questions, a combination of both quantitative and qualitative studies will be performed. Through the course of the project, we aim to develop and evaluate a framework for performing usability evaluations in VR. [1] BERG, L.P. and VANCE, J.M., 2017. Industry use of virtual reality in product design and manufacturing: a survey. Virtual Reality, 21(1):1–17. [2] Boring, S., Jurmu, M. and Butz, A., 2009, November. Scroll, tilt or move it: using mobile phones to continuously control pointers on large public displays. In Proceedings of the 21st Annual Conference of the Australian Computer-Human Interaction Special Interest Group: Design: Open 24/7 (pp. 161-168). ACM. [3] Mubin, O., Obaid, M., Jordan, P., Alves-Oliveria, P., Eriksson, T., Barendregt, W., Sjolle, D., Fjeld, M., Simoff, S. and Billinghurst, M., 2016, November. Towards an agenda for Sci-Fi inspired HCI research. In Proceedings of the 13th International Conference on Advances in Computer Entertainment Technology (p. 10). ACM. [4] VOIT, A., MAYER, S., SCHWIND, V., and HENZE, N., 2019. Online, VR, AR, Lab, and In-Situ: Comparison of Research Methods to Evaluate Smart Artifacts. In To appear in Proc. of CHI 2019.