Humanity’s dream of autonomous mobility dates back almost a century. However, its fulfillment always continued to stay about 20 years away. With recent developments, it seems a tipping point in automotive history is reached. Autonomous vehicles (AVs; SAE levels 4 and 5) are increasingly deployed on test tracks and even tested on public roads all over the world – e.g., in the United States, in the United Kingdom, in China, or in Germany. Although most test operations are still conducted with limitations – e.g., in terms of low-speed limits, restricted areas like airports or university campuses, limited operation times, or special safety regulations like the continuous oversight of a safety/backup driver – it seems that the time gap to the ’always 20-year away utopia’ will diminish. Emerging commercial test operations with driverless vehicles indicate that the automotive industry might be able to fulfill that promise within the next decade.
Problem Statement
The adoption of AVs faces significant barriers, including issues of user acceptance, human factors, and the quality of the user experience. Human-machine interfaces (HMIs) have the potential to address these challenges and foster AV adoption, but their development requires suitable prototyping methods, especially during early design phases when AV capabilities and availability are still limited. Existing methods often fail to adequately consider the physical and social contexts that are critical to human-centered design.
To bridge this gap, there is a need for context-based prototyping approaches that enable the creation, evaluation, and optimization of novel HMI concepts. These approaches should integrate contextual factors, simulate real-world scenarios, and provide practical insights for improving human-AV interactions. Addressing this problem will empower researchers and practitioners to design HMIs that enhance user acceptance, improve interaction quality, and accelerate AV adoption.
Motivation, Research Questions, & Dissertation Outline
To enfold their potential for private and public transportation, AVs need to tackle technical and psychological challenges. Their adoption and their success highly depends on future users’ acceptance of the technology. Challenges that might hinder their acceptance need to be overcome first – starting already in early development phases of new systems. As a human driver will no longer be required onboard an AV, future users will solely interact with the AI-infused systems via digital interfaces. Consequently, the interaction and communication between humans and AVs and the resulting UX are key determinants of the technology’s success. Focusing on internal communication, we derived the following general research question (RQ) as a starting point for this thesis:
- RQ1. How might we design suitable HMIs for (internal) human-AV interaction that counteract acceptance challenges?
In order to facilitate public adoption, system designers need to solve human factors and acceptance challenges before AVs arrive for general public use. To do this, we need to simplify access to the context of driverless AV rides. Thus, we need to find ways to create flexible but standardizable environments while considering the potentially complex nature and dynamics of human-AV interaction in the design process. Consequently, we derived our second RQ:
- RQ2. How might we prototype and evaluate (internal) human-AV interactions early, considering their dynamic context?
To answer the laid out research questions, this work makes use of established HCI frameworks. First, we adopted a research in and through design approach as proposed by Dalsgaard. Furthermore, we embraced the human-centered design process as described in the ISO 9241-210 and combined it with the collaborative UX design methods conflated by Steimle and Wallach.
Empirical Research
The PhD project and dissertation is based on empirical user research and consecutive publications. Below is a concise overview of the papers and studies that were conducted. Please have a look at the dissertation and the respective papers for more details.
Background & Related Work
Abstract: Before AVs become broadly available, acceptance challenges such as trust and safety concerns must be overcome. In the development of appropriate HMIs that will tackle these challenges, physical and social context play essential roles. Contextual factors thus need to be considered in early prototyping stages. Based on a qualitative semi-systematic literature review and knowledge from our research, this paper elaborates on the value of context-based interface prototyping in the AV domain. It provides a comprehensive overview and a discussion of applicable methods, including physical lab-based prototyping (mock-up, ride simulation with virtual and mixed reality, and immersive video), social context simulation (actors, enactment, items and props, and sound), wizard-of-oz, and experimental vehicles. Finally, the paper discusses factors affecting the impact of prototyping and derives recommendations for the application of prototyping methods in future AV studies.
Lukas A. Flohr and Dieter P. Wallach. 2023. The Value of Context-Based Interface Prototyping for the Autonomous Vehicle Domain: A Method Overview. Multimodal Technologies and Interaction 7, 4 (2023), 1–17. https://doi.org/10.3390/mti7010004 Download PDF
Immersive Video-Based Simulation
Abstract: AVs develop rapidly, whereas appropriate methods for interface design and development for such driverless vehicles are still in their infancy. This paper presents a simple approach for context-based prototyping and evaluation of human-machine
interfaces for (shared) AVs in public transportation. It demonstrates how to set up a lightweight immersive video-based AV simulator using real-world video and audio footage captured in urban traffic. In two user studies (n1 = 9; n2 = 31) we investigate presence perception and simulator sickness to provide initial evidence for the suitability of this cost-effective method. Furthermore, with the intent to increase presence perception and technology acceptance, we combine the AV simulator with a human actor imitating a passenger that gets on and off a shared AV ride.
Lukas A. Flohr, Dominik Janetzko, Dieter P. Wallach, Sebastian C. Scholz, and Antonio Krüger. 2020. Context-Based Interface Prototyping and Evaluation for (Shared) Autonomous Vehicles Using a Lightweight Immersive Video-Based Simulator. In Proceedings of the 2020 ACM Designing Interactive Systems Conference (Eindhoven, Netherlands) (DIS ’20). Association for Computing Machinery, New York, NY, USA, 1379–1390. https://doi.org/10.1145/3357236.3395468 Download PDF
Wizard-of-Oz & Object Detection
Abstract: AVs face substantial challenges regarding acceptance and UX. Novel human-machine interfaces (HMIs) providing transparent system information could account for those and facilitate adoption. However, since the availability of AVs for early concept studies is limited, context-based interface prototyping is required. This paper demonstrates the prototype and wizard-of-oz-based on-road evaluation of a futuristic windshield HMI concept that visualizes real-time object detections via augmented reality (AR). In a mixed-methods within-subjects study (N = 30), participants assessed three early-stage concept variants to explore whether object detection visualization can counteract the aforementioned challenges. The findings confirm that transparent system feedback can increase understandability, perceived usefulness, and hedonic UX, but the amount and the timing of the provided information are crucial. The applied prototyping method proved suitable for investigating HMI concepts with real-time AR on urban roads. Based on a critical discussion, the paper concludes with design and prototyping recommendations
Lukas A. Flohr, Joseph S. Valiyaveettil, Antonio Krüger, and Dieter P. Wallach. 2023. Prototyping Autonomous Vehicle Windshields with AR and Real-Time Object Detection Visualization: An On-Road Wizard-of-Oz Study. Proceedings of the 2023 ACM Designing Interactive Systems Conference (Pittsburgh, PA, USA) (DIS ’23). Association for Computing Machinery, New York, NY, USA, 1–15. https://doi.org/10.1145/3563657.3596051 Download PDF
Mobile Interaction with Autonomous Mobility-on-Demand (AMoD)
Abstract: In autonomous mobility-on-demand (AMoD) systems, passengers will solely interact with autonomous vehicles via digital user interfaces (UIs). Hence, UIs are crucial for acceptance and user experience (UX). As a foundation for deriving empirically grounded design guidelines, we investigate two approaches for mobile interaction: chatbots and ‘classic’ graphical UIs (GUIs). We evaluated prototypes of both in expert studies (nGUI = 6; nChatbot = 5) and a between-subjects simulator user study (n = 34). The latter enabled participants to experience a complete AMoD journey. While both concepts receive good acceptance and positive UX evaluations, the GUI results in significantly higher attractiveness and user satisfaction ratings. A significant interaction effect reveals a higher intention to use the chatbot in scenarios with a change of plans, but the GUI in ‘happy path’ scenarios. Interview data and emotion curves support this effect. Balancing the concepts‘ advantages and disadvantages, we provide design recommendations and propose to use GUI-based mobile applications with integrated (text-based) conversational elements for future human-AMoD interaction.
Lukas A. Flohr, Sofie Kalinke, Antonio Krüger, and Dieter P. Wallach. 2021. Chat or Tap? – Comparing Chatbots with ‘Classic’ Graphical User Interfaces for Mobile Interaction with Autonomous Mobility-on-Demand Systems. In Proceedings of the 23rd International Conference on Mobile Human-Computer Interaction (Toulouse & Virtual, France) (MobileHCI ’21). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3447526.3472036 Download PDF
Information Demands at Day and Night
Abstract: Shared automated mobility-on-demand promises efficient, sustainable, and flexible transportation. Nevertheless, security concerns, resilience, and their mutual influence – especially at night – will likely be the most critical barriers to public adoption since passengers have to share rides with strangers without a human driver on board. Prior research points out that having information about fellow travelers could alleviate the concerns of passengers and we designed two user interface variants to investigate the role of this information in an exploratory within-subjects user study (N = 24). Participants experienced four automated day and night rides with varying personal information about co-passengers in a simulated environment. The results of the mixed-method study indicate that having information about other passengers (e.g., photo, gender, and name) positively affects user experience at night. In contrast, it is less necessary during the day. Considering participants‘ simultaneously raised privacy concerns, balancing security and privacy demands poses a substantial challenge for resilient system design.
Lukas A. Flohr*, Martina Schuß*, Dieter P. Wallach, Antonio Krüger, and Andreas Riener. 2024. Designing for Passengers’ Information Needs on Fellow Travelers: A Comparison of Day and Night Rides in Shared Automated Vehicles. Applied Ergonomics 116 (2024), 104198. https://doi.org/10.1016/j.apergo.2023.104198
*Note: both authors contributed equally to this paper. Download PDF
Conclusion: Answers to the Main Research Questions & Future Work
PhD Defense Impressions
Metadata
Dissertation
Advisors & Reviewers
Institutions
Digital Object Identifier (DOI)