Context-Based Interface Prototyping of Human-Machine Interfaces for Autonomous Vehicles

Autonomous vehicles (AVs; SAE levels 4 and 5) face substantial challenges regarding acceptance, human factors, and user experience. Human-machine interfaces (HMIs) offer the potential to account for those and facilitate AV adoption. Since AVs’ capabilities and availability are still limited, suitable prototyping methods are required to create, evaluate, and optimize novel HMI concepts from early development phases. In all human-centered design activities, physical and social contexts are vital. This thesis argues for applying context-based interface prototyping of human-AV interactions to account for their interrelation with contextual factors. We adopt a ’research in and through design’ approach and explore the two intertwined areas: design and prototyping. Regarding the latter, we concentrate on straightforward methods. We demonstrate an immersive video-based approach for lab simulation of AVs and a wizard-of-oz-based method for on-road AV simulation and prototyping of HMIs providing real-time information. We apply these methods in empirical studies to assess their suitability and explore HMI concepts created to counter the aforementioned challenges. Thereby, we investigate the potential of (AR-based) object detection visualization and concepts for mobile and in-vehicle interaction with (shared) AVs. Based on the findings, we provide design and prototyping recommendations that will aid researchers and practitioners in creating suitable human-AV interactions.

Thesis Outline

Key Publications

Background & Related Work

Lukas A. Flohr and Dieter P. Wallach. 2023. The Value of Context-Based Interface Prototyping for the Autonomous Vehicle Domain: A Method Overview. Multimodal Technologies and Interaction 7, 4 (2023), 1–17. https://doi.org/10.3390/mti7010004 Download PDF

Immersive Video-Based Simulation

Lukas A. Flohr, Dominik Janetzko, Dieter P. Wallach, Sebastian C. Scholz, and Antonio Krüger. 2020. Context-Based Interface Prototyping and Evaluation for (Shared) Autonomous Vehicles Using a Lightweight Immersive Video-Based Simulator. In Proceedings of the 2020 ACM Designing Interactive Systems Conference (Eindhoven, Netherlands) (DIS ’20). Association for Computing Machinery, New York, NY, USA, 1379–1390. https://doi.org/10.1145/3357236.3395468 Download PDF

Wizard-of-Oz & Object Detection

Lukas A. Flohr, Joseph S. Valiyaveettil, Antonio Krüger, and Dieter P. Wallach. 2023. Prototyping Autonomous Vehicle Windshields with AR and Real-Time Object Detection Visualization: An On-Road Wizard-of-Oz Study. Proceedings of the 2023 ACM Designing Interactive Systems Conference (Pittsburgh, PA, USA) (DIS ’23). Association for Computing Machinery, New York, NY, USA, 1–15. https://doi.org/10.1145/3563657.3596051 Download PDF

Mobile Interaction with Autonomous Mobility-on-Demand (AMoD)

Lukas A. Flohr, Sofie Kalinke, Antonio Krüger, and Dieter P. Wallach. 2021. Chat or Tap? – Comparing Chatbots with ‘Classic’ Graphical User Interfaces for Mobile Interaction with Autonomous Mobility-on-Demand Systems. In Proceedings of the 23rd International Conference on Mobile Human-Computer Interaction (Toulouse & Virtual, France) (MobileHCI ’21). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3447526.3472036 Download PDF

Information Demands at Day and Night

Lukas A. Flohr*, Martina Schuß*, Dieter P. Wallach, Antonio Krüger, and Andreas Riener. 2024. Designing for Passengers’ Information Needs on Fellow Travelers: A Comparison of Day and Night Rides in Shared Automated Vehicles. Applied Ergonomics 116 (2024), 104198. https://doi.org/10.1016/j.apergo.2023.104198
*Note: both authors contributed equally to this paper. Download PDF