Deutsch

All projects

MIA-PROM

Multimodal Interactive Assistant (MIA) - Patient Reported Outcome Measures (PROMs)

Collecting patient feedback is essential for evaluating medical treatments. One valuable method is using Patient Reported Outcome Measures (PROMs) questionnaires that capture patients’ experiences before and after care. Yet, challenges like language barriers, reading difficulties, or low motivation can prevent many from completing them, especially those from disadvantaged groups. The MIA-PROM project addresses this issue by developing a Multimodal Interactive Assistant (MIA) featuring a virtual and a physical assistant agent that guides patients through PROM questionnaires, explaining terms, answering questions, and encouraging completion in an accessible and engaging way. The project is a collaboration between four academic, industrial, and clinical partners and has a duration of 40 months. Our main contributions to the project were the development and integration of the virtual avatar assistant in the system, UX evaluation, and data analysis.

  • Funders Federal Ministry of Education and Research
  • Funding program AI-based assistance systems for process-accompanying health applications (KIAS)
  • Funding amount 2.13 Million Euro
  • Duration 2022–2025

Funding

  • Bundesministerium für Bildung und Forschung

Partners

  • Charité - University Berlin
  • University of Applied Sciences Munich
  • Technical University of Berlin
  • Hamm-Lippstadt University of Applied Sciences
  • Dexter Health GmbH
  • Acalta GmbH
  • ZAR rehabilitation center Berlin
  • Seehof rehabilitation center Berlin

Team

Project website →

MIA-PROM is a collaborative effort, supported by the German Federal Ministry of Education and Research and involving academic, industrial, and clinical partners including Charité – University Berlin, University of Applied Sciences Munich, Technical University of Berlin, Hamm-Lippstadt University of Applied Sciences, Dexter Health GmbH, Acalta GmbH, ZAR Rehabilitation Center Berlin, and Seehof Rehabilitation Center Berlin. The project runs from 2022 to 2025 with a funding amount of 2.13 million euros. The key contributions from research teams have been the development and integration of the virtual avatar assistant, comprehensive user experience evaluation, and robust data analysis. By focusing on patient-centered communication and technological innovation, MIA-PROM aims to set a new standard for digital health data collection and improve medical care for diverse populations.

A workplace showing two tablets, with the miaprom software. On of them is showing the virtual avatar

The MIA-PROM project is designed to revolutionize the way patient feedback is collected in clinical environments by utilizing advanced AI-driven assistance. Traditionally, Patient Reported Outcome Measures (PROMs) questionnaires are used to gather detailed data on treatment experiences and outcomes. However, the process is often hampered by barriers such as language difficulties, lack of understanding of medical terminology, and low motivation to participate—especially among disadvantaged groups. The MIA-PROM project addresses these challenges by introducing a Multimodal Interactive Assistant (MIA) that guides patients through the PROM process in a more engaging and accessible manner. This assistant appears both as a virtual avatar and a physical agent (Furhat robot), providing explanations of terms, answering questions, and motivating patients to complete the questionnaires fully, thereby improving the quality and inclusivity of the data collected.

An illustration of the questionnaire

One of the core innovations of MIA-PROM lies in its commitment to accessibility and social inclusion. The assistant is designed to read questions aloud, translate medical terms into simpler language, and offer support tailored to individual patient needs. The project team has also adopted a participatory design approach, involving a patient advisory board throughout the development process to ensure that key design decisions reflect real patient perspectives. Workshops are used to gather feedback and discuss the preferred modality of the assistant whether a small robot-like agent for physical presence or a virtual avatar displayed on the screen. The goal is to determine which format most effectively encourages patient participation and data quality. Additionally, the system’s speech module is powered by artificial intelligence that learns and adapts during real interactions, continuously improving its effectiveness over time.

Two people infront of the Software. Two tablets, one showing the questionnaire, and one showing the virtual avatar

Publications