PREPARE
Multimodal Assessment and Intelligent Learning Management System Platform
PREdiction of Healthcare Provider Skill Acquisition and Future Training REquirements (PREPARE) was developed in-house at The University of Toledo starting in 2017 and is currently being leveraged across the University of Toledo’s College of Medicine and Life Sciences to support assessment during training/education (simulation-based or otherwise) of medical students, residents, and other learners enrolled to professional programs across the COMLS.
Vision
The vision for PREPARE is to personalize training/education of healthcare professionals and to provide a data driven infrastructure to avoid institution of a one-size-fits-all curriculum that is currently common practice amongst healthcare professionals and other professional programs. A key feature of the platform is to collect, process, and synchronize data at the instructor (e.g., observer-based performance assessments), operational/training environment (e.g., audio and video), and learner (physiological data and measures related to stress, cognitive/mental workload that are correlated with performance or competency/proficiency of core knowledge/skills) levels.
Capabilities of PREPARE
The University of Toledo has developed an initial version of a novel learning management system platform, PREdiction of Healthcare Provider Skill Acquisition and Future Training REquirements, (PREPARE) which standardizes creation and assessment of training curriculum for healthcare professionals of any background, discipline, or specialty.
The current version (v1.3) of the software platform provides:
- multimodal assessment capabilities at the learner, instructor, and training environment levels
- the ability to collect and process data from commercial wearable devices and physiological monitors to collect data from learners during training to provide objective performance measures
- an observer-based assessment module as well as a customizable module to generate pre- and post- assessment surveys
- export of data in CSV spreadsheet format
- built-in natural language processing module (NLP*) which serves to detect and analyze audio collected during training to provide automated performance measurement capabilities and multimodal data synchronization
All multimodal data sources collected by the platform are synchronized in time and embedded intelligent data visualizations promote data exploration, and analysis capabilities.
Measurement standardization provided by the platform allows for a common set of evaluation criteria such that large datasets can be generated to develop machine learning-based models to personalize training and education.
- Additionally, standardized measures can also be evaluated in real-world operational settings that will allow the research community to evaluate the transfer of knowledge and skills from training to real- world.
The PREPARE platform has been and is currently being used in training and real-world patient care settings across numerous academic and clinical settings at the University of Toledo, and there is current collaboration amongst various external organizations (military/government laboratories and civilian academic institutions/medical centers).
*Paudel P, Pappada S, Cheng L. Automated Multimodal Performance Evaluation in Simulation-based Medical Education using Natural Language Processing. InProceedings of the ACM/IEEE 14th International Conference on Cyber-Physical Systems (with CPS-IoT Week 2023) 2023 May 9 (pp. 258-259).
Academic Institution Collaborators
Research collaboration with various academic institutions is ongoing, under prior and active collaborations, the platform has been or is in process of being deployed for use for a variety of research and development activities including: medical simulation research, human performance monitoring and assessment research:
Government Collaborators
The University of Toledo has also established Cooperative Research and Development Agreements with the US Military/Defense Health Agency. These organizations across the Defense Health Agency have been provided the PREPARE platform to promote acquisition and sustainment of knowledge and skills required for optimal patient care across the Military Healthcare System. Current collaborating organizations include:
- U.S. Army Telemedicine and Advanced Technology Research Center (TATRC)
- United State’s Air Force 59th Medical Wing and its En Route Care Research Center (ECRC)
Strategic Partnership with Society for Simulation in Healthcare
The University of Toledo has formed a strategic collaboration with the Society for Simulation in Healthcare (SSH) and its International Simulation Data Registry (ISDR). As of June 2023, SSH and the ISDR have agreed to adopt the PREPARE platform as the primary multimodal data acquisition and assessment platform for the ISDR. The registry was founded to collect and archive standardized data derived from simulation-based educational activities at various institutions throughout the World. In support of this mission, the PREPARE platform will be the core platform providing the standardized data acquisition capability and vision of the ISDR.
Broadly, the purpose of the ISDR is three-fold:
- Enhancement of the simulation community’s efficacy in translating proven educational methods into standard practice.
- Enhancement of the efficiency of simulation-based translational research.
- Provision of ongoing site-specific educational and quality benchmarking data to improve programmatic performance.
As part of the strategic collaboration and initiative with SSH and the ISDR:
- A number of research studies leveraging PREPARE will be executed across institutional members of the ISDR.
- The goals of the collaborative research will involve improving and standardizing assessment across all healthcare disciplines and specialties, and within specialties.
- Collaborating institutions will have the opportunity to share data and results, disseminate results of research in high impact peer reviewed journals and national/international conference proceedings, and pursue research funding to advance simulation-based education and consequently advance quality of care and patient outcomes.
- The ISDR is a member only section of SSH. Institutions who wish to gain membership and collaborate with the ISDR should inquire on the SSH website.
SAFE-T-NET
PREPARE lays the groundwork for SAFE-T-NET, a groundbreaking platform that assesses healthcare professionals' performance in real-world and training settings. SAFE-T-NET's modular structure facilitates real-time evaluation using standardized measurements and automated analyses from diverse data sources.
While PREPARE contributes essential components, collaborative research is sought to fully realize SAFE-T-NET's potential. Through continuous monitoring and machine learning, the platform identifies performance-influencing factors, enabling real-time interventions and adaptive support for healthcare professionals. This extends beyond healthcare, optimizing networks and refining training interventions for enhanced operational efficiency and effectiveness.
Contact
Scott M. Pappada, Ph.D.
Associate Professor and Director of Research, Department of Anesthesiology
The University of Toledo College of Medicine and Life Sciences
Associate Professor, Department of Bioengineering
Prestige Faculty, Department of Electrical Engineering and Computer Science
The University of Toledo College of Engineering
Submit your information
Webform coming soon