Physical object interaction in first responder mixed reality training

Christian Schönauer; Chris Bösch; Thomas Wechdorn; Johannes Göllner; Andreas Peer; Annette Mossel; Hannes Kaufmann

doi:10.1117/12.2557396

23 April 2020 Physical object interaction in first responder mixed reality training

Christian Schönauer, Chris Bösch, Thomas Wechdorn, Johannes Göllner, Andreas Peer, Annette Mossel, Hannes Kaufmann

Proceedings Volume 11426, Virtual, Augmented, and Mixed Reality (XR) Technology for Multi-Domain Operations; 114260F (2020) https://doi.org/10.1117/12.2557396
Event: SPIE Defense + Commercial Sensing, 2020, Online Only

Abstract

Virtual Reality (VR) systems can improve the training of first responders and soldiers in multi-domain operations in a number of ways. Realistic simulation of physical objects, however, is challenging. The huge variety of equipment pieces and other objects specialists in first responder units - and in particular CBRN-troops - interact with further increases the effort. In this paper, we present a novel and flexible Mixed Reality (MR) training system for first responders that enables the integration of physical objects by using Augmented Virtuality (AV) and binary tracking". A Head Mounted Display (HMD) immerses the user in VR, while augmenting the visualization with 3D imagery of real objects, captured by an RGB-D sensor. In addition, a RFID-reader at the user's hand detects the presence or absence (binary response) of certain equipment items. Our proposed MR system fuses this information with data of an inertial motion capture suit to an approximate global object pose and distributes it. Our solution provides a wide range of options for physical object interaction and collaboration in a multi-user MR environment. In addition, we demonstrate the training capabilities of our proposed system with a multi-user training scenario, simulating a CBRN crisis. Results from our technical and quantitative user evaluation with 13 experts in CBRN response from the Austrian Armed Forces (National Defense Academy and Competence Center NBC Defense) indicate strong applicability and user acceptance. Over 80% of the participants found it easy or very easy to interact with physical objects and liked the multi-user training much or very much.

Citation Download Citation

Christian Schönauer, Chris Bösch, Thomas Wechdorn, Johannes Göllner, Andreas Peer, Annette Mossel, and Hannes Kaufmann "Physical object interaction in first responder mixed reality training", Proc. SPIE 11426, Virtual, Augmented, and Mixed Reality (XR) Technology for Multi-Domain Operations, 114260F (23 April 2020); https://doi.org/10.1117/12.2557396

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available