Introduction

On November 1, Levi and I attended the 2019 L.A. Tech to Protect Hackathon , a hackathon to create innovative technologies for public safety agencies. In this contest, there were 10 different prompts to choose from. Since Levi and I were interested in learning augmented reality, we decided to enter Contest #003 - Looking Under the Hood: Using Augmented Reality to Help Save Trapped Passengers.

Problem Statement

When first responders arrive on the scene of a vehicle accident, the top priority is to rescue the passengers and treat their injuries. When passengers are trapped inside the vehicle, tools such as the “Jaws-of-Life” are needed to physically remove vehicle parts around the passenger. To do their jobs safely and efficiently, emergency responders need up-to-date information about the vehicle design and construction to know the safe and unsafe extrication cut points. Accidentally cutting into airbags or electric vehicle batteries would be extremely dangerous to both the passenger and emergency responders. While vehicle diagrams currently exist, it may be cumbersome to locate information for the correct vehicle make and model, the vehicle orientation may be different at the accident scene, and 2D images may be difficult to sift through.

Solution

To solve these problems, we propose Extrication Buddy. Extrication Buddy is a phone application that uses machine learning to identify the vehicle make and model, augmented reality to overlay hazards and extrication cut zones on the vehicle, and an interactive 3D model to display additional extrication information. Below, you can see a demo of Extrication Buddy in action:

Several technologies were needed to create Extrication Buddy. For the demo, the 3D model vehicle was downloaded from Free3d.com and further modified on Autodesk to include the extrication cut zones and hazardous parts. The augmented reality model targets were created using Vuforia Model Target Generator (MTG). Vuforia MTG takes a 3D object and converts it to a dataset that interfaces with the Vuforia SDK. Since Unity can be integrated with the Vuforia engine, we used Unity as our platform for Extrication Buddy. You can view the source code of Extrication Buddy at the link below:

Summary

During our time working on Extrication Buddy, we really enjoyed talking to the great community of emergency responders and learning from their great wealth of experience. We admire their strength and dedication to the safety of society. If selected to continue work on Extrication Buddy, we would love to continue working with them, NFPA, and car manufacturers to help the emergency responders more effectively and safely rescue passengers from vehicle accidents.