A beautiful, clear-blue sky hung over New York City on a warm September morning.
It was a local election day, so much of the city was running a little late into work. But the first responders — firefighters, EMS and police officers — had already been at work for hours.
Just after 8:46 am, calls began flooding 9-1-1. New York City firefighters, police and emergency medical responders began dispatching to the World Trade Center where a plane had just flown into the North Tower. At the time, no one had any idea how just big the disaster would become, or the hazardous risks involved in being on the front lines.
As trained first responders, the priority was on rescuing as many civilians as possible. Fire and police units began moving survivors away from the rapidly deteriorating World Trade Center One and Two, and their surrounding skyscrapers.
The air was thick as dust and debris mixed with jet fuel, asbestos, lead, mercury and dangerous carcinogens fell from the crash zones 90 stories above. Less than two hours after the initial impact, both towers collapsed, kicking up a dust cloud containing over 2,500 contaminants that quickly engulfed most of lower Manhattan. Rising from the scene, and already spreading across the sky into Brooklyn and New Jersey, was a toxic dust cloud visible from space. The smoke continued to rise for 100 days before the site finally stopped burning.
As September 12th dawned, emergency teams were faced with a new challenge — first search and rescue, then recovery over a 14+ acre smoldering pile of rubble already dubbed “Ground Zero.” They also spent weeks searching the surrounding subterraneous tunnels where people may have taken refuge. Recovery operations continued for nine months, during which time the site and air contained hazardous levels of toxins.
In June 2002, the air quality in lower Manhattan had finally returned to pre-September 11th levels. But the damage had been done. Approximately 75,000 people were enrolled in the World Trade Center Health Program as of 2016, having reported health impacts from the toxins in the air as a result of the attacks.
Everyday Workers at Risk of HAZMAT
While the scale of an incident such as September 11th, 2001 might be rare, workers and responders deal every day with incidents involving hazardous materials (HAZMAT) where even a small exposure can leave lasting, even deadly, effects.
Government regulations require anyone working in an environment that may expose them to hazardous substances to complete general training. This includes military, firefighters, EMS and police, who could come in contact with a hazard while responding to an emergency. But it also includes millions of workers worldwide in other industries—such as manufacturing, mining, drilling and refinery—who go to work every day at risk of exposure to dangerous chemicals should a leak or accident occur.
For an employee who finds themselves in the rare emergency situation, understanding how to identify a toxic leak and take the right actions to protect themselves is very important. But given the extremely tense nature of many HAZMAT situations, it’s not always easy to recall what you learned in training in the heat of the moment.
Empowered by Real Experience
Dr. Mitchel A. Rosen Directs the Center for Public Health Workforce Development at Rutgers School of Public Health, and runs HAZMAT training. According to Dr. Rosen, HAZMAT training includes a forty-hour course consisting of a mix of classroom, computer and field simulations.
While the training Dr. Rosen has been conducting for years is good at giving workers a solid education around HAZMAT procedure, it lacked the feel of realism. “Typically, when students are taking their readings with the meters, the instructors would have to say to them, ‘we’ve got 21% oxygen, 2% LEL, and so forth.”
“I thought it would be great to develop a program where we didn’t have to have the instructor walking around talking them through it, one where the students could get ‘real’ meter readings based on a programmable scenario.”
Dr. Rosen reached out to a colleague at the New Jersey Institute of Technology, where Dr. Cesar Bandera heads the entrepreneurship program. He is also the co-owner of Cell Podium, a tech startup in Newark. They had worked together on a prior project developing technology to deliver “just in time” emergency training via bite-sized video clips that could be received on any cell phone. “Cesar was the technology guy and I’m the content guy,” explained Dr. Rosen.
They agreed that utilizing a combination of sensors, cloud computing and smartphones, it would be possible to create an augmented reality that simulates a wide array of HAZMAT scenarios, allowing students to feel more responsible in the moment and begin building muscle memory for longer-term information retention.
Creating the Augmented Reality
Traditional field exercises consist of a multi-acre area set with an array of hazards that might be found at a spill site. Students must “solve” hazards through a process of interpreting meter readings, assessing the situation, and utilizing the right protocol to control the hazards.
In the real world, any number of factors can rapidly change a HAZMAT situation. Wind direction shifts, driving toxins directly towards rescue workers. A leak expands, increasing toxin levels to deadly within seconds. In the current setup, there was just no way to simulate these important reactionary moments without an instructor verbalizing each change, which pulls students right out of the simulation.
Bandera began by defining the data it would require to create the augmented reality.
In order to map the simulation, each participant’s exact location at any given moment was required. Each hazard’s location was also necessary.
Each hazard’s “toxic persona,” or mix of meter readings had to be created.
Wind, rain, water and any other environmental factor had to be both tracked and programmable to interact with the simulated toxin readings.
The next step was to decide on the technology required for data capture and processing. Bandera chose Bluetooth technology for its ability to capture location information accurately down to the centimeter. Beacons were placed on the hazards and in participants’ sensor simulator meters (smart devices) to transmit their location in real-time. Bandera stresses, “The simulation and the reality must be in sync, within milliseconds, for it to work.”
Generating the augmented reality also uses environmental data, which the trainer can specify to create the desired training scenario, or which can come from real-time meteorological data feeds.
All this data must be sent up to the cloud, processed instantaneously, and fed back to the students via sensor simulator “meters.” Bandera used smart devices to receive this data, which manifested in the form of toxin readings that appear on the screen. 3D printed boxes completed the experience to replicate a real meter typically used.
AR Training for Everyone
Since its initial testing at the Rutgers training site in summer of 2017, six trainings have been conducted, with eight more planned through 2018. Dr. Rosen has heard the best feedback from instructors, who really appreciate the realistic approach the technology has enabled.
Each year, the Worker Training Program of the National Institute of Environmental Health Sciences (NIEHS) helps over one hundred thousand workers around the country receive HAZMAT training. This organization, led by Director Joseph “Chip” Hughes, has funded the research behind the augmented reality technology to date, enabling Bandera and Rosen to bring this new technology to the Worker Training Program’s diverse community.
This non-traditional, innovative use of augmented reality has made the HAZMAT training experience significantly more accessible, and Bandera is already seeing its potential for much broader applications. He plans to take the tech to every HAZMAT training facility in the country, and is excited at the possibilities of supplying the tech to private companies as well to help employees be better prepared for accidents or emergency situations.
“This kind of augmented reality is only possible because of the advances in mobile device technology and cloud computing. We can deliver better training to more people, and that’s going to save lives.” — Cesar Bandera, co-founder of Cell Podium
Reader Question: What other kinds of training do you think AR would benefit?
Let us know in the comments!