Mining Contract: Development of Automated Rockfall Detection via Thermal Video Cameras in Open Pit Mines

Keywords: Geological mapping Thermal detection

Contract #	75D30122C14875
Start Date	9/1/2021
Research Concept	Rockfalls and slope failures are a critical and increasing risk for the mining industry. Despite great progress in slope monitoring, there are limited methods to detect rockfalls in real time in the mining environment. It is possible to monitor rockfall sources and deposition zones with terrestrial and unmanned aerial vehicle (UAV) light detection and ranging (LiDAR) systems, but these options require post-processing and are not conducted in real time. Two Doppler radar systems are in commercial development for rockfall detection, but they have not been widely adopted in the U.S. mining industry. Research under a previous NIOSH Mining pilot contract, “Application Testing of Thermal Imaging Cameras for the Detection of Rockfall Events and Conditions in Open Pit Mines,” has proven that thermal video cameras can see rockfall, and, more specifically, the changes left on a slope as objects fall, impact, and disturb the ground. This research recorded over 1,000 rockfalls, seven slope failures, and over 32,000 hours of thermal video in mining operations. The University of Arizona Geotechnical Center of Excellence (GCE), which partnered with NIOSH on the previous pilot contract, has received multiple industry requests to develop an automated rockfall detection algorithm to be applied to thermal imagery which would automatically activate alarms when rockfalls occur and/or impact an area of concern. Such algorithms will also support the integration of thermal and Doppler radar systems.
Topic Area	Ground Monitoring

Contract Status & Impact

This contract is ongoing. For more information on this contract, send a request to mining@cdc.gov.

Based on the above research concept and past related research, the current contract research addresses this issue of rockfalls and slope failures by way of two aims:

To develop an automated rockfall recognition algorithm using computer vision approaches applied to thermal video from open pit mines.
To develop empirical approaches for rockfall forecasting and slope hazard recognition based on rockfall frequency and location.

The first aim of developing an automated rockfall detection algorithm will improve upon the pilot contract algorithm, developed in Python, by focusing on (1) reliable detection of motion and tracking, (2) reliable classification of motion, and (3) alarming and presentation of results. For motion detection and tracking, a series of test implementations will be programmed and evaluated using NIOSH’s extensive data set of thermal camera videos. To ensure reliability, a motion classifier will be implemented by developing a “masking” process to exclude active mining areas, allowing the detection algorithm to focus on regions of interest and ignore motion under the mask due to mining equipment. For alarming, thermal camera data will be geo-referenced to a 3D pit digital terrain model (DTM), with the rockfall locations and tracking points on the pit DTM exported to a user in a mine coordinate system.

The second aim, developing empirical approaches for rockfall forecasting and slope hazard recognition, will be achieved once an automated rockfall detection algorithm is running. This algorithm will be pointed to NIOSH’s 32,000 hours of thermal video, with more than 1,000 rockfalls, and the detection algorithm will document the time and spatial location of the rockfall. The GCE team will compare that data against rainfall, wind, temperature, and air pressure data to develop an empirical tool to help highlight times of elevated rockfall risk. It is expected that these empirical correlations will help geotechnical engineers to develop weather trigger action response plans for rockfall and slope failures. By comparing rockfall events with the associated data, contract researchers will develop empirical correlations to identify high-risk time periods for rockfall events. Having a better understanding of the times of elevated risk will allow for the creation of guidelines to aid geotechnical engineers and mine operation staff to better manage risks.

The ultimate goal is to develop a standalone system that uses a single thermal camera and the new detection algorithm. This system can be transported in a pickup truck and installed in less than 1 hour by a single person, and it will include a single, high-resolution, security-type thermal camera; a tripod; plug-in capability to a mine power source; and plug-in capability to a mine’s wireless network. Components will be added for weatherproofing if needed, especially if determined that the detection algorithm process must be conducted at the camera in the field. The GCE will then deploy and test this system at mines close to the University campus to further refine the setup.