Mining Project: Machine Situational Awareness (MSA) - Framework and Algorithms for Assured Autonomy in Mining

Principal Investigator	Michael McNinch - Request information
Start Date	10/1/2023
Objective	The Machine Situation Awareness (MSA) project will research methods to ensure that autonomous mining equipment is safe for operation near personnel. This is known as Assured Autonomy (AA), and the principal goal of this work is the creation of a supervisory control framework capable of understanding its environment, identifying possible hazards, and recommending actions that minimize associated risk.
Topic Area	Programmable Electronic Systems

Research Summary

Approach

This project will place emphasis on development of industry partnerships. NIOSH recognizes that the project’s success is dependent upon more timely and more effective implementation of AA solutions for mining. This requires that much of the project effort will be focused on establishing relationships and ongoing interactions with original equipment manufacturers (OEMs), other government agencies, mine operators, autonomy retrofit firms, and international interest organizations (Global Mining Group [GMG], the International Organization for Standardization [ISO], etc.). This will involve regular correspondence, conference/exhibition attendance, industry/research involvement, and solicitation of feedback and critique from academia and the private sector.

Logistically, first steps will happen in parallel: development of a preliminary framework, engagement with industry partners, and establishment of testing goals and relationships. Much of the first year will be dedicated to (1) understanding operations and maintenance processes to properly define safety requirements, and (2) gathering and evaluating available technologies that may be useful for the ultimate implementation of the system. Project researchers expect in the second year to be developing specific test plans based on partner involvement and identified validation requirements. This will certainly be an iterative and evolving process that will culminate in proof-of-concept testing, evaluation, and dissemination. In general, researchers expect the final two years of the project to be heavily involved with in-field testing with industry partners.

Field testing will necessarily focus on reliability and predictability of responses in real-world environments, computer processing demand, adaptability of the framework, communication requirements and protocols, projected cost of implementation, ability to replay events repeatedly, and identification of areas for improvement. This framework is intended to be adaptable across a range of sensors and algorithms, as well as be applicable to a variety of equipment. The goal of the prototype is to demonstrate viability and present a practical solution to OEMs that meets the proposed requirements.

Milestones and Accomplishments

Description	Audience	Year
Author final safety requirements for selected autonomous equipment	Academia and OEMs	2024
Present webinar covering draft framework prior to field testing	Academia and OEMs	2025
Complete full documentation of MSA framework and supporting algorithm	Academia and OEMs	2027

Planned Impacts and Outcomes

This project will promote safety through adaptable MSA and AA while leveraging stakeholder relationships to ensure adoption by industry—the foremost measure of impact. This project is forward-looking as the goal is to avoid accidents before a wider rollout of the technology in the U.S. The purpose of prototype development is to demonstrate viable algorithms that can meet safety requirements for tracking and hazard evaluation. These methods will ease integration of the framework and provide state-of-the-art solutions.

Outputs

There are seven planned outputs for the first year of the project. A journal article for MMEX, “Evaluation of Models for Interaction Probability in Autonomous Monitor and Control Environments” will describe the logistics and approach to evaluating interaction probabilities in the MSA environment. This article will be accompanied by a press release. “Perceptive Track Projection – Creating Context Sensitive Path, Velocity, and Auxiliary Activity Projections for Use in Autonomous Safety Intervention System” will be a conference proceeding describing path projection methods and context for tracking. A trade publication is planned that will act as an introduction to the project and allow for a wider audience. “Machine Situation Awareness in Mining Systems: Addressing Special Circumstances and Barriers to Autonomy” will be a conference proceeding describing challenges in adoption and how they may be mitigated by MSA.

Because outreach is vital to the project, several presentations are planned in addition to the aforementioned conference proceedings. “Overview of Machine Situational Awareness for NORA Construction Sector Council-Presentation” will provide a high-level project description for the National Occupational Research Agenda partnership. “Who goes there? Creating robust object detection and multi-target tracking in support of autonomous mining equipment” will discuss early progress in adapting tracking algorithms. Finally, there will be a presentation on autonomous safety for the Mine Safety and Health Administration (MSHA).

The project team has recently completed four outputs from the pilot phase. These include overview presentations for the NIOSH Mine Automation and Emerging Technologies Health and Safety Partnership and the Society for Mining, Metallurgy & Exploration (SME) Spring Series. Additionally, presentations on probability processing were given at SME MineXchange and SME APCOM, alongside a journal article for APCOM.

Supporting Visuals

Lidar and thermal camera images taken at the project test bench during the pilot phase.

Still of skeletonization test for aiding in activity recognition for mine workers.

---