FRA RD&T: Using AI to Improve Safety

Written by Jay P. Baillargeon, FRA Office of Research, Development and Technology – Track Research Division

Ingenuity and innovation have made rail one of the safest modes of transportation in our nation today. Hard lessons were learned over many decades by railroads, workers and regulators. Significant strides were made possible by the development and widespread adoption of evolving technologies and modern business practices. Among the contemporary cutting-edge technologies making their mark on the railroad industry is artificial intelligence, or “AI.”

The U.S. Office of Science and Technology Policy defines AI as technology that enables computers and other automated systems to perform tasks that have historically required human cognition and what we typically consider to be human decision-making abilities. The history of railroading is replete with advances in mechanical, civil, electrical and chemical engineering. In no small part, advances in AI and computer science are generating even more momentum and driving a new technological revolution expected to dramatically transform all fields of engineering and the future of railroading.

Building the Foundation for AI Research

Increasing automation of operations, inspections, equipment and safety processes are widely expected as new and emerging AI-based technology is used in railroading. In response to previous developments and anticipating new ones, the Federal Railroad Administration (FRA) is building the capacity to understand and assess the safety implications of new technology. Since the early 2000s, FRA’s Track Research Division, in the Office of Research, Development and Technology (RD&T), has been actively engaged in AI research—including research into neural net applications, machine vision and machine-learning capabilities for complex analyses as well as new and innovative inspection technologies incorporating AI-based processing techniques. RD&T has been a key proponent of AI for nearly two decades and has seen its efforts translate into viable technology with widespread implementation in the railroad industry.

For example, the FRA RD&T automated joint bar inspection tool, developed primarily between 2002 and 2009, epitomizes the agency’s focus on leveraging AI tools and applications. In 2009, the technology was successfully commercialized and has since become a standard and widely accepted method for automatically inspecting joint bars. This system, which can be deployed either on a hi-rail vehicle or inspection car, takes illuminated images of the joint bar at speeds up to 60 mph and runs them through a series of complex machine-learning algorithms. The imagery is then processed to determine whether even minute hairline cracks are present. The images, along with detailed geolocation information, are then provided to railroad maintenance personnel for remediation.

RD&T will study other facets of AI over the next 5 years, including two specific areas:

  • AI-based risk analysis—in which a suite of technologies will be developed to increase safety and reduce human error by improving the speed, accuracy and consistency of routine inspection processes. The primary focus of this initiative will be the application of predictive analytics.
  • Expansion of autonomous inspection technologies—so that key inspections of equipment or infrastructure occur seamlessly during routine operations, instead of as a separate, dedicated process. 

While efforts to date have been focused primarily on track-related applications, RD&T is expanding the scope of AI-related research into other areas, such as highway-rail grade crossing safety enhancements and trespass deterrence and prevention.

AI-Based Predictive Analytics for Track Data

Predictive analytics, in the context of track-related research, refers to the analysis and application of track measurement data. Such information is needed to build computational tools designed to accurately predict adverse track structure/substructure conditions. The primary goal of this research is to help railroads more easily identify high-risk track segments and, in turn, prevent unsafe conditions long before they become problematic by augmenting current inspection capabilities, optimizing inspection vehicle routing and enabling risk-based preventive maintenance approaches. Also, by incorporating innovative AI-based techniques such as machine learning, RD&T is exploring ways of automating the processing and reporting of analytical results to enable real-time decision making in the future, getting relevant data-driven information to field personnel quicker.

Predictive analytics requires a significant amount of data to create algorithms that accurately and reliably predict adverse conditions with a minimal false-positive rate. Fortunately, autonomous track geometry measurement systems (ATGMS) permit more frequent inspections during revenue service—without degrading operational efficiency. Along with the vast distances these systems cover each year, an equally substantial amount of raw data is collected that contain valuable insight into long-term trends. These data allow operators to monitor track geometry conditions as they develop over time, but the current process by which it is extrapolated can be labor intensive and time consuming. The increased speed of data processing now allows railroads to predict degradation rates, optimize maintenance efforts and, in turn, prevent safety-critical issues from occurring.

As part of a new research initiative, RD&T is developing computational strategies that will direct the automated management of recursive track geometry measurements gathered by ATGMS vehicles. Using raw data collected from the ATGMS fleet of a U.S. passenger railroad, a process is being developed that: (1) segments and aligns the track geometry measurements from multiple time-separated runs; (2) identifies and processes peak-value deviations; and (3) reports the appropriate severity level of the deviations as they relate to established maintenance and safety thresholds. This automated process will employ machine learning to streamline the steps taken to transfer actionable information from the ATGMS vehicle to the decision maker responsible for maintenance and regulatory compliance. From here, the foundational elements of the research can be applied to other track geometry systems, both manned and autonomous, and establish a framework for other track-related datasets.

Automated Inspection/Monitoring Technologies

Automated change detection is another area of interest for RD&T. Change detection is the ability to determine whether a change (or changes) has/have occurred in two or more identical images separated by time. The focus is on relevant changes to track structure that might suggest a safety-critical issue, as opposed to irrelevant, unimportant changes, such as a piece of trash that appears during a run, or minor disturbances to ballast. This is where AI comes in because algorithms can be used to properly process and align the images gathered from an inspection car or even an unmanned aerial system (UAS) and to highlight any changes that may have occurred, like a missing fastener clip or a disintegrated crosstie. RD&T is actively engaged in multiple research projects aimed at further developing this technology, using not only traditional photographic images of the track but three-dimensional laser-based triangulation techniques as well.

Beyond efforts to leverage and expand the use of AI for track and structures, RD&T is also conducting exploratory research in remote trespasser detection. AI-aided algorithms are helping to automatically process live video footage from both ground- and UAS-based systems to detect trespassers on railroad property. The application of AI allows for real-time processing and notification with minimal human supervision, while minimizing false alarms (e.g., animals passing by the camera), so law enforcement personnel may respond in a timely manner. Another project will study the effectiveness of using incidental video footage obtained from cameras along rail rights-of-way. In this case, researchers will explore using AI to automate detection from camera feeds.

The Future of Railroading

With the use of AI and other technologies, there is great potential for railroads to further reduce the occurrence of high-consequence accidents and derailments altogether. To realize such a future for rail transportation, RD&T is focused on dedicated research initiatives aimed at Improving, Implementing and Inspiring:

  • Improve: High-quality inspection/measurement data necessary to properly train AI, which results in a need for (1) less time-consuming and more efficient inspection/measurement strategies (e.g., autonomous systems) and (2) the development of new technologies to fill gaps in the data.
  • Implement: FRA has a proven record of facilitating and hastening industry implementation of AI-enabled technologies. The agency will continue to sponsor AI research to address elusive safety issues facing the railroad industry now and in the future.
  • Inspire: Continued advancements made possible by AI-enabled technologies in the railroad industry will only be possible through the recruitment and retention of recognized subject matter experts.

FRA will continue to explore the multitude of ways AI and other technologies can enhance railway safety. The agency is committed to fostering innovations essential to realizing a future where accidents and derailments in the railroad industry are a distant memory.

Resource: Download the FRA RD&T Current Research Projects PDF from the TRB 2020 Annual Conference:


The author would like to acknowledge Francesco Bedini Jacobini, who is currently spearheading FRA’s AI-related efforts in grade crossing safety and trespasser prevention. In addition, the author would like to acknowledge the support and guidance of Gary Carr, former Division Chief of the Track Research Division. Many contractors and subcontractors have also contributed to the success of research initiatives focused on applying AI for safety-enhancing technologies.

JAY P. BAILLARGEON leads the Predictive Analytics Program for the FRA Office of Research, Development and Technology Track Research Division and is based at FRA’s Transportation Technology Center in Pueblo, Col. The Predictive Analytics Program focuses on the enhancement of railroad safety through innovative analytical strategies, including AI applications for track-related datasets. Jay serves on multiple interagency task forces related to data management and AI at the U.S. Department of Transportation, including the DOT AI Task Force in response to the Presidential Executive Order on AI, and is a member of the Institute for Operations Research and the Management Sciences (INFORMS) Railway Applications Section, the American Society of Mechanical Engineers (ASME) and the American Railway Engineering and Maintenance-of-Way Association (AREMA).

Tags: , ,