TTCI R&D: Machine Learning for Machine Vision Systems

Barber® S-2-HD® with Frame Brace®

RAILWAY AGE, JULY 2021 ISSUE: Reliable, real-time monitoring of in-service railcar components will enhance the potential for maintenance planning.

BY ANISH POUDEL, PH.D – PRINCIPAL INVESTIGATOR I (NDT); ABE MEDDAH – PRINCIPAL INVESTIGATOR I; AND MATT WITTE, PH.D – SCIENTIST, TRANSPORTATION TECHNOLOGY CENTER, INC. 

Through the Association of American Railroads (AAR) Strategic Research Initiatives (SRI) program, Transportation Technology Center, Inc. (TTCI) has been assisting suppliers and other stakeholders in the development of machine vision technologies and related algorithms for evaluating railcar components and conditions. To enhance safety and reduce worker exposure to yard risk, North American railroads have begun to install machine vision inspection systems in revenue service. 

Using commercially available deep learning system platforms, TTCI researchers developed and demonstrated three convolutional neural network-based applications for analyzing visual images. A convolutional neural network is a type of artificial neural network that uses machine learning algorithms to analyze digital images. Convolutional neural networks are more powerful and effective than traditional artificial neural networks at recognizing, interpreting, and categorizing large, unstructured data sets; particularly those comprised of visual imagery. The convolutional neural networks that TTCI demonstrated were used to identify and evaluate components on the railcar truck. Specifically, convolutional neural networks were developed to identify truck type, detect the location of a spring group and measure the compression of spring coils to determine load conditions.

A convolutional neural network must be trained to recognize and categorize data. This training is accomplished “by example” and requires two components: 1) A sampling of images representative of the type expected to be processed by the convolutional neural network, and 2) A specified desired output for when the network recognizes—or does not recognize—certain objects in each of these images. Grayscale JPEG images with 2048 × 2048 resolution were deemed adequate for developing and testing the applications. These images were obtained from TTCI’s manual viewer database system, which contains machine vision data from vision systems on site at the Federal Railroad Administration Transportation Technology Center (TTC), as well as images from a revenue service location (Stewart, Monique, Matthew Witte, and Abe Meddah. August 2020. “In-Service Performance of a Truck Component Inspection System,” Federal Railroad Administration, Office of Research and Development, DOT/FRA/ORD-20/31, Washington, D.C.). A total of 1,276 images were used to build and test these applications and to evaluate the overall possibility of machine learning technology.

In the platform used to demonstrate this application, users must label several images as they are uploaded initially; however, when enough images are labeled, the system’s automated process can take over and finish the task. For each application developed, the automated process randomly selected 75% of the images used as the training set. The remaining 25% of the images were withheld as a validation set and used later to evaluate accuracy. The Truck Type Classification, Spring Group Detection, and Spring Group Load applications attained 97%, 100% and 91% accuracy rates, respectively, for randomly selected validation sets. 

This program demonstrated the viability of using convolutional neural network-based machine learning applications to detect, identify, and classify specific patterns from images of railcar components. Further collaboration from railroads and railroad industry stakeholders is anticipated. Coordinated collaboration would generate uniform sets of railcar component/condition images for use by the algorithm developers to develop more convolutional neural network-based applications. Such applications could then be tested and validated with a goal of implementation. 

This, in turn, would allow for reliable, real-time monitoring of in-service railcar components that would enhance the potential for maintenance planning. Work has already begun at Railinc to build a machine vision image repository for the industry.