Machine Vision: You CAN Fix What You Can’t See

Written by Railway Age Staff
image description

BNSF Railway trains pass by wayside detectors

RAILWAY AGE, SEPTEMBER 2020 ISSUE: Whether it’s the track structure or the equipment that operates on it, there are many things that the naked eye cannot readily see. Increasingly, machine vision technology is becoming the best way to identify potential flaws before they lead to failures.

“The various machine vision technologies deployed detect thousands of conditions each year that could potentially lead to accidents,” says Robert Coakley, Director of Business Development, ENSCO Rail. Compared to manual visual inspections, he says, autonomous machine vision offers advantages of speed, reduced track occupancy, inspection frequency and consistency. The equipment is installed on revenue service trains, can perform inspections at track speed and does not require the additional occupancy of a hi-rail vehicle.

ENSCO Autonomous Fastener Inspection utilizing ML/AI to automatically detect fasteners and fastener condition.

“ENSCO’s image evaluation processes apply a consistent standard and present track condition in a strip chart format to enable data trending and data modeling,” Coakley says. “This approach provides customers the ability to apply a consistent standard to image review and further allows for images to be evaluated over multiple inspections to create trend data analytics.”

ENSCO Autonomous Ballast Condition Inspection utilizing ML/AI to automatically evaluate ballast and convert ballast condition to strip chart output.

He adds that ENSCO uses high-resolution camera systems and practical machine vision algorithms and applies advanced image processing techniques such as deep learning to detect objects and features. The result is “high-speed and quality track imaging systems that provide extremely reliable image acquisition and processing capabilities for comprehensive track inspection and evaluation.”

ENSCO’s proprietary Virtual Track Walk software synchronizes measurement and imaging inspection data to allow for synchronized side-by-side viewing of survey data and imaging.

Kim Bowling, Director Car Monitoring & Diagnostics, CSX, says that on the rail side, her company uses two types of machine vision systems: targeted machine vision and whole-car imaging systems. The former uses “very optimized lighting and optics to help us look at a single component. Our whole-car imaging system is called a Train Inspection Portal, and today it uses 17 cameras to image the entire train as it passes at track speed,” she says.

Gary Van Tassel, Director Intermodal Ops Planning & Network Design, CSX, says on the intermodal side, the railroad has “deployed 60 optical character recognition portals across the majority of our intermodal network. These are sort of a rapidly evolving system … right now we’re capturing all camera angles—coming in and out of the terminal—and storing those for identification of damages.”

CSX intermodal terminal optical character recognition portal.

Van Tassel adds that on the crane automation side, CSX is using myriad machine vision tools, “whether it’s LIDAR (Light Detection and Ranging), lasers, or OCR (Optical Character Recognition) to identify obstacles and ultimately guide our automated cranes, which run about 80% in an automated state.”

CSX intermodal terminal optical character recognition portal.
CSX intermodal terminal optical character recognition portal.

Rail solutions company Trimble’s Beena Vision range of vision-based wayside non-contact measurement and inspection technologies aim to enable the automated, proactive monitoring of rolling stock condition, providing data feeds that can be processed to effectively assess rolling stock condition from component level to full train inspection. The solution suite includes, but is not limited to: Imaging units to inspect almost all rolling stock components visible while a train is in motion; MVAs (Machine Vision Algorithms) to process and provide information related to those images; and databases and user interfaces to access and view data with short- and long-term information for trending and prediction.

Trimble Beena Vision ML array.

“Our solutions are designed to capture data for specific components, and based on the type of measurement and inspection carried out, Trimble Beena Vision systems use the relevant sensor technology to ensure high quality and accuracy of image and data outputs,” says Ken Vilardebo, Director of Engineering.

Trimble Beena Vision display.
Trimble Beena Vision image output.

How It Works

Class I BNSF began its use of machine vision systems (MVS) with its network of wayside detectors to reduce rail equipment incidents and service interruptions. Specifically, its mechanical experts and data scientists sought to spot trends with the urgency of equipment repairs to indicate when maintenance should occur. The technology works, explains John Martin, Director, Technology Services, by using artificial intelligence (AI) in combination with MVS to analyze equipment images and identify small defects before they lead to larger problems, such as equipment failure.

BNSF wayside detector array. BNSF photo.

“Today’s AI models have advanced with increases in computer processing capabilities,” Martin says. “With these advances, we have integrated BNSF’s Mechanical Image Driven Analytics System (MIDAS) with an equipment monitoring quality system that allows us to easily cross-reference MVS information with other data from our network of 4,000 sensors that monitor rolling stock across the BNSF network.”

BNSF photo.

CSX’s Bowling notes that algorithms “are the key piece of logic that help us sort out these images and determine, is it a picture of the correct component, and then, is it a good component or a defective component? We work with different companies to help us create these algorithms. Machine Learning (ML) is a computer tool that uses multiple images to help train the model and generate the algorithm. Before we can create the model, we have to have hundreds of images of a component.

CSX’s Van Tassel adds that intermodal uses similar technology for a different but related application. “Using doors as an example,” he says, “when we’re automating our cranes and the machine is deciding which way to orient the container—obviously you have to position the container doors on the back of the truck—that’s a relatively quick and simple decision for a human operator, to be able to press a button to take them and orient it correctly. We use a similar sort of neural network where we are showing hundreds, if not thousands, of images of the front side of the container and the door side, which are a slightly different configuration. Doors have lock handles and bars to gain access to the container. We’re showing it a bunch of pictures of the front and back. And then the algorithms come in and begin to learn that configuration, allowing the machine to actually reorient the container vs. having to wait for human intervention.”

ENSCO Autonomous Fastener Inspection utilizing ML/AI to automatically detect fasteners and fastener condition.

Coakley says that ENSCO applies advanced technologies such as autonomous inspection, AI, ML, signal and image processing and data analytics in an effort to “provide the rail industry the best possible tools to ensure track and rolling stock safety, increase productivity and efficiency and reduce operating cost.”

ENSCO Rail’s Autonomous Track Geometry Measurement System (ATGMS) uses autonomous track inspection sensors and technology. The ATGMS units are installed onboard revenue service vehicles; data is then streamed wirelessly off the revenue vehicle to a cloud-based server where automated AI algorithms review the data, filter out false positives and send alerts in near real-time. ENSCO has deployed more than 25 ATGMS systems, including multiple systems operating across four North American Class I railroads. Since March 1, 2020, ENSCO ATGMS systems have inspected more than 340,000 miles, helping railroads maintain critical infrastructure during the COVID-19 pandemic when manual inspections were not possible/advisable.

ENSCO’s Autonomous Joint Bar Imaging System (AJBIS) continuously collects joint bar images, which are evaluated using deep learning algorithms. Exceptions are identified and streamed wirelessly to the cloud where customer alerts are distributed. The company’s Autonomous Rail Surface Imaging System (ARSIS) continuously collects images of the top of the rail. These images are evaluated using deep learning algorithms that identify excessive surface damage. ENSCO’s Autonomous Track Component Imaging System (ATCIS) continuously collects images of the track bed, including fasteners and ties. These images are evaluated using deep learning algorithms that identify missing/broken fasteners, skewed or broken sleepers, missing rail anchors, etc.

All these technologies identify exceptions and stream them wirelessly to the cloud, where customer alerts are distributed.

Trimble Beena Vision solutions’ Vilardebo says that “in the application of machine vision technology using wayside detectors there are three key phases: image data acquisition, image and data processing, and fault detection and alarm generation. MVS technology is installed on and around tracks where environmental conditions can be extreme. Image quality and fidelity must be independent of environmental conditions such as lighting, temperature, precipitation, etc., for the successful deployment of vision-based systems. Proper lighting is the second significant factor. After capturing, labeling and storing images, Machine Vision Algorithms (MVAs) are then deployed to process the images and create the relevant information for the identified components. The final stage of data and image processing is deciding which outputs of MVAs are used to create different levels of warnings and alarms.”

“MVA-based alarms are either instantaneous, planning level, or trend-based,” Vilardebo says. “Instantaneous alarms can be generated to indicate significant failure modes such as coupler securement failure, a broken center sill or a condemnable wheel. In these cases, depending on the applicable rules, the train may have to be stopped immediately for urgent corrective action. In other, less urgent cases such as a bearing cap bolt missing or a broken truck spring, the train is usually moved to a more appropriate location for repairs. Planning level information refers to the identification of less serious conditions that do require maintenance to be planned and executed in certain timeframes. These types of events could refer to, for instance, minor structural damage, missing earth straps on cars or other events that can be dealt with when ‘next in shop.’

“Trend-based alarms are usually created based on relatively slowly changing measurements of an asset, where the trend of the change indicates a possible fault. Trend-based alarms are usually generated from a database where all historical data is available and analyzed periodically. Alarms and alerts from Trimble Beena Visions systems can be viewed and managed using one of Trimble’s machine vision condition monitoring data management solutions, Trimble® WISE (Wayside Inspection System Environment) or Trimble® TrainWatch, which are sophisticated software applications for detector data visualizations and analysis.”

Trimble® WISE is a condition-monitoring data management platform that aims to provide a unified environment for wayside detector data. Trimble® TrainWatch is a virtual train inspection portal that provides a comprehensive environment for train inspectors to inspect a full train using data gathered by wayside equipment. Existing Trimble automated inspection algorithms are also supported within the TrainWatch environment, which allows some required inspections to be managed using automated algorithms “making the virtual inspection process even faster,” Vilardebo says.

Duos Technologies will institute increased automated mechanical inspections at one of its existing Railcar Inspection Portals (rip®) along a Class I network under a $1.3 million contract.

Duos Technologies machine vision portal.

The goal of the upgraded rip® system is to identify specific railcar inspection points that can be monitored and addressed through the use of artificial intelligence applications that the Class I is developing and integrating into the Duos centraco® Command and Control Software platform, which “consolidates data and events from multiple sources into a unified and distributive user interface,” according to Duos.

rip® technology consists of a 360-degree modular intelligent visualization system that takes detailed, real-time, full-picture images of railcars moving at speeds of up to 120 miles per hour. The panoramic view can detect oil leaks, damaged parts, open doors and open and missing hatches, alerting inspectors to the problem and showing them the location of the problem. Sophisticated algorithms also identify more complex problems. The upgrade is aimed at creating “new perspectives” within the current system to identify specific railcar inspection points that can be monitored by channeling artificial intelligence (AI) applications being developed by the client and integrated into the Duos centraco® platform. Duos said the expectation is that it will expand to other locations upon a successful proof-of-concept.

Work is slated for completion before year-end. The contract also includes future payments through 2022 for recurring service, maintenance and spare parts components.

Duos CEO Chuck Ferry, who started work at the company earlier this month, said he is “looking forward to growing our relationship with our customer and leveraging this initial upgrade into additional system deployments in the future.” Duos has supplied rip® systems to several Class I railroads. CN, for example, has deployed seven so far.

When to Use It

Martin says BNSF is currently leveraging MIDAS at five locations across its network, monitoring seven main line tracks to specifically identify broken or cracked wheels. These systems capture data from approximately 250 trains per day, producing more than 650,000 wheel images. The system processes images trackside and transmits defects to BNSF’s Network Operations Center 24 hours a day, allowing for alerts to its Mechanical team, which then reviews the images of potential defects and makes necessary repairs.

“Ultimately, these preventative systems allow our Mechanical team to make repairs at shops rather than pulling cars out of service in the field, which keeps our network fluid and reduces service interruptions,” Martin says. “We are currently expanding MIDAS to three additional locations and have plans for future sites. We are continually developing our MVS models to expand the number of components undergoing inspection.”

“This technology has been evolving for the past 15 to 20 years for containers, from the marine industry into intermodal,” CSX’s Van Tassel says. “The initial systems were reactive, basically looking at high-resolution images and hopefully capturing the container number. As the technology evolved, it has become less and less reactive. And what Kim’s doing on her side is looking at the proactive identification of defects. That’s really where the technology is starting to go very rapidly.”

Vilardebo agrees that the practice has come a long way, saying that on both the hardware and software fronts, MVS technologies have matured over the years. “On the hardware side, cameras are more robust with higher resolutions, higher frame rates and more sensitive sensors,” he says. “This helps even in low-light conditions, resulting in better inspection results. Illumination technology has also evolved to provide longer-lasting and more powerful lighting options. On the software side, there has been an explosion of new algorithms since the advent of Graphics Processing Unit (GPU)-based deep learning. Using deep learning algorithms, inspections can be done faster and with a much higher accuracy. This evolution in inspection algorithms paves the way toward the goal of fully automated train inspection.”

Coakley adds that the rise of Big Data has helped drive interest in autonomous systems, as with the increase in inspection frequency comes an increase in track data and a new opportunity to visualize and analyze data. “ENSCO’s approach to image processing extends the effectiveness of the track condition imagery by converting the image to a condition index value and integrated within ENSCO Rail Data Analytics Solutions, thus helping users optimize maintenance and renewal planning, reduce risks through earlier identification of track defects and improve rail network safety,” he says.

Future Vision

Vilardebo believes that as more operators in the rail industry start looking to digitization to reduce costs and streamline processes, digitization initiatives “will inevitably result in significant data volumes that require smart processing and analytics to provide actionable intelligence. Deriving actionable insight from data is the key to improving efficiency and reliability across an operator’s organization. Via faster enhanced data preparation and contextualization, and the application of intelligent analytics, data can be modeled to show trends and patterns that would otherwise remain hidden.”

Coakley adds that railroads face two principal challenges: scheduling critical track activities within limited track time without impacting revenue service, and accomplishing that within budget. More advancements in the field will do just that, he believes. “The integration of advanced technology such as AI and ML offer unprecedented opportunities for planning efficiency and significant maintenance cost savings,” he says. “By integrating these technologies across the entire continuum of asset monitoring and maintenance planning, railroad maintenance engineers are realizing improved safety and fewer revenue service disruptions. In short, the demand for machine vision and autonomous technology continues to grow as railroads seek creative ways to accomplish track inspection more efficiently and accurately with less impact on operations and budget.”

The railroads are on board.

FURTHER READING:

CN ATIP: No Longer Atypical

BNSF geometry train west of Belen, N.Mex. BNSF photo.
Categories: Analytics, C&S, Class I, Freight, Freight Cars, Locomotives, M/W, Mechanical, News, Short Lines & Regionals, Switching & Terminal Tags: , , , , ,