Deep learning applications in railroads: Predicting carloads

Written by Offei Adarkwa, Ph.D and Nii Attoh-Okine, Ph.D., PE
image description

The freight railroad industry is considered by many as having a critical role to play in the development and economic growth of the United States. In 2014 alone, it generated $33 billion in tax revenues as part of a total of $274 billion contribution to the economy (Lord, 2016).

Unsurprisingly, expert analysts and investors including Warren Buffett view freight carloads as one of the most important economic indicators (Perry, 2010). Multiple efforts have been made in the past to predict rail freight volume. Accurate prediction is important for freight railroad companies in planning and decision-making for operations, maintenance, labor allocation and capital investment. In the midst of rapid technological development and marketplace integration, prediction of rail freight volumes is all the more crucial. With the right information on future freight volumes, rail companies can efficiently allocate resources to maximize their returns on investments. At the administrative level, this provides a scientific basis for developing policies aimed to improve service delivery (Yang and Yu, 2015). Due to the recent success of TensorFlow; a machine-learning tool used in business applications, this work explored how these tools can be used to predict rail freight carloads in the United States.

Previous Work on Rail Freight Volume Predictions

Past efforts in predicting rail freight volumes involved the use of a variety of regression models including linear, logistic, polynomial and exponential (Ying et al, 2008). Other time series prediction methods such as the Autoregressive integrated moving average (ARIMA) model have also been used by researchers to predict freight volumes in Estonia (Hunt, 2013). Dimension reduction approaches such as principal component regression (PCR), partial least squares regression (PLSR) among others have been used in past studies (Yang and Yu, 2015). Several neural network architectures have also been employed as predictive tools for freight volumes with varying degrees of success (Li and Lang, 2015, Guo and Fu, 2016, Wen and Zhang, 2009).

Focus of Project

Researchers compared the predictive capabilities of long short-term memory (LSTM) networks with three other approaches: naïve forecasting, ARIMA and multi-layer perceptron regression. The LSTM model was developed using Google’s TensorFlow; an open source software library originally developed by the firm’s researchers for performing machine learning and building deep neural networks (TensorFlow, 2017). Common applications of this powerful tool include image classification and language modeling (Abadi et al., 2016).

Due to the capabilities of deep neural networks, they are inadvertently becoming a part of our daily lives and as such, applications in capital-intensive industries such as railroads need to be explored.

Comparing predictive tools for railroad freight volumes Monthly railroad carload data from the Bureau of Transportation Statistics (BTS, 2016) was used for this project. As shown in Figure 1, the period considered for analysis was from January 2000 to December 2016. The impacts of the 2007-2009 global recession on economic activity is apparent in Figure 1 due to the sharp decline in freight carloads during this period. The lowest number of carloads during the analysis period was 1,012,381 recorded in April 2016. The maximum number of carloads from 2000 to 2016 was recorded in May 2006 at 1,521,036. The average monthly number of freight carloads was 1,311,750.

Rail freight data is highly seasonal as is shown in Figure 2 using a five-year time window starting in January 2000. The annual peak loads recorded in the month of during that month in anticipation of Thanksgiving and the winter holidays:

An interesting observation is that the total freight volume has not been able to recover fully after the recession. The reduction in volumes can be attributed to the slow economic recovery as well losses due to competition from other modes of transport. In effect, railroad freight companies need to employ new strategies to protect and grow their market shares (Ashe, 2017).

The data was divided into two parts; training and test data set. The training set was made up of 60% of the data starting from January 2000. The remaining 40% of the data was used for testing the models. The performance of the models were evaluated using the root mean square error (RMSE) and mean absolute errors (MAE).

The first prediction approach was a naïve forecast where the freight volume for the future is simply the value for the previous current year. The second predictive approach was the ARIMA model, which yielded a better fit compared to the naïve approach. Relatively lower error values were obtained using the MLP prediction model. However, the deep learning approach based on LSTM yielded the best results with the lowest mean absolute error value of 1340.08 as shown in Table 1.

Figure 3 shows how each method fits the data in the test set. A closer look reveals a tighter cluster around the 45-degree line in the LSTM, reinforcing the model’s accuracy.

Table 2 also shows the observed values and predicted freight carloads for the last six months of

Potential Applications in the Industry

Rail freight data is useful indicator of economic activity since demand for rail freight services are the result of demand of other products in the economy (AAR, 2017). Accurate prediction of rail freight volumes can lead to improved operations planning, budgeting and maintenance activities.

By incorporating factors which affect the demand of railroad freight services, the deep learning approach highlighted in this piece can provide better forecasts of freight volumes. A few of these factors include GDP growth rates and fuel prices, taxes, government subsidies as well technological developments have direct impacts on price flexibility in the industry.

Promising results from this study underscore the need for further studies on the use of LSTM networks in freight volume prediction and other applications in the railroad industry.

References

AAR (2017). Freight Rail Traffic Data. https://www.aar.org/data-center/rail-traffic-data Accessed: 12/8/2017

Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M. and Ghemawat, S., 2016. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467.

Ashe, A. (2017). Railroads Must Modernize, Industry Consultant Says. Transportation Topics. http://www.ttnews.com/articles/railroads-must-modernize-industry-consultant-says Accessed: 12/8/2017

Based on Support Vector Machine and NARX Neural Network. Proceedings of 3rd

BTS (2016). Rail Freight Carloads. Bureau of Transportation Statistics. https://www.transtats.bts.gov/osea/seasonaladjustment/?PageVar=RAIL_FRT_CARLOADS Accessed: 12/1/2017

Guo, Z., and Fu, J. (2016). Prediction Method of Railway Freight Volume Based on Improved Neural Network. 2nd International Conference on Electronics, Network and Computer Engineering (ICENCE 2016), pp. 936- 940

Hunt, U. (2003). Forecasting of railway freight volume: Approach of Estonian railway to arise efficiency. Transport, 18:6, pp. 255-258

International Conference on Logistics, Informatics and Service Science, DOI 10.1007/978-3-642-40660-7_129

Koskela, T., Lehtokangas, M., Saarinen, J. and Kaski, K. (1996). Time series prediction with multilayer perceptron, FIR and Elman neural networks. In Proceedings of the World Congress on Neural Networks (pp. 491-496). INNS Press San Diego, USA.

Li, X., and Lang, M. (2015). Combination Prediction of Railway Freight Volume

Lord, N. (2016). Freight Rail Effect: How one industry drives billions in economic activity. Politico. http://www.politico.com/sponsor-content/2016/07/freight-rail-effect Accessed: 11/15/2017

Perry, M.J. (2010). Rail Traffic: Warren Buffett’s Favorite Economic Indicator. Seeking Alpha. https://seekingalpha.com/article/213814-rail-traffic-warren-buffetts-favorite-economic-indicator Accessed: 11/15/2017

TensorFlow (2017). About TensorFlow. https://www.tensorflow.org/ Accessed: 12/4/2017

Wen, P., and Zhang, Z. (2009). A Forecasting Model of City Freight Volume Based on BPNN. The Sixth ISNN 2009, AISC 56, pp. 791-798

Yang, Y., and Yu, C. (2015). Prediction models based on multivariate statistical methods and their applications for predicting railway freight volume. Neurocomputing 158 (2015) pp. 210-215.

Ying, Q., Huapu, L., Haiwei, W. (2008). Prediction Method for Regional Logistics. Tsinghua Science and Technology. ISSN 1007-0214 14/25, Volume 13, Number 5, pp. 660-668.

Categories: Analytics Tags: