Convolutional neural networks for modeling and forecasting nonlinear nonstationary processes
The object of research. The object of research is modeling and forecasting nonlinear nonstationary processes presented in the form of time-series data.
Investigated problem. There are several popular approaches to solving the problems of adequate model constructing and forecasting nonlinear nonstationary processes, such as autoregressive models and recurrent neural networks. However, each of them has its advantages and drawbacks. Autoregressive models cannot deal with the nonlinear or combined influence of previous states or external factors. Recurrent neural networks are computationally expensive and cannot work with sequences of high length or frequency.
The main scientific result. The model for forecasting nonlinear nonstationary processes presented in the form of the time series data was built using convolutional neural networks. The current study shows results in which convolutional networks are superior to recurrent ones in terms of both accuracy and complexity. It was possible to build a more accurate model with a much fewer number of parameters. It indicates that one-dimensional convolutional neural networks can be a quite reasonable choice for solving time series forecasting problems.
The area of practical use of the research results. Forecasting dynamics of processes in economy, finances, ecology, healthcare, technical systems and other areas exhibiting the types of nonlinear nonstationary processes.
Innovative technological product. Methodology of using convolutional neural networks for modeling and forecasting nonlinear nonstationary processes presented in the form of time-series data.
Scope of the innovative technological product. Nonlinear nonstationary processes presented in the form of time-series data.
Palit, A., Popovic, D. (2005). Computational intelligence in time series forecasting: theory and engineering applications. Springer Science & Business Media, 372. doi: http://doi.org/10.1007/1-84628-184-9
Bidyuk P., Romanenko V., Timoschuk O. (2010). Analysis of time series. Kyiv: NTUU «KPI».
Hyndman R., Athanasopoulos G. (2013). Forecasting: Principles and Practice. OTexts.
Belas, O., Bidiuk, P. Belas, A. (2019). Comparative analysis of autoregressive approaches and recurrent neural networks for modeling and forecasting nonlinear nonstationary processes. Information Technology and Security, 7 (1), 91–99. doi: http://doi.org/10.20535/2411-1031.2019.7.1.184395
Gers, F. A., Eck, D., Schmidhuber, J. (2001). Applying LSTM to Time Series Predictable through Time-Window Approaches. Proceedings of International Conference on Artificial Neural Networks, 669–676. doi: http://doi.org/10.1007/3-540-44668-0_93
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., Jackel, L. D. (1989). Backpropagation Applied to Handwritten Zip Code Recognition. Neural Computation, 1 (4), 541–551. doi: http://doi.org/10.1162/neco.19184.108.40.2061
Goodfellow I., Bengio, J., Courville, A. (2018). Deep learning. МIT Press.
He, K., Zhang, X., Ren, S., Sun, J. (2016). Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). doi: http://doi.org/10.1109/cvpr.2016.90
Hochreiter, S., Schmidhuber, J. (1997). Long Short-Term Memory. Neural Computation, 9 (8), 1735–1780. doi: https://doi.org/10.1162/neco.19220.127.116.115
Hochreiter S., Bengio Y., Schmidhuber, J. (2001) Gradient flow in recurrent nets: the difficulty of learning long-term dependencies. IEEE Press.
Nikolenko, S., Kadurin, А., Arkhangelskaya, Е. (2018). Deep learning. Saint Petersburg: Peter, 479.
Chollet, F. (2017). Deep Learning with R. Manning. Black & White, 384.
Zeiler, M. D., Fergus, R. (2014). Visualizing and Understanding Convolutional Networks. ECCV Press, 818–833. doi: http://doi.org/10.1007/978-3-319-10590-1_53
Lecun, Y., Bottou, L., Bengio, Y., Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86 (11), 2278–2324. doi: http://doi.org/10.1109/5.726791
Krizhevsky, A., Sutskev I., Hinton, J. (2012). Imagenet classification with deep convolutional neural networks. NIPS, 1106–1114.
LeCun, Y., Kavukcuoglu, K., Farabet, C. (2010). Convolutional networks and applications in vision. Proceedings of 2010 IEEE International Symposium on Circuits and Systems, 2 (4), 253–256. doi: http://doi.org/10.1109/iscas.2010.5537907
Walmart. (2014). Walmart Recruiting - Store Sales Forecasting [Dataset]. https://www.kaggle.com/c/walmart-recruiting-store-sales-forecasting/data.
Laptev, N., Smyl, S., Shanmugam, S. (2017). Engineering Extreme Event Forecasting at Uber with Recurrent Neural Networks. Uber Engineering. Available at: http://roseyu.com/time-series-workshop/submissions/TSW2017_paper_3.pdf
Belas, O., Belas, A. (2021). General methods of forecasting nonlinear nonstationary processes based on mathematical models using statistical data. System research and information technologies, 1 (1), 79–86.
Bergmeir, C., Hyndman, R. J., Benítez, J. M. (2016). Bagging exponential smoothing methods using STL decomposition and Box–Cox transformation. International Journal of Forecasting, 32 (2), 303–312. doi: http://doi.org/10.1016/j.ijforecast.2015.07.002
De Livera, A. M., Hyndman, R. J., Snyder, R. D. (2011). Forecasting Time Series With Complex Seasonal Patterns Using Exponential Smoothing. Journal of the American Statistical Association, 106 (496), 1513–1527. doi: http://doi.org/10.1198/jasa.2011.tm09771
Sagheer, A., Kotb, M. (2019). Time series forecasting of petroleum production using deep LSTM recurrent networks. Neurocomputing, 323, 203–213. doi: http://doi.org/10.1016/j.neucom.2018.09.082
Chimmula, V. K. R., Zhang, L. (2020). Time series forecasting of COVID-19 transmission in Canada using LSTM networks. Chaos, Solitons & Fractals, 135, 109864. doi: http://doi.org/10.1016/j.chaos.2020.109864
Colon, S., Gil, J. (2019). Data Mining Techniques and Machine Learning Model for Walmart Weekly Sales Forecast. Puerto Rico. Available at: https://prcrepository.org/xmlui/bitstream/handle/20.500.12475/174/FA-19_Articulo %20Final_Jose %20Santaella.pdf
Elias, N., Singh, S. (2018). Forecasting of Walmart sales using machine learning algorithms. Available at: https://api.semanticscholar.org/CorpusID:209465807
Copyright (c) 2021 Andrii Belas, Petro Bidyuk
This work is licensed under a Creative Commons Attribution 4.0 International License.
Our journal abides by the Creative Commons CC BY copyright rights and permissions for open access journals.
Authors, who are published in this journal, agree to the following conditions:
1. The authors reserve the right to authorship of the work and pass the first publication right of this work to the journal under the terms of a Creative Commons CC BY, which allows others to freely distribute the published research with the obligatory reference to the authors of the original work and the first publication of the work in this journal.
2. The authors have the right to conclude separate supplement agreements that relate to non-exclusive work distribution in the form in which it has been published by the journal (for example, to upload the work to the online storage of the journal or publish it as part of a monograph), provided that the reference to the first publication of the work in this journal is included.