Convolutional neural networks for modeling and forecasting nonlinear nonstationary processes

Keywords: mathematical modeling, signal processing, time-series, nonstationary processes, convolutional neural networks, recurrent neural networks

Abstract

The object of research. The object of research is modeling and forecasting nonlinear nonstationary processes presented in the form of time-series data.

Investigated problem. There are several popular approaches to solving the problems of adequate model constructing and forecasting nonlinear nonstationary processes, such as autoregressive models and recurrent neural networks. However, each of them has its advantages and drawbacks. Autoregressive models cannot deal with the nonlinear or combined influence of previous states or external factors. Recurrent neural networks are computationally expensive and cannot work with sequences of high length or frequency.

The main scientific result. The model for forecasting nonlinear nonstationary processes presented in the form of the time series data was built using convolutional neural networks. The current study shows results in which convolutional networks are superior to recurrent ones in terms of both accuracy and complexity. It was possible to build a more accurate model with a much fewer number of parameters. It indicates that one-dimensional convolutional neural networks can be a quite reasonable choice for solving time series forecasting problems.

The area of practical use of the research results. Forecasting dynamics of processes in economy, finances, ecology, healthcare, technical systems and other areas exhibiting the types of nonlinear nonstationary processes.

Innovative technological product. Methodology of using convolutional neural networks for modeling and forecasting nonlinear nonstationary processes presented in the form of time-series data.

Scope of the innovative technological product. Nonlinear nonstationary processes presented in the form of time-series data.

Downloads

Download data is not yet available.

Author Biographies

Andrii Belas, National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic institute”

Department of Mathematical methods for System Analysis

Institute for applied system analysis

Petro Bidyuk, National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic institute”

Department of Mathematical methods for System Analysis

Institute for applied system analysis

References

Palit, A., Popovic, D. (2005). Computational intelligence in time series forecasting: theory and engineering applications. Springer Science & Business Media, 372. doi: http://doi.org/10.1007/1-84628-184-9

Bidyuk P., Romanenko V., Timoschuk O. (2010). Analysis of time series. Kyiv: NTUU «KPI».

Hyndman R., Athanasopoulos G. (2013). Forecasting: Principles and Practice. OTexts.

Belas, O., Bidiuk, P. Belas, A. (2019). Comparative analysis of autoregressive approaches and recurrent neural networks for modeling and forecasting nonlinear nonstationary processes. Information Technology and Security, 7 (1), 91–99. doi: http://doi.org/10.20535/2411-1031.2019.7.1.184395

Gers, F. A., Eck, D., Schmidhuber, J. (2001). Applying LSTM to Time Series Predictable through Time-Window Approaches. Proceedings of International Conference on Artificial Neural Networks, 669–676. doi: http://doi.org/10.1007/3-540-44668-0_93

LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., Jackel, L. D. (1989). Backpropagation Applied to Handwritten Zip Code Recognition. Neural Computation, 1 (4), 541–551. doi: http://doi.org/10.1162/neco.1989.1.4.541

Goodfellow I., Bengio, J., Courville, A. (2018). Deep learning. МIT Press.

He, K., Zhang, X., Ren, S., Sun, J. (2016). Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). doi: http://doi.org/10.1109/cvpr.2016.90

Hochreiter, S., Schmidhuber, J. (1997). Long Short-Term Memory. Neural Computation, 9 (8), 1735–1780. doi: https://doi.org/10.1162/neco.1997.9.8.1735

Hochreiter S., Bengio Y., Schmidhuber, J. (2001) Gradient flow in recurrent nets: the difficulty of learning long-term dependencies. IEEE Press.

Nikolenko, S., Kadurin, А., Arkhangelskaya, Е. (2018). Deep learning. Saint Petersburg: Peter, 479.

Chollet, F. (2017). Deep Learning with R. Manning. Black & White, 384.

Zeiler, M. D., Fergus, R. (2014). Visualizing and Understanding Convolutional Networks. ECCV Press, 818–833. doi: http://doi.org/10.1007/978-3-319-10590-1_53

Lecun, Y., Bottou, L., Bengio, Y., Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86 (11), 2278–2324. doi: http://doi.org/10.1109/5.726791

Krizhevsky, A., Sutskev I., Hinton, J. (2012). Imagenet classification with deep convolutional neural networks. NIPS, 1106–1114.

LeCun, Y., Kavukcuoglu, K., Farabet, C. (2010). Convolutional networks and applications in vision. Proceedings of 2010 IEEE International Symposium on Circuits and Systems, 2 (4), 253–256. doi: http://doi.org/10.1109/iscas.2010.5537907

Walmart. (2014). Walmart Recruiting - Store Sales Forecasting [Dataset]. https://www.kaggle.com/c/walmart-recruiting-store-sales-forecasting/data.

Laptev, N., Smyl, S., Shanmugam, S. (2017). Engineering Extreme Event Forecasting at Uber with Recurrent Neural Networks. Uber Engineering. Available at: http://roseyu.com/time-series-workshop/submissions/TSW2017_paper_3.pdf

Belas, O., Belas, A. (2021). General methods of forecasting nonlinear nonstationary processes based on mathematical models using statistical data. System research and information technologies, 1 (1), 79–86.

Bergmeir, C., Hyndman, R. J., Benítez, J. M. (2016). Bagging exponential smoothing methods using STL decomposition and Box–Cox transformation. International Journal of Forecasting, 32 (2), 303–312. doi: http://doi.org/10.1016/j.ijforecast.2015.07.002

De Livera, A. M., Hyndman, R. J., Snyder, R. D. (2011). Forecasting Time Series With Complex Seasonal Patterns Using Exponential Smoothing. Journal of the American Statistical Association, 106 (496), 1513–1527. doi: http://doi.org/10.1198/jasa.2011.tm09771

Sagheer, A., Kotb, M. (2019). Time series forecasting of petroleum production using deep LSTM recurrent networks. Neurocomputing, 323, 203–213. doi: http://doi.org/10.1016/j.neucom.2018.09.082

Chimmula, V. K. R., Zhang, L. (2020). Time series forecasting of COVID-19 transmission in Canada using LSTM networks. Chaos, Solitons & Fractals, 135, 109864. doi: http://doi.org/10.1016/j.chaos.2020.109864

Colon, S., Gil, J. (2019). Data Mining Techniques and Machine Learning Model for Walmart Weekly Sales Forecast. Puerto Rico. Available at: https://prcrepository.org/xmlui/bitstream/handle/20.500.12475/174/FA-19_Articulo %20Final_Jose %20Santaella.pdf

Elias, N., Singh, S. (2018). Forecasting of Walmart sales using machine learning algorithms. Available at: https://api.semanticscholar.org/CorpusID:209465807


👁 30
⬇ 22
Published
2021-06-30
How to Cite
Belas, A., & Bidyuk, P. (2021). Convolutional neural networks for modeling and forecasting nonlinear nonstationary processes. ScienceRise, (3), 12-20. https://doi.org/10.21303/2313-8416.2021.001924
Section
Innovative technologies in industry