Kapılı Tekrarlayan Hücreler Tabanlı Bulanık Zaman Serileri Tahminleme Modeli

Zaman serisi tahminleme hava durumu, iş dünyası, satış verileri ve enerji tüketimi tahminleme gibi bir çok alanda uygulama alanına sahiptir. Bu alanlarda tahminleme yaparken kesin sonuçlar elde etmek çok önemlidir ama aynı zamanda zaman serilerinin karmaşık ve de belirsizlik içeren veriler olması nedeniyle çok zordur. Günümüzde, derin öğrenme metotları bu alanda klasik metotlara göre daha iyi sonuçlar vermektedir. Fakat literatürde bulanık zaman serileri tahminleme konusunda çok az çalışma vardır. Bu çalışmada, zaman serilerindeki karmaşıklığın ve belirsizliğin doğurduğu problemleri yok etmek için Yinelemeli sinir Ağları ile bulanık time serilerini bir arada kullanan bir model ortaya konmuştur. Bu çalışmada, Kapılı Tekrarlayan Hücreler kullanarak geçmiş veriler ile bulanık verilerin üyelik değerleri birleştirilerek tahminleme değeri hesaplanmıştır. Ayrıca, bu çalışmadaki model ilk seviye bulanık ilişkileri ele alabildiği gibi, çoklu seviye bulanık ilişkileri de kapsamaktadır. Testlerde literatürde var olan çalışmalar ilgili model ile iki açık veri seti ile karşılaştırılmış olup bahsi geçen modelin daha iyi veya benzer sonuçlar verdiği gözlemlenmiştir. Ayrıca model Covid-19 verileri kullanılarak da test edilmiş ve Uzun-Kısa Süreli Bellek modellerinden daha iyi sonuç vermiştir.

Anahtar Kelimeler:

Kapılı Tekrarlayan Hücreler, Zaman Serisi Tahminleme, Bulanık Zaman Serisi, Derin Öğrenme

Gated recurrent unit network-based fuzzy time series forecasting model

Time series forecasting has lots of applications in various industries such as weather, business, retail and energy consumption forecasting. Accurate prediction in these applications is very important and also difficult task because of complexity and uncertainty of time series. Nowadays, using deep learning methods is a popular approach in time series forecasting and shows better performance than classical methods. However, in the literature, there are few studies which use deep learning methods in fuzzy time series (FTS) forecasting. In this study, we propose a novel FTS forecasting model which is based upon hybridization of Recurrent Neural Networks with FTS to deal with complexity and also uncertanity of these series. The proposed model utilizes Gated Recurrent Unit (GRU) to make prediction by using combination of membership values and also past value from original time series data as model input and produce real forecast value. Moreover, the proposed model can handle first order fuzzy relations as well as high order ones. In experiments, we have compared our model results with those of state-of-art methods by using two real world datasets; The Taiwan Stock Exchange Capitalization Weighted Stock Index (TAIEX) and Nikkei Stock Average. The results indicate that our model outperforms or performs similar to other methods. The proposed model is also validated by using Covid-19 active case dataset and shows better performance than Long Short-term Memory (LSTM) networks.

Keywords:

Gated Recurrent Unit, Time Series Forecasting, Fuzzy Time Series, Deep Learning,

PDF

___

Aladag, Cagdas H., Basaran, M. A., Egrioglu, E., Yolcu, U., & Uslu, V. R. (2009). Forecasting in high order fuzzy times series by using neural networks to define fuzzy relations. Expert Systems with Applications, 36(3 PART 1), 4228–4231. https://doi.org/10.1016/j.eswa.2008.04.001
Aladag, Cagdas H, Basaran, M. A., Egrioglu, E., Yolcu, U., & Uslu, V. R. (2009). Forecasting in high order fuzzy times series by using neural networks to define fuzzy relations. Expert Systems with Applications, 36(3, Part 1), 4228–4231. https://doi.org/https://doi.org/10.1016/j.eswa.2008.04.001
Aladag, Cagdas Hakan. (2013). Using multiplicative neuron model to establish fuzzy logic relationships. Expert Systems with Applications, 40(3), 850–853. https://doi.org/10.1016/j.eswa.2012.05.039
Aladag, Cagdas Hakan, Yolcu, U., & Egrioglu, E. (2010a). A high order fuzzy time series forecasting model based on adaptive expectation and artificial neural networks. Mathematics and Computers in Simulation, 81(4), 875–882. https://doi.org/https://doi.org/10.1016/j.matcom.2010.09.011
Aladag, Cagdas Hakan, Yolcu, U., & Egrioglu, E. (2010b). A high order fuzzy time series forecasting model based on adaptive expectation and artificial neural networks. Mathematics and Computers in Simulation, 81(4), 875–882. https://doi.org/https://doi.org/10.1016/j.matcom.2010.09.011
Aladag, Cagdas Hakan, Yolcu, U., Egrioglu, E., & Dalar, A. Z. (2012). A new time invariant fuzzy time series forecasting method based on particle swarm optimization. Applied Soft Computing, 12(10), 3291–3299. https://doi.org/https://doi.org/10.1016/j.asoc.2012.05.002
Bahrepour, M., Akbarzadeh-T., M. R., Yaghoobi, M., & Naghibi-S., M. B. (2011). An adaptive ordered fuzzy time series with application to FOREX. Expert Systems with Applications, 38(1), 475–485. https://doi.org/10.1016/j.eswa.2010.06.087
Bas, E., Egrioglu, E., Aladag, C. H., & Yolcu, U. (2015). Fuzzy-time-series network used to forecast linear and nonlinear time series. Applied Intelligence, 43(2), 343–355. https://doi.org/10.1007/s10489-015-0647-0
Bas, E., Grosan, C., Egrioglu, E., & Yolcu, U. (2018). High order fuzzy time series method based on pi-sigma neural network. Engineering Applications of Artificial Intelligence, 72, 350–356. https://doi.org/https://doi.org/10.1016/j.engappai.2018.04.017
Becerra-Rico, J., Aceves-Fernández, M. A., Esquivel-Escalante, K., & Pedraza-Ortega, J. C. (2020). Airborne particle pollution predictive model using Gated Recurrent Unit (GRU) deep neural networks. Earth Science Informatics, 13(3), 821–834. https://doi.org/10.1007/s12145-020-00462-9
Bose, M., & Mali, K. (2019). Designing fuzzy time series forecasting models: A survey. International Journal of Approximate Reasoning, 111, 78–99. https://doi.org/10.1016/j.ijar.2019.05.002
Bulut, E. (2014). Modeling seasonality using the fuzzy integrated logical forecasting (FILF) approach. Expert Systems with Applications, 41(4, Part 2), 1806–1812. https://doi.org/https://doi.org/10.1016/j.eswa.2013.08.079
Cagcag Yolcu, O., & Lam, H. K. (2017). A combined robust fuzzy time series method for prediction of time series. Neurocomputing, 247, 87–101. https://doi.org/10.1016/j.neucom.2017.03.037
Cai, Q., Zhang, D., Zheng, W., & Leung, S. C. H. (2015). A new fuzzy time series forecasting model combined with ant colony optimization and auto-regression. Knowledge-Based Systems, 74, 61–68. https://doi.org/https://doi.org/10.1016/j.knosys.2014.11.003
Castillo, O., Alanis, A., Garcia, M., & Arias, H. (2007). An intuitionistic fuzzy system for time series analysis in plant monitoring and diagnosis. Applied Soft Computing, 7(4), 1227–1233. https://doi.org/https://doi.org/10.1016/j.asoc.2006.01.010
Chang, J.-R., Wei, L.-Y., & Cheng, C.-H. (2011). A hybrid ANFIS model based on AR and volatility for TAIEX forecasting. Applied Soft Computing, 11(1), 1388–1395. https://doi.org/https://doi.org/10.1016/j.asoc.2010.04.010
Chen, M.-Y., & Chen, B.-T. (2014). Online fuzzy time series analysis based on entropy discretization and a Fast Fourier Transform. Applied Soft Computing, 14, 156–166. https://doi.org/https://doi.org/10.1016/j.asoc.2013.07.024
Chen, S.-M., & Kao, P.-Y. (2013). TAIEX forecasting based on fuzzy time series, particle swarm optimization techniques and support vector machines. Information Sciences, 247, 62–71. https://doi.org/https://doi.org/10.1016/j.ins.2013.06.005
Chen, S.-M., & Tanuwijaya, K. (2011). Fuzzy forecasting based on high-order fuzzy logical relationships and automatic clustering techniques. Expert Systems with Applications, 38(12), 15425–15437. https://doi.org/https://doi.org/10.1016/j.eswa.2011.06.019
Chen, S. M. (1996). Forecasting enrollments based on fuzzy time series. Fuzzy Sets and Systems, 81(3), 311–319. https://doi.org/10.1016/0165-0114(95)00220-0
Cho, K., Merrienboer, B. van, Gülçehre, Ç., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. BT - Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, (pp. 1724–1734). https://doi.org/10.3115/v1/d14-1179
Cui, Z., & Wang, Y. (2017). Deep Stacked Bidirectional and Unidirectional LSTM Recurrent Neural Network for Network-wide Traffic Speed Prediction.
De Gooijer, J. G., & Hyndman, R. J. (2006). 25 years of time series forecasting. International Journal of Forecasting, 22(3), 443–473. https://doi.org/https://doi.org/10.1016/j.ijforecast.2006.01.001
Dutta, A., Kumar, S., & Basu, M. (2020). A Gated Recurrent Unit Approach to Bitcoin Price Prediction. In Journal of Risk and Financial Management (Vol. 13, Issue 2). https://doi.org/10.3390/jrfm13020023
Efendi, R., Ismail, Z., & Deris, M. M. (2015). A new linguistic out-sample approach of fuzzy time series for daily forecasting of Malaysian electricity load demand. Applied Soft Computing, 28, 422–430. https://doi.org/https://doi.org/10.1016/j.asoc.2014.11.043
Egrioglu, E., Aladag, C. H., & Yolcu, U. (2013). Fuzzy time series forecasting with a novel hybrid approach combining fuzzy c-means and neural networks. Expert Systems with Applications, 40(3), 854–857. https://doi.org/https://doi.org/10.1016/j.eswa.2012.05.040
Egrioglu, E., Aladag, C. H., Yolcu, U., Uslu, V. R., & Basaran, M. A. (2009). A new approach based on artificial neural networks for high order multivariate fuzzy time series. Expert Systems with Applications, 36(7), 10589–10594. https://doi.org/https://doi.org/10.1016/j.eswa.2009.02.057
George E. P. Box, G. M. J. G. C. R. G. M. L. (2015). Time Series Analysis: Forecasting and Control. John Wiley & Sons.
Graves, A., Mohamed, A., & Hinton, G. E. (2013). Speech Recognition with Deep Recurrent Neural Networks. CoRR, abs/1303.5. http://arxiv.org/abs/1303.5778
Hájek, P., & Olej, V. (2017). Intuitionistic neuro-fuzzy network with evolutionary adaptation. Evolving Systems, 8(1), 35–47. https://doi.org/10.1007/s12530-016-9157-5
Huarng, K. (2001a). Effective lengths of intervals to improve forecasting in fuzzy time series. Fuzzy Sets and Systems, 123(3), 387–394. https://doi.org/https://doi.org/10.1016/S0165-0114(00)00057-9
Huarng, K. (2001b). Heuristic models of fuzzy time series for forecasting. Fuzzy Sets and Systems, 123(3), 369–386. https://doi.org/https://doi.org/10.1016/S0165-0114(00)00093-2
Huarng, K., & Yu, T. H. K. (2006). The application of neural networks to forecast fuzzy time series. Physica A: Statistical Mechanics and Its Applications, 363(2), 481–491. https://doi.org/10.1016/j.physa.2005.08.014
Jang, J.-. R. (1993). ANFIS: adaptive-network-based fuzzy inference system. IEEE Transactions on Systems, Man, and Cybernetics, 23(3), 665–685. https://doi.org/10.1109/21.256541
Jozefowicz, R., Zaremba, W., & Sutskever, I. (2015). An empirical exploration of Recurrent Network architectures. 32nd International Conference on Machine Learning, ICML 2015, 3, 2332–2340.
Kocak, C. (2017). ARMA(p,q) type high order fuzzy time series forecast method based on fuzzy logic relations. Applied Soft Computing, 58, 92–103. https://doi.org/https://doi.org/10.1016/j.asoc.2017.04.021
Kocak, C., Egrioglu, E., & Bas, E. (2021). A new deep intuitionistic fuzzy time series forecasting method based on long short-term memory. Journal of Supercomputing, 77(6), 6178–6196. https://doi.org/10.1007/s11227-020-03503-8
Lee, L.-W., Wang, L.-H., & Chen, S.-M. (2008). Temperature prediction and TAIFEX forecasting based on high-order fuzzy logical relationships and genetic simulated annealing techniques. Expert Systems with Applications, 34(1), 328–336. https://doi.org/https://doi.org/10.1016/j.eswa.2006.09.007
Li, S.-T., Cheng, Y.-C., & Lin, S.-Y. (2008). A FCM-based deterministic forecasting model for fuzzy time series. Computers & Mathematics with Applications, 56(12), 3052–3063. https://doi.org/https://doi.org/10.1016/j.camwa.2008.07.033
Lin, T., Horne, B. G., & Giles, C. L. (1998). How embedded memory in recurrent neural network architectures helps learning long-term temporal dependencies (Vol. 11, Issue 5, pp. 861–868). https://www.scopus.com/inward/record.uri?eid=2-s2.0-0032123428&doi=10.1016%2FS0893-6080%2898%2900018-5&partnerID=40&md5=ab7519875b4103882070c5a56a6c7a1e
Lindemann, B., Müller, T., Vietz, H., Jazdi, N., & Weyrich, M. (2021). A survey on long short-term memory networks for time series prediction. Procedia CIRP, 99, 650–655. https://doi.org/https://doi.org/10.1016/j.procir.2021.03.088
Martín Abadi, Ashish Agarwal, Paul Barham, E. B., Zhifeng Chen, Craig Citro, Greg S. Corrado, A. D., Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, I. G., Andrew Harp, Geoffrey Irving, Michael Isard, Rafal Jozefowicz, Y. J., Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dan Mané, M. S., Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, J. S., Benoit Steiner, Ilya Sutskever, Kunal Talwar, P. T., Vincent Vanhoucke, Vijay Vasudevan, F. V., Oriol Vinyals, Pete Warden, Martin Wattenberg, M. W., & Yuan Yu, and X. Z. (n.d.). TensorFlow:Large-Scale Machine Learning on Heterogeneous Systems. https://www.tensorflow.org/
Nie, J. (1997). Nonlinear time-series forecasting: A fuzzy-neural approach. Neurocomputing, 16(1), 63–76. https://doi.org/https://doi.org/10.1016/S0925-2312(97)00019-2
Novák, V. (1995). Towards Formalized Integrated Theory of Fuzzy Logic BT - Fuzzy Logic and its Applications to Engineering, Information Sciences, and Intelligent Systems (Z. Bien & K. C. Min (eds.); pp. 353–363). Springer Netherlands. https://doi.org/10.1007/978-94-009-0125-4_35
Panigrahi, S., & Behera, H. S. (2020). A study on leading machine learning techniques for high order fuzzy time series forecasting. Engineering Applications of Artificial Intelligence, 87, 103245. https://doi.org/10.1016/j.engappai.2019.103245
Sadaei, H J, Enayatifar, R., Guimarães, F. G., Mahmud, M., & Alzamil, Z. A. (2016). Combining ARFIMA models and fuzzy time series for the forecast of long memory time series (Vol. 175, pp. 782–796). https://www.scopus.com/inward/record.uri?eid=2-s2.0-84963768497&doi=10.1016%2Fj.neucom.2015.10.079&partnerID=40&md5=17238b9bbacc56abf01b11050b134966
Sadaei, Hossein Javedani, de Lima e Silva, P. C., Guimarães, F. G., & Lee, M. H. (2019). Short-term load forecasting by using a combined method of convolutional neural networks and fuzzy time series. Energy, 175, 365–377. https://doi.org/10.1016/j.energy.2019.03.081
Schuster, M., & Paliwal, K. K. (1997). Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11), 2673–2681. https://doi.org/10.1109/78.650093
Shen, G., Tan, Q., Zhang, H., Zeng, P., & Xu, J. (2018a). Deep Learning with Gated Recurrent Unit Networks for Financial Sequence Predictions. Procedia Computer Science, 131, 895–903. https://doi.org/https://doi.org/10.1016/j.procs.2018.04.298
Shen, G., Tan, Q., Zhang, H., Zeng, P., & Xu, J. (2018b). Deep Learning with Gated Recurrent Unit Networks for Financial Sequence Predictions. Procedia Computer Science, 131, 895–903. https://doi.org/https://doi.org/10.1016/j.procs.2018.04.298
Siami-Namini, S., Tavakoli, N., & Namin, A. S. (2019). The Performance of LSTM and BiLSTM in Forecasting Time Series. 2019 IEEE International Conference on Big Data (Big Data), 3285–3292. https://doi.org/10.1109/BigData47090.2019.9005997 Singh, P. (2017). A brief review of modeling approaches based on fuzzy time series. International Journal of Machine Learning and Cybernetics, 8(2), 397–420. https://doi.org/10.1007/s13042-015-0332-y
Singh, P., & Borah, B. (2013). High-order fuzzy-neuro expert system for time series forecasting. Knowledge-Based Systems, 46, 12–21. https://doi.org/10.1016/j.knosys.2013.01.030
Singh, P., & Borah, B. (2014). An effective neural network and fuzzy time series-based hybridized model to handle forecasting problems of two factors. Knowledge and Information Systems, 38(3), 669–690. https://doi.org/10.1007/s10115-012-0603-9
Song, Q., & Chissom, B. S. (1993). Fuzzy time series and its models. Fuzzy Sets and Systems, 54(3), 269–277. https://doi.org/https://doi.org/10.1016/0165-0114(93)90372-O
Stefanakos, C. (2016). Fuzzy time series forecasting of nonstationary wind and wave data. Ocean Engineering, 121, 1–12. https://doi.org/https://doi.org/10.1016/j.oceaneng.2016.05.018
Sullivan, J., & Woodall, W. H. (1994). A comparison of fuzzy forecasting and Markov modeling. Fuzzy Sets and Systems, 64(3), 279–293. https://doi.org/https://doi.org/10.1016/0165-0114(94)90152-X
Tan, Q., Ye, M., Yang, B., Liu, S., Ma, A. J., Yip, T. C.-F., Wong, G. L.-H., & Yuen, P. (2020). DATA-GRU: Dual-Attention Time-Aware Gated Recurrent Unit for Irregular Multivariate Time Series. Proceedings of the AAAI Conference on Artificial Intelligence, 34(01), 930–937. https://doi.org/10.1609/aaai.v34i01.5440
Torbat, S., Khashei, M., & Bijari, M. (2018). A hybrid probabilistic fuzzy ARIMA model for consumption forecasting in commodity markets. Economic Analysis and Policy, 58, 22–31. https://doi.org/https://doi.org/10.1016/j.eap.2017.12.003
Tran, N., Nguyen, T., Nguyen, B. M., & Nguyen, G. (2018). A Multivariate Fuzzy Time Series Resource Forecast Model for Clouds using LSTM and Data Correlation Analysis. Procedia Computer Science, 126, 636–645. https://doi.org/10.1016/j.procS.2018.07.298
Tsaur, R.-C., O Yang, J.-C., & Wang, H.-F. (2005). Fuzzy relation analysis in fuzzy time series model. Computers & Mathematics with Applications, 49(4), 539–548. https://doi.org/https://doi.org/10.1016/j.camwa.2004.07.014
Tseng, F.-M., Tzeng, G.-H., Yu, H.-C., & Yuan, B. J. C. (2001). Fuzzy ARIMA model for forecasting the foreign exchange market. Fuzzy Sets and Systems, 118(1), 9–19. https://doi.org/https://doi.org/10.1016/S0165-0114(98)00286-3
Wang, Y., Liao, W., & Chang, Y. (2018). Gated Recurrent Unit Network-Based Short-Term Photovoltaic Forecasting. In Energies (Vol. 11, Issue 8). https://doi.org/10.3390/en11082163
Wong, H.-L., Tu, Y.-H., & Wang, C.-C. (2010). Application of fuzzy time series models for forecasting the amount of Taiwan export. Expert Systems with Applications, 37(2), 1465–1470. https://doi.org/https://doi.org/10.1016/j.eswa.2009.06.106
Yang, S., Yu, X., Zhou, Y., & Yu, X. (2020). LSTM and GRU Neural Network Performance Comparison Study: Taking Yelp Review Dataset as an Example. Proceedings - 2020 International Workshop on Electronic Communication and Artificial Intelligence, IWECAI 2020, 98–101. https://doi.org/10.1109/IWECAI50956.2020.00027
Yu, T. H. K., & Huarng, K. H. (2010). A neural network-based fuzzy time series model to improve forecasting. Expert Systems with Applications, 37(4), 3366–3372. https://doi.org/10.1016/j.eswa.2009.10.013
Zadeh, L. A. (1965). Fuzzy sets. Information and Control, 8(3), 338–353. https://doi.org/https://doi.org/10.1016/S0019-9958(65)90241-X