A Modified Energy Demand Forecasting Model using Hybrid CNN-LSTM with Transformer for Univariate Time Series
DOI:
https://doi.org/10.22399/ijcesen.2293Keywords:
Energy Demand Forecasting, Univariate Time Series, CNN-LSTM Model, Transformer Layer, Forecasting AccuracyAbstract
Precise energy demand forecasting is important in managing electrical power systems, particularly if univariate time series analysis can be applied. To overcome the shortcomings of traditional hybrid models, this paper proposes an improved deep learning architecture that combines Transformer layers, Long Short-Term Memory (LSTM), and Convolutional Neural Networks (CNN). The proposed architecture was trained and validated on historical hourly energy demand data from 2015 to 2018. Performance evaluation revealed that the CNN-LSTM-Transformer model significantly improved forecasting accuracy compared to the baseline CNN-LSTM model. Specifically, the hybrid model achieved a Mean Absolute Error (MAE) of 234.25, Root Mean Squared Error (RMSE) of 386.15, and Mean Absolute Percentage Error (MAPE) of 0.84%, alongside an R² score of 99.28%. These results confirm the model’s robustness in capturing both local temporal dynamics and long-range dependencies, making it a promising solution for real-time energy forecasting applications.
References
[1] Cui, B., Liu, M., Li, S., Jin, Z., Zeng, Y., & Lin, X. (2023). Deep learning methods for atmospheric PM₂.₅ prediction: A comparative study of transformer and CNN-LSTM-attention. Atmospheric Pollution Research, 14(9), 101833. https://doi.org/10.1016/j.apr.2023.101833
[2] Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Santamaría, J., Fadhel, M. A., Al-Amidie, M., & Farhan, L. (2021). Review of deep learning: Concepts, CNN architectures, challenges, applications, future directions. Journal of Big Data, 8, 53. https://doi.org/10.1186/s40537-021-00444-8
[3]. Kaya, M., Utku, A., & Canbay, Y. (2024). A Hybrid CNN-LSTM Model for Predicting Energy Consumption and Production Across Multiple Energy Sources. Journal of Soft Computing and Artificial Intelligence, 5(2), 63-73. https://doi.org/10.55195/jscai.1577431
[4] Yıldız Doğan, G., Aksoy, A., & Öztürk, N. (2024). A Hybrid Deep Learning Model to Estimate the Future Electricity Demand of Sustainable Cities. Sustainability, 16(15), 6503. https://doi.org/10.3390/su16156503
[5] Guo, W., Liu, S., Weng, L., & Liang, X. (2025). Power Grid Load Forecasting Using a CNN-LSTM Network Based on a Multi-Modal Attention Mechanism. Applied Sciences, 15(5), 2435. https://doi.org/10.3390/app15052435
[6] Wu, H., Xu, J., Wang, J., & Long, M. (2021). Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting. Advances in Neural Information Processing Systems, 34, 22419–22430. https://proceedings.neurips.cc/paper_files/paper/2021/hash/9d86d84eec0f2cd5f2b8f2f707b6b4c1-Abstract.html
[7] Yang, F., Liu, Z., Yan, J., & Geng, Y. (2022). CNN-LSTM neural network with attention mechanism for fault diagnosis of high voltage circuit breakers. In 18th International Conference on AC and DC Power Transmission (ACDC 2022) (pp. 1102–1107). IET. https://doi.org/10.1049/icp.2022.1357
[8] Binbusayyis, A., & Sha, M. (2025). Energy consumption prediction using modified deep CNN-Bi LSTM with attention mechanism. Heliyon, 11(1), e41507. https://doi.org/10.1016/j.heliyon.2024.e41507
[9] K. Roy, A. Ishmam and K. A. Taher, "Demand Forecasting in Smart Grid Using Long Short-Term Memory," 2021 International Conference on Automation, Control and Mechatronics for Industry 4.0 (ACMI), Rajshahi, Bangladesh, 2021, pp. 1-5, doi: 10.1109/ACMI53878.2021.9528277
[10] Lee, K., Ippolito, D., Nystrom, A., Zhang, C., Eck, D., Callison-Burch, C., & Carlini, N. (2021). Deduplicating training data makes language models better. arXiv preprint arXiv:2107.06499. https://arxiv.org/abs/2107.06499
[11] Aghabagherloo, A., Abadi, A., Sarkar, S., Dasu, V. A., & Preneel, B. (2025). Impact of data duplication on deep neural network-based image classifiers: Robust vs. standard models. arXiv preprint arXiv:2504.00638. https://arxiv.org/abs/2504.00638
[12] Alwateer, M., Atlam, E. S., El-Raouf, M. M. A., Ghoneim, O. A., & Gad, I. (2024). Missing data imputation: A comprehensive review. Journal of Computer and Communications, 12(11), 53–75. https://doi.org/10.4236/jcc.2024.1211004
[13]The pandas development team. (n.d.). pandas.DataFrame.fillna. pandas documentation. https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.fillna.html
[14] Wijaya, C. Y. (2023, February 1). The optimal way to input missing data with pandas fillna(). KDnuggets. https://www.kdnuggets.com/2023/02/optimal-way-input-missing-data-pandas-fillna.html
[15] Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., ... & Duchesnay, É. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830. http://jmlr.org/papers/v12/pedregosa11a.html
[16] Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT Press. https://www.deeplearningbook.org/
[17] Chakraborty, A., Tomsett, R., Raghavendra, R., Harborne, D., Alzantot, M., Cerutti, F., ... & Srivastava, M. (2021). Interpretability of deep learning models: A survey of results. IEEE Transactions on Neural Networks and Learning Systems, 32(11), 4793–4813. https://doi.org/10.1109/TNNLS.2020.3029375
[18] Chen, Y., Kang, Y., Chen, Y., & Wang, Z. (2020). Probabilistic forecasting with temporal convolutional neural network. Neurocomputing, 399, 491–501. https://doi.org/10.1016/j.neucom.2020.03.011
[19] Fazel Mojtahedi, F., Yousefpour, N., Chow, S. H., & Cassidy, M. (2025). Deep learning for time series forecasting: Review and applications in geotechnics and geosciences. Archives of Computational Methods in Engineering. https://doi.org/10.1007/s11831-025-10244-5
[20] Chou, J.-S., & Ren, C. (2020). Cloud-based short-term solar power forecasting using LSTM networks. Sustainable Cities and Society, 55, 102010. https://doi.org/10.1016/j.scs.2020.102010
[21] Zhou, K., Yang, S., & Shao, Z. (2022). Short-term electric load forecasting with LSTM networks based on pattern sequence similarity. Applied Energy, 313, 118798. https://doi.org/10.1016/j.apenergy.2022.118798
[22] Chung, J., & Jang, B. (2022). Accurate prediction of electricity consumption using a hybrid CNN-LSTM model based on multivariable data. PLOS ONE, 17(11), e0278071. https://doi.org/10.1371/journal.pone.0278071
[23] Bu, S.-J., & Cho, S.-B. (2020). Time series forecasting with multi-headed attention-based deep learning for residential energy consumption. Energies, 13(18), 4722.
[24] Shi, J., Wang, S., Qu, P., & Shao, J. (2024). Time series prediction model using LSTM-Transformer neural network for mine water inflow. Scientific Reports, 14, Article 18284.
[25] Su, L., Zuo, X., Li, R., Wang, X., Zhao, H., & Huang, B. (2025). A systematic review for transformer-based long-term series forecasting. Artificial Intelligence Review, 58, Article 80.
[26] Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., & Zhang, W. (2021). Informer: Beyond efficient transformer for long sequence time-series forecasting. Proceedings of the AAAI Conference on Artificial Intelligence, 35(12), 11106–11115.
[27] Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., & Zhang, W. (2021). Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, 35(12), 11106–11115.
[28] Wu, H., Xu, J., Wang, J., & Long, M. (2021). Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting. Advances in Neural Information Processing Systems, 34, 22419–22430.
[29] Lim, B., Arık, S. Ö., Loeff, N., & Pfister, T. (2021). Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting. International Journal of Forecasting, 37(4), 1748–1764.
[30] Chicco, D., Warrens, M. J., & Jurman, G. (2021). The coefficient of determination R-squared is more informative than SMAPE, MAE, MAPE, MSE and RMSE in regression analysis evaluation. PeerJ Computer Science, 7, e623. https://doi.org/10.7717/peerj-cs.623
[31]Li, X. (2023). A comparative study of statistical and machine learning models on near-real-time daily emissions prediction. arXiv preprint arXiv:2302.01152. https://arxiv.org/abs/2302.01152
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 International Journal of Computational and Experimental Science and Engineering
This work is licensed under a Creative Commons Attribution 4.0 International License.
