Hyperparameter metaheuristic optimization technique (using salp swarm algorithm) For (Bert transformer) In sentiment analysis. (IMDb dataset)
DOI:
https://doi.org/10.22399/ijcesen.4193Keywords:
BERT, Deep Learning, Hyperparameter Optimization, Natural Language Processing, Salp Swarm Algorithm, Sentiment AnalysisAbstract
Sentiment recognition is a complex task in natural language processing (NLP), and needs training models to handle bulk volumes of data such as IMDb dataset (English movie reviews) with hesitating linguistic trends, posing some considerable computational difficulties.The proposed framework incorporates Bidirectional Encoder Representations Transformers (BERT) and Salp Swarm Algorithm (SSA) to optimize hyperparameters in sentiment analysis using IMDb dataset.Salp swarm intelligence of (SSA) is conducted to optimize the learning rate, batch size, dropout rate, and number of attention heads.Comparative analysis has been conducted against 4 state-of-the-art algorithms (Grid Search, Particle Swarm Optimization, Improved SSA, WOA-AdaBoost) indicating the effectiveness of the proposed SSA-BERT model.The model shows an accuracy of )99.5% (on IMDb dataset performing better than Grid Search (95.5%), Particle Swarm Optimization (PSO) (96%), and Improved SSA (ISSA) (98.5%) as well as the WOA-AdaBoost (99%). Statistical analysis has been conducted using a T-test to prove the model’s superiority.The proposed model achieved a )99.0% (accuracy on the fifth epoch, and an overall accuracy of (99.5%).
References
[1] Abd Elaziz, M., Dahou, A., Abualigah, L., Yu, L., Alshinwan, M., Khasawneh, A. M., & Lu, S. (2021). Advanced metaheuristic optimization techniques in applications of deep neural networks: A review. Neural Computing and Applications, 33(24), 14079–14099.[https://doi.org/10.1007/s00521-021-06253-8]
[2] Al Duhayyim, M., Alazwari, S., Mengash, H. A., Marzouk, R., Alzahrani, J. S., Mahgoub, H., Althukair, F., & Salama, A. S. (2022). Metaheuristics optimization with deep learning enabled automated image captioning system. Applied Sciences, 12(15), 7724. [https://doi.org/10.3390/app12157724]
[3] Al Zubi, A. M. M. (2024). Spam reviews detection models in multilingual contexts applying sentiment analysis, metaheuristics, and advanced word embedding [Doctoral dissertation, Universidad de Granada]. RSITAS GRAN, 1531. [https://digibug.ugr.es/handle/10481/1531]
[4] Alruqimi, M., & Di Persio, L. (2024). Multistep Brent oil price forecasting with a multi-aspect meta-heuristic optimization and ensemble deep learning model. Energy Informatics, 7, 130. [https://doi.org/10.1186/s42162-024-00322-0]
[5] Al-Shathry, N. I., Alghamdi, M., Al-Dobaian, A. S., Darem, A. A., Alotaibi, S. D., Almanea, M., Alghamdi, B. M., & Sorour, S. (2024). Integrating optimal deep learning with natural language processing for Arabic spam and ham tweets recognition. Fractals, 32(9&10), 2540052. [https://doi.org/10.1142/S0218348X25400527]
[6] Alblehai, F. (2025). Artificial intelligence-driven cybersecurity system for internet of things using self-attention deep learning and metaheuristic algorithms. Scientific Reports, 15, 13215. [https://doi.org/10.1038/s41598-025-12315-2]
[7] Antonijevic, M., Jovanovic, L., Bacanin, N., Zivkovic, M., Kaljevic, J., & Zivkovic, T. (2024). Using BERT with modified metaheuristic optimized XGBoost for phishing email identification. In Proceedings of 4th International Conference on Artificial Intelligence and Smart Energy (pp. 358–370). Springer. [https://doi.org/10.1007/978-981-99-8269-0_31]
[8] Bacanin, N., Zivkovic, M., Stoean, C., Antonijevic, M., Janicijevic, S., Sarac, M., & Strumberger, I. (2022). Application of natural language processing and machine learning boosted with swarm intelligence for spam email filtering. Mathematics, 10(22), 4173. [https://doi.org/10.3390/math10224173]
[9] Barut, C., Yildirim, S., Alatas, B., & Yildirim, G. (2024). Innovative multi-objective optimization based automatic fake news detection. Journal of Information Science. Advance online publication. [https://doi.org/10.1177/01655515241261449]
[10] Jain, V., & Kashyap, K. L. (2023). Ensemble hybrid model for Hindi COVID-19 text classification with metaheuristic optimization algorithm. Multimedia Tools and Applications, 82(11), 16839–16859. [https://doi.org/10.1007/s11042-022-13987-5]
[11] Jokic, A., Jovic, N., Gajic, V., Svicevic, M., Pavkovic, M., & Petrovic, A. (2024). Structured query language injection detection with natural language processing techniques optimized by metaheuristics. Journal of Information Security and Applications. Advance online publication. [https://doi.org/10.1016/j.jisa.2024.103730]
[12] Jovanovic, L., Bacanin, N., Radomirovic, B., Zivkovic, M., Njegus, A., & Antonijevic, M. (2024). Natural language processing and AdaBoost optimized by modified metaheuristic for online harassment detection. In Innovations and Advances in Cognitive Systems (pp. 446–463). Springer. [https://doi.org/10.1007/978-3-031-51354-5_30]
[13] Kumar, S., Khan, M. B., Hasanat, M. H. A., Saudagar, A. K. J., AlTameem, A., & AlKhathami, M. (2022). Sigmoidal particle swarm optimization for Twitter sentiment analysis. Computers, Materials & Continua, 74(1), 290–298. [https://doi.org/10.32604/cmc.2022.031282]
[14] Li, B., Li, N., Yang, J., Alfarraj, O., Albelhai, F., Tolba, A., Shaikh, Z. A., Alizadehsani, R., Pławiak, P., & Yee, P. L. (2025). Image steganalysis using active learning and hyperparameter optimization. Scientific Reports, 15, 7340. [https://doi.org/10.1038/s41598-025-07340-5]
[15] Markovic, V., Njegus, A., Bulaja, D., Zivkovic, T, Zivkovic, M., Mani, J. P., & Bacanin, N. (2024). Employee reviews sentiment classification using BERT encoding and AdaBoost classifier tuned by modified PSO algorithm. arXiv:2408.07639, 1. [https://doi.org/10.2991/978-94-6463-482-2_3]
[16] Mu, G., Liao, Z., Li, J., Qin, N., & Yang, Z. (2023). IPSO-LSTM hybrid model for predicting online public opinion trends in emergencies. PLoS ONE, 18(10), e0292677. [https://doi.org/10.1371/journal.pone.0292677]
[17] Ramasamy, L. K., Kadry, S., & Lim, S. (2021). Selection of optimal hyper-parameter values of support vector machine for sentiment analysis tasks using nature-inspired optimization methods. Bulletin of Electrical Engineering and Informatics, 10(1), 290–298. [https://doi.org/10.11591/eei.v10i1.2760]
[18] Stoean, C., Zivkovic, M., Bozovic, A., Bacanin, N., Strulak-Wójcikiewicz, R., Antonijevic, M., & Stoean, R. (2023). Metaheuristic-based hyperparameter tuning for recurrent deep learning: Application to the prediction of solar energy generation. Axioms, 12(3), 266. [https://doi.org/10.3390/axioms12030266]
[19] Sureja, N., Chaudhari, N. M., Bhatt, J., Desai, T., Parikh, V., Panesar, S., Sureja, H., & Kharva, J. (2025). An improved reptile search algorithm-based machine learning for sentiment analysis. International Journal of Electrical and Computer Engineering, 15(1), 755–766. [https://doi.org/10.11591/ijece.v15i1.pp755-766]
[20] Zivkovic, M., Jovanovic, L., Bukumira, M., Antonijevic, M., Mladenovic, D., Al Washahi, M., & Bacanin, N. (2024). Optimizing SQL injection detection using BERT encoding and AdaBoost classification. [https://doi.org/10.2991/978-94-6463-482-2_10]
[21] Mirjalili, S., Gandomi, A. H., Mirjalili, S. Z., Saremi, S., Faris, H., & Mirjalili, S. M. (2017). Salp Swarm Algorithm: A bio-inspired optimizer for engineering design problems. Advances in Engineering Software, 114, 163–191. https://doi.org/10.1016/j.advengsoft.2017.07.002
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 International Journal of Computational and Experimental Science and Engineering
This work is licensed under a Creative Commons Attribution 4.0 International License.
