An Efficient Approach for Encrypted Traffic Classification and Intrusion Detection using Packet Transformer Encoder and CNN
DOI:
https://doi.org/10.22399/ijcesen.1377Keywords:
Encrypted Traffic, Deep Learning, Classification, BERT, CNNAbstract
As encryption technology rapidly progresses and applications experience exponential growth, the research focus on network traffic classification has intensified. Current methods for classifying encrypted traffic exhibit certain constraints. Traditional techniques, including machine learning, heavily depend on feature engineering. Deep learning approaches are vulnerable to the quantity and distribution of labeled data, while pretrained models predominantly emphasize global traffic features, neglecting local features. In addressing these challenges, we introduced a methodology that incorporates both Bidirectional Encoder Representations from Transformers (BERT) and Convolution Neural Networks (CNN). To underscore both global traffic patterns and local features, we leverage the BERT and CNN mechanisms, respectively. Our approach attains state-of-the-art performance on the publicly accessible ISCX-VPN dataset for both traffic service and application identification tasks, achieving impressive F1 scores of 99.11% and 99.41%, respectively, in these domains. The experimental outcomes affirm that our method significantly enhances the performance of encrypted traffic classification.
References
[1] Zhang, Z., Han, X., Liu, Z., Jiang, X., Sun, M., & Liu, Q. (2019). ERNIE: Enhanced language representation with informative entities (arXiv:1905.07129). arXiv. https://arxiv.org/abs/1905.07129
[2] Rezaei, S., & Liu, X. (2019). Deep learning for encrypted traffic classification: An overview. IEEE Communications Magazine, 57(5), 76–81. https://doi.org/10.1109/MCOM.2019.1800819
[3] Zeng, Y., Li, K., He, X., Liu, X., & He, X. (2019). Deep-Full-Range: A deep learning-based network encrypted traffic classification and intrusion detection framework. IEEE Access, 7, 45182–45190. https://doi.org/10.1109/ACCESS.2019.2908430
[4] Lin, P., Ye, K., Xu, C.-Z., Wang, Y., & Li, J. (2021). PEAN: A packet-level end-to-end attentive network for encrypted traffic identification. In 2021 IEEE 23rd Int. Conf. on High Performance Computing & Communications (HPCC/DSS/SmartCity/DependSys) (pp. 1183–1190). IEEE. https://doi.org/10.1109/HPCC-DSS-SmartCity-DependSys52092.2021.00161
[5] Liu, J., Duan, H., Li, B., He, Q., & Zhou, W. (2017). Effective and real-time in-app activity analysis in encrypted internet traffic streams. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 2015–2024). ACM. https://doi.org/10.1145/3097983.3097992
[6] Lin, P., Ye, K., Xu, C.-Z., & Yang, J. (2022). A novel multimodal deep learning framework for encrypted traffic classification. IEEE/ACM Transactions on Networking, 30(2), 536–549. https://doi.org/10.1109/TNET.2021.3134495
[7] Sheela, M., Amirthayogam, G., Hephzipah, J. J., Suganthi, R., Karthikeyan, T., & Gopianand, M. (2024). Advanced brain tumor classification using DEEPBELEIF-CNN method. Babylonian Journal of Machine Learning, 2024, 89–101. https://doi.org/10.58496/BJML/2024/009
[8] Roughan, M., Sen, S., Spatscheck, O., & Duffield, N. (2004). Class-of-service mapping for QoS: A statistical signature-based approach to IP traffic classification. In Proceedings of the 4th ACM SIGCOMM Conference on Internet Measurement (pp. 135–148). ACM. https://doi.org/10.1145/1028788.1028805
[9] Lin, P., Ye, K., & Xu, C.-Z. (2019). Dynamic network anomaly detection system by using deep learning techniques. In Cloud Computing – CLOUD 2019: 12th International Conference (pp. 243–252). Springer. https://doi.org/10.1007/978-3-030-23255-9_19
[10] Alazawi, S. A. H., Abdulbaqi, H. A., & Ali, A. H. (2024). CNN-based intrusion detection software for network operating system environment. Babylonian Journal of Internet of Things (BJIoT), 2024, 79–86. https://doi.org/10.58496/BJIoT/2024/010
[11] Anderson, B., & McGrew, D. (2016). Identifying encrypted malware traffic with contextual flow data. In Proceedings of the 2016 ACM Workshop on Artificial Intelligence and Security (pp. 35–46). ACM. https://doi.org/10.1145/2996429.2996435
[12] Kalnoor, G., Sai, K., Dasari, S. S., Waddenkery, N., & Pragathi, B. (2024). Enhanced brain tumor detection from MRI scans using frequency domain features and hybrid machine learning models. Journal of Modern Technology, 2024, 141–149.
[13] Shen, M., Wei, M., Zhu, L., & Wang, M. (2017). Classification of encrypted traffic with second-order Markov chains and application attribute bigrams. IEEE Transactions on Information Forensics and Security, 12(8), 1830–1843. https://doi.org/10.1109/TIFS.2017.2687799
[14] Liu, C., Cao, Z., Xiong, G., Gou, G., Yiu, S.-M., & He, L. (2018). MaMPF: Encrypted traffic classification based on multi-attribute Markov probability fingerprints. In IEEE/ACM 26th International Symposium on Quality of Service (IWQoS) (pp. 1–10). IEEE. https://doi.org/10.1109/IWQoS.2018.8624182
[15] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems (pp. 5998–6008). https://papers.nips.cc/paper_files/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html
[16] Zejdl, P., Ubik, S., Macek, V., & Oslebo, A. (2008). Traffic classification for portable applications with hardware support. In International Workshop on Intelligent Solutions in Embedded Systems (pp. 1–9). IEEE. https://doi.org/10.1109/WISES.2008.4665073
[17] Park, J.-S., Yoon, S.-H., & Kim, M.-S. (2013). Performance improvement of payload signature-based traffic classification system using application traffic temporal locality. In 15th Asia–Pacific Network Operations and Management Symposium (APNOMS) (pp. 1–6). IEEE. https://doi.org/10.1109/APNOMS.2013.6996521
[18] Sen, S., Spatscheck, O., & Wang, D. (2004). Accurate, scalable in-network identification of P2P traffic using application signatures. In Proceedings of the 13th International World Wide Web Conference (pp. 512–521). ACM. https://doi.org/10.1145/988672.988737
[19] Taylor, V. F., Spolaor, R., Conti, M., & Martinovic, I. (2018). Robust smartphone app identification via encrypted network traffic analysis. IEEE Transactions on Information Forensics and Security, 13(1), 63–78. https://doi.org/10.1109/TIFS.2017.2736559
[20] Taylor, V. F., Spolaor, R., Conti, M., & Martinovic, I. (2016). AppScanner: Automatic fingerprinting of smartphone apps from encrypted network traffic. In Proceedings of the IEEE European Symposium on Security and Privacy (EuroS&P) (pp. 439–454). IEEE. https://doi.org/10.1109/EuroSP.2016.38
[21] Liu, C., He, L., Xiong, G., Cao, Z., & Li, Z. (2019). FS-Net: A flow sequence network for encrypted traffic classification. In IEEE Conference on Computer Communications (INFOCOM) (pp. 1171–1179). IEEE. https://doi.org/10.1109/INFOCOM.2019.8737553
[22] Cho, K., Van Merriënboer, B., Bahdanau, D., & Bengio, Y. (2014). Learning phrase representations using RNN encoder–decoder for statistical machine translation (arXiv:1406.1078). arXiv. https://arxiv.org/abs/1406.1078
[23] Korczynski, M., & Duda, A. (2014). Markov chain fingerprinting to classify encrypted traffic. In IEEE Conference on Computer Communications (INFOCOM) (pp. 781–789). IEEE. https://doi.org/10.1109/INFOCOM.2014.6847993
[24] Lashkari, A. H., Draper-Gil, G., Mamun, M. S. I., & Ghorbani, A. A. (2017). Characterization of Tor traffic using time-based features. In Proceedings of the International Conference on Information Systems Security and Privacy (ICISSP) (pp. 253–262).
[25] Bhavya, P. S., Balaji, K., Banoth, N., & Parhamfar, M. (2024). A light weight Mobile Net SSD algorithm based identification and detection of multiple defects in ceramic insulators. Journal of Modern Technology, 2024, 59–74.
[26] Neelashetty, K., Goel, S., Inamdar, F., Dintakurthy, Y., Varanasi, L. N. S., & Krishna, V. B. M. (2025). Optimal energy management in microgrids: A demand response approach with Monte Carlo scenario synthesis and K-means clustering. International Journal of Computational and Experimental Science and Engineering, 11(1). https://doi.org/10.22399/ijcesen.1023
[27] Shbair, W. M., Cholez, T., Francois, J., & Chrisment, I. (2016, April). A multi-level framework to identify HTTPS services. In Proceedings of the IEEE/IFIP Network Operations and Management Symposium (pp. 240–248).
[28] Cheng, J., He, R., Yuepeng, E., Wu, Y., You, J., & Li, T. (2020, December). Real-time encrypted traffic classification via lightweight neural networks. In Proceedings of the IEEE Global Communications Conference (GLOBECOM) (pp. 1–6).
[29] Wang, W., Zhu, M., Wang, J., Zeng, X., & Yang, Z. (2017, July). End-to-end encrypted traffic classification with one-dimensional convolution neural networks. In Proceedings of the IEEE International Conference on Intelligence and Security Informatics (ISI) (pp. 43–48).
[30] Lotfollahi, M., Siavoshani, M. J., Hossein Zade, R. S., & Saberian, M. (2020). Deep packet: A novel approach for encrypted traffic classification using deep learning. Soft Computing, 24(3), 1999–2012.
[31] Krishna, V. B. M., Melkeri, V. S., Goel, S., & Prasad, K. R. K. V. (2025). Two-stage energy management for maximizing renewable energy penetration. Engineering Review, 45(1). https://doi.org/10.30765/er.2688
[32] Wang, W., et al. (2017). HAST-IDS: Learning hierarchical spatial-temporal features using deep neural networks to improve intrusion detection. IEEE Access, 6, 1792–1806.
[33] Rezaei, S., Kroencke, B., & Liu, X. (2020). Large-scale mobile app identification using deep learning. IEEE Access, 8, 348–362.
[34] He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 770–778).
[35] Hochreiter, S., Bengio, Y., Frasconi, P., & Schmidhuber, J. (2001). Gradient flow in recurrent nets: The difficulty of learning long-term dependencies.
[36] Ba, J. L., Kiros, J. R., & Hinton, G. E. (2016). Layer normalization. arXiv preprint arXiv:1607.06450.
[37] Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. CoRR, abs/1409.0473.
[38] Britz, D., Goldie, A., Luong, M. T., & Le, Q. V. (2017). Massive exploration of neural machine translation architectures. CoRR, abs/1703.03906.
[39] Draper-Gil, G., Lashkari, A. H., Mamun, M. S. I., & Ghorbani, A. A. (2016, February). Characterization of encrypted and VPN traffic using time-related features. In Proceedings of the 2nd International Conference on Information Systems Security and Privacy (ICISSP) (pp. 407–414). Rome, Italy.
[40] Lin, X., Xiong, G., Gou, G., Li, Z., Shi, J., & Yu, J. (2022, April). ET-BERT: A contextualized datagram representation with pre-training transformers for encrypted traffic classification. In Proceedings of the ACM Web Conference 2022 (pp. 633–642). Lyon, France.
[41] Rogers, A., Kovaleva, O., & Rumshisky, A. (2020). A primer in BERTology: What we know about how BERT works. Transactions of the Association for Computational Linguistics, 8, 842–866.
[42] Hubel, D. H., & Wiesel, T. N. (1968). Receptive fields and functional architecture of monkey striate cortex. Journal of Physiology, 195, 215–243.
[43] Dos Santos, C., & Gatti, M. (2014, August). Deep convolutional neural networks for sentiment analysis of short texts. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers (pp. 69–78). Dublin, Ireland.
[44] Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.
[45] Lotfollahi, M., Siavoshani, M. J., Hossein Zade, R. S., & Saberian, M. (2020). Deep packet: A novel approach for encrypted traffic classification using deep learning. Soft Computing, 24, 1999–2012.
[46] Zhao, Z., Chen, H., Zhang, J., Zhao, X., Liu, T., Lu, W., ... & Du, X. (2019, November). UER: An open-source toolkit for pre-training models. In Proceedings of EMNLP-IJCNLP 2019 (p. 241). Hong Kong, China.
[47] Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
[48] van Ede, T., Bortolameotti, R., Continella, A., Ren, J., Dubois, D. J., Lindorfer, M., ... & Peter, A. (2020, February). Flowprint: Semi-supervised mobile-app fingerprinting on encrypted network traffic. In Proceedings of the Network and Distributed System Security Symposium (NDSS). San Diego, CA, USA.
[49] Panchenko, A., Lanze, F., Pennekamp, J., Engel, T., Zinnen, A., Henze, M., & Wehrle, K. (2016, February). Website fingerprinting at internet scale. In Proceedings of NDSS. San Diego, CA, USA.
[50] Taylor, V. F., Spolaor, R., Conti, M., & Martinovic, I. (2017). Robust smartphone app identification via encrypted network traffic analysis. IEEE Transactions on Information Forensics and Security, 13, 63–78.
[51] Al-Naami, K., Chandra, S., Mustafa, A., Khan, L., Lin, Z., Hamlen, K., & Thuraisingham, B. (2016, December). Adaptive encrypted traffic fingerprinting with bi-directional dependence. In Proceedings of the 32nd Annual Conference on Computer Security Applications (pp. 177–188). Los Angeles, CA, USA.
[52] Sirinam, P., Imani, M., Juarez, M., & Wright, M. (2018, October). Deep fingerprinting: Undermining website fingerprinting defenses with deep learning. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security (pp. 1928–1943). Toronto, ON, Canada.
[53] Liu, C., He, L., Xiong, G., Cao, Z., & Li, Z. (2019, April). FS-Net: A flow sequence network for encrypted traffic classification. In Proceedings of the IEEE INFOCOM 2019 - IEEE Conference on Computer Communications (pp. 1171–1179). Paris, France.
[54] Shen, M., Zhang, J., Zhu, L., Xu, K., & Du, X. (2021). Accurate decentralized application identification via encrypted traffic analysis using graph neural networks. IEEE Transactions on Information Forensics and Security, 16, 2367–2380.
[55] He, H. Y., Yang, Z. G., & Chen, X. N. (2020, December). PERT: Payload encoding representation from transformer for encrypted traffic classification. In Proceedings of the 2020 ITU Kaleidoscope: Industry-Driven Digital Transformation (ITU K) (pp. 1–8). Ha Noi, Vietnam.
[56] Shiravi, A., Shiravi, H., Tavallaee, M., & Ghorbani, A. A. (2012). Toward developing a systematic approach to generate benchmark datasets for intrusion detection. Computers & Security, 31(3), 357–374.
[57] Dintakurthy, Y., Innmuri, R. K., Vanteru, A., & Thotakuri, A. (2025). Emerging applications of artificial intelligence in edge computing: A comprehensive review. Journal of Modern Technology, 1(2), 175–185.
[58] Al Barazanchi, I. I., Hashim, W., Thabit, R., & Hussein, N. A. K. (2024, March). Advanced hybrid mask convolutional neural network with backpropagation optimization for precise sensor node classification in wireless body area networks. KHWARIZMIA, 2024, 17–31. https://doi.org/10.70470/KHWARIZMIA/2024/004
[59] Alazawi, S. A. H., Abdulbaqi, H. A., & Ali, A. H. (2024, August). CNN-based intrusion detection software for network operating system environment. BJIoT, 2024, 79–86. https://doi.org/10.58496/BJIoT/2024/010
[60] Harinath, K. R., & Kumar, G. K. (2024). Encrypted network traffic classification and feature selection by ensemble of CNN and TLBO meta-heuristic algorithm. In N. Singh, A. K. Bashir, S. Kadry, & Y. C. Hu (Eds.), Proceedings of the 1st International Conference on Intelligent Healthcare and Computational Neural Modelling. ICIHCNN 2022 (pp. xxx–xxx). Springer. https://doi.org/10.1007/978-981-99-2832-3_65
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 International Journal of Computational and Experimental Science and Engineering
This work is licensed under a Creative Commons Attribution 4.0 International License.
