Semantic Segmentation of Satellite Images using 2 various U-Net Architectures: A Comparison Study
DOI:
https://doi.org/10.22399/ijcesen.1360Keywords:
Aerial imagery, Attention mechanisms, Semantic segmentation, U-Net architecture, Residual blockAbstract
Image segmentation has been a challenging issue in computer vision for years. In contrast to image classification and object detection, semantic segmentation is considered the top tier of the image analysis approach, which gives detailed details of the scene for a given input image. Analysis of aerial images without human intervention has developed a keen interest in research because of its vital importance in various domains. Different applications like disaster response and urban planning depend mostly on semantic segmentation of aerial imagery for their analysis. In a wide range of image processing tasks, convolutional neural networks (CNNs) have manifested their tremendous performance, and the transformation of the computer vision domain is achieved by deep learning. Amongst multiple varieties of CNN, U-Net has proved its efficiency in segmenting aerial images and the medical domain. Nonetheless, U-Net can’t extract potential spatial features from aerial images because of insufficient layers and may output inaccurate boundaries, particularly for objects with compound structures. To circumvent these deficiencies, different varieties of U-Net are experimented with for aerial image segmentation using U-Net, Attention U-Net, Attention Res U-Net, and Recurrent Residual U-Net. We evaluated all these models on a publicly available dataset named semantic segmentation of aerial imagery. Extensive experimental results conclude that Attention Res U-Net and Recurrent Residual U-Netperform better than other U-Net architectures.
References
Feng, X., Jiang, Y., Yang, X., Du, M., Li, X. (2019) Computer vision algorithms and hardware implementations: A survey. Integration 69, 309–320 DOI: https://doi.org/10.1016/j.vlsi.2019.07.005
Vu, H., Le, T.L., Nguyen, V.G., Dinh, T.H. (2018) Semantic regions segmentation using a spatio-temporal model from an uav image sequence with an optimal configuration for data acquisition. Journal of Information and Telecommunication 2(2), 126–146 DOI: https://doi.org/10.1080/24751839.2017.1390654
Sardooi, E.R., Azareh, A., Choubin, B., Barkhori, S., Singh, V.P., Shamshirband, S. (2019) Applying the remotely sensed data to identify homogeneous regions of watersheds using a pixel-based classification approach. Applied Geography 111, 102071 DOI: https://doi.org/10.1016/j.apgeog.2019.102071
Elharrouss, O., Al-Maadeed, S., Subramanian, N., Ottakath, N., Almaadeed, N., Himeur, Y. (2021) Panoptic segmentation: A review. arXiv preprint arXiv:2111.10250
Manickam, R., Kumar Rajan, S., Subramanian, C., Xavi, A., Eanoch, G.J., Yesudhas, H.R. (2020) Person identification with aerial imaginary using segnet based semantic segmentation. Earth Science Informatics 13, 1293–1304 DOI: https://doi.org/10.1007/s12145-020-00516-y
Hao, S., Zhou, Y., Guo, Y. (2020) A brief survey on semantic segmentation with deep learning. Neurocomputing 406, 302–321 DOI: https://doi.org/10.1016/j.neucom.2019.11.118
Ulku, I., Akag¨und¨uz, E. (2022) A survey on deep learning-based architectures for semantic segmentation on 2d images. Applied Artificial Intelligence 36(1), 2032924 DOI: https://doi.org/10.1080/08839514.2022.2032924
Mo, Y., Wu, Y., Yang, X., Liu, F., Liao, Y. (2022) Review the state-of-the-art technologies of semantic segmentation based on deep learning. Neurocomputing 493, 626–646 DOI: https://doi.org/10.1016/j.neucom.2022.01.005
Lv, Z., Wang, F., Cui, G., Benediktsson, J.A., Lei, T., Sun, W. (2022) Spatial–spectral attention network guided with change magnitude image for land cover change detection using remote sensing images. IEEE Transactions on Geoscience and Remote Sensing 60, 1–12 DOI: https://doi.org/10.1109/TGRS.2022.3197901
Safavi, F., Rahnemoonfar, M. (2022) Comparative study of real-time semantic segmentation networks in aerial images during flooding events. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 16, 15–31 DOI: https://doi.org/10.1109/JSTARS.2022.3219724
Zhang, X., Ma, W., Li, C., Wu, J., Tang, X., Jiao, L. (2019) Fully convolutional network-based ensemble method for road extraction from aerial images. IEEE Geoscience and Remote Sensing Letters 17(10), 1777–1781 DOI: https://doi.org/10.1109/LGRS.2019.2953523
Javanmardi, S., Ashtiani, S.-H.M., Verbeek, F.J., Martynenko, A. (2021) Computer-vision classification of corn seed varieties using deep convolutional neural network. Journal of Stored Products Research 92, 101800 DOI: https://doi.org/10.1016/j.jspr.2021.101800
Wang, J., Fu, P., Gao, R.X. (2019) Machine vision intelligence for product defect inspection based on deep learning and hough transform. Journal of Manufacturing Systems 51, 52–60 DOI: https://doi.org/10.1016/j.jmsy.2019.03.002
Zhou, T., Ruan, S., Canu, S. (2019) A review: Deep learning for medical image segmentation using multi-modality fusion. Array 3, 100004 DOI: https://doi.org/10.1016/j.array.2019.100004
Yuan, X., Shi, J., Gu, L. (2021) A review of deep learning methods for semantic segmentation of remote sensing imagery. Expert Systems with Applications 169, 114417 DOI: https://doi.org/10.1016/j.eswa.2020.114417
Lateef, F., Ruichek, Y. (2019) Survey on semantic segmentation using deep learning techniques. Neurocomputing 338, 321–348 DOI: https://doi.org/10.1016/j.neucom.2019.02.003
Wang, R., Lei, T., Cui, R., Zhang, B., Meng, H., Nandi, A.K. (2022) Medical image segmentation using deep learning: A survey. IET Image Processing 16(5), 1243–1267 DOI: https://doi.org/10.1049/ipr2.12419
Ronneberger, O., Fischer, P., Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 Springer DOI: https://doi.org/10.1007/978-3-319-24574-4_28
Erdem, F., Avdan, U. (2020) Comparison of different u-net models for building extraction from high-resolution aerial imagery. International Journal of Environment and Geoinformatics 7(3), 221–227 DOI: https://doi.org/10.30897/ijegeo.684951
Soni, A., Koner, R., Villuri, V.G.K. (2020). M-unet: Modified u-net segmentation framework with satellite imagery. In: Proceedings of the Global AI Congress 2019, pp.47–59 Springer DOI: https://doi.org/10.1007/978-981-15-2188-1_4
Zhang, H., Liu, M., Wang, Y., Shang, J., Liu, X., Li, B., Song, A., Li, Q. (2021) Automated delineation of agricultural field boundaries from sentinel-2 images using recurrent residual u-net. International Journal of Applied Earth Observation and Geoinformation 105, 102557 DOI: https://doi.org/10.1016/j.jag.2021.102557
Wang, Y., Peng, Y., Li, W., Alexandropoulos, G.C., Yu, J., Ge, D., Xiang, W. (2022) Ddu-net: Dual-decoder-u-net for road extraction using high-resolution remote sensing images. IEEE Transactions on Geoscience and Remote Sensing 60, 1–12 DOI: https://doi.org/10.1109/TGRS.2022.3197546
Hou, J., Hou, B., Zhu, M., Zhou, J., Tian, Q. (2023) Sensitivity analysis of parameters of u-net model for semantic segmentation of silt storage dams from remote sensing images. Canadian Journal of Remote Sensing 49(1), 2178834 DOI: https://doi.org/10.1080/07038992.2023.2178834
Anderson, P., He, X., Buehler, C., Teney, D., Johnson, M., Gould, S., Zhang, L. (2018) Bottom-up and top-down attention for image captioning and visual question answering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6077–6086 DOI: https://doi.org/10.1109/CVPR.2018.00636
Jetley, S., Lord, N.A., Lee, N., Torr, P.H. (2018) Learn to pay attention. arXiv preprint arXiv:1804.02391
Sun, Y., Bi, F., Gao, Y., Chen, L., Feng, S. (2022) A multi-attention unet for semantic segmentation in remote sensing images. Symmetry 14(5), 906 DOI: https://doi.org/10.3390/sym14050906
Tang, M., Georgiou, K., Qi, H., Champion, C., Bosch, M. (2023) Semantic segmentation in aerial imagery using multi-level contrastive learning with local consistency. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 3798–3807 DOI: https://doi.org/10.1109/WACV56688.2023.00379
Abdollahi, A., Pradhan, B., Alamri, A.M. (2022) An ensemble architecture of deep convolutional segnet and unet networks for building semantic segmentation from high-resolution aerial images. Geocarto International 37(12), 3355–3370 DOI: https://doi.org/10.1080/10106049.2020.1856199
Wang, S., Hou, X., Zhao, X. (2020) Automatic building extraction from high-resolution aerial imagery via fully convolutional encoder-decoder network with non-local block. IEEE Access 8, 7313–7322 DOI: https://doi.org/10.1109/ACCESS.2020.2964043
Abdollahi, A., Pradhan, B. (2021) Integrating semantic edges and segmentation information for building extraction from aerial images using unet. Machine Learning with Applications 6, 100194 DOI: https://doi.org/10.1016/j.mlwa.2021.100194
Benjdira, B., Bazi, Y., Koubaa, A., Ouni, K. (2019) Unsupervised domain adaptation using generative adversarial networks for semantic segmentation of aerial images. Remote Sensing 11(11), 1369 DOI: https://doi.org/10.3390/rs11111369
Niu, R., Sun, X., Tian, Y., Diao, W., Chen, K., Fu, K. (2021) Hybrid multiple attention network for semantic segmentation in aerial images. IEEE Transactions on Geoscience and Remote Sensing 60, 1–18 DOI: https://doi.org/10.1109/TGRS.2021.3065112
Su, Z., Li, W., Ma, Z., Gao, R. (2022) An improved u-net method for the semantic segmentation of remote sensing images. Applied Intelligence 52(3), 3276–3288 DOI: https://doi.org/10.1007/s10489-021-02542-9
Marmanis, D., Wegner, J.D., Galliani, S., Schindler, K., Datcu, M., Stilla, U. (2016) Semantic segmentation of aerial images with an ensemble of cnns. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences 3, 473–480 DOI: https://doi.org/10.5194/isprs-annals-III-3-473-2016
Li, W., He, C., Fang, J., Fu, H. (2018) Semantic segmentation based building extraction method using multi-source gis map datasets and satellite imagery. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 238–241 DOI: https://doi.org/10.1109/CVPRW.2018.00043
Erdem, F., Bayram, B., Bakirman, T., Bayrak, O.C., Akpinar, B. (2021) An ensemble deep learning based shoreline segmentation approach (waternet) from landsat 8 oli images. Advances in Space Research 67(3), 964–974 DOI: https://doi.org/10.1016/j.asr.2020.10.043
Yuan, K., Zhuang, X., Schaefer, G., Feng, J., Guan, L., Fang, H. (2021) Deep-learning-based multispectral satellite image segmentation for water body detection. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 14, 7422–7434 DOI: https://doi.org/10.1109/JSTARS.2021.3098678
Saifi, M.Y., Singla, J., et al. (2019) Deep learning based framework for semantic segmentation of satellite images. In: 2020 Fourth International Conference on Computing Methodologies and Communication (ICCMC), pp. 369–374 (2020). IEEE [39] Avenash, R., Viswanath, P.: Semantic segmentation of satellite images using a modified cnn with hard-swish activation function. In: VISIGRAPP (4: VISAPP), pp. 413–420 DOI: https://doi.org/10.1109/ICCMC48092.2020.ICCMC-00069
Saralioglu, E., Gungor, O. (2022) Semantic segmentation of land cover from high resolution multispectral satellite images by spectral-spatial convolutional neural network. Geocarto International 37(2), 657–677 DOI: https://doi.org/10.1080/10106049.2020.1734871
Brand, A., Manandhar, A. (2021) Semantic segmentation of burned areas in satellite images using a u-net-based convolutional neural network. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences 43, 47–53 DOI: https://doi.org/10.5194/isprs-archives-XLIII-B3-2021-47-2021
Singh, N.J., Nongmeikapam, K. (2023) Semantic segmentation of satellite images using deep-unet. Arabian Journal for Science and Engineering 48(2), 1193–1205 DOI: https://doi.org/10.1007/s13369-022-06734-4
Wang, X., Hu, Z., Shi, S., Hou, M., Xu, L., Zhang, X. (2023) A deep learning method for optimizing semantic segmentation accuracy of remote sensing images based on improved unet. Scientific Reports 13(1), 7600 DOI: https://doi.org/10.1038/s41598-023-34379-2
Maurya, A., Mittal, P., Kumar, R., et al. (2023) A modified u-net-based architecture for segmentation of satellite images on a novel dataset. Ecological Informatics 75, 102078 DOI: https://doi.org/10.1016/j.ecoinf.2023.102078
Oktay, O., Schlemper, J., Folgoc, L.L., Lee, M., Heinrich, M., Misawa, K., Mori, K., McDonagh, S., Hammerla, N.Y., Kainz, B., et al. (2018) Attention u-net: Learning where to look for the pancreas. arXiv preprint arXiv:1804.03999
Maji, D., Sigedar, P., Singh, M. (2022) Attention res-unet with guided decoder for semantic segmentation of brain tumors. Biomedical Signal Processing and Control 71, 103077 DOI: https://doi.org/10.1016/j.bspc.2021.103077
Resnet:. https://d2l.ai/chapter convolutional-modern/resnet.html Accessed accessed 05-August-2022
Xiang, L., Li, Y., Lin, W., Wang, Q., Shen, D. (2018). Unpaired deep cross-modality synthesis with fast training. In: Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 20, 2018, Proceedings 4, pp. 155–164 Springer DOI: https://doi.org/10.1007/978-3-030-00889-5_18
He, K., Zhang, X., Ren, S., Sun, J. (2016). Identity mappings in deep residual networks. In: Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14, pp. 630–645 Springer DOI: https://doi.org/10.1007/978-3-319-46493-0_38
Alom, M.Z., Yakopcic, C., Hasan, M., Taha, T.M., Asari, V.K. (2019) Recurrent residual u-net for medical image segmentation. Journal of Medical Imaging 6(1), 014006–014006 DOI: https://doi.org/10.1117/1.JMI.6.1.014006
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 International Journal of Computational and Experimental Science and Engineering
This work is licensed under a Creative Commons Attribution 4.0 International License.
