Deep learning for semantic segmentation and feature extraction of Miao costume patterns
Traditional costume patterns of ethnic minorities constitute key elements of intangible cultural heritage, yet the lack of automated, pixel-level analysis tools has hindered their systematic digital documentation and preservation. This study proposes a general deep-learning-based semantic segmentation and feature-extraction framework for traditional ethnic costume patterns and uses Miao patterns as a representative, computationally demanding example, thereby addressing the two coupled challenges of accurate pattern boundary delineation and the generation of discriminative feature representations. The framework integrates a dual-pathway attention fusion mechanism combining channel and spatial attention within a Swin transformer-based encoder–decoder architecture. A composite loss function complements it, jointly optimizing classification accuracy, class-imbalance mitigation, and boundary preservation. We construct a dedicated benchmark dataset comprising pixel-level annotations across eight culturally defined pattern categories to support training and evaluation. Comprehensive experiments showed that the proposed framework achieves a mean intersection over union (mIoU) of 74.6%, a foreground pixel accuracy of 91.3%, an overall pixel accuracy of 94.6%, an F1-score of 82.4%, and a boundary F1 (BF1) of 68.2% on the Miao costume pattern dataset (MCPD) test set, outperforming the strongest baseline by 3.4 percentage points in mIoU and 4.5 percentage points in boundary F1. The extracted image-level 256-dimensional semantic feature representations showed clear inter-class separability, with a Silhouette coefficient of 0.47, and achieved a retrieval mean average precision of 80.6% on the Miao costume pattern dataset MCPD test set, while the framework obtains a mean adapted mIoU of 58.7% on the fabric image data dataset under a target-domain adaptation protocol using 56 labeled fine-tuning images. This research establishes a reproducible computational pipeline for the systematic cataloging and image-level retrieval of Miao costume patterns. The results provide initial evidence that such a segmentation-guided framework may support related textile heritage documentation tasks, although broader validation across additional ethnic textile datasets and cultural contexts remains necessary.
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017). SegNet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(12), 2481–2495. https://doi.org/10.1109/TPAMI.2016.2644615
Basu, A., Paul, S., Ghosh, S., Das, S., Chanda, B., Bhagvati, C., & Snasel, V. (2023). Digital restoration of cultural heritage with data-driven computing: A survey. IEEE Access, 11, 53939–53977. https://doi.org/10.1109/ACCESS.2023.3280639
Boulila, W., Ghandorh, H., Masood, S., Alzahem, A., Koubaa, A., Ahmed, F., Khan, Z., & Ahmad, J. (2024). A transformer-based approach empowered by a self-attention technique for semantic segmentation in remote sensing. Heliyon, 10(8), e29396. https://doi.org/10.1016/j.heliyon.2024.e29396
Cao, H., Wang, Y., Chen, J., Jiang, D., Zhang, X., Tian, Q., & Wang, M. (2022). Swin-Unet: Unet-like pure transformer for medical image segmentation. In Computer Vision – ECCV 2022 Workshops (Lecture Notes in Computer Science, Vol. 13803, pp. 205–218). Springer Nature Switzerland, Cham, Switzerland. https://doi.org/10.1007/978-3-031-25066-8_9
Cărunta, C., Cărunta, A., Mizeranschi, A.-E., & Popa, C.-A. (2026). Real-time semantic segmentation of driving scenes via effective attention-based information fusion and hybrid encoder. Scientific Reports, 16(1), 2419. https://doi.org/10.1038/s41598-025-29931-1
Casillo, M., Colace, F., Gaeta, R., Lorusso, A., Santaniello, D., & Valentino, C. (2024). Revolutionizing cultural heritage preservation: An innovative IoT-based framework for protecting historical buildings. Evolutionary Intelligence, 17, 3815–3831. https://doi.org/10.1007/s12065-024-00959-y
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018). Encoder-decoder with atrous separable convolution for semantic image segmentation. In Computer Vision – ECCV 2018 (Lecture Notes in Computer Science, Vol. 11211, pp. 833–851). Springer International Publishing, Cham, Switzerland. https://doi.org/10.1007/978-3-030-01234-2_49
Chen, T., Hong, Y., & Dai, X. (2025). Deep learning-based recognition of Miao ethnic costumes via YOLOv5s: A step toward digital cultural preservation. Springer Science and Business Media LLC. https://doi.org/10.21203/rs.3.rs-7152683/v1
Cheng, B., Misra, I., Schwing, A. G., Kirillov, A., & Girdhar, R. (2022). Masked-attention mask transformer for universal image segmentation. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 1280–1289). IEEE. https://doi.org/10.1109/cvpr52688.2022.00135
Cui, H., Seyam, A. F., & Shamey, R. (2025). Textile and colour defect detection using deep learning methods. Coloration Technology, 142(4), 481–501. https://doi.org/10.1111/cote.70044
Deng, M., Liu, Y., & Chen, L. (2023). AI-driven innovation in ethnic clothing design: An intersection of machine learning and cultural heritage. Electronic Research Archive, 31(9), 5793–5814. https://doi.org/10.3934/era.2023295
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., & Houlsby, N. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv. https://doi.org/10.48550/arXiv.2010.11929
Emek Soylu, B., Guzel, M. S., Bostanci, G. E., Ekinci, F., Asuroglu, T., & Acici, K. (2023). Deep-learning-based approaches for semantic segmentation of natural scene images: A review. Electronics, 12(12), 2730. https://doi.org/10.3390/electronics12122730
Fan, C.-L. (2023). Multiscale feature extraction by using convolutional neural network: Extraction of objects from multiresolution images of urban areas. ISPRS International Journal of Geo-Information, 13(1), 5. https://doi.org/10.3390/ijgi13010005
Fan, Y., & Li, M. (2025). Traditional Chinese clothing style recognition and design based on deep learning. Computer-Aided Design & Applications, 22, 326–341. https://doi.org/10.14733/cadaps.2025.S1.326-341
Fantin Irudaya Raj, E., & Balaji, M. (2022). Application of deep learning and machine learning in pattern recognition. In Advance concepts of image processing and pattern recognition: Effective solution for global challenges (pp. 63–89). Springer Singapore, Singapore. https://doi.org/10.1007/978-981-16-9324-3_4
Guo, H., Lv, W., Shen, Z., Wang, D.-H., & Zhang, Y. (2026). CPFormer-Net: Correspondence pruning transformer with structured context aggregation. IEEE Signal Processing Letters, 33, 111–115. https://doi.org/10.1109/LSP.2025.3636997
Guo, H., Xiao, G., Su, L., Zhou, J., & Wang, D.-H. (2024). Local neighbor propagation on graphs for mismatch removal. Information Sciences, 653, 119749. https://doi.org/10.1016/j.ins.2023.119749
Harisanty, D., Obille, K. L. B., Anna, N. E. V., Purwanti, E., & Retrialisca, F. (2024). Cultural heritage preservation in the digital age, harnessing artificial intelligence for the future: A bibliometric analysis. Digital Library Perspectives, 40(4), 609–630. https://doi.org/10.1108/DLP-01-2024-0018
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 770–778). IEEE. https://doi.org/10.1109/CVPR.2016.90
Hou, X., Zhao, H., Zhou, X., Liang, X., & Yang, J. (2024). Traditional pattern segmentation algorithm based on semantic prediction and region learning. In 2024 International Conference on Culture-Oriented Science & Technology (CoST) (pp. 218–223). IEEE. https://doi.org/10.1109/CoST64302.2024.00050
Hu, J., Shen, L., & Sun, G. (2018). Squeeze-and-excitation networks. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 7132–7141). IEEE. https://doi.org/10.1109/cvpr.2018.00745
Huang, L., Jiang, B., Lv, S., Liu, Y., & Fu, Y. (2024). Deep-learning-based semantic segmentation of remote sensing images: A survey. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 17, 8370–8396. https://doi.org/10.1109/JSTARS.2023.3335891
Huang, X., Jane, M., Huang, A., & Zhang, Z. (2024). Analysis and recognition of traditional patterns in Guangxi Zhuang ethnic clothing based on an improved YOLOv5 model. In 2024 5th International Symposium on Computer Engineering and Intelligent Communications (ISCEIC) (pp. 677–681). IEEE. https://doi.org/10.1109/ISCEIC63613.2024.10810166
Iqbal Hussain, M. A., Khan, B., Wang, Z., & Ding, S. (2020). Woven fabric pattern recognition and classification based on deep convolutional neural networks. Electronics, 9(6), 1048. https://doi.org/10.3390/electronics9061048
Ji, J., Lao, Y., & Huo, L. (2024). Convolutional neural network application for supply–demand matching in Zhuang ethnic clothing image classification. Scientific Reports, 14(1), 13348. https://doi.org/10.1038/s41598-024-64082-9
Jiangtao, W., Ruhaiyem, N. I. R., & Panpan, F. (2025). A comprehensive review of U-Net and its variants: Advances and applications in medical image segmentation. IET Image Processing, 19(1), e70019. https://doi.org/10.1049/ipr2.70019
Li, X., Chiabrando, F., & Sammartano, G. (2026). Machine learning and deep learning for cultural heritage conservation: A bibliometric and task-oriented review. Remote Sensing, 18(4), 628. https://doi.org/10.3390/rs18040628
Liu, R., Yu, Z., Fan, Q., Sun, Q., & Jiang, Z. (2024). The improved method in fabric image classification using convolutional neural network. Multimedia Tools and Applications, 83(3), 6909–6924. https://doi.org/10.1007/s11042-023-15573-w
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021). Swin Transformer: Hierarchical vision transformer using shifted windows. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 9992–10002). IEEE. https://doi.org/10.1109/ICCV48922.2021.00986
Long, J., Shelhamer, E., & Darrell, T. (2015). Fully convolutional networks for semantic segmentation. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 3431–3440). IEEE. https://doi.org/10.1109/CVPR.2015.7298965
Lv, J., Shen, Q., Lv, M., Li, Y., Shi, L., & Zhang, P. (2023). Deep learning-based semantic segmentation of remote sensing images: A review. Frontiers in Ecology and Evolution, 11, 1201125. https://doi.org/10.3389/fevo.2023.1201125
Ma, H., Zhou, Z., & Wang, Y. (2025). Digital preservation and development of architectural heritage from a virtual perspective: A systematic review. npj Heritage Science, 13(1), 594. https://doi.org/10.1038/s40494-025-02162-3
Ma, J., Zhou, S., & Xu, P. (2025). Lightweight visual feature recognition of Tibetan Opera costumes: Framework design and implementation. npj Heritage Science, 13(1), 660. https://doi.org/10.1038/s40494-025-02231-7
Machado, R., Barros, L. A., Vieira, V., Silva, F. D. d., Costa, H., & Carvalho, V. (2025). Textile defect detection using artificial intelligence and computer vision—A preliminary deep learning approach. Electronics, 14(18), 3692. https://doi.org/10.3390/electronics14183692
Mahanta, L. B., Mahanta, D. R., Rahman, T., & Chakraborty, C. (2024). Handloomed fabrics recognition with deep learning. Scientific Reports, 14(1), 7974. https://doi.org/10.1038/s41598-024-58750-z
Manan, M. A., Feng, J., & Ahmed, S. (2025). Advancement in semantic segmentation techniques: A comprehensive review for semantic segmentation of colorectal polyps using deep learning. Artificial Intelligence Review, 58(12), 396. https://doi.org/10.1007/s10462-025-11369-6
Mirab Golkhatmi, B., Houshmand, M., & Hosseini, S. A. (2025). A multi-scale attention-based Swin transformer model for medical images segmentation. Scientific Reports, 15(1), 38893. https://doi.org/10.1038/s41598-025-22649-0
Mulu, A., & Gizaw, A. (2023). Habesha women's dress embroidery design identification using an ensemble feature extraction approach. In 2023 International Conference on Information and Communication Technology for Development for Africa (ICT4DA) (pp. 1–6). IEEE. https://doi.org/10.1109/ICT4DA59526.2023.10302186
Ouyang, C., Tang, F., Zhu, D., Qiu, Y., & Liu, Y. (2021). A multi-strategy improved sparrow search algorithm. Journal of Physics: Conference Series, 2082(1), 012014. https://doi.org/10.1088/1742-6596/2082/1/012014
Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015 (Lecture Notes in Computer Science, Vol. 9351, pp. 234–241). Springer International Publishing, Cham, Switzerland. https://doi.org/10.1007/978-3-319-24574-4_28
Shi, Y., Li, F., Zhao, S., Yu, H., Chen, X., & Liu, Q. (2025). IAP-TransUNet: Integration of the attention mechanism and pyramid pooling for medical image segmentation. Frontiers in Neurorobotics, 19, 1706626. https://doi.org/10.3389/fnbot.2025.1706626
Urrea, C., & Vélez, M. (2025). Advances in deep learning for semantic segmentation of low-contrast images: A systematic review of methods, challenges, and future directions. Sensors, 25(7), 2043. https://doi.org/10.3390/s25072043
Wang, K., Kumar, V., Zeng, X., Koehl, L., Tao, X., & Chen, Y. (2019). Development of a textile coding tag for the traceability in textile supply chain by using pattern recognition and robust deep learning. International Journal of Computational Intelligence Systems, 12(2), 713–722. https://doi.org/10.2991/ijcis.d.190704.002
Wang, T., & Wen, B. (2024). Research on image recognition of ethnic minority clothing based on improved vision transformer. Mathematical Foundations of Computing, 7(1), 84–97. https://doi.org/10.3934/mfc.2022054
Wang, X., Hu, Z., Shi, S., Hou, M., Xu, L., & Zhang, X. (2023). A deep learning method for optimizing semantic segmentation accuracy of remote sensing images based on improved UNet. Scientific Reports, 13(1), 7600. https://doi.org/10.1038/s41598-023-34379-2
Woo, S., Park, J., Lee, J.-Y., & Kweon, I. S. (2018). CBAM: Convolutional block attention module. In Computer Vision – ECCV 2018 (Lecture Notes in Computer Science, Vol. 11211, pp. 3–19). Springer International Publishing, Cham, Switzerland. https://doi.org/10.1007/978-3-030-01234-2_1
Xie, E., Wang, W., Yu, Z., Anandkumar, A., Alvarez, J. M., & Luo, P. (2021). SegFormer: Simple and efficient design for semantic segmentation with transformers. Advances in Neural Information Processing Systems, 34, 12077–12090.
Yu, A., Quan, Y., Yu, R., Guo, W., Wang, X., Hong, D., Zhang, H., Chen, J., Hu, Q., & He, P. (2023). Deep learning methods for semantic segmentation in remote sensing with small data: A survey. Remote Sensing, 15(20), 4987. https://doi.org/10.3390/rs15204987
Zhang, J., Chen, X., Yang, B., Guan, Q., Chen, Q., Chen, J., Wu, Q., Xie, Y., & Xia, Y. (2025). Advances in attention mechanisms for medical image segmentation. Computer Science Review, 56, 100721. https://doi.org/10.1016/j.cosrev.2024.100721
Zhang, M., Zhang, Y., Liu, S., Han, Y., Cao, H., & Qiao, B. (2024). Dual-attention transformer-based hybrid network for multi-modal medical image segmentation. Scientific Reports, 14(1), 25704. https://doi.org/10.1038/s41598-024-76234-y
Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017). Pyramid scene parsing network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 6230–6239). IEEE. https://doi.org/10.1109/CVPR.2017.660