Real-Time Crack Detection and Segmentation in Metallic Surfaces Using YOLO v11-Seg Model Integrated with HMI for Industrial Applications
Keywords:
Crack Detection, YOLO v11, Crack Segmentation, Mask Selection, Human Machine Interface, Quality Control, Artificial IntelligenceSynopsis
This is a Chapter in:
Book:
Intelligent and Sustainable Solutions
Print ISBN 978-1-6692-0012-3
Online ISBN 978-1-6692-0011-6
Series:
Chronicle of Computing
Chapter Abstract:
Cracks in metallic surfaces fail industrial systems, compromising systems' safety and productivity. It is therefore imperative to detect such cracks in their initial stage, so preventive measures can be taken to avoid downtime and associated risks. In this paper, we report the novel adaptation and application of the YOLO v11-seg model for precise crack segmentation in metallic surfaces. The specialized configuration optimizes both detection accuracy and inference speed, making it suitable for industrial applications. The deep learning model is integrated into a Human-Machine Interface (HMI) using the Open Neural Network Exchange (ONNX) format, to enable seamless real-time visualization and interaction. This enhances usability for industrial operators, bridging the gap between advanced AI models and practical deployment. We report a fully functional and versatile inspection tool by combining the deep learning model with hardware and real-time video processing via Python. A custom dataset comprising 1,111 training and 246 test images was curated, annotated with segmentation masks, and augmented using the Albumentations library to improve generalization. The model showed detection and segmentation precisions of 96% and 94%, respectively.
About this Paper
Cite this paper as:
Qureshia S., Fatimaa A., Daudpotoa J., Shaikha A., Qayooma A. (2025). Real-Time Crack Detection and Segmentation in Metallic Surfaces Using YOLO v11-Seg Model Integrated with HMI for Industrial Applications. In: Tiako P.F. (ed) Intelligent and Sustainable Solutions. Chronicle of Computing. OkIP. CAIF25#9. https://doi.org/10.55432/978-1-6692-0011-6_4
Presented at:
The 2025 OkIP International Conference on Artificial Intelligence Frontiers (CAIF) in Oklahoma City, Oklahoma, USA, and Online, on April 2, 2025
Contact:
Jawaid Daudpoto
jawaid.daudpoto@faculty.muet.edu.pk
References
Al-Huda, Z., Peng, B., Algburi, R. N. A., Al-antari, M. A., AL-Jarazi, R., Al-maqtari, O., & Zhai, D. (2023). Asymmetric dual-decoder-U-Net for pavement crack semantic segmentation. Automation in Construction, 156, 105138. https://doi.org/10.1016/j.autcon.2023.105138
Al-Huda, Z., Peng, B., Algburi, R. N. A., Al-antari, M. A., AL-Jarazi, R., & Zhai, D. (2023). A hybrid deep learning pavement crack semantic segmentation. Engineering Applications of Artificial Intelligence, 122, 106142. https://doi.org/10.1016/j.engappai.2023.106142
Bai, S., Ma, M., Yang, L., & Liu, Y. (2024). Pixel-wise crack defect segmentation with dual-encoder fusion network. Construction and Building Materials, 426, 136179. https://doi.org/10.1016/j.conbuildmat.2024.136179
Chen, T., Cai, Z., Zhao, X., Chen, C., Liang, X., Zou, T., & Wang, P. (2020). Pavement crack detection and recognition using the architecture of segNet. Journal of Industrial Information Integration, 18, 100144. https://doi.org/10.1016/j.jii.2020.100144
Dong, C., Li, L., Yan, J., Zhang, Z., Pan, H., & Catbas, F. N. (2021). Pixel-level fatigue crack segmentation in large-scale images of steel structures using an encoder–decoder network. Sensors, 21(12), 4135. https://doi.org/10.3390/s21124135
Fan, Z., Lin, H., Li, C., Su, J., Bruno, S., & Loprencipe, G. (2022). Use of Parallel ResNet for High-Performance Pavement Crack Detection and Measurement. Sustainability (Switzerland), 14(3), 1825. https://doi.org/10.3390/su14031825
Golding, V. P., Gharineiat, Z., Munawar, H. S., & Ullah, F. (2022). Crack Detection in Concrete Structures Using Deep Learning. Sustainability (Switzerland), 14(13), 8117. https://doi.org/10.3390/su14138117
Guillermo, M., Billones, R. K., Bandala, A., Vicerra, R. R., Sybingco, E., Dadios, E. P., & Fillone, A. (2020). Implementation of Automated Annotation through Mask RCNN Object Detection model in CVAT using AWS EC2 Instance. IEEE Region 10 Annual International Conference, Proceedings/TENCON, 2020-November, 708–713. https://doi.org/10.1109/TENCON50793.2020.9293906
Hang, J., Wu, Y., Li, Y., Lai, T., Zhang, J., & Li, Y. (2023). A deep learning semantic segmentation network with attention mechanism for concrete crack detection. Structural Health Monitoring, 22(5), 3006–3026. https://doi.org/10.1177/14759217221126170
He, T., Li, H., Qian, Z., Niu, C., & Huang, R. (2024). Research on Weakly Supervised Pavement Crack Segmentation Based on Defect Location by Generative Adversarial Network and Target Re‐optimization. Construction and Building Materials, 411, 134668. https://doi.org/10.1016/j.conbuildmat.2023.134668
He, Z., & Xu, W. (2024). Deep learning and image preprocessing-based crack repair trace and secondary crack classification detection method for concrete bridges. Structure and Infrastructure Engineering, 1–17. https://doi.org/10.1080/15732479.2024.2330702
Huang, H., Zhao, S., Zhang, D., & Chen, J. (2022). Deep learning-based instance segmentation of cracks from shield tunnel lining images. Structure and Infrastructure Engineering, 18(2), 183–196. https://doi.org/10.1080/15732479.2020.1838559
Khanam, R., & Hussain, M. (2024). YOLOv11: An Overview of the Key Architectural Enhancements. abs/2410.17725. http://arxiv.org/abs/2410.17725
Liu, K., & Chen, B. M. (2023). Industrial UAV-Based Unsupervised Domain Adaptive Crack Recognitions: From Database Towards Real-Site Infrastructural Inspections. IEEE Transactions on Industrial Electronics, 70(9), 9410–9420. https://doi.org/10.1109/TIE.2022.3204953
Liu, Z., Yeoh, J. K. W., Gu, X., Dong, Q., Chen, Y., Wu, W., Wang, L., & Wang, D. (2023). Automatic pixel-level detection of vertical cracks in asphalt pavement based on GPR investigation and improved mask R-CNN. Automation in Construction, 146, 104689. https://doi.org/10.1016/j.autcon.2022.104689
Luo, J., Lin, H., Wei, X., & Wang, Y. (2023). Adaptive Canny and Semantic Segmentation Networks Based on Feature Fusion for Road Crack Detection. IEEE Access, 11, 51740–51753. https://doi.org/10.1109/ACCESS.2023.3279888
Nguyen, A., Gharehbaghi, V., Le, N. T., Sterling, L., Chaudhry, U. I., & Crawford, S. (2023). ASR crack identification in bridges using deep learning and texture analysis. Structures, 50, 494–507. https://doi.org/10.1016/j.istruc.2023.02.042
ÖNLER, & Eray. (2018). Image Augmentation In Agriculture Using The Albumentations Library. New Trends in Agriculture, Forestry and Aquaculture Sciences, abs/1809.06839, 89. https://www.researchgate.net/publication/366685124
Ranyal, E., Sadhu, A., & Jain, K. (2024). Enhancing pavement health assessment: An attention-based approach for accurate crack detection, measurement, and mapping. Expert Systems with Applications, 247, 123314. https://doi.org/10.1016/j.eswa.2024.123314
Shen, X., Lu, X., Guo, J., Liu, Y., Qi, J., & Lv, Z. (2024). Nondestructive Testing of Metal Cracks: Contemporary Methods and Emerging Challenges. Crystals, 14(1), 54. https://doi.org/10.3390/cryst14010054
Su, H., Wang, X., Han, T., Wang, Z., Zhao, Z., & Zhang, P. (2022). Research on a U-Net Bridge Crack Identification and Feature-Calculation Methods Based on a CBAM Attention Mechanism. Buildings, 12(10), 1561. https://doi.org/10.3390/buildings12101561
Tran, T. S., Nguyen, S. D., Lee, H. J., & Tran, V. P. (2023). Advanced crack detection and segmentation on bridge decks using deep learning. Construction and Building Materials, 400, 132839. https://doi.org/10.1016/j.conbuildmat.2023.132839
Wang, S., Pan, Y., Chen, M., Zhang, Y., & Wu, X. (2020). FCN-SFW: Steel Structure Crack Segmentation Using a Fully Convolutional Network and Structured Forests. IEEE Access, 8, 214358–214373. https://doi.org/10.1109/ACCESS.2020.3040939
Wang, W., & Su, C. (2021). Semi-supervised semantic segmentation network for surface crack detection. Automation in Construction, 128, 103786. https://doi.org/10.1016/j.autcon.2021.103786
Wang, Z., Yang, J., Jiang, H., & Fan, X. (2020). CNN training with twenty samples for crack detection via data augmentation. Sensors (Switzerland), 20(17), 1–17. https://doi.org/10.3390/s20174849
Watari, T., Nansai, K., Giurco, D., Nakajima, K., McLellan, B., & Helbig, C. (2020). Global Metal Use Targets in Line with Climate Goals. Environmental Science and Technology, 54(19), 12476–12483. https://doi.org/10.1021/acs.est.0c02471
Xu, Y., Li, D., Xie, Q., Wu, Q., & Wang, J. (2021). Automatic defect detection and segmentation of tunnel surface using modified Mask R-CNN. Measurement: Journal of the International Measurement Confederation, 178, 109316. https://doi.org/10.1016/j.measurement.2021.109316
Yamane, T., & Chun, P. (2020). Crack Detection from a Concrete Surface Image Based on Semantic Segmentation Using Deep Learning. Journal of Advanced Concrete Technology, 18(9), 493–504. https://doi.org/10.3151/jact.18.493
Zhang, H., Qian, Z., Tan, Y., Xie, Y., & Li, M. (2022). Investigation of pavement crack detection based on deep learning method using weakly supervised instance segmentation framework. Construction and Building Materials, 358, 129117. https://doi.org/10.1016/j.conbuildmat.2022.129117
Zhang, J., Qian, S., & Tan, C. (2023). Automated bridge crack detection method based on lightweight vision models. Complex and Intelligent Systems, 9(2), 1639–1652. https://doi.org/10.1007/s40747-022-00876-6
