Development of validation standards for deep learning models in the context of industrial modelling and optimisation of production lines for defect detection in industrial facilities

Kanan Mikayilov Latafat Gardashova
Received 10.06.2025, Revised 02.11.2025, Accepted 24.12.2025
Abstract

High-speed visual inspection in modern manufacturing suffers when laboratory metrics fail to predict f ield outcomes, causing waste, rework, and safety risks under domain shift and tight cycle-time budgets. The aim of this study was to establish an industry-oriented validation standard for deep-learning defect detection that is explicitly tied to production risk and deployability, with emphasis on operation at very low false-positive rates, high-percentile latency limits, and reproducible procedures for robustness, calibration, and explainability. The methodological design combined analysis of prevailing practices with a digital-twin evaluation of high-speed inspection streams, linking laboratory trials to an anonymised production line. Results dominated the findings: recall measured at very low false-positive rates showed the strongest absolute correlations with field-proximal outcomes (≈ 0.85 -0.88), partial area over the 0-5% false-positive band ranked second, and overall accuracy was weak (≈ 0.38-0.42). Latency acted as an acceptance gate: a configuration near 75 ms (p95) and 100 ms (p99) achieved either maximum realised throughput (≈ 810 parts per minute) or minimum errors, whereas a slower configuration at 141 ms and 171 ms coincided with ≈ 540 parts per minute and higher error counts. Structured validation reduced false positives from 84 to 66 per ten thousand units (-21.4%) and false negatives from 61 to 49 (-19.7%). Under controlled operating shifts, recall fell by 12.9-16.2% without adaptation and by 4.1-6% with threshold-only online calibration; the harmonic mean of precision and recall fell by 8.7-11.5% without adaptation and stabilised to 3.1-4.4% with calibration. Reproducibility and integration were evidenced by coefficients of variation of 2.1 2.4%, inter-operator threshold variance of 0.07-0.09, explainability compliance of 92.8-94.1% of batches at an overlap threshold of 0.5, service uptime of 99.6-99.8%, and deterministic rollback. The resulting standard yields practical value by specifying acceptance targets (prioritise recall at very low false positive rate; enforce p95 ≈ 75 ms/p99 ≈ 100 ms to sustain ≈ 810 ppm), prescribing lightweight online threshold calibration to cap shift-induced losses, and supplying auditable key performance indicators (variation bounds, explainability compliance, uptime, rollback) for commissioning checklists and supplier contracts to reduce deployment risk and life-cycle cost

Keywords:
automated quality control; latency constraints; domain shifts; online calibration; explainability; reproducibility; operational sustainability

Retrieved from Volume 23, No. 2, 2025

Pages 46–58

References Suggested citation
  1. Adeyemi, T.S. (2024). Defect detection in manufacturing: An integrated deep learning approach. Journal of Computer and Communications, 12(10), 153-176. doi: 10.4236/jcc.2024.1210011.
  2. Ameri, R., Hsu, C.-C., & Band, S.S. (2024). A systematic review of deep learning approaches for surface defect detection in industrial applications. Engineering Applications of Artificial Intelligence, 130(3), article number 107717. doi: 10.1016/j.engappai.2023.107717.
  3. Avola, D., Cascio, M., Cinque, L., Fagioli, A., Foresti, G., Marini, M., & Rossi, F. (2022). Real-time deep learning method for automated detection and localisation of structural defects in manufactured products. Computers & Industrial Engineering, 172(A), article number 108512. doi: 10.1016/j.cie.2022.108512.
  4. Bai, J., Wu, D., Shelley, T., Schubel, P., Twine, D., Russell, J., Zeng, X., & Zhang, J. (2024). A comprehensive survey on machine learning driven material defect detection. ACM Computing Surveys, 52(11), article number 275. doi: 10.1145/3730576.
  5. Bimrose, M.V., McGregor, D.J., Wood, C., Tawfick, S., & King, W.P. (2025). Additive manufacturing source identification from photographs using deep learning. NPJ Advanced Manufacturing, 2(1), article number 20. doi: 10.1038/s44334-025-00031-2.
  6. Chen, Y., Ding, Y., Fan, Z., Zhang, E., Wu, Z., & Shao, L. (2021). Surface defect detection methods for industrial products: A review. Applied Sciences, 11(16), article number 7657. doi: 10.3390/app11167657.
  7. Ettalibi, A., Elouadi, A., & Mansour, A. (2024). AI and computer vision-based real-time quality control: A review of industrial applications. Procedia Computer Science, 231, 212-220. doi: 10.1016/j.procs.2023.12.195.
  8. Hicham, T., Khalifa, M., Kamal, E.G., & Fatiha, A. (2024). Machine and deep learning applications in Industry 4.0. In International conference on technology, engineering, and computing applications (pp. 1-5). Semarang: IEEE. doi: 10.1109/ICTECA60133.2023.10490844.
  9. Ige, A.B., Adepoju, P.A., Akinade, A.O., & Afolabi, A.I. (2024). Machine learning in industrial applications: An in-depth review and future directions. International Journal of Multidisciplinary Research and Growth Evaluation, 6(1), 36-44. doi: 10.54660/.IJMRGE.2025.6.1.36-44.
  10. Jamwal, A., Agrawal, R., & Sharma, M. (2022). Deep learning for manufacturing sustainability: Models, applications in Industry 4.0 and implications. International Journal of Information Management Data Insights, 2(2), article number 100107. doi: 10.1016/j.jjimei.2022.100107.
  11. Jia, Z., Wang, M., & Zhao, S. (2023). A review of deep learning-based approaches for defect detection in smart manufacturing. Journal of Optics, 53, 1345-1351. doi: 10.1007/s12596-023-01340-5.
  12. Jing, Y., Li, S., Wang, Z., Dong, H., Wang, J., & Tang, S. (2020). Using deep learning to detect defects in manufacturing: A comprehensive survey and current challenges. Materials, 13(24), article number 5755. doi: 10.3390/ma13245755.
  13. Kaixin, L., Ma, Z., Liu, Y., Yang, J., & Yao, Y. (2021). Enhanced defect detection in carbon fiber reinforced polymer composites via generative kernel principal component thermography. Polymers, 13(5), article number 825. doi: 10.3390/polym13050825.
  14. Kausik, A.K., Rashid, A.B., Baki, R.F., & Maktum, M.M.J. (2025). Machine learning algorithms for manufacturing quality assurance: A systematic review of performance metrics and applications. Array, 26, article number 100393. doi: 10.1016/j.array.2025.100393.
  15. Khan, A.S., Akram, M.U., Khattak, M.A., & Jawed, S. (2024). Deep learning based approaches for intelligent industrial machinery health management & fault diagnosis in resource-constrained environments. Scientific Reports, 15, article number 1114. doi: 10.1038/s41598-024-79151-2.
  16. Khanam, R., Hussain, M., Hill, R., & Allen, P. (2024). A comprehensive review of convolutional neural networks for defect detection in industrial applications. IEEE Access, 12, 94250-94295. doi: 10.1109/ACCESS.2024.3425166.
  17. Kim, M.J., Hussain, A., Munsif, M., & Yoon, S.I. (2023). Industrial defective chip inspection using deep convolutional neural network with attention mechanism. In Conference of Korean institute of next generation computing spring 2023 (pp. 51-54). Changwon-si: KAIST.
  18. Kumari, S., Prabha, C., Karim, A., Hassan, M.M., & Azam, S. (2024). A comprehensive investigation of anomaly detection methods in deep learning and machine learning: 2019-2023. IET Information Security, 2024(1), article number 8821891. doi: 10.1049/2024/8821891.
  19. Lee, C., Kim, Y., & Kim, H. (2024). Computer vision-based product quality inspection and novel counting system. Applied System Innovation, 7, article number 127. doi: 10.20944/preprints202411.0133.v1.
  20. Li, W., & Li, T. (2025). Comparison of deep learning models for predictive maintenance in industrial manufacturing systems using sensor data. Scientific Reports, 15, article number 23545. doi: 10.1038/s41598-025-08515-z.
  21. Luo, Q., Fang, X., Sun, Y., Liu, L., Ai, J., & Yang, C. (2019). Surface defect classification for hot-rolled steel strips by selectively dominant local binary patterns. IEEE Access, 7, 23488-23499. doi: 10.1109/ACCESS.2019.2898215.
  22. Ma, Y., Yin, J., Huang, F., & Li, Q. (2024). Surface defect inspection of industrial products with object detection deep networks: A systematic review. Artificial Intelligence Review, 57, article number 333. doi: 10.1007/s10462024-10956-3.
  23. Maldague, X. (2019). Automatic defect detection for X-Ray inspection: Identifying defects with deep convolutional network. e-Journal of Nondestructive Testing (eJNDT), 24(10).
  24. Mayuravaani, M., & Manivannan, S. (2021). A semi-supervised deep learning approach for the classification of steel surface defects. In International conference on information and automation for sustainability (pp. 179-184). Negambo: IEEE. doi: 10.1109/ICIAfS52090.2021.9606143.
  25. Mezher, A.M., & Marble, A.E. (2023). Computer vision defect detection on unseen backgrounds for manufacturing inspection. Expert Systems with Applications, 243, article number 122749. doi: 10.1016/j.eswa.2023.122749.
  26. Ördek, B., Borgianni, Y., & Coatanéa, E. (2024). Machine learning-supported manufacturing: A review and directions for future research. Production & Manufacturing Research, 12(1), article number 2326526. doi: 10.1080/21693277.2024.2326526.
  27. Park, S.-H., Lee, K.-H., Park, J.-S., & Shin, Y.-S. (2022). Deep learning-based defect detection for sustainable smart manufacturing. Sustainability, 14(5), article number 2697. doi: 10.3390/su14052697.
  28. Raj, A., Bosch, J., Olsson, H.H., Arpteg, A., & Brinne, B. (2022). Data management for production quality deep learning models: Challenges and solutions. Journal of Systems and Software, 191(6), article number 111359.doi: 10.1016/j.jss.2022.111359.
  29. Sundaram, S., & Zeid, A. (2023). Artificial intelligence-based smart quality inspection for manufacturing. Micromachines, 14(3), article number 570. doi: 10.3390/mi14030570.
  30. Tsai, D., & Jen, P. (2021). Autoencoder-based anomaly detection for surface defect inspection. Advanced Engineering Informatics, 48, article number 101272. doi: 10.1016/j.aei.2021.101272.
  31. Ullah, W., Khan, S.U., Kim, M.J., Hussain, A., Munsif, M., Lee, M., Seo, D., & Baik, S. (2024). Industrial defective chips detection using deep convolutional neural network with inverse feature matching mechanism. Journal of Computational Design and Engineering, 11(3), 326-336. doi: 10.1093/jcde/qwae019.
  32. Villegas, W., Gaibor-Naranjo, W., & Sanchez-Viteri, S. (2024). Application of deep learning techniques for the optimisation of industrial processes through the fusion of sensory data. International Journal of Computational Intelligence Systems, 17, article number 187. doi: 10.1007/s44196-024-00596-4.
  33. Yakovenko, I., Dmytrenko, Y., & Bakulina, V. (2022). Construction of analytical coupling model in reinforced concrete structures in the presence of discrete cracks. In A. Bieliatynskyi & V. Breskich (Eds.), Safety in aviation and space technologies. Lecture notes in mechanical engineering (pp. 107-120). Cham: Springer. doi: 10.1007/9783-030-85057-9_10.
  34. Zhang, Z., Zhou, M., Wan, H., Li, M., Li, G., & Han, D. (2023). IDD-Net: Industrial defect detection method based on deep-learning. Engineering Applications of Artificial Intelligence, 123(B), article number 106390. doi: 10.1016/j. engappai.2023.106390.
Mikayilov, K., & Gardashova, L. (2025). Development of validation standards for deep learning models in the context of industrial modelling and optimisation of production lines for defect detection in industrial facilities. Journal of Kryvyi Rih National University, 23(2), 46-58. https://doi.org/10.31721/2306-5451-2025-2-23-46-58
en