A real-time defect detection in printed circuit boards applying deep learning
Abstract
Inspection of defects in the printed circuit boards (PCBs) has both safety and economic significance in the 4.0 industrial manufacturing. Nevertheless, it is still a challenging problem to be studied in-depth due to the complexity of the PCB layouts and the shrinking down tendency of the electronic component size. In this paper, a real-time automated supervision algorithm is proposed to test the PCBs quality among different scenarios. The density of the PCBs layout and the complexity on the surface are analyzed based on deep learning and image feature extraction algorithms. To be more detailed, the ORB feature and the Brute-force matching method are utilized to match perfectly the input images with the PCB templates. After transferring images by aiding the RANSAC algorithm, a hybrid method using modern computer vision algorithms is developed to segment defective areas on the PCBs surface. Then, by applying the enhanced Residual Network –50, the proposed algorithm can classify the groove defects on the surface mount technology electronic components which minimum size up to 1x3 mm. After the training process, the proposed system is capable to categorize various types of overproduced, recycled, and cloned PCBs. The speed of the quality testing operation maintains at a high level with an average precision rate up to 96.29 % in case of good brightness conditions. Finally, the computational experiments demonstrate that the proposed system based on deep learning can obtain superior results and it outperforms several existing works in terms of speed, precision, and robustness
Downloads
References
Zhang, Y. (2021). Improved Numerical-Analytical Thermal Modeling Method of the PCB With Considering Radiation Heat Transfer and Calculation of Components’ Temperature. IEEE Access, 9, 92925–92940. doi: https://doi.org/10.1109/access.2021.3093098
Khiabani, N., Chiang, C.-W., Liu, N.-C., Kuan, Y.-C., Wu, C.-T. M. (2021). An Ultrawide Ku- To W-Band Array Antenna Package Using Flip-Chipped Silicon Integrated Passive Device With Multilayer PCB Technology. IEEE Microwave and Wireless Components Letters, 31 (7), 861–864. doi: https://doi.org/10.1109/lmwc.2021.3074940
Rajabzadeh, M., Ungethum, J., Herkle, A., Schilpp, C., Becker, J., Fauler, M. et. al. (2021). A PCB-Based 24-Ch. MEA-EIS Allowing Fast Measurement of TEER. IEEE Sensors Journal, 21 (12), 13048–13059. doi: https://doi.org/10.1109/jsen.2021.3067823
Llamazares, A., Garcia-Gracia, M., Martin-Arroyo, S. (2021). Characterization of Parasitic Impedance in PCB Using a Flexible Test Probe Based on a Curve-Fitting Method. IEEE Access, 9, 40695–40705. doi: https://doi.org/10.1109/access.2021.3064190
Huang, C.-M., Wang, S.-H., Wu, T.-Y., Huang, M.-C., Wu, R.-B. (2021). Systematic Design for Mitigation of RF Desense by Interleaved Power Line in Two-Layer PCB. IEEE Transactions on Components, Packaging and Manufacturing Technology, 11 (5), 859–864. doi: https://doi.org/10.1109/tcpmt.2021.3063321
Jin, W., Lin, W., Yang, X., Gao, H. (2017). Reference-Free Path-Walking Method for Ball Grid Array Inspection in Surface Mounting Machines. IEEE Transactions on Industrial Electronics, 64 (8), 6310–6318. doi: https://doi.org/10.1109/tie.2017.2682008
Botero, U. J., Ganji, F., Asadizanjani, N., Woodard, D. L., Forte, D. (2020). Semi-Supervised Automated Layer Identification of X-ray Tomography Imaged PCBs. 2020 IEEE Physical Assurance and Inspection of Electronics (PAINE). doi: https://doi.org/10.1109/paine49178.2020.9337738
Xia, S., Wang, F., Xie, F., Huang, L., Wang, Q., Ling, X. (2021). An Efficient and Robust Target Detection Algorithm for Identifying Minor Defects of Printed Circuit Board Based on PHFE and FL-RFCN. Frontiers in Physics, 9. doi: https://doi.org/10.3389/fphy.2021.661091
Zhang, B., Yang, H., Yin, Z. (2015). A Region-Based Normalized Cross Correlation Algorithm for the Vision-Based Positioning of Elongated IC Chips. IEEE Transactions on Semiconductor Manufacturing, 28 (3), 345–352. doi: https://doi.org/10.1109/tsm.2015.2430453
Romero Subirón, F., Rosado Castellano, P., Bruscas Bellido, G., Benavent Nácher, S. (2018). Feature-Based Framework for Inspection Process Planning. Materials, 11 (9), 1504. doi: https://doi.org/10.3390/ma11091504
Zhou, Z., Wang, M., Cao, Y., Su, Y. (2020). CNN Feature-Based Image Copy Detection with Contextual Hash Embedding. Mathematics, 8 (7), 1172. doi: https://doi.org/10.3390/math8071172
Ghaffari, A., Fatemizadeh, E. (2018). Image Registration Based on Low Rank Matrix: Rank-Regularized SSD. IEEE Transactions on Medical Imaging, 37 (1), 138–150. doi: https://doi.org/10.1109/tmi.2017.2744663
Chandran, K. R. S., Chandramani, P. V. (2019). Energy‐efficient system‐on‐chip reconfigurable architecture design for sum of absolute difference computation in motion estimation process of H.265/HEVC video encoding. Concurrency and Computation: Practice and Experience, 34 (8). doi: https://doi.org/10.1002/cpe.5461
Zhang, G., Kuang, Z., Wei, S., Huang, K., Liang, F., Yang, C.-F. (2018). Hardware Implementation for an Improved Full-Pixel Search Algorithm Based on Normalized Cross Correlation Method. Electronics, 7 (12), 428. doi: https://doi.org/10.3390/electronics7120428
Zhang, H., Song, A. (2017). A map-based normalized cross correlation algorithm using dynamic template for vision-guided telerobot. Advances in Mechanical Engineering, 9 (9), 168781401772883. doi: https://doi.org/10.1177/1687814017728839
Kaur, P., Pannu, H. S. (2018). Comprehensive review of continuous and discrete orthogonal moments in biometrics. International Journal of Computer Mathematics: Computer Systems Theory, 3 (2), 64–91. doi: https://doi.org/10.1080/23799927.2018.1457080
He, H., Hu, Z., Wang, B., Luo, D., Lee, W.-J., Li, J. (2020). A Contactless Zero-Value Insulators Detection Method Based on Infrared Images Matching. IEEE Access, 8, 133882–133889. doi: https://doi.org/10.1109/access.2020.3011170
Tsai, D., Hsieh, Y. (2017). Machine Vision-Based Positioning and Inspection Using Expectation–Maximization Technique. IEEE Transactions on Instrumentation and Measurement, 66 (11), 2858–2868. doi: https://doi.org/10.1109/tim.2017.2717284
Edalatifar, M., Tavakoli, M., Setoudeh, F. (2021). A Deep Learning Approach to Predict the Flow Field and Thermal Patterns of Nonencapsulated Phase Change Materials Suspensions in an Enclosure. Journal of Applied and Computational Mechanics. doi: https://doi.org/10.22055/jacm.2021.37805.3092
Alzghoul, A., Jarndal, A., Alsyouf, I., Bingamil, A., Ali, M., AlBaiti, S. (2021). On the Usefulness of Pre-processing Methods in Rotating Machines Faults Classification using Artificial Neural Network. Journal of Applied and Computational Mechanics, 7 (1), 254–261. doi: https://doi.org/10.22055/jacm.2020.35354.2639
Gao, Q., Wu, X. (2021). Real-Time Deep Image Retouching Based on Learnt Semantics Dependent Global Transforms. IEEE Transactions on Image Processing, 30, 7378–7390. doi: https://doi.org/10.1109/tip.2021.3104173
Wang, Y., Ji, S., Zhang, Y. (2021). A Learnable Joint Spatial and Spectral Transformation for High Resolution Remote Sensing Image Retrieval. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14, 8100–8112. doi: https://doi.org/10.1109/jstars.2021.3103216
Attaran, B., Ghanbarzadeh, A. (2014). Bearing Fault Detection Based on Maximum Likelihood Estimation and Optimized ANN Using the Bees Algorithm, Journal of Applied and Computational Mechanics, 1 (1), 35–43. doi: https://doi.org/10.22055/jacm.2014.10547
Tsai, D.-M., Chou, Y.-H. (2020). Fast and Precise Positioning in PCBs Using Deep Neural Network Regression. IEEE Transactions on Instrumentation and Measurement, 69 (7), 4692–4701. doi: https://doi.org/10.1109/tim.2019.2957866
Hu, B., Wang, J. (2020). Detection of PCB Surface Defects With Improved Faster-RCNN and Feature Pyramid Network. IEEE Access, 8, 108335–108345. doi: https://doi.org/10.1109/access.2020.3001349
Park, J.-Y., Hwang, Y., Lee, D., Kim, J.-H. (2020). MarsNet: Multi-Label Classification Network for Images of Various Sizes. IEEE Access, 8, 21832–21846. doi: https://doi.org/10.1109/access.2020.2969217
Zhang, Q., Liu, H. (2021). Multi-scale defect detection of printed circuit board based on feature pyramid network. 2021 IEEE International Conference on Artificial Intelligence and Computer Applications (ICAICA). doi: https://doi.org/10.1109/icaica52286.2021.9498174
Adibhatla, V. A., Chih, H.-C., Hsu, C.-C., Cheng, J., Abbod, M. F., Shieh, J.-S. (2020). Defect Detection in Printed Circuit Boards Using You-Only-Look-Once Convolutional Neural Networks. Electronics, 9 (9), 1547. doi: https://doi.org/10.3390/electronics9091547
Nguyen, V.-T., Nguyen, A.-T., Nguyen, V.-T., Bui, H.-A., Nguyen, X.-T. (2021). Real-time Target Human Tracking using Camshift and LucasKanade Optical Flow Algorithm. Advances in Science, Technology and Engineering Systems Journal, 6 (2), 907–914. doi: https://doi.org/10.25046/aj0602103
Nguyen, V.-T., Nguyen, A.-T., Nguyen, V.-T., Bui, H.-A. (2021). A Real-Time Human Tracking System Using Convolutional Neural Network and Particle Filter. Lecture Notes in Networks and Systems, 411–417. doi: https://doi.org/10.1007/978-981-16-2094-2_50
Helwan, A., Sallam Ma’aitah, M. K., Abiyev, R. H., Uzelaltinbulat, S., Sonyel, B. (2021). Deep Learning Based on Residual Networks for Automatic Sorting of Bananas. Journal of Food Quality, 2021, 1–11. doi: https://doi.org/10.1155/2021/5516368
Liu, C., Zhou, W., Chen, Y., Lei, J. (2020). Asymmetric Deeply Fused Network for Detecting Salient Objects in RGB-D Images. IEEE Signal Processing Letters, 27, 1620–1624. doi: https://doi.org/10.1109/lsp.2020.3023349
Akritas, A. G., Malaschonok, G. I. (2004). Applications of singular-value decomposition (SVD). Mathematics and Computers in Simulation, 67 (1-2), 15–31. doi: https://doi.org/10.1016/j.matcom.2004.05.005
Tran, Q.-V., Su, S.-F., Nguyen, V.-T. (2020). Pyramidal Lucas – Kanade-Based Noncontact Breath Motion Detection. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 50 (7), 2659–2670. doi: https://doi.org/10.1109/tsmc.2018.2825458
Ma, S.-Y., Khalil, A., Hajjdiab, H., Eleuch, H. (2020). Quantum Dilation and Erosion. Applied Sciences, 10 (11), 4040. doi: https://doi.org/10.3390/app10114040
Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollar, P. (2020). Focal Loss for Dense Object Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42 (2), 318–327. doi: https://doi.org/10.1109/tpami.2018.2858826
Copyright (c) 2022 Van-Truong Nguyen, Huy-Anh Bui
This work is licensed under a Creative Commons Attribution 4.0 International License.
Our journal abides by the Creative Commons CC BY copyright rights and permissions for open access journals.
Authors, who are published in this journal, agree to the following conditions:
1. The authors reserve the right to authorship of the work and pass the first publication right of this work to the journal under the terms of a Creative Commons CC BY, which allows others to freely distribute the published research with the obligatory reference to the authors of the original work and the first publication of the work in this journal.
2. The authors have the right to conclude separate supplement agreements that relate to non-exclusive work distribution in the form in which it has been published by the journal (for example, to upload the work to the online storage of the journal or publish it as part of a monograph), provided that the reference to the first publication of the work in this journal is included.
