A real-time defect detection in printed circuit boards applying deep learning

Keywords: Printed circuit board, defect detection, deep learning, computer vision, RESNET

Abstract

Inspection of defects in the printed circuit boards (PCBs) has both safety and economic significance in the 4.0 industrial manufacturing. Nevertheless, it is still a challenging problem to be studied in-depth due to the complexity of the PCB layouts and the shrinking down tendency of the electronic component size. In this paper, a real-time automated supervision algorithm is proposed to test the PCBs quality among different scenarios. The density of the PCBs layout and the complexity on the surface are analyzed based on deep learning and image feature extraction algorithms. To be more detailed, the ORB feature and the Brute-force matching method are utilized to match perfectly the input images with the PCB templates. After transferring images by aiding the RANSAC algorithm, a hybrid method using modern computer vision algorithms is developed to segment defective areas on the PCBs surface. Then, by applying the enhanced Residual Network –50, the proposed algorithm can classify the groove defects on the surface mount technology electronic components which minimum size up to 1x3 mm. After the training process, the proposed system is capable to categorize various types of overproduced, recycled, and cloned PCBs. The speed of the quality testing operation maintains at a high level with an average precision rate up to 96.29 % in case of good brightness conditions. Finally, the computational experiments demonstrate that the proposed system based on deep learning can obtain superior results and it outperforms several existing works in terms of speed, precision, and robustness

Downloads

Download data is not yet available.

Author Biographies

Van-Truong Nguyen, Hanoi University of Industry

Department of Mechatronics Engineering

Huy-Anh Bui, Hanoi University of Industry

Department of Mechatronics Engineering

References

Zhang, Y. (2021). Improved Numerical-Analytical Thermal Modeling Method of the PCB With Considering Radiation Heat Transfer and Calculation of Components’ Temperature. IEEE Access, 9, 92925–92940. doi: https://doi.org/10.1109/access.2021.3093098

Khiabani, N., Chiang, C.-W., Liu, N.-C., Kuan, Y.-C., Wu, C.-T. M. (2021). An Ultrawide Ku- To W-Band Array Antenna Package Using Flip-Chipped Silicon Integrated Passive Device With Multilayer PCB Technology. IEEE Microwave and Wireless Components Letters, 31 (7), 861–864. doi: https://doi.org/10.1109/lmwc.2021.3074940

Rajabzadeh, M., Ungethum, J., Herkle, A., Schilpp, C., Becker, J., Fauler, M. et. al. (2021). A PCB-Based 24-Ch. MEA-EIS Allowing Fast Measurement of TEER. IEEE Sensors Journal, 21 (12), 13048–13059. doi: https://doi.org/10.1109/jsen.2021.3067823

Llamazares, A., Garcia-Gracia, M., Martin-Arroyo, S. (2021). Characterization of Parasitic Impedance in PCB Using a Flexible Test Probe Based on a Curve-Fitting Method. IEEE Access, 9, 40695–40705. doi: https://doi.org/10.1109/access.2021.3064190

Huang, C.-M., Wang, S.-H., Wu, T.-Y., Huang, M.-C., Wu, R.-B. (2021). Systematic Design for Mitigation of RF Desense by Interleaved Power Line in Two-Layer PCB. IEEE Transactions on Components, Packaging and Manufacturing Technology, 11 (5), 859–864. doi: https://doi.org/10.1109/tcpmt.2021.3063321

Jin, W., Lin, W., Yang, X., Gao, H. (2017). Reference-Free Path-Walking Method for Ball Grid Array Inspection in Surface Mounting Machines. IEEE Transactions on Industrial Electronics, 64 (8), 6310–6318. doi: https://doi.org/10.1109/tie.2017.2682008

Botero, U. J., Ganji, F., Asadizanjani, N., Woodard, D. L., Forte, D. (2020). Semi-Supervised Automated Layer Identification of X-ray Tomography Imaged PCBs. 2020 IEEE Physical Assurance and Inspection of Electronics (PAINE). doi: https://doi.org/10.1109/paine49178.2020.9337738

Xia, S., Wang, F., Xie, F., Huang, L., Wang, Q., Ling, X. (2021). An Efficient and Robust Target Detection Algorithm for Identifying Minor Defects of Printed Circuit Board Based on PHFE and FL-RFCN. Frontiers in Physics, 9. doi: https://doi.org/10.3389/fphy.2021.661091

Zhang, B., Yang, H., Yin, Z. (2015). A Region-Based Normalized Cross Correlation Algorithm for the Vision-Based Positioning of Elongated IC Chips. IEEE Transactions on Semiconductor Manufacturing, 28 (3), 345–352. doi: https://doi.org/10.1109/tsm.2015.2430453

Romero Subirón, F., Rosado Castellano, P., Bruscas Bellido, G., Benavent Nácher, S. (2018). Feature-Based Framework for Inspection Process Planning. Materials, 11 (9), 1504. doi: https://doi.org/10.3390/ma11091504

Zhou, Z., Wang, M., Cao, Y., Su, Y. (2020). CNN Feature-Based Image Copy Detection with Contextual Hash Embedding. Mathematics, 8 (7), 1172. doi: https://doi.org/10.3390/math8071172

Ghaffari, A., Fatemizadeh, E. (2018). Image Registration Based on Low Rank Matrix: Rank-Regularized SSD. IEEE Transactions on Medical Imaging, 37 (1), 138–150. doi: https://doi.org/10.1109/tmi.2017.2744663

Chandran, K. R. S., Chandramani, P. V. (2019). Energy‐efficient system‐on‐chip reconfigurable architecture design for sum of absolute difference computation in motion estimation process of H.265/HEVC video encoding. Concurrency and Computation: Practice and Experience, 34 (8). doi: https://doi.org/10.1002/cpe.5461

Zhang, G., Kuang, Z., Wei, S., Huang, K., Liang, F., Yang, C.-F. (2018). Hardware Implementation for an Improved Full-Pixel Search Algorithm Based on Normalized Cross Correlation Method. Electronics, 7 (12), 428. doi: https://doi.org/10.3390/electronics7120428

Zhang, H., Song, A. (2017). A map-based normalized cross correlation algorithm using dynamic template for vision-guided telerobot. Advances in Mechanical Engineering, 9 (9), 168781401772883. doi: https://doi.org/10.1177/1687814017728839

Kaur, P., Pannu, H. S. (2018). Comprehensive review of continuous and discrete orthogonal moments in biometrics. International Journal of Computer Mathematics: Computer Systems Theory, 3 (2), 64–91. doi: https://doi.org/10.1080/23799927.2018.1457080

He, H., Hu, Z., Wang, B., Luo, D., Lee, W.-J., Li, J. (2020). A Contactless Zero-Value Insulators Detection Method Based on Infrared Images Matching. IEEE Access, 8, 133882–133889. doi: https://doi.org/10.1109/access.2020.3011170

Tsai, D., Hsieh, Y. (2017). Machine Vision-Based Positioning and Inspection Using Expectation–Maximization Technique. IEEE Transactions on Instrumentation and Measurement, 66 (11), 2858–2868. doi: https://doi.org/10.1109/tim.2017.2717284

Edalatifar, M., Tavakoli, M., Setoudeh, F. (2021). A Deep Learning Approach to Predict the Flow Field and Thermal ‎Patterns of Nonencapsulated Phase Change Materials ‎Suspensions in an Enclosure. Journal of Applied and Computational Mechanics. doi: https://doi.org/10.22055/jacm.2021.37805.3092

Alzghoul, A., Jarndal, A., Alsyouf, I., Bingamil, A., Ali, M., AlBaiti, S. (2021). On the Usefulness of Pre-processing Methods in Rotating ‎Machines Faults Classification using Artificial Neural Network. Journal of Applied and Computational Mechanics, 7 (1), 254–261. doi: https://doi.org/10.22055/jacm.2020.35354.2639

Gao, Q., Wu, X. (2021). Real-Time Deep Image Retouching Based on Learnt Semantics Dependent Global Transforms. IEEE Transactions on Image Processing, 30, 7378–7390. doi: https://doi.org/10.1109/tip.2021.3104173

Wang, Y., Ji, S., Zhang, Y. (2021). A Learnable Joint Spatial and Spectral Transformation for High Resolution Remote Sensing Image Retrieval. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14, 8100–8112. doi: https://doi.org/10.1109/jstars.2021.3103216

Attaran, B., Ghanbarzadeh, A. (2014). Bearing Fault Detection Based on Maximum Likelihood Estimation and Optimized ANN Using the Bees Algorithm, Journal of Applied and Computational Mechanics, 1 (1), 35–43. doi: https://doi.org/10.22055/jacm.2014.10547

Tsai, D.-M., Chou, Y.-H. (2020). Fast and Precise Positioning in PCBs Using Deep Neural Network Regression. IEEE Transactions on Instrumentation and Measurement, 69 (7), 4692–4701. doi: https://doi.org/10.1109/tim.2019.2957866

Hu, B., Wang, J. (2020). Detection of PCB Surface Defects With Improved Faster-RCNN and Feature Pyramid Network. IEEE Access, 8, 108335–108345. doi: https://doi.org/10.1109/access.2020.3001349

Park, J.-Y., Hwang, Y., Lee, D., Kim, J.-H. (2020). MarsNet: Multi-Label Classification Network for Images of Various Sizes. IEEE Access, 8, 21832–21846. doi: https://doi.org/10.1109/access.2020.2969217

Zhang, Q., Liu, H. (2021). Multi-scale defect detection of printed circuit board based on feature pyramid network. 2021 IEEE International Conference on Artificial Intelligence and Computer Applications (ICAICA). doi: https://doi.org/10.1109/icaica52286.2021.9498174

Adibhatla, V. A., Chih, H.-C., Hsu, C.-C., Cheng, J., Abbod, M. F., Shieh, J.-S. (2020). Defect Detection in Printed Circuit Boards Using You-Only-Look-Once Convolutional Neural Networks. Electronics, 9 (9), 1547. doi: https://doi.org/10.3390/electronics9091547

Nguyen, V.-T., Nguyen, A.-T., Nguyen, V.-T., Bui, H.-A., Nguyen, X.-T. (2021). Real-time Target Human Tracking using Camshift and LucasKanade Optical Flow Algorithm. Advances in Science, Technology and Engineering Systems Journal, 6 (2), 907–914. doi: https://doi.org/10.25046/aj0602103

Nguyen, V.-T., Nguyen, A.-T., Nguyen, V.-T., Bui, H.-A. (2021). A Real-Time Human Tracking System Using Convolutional Neural Network and Particle Filter. Lecture Notes in Networks and Systems, 411–417. doi: https://doi.org/10.1007/978-981-16-2094-2_50

Helwan, A., Sallam Ma’aitah, M. K., Abiyev, R. H., Uzelaltinbulat, S., Sonyel, B. (2021). Deep Learning Based on Residual Networks for Automatic Sorting of Bananas. Journal of Food Quality, 2021, 1–11. doi: https://doi.org/10.1155/2021/5516368

Liu, C., Zhou, W., Chen, Y., Lei, J. (2020). Asymmetric Deeply Fused Network for Detecting Salient Objects in RGB-D Images. IEEE Signal Processing Letters, 27, 1620–1624. doi: https://doi.org/10.1109/lsp.2020.3023349

Akritas, A. G., Malaschonok, G. I. (2004). Applications of singular-value decomposition (SVD). Mathematics and Computers in Simulation, 67 (1-2), 15–31. doi: https://doi.org/10.1016/j.matcom.2004.05.005

Tran, Q.-V., Su, S.-F., Nguyen, V.-T. (2020). Pyramidal Lucas – Kanade-Based Noncontact Breath Motion Detection. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 50 (7), 2659–2670. doi: https://doi.org/10.1109/tsmc.2018.2825458

Ma, S.-Y., Khalil, A., Hajjdiab, H., Eleuch, H. (2020). Quantum Dilation and Erosion. Applied Sciences, 10 (11), 4040. doi: https://doi.org/10.3390/app10114040

Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollar, P. (2020). Focal Loss for Dense Object Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42 (2), 318–327. doi: https://doi.org/10.1109/tpami.2018.2858826


👁 87
⬇ 53
Published
2022-03-31
How to Cite
Nguyen, V.-T., & Bui, H.-A. (2022). A real-time defect detection in printed circuit boards applying deep learning. EUREKA: Physics and Engineering, (2), 143-153. https://doi.org/10.21303/2461-4262.2022.002127
Section
Computer Science