A Study on Welding Bead Detection and Inspection Using Computer Vision Algorithms

Abstract

Welding activities are very hazardous and sometimes disadvantageous to its clients when it comes to accidents or uncertainty coincidences. The uncertainties might be caused by machines such as robot crushes and breakdowns which sometimes cause huge damage such as physical injuries to its clients leading to death and environmental pollution at large. Apart from causing physical injuries to the victims, if happens repeatedly, they might cause mental torture and psychological panic to workers who might be intimidated by the situation at the working station. The welding activities require regular and scheduled routines for assessment and evaluation of the machines involved in welding activities to avoid environmental bizarre such as fire outbreaks caused by defective tools. That’s why we propose an automatic multi-camera system that uses computer vision to detect and inspect the quality of welding beads in a factory. Additionally, we use the same system to evaluate the perfectness or defectiveness of the welding robot. In this study, we propose computer vision algorithms to detect and inspect welding beads on shear reinforcement with dual anchorage (SRD) which are recently used for building mansions and apartments in the field of building construction. To achieve the goal of this work, the author employed both rule-based and deep learning-based algorithms with image processing techniques for object detection, localization, and image classification. First, this study involves the development of an automatic multi-camera system for image and video frame acquisition, processing, and analysis. Second, we develop an algorithm for welding bead detection based on the Morphological Geodesic Active Contour (MGAC) by adding image processing techniques such as the inverse Gaussian operator, and histogram equalization algorithm because of the nature of the environment where the experiments were performed. The proposed algorithm, performed better however, it was slow compared to the conventional ones at a computational time of 0.68s for a single frame with an average of 0.99, 0.98, 0.99, and 0.94 metrics in terms of recall, precision, F-Measure, and IOU respectively. Geodesic active contours are well known for being slow and sometimes are not suitable for real-time applications, therefore, we propose a histogram-based algorithm to determine the lower and upper boundaries of the inhomogeneous pixel’s distribution using Chebyshev’s inequality boundaries at a clipping-off value of 25% and k = 2. The proposed algorithm performed better in terms of computational time by 20 times the first proposed one however, the metrics of evaluation were lagging at 0.94, 0.95, 0.94, 0.90 for recall, precision, F-Measure, and IOU respectively. We modified the pre-trained model (Mask-RCNN) by adding a classification algorithm to the mask layer. The model achieved a classification accuracy of 92% with just three (3) misclassified class labels, after training the model for 20 epochs, however, testing procedures took at least 1.84s per frame. Additionally, the modified Mask-RCNN achieved a segmentation accuracy of 0.96, with 1.00, and 1.00 for precision and recall values respectively regardless of the environment’s predefined conditions.