Product Defect Identification System

Abstract

Some uncontrollable defects will occur on the surface of metal work pieces during processing. The existence of surface defects not only affects the appearance of the finished product, but also affects the quality to a certain extent. Surface defect detection of metal work pieces can effectively improve product quality and production efficiency, and is an important link in the process of product quality control. This proposed system uses the convolutional neural network algorithm in deep learning to classify and detect metal surface defects. The surface defect recognition accuracy and defect detection rate of metal work is computed.

Introduction

I. INTRODUCTION

With the continuous improvement of science and technology, the intelligent, automated, and unmanned manufacturing industry will be an inevitable trend in the future. China has also put forward the "Made in China 2025" strategy to become a powerful country in science and technology to realize the transformation from a manufacturing country to a manufacturing country. As an important form of metal materials, metal workpieces are widely used in daily life and industrial production by virtue of their excellent mechanical and physical properties. In the production process of the product, due to the influence of equipment and technology, different kinds of defects often appear on the surface of the product, such as scratches, holes, and cracks in the metal work piece. The surface quality of metal workpieces not only affects the appearance and image of the product, but may also affect the functional characteristics of the product and cause significant losses to the enterprise. Therefore, it is very necessary to detect the surface defects of the product, and it is particularly important to design a real-time and effective surface defect detection method for metal workpieces. Conventional non-destructive inspection methods for surface defects of metal workpieces include magnetic particle inspection, penetrant inspection, infrared thermal imaging inspection, ultrasonic inspection, visual inspection, etc. Machine vision inspection technology is constantly being used in inspections in various fields. Machine vision inspection mainly uses high-resolution industrial cameras to obtain images of specimens to be inspected, and uses digital image processing inspection algorithms to complete the inspection of defects. The visual inspection is a non-contact inspection, and it will not cause any damage to the workpiece to be inspected during inspection. At the same time, the visual inspection has a high degree of automation, which can be realized for a long time and work continuously and smoothly. In recent years, research on artificial intelligence technology has continued to deepen, and machine learning and deep learning methods have been rapidly developed, and have been gradually applied in various fields, providing a new method for solving the detection problem of metal work piece surface defects. Deep learning can directly learn two-dimensional images, reducing image pre-processing, without manually extracting features, and can automatically learn more appropriate features layer by layer, greatly reducing the impact of human factors.  Quality management has become a central concern for manufacturing organizations, as achieving good quality is necessary to remain competitive in the market (Harik & Wuest, 2020). In addition to the establishment of international standards (European Committee for Standardization, 2015a, 2015b), different frameworks have been developed to aid organizations in establishing well-functioning quality management systems, such as Total Quality Management and Six Sigma (Oakland, 2014). A prominent element within these larger quality management frameworks is quality control, which is implemented for the expressed purpose of ensuring manufactured products comply with quality requirements. It is also accepted that visual classification tasks with higher complexity require more data, larger networks and more resource-intensive training of the neural network (Ameer& Maul, 2019). For quality inspection applications, this complexity may increase when more product variants are introduced to the network, resulting in a wider range of, for example, colours, materials, defect types and geometry.

A. Purpose

Defect detection during manufacturing processes is a vital step to ensure product quality. The timely detection of faults or defects and taking appropriate actions are essential to reduce operational and quality-related costs. It is also efficient at inspecting large production lines and spotting faults even on the smallest parts of a final product. 

B. Problem Statement

We'll look into whether deep learning models are appropriate for identifying steel product flaws. The key difficulty in this task is developing a reliable system from a small number of samples. In order to categorise and compare the performance of various neural network architectures and data augmentation strategies in addressing the key issues previously exposed, we will analyse the performance of these different neural network architectures in this project.

C. Objectives

The aim of this work is to develop image processing algorithms for product identification, defect detection and grading. For the purpose, it is also proposed to develop a specially designed product image acquisition setup. This aim is proposed to be achieved in the study by the following objectives:

1. Design and develop a robust machine learning algorithm for product classification.
2. Design and develop a lab prototype system to acquire images of surface defects over the entire area of the product.
3. Develop texture feature descriptors to discriminate defective and non-defective regions of the product.
4. Evaluate and validate the performance of the feature extraction algorithm and classifier results for unseen test data sets.
5. Develop grading rules based on computation of effective cutting value of the product for objective product grading.

D. The Merits Of The Project

1. It improves the Manufacturing process and optimizes costs.
2. It minimizes the manufacturer's costs by identifying detects earlier and faster.
3. It also reduces scrap and improves employee productivity.

E. Scope

The project has a focus on improving the performance and accuracy of surface defect detection by applying the model of CNN. In other words, the data will principally be assessed via the model of CNN and in contrast to the other two semantic segmentation models called ResUnet and Deeplab v3 plus. Finally, according to the analysis, an improvement will be made to CNN model. The dataset contains only steel surfaces from Kaggle competition as mentioned before.

II. METHODOLOGY

A. Block Diagram Of Product Defect Detection System

The manufacturing industries have been searching and developing new solutions to increase the product quality and to decrease the time taken and costs of production. The defect in the products can be detected using pre- processing defect. This defect detection in industrial applications produces high detection accuracy than the traditional methods acquired by the manufacturing industries for examining defects. The defect is detected through image processing.

B. Arduino Uno Board

Arduino UNO is based on an ATmega328P microcontroller. It is easy to use compared to other boards, such as the Arduino Mega board, etc. The board consists of digital and analog Input/Output pins (I/O), shields, and other circuits. The Arduino UNO includes 6 analog pin inputs, 14 digital pins, a USB connector, a power jack, and an ICSP (In-Circuit Serial Programming) header. It is programmed based on IDE, which stands for Integrated Development Environment. It can run on both online and offline platforms.

C. H-Bridge

An H bridge is an electronic circuit that switches the polarity of a voltage applied to a load. These circuits are often used in robotics and other applications to allow DC motors to run forwards or backwards. The H-bridge arrangement is generally used to reverse the polarity/direction of the motor, but can also be used to 'brake' the motor, where the motor comes to a sudden stop, as the motor's terminals are shorted, or to let the motor 'free run' to a stop, as the motor is effectively disconnected from the circuit. An H Bridge is a set of four switches that are assembled in such a way that an arbitrary load impedance is decoupled from a direct current (DC) power rail and ground.

D. Motor

A DC motor is any of a class of rotary electrical machines that converts direct current electrical energy into mechanical energy. The most common types rely on the forces produced by magnetic fields. Nearly all types of DC motors have some internal mechanism, either electromechanical or electronic, to periodically change the direction of current flow in part of the motor.DC motors were the first form of motor widely used, as they could be powered from existing direct- current lighting power distribution systems. A DC motor's speed can be controlled over a wide range, using either a variable supply voltage or by changing the strength of current in its field windings. Small DC motors are used in tools, toys, and appliances

E. Power Supply

A power supply is an electrical device that supplies electric power to an electrical load. The main purpose of a power supply is to convert electric current from a source to the correct voltage, current, and frequency to power the load. As a result, power supplies are sometimes referred to as electric power converters. Power is the backbone of any electronic system and the power supply is what feeds the system. Power supplies are used in most electric equipment. Their applications cut across a wide spectrum of product types, ranging from consumer appliances to industrial utilities, from milliwatts to megawatts, and from handheld tools to satellite communications.

F. Buzzer

An audio signaling device like a beeper or buzzer may be electro mechanical or piezo electric or mechanical type. The main function of this is to convert the signal from audio to sound. Generally, it is powered through DC voltage and used in timers, alarm devices, printers, alarms, computers, etc. Based on the various designs, it can generate different sounds like alarm, music, bell & siren.It includes two pins namely positive and negative. The positive terminal of this is represented with the ‘+’ symbol or a longer terminal.

G. LCD Display

A liquid-crystal display (LCD) is a flat-panel display or other electronically modulated optical device that uses the light-modulating properties of liquid crystals. Liquid crystals do not emit light directly,  instead  using  a backlight or reflector to  produce  images  in  color or monochrome. LCDs are available to display arbitrary images (as in a general-purpose computer display) or fixed images with low information content, which can be displayed or hidden, such as preset words, digits, and 7-segment displays, as in a digital clock. They use the same basic technology, except that arbitrary images are made up of a large number of small pixels, while other displays have larger elements

III. IMPLEMENTTION

A. Process Of Defect Detection System

Detection of a defect by image processing broadly follows some of the basic steps which include feature extraction, edge detection, morphological operators, and training of data..A real-time defect detection system is presented to help classify product quality automatically based on the YOLO (You only look once) algorithm. The system can be integrated into factories and production lines, helping to optimize efficiency and save operating costs.

Based on the YOLO algorithm , we trained a model to predict good and defected products during product manufacturing process in the factory. Later on, with the trained model, we built a system to detect defective products in real-time. The system can be readily installed and deployed at factories with existing infrastructure (CCTV cameras and connected computers).

B. The Proposed System For Detecting Defects In Real Time

1. YOLO Network: First of all, the outside images of the packaging boxes collected from the cameras will be labelled and classified as defective/non-defective or damaged/intact through external appearance assessment. Then this module builds the YOLO network as described and trains the network with annotated images to produce the trained model. After that, the model will be put into the packaging defect detection system for detecting defects of boxes from the conveyor belts.
2. Real-time image receiving: This module collects data from the cameras mounted on the conveyor belts and transmits frames of packages in real-time to the server. We utilize FFmpeg, a popular open-source, cross platform solution used in image processing and streaming to support real-time frame processing.
3. Packaging defect detection: This module manages frames sent from the Real-time video receiving module in a queue. Each frame in this queue will be resized and then put into the trained model to predict the probability values of which class the packaging boxes belong to. Finally, the prediction results of this frame are transmitted to the Automatic classifier module.
4. Automatic classifier: Based on the probability values received from the Package defect detection module with a given threshold, this module will automatically process the classification of packages, giving the decision whether a package is defective or not. After that, information of defect package is sent to the robot arm to move the damaged package out of the conveyor belt.

C. System Architecture

A pre-trained model is a saved network that was previously trained on a large dataset, typically on a large-scale image-classification task. Some highly optimized and extraordinarily efficient pre-trained models are available on the internet. Different models are used to perform different tasks. Some of the pre-trained models are VGG-16, VGG-19, YOLOv5, YOLOv3, and ResNet 50.

The system consists of 3 main stages:

1. Stage1

Acquiring the image of the product

It involves the capturing of the images of the product using camera. In this system we collected the number of database of product images that is good and bad quality images. These product image databases are helpful for more accurate result. So in this system we collected the camera images as database 225 database and these images used as input images in this system.

2. Stage 2:

Detection process:

Choose an input image from collected database images. Product is detected by feature extraction process. The proposed methodology in this paper, to perform the  analysis for image features extracts using following steps

a. Capture input images using camera and collect number of images as a database images. It includes good as well as bad quality images.

b. RGB image is converted to HSV color space. Then lower  and upper ranges are defined. Then ranges of binary image are defined. Then convert single channel mask back into 3 channels.

c. For extracts a colored object to detect the color, here we use HSV colorthresholder script to determine the lower/upper thresholds. HSV color space is also give the information about the image that is, it either present or not in this system.

d. Using by this input image we obtain the mask images. In mask image we get black and white colored image.

3. Stage 3:

Detection of defective product:

Find out defective product is one of the most important preprocessing steps. The defective skin is calculated. A color image of the product was used for the analysis. If the pixel value is less than the selected threshold value then it is considered as a part of defective i.e. bad quality product..Any pixel value greater than the selected threshold value is a part of pure skin i.e. good quality product.

The image is mask then pure part of the image indicated by black while the damaged ones white. Then the total number of white pixels are calculated which will be equal to the total number of pixels corresponding to damaged part.

A pre-processing procedure is performed to enable more accurate classification as well as ensuring the image resolution corresponds to the expected size of input images in the network. Image Processing for a more detailed description of the implemented image processing algorithm. Once processed, the image is transferred to a classification network which attempts to determine if the product is defective or not.

The framework allows for easy exchange of the classification network to test how different approaches perform in the same conditions. Monolithic approaches, of which there are many architectures available for use, do not need any further design to function and can be implemented as soon as they have been trained. However, the implementation of a modular network requires additional steps, especially given how the very structure of modular networks can be tailored to specific use cases and specific sets of data.

V. FUTURE SCOPE

We hope that in the future, this model will be implemented with a much more efficient dataset for a specific piece of product or object that contains various information, and so on. So that no product or object is wasted by quality inspection and this system is more productive. We want this model to be used all over the world to help the growth of the industrial sector. Both researchers and entrepreneurs may be interested in this field. In the future, we hope to create a cloud platform for all of the industries that will be using this model to share information all over the industries.

Conclusion

In this modern world, almost every sector is being enlightened by different technological innovations and findings. India is also moving forward with these blessings although the most significant economic resource of our country. We believe this model can play a very essential part in today\'s world. Industrialization is a fundamental aspect of modern civilization. With increasing modernization and Industrialization, industrial growth and demand emerge as a massive factor in this. But this is lacking in using new technologies of machine learning. As a result, our industry should be familiar with all of the latest machine learning and other techniques. In this study, the image processing technique and application of image processing expertise for automatic inspection and defect detection is discussed. Although lot of research carried by different researcher doing research in images processing, there is scope to apply image processing techniques for quality control of industrial product. The image processing techniques are very powerful tool for automatic, fast and easier defect detection and quality control of various types of products. Algorithm is proposed for real time quality monitoring of manufactured product. This proposed system can replace manual inspection of industrial product. Result will indicate product is defective or non-defective. Using this automatic inspection system cost of inspection will be reduced also accuracy of inspection will increase.

References

[1] B.Z. Chen, Z.H. Fang, Y. Xia, etc. Automatic detection of pore defects in thick steel pipe welds based on X-ray images. Computer Applications, 2017, 37(03): 849-853. [2] X.L. Qian, H.Q. Zhang, Y.X. Chen, etc. Research status and prospect of surface defect detection for solar cells based on machine vision. Journal of Beijing University of Technology, 2017, 43(01): 76-85. [3] L.Z. Zhu, M.Y. Chen. A Research on Auto Glue Defect Detection Based on Machine Learning. Journal of Shenyang Ligong University, 2018, 37(04): 18-22. [4] L. Guo, L. Shu, G.C. Wu. Detection method of arc-extinguishing grid surface defects based on machine learning. Mechanical Engineering and Automation, 2019(01): 4-7. [5] C. Wang, Y.T. Liu, X.Y. Xu, et al. Research on magnetic tile defect detection based on convolutional neural network. Journal of Dalian Nationalities University, 2019, 21(03): 220-224 [6] Y.J. Li, H.N. Zhang, J.L. Huang. Part defect detection system based on computer vision. Electronic Technology and Software Engineering, 2019(15): 111-112. [7] R.M. Liu, J.W. Huang, Y. Liu, etc. Detection of small spot defects on the surface of mud-coated wooden lines based on correlation vector machine regression. Journal of Forestry Engineering, 2019, 4(05): 115-120. [8] Yibin Huang, Congying Qiu, Yue Guo, Xiaonan Wang, and Kui Yuan, “Surface Defect Saliency of Magnetic Tile”, 2018 IEEE 14th International Conference on Automation Science and Engineering (CASE), Munich, Germany, August 20-24, 2018 DOI: 10.1109/COASE.2018.8560423. [9] Rasheed, A.; Zafar, B.; Rasheed, A.; Ail, N.; Sajid, M.; Dar, S.H.; Habib, U.; Shehryar, T.; Mahmood, M.T. Fabric Defect Detection Using Computer Vision Techniques: A Comprehensive Review. Math. Probl. Eng. 2020, 2020, 8189403. [10] Song, W.; Chen, T.; Gu, Z.; Gai, W.; Huang, W.; Wang, B. Wood Materials Defects Detection Using Image Block Percentile Color Histogram and Eigenvector Texture Feature. In Proceedings of the First International Conference on Information Sciences, Machinery, Materials and Energy, Chongqing China, 11–13 April 2015. [11] Ma, N.; Gao, X.; Wang, C.; Zhang, Y.; You, D.; Zhang, N. Influence of Hysteresis Effect on Contrast of Welding Defects Profile in Magneto-Optical Image. IEEE Sens. J. 2020, 20 4.Prasitmeeboon, P.; Yau, H. Defect Detection of Particleboards by Visual Analysis and Machine Learning. In Proceedings of the 2019 5th International Conference on Engineering, Applied Sciences and Technology (ICEAST), Luang Prabang, Laos, 2–5 July 2019; [12] Li, J.H.; Quan, X.X.; Wang, Y.L. Research on Defect Detection Algorithm of Ceramic Tile Surface with Multi-feature Fusion. Comput. Eng. Appl. 2020, 56, 191–198. [13] Wang, F.l.; Zuo, B. Detection of surface cutting defect on magnet using Fourier image reconstruction. J. Cent. South Univ. 2016, 23, 1123–1131. [14] Wang, J.; Fu, P.; Gao, R.X. Machine vision intelligence for product defect inspection based on deep learning and Hough transform. J. Manuf. Syst. 2019, 51, 52–60.

Copyright

Copyright © 2023 Rutu ., Rakesh M R, Pooja L, Hema C. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.