Across many industries visual inspection is key to ensure that products meet the quality standards. For a customer focused mentality there is nothing worse than shipping devices with defects to your client. Typically companies hire employees to carry out visual inspection as a measure of quality control. However, there are some disadvantages:
- Visual inspection is cost and time consuming (especially for large volume production)
- Variability (not every operator classifies the same pattern as defect)
With the recent progress in data science and image processing in particular, computers and designated visual inspection algorithms which are based on pattern recognition might become a powerful tool. Within this article I would like to give an overview about what is already possible in terms of pattern and defect recognition and how software can help you to classify visual fails correctly.
Taking images of the product
As a first step a procedure is required that takes pictures of the product. Details of this process highly depend on the actual product but a automated procedure is highly desirable: In semiconductor industries for instance often microscopes are required to take images due to the small product sizes. Often several images of one single product are required to cover all surfaces or to give the required resolution. Several companies offer out of the box solutions for this task. Critical is the following task, the feature extraction and the correct classification of failures.
Software for automated visual inspection
Unfortunately, there is no general approach to this problem. An algorithm that classifies well on a certain product does not necessarily work well on similar products. Therefore, an individual solution is required taking into account the properties of the specific product and typical failure mechanism.
As a first step, the image with width w and height h is converted into a matrix. Each pixel of the image can be represented by the rgb(a) color code which is nothing else than a set of numbers for each pixel. In many cases the image can be converted into grey scale where a single value is representing the brightness/darkness of the pixel in the image. Using this conversion the image can written as a simple w x h matrix. The correct detection and classification of defects is then a purely mathematical process. I would like to give some examples:
Defect detection using blob detection
Blob detection is a common method in computer vision and can give surprisingly good results on images that should not show any pattern. I don't want to dig into the mathematical details, I would just like to show some images to illustrate the potential of this method.
The image on the left hand side is the input image. It is just a surface showing one small defect, one larger defect which illustrates some contamination and a scratch. On the right hand side the same image is shown again, in addition blobs are highlighted with red circles. In this case all 3 defects were classified successfully.
The algorithm can be further improved so that overlapping defects are joined together. With this procedure it might become possible to differentiate scratches from other defects for instance as scratches typically show a large extension along one dimension whilst staying compact in the other one.
Advanced algorithms for pattern recognition
The blob detection mentioned above works pretty well when there is no product specific feature at all. However, in most cases this is not true. The blob detection alone can not successfully differentiate between wanted (product specific) and unwanted patterns such as defects.
In recent years machine learning and artificial intelligence have produced state of the art results in the area of image recognition and feature extraction. Especially deep learning algorithms often provide surprisingly good performance with very low error rates.
Please get in contact with us if you are interested in exploring the potential of automated defect detection in a production environment. Typically we would need a set of example images that we use to develop and train a suitable algorithm. In a next step we would use a different set of images to validate the algorithm. Depending on the result it might then be taken into production to simplify visual inspection processes.