Surface Detection

Owned by Monika Rehakova (Unlicensed)
Last updated: Aug 29, 2023 by Kristyna Janku
4 min read

Surface Detection can be trained to detect a specific surface or known surface defects, it is suitable especially for detecting scratches, cracks, and defects on a highly variable background.

This module serves to detect a specific surface and it can also classify the types of surfaces, which are to be detected, into different classes like the Classifier.

Training

Annotations

Surfaces can be marked in two ways - by brush painting in the image or by making polygons, those ways can also be combined. Each surface class has one color assigned. For quick color switching, you can press a numeric key indicating the number of the class that is given in square brackets.

Brush painting

Basic painting into the image by a brush of selected size. To change the brush size or switch between paint- and erase mode, you can either use the sliders at the top, or you can use shortcuts, which can be found in the Help section under the ‘?’ symbol. Currently, you can press or hold the 'k' and 'l' keys to decrease resp. increase the brush size, and holding 'e' temporarily switches the paintbrush to the eraser.

 

Polygons

When you choose the polygon mode, you left-click into the image to define the vertices of the polygon, which are being connected with a line in the order you put them in. When you're satisfied with the shape, click on the initial point marked with a circle, which joins the last vertex with the first one and fills the polygon with the chosen color. Polygons work in the erase mode as well.

 

Include

It is used to include the selected image for training if it does not contain any defects. The picture does not fall into one of the classes which were created, its whole surface is marked as defect-free.

It can be used when a defect-free/OK image has some noise or it’s slightly scratched for instance, or it has some very variable background and we get false defects detected on it. It can help achieve higher accuracy.

Ignore

Surfaces marked with ‘Ignore’ class are excluded from the training. Whereas, parts of the image which are left unannotated, are part of the training, but are considered an ‘OK’ class, which is not displayed when detected. Can be used e.g. in case we are unsure whether a specific part of an image is OK or NG or if there’s some distortion and we don’t want to put that part into training. The standard type of surface detector doesn't use Ignore class.

Settings

Types of Surface Detection

Two types of surface detectors are available, each based on a different kind of neural network.

 

Lightweight:

  • Detects a wide range of defects, however, can suffer from a higher amount of false positives (good parts evaluated as bad by the software), adjusting the Confidence threshold after training can help reduce this.

  • Faster inference and training.

  • Multiple options for view-finder size.

  • The model can optionally be optimized using Tensor RT for the specific GPU used when you check the TRT switch in the model list. The optimization improves inference times when processing images using this model.

The model is optimized for specific GPU, so if you transfer the project to a PC with a different type of GPU, the optimization needs to be done again for the new graphics card.

The TRT optimization is available only for models with one class at the moment.

  • This type does not require large datasets, however, performance on highly variable patterns may be more suitable for the Standard option.

 

Standard:

  • This type produces very precise heatmaps and usually doesn’t have too many false positives. It's ideal for searching for objects/defects with closed shapes. It also takes into account the surroundings, not just the surface itself.

  • Training time is usually longer.

  • View-finder size is not used in this type & ignore function is not included.

 

View-finder

The size of the view-finder determines how much the inspection will be focused on. The size is selected depending on how detailed the inspection model is desired. If you choose a size that is too small, you lose the knowledge of the surroundings and therefore can miss some defects; on the other hand, if the chosen size is too large, details can be overlooked.

Just as when a human eye focuses on detecting errors. Some errors are seen from a larger distance, and others can only be seen through a magnifying glass.

Along with the size, the recognition speed also varies. There is no general rule on how to set the right size, you need to try out a number of sizes and learn how to estimate the best size on the first try. The size of the defects you are searching for might be of help.

When using smaller or narrow images, be careful that the viewfinder size must be smaller than the smallest image dimension. When rotation is on, there is also additional padding, so even if the viewfinder is slightly smaller than the image, it may not fit - in that case, use smaller viewfinder, upscale image in Preprocess, or turn off the rotation.

The Standard type of Surface Detection doesn't use this parameter.

Detection

The detection result is a set of heatmaps (one for each class). When validating, heatmaps are plotted in the image for better illustration. Heatmaps for the whole image are added to 'heatmaps' in Context, and all the rectangles surrounding the searched-for areas are added to ‘detectedRectangles’. Each rectangle has a class assigned, depending on which heatmap it was found in.