Training Overview

Training Overview - Supervised & Unsupervised (Deep-Learning)

To perform deep learning supervised training, firstly, end-users need to perform annotations on a set of images that is called Training Images, and the remaining will be called Test Images.

The algorithm (machine learning) learns from the annotated data from the Training set of images, producing a Prediction Model.

The set of Testing Images is then evaluated using the Prediction Model, which can be used to provide a set of statistics. The statistics can help us verify if the Bad Parts from Test Images are indeed identified by the Prediction Model.

On the other hand, for the unsupervised model, unannotated (unlabeled) data is provided and the algorithm tries to make sense of it by extracting features and patterns on its own.

PEKAT VISION software offers training of models using both Supervised and Unsupervised methods. Detailed modules information can be found within the article Modules.

The Unsupervised module available is Anomaly Detector, where the only required end-user input is to assign at least 30% of the images to the OK class.

The remaining modules are supervised, where the end-user is required to perform detailed annotations, however, the results are extremely accurate.

To provide a better understanding, find below a top-level schematic of how supervised training works in machine learning:

Open

The Training mode for PEKAT is available for the following AI Modules:

• Anomaly Detector

• Classifier

• Detector

• Surface Detection

• OCR

Below you can find a Training Overview Flowchart, which demonstrates the overall training steps. However, for detailed training information about a specific module, please access the desired module’s page.

Open

Training Graph - Loss Function Chart

The Training begins after clicking on the Start Training button, from that point a dialog window will display a training chart only for the following modules:

Surface Detection

Based on the chart displayed, its graph gives you an indication of the ideal moment to stop the training.

The graph should gradually decrease over the training duration because as the graph decreases, the model is improving.

Note that training can be finished at any time, however, the optimal moment is when the chart is not decreasing anymore (sideways graph).

As shown below, the graph decreased over time at first, and then near the end, it had not been decreasing for a while, therefore, the end-user should click on the Finish Training button to evaluate the results.

Please note the chart is not available for Anomaly Detector,Detector,OCR modules. The training requirements are determined before training, therefore loss function graph is not available.

These modules are trained either by pre-determined cycles or by user-specified iterations.

Avoiding Overfitting

Overfitting is a situation when the model becomes so specific to the training set that it starts to lose the ability to generalize and has bad results on different (testing) images. While performing training, the Loss Function graph is displayed, showing the error about the Training image set. In some cases, the Validation image set can be used to evaluate how well the model performs on unseen images already during training. Overfitting zones are likely to happen in machine learning when the loss function for the Validation set reaches its lowest peak as can be seen in this comparison between Training and Validation loss graphs.

Several methods help to prevent this situation from happening and we would like to highlight that our algorithm is designed to prevent overfitting.

If training is not interrupted when the graph is not decreasing anymore, the model will stop improving, meaning that you will not achieve better results. However, this will not cause weakness in the model, in other words, this will not increase the presence of false positives/negatives - it will just not provide better results. Therefore, if the graph is not decreasing anymore, it means the model stopped improving.

Extend Model

Extend model function is only available for the modules Classifier, Detector, OCR, Surface Detection.

This function allows you to perform further training on an already-trained model. The option to extend model is only available if you have at least one already trained model (otherwise the option is hidden).

Training data split

By adjusting the training data split you can choose how many of the images you annotated will be used for training and how many will be used for testin (the testing images will be used to calculate statistics that are displayed in the Confusion matrix).

The images are split between training and testing pseudo-randomly. A seed is used to initiate a random number generator that is used to split the images. This means that the same seed results in the same data split.

Augmentation

For a detailed explanation regarding augmentation, visit the article https://pekatvision.atlassian.net/wiki/spaces/KB32/pages/934185055.

Model Validation

After training, the new model is automatically selected and its preview is shown. A different model can be selected from the model list using the Change button and then clicking on the desired model to check the model’s results, as shown in the pictures below. When a model is selected, the statistics will then be based on the selected model.

Information on the training process and conditions used for a particular model are available by clicking on the info icon in the list of models, as shown in the picture below.

Model deletion

To delete an unwanted trained model click on the trash icon on the right.

To delete multiple models use CTRL for selecting the models and then click the DELETE SELECTED button.

Model importing

To import a model click on the IMPORT MODEL button and then find your model (.zip file) in the file system.