Evaluation

Owned by Vojtěch Kováčik
Last updated: Apr 30, 2024Version comment
3 min read

Automatically decides whether an image is ok or ng . This information is represented by a green or red stripe in the corner of each image in the image preview.

evaluation.png

Evaluation can be used in the following modules:

If one of the modules in flow uses evaluation, this evaluation will affect the next modules. In case the next module uses evaluation as well it will overwrite the results of the previous evaluation.

When you use Parallelism, a boolean operator AND is used among the branches. That means if one of the branches has an ng result, the overall result will be ng as well.

Anomaly Detector

The evaluation considers the number of anomalies and compares it with a threshold value. More information on calculating the threshold can be found Anomaly Detector. If the values are bigger than the threshold, the image is evaluated as ng, otherwise, it is considered ok.

Evaluation

To activate this function, you need to tick the Evaluation button above the generated sensitivity graph. When processing is active, it evaluates the image as valid (green stripe) or invalid (red stripe).

Enable eval.png

Threshold

The value determines the number of anomalies that are tolerated. Higher values filter out the defects that have lower confidence. Lower values keep most of the defect detections.

The threshold can be recalculated and tweaked to fit the needs of the project.

Evaluation conditions

Images are evaluated based on a set rules. If the image follows all of the rules it is considered ok , otherwise it is ng .

It is possible to set multiple groups of conditions. Each being evaluated with a logical AND or OR based on the current needs. You are able to select all detections as well as individual classes.

If there are multiple classes, the rules can be set for a specific class or one of Any/Every/Together.

  • Any - OK if the rule is true for at least one of the classes (e.g. rule “Any Count = 10” is true if Class1 count is 10 and Class2 count is 4).

  • Every - OK if the rule is true for each of the classes found in the image (e.g. rule “Every Count = 10” is true if Class1 count is 10 and Class2 count is 10).

  • Together - OK if the rule is true for all classes present in the image combined (e.g. rule “Together Count = 10” is true if Class1 count is 6 and Class2 count is 4).

Then it is possible to choose the type of evaluation based on the model: Count/Edge length.

  • Count - represents the number of detected rectangles.

  • Edge length - represents the length of the chosen rectangle/rectangles.

If you set the rules for a trained model and then train another one, the rules are automatically copied to the new model, so you don’t need to set them again if no changes in evaluation are needed.

OCR

The evaluation in the OCR module is based on the comparison of the results found in the image with the specified regex. It is possible to specify multiple rules and check if at least one is filled (use OR)