Managed Validation

This tutorial will describe the steps to validate the performance of ModelPack Vision models as managed sessions in EdgeFirst Studio that have been trained through the end-to-end workflows or Training ModelPack. For a tutorial to validate Fusion models, see Validating Fusion Models. A managed validation session is self hosted in an EC2 instance which is suited for users that do not have an embedded platform available to host the validation process. A user-managed validation session deploys the model in an embedded platform for validation.

Specify Project Experiments

From the projects page, choose the project that contains the training session with the models you want to validate. In this example, the project chosen is the "Object Detection" project that was created in the Getting Started. Next click the "Model Experiments" button as indicated in red.

Model Experiments

Create Validation Session

In the experiment card, click the "Validate Sessions" button as indicated in red below.

Validate Sessions

You will be greeted to the "Validate Sessions" page as shown below.

Validate Sessions Page

Start a validation session by clicking on the "New Session" button on the top right corner of the page.

New Session Button

You will be greeted with a validation session dialog. In this dialog, specify the name of the validation session, the model to validate, and the dataset to deploy. In this example, the TFLite model will be validated and the "Coffee Cup" dataset with the validation partition will be used. Next specify, the validation parameters on the right. Additional information on these parameters are provided by hovering over the info button .

Validation Session Fields

Once the settings have been specified, go ahead and click on the "Start Session" button on the bottom right of the dialog. This will start the validation session which will validate the model using the validation partition of the dataset.

Session Progress

Once the validation session has started, the progress with the stages will be shown on the left and additional information and status is shown on the right.

Validation Session

Completed Session

The completed session will look as follows with the status set to "Complete".

Completed Session

The attributes of the validation sessions in EdgeFirst Studio are labeled below.

Validation Session Attributes

Validation Metrics

Once the validation session completes, you can view the validation metrics by clicking the "View Validation Charts" button on the top of the session card.

Validation Charts

Info

See detection and segmentation metrics for further details.

You can go back to the validation session card by pressing the "Back" button as indicated in red below on the top left corner of the page.

Back to the Session Card

Comparing Metrics

It is also possible to compare validation metrics for multiple sessions. See Validation Sessions in the Model Experiments Dashboard.

Next Steps

Now that you have validated your Vision model, you can find examples for deploying your model in the PC, EVK, and Maivin Platform.