Web Workflow
In this workflow, you will explore recording a video or capturing images using a mobile device and then upload the captured data into EdgeFirst Studio for annotation and then model training, validation, and deployment using the PC. This workflow requires the user to have signed up and logged in to EdgeFirst Studio and followed the initial steps described in the EdgeFirst Studio Quickstart.
Capture with a Phone
The examples below will show recording of a five second video and image captures of coffee cups using a phone for training a Vision model that detects coffee cups. However, you can choose any type of objects in your dataset.
Data Usage
It is recommended to use a phone connected to a Wifi network. A device connected to mobile data might be subject to intense usage when uploading files; video files or image files can be large in size. In the examples below, the video file used was ~15MB and the image files were ~2MB each.
Record Video
Using a smartphone, you can record a video with the camera application as shown below. Typically, the video recording can be started by pressing the red circular button. The video can be stopped by pressing the same button again.
Capture Images
Furthermore, you can also capture individual images as shown below. You can take image snapshots from the camera by pressing the white circular button.
Leveraging Videos
It is recommended to use videos rather than individual images. This is because the Automatic Ground Truth Generation (AGTG) feature leverages SAM-2 with tracking information which only needs a single annotation to annotate all frames. However, individual images requires more effort by annotating each image separately.
Limited Datasets
Throughout the demos, the dataset is kept small. However, training on limited datasets will result in poor model performances when the model is deployed under conditions that differs from the dataset samples. It is suggested to increase the amount of training data under various conditions and backgrounds to train a more robust model.
Create Dataset
If you have a video recording or sample images for your dataset, you can create a dataset container in EdgeFirst Studio to contain your video frames or images and annotations.
Navigate to a web browser and login to EdgeFirst Studio. Once logged in to EdgeFirst Studio, navigate to your project. In this case the project name is "Object Detection". Click on the "Datasets" button that is indicated in red below.
This will bring you to the "Datasets" page of the selected project. Create a new dataset container by clicking the "New Dataset" button that is indicated in red.
Add the dataset and annotation container name, labels, and dataset description as indicated by the fields below. It is up to you to specify the information in the fields and you do not have to strictly follow the example shown below. Click the "Create" button once the fields have been filled.
Your created dataset will look as follows.
Upload Video
Video files can be uploaded into any dataset container in EdgeFirst Studio. Choose the dataset container to upload the video file. In this case, the dataset is called "Coffee Cup". Click on the dataset context menu (three dots) and select import.
This will bring you to the "Import Dataset" page.
Click on the drop-down that says "Import Type" and then specify "Videos" and then click "Done" as shown below.
Now that the import type is specified to a "Videos", click on "select files" as indicated.
On an android device, this will bring up the option to specify the location of the files.
In my current setup, I have selected "My Files" from the options above and then "Videos" which will allow me to pinpoint the location of the video I have recorded.
Once the video file has been selected, set the desired FPS (frames per second), and then go ahead and click the "Start Import" button to start importing the video file.
This will start the import process and once it is completed, you should see the number of images in the dataset increased. If you do not see any changes, refresh the browser.
Upload Images
Image files can be uploaded into any dataset container in EdgeFirst Studio. Choose the dataset container to upload image files. In this case, the dataset is called "Coffee Cup". Click on the dataset context menu (three dots) and select import.
This will bring you to the "Import Dataset" page.
Click on "select files". This will bring up the option to specify the location of the files.
In my current setup, I have selected "Photos & Videos" from the options above and then I have multi-selected the images I want to import by press and hold on a single image to enable multi-select. To import, I pressed "Select".
Once the image files have been selected, the progress for the image import will be shown.
Once it completes, you should see the number of images in the dataset increase by the amount of selected images. If you do not see any changes, refresh your browser.
Next view the gallery of the dataset to confirm all the captured data has been uploaded. You should see the imported video file and images in the gallery. Note that videos appear as sequences with a play button overlay on the preview thumbnail.
Once all the captured data has been uploaded to the dataset container, you will now assign groups to the data to split the data into training and validation sets. Follow the tutorial for creating groups with an 80% partition to training and 20% partition to validation. The final outcome for the groups should look as follows.
Now that you have imported captured images or videos into EdgeFirst Studio and have split the captured data into training and validation partitions, you can now start annotating your data as shown in the next section below.
Annotate Dataset
Now that you have a dataset in your project, you can start annotating the dataset. This will briefly show the steps for annotating the dataset, but for an in depth tutorial on the annotation process, please see Dataset Annotations.
To annotate a dataset, first create an annotation set on the dataset card.
A new annotation set was created called "new-annotations".
Next, open the dataset gallery, by clicking on the gallery button 
on the top left of the dataset card. The dataset will contain sequences (video) 
and images. Click on any sequence card to start annotating sequences.
On the top navbar, switch to the right annotation set.
Start the AGTG server by clicking on the "AI Segment Tool" and follow the prompts as indicated.
Once the AGTG server has started, go ahead and annotate the starting frame.
Once the starting frame has been annotate, go ahead and propagate the annotations throughout the rest of the frames.
Repeat the steps for all the sequences in the dataset. For the case of individual images, the same steps apply except there is no propagation step. More details are provided in the manual annotations.
Train a Vision Model
Now that you have a fully annotated dataset that is split into training and validation samples, you can start training a Vision model. This will briefly show the steps for training a model, but for an in depth tutorial, please see Training ModelPack.
From the "Projects" page, click on "Model Experiments" of your project.
Create a new experiment by clicking "New Experiment" on the top right corner. Enter the name the description of this experiment. Click "Create New Experiment".
Navigate to the "Training Sessions".
Create a new training session by clicking on the "New Session" button on the top right corner.
Follow the settings indicated in red and keep the rest of the settings by their default. Click "Start Session" to start the training session.
The session progress will be shown like the following below.
Once completed the session card will appear like the following below.
On the train session card, expand the session details.
The trained models will be listed under "Artifacts".
| Session Details | Artifacts |
|---|---|
 |
 |
Validate Vision Model
Now that you have trained a Vision model, you can now start validating your Vision model. This will briefly show the steps for validating a model, but for an in depth tutorial, please see Validating ModelPack.
On the train session card, expand the session details.
Click the "Validate" button.
Specify the name of the validation session and the model and the dataset for validation. The rest of the settings were kept as defaults. Click "Start Session" at the bottom to start the validation session.
The validation session progress will appear in the "Validation" page as shown below.
Once completed the session card will appear like the following below.
The validation metrics are displayed as charts which can be found by clicking the validation charts.
Deploy the Model
Once you have validated your trained model, let's take a look at an example of how this model can be deployed in your PC by following the tutorial Deploying to the PC.
If you have an NXP i.MX 8M Plus EVK you can also run your model directly on the device using the EdgeFirst Middleware by following the tutorial Deploying to Embedded Targets.
Additional Platforms
Support for additional platforms beyond the NXP i.MX 8M Plus will be available soon. Let us know which platform you'd like to see supported next!
If you have an EdgeFirst Platform such as the Maivin or Raivin then you can deploy and run the model using the bundled EdgeFirst Middleware by following the tutorial Deploying to EdgeFirst Platforms.
No Studio Costs
Deployment of Vision models will not cost any credits from Studio.
Nest Steps
Explore more features by following the Maivin Workflow.