Deploying Vision Models
In this tutorial we are going to give you the tools needed to run Vision models on a PC for object detection or segmentation allowing you to build your own applications in just a few lines of code!
It is important to complete the tutorials on the EdgeFirst Studio Quickstart Guide first before moving forward with the tutorials in this notebook. The workflows presented in the Quickstart Guide ultimately stop at the the model deployment stages which will be the primary focus of this notebook.
Warning
The tutorials presented in this notebook requires a trained and validated Vision model.
Requirements
To use this notebook, let's first install the following dependencies.
%pip install edgefirst-client
%pip install tensorflow
%pip install pillow
%pip install numpy
%pip install matplotlib
Connect to EdgeFirst Client
Once all the dependencies have been installed, the client needs to be connected to EdgeFirst Studio to fetch the model artifacts to your PC. Run the following code block below to connect the EdgeFirst Client to the Studio. Modify the "username" to be your own. Whrn running the cell block below, you will be prompted to enter you password. Once entered, press the "Enter" key to accept. The code block will execute successfully if the credentials are correct.
from edgefirst_client import Client
from getpass import getpass
username = 'john-trial'
client = Client()
password = getpass()
client.login_sync(username, password)
Retrieve Training Session ID
Once you have connected to EdgeFirst Client, let's move forward by finding the training session ID that contains the trained model artifacts. Once we have located the model artifacts, we can fetch these files locally to run for inference in this demo.
There are two possible ways to get the artifacts from the training session:
- Manually download
modelpack.tfliteandlabels.txtfrom the EdgeFirst Studio training session. - Using
edgefirst-clientcommand line interface.
In this tutorial, we will explore option two which is to run the edgefirst-client command to fetch the model artifacts.
Verify that the connection is successful by listing the projects available to the user.
client.projects_sync() # This will list all the projects available to the user
By following the EdgeFirst Studio Quickstart, you should have created a project. In this example, the project that was created is called "Object Detection". Make a note of the your project ID. In this case, it is 365. Adjust the code block below to replace with your project ID.
# Retrieve the project ID where the dataset is stored (all experiments/training/validation sessions are stored in the same project)
project_id = 365
# List all the experiments/training/validation sessions available for the project
client.experiments_sync(project_id)
The command above will list all experiments in the project. Make a note of the experiment ID that contains the training and validation sessions you deployed. In this case, it is 496. Adjust the code block below to replace with your experiment ID.
# By using the experiment ID, the user can retrieve all the training sessions available for that experiment.
experiment_id = 496
trainers = client.trainer_sessions_sync(experiment_id)
for i, trainer in enumerate(trainers):
print(f"session {i+1}: ID [{trainer.id()}], Name: {trainer.name()}")
The command above will list all the training sessions in the experiment. Make a note of the training session ID that contains your model artifacts. In this case it is 859. Now that you have isolated the training session that contains your model artifacts, run the code block below to list the artifacts stored in the training session. Adjust the code block to replace with your training session ID.
session_id = 859
# The user can also list the artifacts stored in the training session by using the trainer session ID
client.artifacts_sync(session_id)
Download the Model Artifacts
Now that we have located the training session ID of our artifacts, we can move forward with downloading the model artifacts locally. Execute the code block below to download the artifacts modelpack.tflite (model file) and labels.txt (unique labels). Ensure these files exist. Otherwise, update the code block to match your model files.
# Download the artifacts
client.download_artifact_sync(session_id, 'modelpack.tflite', filename='modelpack.tflite')
client.download_artifact_sync(session_id, 'labels.txt', filename='labels.txt')
Let's take a look at the labels. As you can see the only class in the dataset is "Coffee Cup"
!cat ./labels.txt
Object Detection
In this demo, we will show running the Vision model fetched above for object detection. This model is a quantized TFLite model suited for inference in edge devices. This model will generate bounding boxes around the detected objects in the image.
To run inference on the model we need to have an input image. We can capture an image with a mobile device. A sample image is shown below.
Import Dependencies
For this demo, let's start by importing the dependencies.
import numpy as np
import matplotlib.pyplot as plt
from PIL import Image
import tensorflow as tf
Load the Model
Next let's load TFLite model for inference.
# Loading the model
interpreter = tf.lite.Interpreter(model_path='modelpack.tflite')
interpreter.allocate_tensors()
# Get input and output tensors.
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
H, W = interpreter.get_input_details()[0]['shape'][1:3]
output_details = interpreter.get_output_details()
# Bounding boxes (xmin, ymin, xmax, ymax). Shape must be (1, NB, 1, 4)
box_details = output_details[0]
score_details = output_details[1] # Scores. Shape must be (1, NB, 1)
print("Input shape: ", input_details[0]['shape'])
print("Boxes Output shape: ", output_details[0]['shape'])
print("Scores Output shape: ", output_details[1]['shape'])
Run Model Inference
Now that the model has been loaded, we can pass the input image to the model for inference. Run the next code blocks below. Change the path to the image file as needed. First we are preprocessing the input image to match the input requirements of the model such as the image resolution and data type.
# Loading the image
image_path = "../../assets/dev/sample-coffee-cup.jpg"
image = Image.open(image_path)
image = image.resize((W, H))
image = np.array(image).astype(np.uint8) # Model is quantized with uint8 input
plt.imshow(image)
# Preparing input tensor
input_tensor = np.expand_dims(image, axis=0)
# Run inference
interpreter.set_tensor(input_details[0]['index'], input_tensor)
interpreter.invoke()
Next we will post process the model outputs such as dequantizing and passing the output to the Tensorflow NMS for filtered boxes.
# Retrieve the output tensors
boxes = interpreter.get_tensor(output_details[0]['index'])
scores = interpreter.get_tensor(output_details[1]['index'])
# Dequantize the boxes and scores
if score_details["dtype"] == np.int8:
scale, zero_point = score_details["quantization"]
scores = (scores.astype(np.float32) - zero_point) * scale
if box_details["dtype"] == np.int8:
scale, zero_point = box_details["quantization"]
boxes = (boxes.astype(np.float32) - zero_point) * scale
The following code block will run NMS on the model outputs based on the parameters specified for IoU and score thresholds and the maximum boxes to output from NMS. This parameters can be specified by the user for tweaking the model outputs.
max_boxes = 50
iou_threshold = 0.45
score_threshold = 0.50
# PostProcessing
# We are using the NMS to remove the boxes that have a low score and are too close to each other
nmsed_boxes, nmsed_scores, nmsed_classes, valid_boxes = \
tf.image.combined_non_max_suppression(
boxes,
scores,
max_boxes,
max_boxes,
iou_threshold=iou_threshold,
score_threshold=score_threshold
)
# Get valid boxes
valid_boxes = valid_boxes.numpy()[0]
nmsed_boxes = nmsed_boxes.numpy()[0]
nmsed_classes = nmsed_classes.numpy()[0]
nmsed_scores = nmsed_scores.numpy()[0]
boxes = nmsed_boxes[:valid_boxes]
scores = nmsed_scores[:valid_boxes]
classes = nmsed_classes[:valid_boxes]
Next we will load the labels.txt files to convert model output indices into meaningful names. Each bounding box gets a label, the next steps will show visualization of the postprocessed model outputs with meaningful labels.
# Loading the labels to assign to bounding boxes
with open('labels.txt', 'r') as f:
labels = f.readlines()
labels = [label.strip() for label in labels]
Now we will visualize the model outputs using matplotlib.
import matplotlib.pyplot as plt
# Draw the bounding boxes
for box, score, cls in zip(boxes, scores, classes):
xmin, ymin, xmax, ymax = box
xmin = int(xmin * W)
ymin = int(ymin * H)
xmax = int(xmax * W)
ymax = int(ymax * H)
print(f"Label: {labels[int(cls)]}, Box: [{xmin}, {ymin}, {xmax}, {ymax}], Score: {score:.2f}")
rect = plt.Rectangle((xmin, ymin), xmax - xmin, ymax - ymin, fill=False, edgecolor='green', linewidth=2)
plt.gca().add_patch(rect)
plt.text(xmin - 10, ymin - 10,
f'{labels[int(cls)]}: {score:.2f}', color='green')
plt.imshow(image)
plt.show()
Segmentation
(Coming Soon)