ModelPack Overview
ModelPack is an advanced computer vision solution developed by Au-Zone Technologies as part of their EdgeFirst.AI middleware. It provides both object detection and semantic segmentation capabilities, enabling high-performance, low-latency AI inference on embedded devices, particularly those with AI accelerators (NPUs) in the 0.5 TOPS and up range.
| Detection | Segmentation | Multitask |
|---|---|---|
![]() |
![]() |
![]() |
ModelPack is optimized for real-time vision applications such as industrial automation, robotics, and autonomous systems. It combines object detection — locating multiple objects within an image using bounding boxes — with instance segmentation, which outlines each object’s exact shape at the pixel level. This unified approach enables detailed scene understanding at the edge and can contribute in a late fusion with the radar model.
Getting Started
ModelPack can be trained in EdgeFirst Studio by following four simple steps:
- Set a name and description (optional) for the training session.

- Configure the launch form (input, model, training, augmentation and export) and start the session.

The ModelPack launch form groups its fields as follows.
- Training Session
- Name: The name of the training session, also used to name the artifacts (e.g.
modelpack-coffecup-640x640-rgba-t-<session ID>.onnx) - Description: An optional description of the training session, commonly used to highlight some parameters
- Name: The name of the training session, also used to name the artifacts (e.g.
- Source Dataset: The dataset the session trains and validates on, with its annotation set and tag
- Input
- Input Resolution: The network input resolution (width x height), 480x270 by default. The list covers 1:1, 16:9 and 4:3 resolutions from 320x180 up to 3840x2160 (4K UHD) and 2592x1944. Pick the resolution closest to your deployment camera. Higher resolutions need more GPU memory, and the trainer reduces the batch size automatically when it does not fit. With Input Tiling (SAHI) the Input Resolution is the tile size. In case you need a different resolution to be supported, please reach out and email our support team
- Camera Adaptor: The target camera format of the model input, one of RGB (default), BGR, RGBA, BGRA, Greyscale or YUYV. See Camera Adaptor and Letterbox
- Input Tiling (SAHI): Off by default. Trains on native-resolution tiles of each image and validates on full images. The tiling settings and Tiled Validation that follow it configure the tiles. See Input Tiling (SAHI)
- Letterbox Resize: On by default. Resizes each image into the model input while preserving its aspect ratio. See Camera Adaptor and Letterbox
- Model Parameters: Configures the model architecture and the tasks to train
- Model Backbone: CSPDarkNet19 (default), optimized for inference time, or CSPDarkNet53, optimized for accuracy
- Model Size: Nano (default), Small, Medium or Large. Similar to modern architectures, ModelPack scales the channel width and the block depth with the size (width
0.25,0.375,0.5,0.75and depth0.33,0.33,0.33,0.67from Nano to Large) - Activation Function: The main activation used in the model, one of ReLU, ReLU6 (default) and SiLU. ReLU6 produces the best tradeoff between speed and accuracy in most cases
- Object Detection: Trains an object detection model (enabled by default)
- Segmentation: Trains a semantic segmentation model. Selecting both Object Detection and Segmentation produces a multitask model with one output per task
- Focus Input: Off by default. Reshapes the input as a 2x2 patch grid before the first convolution, trading per-pixel processing for a higher channel count at half resolution. This is often a small inference speedup on NPUs, with a possible accuracy cost on very small objects
- Fine Detection Head: Off by default. Adds a stride-8 (P3) detection head so small objects, below 32 pixels, have enough feature cells to localize. Recommended together with Input Tiling. It applies to CSPDarkNet19; CSPDarkNet53 always has the stride-8 head
-
Training Parameters:
- Epochs: The number of epochs to train the model, 50 by default (1 to 500)
- Batch Size: The number of samples processed in a single training step, 16 by default (1 to 64). Remember the larger the input resolution the smaller the batch size
ModelPack always trains
Every ModelPack session trains on the selected dataset. The backbone starts from ImageNet-pretrained weights and the detection and segmentation heads start fresh.
-
Data Augmentation: The probability of each augmentation technique, in percent. Augmentation is crucial for training models and reducing overfitting, especially on small datasets. See Vision Augmentations
- Random Brightness and Contrast: 50 by default
- Random HSV: Random hue, saturation and value, 30 by default
- Random Grayscale: 10 by default
- Random Horizontal Flip: 50 by default
- Random Vertical Flip: 0 by default. A vertical mirror is not label-preserving for ground-referenced scenes; enable it for overhead or aerial imagery
- Random Shift and Scale: 25 by default. Not used with Input Tiling
- Random Mosaic: 25 by default. With Input Tiling, four tiles are combined at native scale
- Mosaic Center Jitter: 0 by default (0 to 0.45). The half-range for randomizing the center point of a mosaic, as a fraction of the tile size. 0 keeps the fixed-center 2x2 mosaic layout; higher values vary the size of each quadrant. It has an effect only when Random Mosaic is above 0
- Export Parameters: The session publishes FP32 and FP16 ONNX models, a TensorFlow SavedModel and an INT8 calibration snapshot. The converter apps use them to produce INT8 quantized models for each target NPU
- Calibration Samples: The number of images in the INT8 calibration snapshot, 500 by default (100 to 2000). Very low counts can reduce quantized accuracy. Snapshots are reused across sessions with matching settings
- ONNX Opset Version: The ONNX opset version of the exported model, 11, 12 or 13 (default). Keep the default unless a downstream tool requires an older version
- Start App: This button starts the training session
Camera Adaptor and Letterbox
The Camera Adaptor dropdown in the Input group selects the target camera format of the model input. Select the format that matches your deployment camera or ISP output to avoid runtime color conversions. See Camera Adaptor for the supported formats and platform guidance.
The photometric augmentations (brightness and contrast, HSV and grayscale) apply to the RGB, BGR and Greyscale formats. The RGBA, BGRA and YUYV formats receive the geometric augmentations only.
Letterbox Resize fits each image into the model input with an aspect-preserving letterbox: the image is scaled to fit, centred, and padded with gray (114). Boxes are remapped into the padded image, and segmentation masks are padded with the background class. Leave it on for general datasets. For fixed-camera datasets whose frames already match the input aspect ratio it has no effect and can be turned off, which resizes the image directly to the model input and skips the padding.
The resize mode is recorded in the model metadata so deployment runtimes preprocess identically, and the calibration snapshot uses the same resize mode, so INT8 calibration sees the deployment input.
Input Tiling (SAHI)
Small objects in high-resolution images lose most of their pixels when the image is downscaled to the model's input size. Input Tiling (SAHI) trains on native-resolution tiles of each image instead of downscaling the whole image, then validates on full images, so small objects keep their native pixel scale.
With Input Tiling enabled, the Input Resolution is the tile size; 640x640 is the preferred choice. Input Tiling supports object detection with the RGB or BGR camera adaptor.
The following settings, in the Input group, control how tiles are sampled. Their defaults are the validated recipe.
| Setting | Default | Description |
|---|---|---|
| Tiles per Image | 4 (1 to 8) | Tiles cut from each decoded image per epoch. |
| Background Tiles (%) | 15 (0 to 50) | Percentage of tiles that contain no annotated object. Lower it for densely annotated datasets. |
| Full-Image Tiles (%) | 10 (0 to 50) | Percentage of tiles that show the whole image fitted into the tile, keeping large objects and scene context. The remaining tiles are placed around objects. |
| Grid-Aligned Tiles (%) | 25 (0 to 100) | Percentage of object and background tiles taken from the deployment tile grid, so the model sees the tile edges it meets at inference. |
| Native-Scale Tiles (%) | 50 (0 to 100) | Percentage of tiles cropped at exactly the deployment scale. The rest are zoomed within the Tile Zoom Range. |
| Tile Zoom Range | 1.33 (1.0 to 2.0) | Largest zoom applied to non-native tiles, in and out equally: 1.33 samples between 0.75x and 1.33x. 1.0 disables zoom. |
| Minimum Visible Fraction | 0.25 (0.05 to 0.9) | An object cut by a tile edge is kept as a target only when at least this fraction of its area is inside the tile. |
| Final Native Epochs | 10 (0 to 50) | Closing epochs trained on native-scale tiles only, with the tile mosaic off, to match deployment. Limited to half of the epochs. |
| Class-Balanced Tiles | Off | Places more tiles around objects of rare classes. |
| Tiled Validation | Whole Image | How validation scores the model on full images. See Tiled Validation. |
A tile smaller than the Input Resolution, such as a full-image tile or a tile cut from an image smaller than the tile, is letterboxed, centred, into it. With Input Tiling, Random Mosaic combines four tiles at native scale and Random Shift and Scale is not used, because the tile sampler applies its own zoom. The anchors are fit to the boxes as they appear in the tiles.
Tiled Validation
A tile-trained model is scored on full images, the deployment metric, rather than per tile. Tiled Validation selects how.
| Tiled Validation | Description |
|---|---|
| Whole Image (default) | Each full image runs through the model in one pass at native resolution. |
| Tile Grid | Each image is cut into the deployment tile grid, an evenly distributed grid with at least 10% overlap between tiles. Each tile is detected separately, and the detections are lifted to image coordinates and merged across tiles. |
| Both | Runs both and reports the tile grid metrics alongside the whole-image metrics. This is slower. |
Deploying Tile-Trained Models
A tile-trained model exports at the tile size. Its model metadata carries a tiling section describing the deployment tile grid and the merge of detections across tiles, so the runtime cuts each frame into tiles of the Input Resolution. The calibration snapshot holds deployment-grid tiles rather than whole images; see Tiled Models.
ModelPack Architecture
ModelPack is a modern object detector and it adopts similar scaling strategies seen in the YOLO family models. The model expands and contracts based on the width and height parameters. ModelPack shares two main backbones: a Darknet53 backbone similar to YOLOx which maximizes accuracy and a Darknet19 backbone for boosting inference time. Different than YOLOx, ModelPack is NOT anchor free, which makes the model more accurate and stable after quantization.
Figure reproduced from: Yang, L., Chen, G. & Ci, W. Multiclass objects detection
As mentioned above, ModelPack merges Semantic Segmentation and Object Detection on the same model and it is user responsibility depending on problem requirements. Semantic Segmentation only uses two scales (Scale 1 and Scale 2). On the other hand, Object Detection task uses the three scales.
While solving both tasks in the same inference cycle, the three scales are used.
ModelPack outputs are configured in the launch form of the training session. See Training Vision Models for the training workflow.




