Skip to content

EdgeFirst Messages

The EdgeFirst messages are the custom schemas of the EdgeFirst Perception Middleware, published in the EdgeFirst Schemas repository. The current wire format is EdgeFirst Schemas 4.0 which introduced the Tensor family of messages and replaced the DmaBuffer message on camera/dma with CameraFrame on camera/frame.

Box

CameraFrame

edgefirst_msgs.CameraFrame — a timestamped camera frame, carried as a tensor.

A header (stamp, frame_id), a seq, and an embedded :class:Tensor reached through :attr:tensor.

seq is more than drop detection: it is a uint64, so it forces the embedded tensor to an 8-aligned offset regardless of frame_id length. That is what makes the nested tensor byte-identical in every wrapper — and what lets :meth:Tensor.to_standalone_cdr re-head an embedded tensor without re-encoding it.

tensor property 

tensor: Tensor

The embedded tensor, sharing this message's buffer.

For a message decoded from bytes this shares the underlying object outright — no copy. For one just constructed in-process the metadata is duplicated; plane payloads travel behind handles either way, so nothing frame-sized is copied.

Detect

edgefirst_msgs.Detect — detection result with header + boxes.

Carries a sequence of :class:DetectBox results plus timing metadata.

Mask

edgefirst_msgs.Mask — segmentation mask (H × W × L bytes).

encoding is "" for raw uint8 or "zstd" for zstd-compressed payloads. mask exposes the bytes via :class:BorrowedBuf for zero-copy numpy access::

arr = np.frombuffer(mask.mask, dtype=np.uint8).reshape(L, H, W)

On abi3-py38 use mask.mask.view() instead of np.frombuffer.

length property 

length: int

Number of channels (depth dimension of the mask tensor).

mask property 

mask: BorrowedBuf

Zero-copy view of the mask bytes (H × W × L uint8).

Model

edgefirst_msgs.Model — full model inference result with boxes and segmentation masks.

ModelInfo

edgefirst_msgs.ModelInfo — model metadata (name, type, format, shape, labels).

RadarCube

edgefirst_msgs.RadarCube — radar tensor with typed metadata arrays and an int16 cube payload.

Bulk-array accessors return :class:BorrowedBuf; reinterpret with the documented dtype:

  • layout → bytes (1 byte per axis index)
  • shape → np.uint16 (number of bins per axis)
  • scales → np.float32 (real-world scale per axis)
  • cube → np.int16 (interleaved I/Q samples)

Example

::

cube = RadarCube(
    header=Header(stamp=Time(1, 0), frame_id="radar"),
    timestamp=1234567890123456,
    layout=np.array([6, 1, 5, 2], dtype=np.uint8),
    shape=np.array([2, 128, 12, 128], dtype=np.uint16),
    scales=np.array([1.0, 0.117, 1.0, 0.156], dtype=np.float32),
    cube=np.zeros(2 * 128 * 12 * 128, dtype=np.int16),
    is_complex=True,
)

cube_view = np.frombuffer(cube.cube, dtype=np.int16)
shape    = np.frombuffer(cube.shape, dtype=np.uint16)
cube_arr = cube_view.reshape(*shape)

cube property 

cube: BorrowedBuf

Cube data — np.frombuffer(..., dtype=np.int16).

layout property 

layout: BorrowedBuf

Layout codes — uint8 sequence; one entry per axis identifying SEQUENCE / RANGE / RX_CHANNEL / DOPPLER (see radarpub docs).

scales property 

scales: BorrowedBuf

Per-axis scales — np.frombuffer(..., dtype=np.float32).

shape property 

shape: BorrowedBuf

Shape vector — np.frombuffer(..., dtype=np.uint16).

timestamp property 

timestamp: int

Sensor-supplied microsecond timestamp (radar ASIC clock).

RadarInfo

edgefirst_msgs.RadarInfo — radar configuration parameters.

Tensor

edgefirst_msgs.Tensor — the unstamped tensor payload.

Carries the element type, the addressing grid, optional quantization parameters, optional colorimetry, and one or more planes.

shape is the addressing grid, NOT the byte layout: an NV12 frame carries shape == [h, w] with a U8 dtype against an h*w*3/2 allocation. It is deliberately never validated against any buffer size. strides is in BYTES, and is either empty or exactly as long as shape.

quant_axis selects which shape the quantization parameters take, and the encoder enforces the match:

  • -2 unquantized — quant_scales must be empty
  • -1 per-tensor — exactly one scale
  • >= 0 per-axis — exactly shape[quant_axis] scales

quant_zero_points is either empty or the same length as quant_scales. Colorimetry may only be set when format is.

Note that fence_fd and quant_axis default to -1 and -2 respectively — the schema's "absent" values, not zero.

Example

::

t = Tensor(
    storage_kind=2,
    pid=os.getpid(),
    dtype=1,
    shape=[480, 640],
    strides=[640, 1],
    format="NV12",
    color_space="bt709",
    color_range="limited",
    planes=[
        TensorPlane(handle=fd, offset=0, stride=640,
                    size=640 * 480, used=640 * 480),
        TensorPlane(handle=fd, offset=640 * 480, stride=640,
                    size=640 * 480 // 2, used=640 * 480 // 2),
    ],
)

dtype property 

dtype: int

Element type (HAL dtype codes).

fence_fd property 

fence_fd: int

ACQUIRE fence fd; -1 when there is no fence.

pid property 

pid: int

Producer PID, for handle resolution; 0 when not applicable.

storage_kind property 

storage_kind: int

Storage class shared by every plane (HAL storage_kind codes).

plane_data 

plane_data(index: int) -> BorrowedBuf

Zero-copy view of one plane's inline bytes.

:attr:planes copies each plane's data; this does not::

arr = np.frombuffer(t.plane_data(0), dtype=np.uint8)

Returns an empty view for a plane whose bytes travel behind a handle. Raises :class:ValueError if index is out of range.

to_standalone_cdr 

to_standalone_cdr() -> bytes

Re-head this tensor as a standalone Tensor CDR message.

The republish path — forwarding a camera frame's tensor onto a tensor topic. Copies metadata only; plane payloads stay behind their handles. Because the layout is position-independent the result is byte-identical to encoding the same tensor standalone from scratch.

TensorPlane

edgefirst_msgs.TensorPlane — one plane of a :class:Tensor.

Two mutually exclusive transport modes:

  • handle >= 0 — the bytes live behind the platform handle and data is empty. This is the dma-buf / shared-memory path.
  • handle == -1 — the bytes are inline in data; is_inline is True, size == len(data), modifier == 0 and handle_bytes is empty.

A frame must not mix modes: all planes inline, or none. The tensor carries a single storage_kind, pid and fence_fd covering every plane, so a mixed set has no coherent meaning and is rejected.

Read back from a :class:Tensor this is a value copy, data included. For a large inline payload prefer :meth:Tensor.plane_data, which returns a zero-copy view.

modifier property 

modifier: int

Format modifier (tiling / compression); 0 for linear.

size property 

size: int

Allocated size of the plane, in bytes.

stride property 

stride: int

Row stride in BYTES.

used property 

used: int

Bytes actually populated; always <= size.

TensorStamped

edgefirst_msgs.TensorStamped — a timestamped tensor, for model input and output topics.

A header (stamp, frame_id), a seq, and an embedded :class:Tensor reached through :attr:tensor.

seq is more than drop detection: it is a uint64, so it forces the embedded tensor to an 8-aligned offset regardless of frame_id length. That is what makes the nested tensor byte-identical in every wrapper — and what lets :meth:Tensor.to_standalone_cdr re-head an embedded tensor without re-encoding it.

tensor property 

tensor: Tensor

The embedded tensor, sharing this message's buffer.

For a message decoded from bytes this shares the underlying object outright — no copy. For one just constructed in-process the metadata is duplicated; plane payloads travel behind handles either way, so nothing frame-sized is copied.

Track

edgefirst_msgs.Track — object-tracking record (no header).

id is empty when the object isn't being tracked. lifetime counts consecutive frames the track has been seen. created is the timestamp of the first sighting.