Introduction

This article describes setting up the Maivin Detect application to run models and collect the annotations into an MCAP file for review or for further training.

Overview

The Maivin Detect application will recieve camera frames from the Maivin Camera application, load the camera frames into the model, run the inference, apply output decoding and NMS, then publish the inference results into Zenoh.

The Maivin Detect application allows the user to change the inference engine, score threshold, iou threshold, the maximum boxes before NMS, and the label offset. Additionally, the user can use 2 stage models, where the output of the first model will be sent to the second model, and the final output comes from the second model. The second model is always run on the CPU.

Recording Annotations

1. After deploying ModelPack to Maivin, the Maivin Detect application can be used to run the model. See Deploy ModelPack to Maivin for details on the deployment process and configuring Maivin Detect to use the new model.
2. Configure Maivin Detect for the appropriate score and IOU threshold for the model by edting the fields in /etc/default/detect. See the Configuration section of Maivin Dataset Recording and Object Tracking using the Maivin Detect Service for details about the configuration options available for the Maivin Detect service.
   • After editing /etc/default/detect, run the following commands to apply the changes

     sudo systemctl daemon-reload
     sudo systemctl restart detect

3. Record the annotations using the recorder. The below command is an example that records the listed topics for 10s.

   recorder -d 10 /imu /gps /camera/info /camera/h264 /detect/boxes2d /detect/visualization

   See the Recording section of the Maivin Dataset Recording article for more details on recording.

Visualizing the Results

1. Copy the recorded MCAP file to a computer that has Foxglove installed.
2. Open the MCAP file in Foxglove.
3. In Foxglove, view the appropriate Camera stream in an Image Panel. (/camera/h264 or /camera/jpeg)
4. Under panel settings, go to the Image Annotations section and click the eye next to the rt/detect/visualization topic.

For more details about the Foxglove visualization, please see the Playback MCAP with Foxglove Studio article.

Exporting the Results

To export the data in the MCAP to DarkNet format or to Deep View Enterprise, we can use the Maivin Publisher. This can be done on device or on a PC.

maivin-publisher -m DIR --out <PATH to where to store the darknet files> <Path to the .mcap file>

See the Maivin Dataset Publisher article for details on exporting process, as well as a download for a Linux AMD64 executable of the maivin-publisher