Introduction
This article explains how to playback MCAP files captured using the Maivin Dataset Recorder on a PC using Foxglove Studio. If you prefer to directly export the MCAP to a Darknet format or into Deep View Enterprise please refer to the Maivin Dataset Publisher article instead.
Foxglove Studio is an open-source application from the ROS (Robot Operating System) ecosystem. It supports playback for "rosbag" recordings, including those using MCAP. Maivin Dataset Recorder saves into an MCAP file which is compatible with ROS MCAP files and can be loaded into Foxglove Studio.
Requirements
-
Install Foxglove Studio on PC
- Windows, Linux, and Mac are supported.
- Record an MCAP on Maivin using the Maivin Dataset Recorder
- Copy the MCAP recording from Maivin to PC
Getting Started
We've provided a Foxglove Studio Layout for Maivin 2 which can be downloaded at the bottom of this article. Once downloaded it can be imported into Foxglove Studio from the Layout menu, as shown in the following screenshot.
Layouts in Foxglove Studio are customizable, the provided Maivin 2 layout is as follows.
- The top panel provides the H.264 camera stream along with bounding box overlays, if generated by a detection model on the /detect/boxes2d topic.
- In this example you can see the white box around the subject's face, we're running the facedetection model available on Maivin under /usr/share/detect/facedetect.rtm
- The bottom right panel shows a map along with the GPS coordinates of the camera for the current frame.
- You can zoom in to the blue target. If the camera is moved during recording you will see the position change as well.
- The bottom left panel shows the IMU readings of the camera for the current frame.
- This panel could be updated to plot the IMU readings, see next section for details.
- At the very bottom is the timeline and play button.
- The timeline can be used to move forward or backward in the sequence.
Missing Bounding Boxes
The /detect/boxes2d stream is the default output stream from the Maivin detection services. Because of the amount of data included within this stream, it is not compatible with the default bounding-box overlay API for Foxglove Studio. To visualize bounding boxes, there are two methods:
- Use the EdgeFirst Detect Foxglove plugin to enable Foxglove to render the /detect/boxes2d stream. Details can be found at How to Download and Install Foxglove Plugin to view Maivin Detect and Mask Messages
- Enable the VISUALIZATION setting in the Maivin detect service and record the /detect/visualization topic into the MCAP. The process is detailed below.
How to enable Visualization
- On the target update the detect config:
sudo vi /etc/default/detect
- Now navigate to VISUALIZATION and set it to true as follow:
.....
# Enables publishing the visualization message
VISUALIZATION = "true"
..... - Once the visualization is set to true restart the detect service:
sudo systemctl restart detect
- Now that the service has restarted the visualization topic should be being published and can be recorded using the recorder by adding the new topic to the TOPIC flag in the recorder config
How to add Visualization topic
- On the target update the recorder config:
sudo vi /etc/default/recorder
- Now navigate to TOPIC and set it to true as follow:
.....
# Record the following topics. If certain topics are unavailable the recording
# will still be performed but the missing topics will be logged in a warning
# message. If none of the topics are available the recording will terminate.
TOPICS = "/imu /gps /camera/info /camera/h264 /detect/info /detect/boxes2d /detect/visualization"
..... - Once the topic is set restart the recorder service:
sudo systemctl restart recorder
- Now that the service has restarted the visualization topic should be recorded using the recorder
From Foxglove you can list topics, you should see the /detect/visualization topic.
Ensure the boxes2d topic is selected to produce image annotations in the camera panel.
- Select the gear icon from the top right of the camera panel.
- Ensure the purple eye is not greyed out for the rt/detect/visualization topic.
NOTE: The synchronize annotations field will ensure a better "lock" between the annotations that are found and the image displayed. However, if there are no annotations in the /detect/boxes2d stream, no image will be displayed.
Plotting IMU
The IMU sensor readings can also be plotted.
- Change the bottom left panel from Raw Messages into Plot.
- Click the Add a Series button
- Select rt/imu as the topic
- Select the desired sensor paramater, such as angular_velocity.y.
- Repeat from Step #2 to add additional plot series for the other parameters, for example to plot all of x,y,z from angular_velocity.
Viewing Segmentation Masks
The /detect/mask stream is the default output stream from the Maivin detection services. Because of the amount of data included within this stream, it is not compatible with the default Image drawing API in Foxglove Studio. To visualize the Segmentation Mask, please use the EdgeFirst Detect plugin. See How to Download and Install Foxglove Plugin to view Maivin Detect and Mask Messages for details.
Resources
Foxglove Studio provides many additional features for exploring the Maivin Dataset Recordings, refer to their documentation for details.
https://docs.foxglove.dev/docs/introduction/
Comments
0 comments
Please sign in to leave a comment.