Software Integration

Getting Started with ROS 2 and ZED

Open in ClaudeOpen in ChatGPT

The ZED ROS 2 wrapper lets you use the ZED stereo and monocular cameras with the second version of ROS, the Robot Operating System, a set of software libraries and tools that help you build robot applications.

It provides access to the following data:

  • Rectified/Unrectified images
  • Depth data
  • Colored 3D point cloud
  • Inertial information
  • Sensors information
  • Visual Inertial Odometry: Position and Orientation of the camera
  • Pose tracking: localization of the camera with loop closure
  • AI Detected objects
  • Human body skeleton tracking

image

The ROS 2 packages

The ZED ROS 2 integration is split across several packages. The core ones live in the zed-ros2-wrapper repository, while interfaces, robot models, and examples are distributed in dedicated repositories so that they can also be installed on machines without an NVIDIA® GPU.

Packages in the zed-ros2-wrapper repository

PackageDescription
zed_ros2A metapackage that depends on all the others, allowing you to install the full wrapper with a single dependency.
zed_wrapperThe main package. It provides the launch files, the YAML configuration files for every camera model, and automatically generates the camera URDF. It loads the zed_components in a single process.
zed_componentsThe actual ROS 2 components. It contains the ZedCamera component, used by the ZED Stereo Node, and the ZedCameraOne component, used by the ZED Monocular Node. These components can be composed in a single process for zero-copy communication — see Composition and IPC.
zed_debugA package intended for internal development only, used to debug the components by loading them in a single C++ process. You can safely exclude it from your build with --packages-skip zed_debug.

These packages are required by the wrapper but maintained in their own repositories. They do not depend on CUDA or the ZED SDK, so they can be installed on any ROS 2 machine (for example a remote workstation receiving data over the network).

PackageRepositoryDescription
zed_msgszed-ros2-interfacesAll the custom topics and services used by the ZED nodes. See The “zed_msgs” Package and the Custom Message Types reference.
zed_descriptionzed-ros2-descriptionThe URDF/xacro models and 3D meshes of the ZED camera family, used to build the camera TF tree and to display the camera in RViz 2. See The “zed_description” Package.
zed_display_rviz2zed-ros2-examplesPre-configured RViz 2 launch files and visualization tools, plus a set of tutorials and advanced examples. See Data display with RViz2.

Installation

Prerequisites

The ZED ROS 2 Wrapper has limited compatibility with ROS 2 Foxy on Ubuntu 20.04. Modules requiring ROS 2 Humble (or newer) features are disabled in Foxy (e.g. benchmarking and point cloud compression).

We recommend setting up your environment according to the recommended DDS and Network configurations to obtain the maximum performance.

Build the package

The zed-ros2-wrapper uses colcon to build packages.

If you haven’t set up your colcon workspace yet, please follow this tutorial.

To install the zed-ros2-wrapper packages, open a bash terminal, clone the repository from GitHub, and build:

$# Create your ROS 2 Workspace if you do not have one
$mkdir -p ~/ros2_ws/src/
$# Move to the `src` folder of the ROS 2 Workspace
$cd ~/ros2_ws/src/ 
$git clone https://github.com/stereolabs/zed-ros2-wrapper.git
$cd ..
$sudo apt update
$# Install the required dependencies
$rosdep install --from-paths src --ignore-src -r -y
$# Build the wrapper
$colcon build --symlink-install --cmake-args=-DCMAKE_BUILD_TYPE=Release
$# Setup the environment variables
$echo source $(pwd)/install/local_setup.bash >> ~/.bashrc
$source ~/.bashrc

The option --symlink-install is useful while developing your robotics system. It allows using symbolic links instead of copying files to the ROS 2 folders during the installation, where possible. Each package in ROS 2 must be installed and all the files used by the nodes must be copied into the installation folders. Using symbolic links allows you to modify them in your workspace, reflecting the modification during the next executions without needing to issue a new colcon build command to overwrite the files in the install folders. This is true only for the files that don’t need to be compiled (Python launch scripts, YAML configurations, etc).

If you are using a different console interface like zsh, you have to change the source command as follows: echo source $(pwd)/install/local_setup.zsh >> ~/.zshrc and source ~/.zshrc.

Starting the ZED node

To start the ZED node, open a terminal and use the CLI command ros2 launch:

$ros2 launch zed_wrapper zed_camera.launch.py camera_model:=<camera model>

The zed_camera.launch.py is a ROS 2 Python launch script that automatically starts the ZED node using “manual composition”, loading the parameters from the right “YAML files” associated with the specific camera model, and creating the static TF tree from the “URDF file” that is automatically generated from a xacro configuration file.

You can set your own configurations by modifying the parameters in the files common_stereo.yaml, common_mono.yaml, zed.yaml, zedm.yaml, zed2.yaml, zed2i.yaml, zedx.yaml, zedxm.yaml, virtual.yaml, zedxonegs.yaml, and zedxone4k.yaml available in the folder zed_wrapper/config. For a full description of each parameter, follow the complete guide.

Displaying ZED data

Using RViz 2

RViz 2 is a useful visualization tool in ROS 2. With RViz 2, you can visualize the left and right images acquired by the ZED cameras, the depth image, and the 3D colored point cloud, plus other useful information.

Launch the ZED wrapper along with RViz 2 by using the following command (installing the packages included in the zed-ros2-examples repository is required):

$ros2 launch zed_display_rviz2 display_zed_cam.launch.py camera_model:=<camera model>

If you haven’t yet configured your own RViz 2 interface, you can find a detailed tutorial here.

Using RViz 2 on NVIDIA® Jetson™ devices is not recommended for heavy tasks such as processing point clouds and full-resolution images; doing so may slow down the system and affect the reliability of the ZED nodes.

Displaying Images

The ZED node publishes both original (raw) and stereo-rectified (aligned) left and right images. In RViz 2, select a topic and use the image preview mode.

Here is a list of the main available image topics (more image topics are available):

  • rgb/color/rect/image: Color rectified image (left image by default)
  • rgb/color/raw/image: Color unrectified image (left image by default)
  • right/color/rect/image: Color rectified right image
  • right/color/raw/image: Color unrectified right image
  • left/color/rect/image: Color rectified left image
  • left/color/raw/image: Color unrectified left image
  • confidence/confidence_image: Confidence map

image

For a step-by-step tutorial on configuring an RViz 2 session to display only the video data you need, see Using Video Capture with ROS 2.

Displaying Depth

The depth map can be displayed in RViz 2 by subscribing to the following topic:

  • depth/depth_registered: 32-bit depth values in meters. RViz 2 will normalize the depth map on 8-bit and display it as a grayscale depth image.

An OpenNI compatibility mode is available by modifying the config/common_stereo.yaml file. Set depth.openni_depth_mode to true to get depth in millimeters with 16-bit precision, then restart the ZED node.

To learn how to fine-tune the depth and point cloud configuration and set up a dedicated RViz 2 session, see the Using Depth Perception with ROS 2 tutorial.

image

Displaying the Point cloud

A 3D colored point cloud can be displayed in RViz 2 by subscribing to the point_cloud/cloud_registered topic.

Add it in RViz 2 with point_cloud -> cloud -> PointCloud2.

Using RViz 2 on NVIDIA® Jetson™ devices is not recommended for heavy tasks such as processing point clouds and full-resolution images; doing so may slow down the system and affect the reliability of the ZED nodes.

image

Displaying Position and Path

The ZED position and orientation in space over time are published to the following topics:

  • odom: Odometry pose referred to the odometry frame
  • pose: Camera pose referred to the Map frame (the complete data fusion algorithm is applied)
  • pose with covariance: Camera pose referred to the Map frame with covariance
  • odometry path: The sequence of camera odometry poses in the Map frame
  • map path: The sequence of camera poses in the Map frame

For a detailed explanation of the reference frames, the TF tree, and how to configure the positional tracking module, see the Using Positional Tracking with ROS 2 tutorial. If you want to fuse GNSS data to improve outdoor localization, refer to the Using Geo Tracking with ROS 2 tutorial.

Launching with a recorded SVO video

With the ZED SDK, you can record and play back stereo video by using StereoLabs’ SVO file format. To record a sequence, open the ZED Explorer app (/usr/local/zed/tools) and click on the REC button.

To launch the ROS 2 wrapper with an SVO file, set the related parameter while starting the node. For example, to start the ZED 2 node using the SVO as input source:

$ros2 launch zed_wrapper zed_camera.launch.py camera_model:=<camera model> svo_path:=<full_path_to_svo_file>

with RViz 2:

$ros2 launch zed_display_rviz2 display_zed_cam.launch.py camera_model:=<camera model> svo_path:=<full_path_to_svo_file>

The ZED ROS 2 Wrapper now supports the SVO v2 format, which is available from the ZED SDK v4.1.

You can also record and replay data directly as ROS 2 bag files. For a complete walkthrough of recording, replaying, and synchronizing camera data, see the Record and Replay Camera Data with ROS 2 guide.

Dynamic reconfigure

You can dynamically change many configuration parameters during the execution of the ZED node. All the parameters that can be dynamically changed while the node is running are marked as [DYNAMIC] in the YAML configuration files. For a full description of each parameter please read the complete guide.

You can set the parameters by using the CLI command ros2 param set, e.g.:

$ros2 param set /zed/zed_node depth.depth_confidence 80

If the parameter is set successfully, you will get a confirmation message:

$Set parameter successful

If you try to set a parameter that’s not dynamically reconfigurable, or if you provided an invalid value, you will get this type of error:

$ros2 param set /zed/zed_node depth.depth_confidence 150
$Setting parameter failed: depth.depth_confidence must be a positive integer in the range [0,100]

You can also use a GUI to set dynamic parameter values:

$rqt

select Plugins -> Configuration -> Dynamic Reconfigure

image

Node Diagnostics

The ZED ROS 2 node publishes useful diagnostic information aggregated into the /diagnostics topic using the diagnostic_updater package.

These diagnostics report the health and the real-time performance of the node, including the camera grabbing frequency, the processing time of each enabled module (depth, positional tracking, object detection, etc.), the publishing rate of the main topics, and the camera temperature when available. Monitoring this information is the recommended way to verify that the node is running at the expected frequency and to detect performance bottlenecks on your hardware.

Diagnostic information can be analyzed and parsed by using ROS 2 tools, like for example the Runtime Monitor plugin of rqt.

image

If the node is not running at the expected frequency, refer to the Stereo Node Frequency Tuning guide to learn how to balance the processing load across the modules.

Going further

The pages above cover the basics of installing and running the ZED ROS 2 Wrapper. The following guides and tutorials describe the individual nodes, the perception modules, and the advanced topics in detail.

Nodes and camera setups

  • ZED Stereo Node — the complete reference of the node for stereo cameras, including all the published topics and configuration parameters.
  • ZED Monocular Node — the node dedicated to monocular cameras (ZED X One).
  • Custom Stereo Rig — how to build a custom stereo rig from a dual ZED X One setup.
  • Multi-Camera setup — how to run several ZED cameras on the same system.

Perception modules

Configuration, performance, and integration

Interfaces and models