Depth Settings

Sensing Modes

The ZED SDK provides two modes for depth sensing: STANDARD and FILL.

Standard Mode

The STANDARD mode is the default depth sensing mode with the ZED. The STANDARD mode preserves distance metrics and shapes, and runs faster than the FILL mode, but it contains holes due to visual occlusions and filtering. These holes are represented by black areas in the depth image. Use the STANDARD mode for applications such as autonomous navigation, obstacle detection, 3D mapping, people detection and tracking.

In STANDARD mode, holes (black pixels) have different values depending on their type. They are associated with an enum:

• NAN referred as OCCLUSION_VALUE. The depth of the pixel cannot be estimated as it is occluded or an outlier.
• -INFINITY referred as TOO_CLOSE. The depth of the pixel cannot be estimated as it is too close to the camera.
• INFINITY referred as TOO_FAR. The depth of the pixel cannot be estimated as it is too far from the camera.

You can check for valid depth values using isValidMeasure().

Fill Mode

The FILL mode provides a fully dense depth map with a Z value for every pixel (X, Y) in the left image. The FILL mode fills holes and occlusions in the depth map and adds a filtering stage that improves edges and temporal stability but can alter the actual distance of objects in the scene.

This mode is recommended for applications such as mixed reality capture and visual effects. The FILL mode requires more resources and runs at a lower FPS than the STANDARD mode.

To configure depth sensing in FILL mode, set SENSING_MODE_FILL in RuntimeParameters.

// Set sensing mode in FILL
RuntimeParameters runtime_parameters;
runtime_parameters.sensing_mode = SENSING_MODE::FILL;

# Set sensing mode in FILL
runtime_parameters =sl.RuntimeParameters()
runtime_parameters.sensing_mode = sl.SENSING_MODE.FILL

// Set sensing mode in FILL
RuntimeParameters runtime_parameters = new RuntimeParameters();
runtime_parameters.sensingMode = SENSING_MODE.FILL;

Depth Modes

Several depth modes are available to fit your application’s needs. These settings adjust the level of accuracy, range and computational performance of the depth sensing module. They are listed ranked below from the better one to the faster one.

• NEURAL: brings the depth sensing to the next level of accuracy by using AI techniques. Accurate and smooth, even in the most challenging situations.
• ULTRA: offers the highest depth range and better preserves Z-accuracy along the sensing range for computer vision based technique.
• QUALITY: has a strong filtering stage giving smooth surfaces.
• PERFORMANCE: designed to be smooth, can miss some details.

Generally speaking, we recommend using the ULTRA mode for both desktop and embedded applications. If your application requires a lot of resources, switch to PERFORMANCE mode.

// Set depth mode in ULTRA
InitParameters init_parameters;
init_parameters.depth_mode = DEPTH_MODE::ULTRA;

# Set depth mode in ULTRA
init_parameters = sl.InitParameters()
init_parameters.depth_mode = sl.DEPTH_MODE.ULTRA

// Set depth mode in ULTRA
InitParameters init_parameters = new InitParameters();
init_parameters.depthMode = DEPTH_MODE.ULTRA;

Depth Range

Depth range corresponds to the minimum and maximum distance at which the depth of an object can be estimated.

Minimum Range

The minimum range can be lowered from its default value using the depth_minimum_distance setting in InitParameters.

InitParameters init_parameters;
init_parameters.coordinate_units = UNIT::METERS;
init_parameters.depth_minimum_distance = 0.15 ; // Set the minimum depth perception distance to 15cm

init_params = sl.InitParameters()
init_parameters.coordinate_units = sl.UNIT.METER
init_parameters.depth_minimum_distance = 0.15 # Set the minimum depth perception distance to 15cm

InitParameters init_parameters = new InitParameters();
init_parameters.coordinateUnits = UNIT.METER;
init_parameters.depthMinimumDistance = 0.15f ; // Set the minimum depth perception distance to 15cm

Tips:

• Lowering the minimum range to very small values can dramatically increase memory requirements and reduce FPS. Increase this value to improve performance.
• For applications requiring long-range depth perception, we recommend setting depth_minimum_distance to 1m or more for improved performance.

Maximum Range

The maximum range can be increased using sl::InitParameters::depth_maximum_distance.

InitParameters init_parameters;
init_parameters.depth_mode = DEPTH_MODE::ULTRA ;  // Set the depth mode to ULTRA
init_parameters.coordinate_units = UNIT::METER;
init_parameters.depth_maximum_distance = 40;      // Set the maximum depth perception distance to 40m

init_params = sl.InitParameters()
init_parameters.depth_mode = sl.DEPTH_MODE.ULTRA  # Set the depth mode to ULTRA
init_parameters.coordinate_units = UNIT.METER
init_parameters.depth_maximum_distance = 40       # Set the maximum depth perception distance to 40m

InitParameters init_parameters = new InitParameters();
init_parameters.depthMode = DEPTH_MODE.ULTRA ;    // Set the depth mode to ULTRA
init_parameters.coordinateUnits = UNIT.METER;
init_parameters.depthMaximumDistance = 40;        // Set the maximum depth perception distance to 40m

Tips:

• We recommend using the ULTRA depth mode to improve depth accuracy at long distances.
• The maximum depth range can be reduced to clamp values above a certain distance using sl::InitParameters::depth_maximum_distance. This is useful to reduce depth jitter at long distances.
• Increasing the maximum range has no impact on memory or FPS.

Depth Stabilization

Depth stabilization is a feature which temporally fuses and filters the depth map over several frames. This allows to reduce jitter and improve depth accuracy on static objects. Depth stabilization works even when the camera is moving by using the positional tracking capability of the ZED SDK. It can also detect moving objects to avoid fusing depth of dynamic areas.

Depth stabilization is enabled by default. Since it enables positional tracking in background, you can disable depth stabilization using init_parameters.depth_stabilization = false to improve computational performance.

Tips: For fixed cameras, we recommend enabling PositionalTrackingParameters::set_as_static while using depth stabilization. This allows the depth stabilizer module to know the camera is static so it can disable visual tracking and reduce computational load.

Depth Accuracy

Stereo vision uses triangulation to estimate depth from a disparity image, with the following formula describing how depth resolution changes over the range of a stereo camera:

Dr=Z^2*alpha, where Dr is depth resolution, Z the distance and alpha a constant.

Depth accuracy decreases quadratically over the z-distance, with a stereo depth accuracy of 1% of the distance in the near range to 9% in the far range. Depth accuracy can also be affected by outliers’ measurements on homogenous and textureless surfaces such as white walls, green screens and specular areas. These surfaces usually generate temporal instability in depth measurements.

Best Practices

All Stereolabs cameras can be used both indoors and outdoors at long range (up to 20m). The configuration that offers the best depth accuracy is to place the camera at a working distance of 30cm to 1 meter of the scene. To capture fast movements, use the camera high-speed modes (HD720 @ 60 FPS or VGA @ 100 FPS).

Avoid using the camera in very low-light environments as the camera uses the color images for depth perception. However you can improve low-light performance by reducing the camera frame rate to 15FPS.

You can also extend the range of the USB cable up to 15m by using an USB 3.0 extension cable and up to 100m using an optical fiber USB 3.0 extender.