4.0 Migration Guide

Overview

• Added support for the new ZED X and ZED X Mini cameras.
• Added new camera models MODEL::ZED_X and MODEL::ZED_XM.
• Added new input type INPUT_TYPE::GMSL field to sl::DeviceProperties for GMSL cameras (support for GMSL cameras is only available on hosts with Nvidia Jetson SoC).
• Added INPUT_TYPE input_type field to sl::DeviceProperties struct.
• Adding an asynchronous camera recovery mechanism. If enabled and when the camera disconnect, the grab function will immediately return with a new ERROR_CODE::CAMERA_REBOOTING error code and will initiate a recovery function in a parallel thread. By default, the grab function is blocking and will only return once the camera connection has been restored (or if the time out is reached).
  • Added boolean async_grab_camera_recovery optional parameter to sl::InitParameters to enable this behavior (set to false by default).
  • Added ERROR_CODE::CAMERA_REBOOTING error code.

Video

• setFromCameraID and setFromCameraSerialNumber methods from sl::InputType now take an optional BUS_TYPE to choose between USB or GMSL cameras. When unspecified, the method searches for available USB cameras first, then searches for GMSL.
• Added new methods getInputType(), getConfiguration(), and isInit() for sl::InputType.
• Added a scale utility method to sl::CameraParameters to easily convert camera intrinsic parameters for a given resolution.
• Added RESOLUTION:HD1200 (1920x1200) and RESOLUTION::SVGA (960x600) resolutions for ZED X and ZED X Mini cameras.
• Added RESOLUTION:AUTO resolution which sets RESOLUTION::HD720 resolution for USB cameras, and RESOLUTION::HD1200 resolution for GMSL cameras.
• Added new parameters in VIDEO_SETTINGS :
  • EXPOSURE_TIME : Image sensor exposure time in ms
  • ANALOG_GAIN : Analog sensor gain in dB
  • DIGITAL_GAIN : Digital ISP gain in dB
  • AUTO_EXPOSURE_TIME_RANGE : Defines the range of the exposure time in automatic control
  • AUTO_ANALOG_GAIN_RANGE : Defines the range of sensor gain in automatic control. Min/Max range between [1000 - 16000]mdB.
  • AUTO_DIGITAL_GAIN_RANGE : Defines the range of digital ISP gain in automatic control.
  • EXPOSURE_COMPENSATION : Exposure target compensation made after AE. Reduces the overall illumination by factor of F-stops. values range is [0 - 100] (mapped between [-2.0, 2.0]). Only available for GMSL-based cameras.
  • DENOISING : Defines the level of denoising applied on both left and right images. values range is [0 - 100]. Only available for GMSL-based cameras.
• Camera::getCameraSettings now returns an ERROR_CODE and uses a reference to retrieve the value.
• Camera::setCameraSettings now returns an ERROR_CODE instead of void.

Depth

• Improved previous NEURAL depth mode which is now more robust to challenging situations such as low light, heavy compression, and textureless areas such as interior plain walls, overexposed areas, and exterior sky.
• SENSING_MODE is now deprecated and not usable in the API. To get the same behavior as the deprecated SENSING_MODE::FILL, please use the new sl::RuntimeParameters.enable_fill_mode parameter.
• confidence_threshold in sl::RuntimeParameters is now set by default to 95 instead of 100 previously. Following the SENSING_MODE removal (see above), the default confidence is now set lower to remove the potential incorrect depth value that can produce trails behind objects, while keeping a relatively dense depth map.

Object Detection

Object detection models have been refactored to separate object detection models and human body detection models into two separate classes: OBJECT_DETECTION_MODEL and BODY_TRACKING_MODEL respectively.

• The new OBJECT_DETECTION_MODEL enum contains the following models:
  • MULTI_CLASS_BOX_FAST: Any objects, bounding box based.
  • MULTI_CLASS_BOX_MEDIUM: Any objects, bounding box based, a compromise between accuracy and speed.
  • MULTI_CLASS_BOX_ACCURATE: Any objects, bounding box based, more accurate but slower than the base model.
  • PERSON_HEAD_BOX_FAST: Bounding Box detector specialized in person heads, particularly well suited for crowded environments, person localization is also improved.
  • PERSON_HEAD_BOX_ACCURATE: Bounding Box detector specialized in person heads, particularly well suited for crowded environments, the person localization is also improved, more accurate but slower than the base model.
  • CUSTOM_BOX_OBJECTS: For external inference, using your own custom model and/or frameworks. This mode disables the internal inference engine, the 2D bounding box detection must be provided.
• Added the possibility to use multiple object detection models instances at once using the instance_id parameter in sl::ObjectDetectionParameters (and in related methods such as Camera::getObjectDetectionParameters, Camera::disableObjectDetection, …).
• Renamed MULTI_CLASS_BOX to MULTI_CLASS_BOX_FAST in OBJECT_DETECTION_MODEL.
• Renamed PERSON_HEAD_BOX to PERSON_HEAD_BOX_FAST in OBJECT_DETECTION_MODEL.
• Renamed enable_mask_output to enable_segmentation in sl::ObjectDetectionParameters.
• Removed keypoint_2d, keypoint, keypoint_confidence, local_position_per_joint, local_orientation_per_joint, and global_root_orientation from sl::ObjectData. These attributes have been moved to the sl::BodyData class for Body Tracking.

Body Tracking

• The new BODY_TRACKING_MODEL enum contains the following models:
  • HUMAN_BODY_FAST: Keypoints based, specific to human skeleton, real-time performance even on Jetson or low-end GPU cards.
  • HUMAN_BODY_MEDIUM: Keypoints based, specific to human skeletons, compromise between accuracy and speed.
  • HUMAN_BODY_ACCURATE: Keypoints based, specific to human skeletons, state-of-the-art accuracy, requires powerful GPU.

The model now depends both on the accuracy wanted both also the BODY_FORMAT (see below)

• Changed BODY_FORMAT naming from POSE_XX to BODY_XX. The POSE_18 and POSE_34 formats have been renamed to BODY_18 and BODY_34 respectively.
• Added two new human body formats in BODY_FORMAT:
  • BODY_38: Adds additional keypoints to BODY_34, specifically on the feet and hands. The body fitting now provides accurate orientations for the feet and hands.
  • BODY_70: Adds full hand keypoints to BODY_38, 4 keypoints per finger.
• Added the possibility to use multiple body tracking models at once using the instance_id parameter in sl::BodyTrackingParameters (and in related methods such as Camera::getBodyTrackingParameters, Camera::disableBodyTracking, …).
• Added a BODY_KEYPOINTS_SELECTION enum to filter keypoints for specific use cases. Currently available options include BODY_KEYPOINTS_SELECTION::FULL and BODY_KEYPOINTS_SELECTION::UPPER_BODY.

Please refer to the body tracking documentation for more information on the human body tracking models and formats.

• Analogous to object detection, body tracking objects now have their own methods and parameters exposed separately:
  • Added a new sl::BodyData class which contains detection and tracking information on detected human bodies, such as keypoint, joint, and skeleton root information.
  • Added a new Camera::retrieveBodies method to retrieve human body detection data after a Camera::grab call, similar to Camera::retrieveObjects.
  • Added a new sl::Bodies class which is returned from Camera::retrieveBodies, similar to sl::Objects.
  • Added a new sl::BodyTrackingParameters and sl::BodyTrackingRuntimeParameters classes which define all of the parameters related to body tracking.
• Renamed BODY_PARTS to BODY_18_PARTS.
• Renamed BODY_PARTS_POSE_34 to BODY_34_PARTS.
• Added BODY_38_PARTS and BODY_70_PARTS which list all keypoints for new body formats.
• Renamed BODY_BONES to BODY_18_BONES.
• Renamed BODY_BONES_POSE_34 to BODY_34_BONES.
• Added BODY_38_BONES and BODY_70_BONES bones containers for new body formats.

sl::Orientation

• setRotation has been removed. Please use setRotationMatrix instead.
• getRotation has been removed. Please use getRotationMatrix instead.

sl::CalibrationParameters

• R and T attributes have been removed. Please use stereo_transform instead.

sl::CameraInformation

The following attributes have been removed and are available in sl::CameraConfiguration and sl::SensorsConfiguration

• calibration_parameters → sl::CameraConfiguration::calibration_parameters
• calibration_parameters_raw → sl::CameraConfiguration::calibration_parameters_raw
• camera_imu_transform → sl::CameraConfiguration::camera_imu_transform
• camera_firmware_version → sl::CameraConfiguration::firmware_version
• sensors_firmware_version → sl::SensorsConfiguration::firmware_version
• camera_fps → sl::CameraConfiguration::fps
• camera_resolution → sl::CameraConfiguration::resolution

sl::Pose

• getRotation has been removed. Please use getRotationMatrix instead.

sl::RuntimeParameters

• textureness_confidence_threshold has been removed. Please use texture_confidence_threshold instead.

Utility methods

• model2str has been removed. Please use toString instead.
• errorCode2str has been removed. Please use toString instead.
• unit2str has been removed. Please use toString instead.
• str2unit has been removed.