Release Notes

• What's New

  Apr 3, 2024

  This release of the ZED SDK is now a stable release of the major version 4. It brings significant stability and performance improvements. We're introducing a new generation of the Positional Tracking module for more precise and robust localization. A new version of SVO is also available, recording full sensor data and supporting custom user data for ease of use.
  

• 4.1.0

  New Features

  Positional Tracking GEN2

  • Introduced a new Positional Tracking generation, GEN2. It can be enabled using PositionalTrackingParameters::mode = POSITIONAL_TRACKING_MODE::GEN2. It outputs more robust and precise trajectories compared to GEN1, leveraging IMU data at full frequency for better motion estimation. This version requires the new SVO2 data. A new warning ERROR_CODE::SENSORS_DATA_REQUIRED has been introduced to inform about missing high-frequency data when using the module. The state enum has been reworked for clarity on the module status; ODOMETRY_STATUS, SPATIAL_MEMORY_STATUS, and POSITIONAL_TRACKING_FUSION_STATUS are now available. It's accessible using the new struct PositionalTrackingStatus.

  Recording using SVO2

  • Introduced a new default SVO2 format, recording sensor data at full frequency rate.
  • Supports custom user data using new functions Camera::ingestDataIntoSVO alongside a new struct SVOData.The data can then be retrieved when opening the file using Camera::retrieveSVOData. Each data stream is associated with a string key, the data points are associated with a timestamp to replay synchronously if needed. Camera::getSVODataKeys retrieves all custom data stream keys present in the current SVO2 file. This format is mandatory to use the new Positional Tracking Gen2 with IMU data for best accuracy. The older format can still be recorded by setting the environment variable ZED_SDK_SVO_VERSION to 1.

  Improve Neural Depth

  • Introduced a new NEURAL+ mode providing higher quality than Neural with sharper edges and more accurate depth, though requiring more computation.
  • Improved NEURAL mode accuracy and confidence. This new model is more accurate with no runtime difference. The confidence has been reworked for accurate edges and minimized flying pixels. It also improves confidence estimation in challenging areas such as sky, hard illuminations, very close objects, etc., to remove false depth measurements reliably. Neural modes from previous versions can be accessed on demand.

  Region of interest per module

  • Reworked the Region of Interest to be applied independently for each module instead of implicitly for all at once. This feature allows masking an irrelevant part of the frame to avoid polluting results with unnecessary data. Typical usage includes masking a vehicle part where the camera is mounted to improve positional tracking precision and irrelevant depth data for obstacle detection. It can also be used to detect objects or body skeletons on only some part of the frame. The feature is now applied to each module through a new MODULE enum, with defaults applying to all modules. Functions Camera::setRegionOfInterest and Camera::getRegionOfInterest can be used to set the ROI. Auto-detection mode can find the static part of the image automatically by moving the camera for some time. It can be enabled using Camera::startRegionOfInterestAutoDetection, with RegionOfInterestParameters for adjusting detection parameters.

  Global localization / Geo Tracking

  • Improved Global Localization module accuracy and robustness, compatible with the new Positional Tracking GEN2. It now better behaves across a wider range of settings, especially across GNSS signal precision, from very precise RTK to dead reckoning.
  • Supports and requires an antenna position relative to the camera, as well as the type of GNSS fix (2D, 3D, RTK FIX, etc.) to improve precision of the fusion. Samples have been improved for clarity, utilizing SVO2 custom data to record GNSS data, allowing a single recorded SVO2 file to contain all necessary data.

  General

  • Improved ZED X image quality by reducing motion blur and optimizing ISP to limit exposure time, resulting in better image quality and positional tracking precision improvements. Installing the SDK will automatically install the ISP config override for all ZED X cameras.
  • Improved image validity check module on the ZED 2 and ZED 2i to detect rare cases of auto exposure getting stuck underexposing the left image.
  • Improved ZED X capture process for more stable opening of multiple cameras and an optimized color conversion workflow.
  • Reduced CPU usage and memory allocations in the ZED X capture process.
  • Improved depth minimum range to better estimate working range based on camera baseline, lens, and depth algorithm used.
  • Enhanced performance of Body Tracking models for better runtime performance and accuracy compared to version 4.0.
  • Fixed a random issue where the depth minimum value was clamped at 1.5m.

  Fusion

  • Improved Body Tracking fusion synchronization process to better handle low or irregular framerate
  • Improved the body tracking multi-camera calibration process in ZED 360
  • Fixed the depth unit returned by retrieveMeasure, the user-defined UNIT was not applied.
    

  Bug fixes

  • Fixed AI performance regression for NVIDIA Pascal GPUs (GTX 1070, GTX 1080, etc.), correcting incorrect fp16 execution without hardware support, leading to poor performance compared to fp32.
  • Fixed grab recovery that lost camera video settings.
  • Fixed a crash that could occur when calling the resetPositionalTracking function.
  • Fixed the incorrect reported camera model when recording or streaming with ZED Explorer.
  • Fixed a crash when using the Object detection masks that could get corrupted.

  Tools

  • Improved tools UI.
  • Improved ZED Depth Viewer interface.
  • Improved ZED Explorer when using multiple cameras.

  Samples

  • Improved the Global Localization samples.

  Wrappers

  Python

  • Improved Object Detection and Body Tracking parameters check to avoid passing incorrect arguments leading to invalid behavior (like hanging retrieves functions).
  • Added a new Concurrent Object Detection and Body Tracking sample to facilitate usage.
  • Fixed a crash that could occur when calling get_device_list function when no cameras were detected.
  • Fixed the function that checks if the hardware sensors are available.

  Documentation

  • Improved Docker documentation for ZED X.