We announce today the release of the latest ZED SDK with Stereolabs’ new Neural Depth perception module based on deep neural networks for advanced robotics and spatial analytics applications.
The new Neural Depth mode generates a dense, high-fidelity depth map up to 2k resolution. The map is able to accurately capture challenging environments such as reflective surfaces and untextured areas. It also delivers high Depth completeness with unprecedented edge accuracy for advanced spatial segmentation.
Combined with Stereolabs’ latest Depth Camera, the ZED 2i, the new Neural mode works indoors and outdoors with a range of up to 40m and a 120-degree field of view, making it one of the most advanced depth cameras on the market at just $499.
The new Neural Depth mode relies on combining advanced computer vision with a custom CNN (convolutional neural network) specifically trained on stereo images to improve extraction, matching, and cost aggregation. This combination produces a hyper-accurate depth map while maintaining real-time workflow.
Since Depth Maps are at the heart of every ZED SDK module, users benefit from an overall superior performance, with improved Depth Sensing, Tracking, also called Visual Odometry, Mapping, Object Detection, and Skeleton Tracking, all use depth and benefit from the improvement provided by Neural Depth mode.
The ability to generate a dense Depth Map with high accuracy is a game changer for many applications. Compared to previous releases, the NEURAL depth mode generates a significantly better edge segmentation and brings more stability in low-lighted surfaces. It also provides smoother and planer surfaces to work with.
These improvements allow many concrete enhancements in various applications such as Terrain Mapping for Robotics, Pick & Place applications for Industrial Arms, and Mixed Reality.
Having an accurate depth map is key for Terrain Mapping especially for off-road navigation. The Neural Depth mode provides a robust and dense depth map allowing for precise 3D mapping.
The new Neural Depth mode provides game-changing accuracy that unlocks new applications such as Bin Picking and Pick & Place applications for Industrial Arms.
This new depth map enables precise body edge detection, allowing for photorealistic Mixed Reality applications such as background subtraction. Note that the ZED Plugins for Unity and Unreal support the ZED SDK 3.7.
The ZED SDK 3.7 has been optimized for the latest NVIDIA JetPack 5.0 and Jetson AGX Orin™ developer kit. In this configuration, Neural mode runs at 34 fps on the newest NVIDIA Jetson AGX Orin compared to 12 fps on Jetson Xavier NX and JetPack 4.X.
The performance of the new Jetson AGX Orin unlocks new capabilities such as processing 3D data coming from multiple ZED 2i Depth Cameras, making it the ideal combination for autonomous robotics and edge analytics solutions.
The NVIDIA Jetson AGX Orin delivers 275 trillion operations per second (TOPS), over 8x the compute performance of its predecessor, Jetson AGX Xavier. Developers can jump-start their next-generation edge AI and robotics applications on the Jetson AGX Orin developer kit for seamless deployment on the full range of Orin-based production modules when available starting July 2022.
ZED 2i Stereo Camera and NVIDIA Jetson Orin AGX Developer Kit
The NEURAL Depth mode is available in the new ZED SDK 3.7.
Download and install the new version for your configuration (OS, CUDA version).
At the end of the installation, the depth CNN will be optimized to provide the best results at the maximum speed.
All the SDK examples are already plug-and-play compatible with the new Depth mode.
For your existing code, just change the InitParameters::depth_mode to DEPTH_MODE::NEURAL.
The NEURAL depth mode is available starting from the ZED SDK v3.7
Enjoy and send us your feedback!