Getting Started with GStreamer and ZED GStreamer is a popular framework used to create custom media pipelines by combining modular plugins. The ZED GStreamer plugins greatly simplifies the use of ZED camera and SDK in a GStreamer media pipeline. The ZED GStreamer package is composed of five plugins providing five elements, a metadata library and a Linux RTSP Server application. Plugins Name Classification Source Code Description zedsrc Source Github acquires camera color image, depth map, sensors data and object detection information and pushes them in a GStreamer pipeline. zeddemux Stream Demuxer Github receives a composite zedsrc stream (color left + color right + metadata or color left + depth map + metadata), processes the eventual depth data and pushes them in two separated new streams named src_left and src_aux. A third source pad is optionally created for metadata to be externally processed. zeddatamux Metadata Muxer Github receives a video stream compatible with ZED caps and a ZED Data Stream generated by the zeddemux and adds metadata to the video stream. This is useful if metadata are removed by a filter that does not automatically propagate them. zeddatacsvsink File Sink Github sink plugin that receives ZED metadata, extracts the Positional Tracking and the Sensors Data information and save them in a CSV file. Demonstrates how to retrieve and parse Positional Tracking and the Sensors metadata. zedodoverlay Filter Effect Video Github transform filter plugin that receives ZED combined stream with metadata, extracts object detection information and draws the overlays on the oncoming image frames. Demonstrates how to retrieve and parse Object Detection metadata. Library Name Classification Source Code Description zedmeta Metadata Library Github GStreamer library to define and handle the ZED metadata (Positional Tracking data, Sensors data, Detected Object data, Detected Skeletons data). Application Name Classification Source Code Description RTSP Server Server application Github Linux application that instantiates an RTSP server from a text launch pipeline “gst-launch” like.Detected Skeletons data). Installation Prerequisites Stereo camera: ZED 2, ZED, ZED Mini CMake (v3.1+) GStreamer 1.0 Windows installation Install the latest ZED SDK from the official download page [Optional to compile the zedsrc plugin to acquire data from a ZED camera device] Install Git Install CMake Install a GStreamer distribution (both runtime and development installers). The installer should set the installation path via the GSTREAMER_1_0_ROOT_X86_64 environment variable. Add the path %GSTREAMER_1_0_ROOT_X86_64%\bin to the system variable PATH [Optional] Install OpenCV to build the zedodverlay filter Run the following commands from a terminal or command prompt, assuming CMake and Git are in your PATH. git clone https://github.com/stereolabs/zed-gstreamer.git cd zed-gstreamer mkdir build cd build cmake -G "Visual Studio 16 2019" .. cmake --build . --target INSTALL --config Release Linux installation Install prerequisites Install the latest ZED SDK from the official download page Update list of apt available packages sudo apt update Install GCC compiler and build tools sudo apt install build-essential Install CMake build system sudo apt install cmake Install GStreamer, the development packages and useful tools: sudo apt install libgstreamer1.0-0 gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav libgstrtspserver-1.0-0 gstreamer1.0-doc gstreamer1.0-tools gstreamer1.0-x gstreamer1.0-alsa gstreamer1.0-gl gstreamer1.0-gtk3 gstreamer1.0-qt5 gstreamer1.0-pulseaudio libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libgstrtspserver-1.0-dev [Optional] Install OpenCV to build the zedodverlay filter sudo apt install libopencv-dev libopencv-contrib-dev Clone the repository git clone https://github.com/stereolabs/zed-gstreamer.git cd zed-gstreamer Build mkdir build cd build cmake -DCMAKE_BUILD_TYPE=Release .. make # Note: do not use the `-j` flag because parallel build is not supported sudo make install Installation test Open a command line console and try the following commands to verify that everything is correctly installed: Check ZED Video Source Plugin installation inspecting its properties: gst-inspect-1.0 zedsrc Check ZED Video Demuxer installation inspecting its properties: gst-inspect-1.0 zeddemux Check ZED Data Mux Plugin installation inspecting its properties: gst-inspect-1.0 zeddatamux Check ZED CSV Sink Plugin installation inspecting its properties: `gst-inspect-1.0 zeddatacsvsink` Check ZED Object Detection Overlay Plugin installation inspecting its properties: `gst-inspect-1.0 zedodoverlay` Use cases Following a list of use case examples, linked to the relative documentation page: DISPLAY LEFT IMAGE ON SCREEN DISPLAY COLOR STEREO COUPLE WITH VGA RESOLUTION AT 100 FPS DISPLAY DEPTH MAP WITH HD720 RESOLUTION AT 60 FPS H264 STREAMING OVER UDP ON LOCAL NETWORK SPLIT THE STEREO COUPLE AND DISPLAY SINGLE STREAMS SPLIT THE RGB/LEFT SYNCHRONIZED STREAM AND DISPLAY SINGLE STREAMS SENSORS DATA LOGGING TO CSV FILE DISPLAY OBJECT DETECTION BOUNDING BOXES DISPLAY SKELETON TRACKING RESULTS INJECT ZED METADATA INFORMATION IN A STREAM CREATE AN RTSP SERVER TO STREAM ZED DATA