Local Video Streaming

Using the ZED SDK, you can stream the side by side video of a ZED camera over a local IP network (Ethernet or Wifi).

Devices with proper permissions can access the live feed from anywhere using the ZED SDK on the receiving end. When taking a stream as input, the ZED API will behave as if a camera is directly connected to the PC. Every module of the ZED API will be available: depth, tracking, spatial mapping and more.

To stream the video of a ZED camera, refer to the Camera Streaming samples.

Hardware Requirements

When streaming live video, hardware acceleration is used to perform real-time encoding and decoding with minimal overhead. It is available on Nvidia GeForce, Quadro, Tesla and embedded Jetson boards (Nano, TX2, Xavier). There are some limitations regarding the number of concurrent encoding sessions, so make sure to check the Desktop GPU Support Matrix or Jetson Support Matrix to determine the appropriate hardware for your use case.

Streaming Modes

The ZED SDK can stream video using either H264 or H265 encoding modes.

Recommended Bit Rates

When streaming video content over the network, the user can define a specific bitrate. A low bitrate will degrade the quality of the images but requires less bandwidth to transmit over the network. On the other hand, a high bitrate will provide high-quality images with low compression artifacts but the required bandwidth will go up and might create freeze or drop frames.

Below are recommended streaming bitrates depending on the encoder, target resolution and FPS.

Benefits of Hardware Encoding

Both H264 and H265 encoding modes have been designed to use the hardware encoder built into NVIDIA GPUs (known as NVENC). With encoding offloaded to NVENC, the GPU and CPU are free for other operations. For example, in a compute-heavy scenario, it is possible to stream a video at full frame rate with minimal impact on the main application.

Encoding Quality

At a given bitrate, hardware encoding quality can vary depending on your GPU generation. The updated NVENC encoder on Turing-based NVIDIA GPUs (RTX 20-Series, Jetson Xavier) will typically produce superior quality than encoders on older generation GPUs (GTX 10-Series, Jetson Nano).

Streaming Protocol

The streaming module uses the RTP protocol to send and receive the video feed. If not specified, the sender will use the port 30000 and 30001 while the receiver will use ports that are determined as below:

• Try opening the same PORT as the sender.
• If not available, try opening PORT + 2.
• Continue testing new ports until one is available.

Here is an example where multiple ZED cameras are streaming video to one single host machine:

To get the ports opened by the receiver, use the following command on Linux:

$ sudo lsof -i -P -n

Multi-Camera Stream from One Host

You can encode and stream videos from multiple cameras connected to a single PC. There is a maximum number of concurrent hardware encoding sessions that can be started on a single NVIDIA GPU. Make sure to check the NVENC support matrix or Jetson support matrix to determine the appropriate hardware requirements.

Multi-Camera Stream from Different Hosts

It is also possible to stream video from several cameras connected to edge computers or gateways. Below is an example workflow.

In this configuration, the Senders are responsible for encoding and streaming an attached ZED’s video via the local network. On the other side, the Receivers read the stream and process the images using the ZED SDK. Therefore, the Receivers also require an NVIDIA GPU for NVENC decoding and SDK use.

Both the Senders and Receivers must be connected to the same local network. For improved bandwidth, connect the Senders and Receivers via Gigabit Ethernet instead of Wi-Fi. All ZED 2, ZED and ZED Mini cameras support Streaming with the ZED SDK.

Using the Streaming API

Streaming Configuration

To stream the video content of a ZED camera, you need to enable the streaming module with enableStreaming(). The standard grab() function will grab a frame and send it over the local network.

Use StreamingParameters to specify settings like bitrate, port, etc. Then pass those parameters when you call enableStreaming().

// Set the streaming parameters
sl::StreamingParameters stream_params;
stream_params.codec = sl::STREAMING_CODEC::H264; // Can be H264 or H265
stream_params.bitrate = 8000;
stream_params.port = 30000; // Port used for sending the stream
// Enable streaming with the streaming parameters
err = zed.enableStreaming(stream_params);

while (!exit_app) {
    zed.grab();
}
// Disable streaming
zed.disableStreaming();

# Set the streaming parameters
stream = sl.StreamingParameters()
stream.codec = sl.STREAMING_CODEC.H264 # Can be H264 or H265
stream.bitrate = 8000
stream.port = 30000 # Port used for sending the stream
# Enable streaming with the streaming parameters
err = zed.enable_streaming(stream)

while not exit_app :
    zed.grab()

# Disable streaming
zed.disable_streaming()

// Enable streaming with the streaming parameters
err =  zed.EnableStreaming(STREAMING_CODEC.H264_BASED, 8000, 30000);

RuntimeParameters runtimeParameters = new RuntimeParameters();
while (!exit_app) {
    zed.Grab(ref runtimeParameters);
}
// Disable streaming
zed.DisableStreaming();

Using a Stream as SDK Input

Video content streamed from a ZED camera is accessible remotely and can be used as standard input for the ZED API. Therefore, every ZED API module will work as if the camera was directly connected to the device. To use a remote stream as input, specify the IP address and the port of the sender in InitParameters. Then call open() to open the camera from the stream and grab() to grab a new frame and do the processing you want.

// Set the input from stream
InitParameters initParameters;
initParameters.input.setFromStream("127.0.0.1", 30000); // Specify the IP and port of the sender
// Open the camera
ERROR_CODE err = zed.open(initParameters);
if (err != ERROR_CODE::SUCCESS)
    exit(-1);

while (!exit_app) {
    if (zed.grab() == ERROR_CODE::SUCCESS) {
        // Any processing
    }
}
// Close the camera
zed.close();

# Set the input from stream
init = sl.InitParameters()
init.set_from_stream("127.0.0.1", 30000) # Specify the IP and port of the sender
# Open the camera
err = zed.open(init)
if err != sl.ERROR_CODE.SUCCESS:
	exit(1)

while not exit_app:
    if zed.grab() == sl.ERROR_CODE.SUCCESS:
        # Any processing

# Close the camera
zed.close()

// Set the input from stream
InitParameters initParameters = new InitParameters();
initParameters.inputType = INPUT_TYPE.STREAM; 
initParameters.ipStream = "127.0.0.1"; // Specify the IP of the sender
initParameters.portStream = "30000"; // Specify the port of the sender

RuntimeParameters runtimeParameters = new RuntimeParameters();
// Open the camera
ERROR_CODE err = zed.Open(ref initParameters);
if (err != ERROR_CODE.SUCCESS)
    Environment.Exit(-1);

while (!exit_app) {
    if (zed.Grab(ref runtimeParameters) == ERROR_CODE.SUCCESS) {
        // Any processing
    }
}
// Close the camera
zed.Close();

Code Example

See the Camera Streaming samples for more information on streaming a video and using a stream as input.