Video Recording

The ZED SDK allows you to record large video datasets using H.264, H.265 or lossless compression. The ZED SDK uses StereoLabs’ SVO format to store videos along with additional metadata such as timestamps and sensor data.

When loading SVO files, the ZED API will behave as if a ZED was connected and a live feed was available. Every module of the ZED API will be available: depth, tracking, spatial mapping and more. To record SVO videos, you can use the ZED Explorer or ZED Studio applications in GUI or command-line mode or build your own recording app using the ZED API.

What an SVO File Contains

An SVO file is a recording of what the camera outputs, not of what the ZED SDK computes. It only contains:

• The raw, unrectified video stream, exactly as it comes out of the image sensors: left and right images for a stereo camera, a single image for a monocular camera, encoded with the selected compression mode.
• The sensor data that the camera actually provides, stored as metadata alongside the video: IMU (accelerometer and gyroscope) and temperature on the models that embed them, and magnetometer and barometer on the ZED 2 and ZED 2i only. Check the sensor availability table for your camera model. With SVO2, each sensor is recorded at its own full rate instead of the image frame rate.
• The information required to replay the stream: camera model and serial number, resolution, frame rate, image and sensor timestamps, and any custom data you recorded yourself.

No processed data is ever written to the file. Rectified images, depth maps, point clouds, normal maps, confidence maps, camera poses, spatial maps, detected objects and bodies are not stored in an SVO file.

Everything Is Computed Live at Playback

While an SVO file is played back, the ZED SDK processes the raw frames exactly as it does with a live camera: rectification, depth estimation, positional tracking, spatial mapping, object and body detection all run on your machine as the file is read.

The configuration used during the recording session therefore does not constrain the playback session. A file recorded with DEPTH_MODE::NONE and no module enabled can be replayed with DEPTH_MODE::NEURAL_PLUS, positional tracking and object detection all enabled, and every one of them will produce results. The same applies to the depth range, the confidence and texture thresholds, the coordinate system and the unit system: they are chosen when you open the file, not when you record it.

What Cannot Be Changed at Playback

The recorded images have already been processed by the camera ISP, so everything applied before the frames were written is baked into the pixels:

• Camera control settings (sl::VIDEO_SETTINGS): exposure, gain, white balance, gamma, saturation, sharpness, denoising, and so on. Calling setCameraSettings() has no effect during playback.
• Resolution and frame rate, which are fixed when the file is created.

Changing any of these requires a new recording, which is why you should make sure the image settings are correct before starting a long capture session.

Compression Modes

SVO videos can be recorded using various compression modes. We provide both lossless and compressed modes to preserve image quality or reduce file size.

Benefits of Hardware Encoding

For optimal performance, we recommend using the H.264 and H.265 recording modes. They have been designed to use the hardware-based encoder (referred to as NVENC) built into NVIDIA® graphics cards. With encoding offloaded to NVENC, the GPU and the CPU are free for other operations. For example, in a compute-heavy scenario, it is now possible to record video at a full frame rate with minimal impact on the main application.

Encoding Quality

At a given bitrate, hardware encoding quality can vary depending on your GPU generation. The updated NVENC encoder on Turing-based NVIDIA® GPUs (RTX 20-Series, Jetson™ Xavier) will typically produce superior quality than encoders on older generation GPUs (GTX 10-Series, Jetson™ Nano).

Using SVO2

Release 4.1 of the ZED SDK introduced the SVO2 file format, designed to store high-frequency data from the camera and to record custom data in order to store data from external sensors. This recording format is enabled by default in the ZED SDK starting from version 4.1.

• High-frequency data: SVO(1) files would previously only record sensor data at the camera’s image frame rate (15-120 Hz). With SVO2, sensors are recorded at their respective frequency, enabling all ZED SDK algorithms which use high-frequency data, for example the Positional Tracking Gen 2.
• Custom Data: the SVO2 format exposes methods to record custom data. This data can be anything defined by the user, from metadata to label the specific SVO sequence, to data from external sensors, such as IMU, GPS, etc. All custom data is timestamped to be read alongside the original ZED data. More information on the Custom Data API here: C++ / Python / C# / C.

Please check out the custom data Recording and Playback samples on GitHub for more information.

Recording with Multiple Cameras

You can record videos with multiple cameras connected to the same PC. When using hardware encoding (H.264, H.265), make sure to check the NVENC support matrix or the Jetson one, which shows the maximum number of concurrent recording sessions that can be started on a single NVIDIA® GPU. You can also add multiple GPUs to a single server to increase the number of recording sessions with hardware encoding.

Using the Recording API

Video Recording

To record SVO files, you need to enable the Recording module with enableRecording(). Specify an output file name (eg: output.svo) and SVO_COMPRESSION_MODE, then save each grabbed frame. SVO lets you record video and associated metadata (timestamp, IMU data and more if available).

// Create a ZED camera object
Camera zed;
// Enable recording with the filename specified in argument
String output_path(argv[1]);
RecordingParameters recordingParameters;
recordingParameters.compression_mode = SVO_COMPRESSION_MODE::H264;
recordingParameters.video_filename = output_path;
err = zed.enableRecording(recordingParameters);
while (!exit_app) {
    // Each new frame is added to the SVO file
    zed.grab();      
}
// Disable recording
zed.disableRecording();

Video Playback

To play SVO files, simply add the file path as an argument in setFromSVOFile(). When loading SVO files, the ZED API will behave as if a ZED was connected and a live feed was available. Every module of the ZED API will be available: depth, tracking, spatial mapping and more, regardless of what was enabled during the recording session, as explained in What an SVO File Contains. When an SVO file is read entirely, END_OF_SVOFILE_REACHED error code is returned.

// Create a ZED camera object
Camera zed;
// Set SVO path for playback
String input_path(argv[1]);
InitParameters init_parameters;
init_parameters.input.setFromSVOFile(input_path);
// Open the ZED
err = zed.open(init_parameters);
sl::Mat svo_image;
while (!exit_app) {
    if (zed.grab() == ERROR_CODE::SUCCESS) {
        // Read side by side frames stored in the SVO
        zed.retrieveImage(svo_image, VIEW::SIDE_BY_SIDE);
        // Get frame count      
        int svo_position = zed.getSVOPosition();
    }
    else if (zed.grab() == END_OF_SVOFILE_REACHED) {
        std::cout << "SVO end has been reached. Looping back to first frame" << std::endl;
        zed.setSVOPosition(0);
    }
}

Extracting IMU Data from an SVO2 File

SVO2 files store sensor data at each sensor’s own frequency, as described in Using SVO2. During playback, that data is read back with the same getSensorsData() call used with a live camera, and the TIME_REFERENCE you pass decides what you get:

• TIME_REFERENCE::IMAGE returns the sample closest to the current video frame, so you get one sample per grab() call. This is what the example below does.
• TIME_REFERENCE::CURRENT returns the most recent sample independently of grab(). Call it more often than you call grab() and compare timestamps to walk through the samples recorded between two frames, as explained in Retrieve New Sensor Data.

// Open the SVO file (same as standard playback)
Camera zed;
InitParameters init_parameters;
init_parameters.input.setFromSVOFile("recording.svo2");
zed.open(init_parameters);
SensorsData sensors_data;
while (!exit_app) {
    if (zed.grab() == ERROR_CODE::SUCCESS) {
        // Retrieve the IMU sample closest to the current frame
        zed.getSensorsData(sensors_data, TIME_REFERENCE::IMAGE);
        SensorsData::IMUData imu = sensors_data.imu;
        // imu.linear_acceleration : accelerometer, in m/s²
        // imu.angular_velocity    : gyroscope, in deg/s
        // imu.pose                : orientation (IMU 6-DoF fusion)
    }
}

Code Example

Check out the SVO Recording, SVO Playback and SVO Export samples on GitHub.