Getting Started with ZED Live Link

The ZED Live Link allows developers to send camera tracking information as well as skeleton data into Unreal Engine 4 using the Live Link Plugin integrated into UE4. It also contains a Unreal Engine sample project already set-up to receive this data. This way, you can control a Skeletal Mesh in real-time using the Skeleton tracking module of the ZED SDK.

You don’t need to use the ZED Plugin for Unreal, only the ZED Live Link plugin and the Unreal project we provide you. You can have an overview of the solution on the diagram below:

The machine running ZED Live Link will use the ZED SDK, so it should be installed, download it here. The Unreal Project can run on a separate machine (Windows or Linux).

Live Link is a plugin designed to stream animation data coming from external sources to UE. You can get more information about this plugin in the UE4 Live Link doc. Our custom ZED Live Link plugin is therefore designed to stream ZED SDK skeleton data to UE4.

UE project

We provide you an Unreal project that already contains the different assets necessary to animate avatars properly using the ZED Live Link skeleton data. It is compatible with UE4 and UE5.

You will be able to use directly this project by hitting play while the ZED Live Link Plugin is running and see a default avatar animate. We are showing how to add your own models here: Animate New Avatar.

In this tutorial, you’ll learn how to use our ZED Live Link Plugin, connect it to our Unreal Project and animate a default avatar with the Live Link skeleton data. You must follow these steps:

1. Open the Unreal Project

Open our ZEDUnrealLiveLink.uproject project. You’ll see the following empty scene:

In the Content folder of the editor, you’ll see that there are already 3 Animation Blueprints, that correspond to 3 different avatars already imported in the project, as well as 2 assets called Avatar_livelink and MixamoRemap. We’ll see in the page Animate New Avatar what’s their use and how you can add your own assets to animate new avatars.

This project already handles the Live Link skeleton data in order to instantiate correctly avatars and place them in the scene with the corresponding joint positions and orientations. If you want to see how this is done, you can check the project Level Blueprint. The section Creating your project explains more in detail the different assets and the transformations implemented in our project to use the Live Link skeleton data.

You can follow these steps if the Live Link Plugin is not enabled yet in your project.

  1. Inside your project, In the Menu Bar under Edit, select Plugins.

  1. Under the Animation section, click Enabled for Live Link, and Yes on the confirmation window, then restart the Editor.

At this point, you must run the ZED Live Link plugin executable. It will call the ZED SDK in order to compute skeleton data and stream it to the Unreal project. So you must make sure to have the ZED SDK installed on the machine that will run the ZED Live Link plugin.

To know how to build the ZED Live link plugin, take a look at our Build Zed Live Link documentation page.

  • Open a terminal in the folder containing the ZED Live Link executable, and run
$ ./ZEDLiveLink

You can see the connection status in the terminal.

  • On Windows :

  • On Linux :

Adjust ZED SDK Parameters

The ZED SDK parameters are now available in a Json file next to the executable of the Live link sample. It allows you to change the parameters without re building the sample.

The configuration file looks like this :

{
    "InitParameters":
    {
      "input": "USB",
      "input_path": "",
      "resolution": "HD1080",
      "fps": 30,
      "depth_mode": "ULTRA"
    },
    "PositionalTrackingParameters":
    {
      "enable_pose_smoothing" : true,
      "enable_area_memory": false
    },
    "ObjectDetectionParameters":
    {
      "enable_module": true,
      "detection_model": "HUMAN_BODY_ACCURATE",
      "confidence": 60,
      "max_range": 700,
      "minimum_keypoints_threshold" : 5
    }
}

You can get more information about the different parameters role by looking at the API documentation.

By default, the Live Link sample will try to open a json file called ZEDLiveLinkConfig.json next to the executable.

But you can also give the path to your config file as an argument when running the Live Link sample :

$ ./ZEDLiveLink C:/path/to/config/file.json

This step is normally handled automatically by the Unreal Project provided. If you have data coming from several Live Link sources, you might need to do the following to make sure you’re connected to the right source.

  1. Inside the UE4 Editor, from the Menu Bar under Window, select Live Link.

  1. In the Live Link window, click the Source button, then under Message Bus Source, select your ZED source.

5. Hit Play in Unreal

With the project you’re provided, once the ZED Live Link plugin is running, you can animate a default avatar by directly hitting play in Unreal. You will see avatars animated with skeleton data coming from the connected Live Link source:

Troubleshooting

  • Your firewall might block the data stream. If you do not see the ZED Source in the Live Link window, try to disable it.

  • If the ZED Source is not yet detected in UnrealEngine, enable Enable by default in Edit -> Project Settings -> UDP Messaging -> Enable by default

  • On Linux, if the ZED Live Link plugin executable crashes at the start, try to run the ldd command onto the sl_zed_c.so library :
$ ldd sl_zed_c.so

It will show all the dependencies required by the .so and allow you to install anything that might be missing (for example lib-usb).

Note that the c wrapper used for the Live link plugin is also available here : https://github.com/stereolabs/zed-c-api.

If you encounter issues running the live link plugin, do not hesitate to build the wrapper yourself and place it in the lib/win64 or /linux folder.