AR Object Placement with Unity

In this tutorial, you will learn how to place virtual objects on real-world surfaces.

Introduction

To place an object on a real-world surface, you need to detect planes in the scene. While you could scan and create a mesh of the entire scene with Spatial Mapping, the plugin provides a simpler way of placing objects on horizontal surfaces: Instant Plane Detection.

Plane Detection allows you to capture meshes of real-world planar surfaces without the need for pre-scanning or moving the camera. Using a single image and depth frame, the plugin extracts planes by detecting surfaces with similar normals.

Add Plane Detection

Create a new Scene.
Replace Main Camera with ZED_Rig_Stereo.
Rotate the Directional Light to face straight down, and add a ZEDLight component to it.
Right-click the Hierarchy to make a new Empty GameObject.
Name it Plane Detection Manager.
Add the ZEDPlaneDetectionManager component to it.

With ZEDPlaneDetectionManager in the scene, you can now click on any surface in the Game view and the plugin will instantly create a GameObject that fits that surface. You can also click the “Detect” button to make a plane representing the floor.

Add an Object

End the scene if it’s still running.
Right-click the Hierarchy and click 3D Object -> Cube to add a cube to the scene.
Move the cube’s Position to 0, 0.5, 2.
Set the cube’s Scale to 0.5, 0.5, 0.5.
Position the ZED at table height and start the scene.

You should now see the cube floating above the floor. If there’s something other than floor under the cube, move the cube and/or the ZED until it’s above the floor.

Note: In the image above, we assigned the crate_diffuse material from the Green Screen sample to the cube’s Mesh Renderer component.

Drop the Object

With the scene still running, select the Plane Detection Manager object, and in the Inspector, click “Detect”. The floor should turn blue.
Uncheck “Visible in Game” to hide the plane in the Game view.
Use the Scene view to make sure the cube is floating directly above the floor plane.
Add a RigidBody component to the cube.

The RigidBody component adds gravity. When you add it, it’ll fall down and land on your real-world floor.

Adding Click-to-Place

Unity treats the floor plane you just made like any other mesh with a collider, so there’s many ways you could place an object on it. You can see sophisticated examples in the Plane Detection samples in the plugin.

For now, we’ll add a simple script that teleports the cube to be just above a point you click on.

Stop the scene.
Uncheck “Visible in Game” again.
Add the RigidBody component to the cube again.
Create a new script called “PlaceCube” and add it to the cube.
Open the script and replace the default, empty Update() function with the code below (or find the whole script here). That code will move the cube to one meter above the real-world position of where you clicked. Since clicking on the scene also adds a plane if one is present, the cube will fall and land on the surface you click on.

    void Update ()
    {
		if(Input.GetMouseButtonDown(0))
        {
            Vector2 screenpos = new Vector2(Input.mousePosition.x, Input.mousePosition.y);
            Vector3 worldpos;
            if (ZEDSupportFunctions.GetWorldPositionAtPixel(ZEDManager.GetInstance(sl.ZED_CAMERA_ID.CAMERA_ID_01).zedCamera, screenpos, 
                ZEDManager.GetInstance(sl.ZED_CAMERA_ID.CAMERA_ID_01).GetLeftCameraTransform().GetComponent<Camera>(), out worldpos))
            {
                GetComponent<Rigidbody>().velocity = Vector3.zero;
                transform.position = worldpos + Vector3.up;
            }
        }
	}

Run the scene, and click on flat, horizontal surfaces to see this in action.

Note: If the cube seems to “bounce off” the floor at a strange angle, make sure the floor is well-lit, so the ZED recognizes its surface properly. If using the original ZED and not the ZED Mini, you’ll also need to start the scene with the ZED looking parallel to the floor.