Holds Fusion process data and functions. More...

Functions
	Fusion ()
	Default constructor. More...

	~Fusion ()
	Default destructor. More...

FUSION_ERROR_CODE	init (sl::InitFusionParameters init_parameters=InitFusionParameters())
	Initialize the fusion module with the requested parameters. More...

void	close ()
	Will deactivate all the fusion modules and free internal data. More...

FUSION_ERROR_CODE	subscribe (const sl::FusionConfiguration &fusion_config)
	Set the specified camera as a data provider. More...

FUSION_ERROR_CODE	subscribe (CameraIdentifier uuid, CommunicationParameters param=CommunicationParameters(), sl::Transform pose=sl::Transform(), bool override_gravity=false)
	Set the specified camera as a data provider. More...

FUSION_ERROR_CODE	unsubscribe (CameraIdentifier uuid)
	Remove the specified camera from data provider. More...

FUSION_ERROR_CODE	updatePose (CameraIdentifier uuid, sl::Transform pose)
	Updates the specified camera position inside fusion WORLD. More...

FUSION_ERROR_CODE	getProcessMetrics (FusionMetrics &metrics)
	Get the metrics of the Fusion process, for the fused data as well as individual camera provider data. More...

std::map< CameraIdentifier, SENDER_ERROR_CODE >	getSenderState ()
	Returns the state of each connected data senders. More...

FUSION_ERROR_CODE	process ()
	Runs the main function of the Fusion, this trigger the retrieve and synchronization of all connected senders and updates the enabled modules. More...

FUSION_ERROR_CODE	retrieveImage (sl::Mat &mat, CameraIdentifier uuid, sl::Resolution resolution=sl::Resolution(0, 0))
	Returns the current sl::VIEW::LEFT of the specified camera, the data is synchronized. More...

FUSION_ERROR_CODE	retrieveMeasure (sl::Mat &mat, CameraIdentifier uuid, sl::MEASURE measure=sl::MEASURE::DEPTH, sl::Resolution resolution=sl::Resolution(0, 0))
	Returns the current measure of the specified camera, the data is synchronized. More...

FUSION_ERROR_CODE	enableBodyTracking (const BodyTrackingFusionParameters &params=BodyTrackingFusionParameters())
	Enables the body tracking fusion module. More...

FUSION_ERROR_CODE	retrieveBodies (sl::Bodies &objs, BodyTrackingFusionRuntimeParameters parameters=BodyTrackingFusionRuntimeParameters(), CameraIdentifier uuid=CameraIdentifier())
	Retrieves the body data, can be the fused data (default), or the raw data provided by a specific sender. More...

void	disableBodyTracking ()
	Disable the body fusion tracking module. More...

FUSION_ERROR_CODE	enableObjectDetection (const ObjectDetectionFusionParameters &params=ObjectDetectionFusionParameters())
	Enables the object detection fusion module. More...

FUSION_ERROR_CODE	retrieveObjects (std::unordered_map< sl::String, sl::Objects > &objs)
	Retrieves all the fused objects data. More...

FUSION_ERROR_CODE	retrieveObjects (sl::Objects &objs, const sl::String &fused_od_group_name)
	Retrieves the fused objects of a given fused OD group. More...

FUSION_ERROR_CODE	retrieveObjects (std::unordered_map< unsigned int, sl::Objects > &objs, const CameraIdentifier &uuid)
	Retrieves all the raw objects data provided by a specific sender. More...

FUSION_ERROR_CODE	retrieveObjects (sl::Objects &objs, const CameraIdentifier &uuid, const unsigned int instance_id)
	Retrieves the raw objects data provided by a specific sender and a specific instance id. More...

void	disableObjectDetection ()
	Disable the object detection fusion module. More...

FUSION_ERROR_CODE	enablePositionalTracking (PositionalTrackingFusionParameters parameters=PositionalTrackingFusionParameters())
	Enables positional tracking fusion module. More...

POSITIONAL_TRACKING_STATE	getPosition (Pose &camera_pose, sl::REFERENCE_FRAME reference_frame=REFERENCE_FRAME::WORLD, CameraIdentifier uuid=CameraIdentifier(), POSITION_TYPE position_type=POSITION_TYPE::FUSION)
	Get the Fused Position referenced to the first camera subscribed. If uuid is specified then project position on the referenced camera. More...

FusedPositionalTrackingStatus	getFusedPositionalTrackingStatus ()
	Get the current status of fused position. More...

FUSION_ERROR_CODE	ingestGNSSData (sl::GNSSData gnss_data)
	Ingest GNSS data from an external sensor into the fusion module. More...

POSITIONAL_TRACKING_STATE	getCurrentGNSSData (sl::GNSSData &out)
	Returns the last synchronized gnss data. More...

GNSS_FUSION_STATUS	getGeoPose (sl::GeoPose &pose)
	Returns the current GeoPose. More...

GNSS_FUSION_STATUS	Geo2Camera (sl::LatLng &in, sl::Pose &out)
	Convert latitude / longitude into position in sl::Fusion coordinate system. More...

GNSS_FUSION_STATUS	Camera2Geo (sl::Pose const &in, sl::GeoPose &out)
	Convert a position in sl::Fusion coordinate system in global world coordinate. More...

void	disablePositionalTracking ()
	Disable the fusion positional tracking module. More...

sl::Timestamp	getCurrentTimeStamp ()
	Return the current fusion timestamp, aligned with the synchronized GNSS and camera data. More...

GNSS_FUSION_STATUS	getCurrentGNSSCalibrationSTD (float &yaw_std, sl::float3 &position_std)
	Get the current calibration uncertainty obtained during calibration process. More...

sl::Transform	getGeoTrackingCalibration ()
	Get the calibration found between VIO and GNSS. More...

FUSION_ERROR_CODE	enableSpatialMapping (SpatialMappingFusionParameters parameters=SpatialMappingFusionParameters())
	Initializes and starts the spatial mapping processes. More...

void	requestSpatialMapAsync ()
	Starts the spatial map generation process in a non blocking thread from the spatial mapping process. More...

FUSION_ERROR_CODE	getSpatialMapRequestStatusAsync ()
	Returns the spatial map generation status. This status allows to know if the mesh can be retrieved by calling retrieveSpatialMapAsync. More...

FUSION_ERROR_CODE	retrieveSpatialMapAsync (Mesh &mesh)
	Retrieves the current generated spatial map only if SpatialMappingParameters::map_type was set as SPATIAL_MAP_TYPE::MESH. More...

FUSION_ERROR_CODE	retrieveSpatialMapAsync (FusedPointCloud &fpc)
	Retrieves the current generated spatial map only if SpatialMappingParameters::map_type was set as SPATIAL_MAP_TYPE::FUSED_POINT_CLOUD. After calling requestSpatialMapAsync , this function allows you to retrieve the generated fused point cloud. The fused point cloud will only be available when getMeshRequestStatusAsync() returns SUCCESS. More...

void	disableSpatialMapping ()
	Disables the spatial mapping process. More...

Detailed Description

Holds Fusion process data and functions.

Constructor and Destructor

◆ Fusion()

Fusion ( )

Default constructor.

◆ ~Fusion()

~Fusion ( )

Default destructor.

Functions

◆ init()

FUSION_ERROR_CODE init ( sl::InitFusionParameters init_parameters = InitFusionParameters() )

Initialize the fusion module with the requested parameters.

Parameters

init_parameters Initialization parameters.

Returns: SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

◆ close()

void close ( )

Will deactivate all the fusion modules and free internal data.

◆ subscribe() [1/2]

FUSION_ERROR_CODE subscribe ( const sl::FusionConfiguration & fusion_config )

Set the specified camera as a data provider.

Parameters

uuid	The requested camera identifier.
param	The communication parameters to connect to the camera.
pose	The World position of the camera, regarding the other camera of the setup.
override_gravity	: Indicates the behavior of the fusion with respect to given calibration pose. If true : The calibration pose directly specifies the camera's absolute pose relative to a global reference frame. If false : The calibration pose (Pose_rel) is defined relative to the camera's IMU rotational pose. To determine the true absolute position, the Fusion process will compute Pose_abs = Pose_rel * Rot_IMU_camera. set to false by default

Returns: SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

◆ subscribe() [2/2]

FUSION_ERROR_CODE subscribe	(	CameraIdentifier	uuid,
		CommunicationParameters	param = `CommunicationParameters()`,
		sl::Transform	pose = `sl::Transform()`,
		bool	override_gravity = `false`
	)

Set the specified camera as a data provider.

Parameters

uuid	The requested camera identifier.
param	The communication parameters to connect to the camera.
pose	The World position of the camera, regarding the other camera of the setup.
override_gravity	: Indicates the behavior of the fusion with respect to given calibration pose. If true : The calibration pose directly specifies the camera's absolute pose relative to a global reference frame. If false : The calibration pose (Pose_rel) is defined relative to the camera's IMU rotational pose. To determine the true absolute position, the Fusion process will compute Pose_abs = Pose_rel * Rot_IMU_camera. set to false by default

Warning: deprecated, FUSION_ERROR_CODE Fusion::subscribe(const sl::FusionConfiguration &fusion_config) should be used instead

Returns: SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

◆ unsubscribe()

FUSION_ERROR_CODE unsubscribe ( CameraIdentifier uuid )

Remove the specified camera from data provider.

Parameters

uuid	The requested camera identifier..

Returns: SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

◆ updatePose()

FUSION_ERROR_CODE updatePose	(	CameraIdentifier	uuid,
		sl::Transform	pose
	)

Updates the specified camera position inside fusion WORLD.

Parameters

uuid	The requested camera identifier.
pose	The World position of the camera, regarding the other camera of the setup.

Returns: SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

◆ getProcessMetrics()

FUSION_ERROR_CODE getProcessMetrics ( FusionMetrics & metrics )

Get the metrics of the Fusion process, for the fused data as well as individual camera provider data.

Parameters

metrics The process metrics.

Returns: SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

◆ getSenderState()

std::map<CameraIdentifier, SENDER_ERROR_CODE> getSenderState ( )

Returns the state of each connected data senders.

Returns: The individual state of each connected senders.

◆ process()

FUSION_ERROR_CODE process ( )

Runs the main function of the Fusion, this trigger the retrieve and synchronization of all connected senders and updates the enabled modules.

Returns: SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

◆ retrieveImage()

FUSION_ERROR_CODE retrieveImage	(	sl::Mat &	mat,
		CameraIdentifier	uuid,
		sl::Resolution	resolution = `sl::Resolution(0, 0)`
	)

Returns the current sl::VIEW::LEFT of the specified camera, the data is synchronized.

Parameters

mat	the CPU BGRA image of the requested camera.
uuid	the requested camera identifier.
resolution	the requested resolution of the output image, can be lower or equal (default) to the original image resolution.

Returns: SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

Note: Only the Left BGRA image is available.

Warning: This feature is not available for network cameras.

◆ retrieveMeasure()

FUSION_ERROR_CODE retrieveMeasure	(	sl::Mat &	mat,
		CameraIdentifier	uuid,
		sl::MEASURE	measure = `sl::MEASURE::DEPTH`,
		sl::Resolution	resolution = `sl::Resolution(0, 0)`
	)

Returns the current measure of the specified camera, the data is synchronized.

Parameters

mat	the CPU data of the requested camera.
uuid	the requested camera identifier.
measure	the requested measure type, by default DEPTH (F32_C1)
resolution	the requested resolution of the output image, can be lower or equal (default) to the original image resolution.

Returns: SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

Note: Only MEASURE: DEPTH, XYZ, XYZRGBA, XYZBGRA, XYZARGB, XYZABGR, DEPTH_U16_MM are available.

Warning: This feature is not available for network cameras.

◆ enableBodyTracking()

FUSION_ERROR_CODE enableBodyTracking ( const BodyTrackingFusionParameters & params = BodyTrackingFusionParameters() )

Enables the body tracking fusion module.

Parameters

params Structure containing all specific parameters for body tracking fusion.
For more information, see the BodyTrackingFusionParameters documentation.

Returns: SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

◆ retrieveBodies()

FUSION_ERROR_CODE retrieveBodies	(	sl::Bodies &	objs,
		BodyTrackingFusionRuntimeParameters	parameters = `BodyTrackingFusionRuntimeParameters()`,
		CameraIdentifier	uuid = `CameraIdentifier()`
	)

Retrieves the body data, can be the fused data (default), or the raw data provided by a specific sender.

Parameters

objs	The fused bodies will be saved into this objects.
parameters	Body detection runtime settings, can be changed at each detection.
uuid	If set to a sender serial number (different from 0), this will retrieve the raw data provided by this sender.

Returns: SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

◆ disableBodyTracking()

void disableBodyTracking ( )

Disable the body fusion tracking module.

◆ enableObjectDetection()

FUSION_ERROR_CODE enableObjectDetection ( const ObjectDetectionFusionParameters & params = ObjectDetectionFusionParameters() )

Enables the object detection fusion module.

Parameters

params Structure containing all specific parameters for object detection fusion.
For more information, see the ObjectDetectionFusionParameters documentation.

Returns: SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

◆ retrieveObjects() [1/4]

FUSION_ERROR_CODE retrieveObjects ( std::unordered_map< sl::String, sl::Objects > & objs )

Retrieves all the fused objects data.

Parameters

objs	The fused objects will be saved into this objects.

Returns: SUCCESS if it goes as it should, otherwise it returns a FUSION_ERROR_CODE.

◆ retrieveObjects() [2/4]

FUSION_ERROR_CODE retrieveObjects	(	sl::Objects &	objs,
		const sl::String &	fused_od_group_name
	)

Retrieves the fused objects of a given fused OD group.

Parameters

objs	The fused objects will be saved into this objects.
fused_od_group_name	The name of the fused objects group to retrieve.

Returns: SUCCESS if it goes as it should, otherwise it returns a FUSION_ERROR_CODE.

◆ retrieveObjects() [3/4]

FUSION_ERROR_CODE retrieveObjects	(	std::unordered_map< unsigned int, sl::Objects > &	objs,
		const CameraIdentifier &	uuid
	)

Retrieves all the raw objects data provided by a specific sender.

Parameters

objs	The fused objects will be saved into this objects.
uuid	Retrieve the raw data provided by this sender.

Returns: SUCCESS if it goes as it should, otherwise it returns a FUSION_ERROR_CODE.

◆ retrieveObjects() [4/4]

FUSION_ERROR_CODE retrieveObjects	(	sl::Objects &	objs,
		const CameraIdentifier &	uuid,
		const unsigned int	instance_id
	)

Retrieves the raw objects data provided by a specific sender and a specific instance id.

Parameters

objs	The fused objects will be saved into this objects.
uuid	Retrieve the raw data provided by this sender.
instance_id	Retrieve the objects inferred by the model with this ID only.

Returns: SUCCESS if it goes as it should, otherwise it returns a FUSION_ERROR_CODE.

◆ disableObjectDetection()

void disableObjectDetection ( )

Disable the object detection fusion module.

◆ enablePositionalTracking()

FUSION_ERROR_CODE enablePositionalTracking ( PositionalTrackingFusionParameters parameters = PositionalTrackingFusionParameters() )

Enables positional tracking fusion module.

Parameters

parameters A structure containing all the PositionalTrackingFusionParameters that define positional tracking fusion module.

Returns: SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

◆ getPosition()

POSITIONAL_TRACKING_STATE getPosition	(	Pose &	camera_pose,
		sl::REFERENCE_FRAME	reference_frame = `REFERENCE_FRAME::WORLD`,
		CameraIdentifier	uuid = `CameraIdentifier()`,
		POSITION_TYPE	position_type = `POSITION_TYPE::FUSION`
	)

Get the Fused Position referenced to the first camera subscribed. If uuid is specified then project position on the referenced camera.

Parameters

camera_pose	Will contain the fused position referenced by default in world (world is given by the calibration of the cameras system).
reference_frame	Defines the reference from which you want the pose to be expressed. Default : REFERENCE_FRAME::WORLD.
uuid	If set to a sender serial number (different from 0), this will retrieve position projected on the requested camera if position_type is equal to POSITION_TYPE::FUSION or raw sender position if position_type is equal to POSITION_TYPE::RAW.
position_type	Select if the position should the fused position re-projected in the camera with uuid or if the position should be the raw position (without fusion) of camera with uui.

Returns: POSITIONAL_TRACKING_STATE is the current state of the tracking process.

◆ getFusedPositionalTrackingStatus()

FusedPositionalTrackingStatus getFusedPositionalTrackingStatus ( )

Get the current status of fused position.

Returns: FusedPositionalTrackingStatus is the current status of the tracking process.

◆ ingestGNSSData()

FUSION_ERROR_CODE ingestGNSSData ( sl::GNSSData gnss_data )

Ingest GNSS data from an external sensor into the fusion module.

Parameters

gnss_data The current GNSS data to combine with the current positional tracking data.

Returns: SUCCESS if it goes as it should, otherwise it returns an FUSION_ERROR_CODE.

◆ getCurrentGNSSData()

POSITIONAL_TRACKING_STATE getCurrentGNSSData ( sl::GNSSData & out )

Returns the last synchronized gnss data.

Parameters

out	Last synchronized gnss data.

Returns: POSITIONAL_TRACKING_STATE is the current state of the tracking process.

◆ getGeoPose()

GNSS_FUSION_STATUS getGeoPose ( sl::GeoPose & pose )

Returns the current GeoPose.

Parameters

pose	The current GeoPose.

Returns: GNSS_FUSION_STATUS is the current state of the tracking process.

◆ Geo2Camera()

GNSS_FUSION_STATUS Geo2Camera	(	sl::LatLng &	in,
		sl::Pose &	out
	)

Convert latitude / longitude into position in sl::Fusion coordinate system.

Parameters

in	The latitude / longitude to be converted in sl::Fusion coordinate system.
out	Converted position in sl::Fusion coordinate system.

Returns: GNSS_FUSION_STATUS is the current state of the tracking process.

◆ Camera2Geo()

GNSS_FUSION_STATUS Camera2Geo	(	sl::Pose const &	in,
		sl::GeoPose &	out
	)

Convert a position in sl::Fusion coordinate system in global world coordinate.

Parameters

pose	Position to convert in global world coordinate.
out	Converted position in global world coordinate.

Returns: GNSS_FUSION_STATUS is the current state of the tracking process.

◆ disablePositionalTracking()

void disablePositionalTracking ( )

Disable the fusion positional tracking module.

◆ getCurrentTimeStamp()

sl::Timestamp getCurrentTimeStamp ( )

Return the current fusion timestamp, aligned with the synchronized GNSS and camera data.

Returns: sl::Timestamp current fusion timestamp.

◆ getCurrentGNSSCalibrationSTD()

GNSS_FUSION_STATUS getCurrentGNSSCalibrationSTD	(	float &	yaw_std,
		sl::float3 &	position_std
	)

Get the current calibration uncertainty obtained during calibration process.

Parameters

yaw_std	Output yaw uncertainty in radian.
position_std	Output position uncertainty in meter.

Returns: sl::GNSS_FUSION_STATUS representing current initialisation status.

◆ getGeoTrackingCalibration()

sl::Transform getGeoTrackingCalibration ( )

Get the calibration found between VIO and GNSS.

Returns: sl::Transform is the calibration found between VIO and GNSS during calibration process.

◆ enableSpatialMapping()

FUSION_ERROR_CODE enableSpatialMapping ( SpatialMappingFusionParameters parameters = SpatialMappingFusionParameters() )

Initializes and starts the spatial mapping processes.

The spatial mapping will create a geometric representation of the scene based on both tracking data and 3D point clouds.
The resulting output can be a Mesh or a FusedPointCloud. It can be be obtained by calling retrieveSpatialMapAsync(). Note that retrieveSpatialMapAsync() should be called after requestSpatialMapAsync().

Parameters

spatial_mapping_parameters The structure containing all the specific parameters for the spatial mapping.
Default: a balanced parameter preset between geometric fidelity and output file size. For more information, see the SpatialMappingParameters documentation.

Returns: SUCCESS if everything went fine, ERROR_CODE::FAILURE otherwise.

Note: The tracking (enablePositionalTracking()) needs to be enabled to use the spatial mapping.; Lower SpatialMappingParameters.range_meter and SpatialMappingParameters.resolution_meter for higher performance.

Warning: This fuction is only available for INTRA_PROCESS communication type.

◆ requestSpatialMapAsync()

void requestSpatialMapAsync ( )

Starts the spatial map generation process in a non blocking thread from the spatial mapping process.

The spatial map generation can take a long time depending on the mapping resolution and covered area. This function will trigger the generation of a mesh without blocking the program. You can get info about the current generation using getSpatialMapRequestStatusAsync(), and retrieve the mesh using retrieveSpatialMapAsync(...) .

Note: Only one mesh can be generated at a time. If the previous mesh generation is not over, new calls of the function will be ignored.

◆ getSpatialMapRequestStatusAsync()

FUSION_ERROR_CODE getSpatialMapRequestStatusAsync ( )

Returns the spatial map generation status. This status allows to know if the mesh can be retrieved by calling retrieveSpatialMapAsync.

Returns: SUCCESS if the mesh is ready and not yet retrieved, otherwise ERROR_CODE::FAILURE.

See requestSpatialMapAsync() for an example.

◆ retrieveSpatialMapAsync() [1/2]

FUSION_ERROR_CODE retrieveSpatialMapAsync ( Mesh & mesh )

Retrieves the current generated spatial map only if SpatialMappingParameters::map_type was set as SPATIAL_MAP_TYPE::MESH.

After calling requestSpatialMapAsync , this function allows you to retrieve the generated mesh. The mesh will only be available when getMeshRequestStatusAsync() returns SUCCESS.

Parameters

mesh	: The mesh to be filled with the generated spatial map.

Returns: SUCCESS if the mesh is retrieved, otherwise ERROR_CODE::FAILURE.

Note: This function only updates the necessary chunks and adds the new ones in order to improve update speed.

Warning: You should not modify the mesh between two calls of this function, otherwise it can lead to corrupted mesh.; If the SpatialMappingParameters::map_type has not been setup as SPATIAL_MAP_TYPE::MESH, the object will be empty.
See requestSpatialMapAsync() for an example.

◆ retrieveSpatialMapAsync() [2/2]

FUSION_ERROR_CODE retrieveSpatialMapAsync ( FusedPointCloud & fpc )

Retrieves the current generated spatial map only if SpatialMappingParameters::map_type was set as SPATIAL_MAP_TYPE::FUSED_POINT_CLOUD. After calling requestSpatialMapAsync , this function allows you to retrieve the generated fused point cloud. The fused point cloud will only be available when getMeshRequestStatusAsync() returns SUCCESS.

Parameters

fpc	: The fused point cloud to be filled with the generated spatial map.

Returns: SUCCESS if the fused point cloud is retrieved, otherwise ERROR_CODE::FAILURE.

Note: This function only updates the necessary chunks and adds the new ones in order to improve update speed.

Warning: You should not modify the fused point cloud between two calls of this function, otherwise it can lead to a corrupted fused point cloud.; If the SpatialMappingParameters::map_type has not been setup as SPATIAL_MAP_TYPE::FUSED_POINT_CLOUD, the object will be empty.

See requestSpatialMapAsync() for an example.

◆ disableSpatialMapping()

void disableSpatialMapping ( )

Disables the spatial mapping process.

The spatial mapping is immediately stopped.
If the mapping has been enabled, this function will automatically be called by close().

Note: This function frees the memory allocated for the spatial mapping, consequently, the spatial map cannot be retrieved after this call.