Sensor fusion

Sensor fusion – definition

Sensor fusion is the process of combining measurements from multiple sensors into one estimate of a robot state or of the surrounding environment. In mobile robotics, the fused output is usually more accurate, more stable, or more observable than any single sensor stream used alone. For a UGV, this typically means estimating pose, velocity, heading, terrain-relative motion, or obstacle structure from sources such as wheel odometry, IMU, LiDAR, depth cameras, and GNSS.

In ROS and ROS 2 systems, sensor fusion is usually implemented as a state estimation pipeline. The pipeline consumes time-stamped messages, transforms them into a common coordinate frame, models uncertainty, and publishes filtered outputs such as odom -> base_link or map -> odom. This aligns with standard ROS frame conventions defined in REP 103 for units and coordinate conventions and REP 105 for mobile platform frame semantics.

For UGV platforms such as Leo Rover and Raph Rover, sensor fusion is not optional once the robot leaves a controlled indoor environment. Wheel encoders drift on loose soil. IMU heading drifts without correction. GNSS alone may be noisy or unavailable near buildings or under vegetation. LiDAR or vision can lose features in dust, fog, or repetitive corridors. Fusion reduces these failure modes by combining complementary sensing modalities.

How sensor fusion works in mobile robotics

The core idea is estimation under uncertainty. Each sensor provides a partial and noisy observation. The estimator combines them using a motion model and a measurement model. In practice, most UGV deployments use a Kalman-filter family estimator, graph-based optimization, or a tightly coupled inertial-navigation stack.

The most common outputs in ROS 2 are:

nav_msgs/msg/Odometry – filtered pose and twist
geometry_msgs/msg/PoseWithCovarianceStamped – global pose estimate
sensor_msgs/msg/Imu – orientation, angular velocity, linear acceleration
tf2 transforms – especially map, odom, base_link, and sensor frames

For a differential-drive UGV, a simplified state vector often includes planar position and yaw:

x = [p_x, p_y, yaw, v_x, omega_z]^T

The prediction step propagates state using wheel odometry and inertial measurements. The update step corrects that prediction using observations from LiDAR SLAM, visual odometry, or GNSS. In estimator notation:

x_k = f(x_k-1, u_k) + w_k
z_k = h(x_k) + v_k

where w_k and v_k are process and measurement noise terms. Their covariance values directly affect filter behavior.

Common sensor combinations on UGV platforms

Sensor fusion architecture depends on terrain, speed, payload budget, and compute resources. On compact educational robots and larger field robots, the same principles apply, but the sensor set and update rates differ.

Sensor	Typical ROS message	Typical rate	Main contribution	Typical limitation
Wheel encoders	`nav_msgs/msg/Odometry`	10 – 100 Hz	Short-term motion estimate	Slip on gravel, mud, ramps
IMU	`sensor_msgs/msg/Imu`	50 – 400 Hz	Angular rate, attitude dynamics	Bias drift, vibration sensitivity
2D or 3D LiDAR	`sensor_msgs/msg/LaserScan`, `sensor_msgs/msg/PointCloud2`	5 – 20 Hz	Scan matching, obstacle geometry	Poor returns in heavy rain, dust, some surfaces
Depth camera	`sensor_msgs/msg/Image`, `sensor_msgs/msg/PointCloud2`	15 – 90 Hz	Visual odometry, local obstacles	Sunlight sensitivity, limited range
GNSS / RTK	`sensor_msgs/msg/NavSatFix`	1 – 20 Hz	Global position reference	Multipath, canopy blockage

Manufacturer specifications should be used for actual rates, noise density, field of view, and range. For example, Intel RealSense depth cameras, Slamtec LiDARs, and RTK receivers expose different operating modes that must be reflected in estimator tuning.

Sensor fusion in ROS 2

In ROS 2, a common baseline stack uses robot_localization for EKF or UKF-based state estimation. It can fuse odometry, IMU, and GNSS-derived data into a locally smooth odometric estimate and, optionally, a globally referenced estimate. This approach is widely used because it follows ROS frame conventions and integrates cleanly with Nav2.

A typical dual-filter setup separates local and global estimation:

Local EKF: wheel odometry + IMU, publishes odom -> base_link
Global EKF: wheel odometry + IMU + GNSS or SLAM pose, publishes map -> odom
navsat_transform_node: converts GNSS fixes into the world frame using heading and robot orientation

ekf_filter_node:
  ros__parameters:
    frequency: 30.0
    two_d_mode: true
    publish_tf: true
    map_frame: map
    odom_frame: odom
    base_link_frame: base_link
    world_frame: odom

    odom0: /wheel/odometry
    odom0_config: [false, false, false,
                   false, false, true,
                   true,  false, false,
                   false, false, true,
                   false, false, false]

    imu0: /imu/data
    imu0_config: [false, false, false,
                  false, false, true,
                  false, false, false,
                  false, false, true,
                  true,  false, false]
    imu0_remove_gravitational_acceleration: true

Frame consistency matters. REP 105 defines the relationship between base_link, odom, map, and earth-referenced frames. A frequent integration error is mixing local odometry and global pose in one frame without clear ownership of the transform chain.

Use on Leo Rover and Raph Rover

On Leo Rover, sensor fusion usually starts with differential-drive wheel odometry and an IMU. This is the minimum practical setup for stable local motion estimation on ROS 2. Since Leo Rover runs on an embedded Raspberry Pi-based compute unit by default, estimator choice and sensor bandwidth should match limited CPU and memory resources.

For indoor navigation on Leo Rover, a common extension is 2D LiDAR plus IMU plus wheel odometry. LiDAR scan matching or SLAM provides drift correction beyond pure dead reckoning. For outdoor work, GNSS or RTK can be added, but only after careful antenna placement, time synchronization, and frame calibration.

Raph Rover is better suited to heavier sensor payloads and field-grade positioning stacks. That makes it a more natural platform for combinations such as:

industrial IMU + wheel odometry + RTK GNSS
3D LiDAR + IMU for localization in rough terrain
stereo or depth sensing + LiDAR for obstacle perception

On both platforms, fusion quality depends less on the number of sensors than on calibration, timestamps, and covariance tuning.

Key integration requirements

Good fusion results require consistent geometry and timing. Most field failures come from integration errors rather than from the estimator itself.

Extrinsic calibration – fixed transforms between base_link and each sensor must be correct in tf2
Timestamp quality – message timestamps should represent acquisition time, not delayed publish time
Covariance values – estimator tuning must reflect actual sensor noise, not placeholder zeros
Observability – yaw, velocity, and position must be constrained by at least one reliable source
2D vs 3D mode – planar UGV stacks often use two_d_mode to suppress unobservable vertical states

ros2 topic hz /imu/data
ros2 topic echo /wheel/odometry
ros2 run tf2_tools view_frames
ros2 topic echo /odometry/filtered

Limitations and trade-offs

Sensor fusion does not remove all error. It redistributes uncertainty based on models and measurements. If all inputs are biased, the fused estimate will also be biased. If wheel slip is severe and GNSS is intermittent, the filter may become overconfident unless covariance is tuned conservatively.

There is also a trade-off between computational simplicity and estimator fidelity. EKF-based fusion is efficient and sufficient for many UGV tasks. Tightly coupled visual-inertial or LiDAR-inertial methods can be more accurate, but they require better calibration, more compute, and sometimes GPU-class hardware.

Normative references and standards

The ROS coordinate and frame semantics relevant to sensor fusion are defined in official ROS Enhancement Proposals. These documents should be treated as baseline references during system design.

REP 103 – Standard Units of Measure and Coordinate Conventions
REP 105 – Coordinate Frames for Mobile Platforms
ROS 2 message definitions for sensor_msgs, nav_msgs, and geometry_msgs
tf2 documentation and ROS frame conventions
robot_localization documentation for EKF, UKF, and GNSS integration in ROS

For deployed robots, vendor specifications remain essential. IMU bias stability, LiDAR scan rate, camera depth range, and GNSS update rate should all come from the official datasheet of the specific device.