Synthetic data
Synthetic data – definition
Synthetic data is artificially generated data, typically produced by a computational model rather than directly measured by a physical sensor. In mobile robotics, it usually means simulated sensor outputs and associated ground-truth annotations produced from a virtual environment, robot model, sensor model, and motion scenario. Typical outputs include RGB images, depth images, LiDAR point clouds, IMU measurements, wheel odometry, GNSS positions, semantic masks, object bounding boxes, and robot poses.
For an unmanned ground vehicle (UGV), synthetic data is used to develop, test, and validate perception, mapping, localization, navigation, and sensor-fusion pipelines before collecting data in the field. It does not replace real measurements. A synthetic dataset is only useful when its environment, sensor model, coordinate frames, timestamps, noise characteristics, and robot kinematics are sufficiently aligned with the intended physical system.
In ROS 2, synthetic data is commonly published through the same interfaces used by physical devices. For example, a simulator may publish camera frames as sensor_msgs/msg/Image, point clouds as sensor_msgs/msg/PointCloud2, IMU data as sensor_msgs/msg/Imu, and transforms as geometry_msgs/msg/TransformStamped messages through the tf2 system. This enables a ROS 2 node to be tested against simulated inputs without modifying its application logic.
Synthetic data in UGV development
A synthetic-data pipeline combines a world model with a dynamic robot model and simulated sensors. The world may contain terrain, walls, vegetation, construction materials, reflective surfaces, illumination sources, moving objects, and weather effects. The robot model defines geometry, mass properties, wheel placement, sensor mounting positions, and coordinate frames.
The following dataset components are relevant for a mobile robot operating outdoors or in semi-structured environments:
- Sensor observations – RGB, stereo, depth, thermal, LiDAR, radar, IMU, GNSS, wheel encoder, and odometry streams.
- Ground truth – robot pose, velocity, object pose, object class, depth, segmentation, occupancy state, and collision events as defined by the simulator.
- Temporal information – timestamps, sensor update rates, latency, frame drops, and synchronization offsets.
- Physical parameters – wheel slip, friction, suspension response, sensor noise, rolling-shutter effects, range error, and GNSS uncertainty.
Ground truth is a key distinction between synthetic and manually labelled real-world data. A simulator can provide pixel-level semantic labels or a simulator-defined six-degree-of-freedom pose of an object directly from its scene graph. In a physical dataset, equivalent annotations may require manual labelling, motion-capture infrastructure, RTK measurements, or post-processing.
ROS 2 interfaces and coordinate frames
For ROS 2 integration, synthetic data should follow the same message definitions and frame conventions as the target robot. The ROS message definitions for common sensors are maintained in the sensor_msgs package documentation. Coordinate-frame conventions should follow REP 103, which defines SI units and the standard body-frame orientation, and REP 105, which defines the common map, odom, and base_link frame relationships for mobile robots.
| Simulated source | Typical ROS 2 message | Primary UGV use |
|---|---|---|
| Monocular or RGB-D camera | sensor_msgs/msg/Image, sensor_msgs/msg/CameraInfo |
Detection, segmentation, visual odometry |
| 3D LiDAR | sensor_msgs/msg/PointCloud2 |
Obstacle detection, mapping, localization |
| IMU | sensor_msgs/msg/Imu |
State estimation and sensor fusion |
| Wheel encoders | nav_msgs/msg/Odometry, sensor_msgs/msg/JointState |
Dead reckoning for differential-drive motion |
| GNSS or RTK GNSS | sensor_msgs/msg/NavSatFix |
Global positioning and map alignment |
A simulated transform tree for a mobile robot will often include map, odom, base_link, and sensor frames such as laser_frame or camera_link, where applicable. Incorrect static transforms can make an apparently valid perception dataset unusable for localization or navigation testing.
Simulation, domain randomization, and sim-to-real transfer
Simulation is not automatically representative of real operation. The difference between simulated observations and physical observations is commonly called the sim-to-real gap. It can result from simplified materials, incomplete terrain geometry, inaccurate sensor calibration, unmodelled vibration, lighting changes, motion blur, or wheel slip.
Domain randomization is one method for reducing overfitting to a single virtual scene. Instead of rendering one fixed environment, the generator varies selected parameters across episodes:
- lighting direction, intensity, colour temperature, and shadows;
- surface textures, material reflectance, dirt, and vegetation density;
- camera exposure, noise, blur, and lens distortion;
- LiDAR range noise, missing returns, and reflectivity response;
- terrain slope, obstacle placement, wheel friction, and slip.
The parameter ranges should be documented and justified by real sensor logs or manufacturer specifications. Randomization cannot compensate for a wrong robot model, incorrect sensor extrinsics, or a missing physical effect that is important for the target task.
Use with Leo Rover and Raph Rover
For Leo Rover, synthetic data is useful when developing ROS 2 nodes on a workstation before deployment to the onboard computer. Leo Rover uses a four-wheel differential-drive configuration, so a simulation should represent differential-drive kinematics, wheel odometry, the base_link frame, and any added sensor payload. Leo Rover can be integrated with ROS 2; supported ROS 2 distributions and installation methods depend on the software image and current platform documentation. Autonomous navigation requires integration of a navigation stack and sensors. It is not provided as an autonomous vehicle out of the box.
A practical workflow is to validate a navigation or perception pipeline against simulated topics, record the run using rosbag2, and replay the same data during regression testing. A minimal launch command may use simulated time:
ros2 launch my_ugv_sim bringup.launch.py use_sim_time:=trueFor Raph Rover, synthetic data can support payload-oriented testing where a larger platform carries heavier sensing or compute equipment. The simulated model should include the actual payload position, mass, field of view, and mounting height. These parameters influence camera perspective, LiDAR occlusion, stability assumptions, and the traversability perceived by the robot.
Validation criteria and limitations
Synthetic datasets should be evaluated against recorded physical data from the intended platform and environment. Useful comparisons include sensor timestamp behaviour, image exposure characteristics, point-cloud density, IMU noise distribution, wheel-odometry drift, and localization error. A perception model trained only on synthetic data should be tested on separate real-world sequences before operational use.
Simulation is particularly limited when terrain deformation, dust, rain, direct sunlight, vegetation motion, multipath GNSS effects, and sensor contamination materially affect the task. These effects are difficult to model consistently and may dominate field performance.
Normative references and technical sources
The following references define ROS interfaces and frame conventions used when synthetic sensor data is integrated into a ROS 2 UGV stack.
- ROS REP 103 – Standard Units of Measure and Coordinate Conventions.
- ROS REP 105 – Coordinate Frames for Mobile Platforms.
- ROS 2 sensor_msgs documentation.
- ROS 2 Quality of Service settings documentation.
- ROS 2 rosbag2 recording and playback documentation.