YOLO

YOLO – definition

YOLO, short for You Only Look Once, is a family of single-stage object detection models. A YOLO model processes an image in one inference pass and returns object classes, confidence scores, and bounding boxes. The original YOLO paper was published at CVPR 2016 by Joseph Redmon et al. Subsequent implementations, including YOLOv3, YOLOv5, YOLOv8, and other independently maintained variants, differ in architecture, training procedure, licence, and runtime interfaces.

In mobile robotics, YOLO is normally used as a perception component rather than a navigation system. On a UGV, it can identify classes relevant to the task, such as people, pallets, traffic cones, crop rows, tools, or construction markers. A detection alone does not provide metric distance, obstacle geometry, localisation, or collision-free motion planning. Those functions require sensor calibration, depth or ranging data, coordinate transforms, and integration with the robot navigation stack.

For Leo Rover and Raph Rover, YOLO can run on an onboard computer or on a separate edge computer connected to the robot network. Leo Rover uses a Raspberry Pi-based onboard computer; ROS 2 support depends on the installed operating-system image and software configuration. For real-time inference workloads, the selected model, image resolution, accelerator availability, thermal limits, and required detection rate must be evaluated on the deployed hardware.

How YOLO object detection works

A YOLO detector receives an image tensor and predicts detections directly from learned image features. Unlike two-stage detectors, it does not first generate a separate set of region proposals. The output is filtered using a confidence threshold and usually non-maximum suppression (NMS), which removes overlapping boxes representing the same object.

A detected object is commonly represented by the bounding-box centre, width, height, class label, and confidence score. The basic image-space box can be written as:

b = (x_center, y_center, width, height)

All values above refer to pixels unless the implementation explicitly normalises them. A 2D bounding box is not a 3D object position. To estimate a position in the robot frame, the system needs camera intrinsics, a depth source or multi-view geometry, and a valid transform chain between the camera and robot frames.

YOLO integration in ROS 2

ROS 2 does not define a mandatory YOLO package or a single standard inference node. A deployment commonly consists of a camera driver, an inference node, a visualisation node, and an application-specific consumer. The camera driver publishes images using sensor_msgs/msg/Image and calibration data using sensor_msgs/msg/CameraInfo.

Detection results can be published with messages from the vision_msgs package, including vision_msgs/msg/Detection2DArray. This provides a more interoperable interface than a custom message because it carries an array of detections, hypotheses, class identifiers, scores, and 2D bounding boxes.

ROS 2 interface	Typical purpose	Integration requirement
`sensor_msgs/msg/Image`	Colour, monochrome, or depth image input	Preserve the image header timestamp and frame ID.
`sensor_msgs/msg/CameraInfo`	Camera intrinsic calibration and distortion model	Required for pixel-to-ray or pixel-to-3D projection.
`vision_msgs/msg/Detection2DArray`	2D object detections	Use the source image timestamp for temporal alignment.
`tf2`	Coordinate-frame transforms	Provide a transform from the camera optical frame to `base_link`.

ROS coordinate-frame conventions should follow REP 103. In particular, camera optical frames use the convention where the positive z-axis points forward, the positive x-axis points right, and the positive y-axis points down. A static camera mount should be described in URDF and published through robot_state_publisher or a static transform publisher.

ros2 topic echo /detections vision_msgs/msg/Detection2DArray
ros2 run tf2_ros tf2_echo base_link camera_color_optical_frame

From 2D detections to UGV-relevant information

For autonomous mobile robotics, a useful detection must be associated with space and time. A practical pipeline synchronises the RGB image with a depth image, stereo point cloud, LiDAR cluster, or another ranging source. The system can then project a selected pixel or segmented object region into a camera-frame point and transform that point into base_link, odom, or map.

The following checks are required before using YOLO output in robot behaviour logic.

Verify camera intrinsics and the image resolution used during calibration.
Verify the camera-to-robot extrinsic transform and frame naming convention.
Account for inference latency when matching detections to odometry or depth data.
Define confidence thresholds per class using recorded task data, not default values alone.
Keep safety-critical obstacle avoidance independent of a single semantic detector.

Use with Leo Rover and Raph Rover

On Leo Rover, YOLO is suitable for research and educational tasks such as person-aware teleoperation, marker detection, object counting, or semantic annotations of recorded ROS 2 bags. The platform has differential drive and four wheels. A detection can therefore support high-level behaviour, but it does not replace the localisation, costmap, controller, and recovery components required for autonomous navigation.

Raph Rover can carry larger sensor and compute payloads where an application requires several cameras, a LiDAR, or a dedicated GPU computer. This is relevant for outdoor inspection, agricultural monitoring, and construction-site data collection. The payload design must still consider sensor field of view, vibration, environmental protection, power distribution, network bandwidth, and transform calibration.

Limitations and validation criteria

YOLO performance depends on the training dataset and the operational domain. A model trained on public road scenes may not reliably detect objects in a laboratory, greenhouse, quarry, or low-light inspection route. Class names also do not guarantee task-level semantics. For example, detecting a “person” does not determine intent, direction of movement, or a safe stopping distance.

Model evaluation should separate detection quality from robotic-system quality. Useful detection metrics include precision, recall, mean average precision (mAP), false-positive rate, and per-class confusion matrices. System validation should additionally measure end-to-end latency, timestamp consistency, depth association error, transform correctness, and behaviour under motion blur, occlusion, glare, dust, and lighting changes.

Normative references and primary sources

YOLO itself is not a ROS standard. Its implementation and model format must be documented separately from the ROS interfaces used to transport sensor and detection data.