Affordable 3D cameras for ROS developers

Szymon Dzwończyk, 25.05.2023

Most of the time, when robot developers talk about SLAM, they address localization and mapping with the use of LiDAR and stereo-vision cameras. In this article, we’ll focus on the latter.

How to approach vision-based depth mapping

We didn’t name the article ‘stereo vision cameras for your rovers’ on purpose as there are at least three different approaches to the problem of vision-based depth mapping:

stereo vision cameras,
time-of-flight sensors,
structured light cameras.

With those readings on board, your rover can calculate its position in 3-dimensional workspace knowing not only what obstacles look like, but also how far and how big they are.

Let's just quickly go through differences between the three approaches and then, we'll list different off-the-shelf components that are easy to implement to your ROS-based rovers.

‍

Stereo vision

Stereo vision is the most intuitive concept to understand. It mimics the natural vision perception of humans and many other species. Stereo vision camera is essentially a camera that consists of two sensors (two lenses) located at a certain distance from one another. It relies on a difference between the left and right camera images which will be slightly shifted depending on the distance to the object it is looking at. Although the measurement relies on a high-demanding calculation of the images differences, the stereo-vision cameras tend to be the least sensitive to the object material, background or different light conditions and are great to be used for outdoor localization and mapping.

Structured light

As typically the object you’re looking at is unknown, the biggest issue with stereo vision cameras is that you need to rely on image processing of the object contours and distinctive points to take into the depth calculation. It takes a lot of computational power – just like in nature. On the other hand, the structured light approach relies on projecting a mesh (most commonly infrared light) to the object and looking for a difference between what it should look like and what it looks like in reality. The bigger the mesh cells, the closer the object is, and as the mesh curves on the object surface, we can tell what shape it is as well. Knowing what to look for, we can calculate with less computing power. On the other hand, though, as we rely on projected light reflection, we can’t always be sure how the object’s surface and background light affected the reading.

Time-of-flight

This kind of cameras are widely used in photography and phone cameras. Time-of-flight cameras and sensors rely on a time delay between artificial light release and capture of the light being reflected by an object. As light needs a certain time to travel, the time-of-flight system can calculate distance between camera and the object relying on constant light speed and the delay parameter. The procedure is similar to LiDAR readings, but as the hardware and application are closer to cameras than laser sensors, time-of-flight cameras belong to the list here.

The list of camera-based sensors for your rover

The list is the result of our market research and, hopefully, will be enough for you to choose from. We selected the cameras based on their availability and price and evaded the ones that are so specialized that cost a small fortune to get. For sure you’ll find more interesting cameras on the market, but here we’ll focus on the ones you can actually get and work with.

Stereo-vision cameras

Stereolabs ZED / ZED 2 / ZED Mini

Price: $349 - $449

Type: Embedded stereo

Depth Range: 0.3 to 25 m

FOV: 96° H, 54° V

Physical Dimensions: 175x30x33 mm

Interface: USB 3.0

Link to ROS Driver

e-Con Systems Tara Stereo Camera

Price: $349

Type: Embedded Stereo Camera

FOV: 60° H

Physical Dimensions: 100x30x35 mm

Interface: USB 3.0

Link to ROS Driver

Notes: Inbuilt IMU

Intel® RealSense™ Tracking Camera T265

Price: $199

Type: Stereo camera with tracking hardware and software

Cameras: Two Fisheye lenses with combined 163±5° FOV

Physical Dimensions: 108 mm x 24.5 mm x 12.5 mm

Interface: USB 3.1 Gen 1 Micro B (USB2.0 supported)

Link to ROS Driver

Notes: Integrates wheel odometry

duo3d DUO

Price: $595-$695

Type: Stereo Camera

Depth Range: 0.23 m to 2.5 m for for M series

FOV: 170 W with 30 mm Baseline

Physical Dimensions: 57 x 30.5 x 14.7mm

Interface: 480 Mbps USB 2.0 Micro-B

Link to API

Link to ROS Driver

Notes: Pixel size 6 x 6 micrometers. Shutter Speed 0.3 microseconds to 1- seconds. Control Functions: Exposure, Shutter, Brightness. Enclosure 6021 Aircraft Grade Aluminium.