Visual odometry¶
The visual odometry module is a base module which is available on every rc_visard.
Visual odometry is part of the sensor dynamics module. It is used to estimate the camera’s motion from the motion of characteristic image points (so-called image features) in left camera images. Image features are computed from image corners, which are image regions with high intensity gradients. Image features are used to look for matches between subsequent images to find correspondences. Their 3D coordinates are computed by stereo matching (independent from the disparity image). The camera’s motion is computed from a set of corresponding 3D points between two images. To increase the robustness of visual odometry, correspondences are not only computed to the previous camera image but to a certain number of previous images, which are called keyframes. The best result is then chosen.
The visual-odometry frame rate is independent of the user setting in the stereo camera module. It is internally limited to 12 Hz but can be lower, depending on the number of features and keyframes. To ensure good pose-estimation quality, the frame rate should not drop significantly under 10 Hz.
Note
Using Stereo matching in parallel to the dynamics module may lead to a decreased frame rate of the visual odometry. In this case, we recommend to decrease the frame rate of the Camera (effectively decreasing the frame rate of the depth image computation), to lower the computational load of stereo matching.
The visual odometry module’s measurements are not directly accessible on the rc_visard. Instead, they are internally fused with measurements from the integrated inertial measurement unit to increase robustness and frequency and reduce latency. The result of the sensor data fusion is provided in the form of different streams (see Stereo INS).
Parameters¶
The visual odometry software module is called rc_stereovisodo and it is represented by the
Dynamics page in the Web GUI.
The user can change the visual odometry parameters there, or use the REST-API
(REST-API interface).
Parameter overview¶
This module offers the following run-time parameters:
| Name | Type | Min | Max | Default | Description |
|---|---|---|---|---|---|
disprange |
int32 | 32 | 512 | 256 | Disparity range in pixels |
ncorner |
int32 | 50 | 4000 | 500 | Number of corners |
nfeature |
int32 | 50 | 4000 | 300 | Number of features |
nkey |
int32 | 1 | 4 | 4 | Number of keyframes |
Description of run-time parameters¶
Run-time parameters influence the number of features used to compute visual odometry. More features increase the visual odometry’s robustness at the expense of more run time, which can reduce the frame rate. Although the resulting state estimate will always have a high frequency due to fusion with IMU measurements, high visual-odometry frame rates are nevertheless desirable, since these measurements are much more accurate than IMU measurements alone. A visual-odometry rate of at least 10 Hz should thus be aimed for. The visual-odometry frame rate is provided as a status parameter and is shown below the camera image on the Web GUI’s Dynamics page.
Fig. 24 The Web GUI’s Dynamics page
The camera image shown on this page depicts image features as small green dots. The bold green dots are the features in the current image for which correspondences could be found in a previous keyframe. Green lines depict the motion of these features relative to the previous keyframe. This visualization should help to find a good set of parameters for visual odometry. The number of correspondences is reported as a status parameter and is shown below the camera image on the Web GUI’s Dynamics page. For robust visual-odometry measurements, the parameters should be adjusted so that the resulting number of correspondences in the target environment is around at least 50 when the sensor is moving. The correspondence count will be larger when the rc_visard is static, and the number will change when the rc_visard moves through the environment. Short failures of the visual odometry are tolerated due to the fusion with IMU measurements. Longer failures should be avoided because they lead to large pose uncertainties and can lead to errors in the state estimation.
Each run-time parameter is represented by a row on the Web GUI’s Dynamics page. The name of the row is given in brackets behind the parameter name, and the parameters are listed in the order they appear in the Web GUI:
disprange (Disparity Range)¶
The disparity range gives the maximum disparity value for each image feature related to the resolution of the high-quality disparity image (half image resolution). The disparity range determines the minimum working distance of the visual odometry. When the disparity range is narrow, only more distant features are considered in the visual-odometry estimation. When choosing a broader disparity range, close features can also be used. Broader disparity ranges increase processing time, which can reduce the visual odometry’s frame rate.
Via the REST-API, this parameter can be set as follows.PUT http://<host>/api/v2/pipelines/0/nodes/rc_stereovisodo/parameters?disprange=<value>PUT http://<host>/api/v1/nodes/rc_stereovisodo/parameters?disprange=<value>
nkey (Number of Keyframes)¶
More keyframes can increase the robustness and accuracy of the visual odometry, but they also increase processing time and can decrease the visual-odometry frame rate.
Via the REST-API, this parameter can be set as follows.PUT http://<host>/api/v2/pipelines/0/nodes/rc_stereovisodo/parameters?nkey=<value>PUT http://<host>/api/v1/nodes/rc_stereovisodo/parameters?nkey=<value>
ncorner (Number of Corners)¶
This value gives the approximate number of corners that will be detected in the left image. Larger numbers make visual odometry more robust and accurate but can lead to lower frame rates of the visual odometry.
Via the REST-API, this parameter can be set as follows.PUT http://<host>/api/v2/pipelines/0/nodes/rc_stereovisodo/parameters?ncorner=<value>PUT http://<host>/api/v1/nodes/rc_stereovisodo/parameters?ncorner=<value>
nfeature (Number of Features)¶
This value is the maximum number of features that will be derived from the corners. It is useful to detect more corners and select the best subset as features. Larger numbers make visual odometry more robust and accurate but can lead to lower visual-odometry frame rates. Fewer features might be computed, depending on the scene and movement. The actual number of features is reported below the camera image on the Web GUI’s Dynamics page.
Via the REST-API, this parameter can be set as follows.PUT http://<host>/api/v2/pipelines/0/nodes/rc_stereovisodo/parameters?nfeature=<value>PUT http://<host>/api/v1/nodes/rc_stereovisodo/parameters?nfeature=<value>
Note
Increasing the number of keyframes, corners, or features will also increase robustness but will require more computation time and may reduce the frame rate, depending on other modules active on the rc_visard. The visual-odometry frame rate should be at least 10 Hz.
Status values¶
This module reports the following status values:
| Name | Description |
|---|---|
corner |
Number of detected corners. This value is shown as Corners below the image preview in the Web GUI. |
correspondences |
Number of correspondences. This value is shown as Correspondences below the image preview in the Web GUI. |
feature |
Number of features. This value is shown as Features below the image preview in the Web GUI. |
fps |
Frame rate of the visual odometry in Hertz. This value is shown below the image preview as Visual Odometry FPS (Hz) in the Web GUI. |
time_frame |
Processing time in seconds to compute corners and features for each frame |
time_vo |
Processing time in seconds to compute the motion |
Services¶
This module offers no start or stop services itself, because the dynamics module starts and stops it.
The visual odometry module offers the following services for persisting and restoring parameter settings.
reset_defaults¶
Restores and applies the default values for this module’s parameters (“factory reset”).
Details
This service can be called as follows.
PUT http://<host>/api/v2/pipelines/0/nodes/rc_stereovisodo/services/reset_defaultsPUT http://<host>/api/v1/nodes/rc_stereovisodo/services/reset_defaultsThis service has no arguments.The definition for the response with corresponding datatypes is:
{ "name": "reset_defaults", "response": { "return_code": { "message": "string", "value": "int16" } } }