I'm testing RTAB from a few weeks with kinect and lenovo phab2 in office space environment.
I have better result with the tango phone, thanks to the phone odometry.
I have some trouble to align multi session.
I try on a 3 database set. 2 of the DB align well because they start infront of the same QrCode (I fix 8 QrCode around the office wall to create good anchor points).The third one start infront of a Qrcode previously scan in the other scans, and finish infront of the starting point of the 2 others.
I try replay with TORO optimization in RTAB-map standelone, but I have misalignment.
Is there a way to well align scans without start infront of the same area?
I don't well understand of the rtab settings.
Thank you for sharing the data, I could test more RTAB-Map and fix some issues. First, I would recommend to update RTAB-Map on your phone, the databases you recorded are from version 0.13.2. The current version of RTAB-Map on Tango is 0.16.3. This causes this issue, which makes a lot of loop closures to be rejected. To run with RTAB-Map desktop >= 0.16.3, we should then update your databases like explained in the issue (updating covariance of odometry links from 0.000001 to 0.0001):
sqlite3 AxoneQRCODE001.db "update Link Set information_matrix=x'000000000088C340000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000088C340000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000088C340000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000088C340000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000088C340000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000088C340' where type=0"
sqlite3 AxoneQRCODE002.db "update Link Set information_matrix=x'000000000088C340000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000088C340000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000088C340000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000088C340000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000088C340000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000088C340' where type=0"
sqlite3 AxoneQRCODE003.db "update Link Set information_matrix=x'000000000088C340000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000088C340000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000088C340000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000088C340000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000088C340000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000088C340' where type=0"
On step 5 of the Multi-session mapping tutorial, go to Preferences->Graph Optimization->"Reject loop closures if optimization error ratio..." and set the value to 1 (instead of 0.01).
Afterward, I changed other parameters more like the current default ones in latest RTAB-Map Tango version:
"RGBD/LoopCovLimited" is a new parameter I added in this commit. This reduces the problems of GTSAM not able to optimize the graph in some cases, and thus getting better optimizations. "RGBD/OptimizeMaxError=1" is the parameter we set to 1 previously with the GUI. Using "Kp/DetectorStrategy=6" (GFTT/BRIEF) helps to find more loop closures. Lowering "Kp/MaxFeatures" and "Vis/MaxFeatures" reduces a lot time to update the visual word vocabulary. "Rtabmap/TimeThr" to 0 to keep all frames in WM, in order to be able to detect better wrong graph deformations to reject wrong loop closures (see section below about QR Code generating a lot of bad loop closures).
With all those parameters, I can process the 3 databases, then do Tool->Post-processing to detect more loop closures and get what it seems to be a good optimized map:
Comments on QR Code usage
When I tried to process the first time with the parameters included in the database, here is what I got:
The problem was that RTAB-Map wrongly detected a loop closure between very similar QR codes, and since it was between 2 sessions, the loop closure could not be rejected by detecting a huge graph deformation:
I'll suggest to find "anchors" a lot more distinctive. Here are good anchors I've seen in the databases:
I would rather start/end sessions in front of those unique anchors.
Hi, and thank you for the reply.
I have made some other scan, with april tags. It's a lot better than Qrcode for making anchor position.
I still have issues, but not the same. This time, aprilTags are well find for loop closure, for the 3 or 4 first scan of a batch replay.
Then, I got this message at the start of the 5th scan:
Thrown when a linear system is ill-posed. The most common cause for this
error is having underconstrained variables. Mathematically, the system is
underdetermined. See the GTSAM Doxygen documentation at
http://borg.cc.gatech.edu/ on gtsam::IndeterminantLinearSystemException for
Then the new scan didn't find any loop closure with the other previous scan.
I can send you the database if your are interest, with my setting files.
anyway, I update tango phone and standelone version to last version I can found. I will try some more scans and come back soon with more news.
Last question.Is there a way to stream live tango phone data to a windows standelone version of rtab-map, or do i have to install rtab-map on ubuntu and run from there with tango ros streamer.
I got a lot of "Thrown when a linear system is ill-posed." with your previous databases on my first trial too, particularly for the third session. By adding the "RGBD/LoopCovLimited" parameters, it helped to reduce this problem, though not definitively. If you are using the windows binaries and want to try this parameter, download the latest snapshot here. Check the parameter under Preferences->RGB-D SLAM->"Limit covariance of non-neighbor links...".
There is no built-in option to stream Tango data over RTAB-Map standalone on Windows, we should indeed use ROS for that with Tango ROS Streamer.
I update standelone version this morning with V17.7 and replay my test from friday.
I scanned a full train station in 4 scan and I have far better results with the V17.5.
I saw a lot of disable features in this new version. SURF visual word type is unusable for exemple as well as GTSAM graph optimization.
I know the scan condition was not optimal at all (3 first scan on morning, the other one on aftenoon, projected shadow, direct sunlight exposition) but the result was nearly perfect with V17.5,it was impressive.
It is a bug that GTSAM is not in the appveyor's artifact, there is a problem with the appveyor config, as GTSAM should be included. I don't have access to a Windows computer right now, I'll try to fix this as soon as possible. I was using GTSAM in my tests above.
For SIFT/SURF, I also changed the opencv version used in the automated build, now using OpenCV2 instead of OpenCV3.
I'll let you know when an artifact with GTSAM is available.
I find a way to align my scans with the previous version (17.5)
I create a new database with this settings:
ReplayTango03.ini and replay one database at a time,then save to a new database.
When each database are resaved, I batch them in a new database with the same settings.
That's look much better to find loop closure over differents databases.
But I can align all my scans. RTAB crash before the end if I put all scans in 1 batch.
the database is the same as in my previous message except I resave them with the config file in this message.
I fixed the problem with the artifacts, GTSAM is included, as well as SURF/SIFT. I tried your ini file with this version and I can process all databases. However, they don't automatically merge all together and when I try to reopen the database in rtabmap-databaseViewer, GTAM fails to optimize the map. I didn't try the new option I described in that post to avoid this, I'll test it tomorrow.
Note that on my computer, I cannot show the map (Preferences->3D Rendering, uncheck Map column) online as I don't have enough RAM (only 6GB on my windows computer). I should also disable the following setting in Preferences->General Settings (GUI):
-Insert new data received in the GUI cache. Used to show the loop closure image and the 3D Map.
The crash at the end is maybe related to a full memory (how much RAM do you have?). In the ini file, set also "RGBD\LoopClosureReextractFeatures" to false to increase speed of loop closure estimation (that parameter is deprecated, as in newer version descriptors are re-matched on loop closures).
I have 32Go of Ram and gtx 1080. It's probably not that (crash at 24go when I retry). It's generally crash when I use a other graphic software like photoshop or 3ds max.
I have a other question.
imagine 5 scans as my train station.
Scan 1,2,3 and 4 align well in a new database.
scan 2,4 and 5 align well in a new database.
why when I replay this 2 new database in a batch, rtab-map can't found loop closure for exact same pictures?
Is there a way to use the new pose (calculated camera location during replay) of the new databases for help align all the scans?
thanks for all your help, it's very useful.
2) Open the resulting map (merged.db) in rtabmap-databaseViewer, in Graph View you would have:
3) The red circle shows a wrong loop closure that happened between two sessions. To see better which link is the wrong one, enable inter-session colors, zoom in the red circle above and you would see a green link just here (the corresponding images are also shown):
4) Remove that link with the Constraints View, navigate loop closures until you get 4170 <-> 2441, then click reject.
5) Add the following link to merge the maps manually: 4646 <-> 1237
6) At this point, all maps should be linked together. To make the map more straight between two sides of the railroad, click on Edit->Detect More loop closures. Afterwards the two sidewalks should be almost parallel.
Example exporting the cloud at 5 cm voxel:
With paths of the different sessions colored:
EDIT Some other exports:
Note that the curved path at the bottom left seems a big drift from Google Tango's VIO (not sure what happened), which cannot be corrected by RTAB-Map.
EDIT2 I think it is one of the biggest 3D scan I've seen with Google Tango: 4.5 km! Thx!
Hi mathieu. I made some other test by starting my scan where I finish the last one, and good result is way more easy to obtain with the good settings. Thanks for all your help.
We are looking for a other solution, like Kaarta stencil or GEOSLAM ZEB-REVO that can return distance until 100m and align scan by loop closure like RTAB map. But the price is huge (55000$ for the ZEB-Horizon).
Do you know a cheaper device, all in 1, that can make same results, and work this RTAB, or do we have to develop our own devices, like some other on this forum?
Is that hard to develop and make it work as the price is huge for a pre-build device?
If I attach a lidar on my Tango, is there a way to match the lidar scan to the tango post-processed odometry and IR scan?
When scanning large areas, lidar-based scanners are more convenient than tango because of their larger field-of-view and range. You can add Paracosm PX-80 to your list.
RTAB-Map can be also used with lidars, in theory it is possible to reproduce those devices. For example, you could combine a Velodyne and a camera with RTAB-Map, to get a colored point cloud. The Velodyne could be used alone (or with an IMU) to get already some accurate pose estimation (e.g., odometry) and camera could be used for loop closure in very large areas.
If you can synchronize the clock of Tango with an external lidar, you could feed the poses from tango and scans to rtabmap (e.g., possibly using tango ros streamer and publishing transform between VIO ref frame and lidar via TF). RTAB-Map can already create surfaces of the environment using the lidar contained in the database and apply texture from the camera on them. However, there are probably some non-trivial tuning to do to rtabmap to get good results that I cannot think right now.
It is quite recent, but we can indeed use rtabmap without a camera, only a lidar for mapping. In this mode however, visual loop closure detection is disabled but proximity detection can work (to find small loop closures). If we want to add a camera only for coloring point clouds or texturing a resulting mesh from lidar, there are some works to do on exporting dialog (this has never been tried yet).
Yes, 3D lidars like Quanergy/Ouster/Velodyne can be used in rtabmap. In this paper, we evaluated RTAB-Map with a velodyne for autonomous car (KITTI dataset).
Hand-held 3D lidar scanning is still in development. I just recently had my hands on a velodyne PUCK, so some examples may appear in the next months.
Then I listen to the `/Sensor/points` topic to be sure that the lidar and ros are communicating on this particular topic.
`rostopic hz /Sensor/points`
Then I run another ros package, `pointcloud_to_depthimage` and set the cloud parameter to the `/Sensor/points` topic so that rtabmap can get the point cloud from the lidar and display it - this point may be where I'm wrong right now
`rosrun rtabmap_ros pointcloud_to_depthimage cloud:=/Sensor/point _fixed_frame_id:=start_of_service _decimation:=8 _fill_holes_size:=5`
Finally, I launch rtabmap_ros with the `rtabmap.launch` file. It opens rtabmap but the point cloud doesn't appear.
`roslaunch rtabmap_ros rtabmap.launch rtabmap_args:="--delete_db_on_start --Mem/ImagePreDecimation 2 --Mem/ImagePostDecimation 2" visual_odometry:=false odom_frame_id:="start_of_service" frame_id:="device" rgb_topic:=/tango/camera/color_1/image_raw depth_topic:=/image camera_info_topic:=/tango/camera/color_1/camera_info compressed:=true depth_image_transport:=raw approx_sync:=false`
then I try this command:
rosrun tf static_transform_publisher 0 0 0 0 0 0 1 map TestName 10.
After that, I'm able to change map to "TestName" in global option >fixed frame, but same error message.
What did I miss?
my final goal here is to use both lidar and tango, and make a 3D map in rtab-map.
Is there a way to connect tango TF to this second pointCloud2?
How can I transfert all that data to rtab to make a 3D map?
I would try first to use tango ros streamer example "as is" while adding the lidar input. To add the lidar point clouds, we have to pre-sync the tango image topics together. To do that, rtabmap.launch has for convenience an argument called rgbd_sync:
Thank you, that's work. Not as fine as I expected but see a evolution.
I set my lidar like that:
$ rosrun tf static_transform_publisher 0 0 0 1.57079632679 3.14159265359 0 device Lidar 100
When I start a new map, points from lidar and tando are align.
Tango is facing forward, lidar return points on side. There is no corresponding points between the devices.
The 2 devices look completely differents things. Is that a problem?
problem seems to appear on rotation.
when I show only tango points, all looks ok. Lidar points don't always match.
How can I improve correspondence between the 2 devices.
Thx for the db, there is indeed a TF problem between the device frame and the lidar frame. There is no requirement that lidar should overlap tango's cloud, but it would make it easier to debug the transform. In your transform, xyz = "0 0 0", not sure the lidar is exactly on device frame. For example, if the phone is 10 cm over the lidar, it could be something like "0 0 0.1". Another problem could be the time synchronization between the stamps from the lidar and Tango poses. If you move very slowly, does the lidar look less off-sync from Tango?