I have been working on a home build lidar scanner and it works ok, however from session to session it will sometimes report a wall as being 5-10cm behind where the rgbd return is. Will this break rtabmap (mapping and SLAM) or is it smart enough to figure stuff like that out?
At which distance this error happens? Some low-cost rgbd cameras have relatively accurate values just under a couple of meters. rtabmap assumes that rgbd camera and the lidar are properly aligned together. If the distance/rotation between the camera and lidar changes over time, it should be reflected in TF.
Depending on the usage, aligning accurately the lidar with camera is not that a "game breaker". For example, on one of robot we are using mostly lidar with wheel encoders for 2D mapping, and use the camera only for loop closure detection. As loop closure transforms are refined by lidar afterward, a small error of alignment between the camera and lidar will be corrected. The 3D map is however not perfectly aligned with the 2d map created from lidar, but in this case we do planning in 2D map, so it is ok.