Nerian karmin2 + scenescan pro ros node integration to orb-slam2

Post Reply
pine
Posts: 2
Joined: Thu Aug 01, 2019 1:25 pm

Nerian karmin2 + scenescan pro ros node integration to orb-slam2

Post by pine »

Hey,

I'm trying to get nerian karmin2 + scenescan pro image stream to work with orb-slam2. When I try to run it, I receive message "waiting for images..."

I'm using your nerian_stereo rosnode with default settings (UDP). Nerian_stereo ROS-node is up and running fine. I can see both left and right recrified images when calling the image messages /nerian_stereo/left_image or right_image. Orb-slam2 is also working fine with karmin2 images when I use images that I have collected beforehand. Additionally, orb-slam2 is running well with other cameras I have tried but those are usb3 connected cameras and not with ethernet connection. Just wondering if there is some known tricks to get it running?
k.schauwecker
Posts: 118
Joined: Mon Mar 25, 2019 1:12 pm

Re: Nerian karmin2 + scenescan pro ros node integration to orb-slam2

Post by k.schauwecker »

Unfortunately I don't have experience with using orb-slam. My guess would be that either it is not correctly corrected to a message topic, or some topics don't match. I have had a brief look at the ROS wiki and I see that it requires a depth image, but I couldn't find info on the exact message type.

The disparity map from SceneScan is not really a depth image but an inverse depth image. I would guess that they require a depth image with a floating-point pixel format and our disparity map is encoded in an integer format and conversion requires knowledge of the camera calibration.

So I think that some adapter logic is required for connecting the nodes. This could be a small node/nodelet that converts the message types. The easiest way would be to take the 3D point cloud and copy only the Z channel to a separate image. Alternatively the Reconstruct3D class from our API can be used to convert disparity maps to point-clouds.

I hope that helps.

Regards,
Konstantin
pine
Posts: 2
Joined: Thu Aug 01, 2019 1:25 pm

Re: Nerian karmin2 + scenescan pro ros node integration to orb-slam2

Post by pine »

Thank you for your reply! Indeed when using orb-slam2 with RGB-D mode it requires depth images. However, I'm running stereo mode and it only requires recrified stereo images and those I'm getting from scenescan pro. I end up reinstalling everything and now it is running correctly. It was probably related to some incorrect ros installation.

I might try RGB-D mode later on and your tips were very valuable for that, thank you.
Post Reply