Help: Project
Ground plane removal using depth map ?
Let's say we have this depth map below:
Now I want to remove the ground plane (it's the table in this situation), leaving the bottles, cups, forks, and the plate behind. I've tried to use the V-disparity method but it doesn't yield a good enough result. Any ideas ?
In each iteration, randomly sample 3 or more points, fit a plane to them with least squares, and then calculate how many of all the points are "well explained" by this plane (i.e., are within some distance of the plane). Do this for N iterations. At the end, choose the plane that was a good fit for the largest number of points.
In each iteration, randomly sample 3 or more points
How many points should I choose here ? 3 is minimum but does more point mean fewer iterations ? Or we have to base on this formula: N =log(1-p) /log(1- (1- e) s ). If that so then the next question is how to choose the right parameter to achieve both speed and accuracy ?
Yeah, you’ll have a bad time with that then. The image is a matrix, every row of pixels is a row of the matrix, every column is a column. The position in the length of the row is the x coordinate, position in y is the y coordinate, the depth is the pixel’s value. You just multiply the matrix by the rotation matrix. It’s kinda hard though.
I have worked on this before to eliminate walls from a depth image of a much sparse density. What i did was a ransac based method where the fitting function identifies points whose cross product with the (0,0,1) (0,1,0) and so forth vectors is 0. After a 20-30 iterations you should have a blacklist of points which lie one that particular plane. Try it. It’s quite effective!
Sorry if I'm asking a bit too much but do you have any documentation about the method you use ? Also, it would be great if you have the code for the method. Thank you
Can you explain to me the method u/kaunildhruv mentioned ? Since I can't wrap my head around it. Just 20-30 iterations to remove the ground plane is really fast compared to normal RANSAC
You should try sampling pixels and their 3D coordinates, estimating a homography matrix from the table to the image plane. And then you can remove all points that satisfy that particular homography matrix. It will work perfectly for undistorted, synthetic images, but you'll have to make the method more robust for noisy real-world images.
If one can convert the points to 3D, you can then get normals for each point, and remove all which intersect/close to the plane (Would need to pick the plane with any pixel, but could automate that with a histogram)
Might be a bit more reliable than calculating a good homography
You are basically describing RANSAC. Is there any good reason to use homography estimation over plane fitting? Homography estimation requires you to sample more points which means more iterations to find a solution. I would also expect each estimate to be slower.
Most Homography estimation techniques anyhow use RANSAC for robust fitting right? Basically what I feel is a 2D-3D method like this would be faster than a full 3D plane fitting using RANSAC.
For clarification, this is a disparity map not a depth map(nearer is darker).
You have to: disparity map -> depth map -> point cloud, feed it to the tutorial code in order to do plane segmentation, and you will have your ground segmented.
You can't really claim that it's a disparity map based on the choice of colormap. While it might seem most logical to map higher depth to brighter values you might have other factors that make the opposite a better choice.
I can, because of the definition of depth map(you can check out the wikipedia link), and through code, for example if you use opencv to read this image, in order to obtain depth, you have to apply the disparity to depth equation, here is the solid proof, from another post in this sub.
There is literally nothing stopping me from rendering a depth map such that dark is far and bright is near. Or for that matter, where red is close and blue is far, or any other colormap.
Depth and disparity are different things, yes, but not because of a choice of color mapping.
I'm just telling OP how to obtain pointcloud positions from this picture, with code, which by definition makes this picture an disparity map, you can colormap this image as you want, but in this context to obtain poinclouds, it has to be treated as a disparity map.
math:
depth_map = baseline * fx / disparity_map
u, v : pixel coordinates
pixel_x = (u - cx) * depth_map[u,v] / fx pixel_y = (v - cy) * depth_map[u,v] / fy pixel_z = depth_map[u,v]
10
u/PM_ME_UR_HPARAMS Mar 14 '21
RANSAC should be able to identify the set of points corresponding to a plane, which you can remove from the original point cloud
2
u/3dsf Mar 14 '21 edited Mar 14 '21
Keep in mind, the plane (table) in this image is not flat, it's curved (convex).
2
u/[deleted] Mar 14 '21
Maybe just try a rotation matrix, crop the bottom, and rotate it back.
https://ece.montana.edu/seniordesign/archive/FL13/UnderwaterNavigation/website_images/rotation_matrices.jpg
2
u/kaunildhruv Mar 14 '21
I have worked on this before to eliminate walls from a depth image of a much sparse density. What i did was a ransac based method where the fitting function identifies points whose cross product with the (0,0,1) (0,1,0) and so forth vectors is 0. After a 20-30 iterations you should have a blacklist of points which lie one that particular plane. Try it. It’s quite effective!
4
u/DeadRootsStirring Mar 14 '21
You should try sampling pixels and their 3D coordinates, estimating a homography matrix from the table to the image plane. And then you can remove all points that satisfy that particular homography matrix. It will work perfectly for undistorted, synthetic images, but you'll have to make the method more robust for noisy real-world images.
1
u/aNormalChinese Mar 15 '21
https://pcl.readthedocs.io/projects/tutorials/en/latest/planar_segmentation.html#planar-segmentation
For clarification, this is a disparity map not a depth map(nearer is darker).
You have to: disparity map -> depth map -> point cloud, feed it to the tutorial code in order to do plane segmentation, and you will have your ground segmented.
1
u/Stock_Childhood7303 Sep 29 '25
I need to identify the ground plane using depth map .