During my stay at Willow Garage this summer, I am focussing on object pose registration from point clouds. In particular, I want to be able to detect, register and track rectangular surfaces that are common in indoor (kitchen) environments, such as cabinet drawers and doors. These pose and shape estimates could then be used to learn articulation models for the interior of a kitchen, as shown in my previous work.

While 6D pose registration is a challenging problem for machine perception in the general case, it becomes (more) feasible when additional assumptions are being made. The stereo cameras of PR2 use structured light to improve the matching process. As a result, they produce highly accurate and dense depth maps. In contrast to 3D laser scanners, the point clouds can be obtained at a normal video frame rates.

We first apply a RANSAC-based approach for segmenting out individual planes in the depth image. Then, for each individual plane, we start at a random start pixel with a rectangle of an initial size, and optimize iteratively its pose and size using a greedy search.

Willow Garage Blog

Video on YouTube

Code API

Package description

To run the demos, change to the demo folder in the planar_objects package. Additionally, you need to download the corresponding bag files. The easiest way to do that is by running:

roscd planar_objects
cd demo
./get_bags.sh

• Demo showing the detection on rectangles on the door dataset

roslauch demo-detector-door.launch

• Demo showing the detection on rectangles on the drawer dataset

roslauch demo-detector-drawer.launch

• Demo showing the tracking on the door dataset

roslauch demo-tracker-door.launch

• Demo showing the tracking on the drawer dataset

roslauch demo-tracker-drawer.launch

• Demo showing the articulation model learner on the door dataset

roslauch demo-tracker-door.launch

• Demo showing the articulation model learner on the drawer dataset

roslauch demo-tracker-drawer.launch

• Demo showing the comparison of the ground truth versus the detected pose

roslaunch demo-phasespace.launch

Note that you can download all available data sets, by running

cd planar_objects
./get_bags.sh

Note also that the message format has changed in revision 24201; get_bags.sh will automatically download the correct bag file, corresponding to your revision.

Manual downloads:

• Drawer dataset
  • drawer-simple-24sept09.bag (20MB) (current)
  • drawer-simple.bag (20MB) (old message format, used for revision 24017)

* Door dataset

• door-simple.bag-24sept09.bag (102MB) (current)
• door-simple.bag (102MB) (old message format, used for revision 24017)

• Ground-truth evaluation (1 of 5 logfiles)
  • phasespace-phi.bag-24sept09.bag (438MB) (current)
  • phasespace-phi.bag (438MB) (old message format, used for revision 24017)

More data is available upon request.