Dorian Tsai, Donald Dansereau, Thierry Peynot and Peter Corke
To be effective, robots will need to reliably operate in scenes with refractive objects in a variety of applications; however, refractive objects can cause many robotic vision algorithms, such as structure from motion, to become unreliable or even fail. We propose a novel method to distinguish between refracted and Lambertian image features using a light field camera.
For previous refracted feature detection methods that are limited to light field cameras with large baselines relative to the refractive object, our method achieves state-of-the-art performance. We extend these capabilities to light field cameras with much smaller baselines than previously considered, where we achieve up to 50% higher refracted feature detection rates. Specifically, we propose to use textural cross-correlation to characterise apparent feature motion in a single light field, and compare this motion to its Lambertian equivalent based on 4D light field geometry.
For structure from motion, we demonstrate that rejecting refracted features using our distinguisher yields lower reprojection error, lower failure rates, and more accurate pose estimates when the robot is approaching refractive objects. Our method is a critical step towards allowing robots to operate in the presence of refractive objects.