Integrating Tactile and Visual Perception for Robotics

Roger A. Browse and James C. Rodger

Introduction

There has recently been a growing interest in developing methods for robotic tactile perception which can complement robotic vision. One reason for this interest is that contact sensing does not underconstrain scene interpretation (as does vision), and with proper proprioceptive feedback, touch can provide exactly the sort of absolute distance information that will complement vision best. Another reason for interest in tactile perception is that robots are usually engaged in contacting and manipulating objects in the course of task accomplishment, and so tactile sensing in the course of that contact utilizes the robots inherent capabilities. Finally, it is clear that tactile perception may provide scene information which is not available to visual sensors. Such information includes details of gripper placement, characteristics of visually occluded surfaces, compliance, roughness and physical resistance.

Our research into robotic perception has proceeded in the following steps:

• Devise a set of appropriate tactile features, and develop methods for the extraction of these features from array force-sensed images.
• Develop (in simulation) methods by which sparse tactile data may be used to efficiently detect object identity and placement from a set of predetermined objects.
• Extend the object identification methods to permit the use of sensory information from arbitrary sources, and to test the extension in the integration of simple visual data along with the tactile features.