Techniques for Combining Tactile and Visual Perception for Robotics

Roger A. Browse and James C. Rodger

Introduction

Recently there has been a growing interest in developing methods for robotic tactile perception which can complement robotic vision. One reason for this interest is that contact sensing does not underconstrain scene interpretation (as does vision), and with proper proprioceptive feedback, touch can provide exactly the sort of absolute distance information that will complement vision best. Another reason for interest in tactile perception is that robots are usually engaged in contacting the manipulating objects in the course of task accomplishment, so that tactile sensing utilizes the robot's inherent capabilities. Finally, it is clear that tactile perception may provide scene information which is not available to visual sensors. Such information includes details of gripper placement, characteristics of visually occluded surfaces, compliance, roughness, and physical resistance.

Human tactile perception is involved in a variety of tasks (see Lederman and Browse, 1988). Thus, in the development of computational tactile systems it is possible to consider any of a number of useful operations. For example, tactile exploration has been studied both as a technique for the development of object representations (Bajcsy and Goldberg, 1984), and as a form of surface inspection. It is also reasonable to address the fundamental aspects of tactile perception in terms of its role in the gripping and manipulation of objects, providing force application information as well as gripper placement (Fearing and Hollerbach, 1984).

One particularly intriguing aspect of human tactile perception is our remarkable ability to recognize objects through touch (Klatzky, Lederman, and Metzger, 1985). It is apparent that humans are capable not only of accurately recognizing objects when permitted to handle them, but-given a set of known objects- humans can perform recognition on the basis of very tactile contacts.

Consider, as an example, the experience that most of us have probably had of entering a darkened room in our home, and being somewhat disoriented. After groping about for a few seconds, contact is made with some object in the room, and suddenly the entire room is instantiated, and a path may be followed in the dark to the light switch. The research reported here is motivated by the need to develop computational methods which can accomplish this form of rapid determination of object identity and placement on the basis of sparse contacts.

In addition to this primary motivation, we believe it is important to have computational systems which meet the further requirements that the underlying methods are such that sensor contacts can be used alone or they can be combined in the recognition process. The methods should also offer the possibility of extension to include integration with some forms of visual input.

The following sections describe the steps that our research has taken towards meeting these objectives. First, we discuss the set of primitive tactile features which are defined for the system, and provide an outline of the methods by which these features are extracted from array force-sensed images. The parameters which define the features are described. Next, we discuss the importance of the use of proprioceptive information in both human and machine tactile perception. Several existing approaches which involve proprioception are described. In the next section, we outline the simulated tactile object recognition system devised to demonstrate the utility of tactile features when combined with proprioception. The system demonstrates that only a few tactile contacts are necessary to identify and locate objects selected from a set of nine models. Finally, we describe the extension of the tactile system to include integration with simple forms of visual input. The control structures which accept and manipulate the object constraints for the tactile contacts are also able to accept similarly formulated constraints which originate in other sensory modalities.