Measuring Operator Performance with Telerobotic Perceptual Enhancements

Roger Browse, Marcia McDonald and Scott Little

Abstract

Teleoperational robots (telerobots) utilize human perception and judgement in the control of remote manipulators. The tasks for which this type of manipulation is required are typically too hazardous, inaccessible, or of the wrong scale for direct human action. These same task characteristics often introduce perceptual degradation (such as undersea operation) or a need for unusually careful handling (such as in space, or in nuclear reactors). Thus telerobotic situations often entail higher demands for detailed and accurate perceptual information than is available through a direct video feed from the remote site.

Research efforts in the development of perceptual enhancements for telerobotics have been directed towards the provision of depth cues through both stereo and motion parallax, real-time graphic display simulation, force feedback and integration of contact and proximity sensed data. Each of these enhancements can involve expensive equipment and long development and refining stages. It is therefore important to carry out empirical measurements of the actual performance gain that is associated with these enhancements, in order to first of all determine if they are actually justified, and secondly to discover the most effective means of presenting the information to the operator.

Within telerobotic research there is a prevalent approach which we refer to as the telepresence assumption. Briefly, this assumption holds that any perceptual enhancement that leads to a reduction in the operator's sense of being remote to the manipulator site will therefore increase performance. Few studies have been carried out which shed light on this assumption. Within psychological research, Beiderman has shown that human subjects are no faster to recognize objects in full colour photographs than they are to recognize the same objects from outline drawings. This paper describes some of the perceptual enhancements that we have been developing for telerobotics, with emphasis on the experimentation we have carried out to test the performance increases that accompany the enhancements.

In the Robotics and Perception Laboratory at Queen's University, we are developing an experimental telerobotic system which includes force reflection, graphic simulation, integration of array force sensors, and computational perception support. The robot is a CRS-Plus M1A5 axis small industrial robot with a parallel axis servo gripper.

A LORD LTS-210 array force sensor is mounted on the gripper, providing force data as the robot grasps objects. We have carried out three formal experiments testing a variety of methods for displaying this tactile information for the operator. The overall conclusions are (1) that human operators are able to integrate visual displays of tactile data into the understanding of the remote workplace, (2) that the tactile information serves only the needs created by the inadequacies of the operator's model of the workspace developed through vision alone, (3) that displays which subjects report as the most "lifelike" and most "informative" are actually inferior to more simple displays in terms of subject performance.

A real-time graphic simulation of the robot has been implemented on a Silicon Graphics 4D-85GT workstation. The simulation has a variety of display modes: wire-frame, solid rendered, transparent, and a variety of operation modes: mouse-based "teach pendant", animation, and "mimic-mode" in which joint angles from the actually operating robot feed the simulation. By setting the viewport parameters of the simulation to values determined by calibrating a video camera at the remote site, then genlocking the simulation output and mixing it with the live video feed, we are able to create a simulation overlay for perceptual enhancement. We have carried out experiments which show that an operator's ability to correctly predict a collision that will occur in the workspace can be increase by 500% if viewpoint control is provided within the simulation. These experiments have also shown no operational difference between the use of wire frame or full shaded graphic rendering.

To summarize, we have determined that significant performance increases can accompany perceptual enhancements in telerobotics, but that continued effort to make displays more lifelike do not necessarily provide continued performance benefits.