The Effectiveness of Real-Time Graphic Simulation in Telerobotics

Roger A. Browse and Scott A. Little

Abstract

An experimental telerobotic system has been developed which uses graphic simulation support. The operator may select from a variety of display methods including the overlay of the simulation aligned with live video images from the remote site. The robot may also be directly controlled from the simulation, using a six degree-of-freedom hand controller. An experiment has been carried out to determine the effect having perspective control of simulation has upon the performance of a teleoperation task. The experiment also compared the effectiveness of different display methods.

Introduction

The development of useful teleoperational robotic systems (telerobots) requires an effective subsystem for remote perception of the robot's workspace. Such systems in use today usually consist of remote video cameras, perhaps equipped with capabilities to permit the operator to alter the orientation and zoom of the cameras, and it is also common to use a camera fixed to a link of the manipulator. Despite these video sources, it may still be the case that a desired view of the workspace is unavailable or obstructed. Lighting conditions may be inadequate, preventing the operator from attaining a clear understanding of the workspace. The lack of depth cues from fixed position video sources may also hamper effective operation.

As high speed graphics becomes more widely available, the use of graphic-enhanced video images, or complete graphic substitution of video images is gaining favour as a method for enabling telerobotic perception. One approach is to superimpose a grid upon the video image corresponding to the base horizontal plane of the workspace. This grid can then be marked to indicate the position, and even orientation, of objects and items of interest to the telerobotic task. This approach can eliminate some of the problems of poor lighting and limited perspective, but is still subject to many forms of perceptual degradation.

Another approach is to utilize wire frame or solid rendered images of the robot and objects in its environment. This approach has the additional advantage of offering any desired perspective or zoom under operator control. Providing that sensors are capable of determining position and motion of objects, such graphic enhancements can also become predictive displays capable of reducing the limitations introduced by time delays.

We have been developing a real-time graphic simulation as a part of our experimental telerobotic system, which also includes bilateral force reflection, integration of tactile and force sensed data, and computational perception support.

This paper describes the graphic simulation and outlines experimentation carried out to determine the simulation configuration which will offer the greatest enhancement to operator performance.