“Host Body/Coupled Gestures: Event for Virtual Arm, Robot Manipulator and Third Hand” by Stelarc

Exhibiting Artist(s):

• Stelarc

The Host Body/Coupled Gestures: Event for Virtual Arm, Robot Manipulator and Third Hand is a computer generated human-like manipulator interactively controlled by VPL VR equipment. Data Gloves with flexion and position-orientation sensors and using a gesture-based command language allow real-time intuitive operation and additional extended capabilities. Functions are mapped to finger gestures, with parameters for each function allowing elaboration — for example the continuous rotation function can apply to the fingers as well as the wrist and the speed can be varied It is also hoped that hand state and motion gestures will also be recognised. The Cyber Glove, with finger abduction and thumb roll sensors will allow a more extensive gesture language. Another strategy that has been used is to only depend on a small number of gestures — with different functions mapped in different tables. By swapping tables, it is possible to access a new set of functions and parameters.

An Arm Editor allows the Virtual Arm to be customised for particular performances and virtual tasks — not being limited by anatomical or engineering constraints of the real world. Use of Limiters provide constraints that define the functional motion. The Virtual Arm is able to behave ambidextrously —as a left hand or right hand by switching the user’s control and simulating hands.
Some of the Virtual Arm’s extended capabilities include ‘stretching’ or telescoping of limb and finger segments, ‘grafting’ of extra hands on the arm and ‘cloning’ or calling up another arm. The ‘record and playback’ function allows the sampling and looping of motion sequences. A ‘clutch’ command enables the operator to freeze the arm, disengaging the simulated hand. For teleoperation systems such features as ‘locking’ — allowing the fixing of the limb in position for precise operations with the hand; in ‘macro mode’ complex commands can be generated with a single gesture and in ‘fine control’ delicate tasks can be completed by the transformation of large operator movements to small movements of the Virtual Arm.
Gesture interaction is an effective and efficient means of remote controlling robot manipulators. As the Virtual Arm is a generalised or universal robot with extended capabilities, it may be useful for tele-operation environments or it can be seen as a sophisticated human-like manipulator for handling objects in a virtual task environment.

Host Body/Coupled Gestures: Event for Virtual Arm, Robot Manipulator and Third Hand is an interactive event that controls counterpoints and choreographs the motions of the virtual arm, a robot manipulator, an electronic third hand and the arms of the body. It combines real-time gesture control of the Virtual Arm, pre-programmed robot scanning, symbiotic EMG activation of the Third Hand and improvised body movements. Sensors on the head and limbs allow the body to switch images from cameras positioned above the body, on the robot manipulator and from a miniature camera attached to the left arm — with the Virtual Arm being the default image. A relationship between body posture and images is established, with body movements determining the flow of images on the large screen — displayed either singly, superimposed or in split configurations. Amplified body and machine signals acoustically configure the virtual and robot operations. There is an interface and interplay of virtual and machine systems; of simulation and physical action; of actuation and automation. The interest is how electronic systems can extend performance parameters and how the body copes with the complexity of controlling information video and machine loops on-line and in real-time. The artist has always been intrigued by the phantom limb effect that amputees often experience —the body can now experience an additional virtual arm.

The virtual arm project was completed with the assistance of Mike Papper, Craig McNaughton, James Boyle, Dean Hansen and Robert Webb — supervised by Mike Gigante at the RMIT advanced computer graphics centre, CITRI, melbourne, 1992.