Augmenting Virtual Worlds with Musical Robotics

(Long paper)

Keywords: Virtual Reality, Video Games, Music, Sound Effects, Musical Robotics, Solenoid, MIDI.

This paper introduces the concept of augmenting the experience of interacting with virtual worlds by making use of musical robotics systems. By creating or interpreting real-time control data from the music and sound-effect channels of interactive software while it is being used, signals for controlling robotic musical instruments and other acoustic sound-creating devices can be generated. These acoustic instruments add a physicality to virtual environments by bringing previously virtual sound into the real world. A proof of concept is described, making use of a set of custom-built robotic pitched and non-pitched percussion instruments playing in conjunction with a well-known vintage videogame. The system is presented as an installation of kinetic art and sound for participants to experience. The design of the robotic sound-objects, their control systems and the software used is described and the paper concludes with an outline of future work and a summary of some of the many potential applications for this technology.