Playground Gaming with Wii, Kinect and Processing
Symposium:
- ISEA2011: 17th International Symposium on Electronic Art
-
More presentations from ISEA2011:
Session Title:
- Game and Movement
Presentation Title:
- Playground Gaming with Wii, Kinect and Processing
Presenter(s):
Venue(s):
Abstract:
This paper is a case study in the design and development of an interactive application produced as part of an AHRC “Beyond Text” Large Grant project. It uses open source software (Processing, libfreenect, Darwiinremote, OSC) and hacked games hardware (both the Microsoft Kinect sensor and Nintendo Wiimotes) to create a low-cost motion tracking system that allows the recording, playback and analysis of children’s playground games in 3D.
By doing this, the application provides researchers in the humanities with a new and innovative way of analysing and archiving these games. The paper covers both the technical aspects of the project and the academic/editorial challenges in making the application enjoyable to children as a game and useful to researchers as an archiving tool. It also provides insights into the potential for hacking videogames hardware (particularly motion-sensitive hardware such as the Kinect and Wiimote) for artistic purposes.
The application presents children with an attractive and intuitive interface, which they can use to record and replay the sound and moves of their game. As the child performs their clapping game, the computer tracks their hand movements and displays them as two animated hands on screen (which mimic their real hand movements realistically). Both their moves and their song/rhyme are recorded. Once they have finished recording, they can replay their own recorded movements – or those of any previous player – which are shown as two additional on-screen hands. As a result, the child can play with/against the recording, with the computer detecting when the player’s real hands “clap” with the virtual computer-controlled ones.
Every game is archived and researchers can examine and analyse this raw data. The application provides various ways doing this – ranging from simple playback to complex visualisations which show the path taken by the hands throughout the entire game.
T his 3D motion tracking system provides significant advantages over conventional archiving techniques, such as video recordings, which only allows the action to be viewed from a single angle (possibly omitting significant details). There is also the potential for the movement data to be analysed by computer (e.g. to identify similar patterns or rhythms).
