“Beyond the Table Top, Everyman, The Ultimate Commodity” presented by Pensyl, Jernigan, Ping and Duy Nguyen
Symposium:
Session Title:
- Performing Technology
Presentation Title:
- Beyond the Table Top, Everyman, The Ultimate Commodity
Presenter(s):
Venue(s):
Abstract:
The application of mixed and augmented reality in large volume theatrical spaces and in live action performance.
This paper covers two aspects of a real world application of mixed and augmented reality technologies in theatrical performance. These are the Technological Framework and the implications of the use of such media technologies on development and presentation of narrative strategies in theatrical and entertainment performance experiences.
Within the technological framework, the paper discusses at length the processes and system architecture for mixed reality in performance, methodologies for creating 3d modeled and animated assets (virtual characters,) recording and lipsyncing of vocal tracks, creation of augmented reality environments where the virtual characters can interact with live action actors. These virtual characters are design to interpolate or “morph” through a series of “visemes” and facial expressions, thus creating responsive and emotive characters.
The technological framework also includes an in-depth discussion of the hybrid vision and sensor tracking system designed to accommodate mixed reality in large volume spaces and in the inconsistent lighting environment of the theatre stage. This process entails a custom designed extrinsic and intrinsic calibration methods for the use of 3 or more cameras and the 3 axis gyroscopic accellerometers to control and present the virtual characters in a runtime environment during theatrical performance.
The paper discusses the scripting and negotiation processes that mediate between the technology and the dramatic and artistic goals in the production of theatrical and entertainment experiences. For the dramatist or artist the application of the technology is in service of the narrative content or conceptual framework that defines the work. Process or methods for technological development are mostly secondary to the construct of the art work itself. In our project the technology and it’s application inherently modifies and mitigates the artistic or dramatic considerations. While we hold true to the narriative as the driving force of the work, the implication of injecting a altered state of character within the drama is impacted by the technology, the presentation and the juxtaposition of media and live action performance. Our paper attempts to reconcile, or at least recognize the tensions that occur in the creative process and how these can extend the potential of theatrical performance and enhance the audience’s experiences. To our knowledge, this is the first time augmented reality technology is used in theatre performance.
