Sound Initiated Drawing and Short-Term Memory Impairment

The presentation includes a participatory workshop of an interactive interface and program for creating, storing and replaying animated drawings with sound, a description of its development and the case study for which it was created. This interface was initially developed to reestablish and maintain coherent connection with a longtime friend and artist who suffered severe memory impairment due to metastasized breast cancer. Through this effort we found a way for her to continue to work as an artist by increasing and sustaining her ability to focus and concentrate for extended periods of time.

The interface includes a computer, drawing tablet and a series of programs I have written to record and assign personally significant sounds (i.e. sounds from familiar environments, daily activities, speech, music, etc.) to regions on a two-dimensional area corresponding to the surface of the drawing tablet. Once sounds are assigned, the movement of the hand is tracked as it initiates and scrubs through the sampled sounds, and the data of the movement is recorded and stored. When this data is fed back through the interface, the drawing recreates itself, unfolding both a map and legacy of the attention of the person who made the drawing.