“Multisensory Science Books for Inclusion” by ,

• 

Multisensory Science Book for Inclusion

Title:

Multisensory Science Books for Inclusion

Artist(s) and People Involved:

Composer, Director, Choreographer:



• Stu Favilla

Symposium:

• ISEA2024: 29th International Symposium on Electronic Art

Venue(s):

• The Edge, State Library of Queensland , Brisbane , Australia

Artist Statement:

Multisensory science books leverage the lived experience of a legally blind artist, musician/interaction designer, an industrial designer and a laboratory of scientists, to communicate cutting-edge biomedicine discovery through exhibitions for blind, low vision and diverse needs audiences. Multisensory science books are a portable exhibition format enabling inclusion, through visual and tactile design, novel technologies, interactions and experiences, music, audio design, and sonification.
The advent of COVID lockdowns and exhibition cancellations led to the creation of the unique format. The books were created in A3 size format with laminated pages on a foam core base and include tac-tile artworks, large print, braille supplements, software tracking optical fiducials (reacTivision, Jordà et.al, 2007) narration, music and sonification.
Tactile artworks throughout the books include ‘braille’ inspired molecular structures, and tactile depictions of immune cells, viruses, bacteria, and the digestive system. A combination of handmade cell sculptures appears throughout with audio text to accompany each topic and associated artwork.
Sensory science has shifted from multimodal representation to multisensory representation engaging the sensory memory and perceptual understanding that all of us carry from childhood through to our current experiences of daily living. Experience has shown that 3D printing is not the only way to communicate tactile information. Using food, found objects and a range of other materials allows for the sense memory of participants to be accessed and deeply engaged.
The message is in the material! This may be a simple metaphor such as covering a kidney sculpture with dried kidney beans or using burnt coffee grounds to represent carbon atoms in a braille-inspired molecule model. Plant stems, buds, rhizomes, feathers, bark, pasta shapes, and grains can all depict complex structures and interactions, or present well-established sensory metaphors for function that people understand immediately. 3D printing is amazing, but it lacks a sensory fidelity that can be represented by found objects.
A major component of the books was the use of interactive sonification of tactile artworks and electronic music. A sonification method was developed to display 2D art and images into sound utilizing MaxMSP/Jitter. The books were designed to sit on individual inclined (30-degree) stands. The book stands also included headphones and Mac mini computers driving the software interactions. The project generated around seven hours of original music, illustrating and representing protein and cellular interactions and themes. The protein and immune-system-inspired music was composed using modular electronic synthesisers.
The books have sought to bring blindness, low vision and diverse needs groups recognition and understanding through designing specifically for their needs. What the participating designers, scientists and researchers have all discovered through their involvement, is that multisensory design also en-gages sighted people and those living without disabilities extremely well. During exhibitions, understanding and empathy are also shared. As the barriers that exist between vision-based information and blind and low-vision people are removed, so too are the social barriers and fears that exist across society.

Category:

• Public Event
• Participatory Art Event