Inclusive Technologies Research
My research is also experimenting with new immersive technology in the areas of vibrotactile experiences, gesture recognition devices and sensor technology that translates body movement into digitally generated sound and image. Each of these technologies has initially been developed to enhance the sensory experiences of those with sensory loss, mental illness or individuals with disabilities.
The Centre for Learning Technologies (CLT) and the SMART Lab (Science of Music, Auditory Reasearch and Technology), Ryerson University have completed a research initiative exploring ways of making sound and music more accessible to all people, regardless of hearing ability, and have developed a cross-modal audiotactile program that supports that supports the presentation of audio information as vibrotactile stimuli. The core assumption is that emotional aspects of sound and music can be "translated" across modalities. The Alternative Sensory Information Display (ASID), also referred to as the Emoti-Chair, allows for the creation of cross-modal experiences representing sound stimuli that are currently unavailable to deaf or hard of hearing individuals. The technology, designed for the purpose of theatre chairs for the deaf, separates audio signals into discrete vibrotactile output channels (voice coils) that can be presented on the body to create a high resolution audiotactile experience through direct connection with musical instruments, live sound and digital sound files.
In researching and using the vibrotactile technology to create 'vibratory compositions' or tactile components, with the generous support of TAD Inc., I will have the opportunity of investigating a much broader application of the sensory interpretation and emotionality of sound.
Grateful aknowledgements to:
Ryerson Research Team: Dr. Deb Fels, Dr. Frank Russo, Carme Branje (PhD), Sai Cherukumilli (MEng)
Tactile Audio Displays Inc. (TAD) Director: Dr. Maria Karam
Centre for Learning Technologies - http://imdc.ca/asid/ASID.html
SMART Lab - http://www.ryerson.ca/smart
TAD - http://tadsinc.com
Views of Emoti-Chair prototypes and wearable ASID system
The Mark, 'Emoti-chair', (published Video)
Ryerson University, 'Emoti-Chair'
Gesture Interfaces and the Vibrotactile
This area of research investigates how computer vision and gesture interfaces could be used by deaf and hearing artists and musicians as a creative medium. We have begun exploring various models that could be used to generate tactile feedback using gestural interfaces. In addition, we are also working with the Xbox Kinect hardware to implement the computer vision, and gesture recognition. The gesture interface will be developed to work with the Emoti-chair system, and it will take the user’s movements as an input and provide vibro-tactile stimulation on the surface of the skin.