Super­‐Normal Integration of Sound and Vision in Performance

Abstract

In the world of rock, blues, and popular music, guitarists who pull faces when they play are somehow more entertaining to watch – as if there is an agreement between the performer and the audience that certain facial gestures enhance the expression of the music at key moments. How does this agreement come about? What is it that viewers and performers share? Research in the field of cross-modal perception has demonstrated that humans share systematic linkages between the senses, including vision and audition. For example, people link higher pitched sounds with smaller, paler, and higher-up objects than lower sounds (e.g., Mondloch & Maurer, 2004). Yet few investigations have explored how these linkages relate to performance in the creative arts. One theory suggests that artists have stronger functional connectivity between sensory areas, and are therefore able to invoke or translate sensory experiences across modalities, using sensory metaphors, and strategic sensory exaggeration (Ramachandran, 2003). To test if viewers share a sensory experience with performers, we asked participants in an online quiz to guess which of two guitar faces ‘went with’ particular moments in guitar solos. One-hundred and seventeen volunteers guessed guitar faces at rates higher than would be predicted by chance, indicating that people were able to draw meaningful information about music from still images of the guitarist’s face. In addition to presenting the results of the study in a poster format, this demonstration will give visitors a chance to engage with the interaction between musical performance and performative gesture, in two activities.