For the last while, I’ve been working with Ricky Graham (Stevens Institute of Technology, NJ) on music performance systems designs that are informed by embodied perspectives on musical structure. Our previous work has centred on traditional tonal music forms as applied to spatialisation (via Emmerson’s space–frames and the embodied image schemas of Lakoff and Johnson). We’ve also developed some new ideas about how theories of electroacoustic music exhibit strong similarities to theories of embodied cognition. (Given electroacoustic/acousmatic music’s focus on environmental sound materials and immersive sound environments via spatialisation and other processes, this part seems intuitively obvious, right?)
But what’s really striking to us is just how closely some of the theories follow each other. Our latest paper, Electroacoustic Music as Embodied Cognitive Praxis (presented at EMS15, Sheffield) proposes that Denis Smalley’s theory of spectromorphology is a theory of embodied cognition. And that electroacoustic music, therefore, is ‘embodied–cognitive praxis’.
At a basic level, Smalley’s spectromorphology uses a language of gestures and (environmental–style) causality in discussing musical form. So it’s already (more than) halfway to embodied cognition. It also takes a variety of musical cases which are strikingly similar to Lakoff and Johnson’s image schemas: cycles, paths via vertical and horizontal trajectories, centre–periphery forms and balancing dynamics via combinations of gestures; we looked at a few key cases from Smalley and broke them down into image schemas. Where there are differences, we propose that Smalley’s spectromorphologies actually extend image schema theory.
A key plank of Smalley’s theory is that different types of gestures have different embodied–functional associations and, hence, causal dynamics. Based on this idea, we’ve outlined a number of specifically sounded schemas (based on common raw materials or audio processes in electroacoustic music) and their embodied associations.
- Flocking/streaming…grouping and segregation (grow/integrate/dissolve)
- Rupture/breaking/glitch (break/sudden change of state, foregrounded act)
- Stretching (extension/sustaining tension, investigating limits of system)
- Bouncing (equilibrium/balance schemas; bounce–back, echoes, decay, inertial effects, coming to rest…cue for new event entries)
- Slow oscillation/breathing (cycle, balance; pace, tension/relaxation)
- Dilation/diffusion–to–point source (expand/contract/coverage,density)
In this view, the embodied dynamics/associations are the generative imperatives and grammars which underpin electroacoustic/acousmatic music (and other similar timbre–based forms).
We then moved on to look at timbre itself, via a model of forces and gestural dynamics. We saw some striking parallels between many of Smalley’s basicgestural dimensions, his energy–motion profiles, and Johnson’s (2007) qualitative dimensions of movement.
This leaves us with a three–dimensional model of gestural dynamics: (1) tension, (2) projection and (3) linearity. We sought to apply these dynamics to an embodied–cognitive theory of timbral ‘space’. We blended Patton’s (2007) theory of 3D spectromorphology–influenced notation with the classical 3D cognitive timbre–space proposed by Grey (1977). Combining these models with the gestural dynamics noted above gave us an outline of an embodied timbre–space which is organised around the qualitative dimensions of movement (see below). We hope to develop this theory in future work.
This type of model provides a starting point for an embodied theory of timbral relations for a wide range of musical materials and functions (and may provide insights into how larger–scale musical form works for timbre–based cases). Also, the treatment of timbre on the basis of dimensions of movement may help us when considering how to mediate between performance gestures and musical/sonic structures in musical interaction design/HCI…music performance systems, DMIs (digital musical instruments), software interface, sonification, etc.
Our conclusions are as follows:
- Spectromorphology is an implicit theory of embodied cognition and may be extended via the gestural dimensions and forces of image schema theory, providing insight into a broader range of musical/sonic forms and practices via these typologies.
- Potential connections beyond composition: embodied timbre/gesture spaces may inform musical/sonic HCI, DMIs/NIMEs, sonification, and how we theorise sonic and digital arts practices.
- Also, the embodied resonances of electroacoustic music (and related forms) make it a space for embodied–cognitive exploration (it’s embodied–cognitive praxis), potentially informing and extending theories of embodied cognition.
The slides from this paper are available on Academia.edu. The full text of the paper will be available via the EMS proceedings site from October.
Selected references:
Adlington, R. (2003). Moving beyond motion: Metaphors for changing sound. Journal of the Royal Musical Association, 128(2), 297-318.
Blackburn, M. (2011). The Visual Sound-Shapes of Spectromorphology: an illustrative guide to composition. Organised Sound, 16(01), 5-13.
Brower, C. (2000). A Cognitive Theory of Musical Meaning. Journal of Music Theory, 44,2, pp.323–379
Godøy, R. I. (2006). Gestural-Sonorous Objects: embodied extensions of Schaeffer’s conceptual apparatus. Organised Sound, 11(02), 149-157.
Graham, R., & Bridges, B. (2014). Gesture and Embodied Metaphor in Spatial Music Performance Systems Design. Proc. NIME 2014. Goldsmiths University of London.
Grey, J. M. (1977). Multidimensional perceptual scaling of musical timbres. The Journal of the Acoustical Society of America, 61(5), 1270-1277.
Johnson, M. (2007). The Meaning of the Body: Aesthetics of Human Understanding. University of Chicago Press.
Lakoff, G. & Johnson, M. (1980). Metaphors We Live By. Chicago, University of Chicago Press.
Patton, K. (2007). Morphological notation for interactive electroacoustic music. Organised Sound, 12(02), 123-128.
Roddy, S., & Furlong, D. (2014). Embodied Aesthetics in Auditory Display. Organised Sound, 19(01), 70-77.
Wilkie, K., Holland, S. & Mulholland, P. (2010). What Can the Language of Musicians Tell Us about Music Interaction Design? Computer Music Journal. Winter 2010, Vol. 34, No. 4, pp. 34–48