ISSTC2015 paper: Embodied Resonances and Thoughts of Electroacoustic Music

I’m due to go to the Irish Sound, Science and Technology Convocation in Limerick next week to present another paper from my collaboration with Ricky Graham, entitled, Embodied Resonances and Thoughts of Electroacoustic Music.

You can find the abstract below.

If you’re interested, my previous posts on ‘Embodied Thinking and Music’ relate to the same general subject matter.

 


 

Embodied Resonances and Thoughts of Electroacoustic Music: What can we learn from theorising disembodied sound using embodied schemas?

Brian Bridges, Ulster University

Ricky Graham, Stevens Institute of Technology

 

Abstract

Music technologies have frequently been conceptualised in terms of how they break or creatively challenge the normal causality of sounding actions. Sound recording and processing technologies, by their very definition, modify the temporality of sounds through encoding, processing and diffusion. They also disturb gestural causality, making sounds available using gestures which may be far removed from the effort, force or structure of action involved in the original sound event.

Fixed–media acousmatic presentations are one extreme on this scale. However, even when describing the most disembodied acousmatic presentations, embodied resonances haunt our discourses. The abstracting imperatives of reduced listening may nonetheless be seen as drawing attention to the basic articulatory acoustics of sound events; in particular, archetypal envelope structures and stages. In addition, when we turn to describing larger–scale form in electroacoustic music, there is a frequent tendency to apply embodied and environmental metaphors and force–based models. For example, Smalley’s (1986, 1997) influential theory of spectromorphology describes a variety of prototypical schemas for joining individual sound objects based on apparent causality. Even with unfamiliar sound materials (or, arguably, especially with unfamiliar materials, which may promote reduced listening) an implicit assumption of such theories of electroacoustic music is that perceived connections between sound materials may echo a ‘gestural grammar’ of causal schemas drawn from our everyday environmental and embodied experience.

This descriptive strategy may be of more significance than mere rhetorical convenience (i.e., the use of familiar metaphors as convenient approximations when describing the unfamiliar). An influential strand of theory within contemporary cognitive science, embodied cognition, argues that our thought processes are shaped by learned embodied and environmentally–derived schemas. The embodied image schema theories of Lakoff and Johnson (Lakoff and Johnson, 1980; 1999) propose a series of common embodied forms as structural primitives for cognition, including the conceptualisation of formal relations. A growing interdisciplinary interest in these theories has seen them being applied to common practice music (Brower, 2000; Johnson, 2007; Wilkie, Holland and Mulholland, 2010). However, we consider such theories to be even more directly applicable to electroacoustic music and related forms. We identify parallels between some of Smalley’s formal structuring and dynamic principles and image schema theories. The identification of embodied resonances in electroacoustic music may allow us to extend our current descriptive theories by comparing them with broader theories of embodied cognition. Furthermore, certain specifics our sounded forms may provide insights which extend the image–schematic theories of Lakoff and Johnson and other embodied cognition theorists. Such an approach also has the potential to inform developments in design, such as the development of music performance and production systems which provide models of musical structure appropriate to electroacoustic music.

Keywords

Spectromorphology, embodied cognition, schemas, environment, form, structure, theory

Selected references

Brower, C. (2000). A Cognitive Theory of Musical Meaning. Journal of Music Theory, 44(2), pp. 323–379.

Johnson, M. (2007). The Meaning of the Body: Aesthetics of Human Understanding. Chicago: University of Chicago Press.

Lakoff, G. and Johnson, M. (1980). Metaphors we Live By. Chicago: University of Chicago Press.

Lakoff, G. and Johnson, M. (1999). Philosophy in the Flesh: The Embodied Mind and Its Challenge to Western Thought. New York: Basic Books.

Smalley, D. (1986). Spectro-morphology and Structuring Processes. In: Emmerson, S. (ed.) The Language of Electroacoustic Music. London: Macmillan, pp. 61–93.

Smalley, D. (1997). Spectromorphology: explaining sound-shapes. Organised Sound, 2(2), pp 107-126.

Wilkie, K., Holland, S., and Mulholland, P. (2010). What Can the Language of Musicians Tell Us about Music Interaction Design? Computer Music Journal, 34(40), pp. 34–48.

 

Embodied Thinking and Music (Part 2): Electroacoustic Music as Embodied Cognitive Praxis (EMS15)

For the last while, I’ve been working with Ricky Graham (Stevens Institute of Technology, NJ) on music performance systems designs that are informed by embodied perspectives on musical structure. Our previous work has centred on traditional tonal music forms as applied to spatialisation (via Emmerson’s space–frames and the embodied image schemas of Lakoff and Johnson). We’ve also developed some new ideas about how theories of electroacoustic music exhibit strong similarities to theories of embodied cognition. (Given electroacoustic/acousmatic music’s focus on environmental sound materials and immersive sound environments via spatialisation and other processes, this part seems intuitively obvious, right?)

But what’s really striking to us is just how closely some of the theories follow each other. Our latest paper, Electroacoustic Music as Embodied Cognitive Praxis (presented at EMS15, Sheffield) proposes that Denis Smalley’s theory of spectromorphology is a theory of embodied cognition. And that electroacoustic music, therefore, is ‘embodied–cognitive praxis’.

At a basic level, Smalley’s spectromorphology uses a language of gestures and (environmental–style) causality in discussing musical form. So it’s already (more than) halfway to embodied cognition. It also takes a variety of musical cases which are strikingly similar to Lakoff and Johnson’s image schemas: cycles, paths via vertical and horizontal trajectories, centre–periphery forms and balancing dynamics via combinations of gestures; we looked at a few key cases from Smalley and broke them down into image schemas. Where there are differences, we propose that Smalley’s spectromorphologies actually extend image schema theory.

spectromorphology and image schemas

A key plank of Smalley’s theory is that different types of gestures have different embodied–functional associations and, hence, causal dynamics. Based on this idea, we’ve outlined a number of specifically sounded schemas (based on common raw materials or audio processes in electroacoustic music) and their embodied associations.

  • Flocking/streaming…grouping and segregation (grow/integrate/dissolve)
  • Rupture/breaking/glitch (break/sudden change of state, foregrounded act)
  • Stretching (extension/sustaining tension, investigating limits of system)
  • Bouncing (equilibrium/balance schemas; bounce–back, echoes, decay, inertial effects, coming to rest…cue for new event entries)
  • Slow oscillation/breathing (cycle, balance; pace, tension/relaxation)
  • Dilation/diffusion–to–point source (expand/contract/coverage,density)

In this view, the embodied dynamics/associations are the generative imperatives and grammars which underpin electroacoustic/acousmatic music (and other similar timbre–based forms).

We then moved on to look at timbre itself, via a model of forces and gestural dynamics. We saw some striking parallels between many of Smalley’s basicgestural dimensions, his energy–motion profiles, and Johnson’s (2007) qualitative dimensions of movement.

qualitative dimensions of movement

This leaves us with a three–dimensional model of gestural dynamics: (1) tension, (2) projection and (3) linearity.  We sought to apply these dynamics to an embodied–cognitive theory of timbral ‘space’. We blended Patton’s (2007) theory of 3D spectromorphology–influenced notation with the classical 3D cognitive timbre–space proposed by Grey (1977). Combining these models with the gestural dynamics noted above gave us an outline of an embodied timbre–space which is organised around the qualitative dimensions of movement (see below). We hope to develop this theory in future work.


 

towards an embodied timbre space


This type of model provides a starting point for an embodied theory of timbral relations for a wide range of musical materials and functions (and may provide insights into how larger–scale musical form works for timbre–based cases). Also, the treatment of timbre on the basis of dimensions of movement may help us when considering how to mediate between performance gestures and musical/sonic structures in musical interaction design/HCI…music performance systems, DMIs (digital musical instruments), software interface, sonification, etc.

gesture timbre space

Our conclusions are as follows:

  • Spectromorphology is an implicit theory of embodied cognition and may be extended via the gestural dimensions and forces of image schema theory, providing insight into a broader range of musical/sonic forms and practices via these typologies.
  • Potential connections beyond composition: embodied timbre/gesture spaces may inform musical/sonic HCI, DMIs/NIMEs, sonification, and how we theorise sonic and digital arts practices.
  • Also, the embodied resonances of electroacoustic music (and related forms) make it a space for embodied–cognitive exploration (it’s embodied–cognitive praxis), potentially informing and extending theories of embodied cognition.

The slides from this paper are available on Academia.edu. The full text of the paper will be available via the EMS proceedings site from October.

Selected references:

Adlington, R. (2003). Moving beyond motion: Metaphors for changing sound. Journal of the Royal Musical Association, 128(2), 297-318.
Blackburn, M. (2011). The Visual Sound-Shapes of Spectromorphology: an illustrative guide to composition. Organised Sound, 16(01), 5-13.
Brower, C. (2000).  A Cognitive Theory of Musical Meaning. Journal of Music Theory, 44,2, pp.323–379
Godøy, R. I. (2006). Gestural-Sonorous Objects: embodied extensions of Schaeffer’s conceptual apparatus. Organised Sound, 11(02), 149-157.
Graham, R., & Bridges, B. (2014). Gesture and Embodied Metaphor in Spatial Music Performance Systems Design. Proc. NIME 2014. Goldsmiths University of London.
Grey, J. M. (1977). Multidimensional perceptual scaling of musical timbres. The Journal of the Acoustical Society of America, 61(5), 1270-1277.
Johnson, M. (2007).  The Meaning of the Body: Aesthetics of Human Understanding. University of Chicago Press.
Lakoff, G. & Johnson, M. (1980). Metaphors We Live By. Chicago, University of Chicago Press.
Patton, K. (2007). Morphological notation for interactive electroacoustic music. Organised Sound, 12(02), 123-128.
Roddy, S., & Furlong, D. (2014). Embodied Aesthetics in Auditory Display. Organised Sound, 19(01), 70-77.
Wilkie, K., Holland, S. & Mulholland, P.  (2010). What Can the Language of Musicians Tell Us about Music Interaction Design? Computer Music Journal. Winter 2010, Vol. 34, No. 4, pp. 34–48