How does UX Design relate to Music?

Summary

UX design is about designing environments (services, digital platforms and products) wherein people have delightful and meaningful experiences through interactions. This overlaps closely with my artistic work in creating original performances, where I leverage interdisciplinary collaborative methods, graphical communication systems, and custom motion-sensitive software to design interactive systems for musicians and dancers. This case study will examine how the UX design process relates to the creation of interdisciplinary artistic media.

Role and situation: Composer/computer music designer, collaboration with cellist and dancer, Institute de recherche et coordination acoustique/musique (IRCAM), September 2017-June 2018

The Brief

in tensions is a work for dancer, cellist, motion sensitive live electronics, and technician that investigates themes of control, intimacy, and the sublime through interrogating questions of expectations, role, and power structures.

Here’s a video of the performance:

Background

Usually, dancers dance to music. Whether it’s a live band or a recording, the music plays unchanged and the dancers react. How can artists disrupt this power relationship? How can one create a situation where human-produced sound and movement can inform and influence each other in a bi-directional feedback loop?

Goals and Constraints

IRCAM is a computer music research center in Paris that serves as a collaborative space for composers, performers, scientists, and engineers. Their projects center on exploring the intersections between technology and artistic creation through research in artificial intelligence, music representation, movement, and digital signal processing. I was given the opportunity at IRCAM to create a performance for cello and dance.

Having worked on incorporating music and dance for the past 4 years, I had a unique set of values that heavily shaped the collaborative process:

1. Collaborative workshop sessions to create close rapport between the performers 

2. Improvisation-based material that the performers naturally embody over time 

3. Performers that traverse the boundaries of what they’re traditionally expected to do 

4. No music stands on stage during the show - dancers routinely memorize their performances, so the musician should too

5. Both performers are equally responsible for all visual and sonic aspects of the performance to create an immersive theater-like atmosphere

These values disrupt the traditional process of “classical” music making, approaching the process of rock bands. Typically classical composers toil for hours alone in their studio notating their music precisely on paper. This paper is delivered to the musician, who practices from it for hours. During the performance, this artifact is played on stage (usually without a second thought), where it serves as a mediation between the artist and the audience. Think about how an orchestra advertises: they usually focus on the conductor: the one musician on stage not playing an instrument, but fully engaging with the other musicians in a highly choreographic fashion. They don’t show the sea of violinists buried deep into their music stands. 

Additionally, there would be 2 electronic components. 

1. Pre-fabricated sound files that are triggered by a technician (me) in the sound booth at precise moments

2. Two motion sensors, worn on the dancer’s arms, that stream data to a computer that maps this data into sonic events in real time

Artistically, I wanted to make a performance about relationships: those between the performers themselves, their individual and collective wills and desires, and their environments. When we typically hear music, we tend to focus on the sound that the performer makes. I was interested in developing the performers as characters who behave in a way that produce sound and movement, where these are the byproduct of physical and psychological behavior, not the default goal of a situation. 

Process

To create a performance that reflected my system of values, it was necessary to evaluate how information was going to be transmitted to the performers in a way that facilitated both memorization and flexibility. Therefore, I viewed this document as a tool for creation and internalization of the material. The solution should be:

1. an artifact/byproduct of the collaboration, not the finality 

2. studied out of time, not to be read/performed form in real time

3. an aid to the performers’ memory by working through aural transmission

Thus, the score makes frequent references to supplementary video recordings. These recordings were captured during the workshop process and demonstrate performative behaviors.

Behavior is how I refer to the general conduct of the performers. For instance, you can instruct someone to behave in a deceptive manner without giving them explicit things to say, thus providing a system of limited flexibility while granting the person agency to choose the best action in a given moment and situation. Behaviors in this piece are governed by sets of rules permitting a limited system of actions, and I would rather show performers these rules than tell them. Through referencing videos in the scores, performers can infer their own low-level rules (pitches, rhythms, quality and geometry of movement) from an analysis of high-level behavior descriptors (move as if your spinal column is made of concrete). Since the majority of the piece does not make heavy use of non-flexible acoustic sound/movement sequences, I believe this communication method is effective for this specific situation.

Human-Human interaction with Humans-computer Interaction

This work is an investigation human-human interaction (behavior) and humans-computer interaction. The flow of the HCI is outlined in the figure below.

The process above produces layers of sound that combine raw acoustic signal from the cello with the data streamed live from the dancer’s movements, shaping human-created timbres with human-created data into un-human soundscapes.

Composing with all of these parameters creates situations where the traditional hierarchies of musicians and dancers are put into a constant state of flux. We are constantly evaluating and questioning the relationships of the actors at play, and how they relate to their omnipresent environment.

Why Is This Important?

in tensions takes the performers out of their traditional roles as “cellist” and “dancer”. Instead, they become entangled in a complex network of interactions as actors with affordances. The composition sets up dynamic systems where these actors explore their boundaries with one another, between themselves and their environments, and within their own psychological states. This is a step away from typical contemporary music making in that the it proposes novel channels of performer communication, collaboration, and the execution of a performance.