Gestural Mapping and Physical Modelling in Digital Instruments

Abstract

This paper considers the hand as a digital interface in composer Laetitia Sonami’s Lady’s Glove instrument. For nearly twenty-five years Sonami used different versions of the wearable controller for the real-time manipulation of digital sound. Sonami introduced the first, “kitchen wear” version of the Lady’s Glove at Ars Electronica in 1991 for a collaborative performance with the artist Paul DeMarinis based on Mechanization Takes Command: A Contribution to Anonymous History, a 1947 text by the Swiss architectural historian Sigfried Giedion. Giedion’s text describes among other things the historical effects of automation and mechanization on the human body, even anticipating a kind of posthuman conception of the hand in referring to the organ as a “prehensile tool.” In their multimedia text setting, Sonami and DeMarinis refract Giedion’s Mechanization through manualization and competing forms of digitality. 

Sonami’s performances imbue computer music with the analog contingency of the gendered body, while her Lady’s Glove points to a tacit continuity between the body and number inherent to the concept of the digital. Sonami describes the Lady’s Glove, in part, as a response to the “heavy masculine apparel” found in virtual reality systems. In the third version of her glove, she nevertheless incorporated sensors from the Power Glove, a device that brought such VR aesthetics to consumers, and used a tracking function based on the reflection of audio signals above the range of human hearing—a technique associated with obstetrics known as ultrasound. The paper considers the relationships between gestation and gesture in Sonami’s What Happened II (1980), which portrays familial reproduction and its disintegration. It concludes that Sonami’s Lady’s Glove musically parallels the use of the hands as an interface for domestic care, reproductive labor, and housework. 

Biography

G Douglas Barrett works on experimental music, media theory, and contemporary art. His new book, Experimenting the Human: Art, Music, and the Contemporary Posthuman, was published in 2023 by the University of Chicago Press. His book, After Sound: Toward a Critical Music, was published in 2016. His articles have appeared in Cultural Critique, Discourse, Postmodern Culture, Mosaic, Twentieth-Century Music, and Contemporary Music Review. Recently, he joined Syracuse University’s Television, Radio, and Film Department as an Assistant Professor.

Abstract

Physical modelling (PhM) synthesis (also known as synthesis by rule or synthesis from first principles) refers to a wealth of techniques able to reproduce the sound of acoustic musical instruments algorithmically. This practice evolved parallel to the advent of computers, though the enabling ideas behind PhM trace back to the early works on differential calculus by D’Alembert and Euler in XVIII century Europe when the musical problem of the vibrating string was investigated and formalised mathematically. In the 1990s, scientists and musicians implemented these ideas in digital computers, giving rise to the first successful examples of PhM synthesis. Among these, digital waveguides and modal methods occupied a central role due to their remarkable efficiency.

Soon, research in PhM evolved from purely artistic to scientific after researchers started applying mainstream numerical methods to solve more complicated physical systems. This came at the expense of real-time synthesis but allowed the exploration of systems that were out of reach for the early synthesis techniques. Finite-difference methods, a numerical technique in which the differential operators are approximated by algebraic operations on a grid, allowed to simulate objects such as bars and plates and have since become a standard in PhM synthesis. Today, the frontiers of physical modelling extend far beyond academia and embrace fields outside of musical acoustics: a quick survey of commercial music software branded as physical modelling is proof of the growing appeal of such techniques to the public. Commercial applications have appeared together with a growing body of academic work but have yet to reach maturity: many current applications are still out-of-reach for real-time synthesis. In this talk, I will review the enabling ideas of PhM from a historical perspective, beginning with the early models of musical string vibration to the modern networks of nonlinearly coupled objects. I will focus on two recent ERC-funded projects exploring ideas in PhM synthesis, the NESS project (2011-2016), led by Prof Stefan Bilbao at the University of Edinburgh, and the NEMUS project (2021-2026), led by myself. Examples of real-time applications running on consumer hardware will compound the slideshow presentation.

Biography

Michele Ducceschi is an Associate Professor of Acoustics in the Department of Engineering at the University of Bologna, Italy (2021-permanent). Previously, he was a Leverhulme Early Career Fellow (2017-2020) at the University of Edinburgh, Scotland, and a Royal Society Newton International Fellow (2015-2017), again at the University of Edinburgh. In 2020, he was awarded an ERC Starting grant for my project NEMUS – Numerical Restoration of Historical Musical Instruments, creating digital replicas of historical musical instruments that are out of playing condition. His research sits at the intersection of music, physics, engineering, computer science and applied mathematics. He is interested in developing fast and reliable numerical schemes for the simulation of nonlinear vibrating systems, enabling a deeper understanding of the sounding object from a simulation perspective. He works with musicians and museum curators to bridge the gap between technology, music composition and organology.