Workshop: Open-source and Networking Tools to Deploy Ephemeral Immersive Spaces

Abstract

Music, media, and technology research are intrinsically interdisciplinary. From a research perspective, research involving the development of art-oriented tools usually lacks larger user bases for evaluation and data collection in production situations or real-life usage. From the musical practice perspective, exploring state-of-art tools during their development allows artists to tailor these arti facts according to production needs while having the feedback needed to exper iment with a little compromise on the quality of the artistic work. Developing open-source tools such as Puara, a framework to create gestural controllers and self-contained systems for artistic exploration, allows artists and researchers to deploy immersive spaces and set artistic performances using off-the-self plug and-play hardware and software components. 

Puara provides interconnection with standard artistic software/hardware for easy integration with the artist’s workflow. This integration allows artists to ex plore sound spatialization, video mapping, gestural control, and sound synthesis with embedded devices such as Raspberry Pi or Nvidia Jetson. 

Open-source and Networking tools to Deploy Ephemeral Immersive Spaces is aimed at art researchers, artists, designers, content creators, and other creatives interested in creating immersive spaces using research-developed tools. We will use off-the-shelf hardware and open-source software to explore SAT/Metalab tools, connecting them with the existing workflow of researchers and artists. Participants will learn about Metalab tools for different artistic tasks, including sound spatialization, video mapping, and computer vision. The topics include evaluating the available space, setting up projection mapping, calibrating video projectors and sound equipment, setting up cameras for live pose tracking and estimation, and employing a software pipeline to map everything together. 

The devices used in this exploration have embedded tools such as SATIE, Splash, and libmapper. Digital Musical Instruments and gestural controllers can be used to interact with the system. The T-Stick, the GuitarAMI, and the TorqueTuner are examples of devices built with Puara modules that allow fast connectivity through OSC (Open Sound Control) or libmapper.

Biographies

Founded in 2002, Metalab is the research laboratory of the Society for Arts and Technology (SAT). Metalab’s missions are to stimulate the emergence of innovative immersive experiences and to make their design accessible to artists and art creators. Applications for the tools developed at Metalab include mixing video mapping, 3D audio, telepresence, and interaction. 

Edu Meneses is a music technologist, digital luthier, composer, and performer. A Ph.D. in music technology at McGill University (IDMIL and CIRMMT), Edu is currently a researcher-developer at SAT. He works with embedded systems for artistic tools and sound spatialization.

Thomas Piquet is a research engineer at SAT. He has a Ph.D. in microprocessor architecture and is specialized in embedded computing. Thomas worked for several years in the telecommunications industry before joining the Metalab team where he combines his research interests while continuing to develop his artistic skills in composition, sound design, mixing, and acoustics.