Project report

Sonification

What does a coral reef sound like? Can the sounds even become music? Sonification is the representation of data as sounds. Maria Kallionpää, Axel Berndt and Hans-Peter Gasselseder take this approach further and even create compositions from the data.

Project participants

Maria Elina Kallionpää Axel Berndt
Mag. rer. nat., Dipl.-Psy. Hans-Peter Gasselseder

Project status

Compositions in Performance

Sonification as a Composition Technique and Meansof Artistic Expression

Making data audible - that's what sonification is all about. These sounds can then even become compositions! A research project and a scientific article were created for this purpose, which is now finding a stage in lectures and performances. The research project on sonification was originally published in December 2023 at the XXV Generative Art Conference - GA2023 in Rome and subsequently at the IRCAM anniversary workshops in March 2024. In lectures by the project participants Axel Berndt, Maria Kallionpää and Hans-Peter Gasselseder, various data in rhythms and pitches will be discussed and a particle simulation model will be presented. Interested? There will also be further presentations on this topic, see below.

  • Abstract of the scientific article on sonification

    Despite of the ever-expanding variety of technological means accessible to composers, the essence of music composing has mostly remained the same for centuries: the key question is, how to form an artistically innovative core idea of a musical piece and to find the most ideal instruments for it? Our research discusses composers' explorations on establishing their individual voices, as well as finding and selecting the tools and techniques that would best serve one's artistic goals. Various software solutions have surfaced to enable a wider palette of sounds to be used as musical "raw material". Rather than using pre-composed music based on the rhythms and pitches organised by the composer, we will focus on how to translate other kinds of data into "notes", or more largely "sound events", and how to use it in an artistically meaningful manner, resulting into musical compositions in their own right.

    You can find the full article here: https://generativeart.com/GA2023/papersDOC/OK/24_Kallionpaa_Berndt_Gasselseder.pdf

Presentation of the research project by Dr. Maria Kallionpää at the All Around Audio Symposium on 27.11.2024

The Reef

Weltpremiere des Theaterstücks „The Reef“ mit Visualisierungen des Künstlers Andrew Veloux in San Francisco

Several compositions were created as part of the research project - including “The Reef”.

With its oceanic topic and use of data sonification as a composition technique, “The Reef” is closely related to my works “El Canto del Mar Infinito” (2020) and “La Danza Invisible del Agua” (2021). The work is based on a sample recorded at Thailand's Mu Koh Lanta National Marine Park. Being somewhat limited in its spectral composition, the excerpt consisted of a swimmer proceeding towards a coral reef. Furthermore, the sample’s procedural structure would lend itself as the dramatic arch of the composition. Upon finishing the composition process, a MIDI rendition served as reference for the video artwork created by the visual artist Andre Veloux. Its abstract visual surfaces follow the music in a detailed manner: on top of serving as a structure of the musical composition, the timing of the original sound recording also informed the structure of the live visuals. The video element of the premiere was created by assigning colours to different hexadecimal bytes of source code of sound files of the piece, producing colour bar codes. These were then combined to create the video work. Andre used the Lego colour palette to facilitate the creation of physical artworks from these colour bar codes. 

The goal of this series of works discussed above is to draw attention on the importance of protecting our marine environments. Together with the real-time video artwork, the music of “The Reef” paints a picture of the gradual damaging that pollution causes to the coral reefs. However, with the help of computer-based analysis, also the vivid colors of such ecosystems were translated into music, giving us a gleam of hope. Maybe it is not too late to do something?

World Premiere of "The Reef" in San Francisco

In collaboration with the New Arts Collaboration in the USA, “The Reef” by pianist Luo Ting was premiered. The video artwork by Andrew Veloux, who is interviewed at the beginning of the video, can be seen in the background.