In order to explore the potential of audience interaction and live coding, we created 'ASCIImage Rhapsody', a piece of music that falls somewhere between installation and performance.

In this piece, compositional material is generated through audience interaction using a technical system with an ASCII-fication algorithm that converts webcam photos into program code for the esoteric live coding programming language ORCA.

This code then controls sound synthesizers developed in SuperCollider to create the music. ASCIImage Rhapsody was performed in 2025 in the context of the exhibition "... und wir fangen gerade erst an" at the Coding Islands concerts of TOPLAP Düsseldorf at Kunsthalle Düsseldorf.

The central idea behind 'ASCIImage Rhapsody' is to combine live coding performances with real-time interaction from the audience. As most audiences are not usually familiar with programming or live coding in a specific language, a bridge is required to connect interaction and code. In order to develop this novel format for installation performances, research was conducted into how user interactions and the environment could be effectively incorporated into code generation.

One promising approach was to use two-dimensional images from a webcam stream for code generation.

The live image from the webcam is converted to greyscale and downsampled to a resolution of 60 x 40 pixels. Each pixel is then converted into an ASCII character that corresponds to ORCA's programming instructions. A conversion table is used to select instruction characters according to brightness.

The result is an ORCA code corresponding to the image captured by the camera, which generates MIDI data containing information about the notes played. This data is then played in real time using sound synthesizers developed in the SuperCollider live coding programming language.

For the performance in the Kunsthalle Düsseldorf, a table with three small monitors was set up in one of the exhibition rooms. Each monitor showed a different aspect of the process, from the ASCII conversion of the camera facing the audience to the ORCA code or the SuperCollider sound synthesis code.

Visitors can interact with the performance via the audience-facing camera and influence the live coding that ultimately controls the musical result.

A publication on “ASCIImage Rhapsody” was submitted to the Workshop for Innovative Computer-based Music Interfaces (ICMI) of the Mensch und Computer Konferenz 2025.