Project report

TRIPLORATION –
An Audiovisual Live Coding Performance in the Metaverse

“Triploration” shows how sound, code and virtual worlds merge to create a unique experience.

Project participants

Damian Dziwis
Markus van Well (TOPLAP Düsseldorf)

Project status

Successfully finalised

In this cooperation project with live coder Markus van Well from Toplap Düsseldorf a unique live coding performance in the metaverse was realized using innovative developments from the field of musical metaverse research.

About "TRIPLORATION"

"Triploration" is an audiovisual live coding performance with two performers (Markus van Well and Damian T. Dziwis) in an immersive, virtual, web-based metaverse environment. The term “Triploration” is a neologism composed of the words “triangle” and “exploration”, used as a metaphor for the presented live coding performance.

artistic concept

TRIANGLE & EXPLORATION

The triangle is a key element in the musical and visual composition of "Triploration."
It uses algorithms for fractal compositions with Sierpinski triangles, which in the third dimension as pyramids also represent the virtual stage for the performance.

The sound synthesis of the "IBNIZ" programming language for live coding is based on a sawtooth oscillator that generates a sequence of triangles on the zero axis.

Just as the relationships between the sides of the triangle merge into a geometric form, the relationships between the performers and the audience create a special experience. Based on an artistic concept of worldbuilding (Dziwis, Damian (2023): VERSNIZ – Audiovisual Worldbuilding through Live Coding as a Performance Practice in the Metaverse), the performers fill the virtual environment in real time with audiovisual fragments using algorithms written in the IBNIZ programming language, creating a constantly evolving audio visual world.

The viewer's movement through
the environment determines
the time, place, and perspective....

...from which the
fragments are perceived.

This creates a unique spatial composition through the possibilities of immersive technology and spatial audio, providing an individual experience for each participant, even in a shared virtual environment where different experiences are created through personal perception.

technical implementation

"Triploration" was developed with the Orchestra toolbox (Dziwis, Damian; Coler, Henrik von.; Pörschmann, Christoph (2023): Orchestra : A Toolbox for Live Music Performances in a Web-Based Metaverse) (GitHub) and implemented as a WebXR metaverse application. This allows not only to bring together the performers over the internet from different locations but also to unite the audience from anywhere in the world with the performers in a virtual environment. The experience can be accessed via various devices such as computers, smartphones, and especially immersive devices such as VR or AR headsets.

Public Performance

Networked Music Concert 2024

The premiere of the performance took place in 2024 at the “Networked Music Concert” of the Detmold University of Music at KreativeInstitut.OWL, and was performed again internationally at the NowNet Arts Conference 2024.

The photo shows the HfM Detmold jazz combo at the Networked Music Concert 2024 at KreativInstitut.OWL

You can find our event recap for this event here:

Credits:

· Screenshots – Damian Dziwis