- Client
Sónar Festival
Exploring Feedback Loops and Artificial Intelligence in Audiovisual Art
This project began with a provocative thought: could AI models, dependent on vast datasets, eventually train on data generated by other AIs due to the depletion of human-created content? This concept mirrors feedback loops—a technique I often use in live visuals. Inspired by this idea, I decided to experiment by replicating feedback, not with my usual analog video gear, but with an AI model.
The challenge was set: use AI to generate image and audio in unconventional ways, prioritizing creative exploration over efficiency or technical precision. The goal wasn’t to achieve detailed outputs, but to embrace imperfections and unpredictability.
To achieve this, I developed a system in Max/MSP and Node.js, integrating the Stable Diffusion API for image generation. The Node.js script automated file naming and video creation using ffmpeg, enabling a seamless iteration process. Using img2img with only one generation step revealed fascinating results—after about 200 iterations, the system “collapsed,” creating mesmerizing ripple-like patterns reminiscent of analog video feedback.
On the audio side, I trained a timbral transfer model called RAVE with distorted feedback and noise, abandoning the use of traditional instruments. To further push boundaries, I utilized the RGB and brightness data from video footage to modulate a sine wave feedback FM system, creating a dynamic arpeggiator controlled by the visuals. These two audio signals were processed and filtered in real-time using the RAVE model, resulting in a mysterious and harmonious soundscape that perfectly complemented the visuals.
This experiment challenged the typical relationship between sound and visuals, flipping the paradigm by letting video data drive the sound design. It was an exciting opportunity to step out of my comfort zone, embrace new tools, and reimagine creative workflows.
Init Image: none
Steps: 1
CFG Scale: 7.5
Subseed Strenght: 0.6
Denoising Strenght: 0.2
Init Image: none
Steps: 1
CFG Scale: 7.5
Subseed Strenght: 0.6
Denoising Strenght: 0.3
Init Image: none
Steps: 1
CFG Scale: 7.5
Subseed Strenght: 0.6
Denoising Strenght: 0.2
Init Image: square
Steps: 1
CFG Scale: 5
Subseed Strenght: 0.3
Denoising Strenght: 0.7
*** Project developed to showcase at Sónar + D 2024 and presented at Ars Electronica 2024