TawnyAV
TawnyAV is an audio visual software designed to display procedural visuals that adapt dynamically to audio picked up by the device's microphone.
It has been developed for a specific use case; as a home grown live AV solution for the electronic music scene here in the UK. Tawny started out from my involvement as a fan of the electronic music scene here in the UK. I see a lot of events with small Djs, passionately sharing their curated music and style, and sometimes think it would be really cool to have some visuals on a display with the DJ. With that in mind, and a general itch for me to do something creative and technical, I wrote Tawny AV.
What TawnyAV Is (and Isn't)
Before diving into the tech, it's important to understand what TawnyAV is at its heart: it's fundamentally a digital art piece. While it has the bones of a simple 'game' engine, it's not a polished, user-friendly software package like you might find elsewhere. The goal behind TawnyAV isn't to sell the software, but to collaborate directly with local DJs and musicians here in the UK. I want to manually craft unique, tailored AV shows that complement s specific sound and style for live sets. If that sounds interesting, get in touch! (Contact details at the end).
Scenes and Shaders
Fundamentally, Tawny AV works in a very similar way to other game engines I have written. It is written in rust and uses Macroquad for window management, input, rendering and UI. The visuals themselves are based around ‘scenes’.
Each visual element in Tawny AV is encapsulated within a 'Scene'. Think of a Scene as a self contained module responsible for a specific type of visual display. Every scene handles its own logic for loading resources, updating its state frame by frame, and drawing itself to the screen using Macroquad.
A good example is the ShaderScene. When this scene is active, it loads GLSL vertex and fragment shader files from disk. In each frame's update, it receives global data such as the current time. These parameters are passed into the shader as uniforms. This allows the shader code itself to react dynamically to the music.
At the moment, the uniforms passed into the shader is the shaders timestep as well as the background image texture. When a beat is detected by the audio processor we simply randomise the timestep to create a randomisation in the visual.
This structure makes it relatively straightforward to create new visualisers by implementing the basic Scene requirements: new(), update(), and draw().
Audio Reactivity
The core of TawnyAV's reactivity comes from its audio processing pipeline. It uses the cpal library to capture audio directly from the default microphone input device. This raw audio stream is then processed in chunks.
Currently, the focus is on beat detection. For each incoming chunk of audio data, the system calculates its energy level (essentially, its loudness). It keeps track of the average energy over a recent time window (a rolling average). When the energy of a new chunk significantly exceeds this recent average (based on a configurable sensitivity), and enough time has passed since the last beat (a cooldown period), it registers a "beat".
This beat detection flag is then made available globally, allowing active scenes like the ShaderScene and Cellular Automata scenes to react and change their visuals in sync with the music's rhythm.
(In the future, I want to use Fourier transforms to get more data about the intensity of bass, mids and treble for any given moment to add more reactivity to visual effects.)
Current Visuals
So far, I've developed a couple of core scene types for TawnyAV:
- Shader Scenes: As described earlier, these scenes use GLSL shaders for visuals. They are highly flexible, as the visual logic is defined entirely within the shader code, reacting to the detected beat.
- Cellular Automata Scenes: This scene type generates visuals based on cellular automata rules, like Conway's Game of Life. It simulates a grid of cells evolving over time based on simple rules defined in configuration files. The cellular automata grid is randomized on every beat.
Why
Tawny AV fundamentally has quite a limited nature, at least at the moment. As mentioned earlier, it's a digital art piece. While its structured like a basic game engine, its forward facing user interface is bare bones at best. Many of the core ideas found in other software, like actual game engines or visual programming environments like TouchDesigner, are hidden behind configuration files or even require modifying the source code itself.
That's why, while the software and source code is available on itch.io for people to use, my main goal is to manually craft an AV show for local Djs and musicians who want some visuals. I want to collaborate to create unique, visual experiences that complement a specific sound and style for live sets.
Tawny AV is designed to be modular, so if you have some programming knowledge or are interested in just having a browse, find the source code here.
If that is something you are interested in, contact me at leonstansfield208@gmail.com!