Blotter is a proof of concept I hacked together that does a bit of audio recognition combined with a few generative AI models (both text and img) to create visuals that are relevant to the song.
The video stream is generated in real time at 24fps - you can try it yourself by requesting visuals in the Twitch chat using the "!v" command!
Right now it's mostly a fun hack project, but I am tinkering with new model architectures for higher fidelity video as well as an interactive tool so people can make videos with their own audio files.
I'd love to hear any feedback or suggestions, thanks!