Combine audio and visual#
This took some experimentation and lengthy chats with GitHub Copilot to figure out how to convert the midi to audio and the animation in such a way they could be combined.
I ended up using bash for this, as my experiments with R wrappers failed.
Convert the midi to audio#
Using timidity
and ffmpeg
:
timidity midi/wikisource-contrapunctus-subject.midi -Ow -o - | ffmpeg -i - -acodec libmp3lame outputs/av/audio.mp3
Combine audio and visual#
Using ffmpeg
:
ffmpeg -i outputs/av/animation.mp4 -i outputs/av/audio.mp3 -c:v copy -c:a aac -strict experimental -shortest contrapunctus-i.mp4
Unsolved problem#
Somewhat disappointingly, the first edge drops from the animation due to the self loop.
(Sound on.)