I wasn't able to find anything simple enough that works, so I built my own. For now, it's just a basic CLI that works on Linux only using whisper.cpp.
It is incredibly simple. You hold down the buttons, say stuff, release and then it will pipe the transcription lines to stdout.
I'm using it to write this comment :)