I haven't tried it directly myself but there is this project, Dragonfire, which looks quite reasonable using DeepSpeech:

https://github.com/DragonComputer/Dragonfire

There's a minimal demo app I put together here too: https://github.com/nmstoker/SimpleSpeechLoop