If the whole subtitle file is out of sync (e.g. every line is late 2s), one shortcut is all it takes. In that 1/10 case, you do need to 'fiddle around for ages', but given that all other options (e.g. SubtitleEdit[1]) are way more complicated and time consuming, mpv is all I could suggest. For the problem with audio and image being out-of-sync, I have no idea :)
For AV sync you can do this in the receiver (and some devices with e.g. Bluetooth support for speakers) because it's not really a problem with the source material (typically anyway) it's just something that does need tuning to the room & hardware.
---
[1] https://www.nikse.dk/subtitleedit