But would you only need 5V? DSLAMs here have an hour or so of UPS capacity as well, so if the router stays up, so does the internet connection, same with 4G/5G routers. Though I guess a 5/12/24V DC mini-UPS would still be a lot easier than actual 110/220V AC.
Still a nice option!