Part of the problem is that academic progression is based so heavily on churning out papers that there's a race to the bottom for 'minimum publishable units'.

A single codebase might provide the analysis framework for tens of individual papers, and releasing early just increases the chances of getting scooped on some of them.

I think there are precedents for delayed data distribution in other fields (biology / chemistry?), so maybe a partial solution would be to embargo the code for a year or so after publication. The downside is that you can never expect to see code for the really cutting edge stuff, but perhaps that will be the price to pay?

Having a delay also might serve to blunt the "Yeah, well. Your code sucks." argument, in that it would need to really suck in a massive, conclusion changing way, in order to gather new headlines. Otherwise, it just sort of sinks away into obscurity unless you're looking directly at that result, in which case knowing about possible defects and improvements is a net benefit.

Overall, I'm offering no new benefits here, but at least a couple of possible mitigations of the downsides you mention.