It makes more sense to farm in places that have reliable sources of water. I don't see why we need to subsidize California's crops, especially when we have so much great soil wasted on corn. Farming, in general, in the US is fairly messed up. Perhaps this crisis will lead to a fix. Maintaining the status quo and pretending everything is okay via regulation and political corruption is what got us here. Lets not continue that.
Even if we just grant that agriculture should be moved away from CA, no one wants to just see it stop immediately for the same reason that the government didn't demolish all buildings that were made with asbestos when they determined that it was harmful to human health.
Define "reliable". California DOES have a reliable source of water. There's a reason the crops are grown here in the first place! The problem is, it's not a reliable source of water for the demands being placed on it. Which is something you'll find, oh, anywhere you start trying to move this agriculture project. It'll work great, and there will be enough for everyone, right up until there isn't.
Why not? Prices would rise, producers in wetter areas would produce more to take advantage, and everyone eats. We'd pay more for food, but we're already paying it, just indirectly and in a more damaging fashion with subsidized water.