There's probably various methods, but if you don't already have quantitative estimates, you'll probably end up polling SMEs you'd trust for this on how much increase in retention and decrease in acquisition costs you'd expect for a notational more robust Microsoft. Average those out.
Even 5 people is enough to significantly narrow the uncertainty (at least, as compared to the prior hypothesis that the true change in odds is somewhere from -1 to 1).
And like a good spam filter, you don't need to be perfect, you just need to be able to turn your initiatives into a forecast of future changes if implemented, and then measure what actually does change for those initiatives you do implement. Even simple linear regression models can be useful in tracking results compared to forecasts.
Most people would probably expect that the improvement is close enough to zero to be nearly indistinguishable from it. And this is why most enterprises don't invest much in reliability work like this, because it doesn't actually seem to move the needle for customer purchasing.
Even 5 people is enough to significantly narrow the uncertainty (at least, as compared to the prior hypothesis that the true change in odds is somewhere from -1 to 1).
And like a good spam filter, you don't need to be perfect, you just need to be able to turn your initiatives into a forecast of future changes if implemented, and then measure what actually does change for those initiatives you do implement. Even simple linear regression models can be useful in tracking results compared to forecasts.
Most people would probably expect that the improvement is close enough to zero to be nearly indistinguishable from it. And this is why most enterprises don't invest much in reliability work like this, because it doesn't actually seem to move the needle for customer purchasing.