Hmm but isn't it that AI has always been intended to augment Human actions, to make life easier. In that process, if we look closer it only means that at a deeper level AI has always been able to lead (For example autonomous driving) and it's still under our CONTROL. It's just that the AI actions as of today are limited to a specific sub system and is not aware of the other systems around it. This is what I wanted to propose through this question. Why hasn't anyone built anything like that yet, because regulations are best made when you have an approximate projection of what the Actual behavior is going to be like.
In order to be aware of systems around it those systems have to provide quantified data and make them accessible via API. World is just too complex. And perhaps you should also break down what "running a company" means. Because running a company is a lot... people, politics, customers, financials, investors, paradoxes, ... where should the AI rule and where not? It's just not that simple.
