Firstly, evidentially, people do care about disparity: increasing disparity seems to adversely affect people physiologically, independently from wealth. That is, having someone else be significantly richer than you, independently of your own health, seems to create stress effects in a population. By the metrics of national health and well-being, it seems like wealth disparity is a bad thing in its own right.

Secondly, the problem with billionaires is not that they simply have so much money, it's that one's ability to influence society, including making and breaking the rules of society, is intrinsically tied to money. For example, consider the recent news about Bezos buying a new boat, and having to take down and rebuild a bridge to get it to sea. On the one hand, it doesn't really matter to me how much he spends on that boat - it's his money, and he can use it as he likes. However, the people of Rotterdam were promised that the bridge would remain put, yet Bezos' money (presumably via the shipyard that organised this) was enough to override the democratic process, presumably alongside adding a significant inconvenience to the people living there.

Or consider the recent trend of billionaires buying media companies. On the one hand, it's kind of irrelevant how they want to invest their money, but on the other hand, these media companies afford significant impact on the views and perspectives seen by society. If we really want to claim that we live in a democracy, it seems dangerous to also accept that one person can, essentially on a whim, buy one of the largest social media platforms with only vague hints as to what he plans to do with it. That sort of power is absolutely not something that you (I assume) or I personally can wield, yet it could well have a significant impact in shaping public opinion.

As long as money can be roughly equated to power, then wealth disparity will remain a very important thing to be concerned about.