> Luckily machine learning came along and has shown empirically that the solution is to increase the number of dimensions.
Ah yes, that's just what brilliant scientists like Mendel, Darwin, Fisher, Wright, Watson et al. were missing: Deep Learning! /s. The number of local optima grows exponentially in problem dimension, so increasing the dimension simply provides an even stronger guarantee that your optimizer will converge to one. Evolution, for its part, is definitely stuck in one, as evidenced by the fact that I cannot fly.
It is true there are more local minima, but they're generally pretty equivalent. In fact, in high dimensions, saddle points are a much bigger problem than local minima. [0] The fact that optimization works on these large, non-convex problems is kind of a marvel, and raises really interesting questions. (mostly "this shouldn't work this well unless there's some kind of regularity to these problems that makes them nicely behaved"). What this says about evolution I neither know nor care to speculate (above my pay grade).
> The number of local optima grows exponentially in problem dimension, so increasing the dimension simply provides an even stronger guarantee that your optimizer will converge to one.
Forgive my potential ignorance, but it seems to me that there's a difference between the dimensionality of a problem and the dimensionality of a neural net working on the problem. Did you mean to imply that adding more dimensions to a neural net increases the number of local maxima for the problem?
> Evolution, for its part, is definitely stuck in one, as evidenced by the fact that I cannot fly.
Flying requires a large number of biological tweaks that are not necessarily good for every organism. For instance, lower bone density. When humans have this, we call it osteoporosis and it's generally a Bad Thing(tm).
Also, I could argue that evolution has done a better job than you give it credit for, considering that I don't know any other animal that can travel over land at sustained high speeds, fly at extreme altitude, swim to extreme depths, and burrow through mountains.
Ah yes, that's just what brilliant scientists like Mendel, Darwin, Fisher, Wright, Watson et al. were missing: Deep Learning! /s. The number of local optima grows exponentially in problem dimension, so increasing the dimension simply provides an even stronger guarantee that your optimizer will converge to one. Evolution, for its part, is definitely stuck in one, as evidenced by the fact that I cannot fly.