The scientist ambition is a grand unifying theory of minimalistic, reductionist and elegant principles that explain everything. Some even argue that we are already there.
But the truth is: when it comes to neurons, all those theories are effectively inferior to what evolution has achieved. They can explain some of what is going on, but they cannot reproduce the results of the biological counterparts.
The artificial results either require orders of magnitude more power, or examples, or has to be hardwired or trained in advance, or requires a billion dollars facility to manufacture the hardware involved.
Biological neurons get trained as they do inference, require fewer examples, use less power and the agent can get drunk and high and lose millions of neurons and synaptic connections and their brain will either keep working as usual, or everything will get rewired after a while.
We don't understand as much as we claim to do yet, if we did, we would have the same results at least.
Those neurons are being trained the day we were born. Reality corresponds to about 11 million bits per second. What I suspect’s happening is that we train higher and higher levels of abstraction and we get to a point where new knowledge is involves training a new permutation of a few high level neurons.
