"We are very, very far from matching raw compute power of the human brain." This has been said for decades with viral conviction. While not equivalent -- we don't understand how these numbers relate -- the human brain averages having approximately 86 billion neurons, and GPT-4 has an estimated 1.8 trillion parameters. At some point we might have to re-evaluate the value of this dogmatic comparison between the human brain and existing AI systems.

There are some 100T to 1P synapses in the 20W human brain (of which we have 8bn). Every synapse is a little molecular computer with several kinds of memory and its own inference and learning logic.

AFAIK the biggest limitation of von Neumann computers is memory bandwidth. Brain is moving 10+ PB/s between neurons. This does not count synapse-local and intra-neuron bandwidth.

> the human brain averages having approximately 86 billion neurons, and GPT-4 has an estimated 1.8 trillion parameters.

But that's not an apples-to-apples comparison, as the artificial neurons of neural networks are just a rough approximation of our neurons. They are also connected in a much simpler way too (nicely divided into layers, where each layer can only pass signals to the very next layer), so it could be that it isn't just a matter of how many parameters you have.

And, it isn't just number of neurons, it is connections between neurons. I think brain still outnumbers on a count of connections (sorry, don't have source handy).

You didn't read what I said. Did I need to use a specific fruit metaphor so you would understand that I claimed they weren't directly comparable?

Yes, you said that those aren't directly comparable, but your comment seems to imply that you can account for the difference by having more artificial neurons than the number of biological neurons that we have. Sorry if this is not the case.

A parameter is comparable to a synapse, and the brain has around 100 trillion.