On Mon, Mar 23, 2026, 1:54 PM James Bowery <[email protected]> wrote:
> > > On Sat, Mar 21, 2026 at 12:28 PM Matt Mahoney <[email protected]> > wrote: > >> ... >> >> There's something I'm not getting. Why does the brain need 10^15 >> synapses to store 10^9 bits? Maybe it's a speed optimization, like how >> a server farm has a million copies of Linux, or your body has 10^13 >> copies of your DNA. Or is it something else? Is it the reason we >> didn't solve AI in 2000? >> > > I suspect it has to do with Bekenstein bound placing data points in such a > high dimensional space that they are all on a surface where they can be > treated as orthogonal. > How is that so? I realize that random bit vectors all have an average Hamming distance of n/2, which puts them all on a hypersphere surface surrounding any one of them. But word vectors are not like that. Some are more correlated than others. They would have to be, because otherwise text would not be predictable and we wouldn't have AI. I realize that other parts of the brain are highly repetitive, like thousands of copies of line and edge detectors in the visual cortex, or thousands of motor neurons controlling the same muscle. Language evolved relatively recently and there is not a lot of evolutionary pressure to optimize it. It uses maybe 10% of our brain, or 2% of resting metabolism. The Bekenstein bound is different. Space only has 3 dimensions. It might explain the size of the proton* but I don't see how it explains language. We already have LLMs that are not far off the 0.3 bits per parameter stored in a Hopfield net. * The Bekenstein bound of the Hubble radius is A/ln(16) of its surface area in Planck units, or 2.95 x 10^122 bits. This is about the number of protons or neutrons that would fit inside, which is a strange coincidence given that the number depends only on h, G, c, and the age of the universe. ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/Tc9fe35df94409188-Mad595c3e3a8538380e77b94f Delivery options: https://agi.topicbox.com/groups/agi/subscription
