A recursively self improving program after n iterations has only increased its Kolmogorov complexity by log n bits. This does not rule out acquisition of computing power, the other requirement for intelligence, nor the acquisition and storage of more knowledge made possible by more hardware. The limitation of programs in isolation was more relevant 15-20 years ago when there were still serious proposals on SL4 for developing AI in a box as a precaution against unfriendly AI.
I realize that the complexity of human civilization is about 500 bits given infinite computing power. I'm assuming 400 bits to describe the laws of physics, given the normal tradeoff between compression and computing power, 80 bits to say which of 10^24 planets, and 20 bits to specify the time interval for homo sapiens. Such a program could predict any question about the future, such as tomorrow's lottery numbers or the exact date of human extinction. Alas, it is not possible for any computer to model the universe that contains it, because otherwise it could beat itself at rock paper scissors. By my estimate of 10^17 bits of human knowledge (costing $1 quadrillion at one cent per bit), I mean observer dependent information, not probability in the absolute sense of Kolmogorov or Solomonoff induction. By that, I mean if I flip a coin and peek at it, the probability of heads is different for you than for me. I am counting bits that must be transferred from carbon to silicon through slow channels made of human flesh. On Tue, Sep 17, 2024, 10:52 AM James Bowery <jabow...@gmail.com> wrote: > > > On Mon, Sep 16, 2024 at 2:26 PM Matt Mahoney <mattmahone...@gmail.com> > wrote: > >> As I explained in my 2013 paper ( https://mattmahoney.net/costofai.pdf > > > The closest you come to a rigorous definition of "knowledge" is Table 2. > It would be helpful to be more careful in using that term in statements > such as: > > "Third, it is fundamentally impossible for a program to increase its own > knowledge ..." > > For example, Newtonian mechanics can *compute* "knowledge" derived from > collective behaviors like fluid mechanics, without a reductio ad absurdum > of a computer the size of the universe. > *Artificial General Intelligence List <https://agi.topicbox.com/latest>* > / AGI / see discussions <https://agi.topicbox.com/groups/agi> + > participants <https://agi.topicbox.com/groups/agi/members> + > delivery options <https://agi.topicbox.com/groups/agi/subscription> > Permalink > <https://agi.topicbox.com/groups/agi/T11c1e7172d92d3cd-Me8f3739c01c99b55edee8da6> > ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T11c1e7172d92d3cd-M46b6f23c89a47f9d82f017e3 Delivery options: https://agi.topicbox.com/groups/agi/subscription
