James Rogers wrote:
I'm not sure whether your definition of AIC precludes this, but it is possible for a small physical system to perfectly model a large physical system providing that the large physical system possesses perfect, large regularities such that its state can be fully represented within the small regularities of the small physical system.
Your intuition is correct, depending on how strict you are about
"knowledge". The intrinsic algorithmic information content of any
machine is greater (sometimes much greater) than the algorithmic
information content of its static state. The intrinsic AIC doesn't
change even though the AIC of the machine state may. For this reason,
it is not possible for a machine with a smaller AIC to perfectly model a
machine with greater or even equal AIC. By extension, it is also not
possible to have perfect self-knowledge. It is a common misapplication
and/or misunderstanding to interchangeably use the intrinsic AIC of a
machine with the AIC of the machine's state; I'm not saying that is
happening here, but I see it regularly in other less rigorous forums and
so it is worth bringing up.
All this does not preclude a smaller machine from having a very good
predictive model of a larger machine. Just not a perfect one.
It has recently occurred to me that "fragile" perfect self-knowledge may be possible if a system deliberately configures itself to avoid the kind of dilemnas that are usually used to demonstrate the impossibility of perfect self-knowledge - if the system makes itself a Henkin sentence rather than a Godel sentence. It would not be knowably perfect self-knowledge, but it would be actually perfect self-knowledge. I haven't essayed an actual proof, though, so I may be mistaken.
Though I also think that nonfragile perfect self-knowledge is probably impossible, I would agree with Ben in not trusting this belief. I have found that I have a very poor grasp of what is "impossible" or even what is "too difficult for human-level intelligence", in the sense that often a relatively small improvement in knowledge will show me how to do things that I previously thought were humanly absurd or ontologically impossible. I mind the time that I posted to the Extropians list about the absolute impossibility of encoding messages into pi without Type IV godhood and then posted a retraction less than 48 hours later. The people who are absolutely sure that not even a transhuman AI could persuade them to let it out of the box are another case in point.
In general, my experience suggests that it is *very very* hard to figure out what I will think is "impossible" or "too difficult for humans" even tomorrow, let alone what an SI will think is impossible. "Impossible" or "too difficult for humans" are simply not useful terms because they are indistinguishable in their application from "Eliezer Yudkowsky doesn't know how to do this as of today." I have not yet found any way to firmly predict which items in this class as of January 10th will still lie within it on January 11th, and that makes the subclasses "impossible" or "too difficult for humans" unhelpful. Now if only some AI theorists would start distrusting their intuitions that figuring out certain parts of AI theory in advance is "too difficult for humans"...
--
Eliezer S. Yudkowsky http://singinst.org/
Research Fellow, Singularity Institute for Artificial Intelligence
-------
To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/?[EMAIL PROTECTED]
