Dr. Omni wrote:
In particular cases a less intelligent entity is perfectly able to
predict the behavior of a more intelligent one. For instance, my cats
are less intelligent than me (or so I hope ;-) and they can predict
several of my actions and take decisions based on that. For instance
"Lúcio has finished dinner and so he will not be at the kitchen
anymore tonight, so I should better meow for more food".
I guess they can predict that based on previous cases - countless
times that I finished dinner, turned the kitchen light off and went to
my bedroom. Which by the way may hint at a way to predict (in the same
cat-like statistical way) the friendliness of an AI:
- Start the AI inside a virtual environment approximating reality, but
don't tell the AI that it's virtual.
- Observe a significant number of the AI actions (and reactions) in
that virtual reality.
- If the AI is considered friendly, then restart it, this time in a
real environment.
Yes, I agree....
However, the problem is that your cat would probably do a BAD job of
predicting your reaction to an environmental situation that was
qualitatively different from anything you or your cat had ever
experienced before..
This is the problem that the "dynamical complexity of the world" poses
for provably Friendly AI.
The universe we are embedded in appears to be highly unpredictable;
chaos and complexity theory are, among other things, attempts to
quantify this unpredictability and comprehend its nature...
We may be able to satisfy ourselves regarding the behavior of a
superhuman AI in environments relatively similar to familiar ones --
via a combination of mathematical, empirical and subjective means, as
you say...
But when an unpredictable world-situation comes up, then empirics and
subjectivity can't be relied upon. Here is where we would like to
have some kind of mathematical guarantee ... which however, I really
doubt is possible.
The basic problem as many have noted is Godelian. Chaitin's version
of Godel's Theorem says "You can't prove a 20 pound theorem with a 10
pound axiom system." We humans cannot prove theorems about things
that are massively more algorithmically complex than ourselves. And,
even if the software code for a superhuman AI is not that tremendously
complex compared to ourselves, the universe itself probably is... and
to prove the ongoing Friendliness about an AI we would likely have to
prove something about how the AI would react to our particular
universe, complex as it is... which is not possible due to the
universe's complexity and our simplicity...
This argument by reference to Godel and the complexity of the universe
is NOT a rigorous proof that Friendly AI is impossible ... it's just a
heuristic argument that it is most likely impossible.
Neither Shane nor I nor anyone else, so far as I know, has proved that
provably Friendly AI is necessarily impossible according to physical
law and mathematical truth. But IMO, when there are such strong
heuristic arguments that something is impossible, the burden of proof
is on those who claim it is possible....
This is not like the past case where narrowminded scientists argued
that airplanes were impossible ... in that case, we had birds as a
proof that at least some other animals could fly... there was meat for
heuristic arguments in both directions. I do not see what is the
argument that provably Friendly AI is possible except "Wouldn't it be
great if it were!", and I see many heuristic arguments that it is not
possible....
But yet, we proceed without rigorous proof in all other aspects of
life and technological development, based on subjective and empirical
considerations.... Presumably we will do so in the domain of AGI as
well, taking whatever empirical proof we can find along the way...
-- Ben G
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/[EMAIL PROTECTED]
