Keith Elis wrote:
Answer me this, if you dare: Do you believe it's possible to design an
artificial intelligence that won't wipe out humanity?
Yes, most certainly I do.
I can hardly stress this enough.
Did you read my previous post on the subject of motivation systems?
This contained much of the idea. I am getting close to the point where
I might have enough spare time to write this into proper form and get it
online, but in the mean time it does exist in the Singularity list archive:
http://www.mail-archive.com/singularity@v2.listbox.com/msg00316.html
The core of the argument for safety is that the types of future AGI
system being discussed now are based on an extrapolation of the
"canonical" AI design of today ... including the egregious flaws in that
canonical design; and including the same flaws that are preventing us
from actually building a generally intelligent system.
The flaw I have in mind (there are many, but this is one of the biggest)
involves the mechanism that drives the system to do what it does.
Currently, this mechanism is assumed to be an extrapolation of the "goal
stack" idea, in which goals are represented as explicit statements in
some logical language. For this to work, the system has to interpret
the statements: it has to know what the terms in the statements mean
and it has to understand how those terms combine to yield the meaning of
the statement. This works (kind of) for narrow AI but is worse than
useless for a real AGI: how the system interprets the meaning of an
abstract statement is totally out of the control of the researcher, and
when the system is growing up (as a real AGI must do, whereas narrow AIs
never do this), it cannot use sophisticated concepts before it has
learned them, so it has to make do with only concepts interpretable by a
baby.... clearly a ridiculous situation.
All in all, there is a need for a more sophisticated motivational
system. When you look into how to do that, it becomes clear that you
should never try to control an AGI with single statements (like "Be
Friendly to all humans and try to help them get what they want") because
this type of control is single-point-of-failure control .... it's a
joke. Cannot ever be made stable.
Instead, what you do is build the motivational system in such a way that
it must always operate from a massive base of thousands of small
constraints. A system that is constrained in a thousand different
directions simply cannot fail in a way that one constrain by a single
supergoal is almost guaranteed to fail.
There is more to the argument than that, of course, but the bottom line
is that when I see arguments relating to what an an AGI "would" do, I am
beside myself with frustration: these are almost always based on the
assumption that the AGI is governed by a crude goal-stack motivational
system, which (a) will probably never yield an AGI that is smart enough
to be a threat (that's why narrow AI is so stupidly useless), and (b) is
so wildly unstable that nobody would try to use it anyway, because thety
will use a broad-based massive-constraint system instead.
The massive constraint system can be designed, I believe, in such a way
that the probability of it going AWOL could be made so low as to be
negligible..... we are talking about a system with as much likelihood of
doing something outside its initial (friendly) motivation as the
likelihood of the sun suddenly qunatum tunneling to the vicinity of
Betelgeuse. If that isn't stable enough for people, I don't know what
would be.
The stupid part of this is that, as you probably know, I tried to get
this argument discussed on SL4 and ran straight into Yudkwosky's gang.
Then I tried to get it discussed here, and there was a but of talk, but
nothing much. Amazing, for a topic that generates so much angst.
I sometimes think people actually would prefer there to be a doomsday
scenario, because they like to be scared, or they want to always believe
the worst. Solutions to scary problems seem ... boring?
Richard Loosemore
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?member_id=4007604&user_secret=7d7fb4d8
