Kaj Sotala wrote:
On Jan 29, 2008 6:52 PM, Richard Loosemore <[EMAIL PROTECTED]> wrote:
Okay, sorry to hit you with incomprehensible technical detail, but maybe
there is a chance that my garbled version of the real picture will
strike a chord.
The message to take home from all of this is that:
1) There are *huge* differences between the way that a system would
behave if it had a single GS, or even a group of conflicting GS modules
(which is the way you interpreted my proposal, above) and the kind of
MES system I just described: the difference would come from the type of
influence exerted, because the vector field is operating on a completely
different level than the symbl processing.
2) The effect of the MES is to bias the system, but this "bias" amounts
to the following system imperative: [Make your goals consistent with
this *massive* set of constraints] .... where the "massive set of
constraints" is a set of ideas built up throughout the entire
development of the system. Rephrasing that in terms of an example: if
the system gets an idea that it should take a certain course of action
because it seems to satisfy an immediate goal, the implications of that
action will be quickly checked against a vast range o constraints, and
if there is any hint of an inconsistency with teh value system, this
will "pull" the thoughts of the AGI toward that issue, whereupon it will
start to elaborate the issue in more detail and try to impose an even
wider net of constraits, finally making a decision based on the broadest
possible set of considerations. This takes care of all the dumb
examples where people suggest that an AGI could start with the goal
"Increase global happiness" and then finally decide that this would be
accomplished by tiling the universe with smiley faces. Another way to
say this: there is no such thing as a single "utility function" in this
type of system, nor is there a small set of utility functions .... there
is a massive-dimensional set of utility functions (as many as there are
concepts or connections in the system), and this "diffuse" utility
function is what gives the system its stability.
I got the general gist of that, I think.
You've previously expressed that you don't think a seriously
"unfriendly" AGI will be likely, apparently because you assume the
motivational-system AGI will be the kind that'll be constructed and
not, for instance, a goal stack-driven one. Now, what makes you so
certain that people will build a this kind of AGI? Even if we assume
that this sort of architecture would be the most viable one, a lot
seems to depend on how tight the constraints on its behavior are, and
what kind they are - you say that they are a "a set of ideas built up
throughout the entire development of the system". The ethics and
values of humans are the result of a long, long period of evolution,
and our ethical system is pretty much of a mess. What makes it likely
that it really will build up a set of ideas constraints that we humans
would *want* it to build? Could it not just as well pick up ones that
are seriously unfriendly, especially if its designers or the ones
"raising" it are in the least bit careless?
Even among humans, there exist radical philosophers whose ideas of a
perfect society are repulsive to the vast majority of the populace,
and a countless number of disagreements about ethics. If we humans
have such disagreements - we who all share the same evolutionary
origin biasing us to develop our moral systems in a certain direction
- what makes it plausible to assume that the first AGIs put together
(probably while our understanding of our own workings is still
incomplete) will develop a morality we'll like?
Perhaps we make too much of the idea of "moral and ethical." As noted,
this leads to endless debate. The alternative is to use "law" even
though it may be arbitrary and haphazard in formulation.
The importance of law is that it establishes "risk." As humans we
understand risk. Will an AI understand risk? Or, should we rephrase
this to read "will there be a risk for an AI?"
examples of what an AI might risk...
1. banishment - not allowed to run. No loading into hardware
2. isolation - prevention of access to published material or experimentation
3. imprisonment - similar to isolation, with more access than isolation
4. close supervision - imposing control through close supervision,
constant oversight, actions subject to approval...
5. economic sanction - not allowed to negotiate any deals or take
control of resources.
I expect Matt Mahoney to point out that "resistance is futile," the AI's
will outsmart us. Does that mean that "criminals" will ultimately be
smarter than non-criminals? Maybe the AI's of the future will want an
even playing field and be motivated to enforce laws.
I see Richards design as easily being able to implement risk factors
that could lead to intelligent and legal behavior. I'm impressed by the
design. Thanks for the explanation.
Stan Nilsen
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?member_id=8660244&id_secret=91507422-909adf
