Kaj,
If I look closely at what you write, you are somewhat close to me - but you
are in fact not saying explicitly and precisely what I am.
I am saying upfront: language and symbol systems are general and abstract
and open to infinite, particular, concrete interpretations, including the
opposite of any interpretation you might prefer. It is therefore impossible
when programming an agent in general language or concepts, to control its
particular interpretations of those concepts - whether those concepts are
"help humanity" or "make humans happy" etc.
You don't say this upfront, and you do seem to imply that it might be
possible to control the agent sometimes, if not at others.
If you basically agree with my statement, then both your exposition and, I'm
sure, mine can be improved.
On 7/12/07, Mike Tintner <[EMAIL PROTECTED]> wrote:
Comment: You and others seem to be missing the point, which obviously
needs
spelling out. There is no way of endowing any agent with conceptual goals
that cannot be interpreted in ways opposite to the designer's
intentions -
that is in the general, abstract nature of language & symbolic systems.
For example, the general, abstract goal of "helping humanity" can
legitimately in particular, concrete situations be interpreted as wiping
out
the entire human race (bar, say, two) - for the sake of future
generations.
And there is no reasonable way to circumvent this. You couldn't, say,
instruct an agent... "help humanity but don't kill any human beings..."
because what if some humans (like, say Bush) are threatening to kill
vastly
greater numbers of other humans...wouldn't you want the agent to
intervene?
And if you decided that even so, you would instruct the agent not to
kill,
it could still as good as kill by rendering humans vegetables while still
alive.
So many people here and everywhere are unaware of the general and
deliberately imprecise nature of language - much stressed by Derrida and
exemplified in the practice of law.
I am confused, now. The sentence from my essay that you quoted was
from a section of it that was *talking about the same very thing as
you are talking about now*. In fact, had you not cut out the rest of
the sentence, it would've been apparent that it was talking exactly
*about* how "helping humanity" is too vague and ill-defined to be
useful:
"" An AI programmed only to help humanity will only help humanity, but
in what way? Were it programmed only to make all humans happy, it
might wirehead us - place us into constant states of pure,
artificially-induced states of orgasmic joy that preclude all other
thought and feeling. While that would be a happy state, many humans
would prefer not to end up in one - but even humans can easily argue
that pure happiness is more important than the satisfaction of desires
(in fact, I have, though I'm unsure of my argument's soundness), so
"forcibly wireheading is a bad thing" is not an obvious conclusion for
a mind."
There are many, many things that we hold valuable, most of which feel
so obvious that we never think about them. An AI would have to be
built to preserve many of them - but it shouldn't preserve them
absolutely, since our values might change over time. Defining the
values in question might also be difficult: producing an exact
definition for any complex, even slightly vague concept often tends to
be next to impossible. We might need to give the AI a somewhat vague
definition and demonstrate by examples what we mean - just as we
humans have learnt them - and then try to make sure that the engine
the AI uses to draw inferences works the same way as ours, so that it
understands the concepts the same way as we do. ""
Isn't this just what you're saying? The /entire section/ was talking
about this very issue.
--
http://www.saunalahti.fi/~tspro1/ | http://xuenay.livejournal.com/
Organizations worth your time:
http://www.singinst.org/ | http://www.crnano.org/ | http://lifeboat.com/
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?&;
--
No virus found in this incoming message.
Checked by AVG Free Edition. Version: 7.5.476 / Virus Database:
269.10.4/897 - Release Date: 11/07/2007 21:57
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?member_id=4007604&id_secret=19912796-c3a35e
