On 10/27/07, Stefan Pernar <[EMAIL PROTECTED]> wrote:
> On 10/27/07, Aleksei Riikonen <[EMAIL PROTECTED]> wrote:
>
>> This is getting ridiculous. As repeatedly stated in this discussion,
>> there is nothing circular about a sequence of steps of the following
>> sort:
>>
>> (1) A superintelligent AI such that it doesn't start killing humans or
>> other nasty things, is created.
>> (2) The one thing that the AI *does* do, is start asking humans what
>> they want to be done. As a part of this asking process, humans (and/or
>> simulated copies of them) are made smarter and more knowledgeable, and
>> are changed in other ways (if any) that the humans want to be changed
>> in.
>> (3) Eventually the process of humans (and/or their copies) getting
>> significantly smarter, more knowledgeable etc stops, and the answers
>> the humans give to questions such as "what should be done?" and "what
>> is 'good'?" become stable.
>> (4) Then the AI just does what the humans want to be done. Happy end,
>> no circular loop.
>
> Aleksei, as it stands there are two interpretations of CEV:
>
> 1) "What would a better humanity want" This I criticize as being circular as
> the system would need a definition for 'good' in order to model a better
> humanity.
> 2) "Let's survey humanity for what it wants" This I criticize as being an
> elaborate way of asking the question of what is 'good' as opposed to
> answering what is good itself. I prefer to coming up with a good answer to
> the question if you will instead of merely asking it.
You seem to have a need to personally give a final answer to "What is
'good'?" -- an answer to what moral rules the universe should be
governed by. If you think that your answer is better than what the
"surveying" process that CEV is would produce, I think your attitude
amounts to delusions of grandeur.
I find it obvious that whatever answer you give, it isn't better than
the answer the smarter and more knowledgeable humans, who are surveyed
in the CEV process, would give. I find it infinitely more preferable
to find out what they would say ("they" would include wiser versions
of you and me), instead of taking whatever the current you says as the
final answer to this question that all the smartest human philosophers
have tried to solve for all of human history, without coming up with
an answer that could be considered to settle the issue.
--
Aleksei Riikonen - http://www.iki.fi/aleksei
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?member_id=4007604&id_secret=58075515-f0452e
