On 10/12/07, Eliezer S. Yudkowsky <[EMAIL PROTECTED]> wrote:
> some of us are much impressed by it. Anyone with even a surface grasp
> of the basic concept on a math level will realize that there's no
> difference between self-modifying and writing an outside copy of
> yourself, but *either one* i
Tim Freeman wrote:
My point is that if one is worried about a self-improving Seed AI
exploding, one should also be worried about any AI that competently
writes software exploding.
There *is* a slight gap between competently writing software and
competently writing minds. Large by human stand
Linas Vepstas:> > >Let's take Novamente as an example. ... It cannot improve
itself> > >until the following things happen:> > >> > >1) It acquires the
knowledge and skills to become a competent> > > programmer, a task that takes a
human many years of directed> > > training and practical experien
From: Derek Zahn <[EMAIL PROTECTED]>
>You seem to think that self-reference buys you nothing at all since it
>is a simple matter for the first AGI projects to reinvent their own
>equivalent from scratch, but I'm not sure that's true.
The "from scratch" part is a straw-man argument. The AGI projec
> >Let's take Novamente as an example. ... It cannot improve itself
> >until the following things happen:
> >
> >1) It acquires the knowledge and skills to become a competent
> > programmer, a task that takes a human many years of directed
> > training and practical experience.
Wrong. This wa
Derek, Tim,
There is no oversight: self-improvement doesn't necessarily refer to
actual instance of self that is to be improved, but to AGI's design.
Next thing must be better than previous one for runaway progress to
happen, and one way of doing it is for next thing to be a refinement
of previous
Tim Freeman:> No value is> added by introducing considerations about
self-reference into> conversations about the consequences of AI engineering.> >
Junior geeks do find it impressive, though.
The point of that conversation was to illustrate that if people are worried
about Seed AI exploding, th
Tim Freeman writes:> >Let's take Novamente as an example. ... It cannot improve
itself> >until the following things happen:> >> >1) It acquires the knowledge
and skills to become a competent> > programmer, a task that takes a human many
years of directed> > training and practical experience.> >
>From: Derek Zahn <[EMAIL PROTECTED]>
>Date: Sun, 30 Sep 2007 08:57:53 -0600
>...
>One thing that could improve safety is to reject the notion that AGI
>projects should be focused on, or even capable of, recursive self
>improvement in the sense of reprogramming its core implementation.
>...
>Let's
