Ben,
OK - so Novamente has a system for
handling 'importance' already and
there is an importance updating function that feeds back to other aspects of
Attention Value. That's good in terms of Novamente having an internal
architecture capable of supporting and ethical system.
> You're asking
the AGI to solve the inverse problem: Find the concept
> that is consistent
with these descriptions and associations, and then
> embody that concept
in your own behavior. I think this is a very hard
> learning problem
.... which presumably means that it
will be put off until the AGI has the
capacity to undertake the leaning process. So why is this a problem?
> and the AGI might
will come up with something subtly but dangerously
> twisted.... I
don't trust this 1/1000 at much as experience-based
> learning.
But it's not either/or - under the
approach that I've suggested, Novamente
would have an itch to learn about certain ethical concepts AND it would
gain experience-based learning - so if experience-based learning is so good
why wouldn't it help Novamente to handle it internal itch-driven learning
without the subtle but dangerous twisting that you fear?
And anyway why would your pure experience-based
learning approach be
any less likely to lead to subtly but dangerously warped ethical systems?
The trainers could make errors and a Novamente's self-learning could be
skewed by the limits of its experience and the modelling it observes.
> Hmmmm.....
I am not certain, but I don't have a good feeling about it.
> I think
it's fine to stimulate things related to compassion and
> inhibit things
opposed to it, but, I think this will be useful as a
> *guide* to an
experience-based learning process, not as a primary means
> of stimulating
the development of a sense of compassion.
Your approach to ethics seems to be
based almost 100% on learning and
you seem to thing that your own team will be training all the Novamentes
before they leave the sandbox. How can you guarantee that your team will
always be the trainers and the quality standards will always be maintained?
For example, why couldn't someone
outside your group get a copy of a
Novamente and just strip out the learned data and then retrain the new
copy of Novamente themselves?
---
Getting back to your basic preferred
concept of using expience-based
learning to build a Novamente's ethical beliefs - this means that every
Novamente has to start as a tabula rasa and in effect learn all the lessons
of evolution all by itself.
With anything less intelligent than
a human-equivalent Novamente this
would be a highly inefficient approach. But with something as intelligent as
human-equivalent Novamente this a hugely dangerous strategy.
Given that ethics were not hard wired
into early animals - you have to ask
why this hard wiring eventually emerged. My guess is that as animals
became more powerful and potentially dangerous to their own kind it was
only the ones with inbuilt ethics that could be activated soon after birth that
were safe enough to survive and pass on their genes.
In other word the lesson of evolution
was that evolutionary recapitulation
could not be relied on to get each animal to a point where it was safe for its
fellows.
----
Just as an aside, it seems that autism
is a condition caused by problems
with a human's pre-wired empathy system. According to your preferred
approach to GI training it should only be a matter of training human GI in
ethics and empathy. Why then does autism exist as a problem since 99%
of autistic kids are put through a major training program by parents and
others to get them to relate socially? I simply can't see why a Novamente
that is without a modicum of ethical hardwiring will not end up being
autisitic - no matter how good the training program you might give it.
Why will your Novamentes not be autistic
- despite the training regime that
you intend?
----
At this stage in the discussion on
the AGI list I haven't heard anything to
convinve me that a certain amount of ethical pre-wiring is certain to cause
problems that are any greater than the problems that could be caused by
NOT having a modicum of carefully designed ethical hardwiring.
----
You have said many times that we need
to suck it an see through
experiment - that the theory of AGI psychological development is too
underdeveloped because we don't know what we are dealing with.
So why not proceed to develop Novamente's
down two different paths
simultaneously - the path you have already designed - where experience-
based learning is virtually the only strategy, and a variant where some
Novamentes have a modicum of carefully designed pre-wiring for ethics?
Then you've got some experiential
basis for comparing the two proposed
strategies - and quick corrective action will be easier if one or other strategy
shows signs of running into problems.
And less negatively, there might be
useful things learned from both
strategies than can be cross fertilised even if major catastrophic problems
don't appear to be emerging.
Would you be prepared to give the
dual (simultaneous) development path
approach a go?
Cheers, Philip
