Ben,
> Ben: That paragraph
gave one possible dynamic in a society of AGI's,
> but there are
many many other possible social dynamics
Of course. What you say is quite
true. But so what?
Let's go back to that one possible
dynamic. Can't you bring yourself to
agree that if a one-and-only super-AGI went feral that humans would
then be at a greater disadvantage relative to it than if there was more
than one AGI around and the humans could call on the help of one or
more of the other AGIs??
Forget about all the other possible
hypotheticals. Is my assessment of
the specific scenario above about right or not - doesn't it have some
element of common sense about
it?
If there is any benefit
in having more than one AGI around in the case
where an AGI does go feral then your comment "I'm just not so sure
that there's any benefit to the
"society of AGI" as opposed to "one big
AGI" approach" no longer holds as an absolute.
It then gets back to ....having a
society of AGIs might be an advantage
in certain cercumstances, but having more than one AGI might have
the following down sides. At this point a balanced risk/benefit
assessment can be made (not definitive of course since we haven't
seen super-intelligent AGIs operation yet). But at least we've got some
relevant issues on the table to think about.
Cheers, Philip
