Joshua Fox wrote:
On the subject of the adoption, or not, of radical new directions, I'm
now reading Richard Rhodes' _The Making of the Atom Bomb_ through
Singularitarian eyes.
It is instructive to consider the analogies--and crucial
differences--between the development of nuclear technology in the
first half of the twentieth century, and AGI in the first half of the
twenty-first.
References:
http://www.sl4.org/archive/0510/12520.html
http://www.sl4.org/archive/0605/14689.html
http://www.mail-archive.com/agi@v2.listbox.com/msg03251.html
http://www.singinst.org/ourresearch/publications/cognitive-biases.pdf
page 20
Similarities
- From the very beginning, science and science fiction which fed into
each other in predicting the Utopia of unlimited energy and the hell
of world destruction which nuclear energy could bring. Some understood
the existential risks and rewards; others just saw nuclear energy as a
science or a technology.
- The boundaries of physics arouse quasi-spiritual feelings more than
most fields of science.
- Even though from the first there was an understanding of the risks
of nuclear science, Szilard had to run around in late 1938 imploring
everyone not to spill the secrets. He had to pursue wealthy sponsors
for private scientific funding.
- Leading scientists, including some who were without doubt
intellectually courageous, did not believe that the chain-reactions
were possible.
- The development went slowly, but the chain reaction went foom!
- It seems to me that there was ongoing acceleration in scientific
development on the way to the atom bomb.
- Nuclear physics entered its home stretch to the bomb with a rising
sense of urgency, just as a world war was brewing. Of course, no one
knows if the struggle against today's extremist totalitarian
anti-Semitic ideology is the precursor to a full-blown world war.
Differences
- Nuclear physics was the rock-star science of its time. Einstein was
_the_ icon of the genius scientist. AGI doesn't seem to have anyone
with this top pop-icon status. But perhaps Pinker, in cognitive
science, is close.
- Although some very intelligent people go into computing and AGI, it
doesn't have the automatic draw for geniuses that cutting-edge physics
did then and still does today.
- Nuclear physics had the full support and funding of the scientific
establishment.
- In nuclear physics, there was never a roller-coaster of excessive
promise followed by temporary disappointment, slowing down subsequent
work, comparable to "AI Winter." (Dare I say "Nuclear Winter"?)
- The Singularity's promise and peril is much greater than that of
nuclear energy.
- Nuclear energy never fulfilled its Utopian promise. Let's hope that
AGI does.
- Nuclear-reaction skeptics could reasonably have said that fission
chain reactions are seen nowhere on earth, and would require extremely
rare materials and massive hardware,.and that therefore that
foreseeable technology might not be able to achieve physical
phenomena known only in astronomy. Intelligence, on the other hand,
exists in 6.5 billion meat blobs on earth .
I think the most important difference is missing from your list.
The superintelligence that is crucial to the Singularity cannot be
considered a "technology" like any previous one: all the technology
that has come before, without exception, was something that people could
decide to *use* in this way or that .... it has no say in how it is
used. That means everything created previously has always been open to
abuse by stupid (albeit sometimes well-meaningly stupid) people.
Superintelligence is not something that anyone could pick up and use in
whatever way they want. It has ideas of its own.
The arguments about exactly what that means are very complex, of course,
but no matter where you stand on the question of how a superintelligence
would react, you are never going to be able to pick it up like a missile
and throw it at someobody. We are talking about not being in Kansas
anymore in a seriously big way.
Richard Loosemore.
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=11983
