Matt Mahoney wrote:
--- "John G. Rose" <[EMAIL PROTECTED]> wrote:
Is there really a bit per synapse? Is representing a synapse with a bit an
accurate enough simulation? One synapse is a very complicated system.
A typical neural network simulation uses several bits per synapse. A Hopfield
net implementation of an associative memory stores 0.15 bits per synapse. But
cognitive models suggest the human brain stores .000001 bits per synapse.
(There are 10^15 synapses but human long term memory capacity is 10^9 bits).
Sorry, I don't buy this at all. This makes profound assumptions about
how information is stored in memory, averagng out the "net" storage and
ignoring the immediate storage capacity. A typical synapse actually
stores a great deal more than a fraction of a bit, as far as we can
tell, but this information is stored in such a way that the system as a
whole can actually use the information in a meaningful way.
In that context, quoting "0.000001 bits per synapse" is a completely
meaningless statement.
Also, "typical" neural network simulations use more than a few bits as
well. When I did a number of backprop NN studies in the early 90s, my
networks had to use floating point numbers because the behavior of the
net deteriorated badly if the numerical precision was reduced. This was
especially important on long training runs or large datasets.
Richard Loosemore
-------------------------------------------
singularity
Archives: http://www.listbox.com/member/archive/11983/=now
RSS Feed: http://www.listbox.com/member/archive/rss/11983/
Modify Your Subscription:
http://www.listbox.com/member/?member_id=4007604&id_secret=96140713-a54b2b
Powered by Listbox: http://www.listbox.com
