Dave Harris wrote: > I started working up the image for the waybak VizANN earlier this week. Once > I have that, I can start on the simulator coding. > > I will be on "vacation" next week (I will be programming at home instead of > at the office.) I plan to spend some time on ANNEvolve. > > Shortly after writing wbreduce and observing the results, I started reading > "The Three-Pound Enigma". The book contains an interview with Crick (a few > months before he died) and Koch. Crick mentioned that his transition from > biochemistry to consciousness studies began with a visit to Tomaso Poggio, > who was studying dendritic spines at the time. The book notes that there is > some evidence the spines are related to learning. > > I visualized a crude analogy where the connections eliminated by wbreduce > were similar to the spines that fade away when learning 'settles'. If > wbreduce ran periodically during the 'runmany' script, it would speed up the > processing (if the connection count were actually reduced), making 'better' > use of CPU time. > > Likewise, connections could be restored (as spines grow) as a part of the > reverse process, nudging the ANN into new behavior. > > I haven't given it a whole lot of thought but I was thinking about the > possibility of having 'fuller-than-fully-connected' ANNs (to correspond to > burgeoning dendritic spine growth.) Obviously, multiple connections with > simple weights would not be useful - superposition would collapse the > connections into a single weight. No, something non-linear would have to be > introduced. <snip>
I agree that there is a valid analogy between the way a brain eliminates synaptic connections that are not needed, and wbreduce. Unfortunately, our present ANN computation method, using our "chromosome", doesn't process any faster if some of the weights are zeroed. These zeroes still appear in the chromosome, and still participate in the calculations. We don't currently have a way to reduce the chromosome size to eliminate zero values. I was thinking about a scheme using a bit mask to create a sparsely connected ANN, but it has the disadvantage of slower processing, because for every potential connection the bit mask needs to be examined to see if the connection exists. m ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
