On Wed, Oct 26, 2011 at 12:53:24AM -0700, BGB wrote: > from what I read, IBM was using a digital crossbar.
It sounds like Kwabena Boahen (Carver Mead's school) is on the right track http://web.cecs.pdx.edu/~strom/onr_workshop/boahen.pdf the group seems to be still publishing http://www.stanford.edu/group/brainsinsilicon/pubs.html >>> I suspect something more "generic" would be needed. >> I don't see how generic will do long-term any than for bootstrap >> (above co-evolution) reasons. > > more generic is more likely to be able to do something interesting. More generic hardware (not optimized for a particular model) also means it's less efficient. On the other hand, we don't have a particular model to optimize for, so right now generic is the way to go. > ARM or similar could work (as could a large number of 386-like cores). > > I had considered something like GPGPU, but depending on the type of > neural-net, there could be issues with mapping it efficiently to > existing GPUs. You could wire it up in a 3d grid, with offsets to mostly local other grid sites and stream through memory. Refresh rate could be 10 Hz or more, depending on how complex the model, and how much random-access like memory accesses you're producing. > also, the strong-areas for GPUs are not necessarily the same as the > requirements for implementing neural nets. again, it could work, but it If you can do plenty of parallel ops on short integers (8-16 bit) then it seems to match well, provided you can feed the shaders. Mapping things to memory is tricky, as otherwise you'll starve the shaders and thus underutilize the hardware. > is just less certain it is "ideal". > > >>> the second part of the question is: >>> assuming one can transition to a purely biology-like model, is this a >>> good tradeoff?... >>> if one gets rid of a few of the limitations of computers but gains some >>> of the limitations of biology, this may not be an ideal solution. >> Biology had never had the issue to deal with high-performance numerics, >> I'm sure if it had it wouldn't do too shabbily. You can always go hybrid >> e.g. if you want to do proofs or cryptography. > > biology also doesn't have software distribution, ability to make > backups, ... Even hybrid analog systems can be converted into a large blob of binary data and loaded back. The loss of precision due to digitization is negligible, as the system will be noisy/low precision (<6 bit) to start with. > ideally, a silicon neural-net strategy would also readily allow the > installation of new software and creation of backups. You can always digitize and serialize your data, put it through a pipe and reinstantialize it in equivalent piece of hardware on the other end. Consider crystalline computation (Margolus/Toffoli) http://people.csail.mit.edu/nhm/cc.pdf which maps very well to 3d lattice of sites in future's molecular hardware. If you can halt the execution, or clone a static shadow copy of the dynamic process you can serialize and extract from the faces of the crystal at leisure. > > the most likely strategy here IMO is to leverage what existing OS's can > already do, essentially treating any neural-net processors as another > piece of hardware as far as the OS is concerned. I'm not sure you need an OS. Many aspects of operation can be mapped to hardware. Consider a 10 GBit/s short reach fiber, the length of fiber through these keystrokes are passing is an optical FIFO conveniently containing a standard MTU (1500 Bytes) frame/packet. Same thing does vacuum to a sufficiently fast line of sight laser link. With the right header layout, you can see how you could make a purely photonic cut-through router or switch, with zero software. Same thing with operating Toffoli's hypothetical computronium crystal, you have stop and go, read and write, and that's it. I/O could be mapped directly to crystal faces. > this would probably mean using neural-net processors along-side > traditional CPU cores (rather than in place of them). > > >>> better would be to try for a strategy where the merits of both can be >>> gained, and as many limitations as possible can be avoided. >>> >>> most likely, this would be via a hybrid model. >> Absolutely. Hybrid at many scales, down to analog computation for neurons. > > yeah. > > analog is an idea I had not personally considered. > > I guess a mystery here is how effectively semiconductor logic (in > integrated circuits) can work with analog signals. > > the main alternative is, of course, 8 or 16-bit digital signals. 8 or 16 bit ALUs are ridiculously complicated, power-hungry and slow if compared with analog computation in transistors or memristor. Of course the precision is low, particularly if you're working at nanoscale. A few atoms displaced shift the parameters of the device visibly, but then, you're engaging a redundant mode of computation anyway. The brain does not just tolerate noise, some aspects of it are noise-driven. > > I had more imagined as hybrids of neural-nets and traditional software. > > granted, there is always a certain risk that it could mean, in some > distant-future setting, people sitting around with Windows running in > their heads (say, because they have incorporated processors and silicon Security issues alone would be a killer. Consider buffer overruns in the optical FIFO I mentioned, how do you cause them? You can't, obviously, since the laws of the unverse instead of limits of human design are asserting correct operation. > neural-nets into their otherwise wetware brains, that or parts, or much, > or all, of their brain functionality has been migrated into silicon). Right spintronics in graphenes looks good. No other good candidates for computronium, apart from self-assembled macromolecular crystals (like viral capsids, only containing CA cells, designed to link up and connect, unlike real virus capids, who don't really want to, unless coaxed by crystallographers). > > or such... -- Eugen* Leitl <a href="http://leitl.org";>leitl</a> http://leitl.org ______________________________________________________________ ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE _______________________________________________ fonc mailing list fonc@vpri.org http://vpri.org/mailman/listinfo/fonc
