On Wed, Mar 14, 2007 at 12:43:01PM -0500, J. Storrs Hall, PhD. wrote: > Besides writing books, Kurzweil builds systems that work.
No arguing with that (though his system-building seems to be all in the past, and self-promotion very much in the present), but he doesn't do AI that works and neither does he do neuroscience that works. As such he has about the same standing as the noted chiromant Jeff Hawkins: pretty damn little. > > I'm not actually just being flippant, the AI crowd has a rather > > bad case of number creep as far as estimates are concerned. > > You can assume 10^23 ops/s on 10^17 sites didn't come out of the > > brown, er, blue. > > I find it completely ridiculous. 1e30 ops total works out to 1000 cubic > meters Allright, that's a reaction. Here's where 10^23 ops/s comes from: there are some 10^11 cells, each of which has about 10^6 computing elements (yes, Virginia, not just the synapse), which each process events at about 10^3 Hz, of which it takes about 10^3 garden-variety machine primitives to compute. I'm going to assume the system runs on a 3d node grid, which makes communication at each step completely local -- which computational neuroscientists (which work with PDEs and large matrices, which scale like crap) would think is completely nuts. The 10^17 sites come from 10^11 cells with 10^6 computing elements each. A site is some 100 bits, probably less, maybe more. > of Drexler nanocomputers. Knowing the relative power density of bio to The reason Drexler proposed scaling down the Difference Engine is not because he considered them practical, but because they're easy to analyze. I'm not sure why you're looking at 10^30 ops btw, my number is some 10^23. He mentions (this is from the web, I'm too lazy to pick up Nanosystems) 10^20 bits/cm^3, which is about the ballpark of 10^17 sites required. That's just the bits, of course, not the crunch. > engineered nanosystems as I (and you) do, I claim that there's no way the > brain can be doing anywhere close to the actual, applied calculation of even > its own volume in nanocomputers, much less a million times as much. The point is that we don't know how exactly the brain does it, so I put some reasonable numbers of hardware required to roughly track what a given biological system is doing. There is still some uncertainty about what unconventional computing can do (that's Chapter 20 in Biophysics of Computation), so I don't claim these numbers are scripture in any way. It could be more, quite a bit more, but most likely it's an upper bound. I would very much wish it was an upper bound that is way too high, because Moore buys you an order of magnitude every 6 years (though commodity clusters go beyond Moore, because of the economies of scale, though not in bottleneck issues like memory bandwidth). > I would confidently undertake to simulate the brain at a level where each > cell > is a system of ODE's with 1e30 ops -- give me 1e36 and I'll give you > molecular dynamics. I think the neuroscientists are confusing what it takes It's not just the ops here, you'll need a lot of storage for atomically accurate representation, too, and here you're running into many m^3 for MD. > to simulate a system and the the amount of useful work it performs. But to find out what useful work it performs you need to run a chunk of the system in the simulation at a rather low level of theory. (Including MD and downstairs, if you want to obtain the switching parameters of an ion channel from first principles -- which you then of course can abstract into a black box, and so further up the hierarchy level). > Using a ratio of 1e6 for power density/speed of a nanoengineered system to > bio, which is fairly low for some of the mechanical designs, the brain should > be doing about as much computation as a microliter of nanocomputers, or about If you'd say ml, I would be more inclined to agree. But at ul, you can't even fit the bits in, nevermind the ops. And ops alone are meaningless, see the 300 nm 40 nm wafer populated by ring oscillators. > 1e18 ops. Given the error bars for this estimation method, i.e. several > orders of magnitude, I'd say this matches Kurzweil's number fine. Then something is obviously severely wrong with Kurzweil's numbers. I keep mentioning number creep, look at http://www.molecularassembler.com/KSRM/5.10.htm Minsky thought 10^6 bits would be enough. So excuse me if I'm a bit careful about those bits straight out of the brown. > I'll just have to wait till those damned lab rats get nanotech working. That > microliter machine (= 1 cubic millimeter) should have ~1e18 B/s memory > bandwidth. If that won't do it we can vary the design to use CAM and get 1e24 > compares per second. But I doubt we'll need it. A 300 mm wafer of 45 nm ring oscillators has a data rate hard to beat. But the rate of bit twirling is not intelligence, not yet. And it would take more than an array of adders or even a multipliers (nobody seems to mention much that meek parts of neurons can do a very nice analog multiply) to do it. > Back to the present: "Amdahl's Rule of Thumb" puts memory size and bandwidth > equal to ops per second for conventional computer systems. I conjecture that > an AI-optimized system may need to be processor-heavy by a factor of 10, i.e. > be able to look at every word in memory in 100 ms, while still being able to > overlay memory from disk in 1 sec. We're looking at needing memory the size > of a very average database, but in RAM. I don't quite follow you here, but at 20 GBytes/s (best case) you'd get an about ~10 Hz refresh rate on your memory/node. Interestingly enough, with ~GBytes and direct-neighbour cube interface GBit Ethernet is quite enough. You only need faster interconnects when you have less memory/node, so your processing rate is >>10 Hz. Machines a la Tera Scale can do much better, provided you can package enough embedded memory. > Bottom line: HEPP for $1M today, $1K in a decade, but only if we have > understood and optimized the software. Do you think what your brain does (what is not require for housekeeping) is grossly inefficient, in the terms of operations, not comparisons to some semi-optimal computronium, and it can be optimized (by skipping all those NOPs, probably)? I'm not quite that confident, I must admit. I'm however quite confident that there is no simple theory lurking in there which can written down as a neat set of equations on a sheet of paper, or even a small pile of such. So there's not much to understand, and very little to optimize. > Let's get to work. Excellent idea. -- Eugen* Leitl <a href="http://leitl.org";>leitl</a> http://leitl.org ______________________________________________________________ ICBM: 48.07100, 11.36820 http://www.ativel.com 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE ----- This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
signature.asc
Description: Digital signature
