Re: [agi] Growth of computer power
> Eugen Leitl Thu, 23 Jun 2005 02:18:14 -0700 > Do any of you here use MPI, and assume 10^3..10^5 node parallelism? I assume 2^14 node parallelism with only a small fraction computing at any time. But then my nodes are really smart memory rather than full-blown processors and not async yet. At this time I use a random access model for my simulation, but I realise this is not going to work in the hardware implementation and will need some form of message passing between nodes Although looking at MPI, this seems far too complex an implementation Numbers of connections between the different units is a tricky question, although I am thinking that a scale-free network may be a good compromise between minimising path length and (silicon) resource usage. Anybody know of any research on this in parallel computing? Will Pearson --- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Growth of computer power
Eugen wrote: > Unfortunately, shared memory is an expensive mirage > to maintain. It doesn't scale very well, due to > coherency issuess (if you have r/w intensive operations > on a memory block, you have to send lots of signals to > and fro until you know it's consistent). This takes lots > of time and logic, and is prohibitive for even > moderately-sized clusters (IIRC there is no shared > memory clusters with 10^3 nodes). No such problems occur > with message passing, as it reflects the nature of > physical reality. I believe current flavors of SGI's NUMALink may do 10^3 nodes, but at a very high cost and with memory latency that is about the same as a Quadrics fabric anyway. At that scale, it is a signal propagation problem. Good ccNUMA like Opterons give excellent results for the money up to about 8 processors. IIRC, an analysis of the first fabric shows that it will scale almost perfectly up to around 7 processors before trailing off. The next version of the Opteron fabric is supposed to double that. Shared memory is a mirage, but it is also very cheap up to a point. Using a big Opteron box has and order of magnitude less memory latency than using a good cluster, which translates into real performance, and shared memory code runs faster than MPI on the same system. Shared memory does not scale easily, but jumping across the chasm to MPI is not cheap either. Some problems (like mine) are so fine-grained that you would be sending a message every hundred machine instructions or so, which is not cheap and bounds the performance to how fast you can throw out messages. I've figured out how to partition my problem such that if I can get an individual partition to fit in a single fast address space, fast cluster MPI should be very acceptable for stitching it all together. The 32-bit version could handle 10^3 partitions, and the 64-bit version has no practical limits. I still have my eyeballs on those Octiga Bay boxes. Opterons, FPGA, and a very fast Tb fabric to bind it all together. Tasty goodness. j. andrew rogers --- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Growth of computer power
On Wed, Jun 29, 2005 at 08:39:15PM -0700, J. Andrew Rogers wrote: > My 32-bit code has built-in support for 10^3 parallelism, and the 64-bit > code gets essentially bottomless parallelism for free (unused bits and > all that). However, as a practical matter I have not used it yet. You > can buy really big shared memory hardware these days for not much money, > and my codes scale pretty nicely if the latency is reasonable. Unfortunately, shared memory is an expensive mirage to maintain. It doesn't scale very well, due to coherency issuess (if you have r/w intensive operations on a memory block, you have to send lots of signals to and fro until you know it's consistent). This takes lots of time and logic, and is prohibitive for even moderately-sized clusters (IIRC there is no shared memory clusters with 10^3 nodes). No such problems occur with message passing, as it reflects the nature of physical reality. > So I guess you could say the codes have been built to support it, but it > has never been implemented in practice. However, I'll be testing that > before the year is out. Very interesting. Good luck! -- Eugen* Leitl http://leitl.org";>leitl __ ICBM: 48.07100, 11.36820http://www.leitl.org 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE --- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED] signature.asc Description: Digital signature
Re: [agi] Growth of computer power
Eugen wrote: > Do any of you here use MPI, and assume 10^3..10^5 node parallelism? My 32-bit code has built-in support for 10^3 parallelism, and the 64-bit code gets essentially bottomless parallelism for free (unused bits and all that). However, as a practical matter I have not used it yet. You can buy really big shared memory hardware these days for not much money, and my codes scale pretty nicely if the latency is reasonable. So I guess you could say the codes have been built to support it, but it has never been implemented in practice. However, I'll be testing that before the year is out. cheers, j. andrew rogers --- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] Growth of computer power
On Thu, Jun 23, 2005 at 09:01:58PM +1200, Shane wrote: > I'm sure this continued exponential growth is good news for all you power > hungry AGI engineers out there... Do any of you here use MPI, and assume 10^3..10^5 node parallelism? -- Eugen* Leitl http://leitl.org";>leitl __ ICBM: 48.07100, 11.36820http://www.leitl.org 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE --- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED] signature.asc Description: Digital signature
[agi] Growth of computer power
Today I came across a new graph from the people who keep the list of the top 500 super computers in the world. It shows, over the period 1993 to present, the power of the most powerful computer, the computer ranked 500th, and the sum of all 500 top supercomputers on their list. http://www.top500.org/lists/2005/06/perfdevel/Projected_Performance_Development_1.png I'm sure this continued exponential growth is good news for all you power hungry AGI engineers out there... Shane Send instant messages to your online friends http://au.messenger.yahoo.com --- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED]
