Re: [agi] SOTA 50 GB memory for Windows
AMD demonstrates the first x86 dual-core processor http://www.digitimes.com/news/a20040831PR200.html Confirms it will re-use the current Opteron 940-pin socket -- Brian Atkins Singularity Institute for Artificial Intelligence http://www.singinst.org/ --- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] SOTA 50 GB memory for Windows
Opteron system are definitely the sweet spot currently, and for the near future. Rumors are that major server companies are working on 32-way systems to be released soon. Also of course Cray bought that OctigaBay company and now has this: http://www.cray.com/products/systems/xd1/ Also rumor has it that when the dual-core Opterons come out in 2H 2005 you will be able to drop them into most existing motherboards/servers that you can buy right now. So upgradability looks potentially excellent. -- Brian Atkins Singularity Institute for Artificial Intelligence http://www.singinst.org/ --- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] SOTA 50 GB memory for Windows
David Hart wrote: Because the memory controller resides on the CPU in Opteron systems, all 8 CPUs must be populated, but this can be achieved with the slowest/cheapest model, the Opteron 840 (1.4 GHz). I would second using the cheapest CPU part available, which currently is the 1.6GHz part, you'll save pocket change by going with the 1.4GHz part (for the 2xx series, the difference is about $10). The low clock speed is deceptive. If you use one of the AMD64 optimized compilers (e.g. http://www.pathscale.com) rather than GCC, the real performance is stunning even on slow CPU parts -- it is as fast or faster than pretty much anything else in the general case. For SMP codes, the only thing comparable is the Unix Big Iron (e.g. IBM's Power series boxen) in terms of how it scales across multiple processors. Of all the different architectures I touch, the Opteron is my favorite. Top-notch Big Iron performance at commodity prices. Itanium is a bit better for floating point (PPC970 only for DSP codes), but not much and it is worse at a lot of other codes and expensive. It is worth noting that the Intel AMD64-compatible chips will be ISA compatible, but missing all the features that make the Opterons scalable. And as Brian noted, it has an aggressive looking roadmap. There is a new version of HyperTransport coming out relatively soon (maybe first part of next year?) which will increase the scalability even more, and the multi-core CPUs should be with us shortly. As the big server vendors put more and more money into big Opteron boxes, Linux being the default OS, this looks like the bang/buck champion for the foreseeable future. 64-bit Windows may be viable, but I have no idea if it will run well on the bigger systems as a practical matter. The Windows VM continues to be funky from what I understand, though I have no personal experience. j. andrew rogers --- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] SOTA 50 GB memory for Windows
Shane wrote: As for more indirect solutions like RAM disks... I think you would loose at least a factor of ten in speed compared to simply accessing system memory directly as you would need to go through the file system and out to an external device with RAM in it pretending to be a disk. If I remember correctly the fastest disk interfaces you can get for a PC are in the order of 100 MB per second while system RAM is more like a few GB per second. So if you need really high speed then RAM is perhaps your only option I think. For RAM, as with disk, it is more about latency than bandwidth. Even with contrived codes, you can't drive RAM at 100% bandwidth even on architectures with extremely low latency (e.g. Opterons), and on more average architectures (e.g. PPC970) it is not even remotely close to theoretical. For big memory systems a la ccNUMA, the performance is less. A big part of the reason is that the latency limits the number of requests to core that you can make per second regardless of the theoretical bandwidth. The disk may have less bandwidth, about 250MB per channel on a normal high-end HBA, but it can drive that close 100% and it is trivial to run multiple channels in parallel which will get you in the same region of real RAM bandwidth-wise. The big downside is that it gets this performance because it works on big chunks of data compared to real RAM, so the number of discrete fetches per second will probably be an order of magnitude less than system core even under ideal circumstances. I think the biggest argument against using a big RAM disk array is that the HBAs and drivers are optimized for very different types of things than a RAM extension. When you get into big ccNUMA systems, the real memory latencies start to creep within spitting distance of network DMA, arguably the most interesting low-cost alternative. j. andrew rogers --- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] SOTA 50 GB memory for Windows
Flash memories could not be used as memory - they withstand no more than 100.000 rewrites. (I do not recall this properly it could be even less then 10.000) Milon Lukasz Kaiser wrote: Hi. Given that your core system is C# this could be a bit of a problem. Just to put my 2c, if you should have the idea to try .NET under linux, better first do some tests. In my experience the linux .NET runtime (mono), although almost fully compatible, is about 5-10 time slower than that on Windows and does not handle over 1GB of memory even if it is there (on 32 bit system at least). Does anyone know how flash drives perform in such setting, can they be used as a slower alternative to RAM ? - lk --- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED] --- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED]
RE: [agi] SOTA 50 GB memory for Windows
Visual Studio (beta) with 64-bit (c#) compilation is available now: http://lab.msdn.microsoft.com/vs2005/productinfo/productline/ as is Windows XP 64-bit for testing. That's all one needs for development. Peter -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Shane Sent: Friday, August 20, 2004 9:57 PM To: [EMAIL PROTECTED] Subject: Re: [agi] SOTA 50 GB memory for Windows Well I guess I have become skeptical about when they will release such a thing as they have been saying that they will put out a 64 bit version for Intel for years now but then always pushing back the release date. No doubt it's related to the difficulties Intel has been having with Itanium. If Intel deliver their AMD compatible 64 bit chips reasonably soon then surely a 64 bit Windows release can't be too far away. The other thing is that when something as big as this changes in the OS it can take a while for various things to straighten themselves out. Things like development tools, devices drivers and so on. I guess for you the key thing is when they will deliver a 64 bit version of C# and associated tools. Curiously, you could get 64 bit Windows for Alpha CPUs about 8 years ago! A friend of mind used to develop things for it way back then, but that version of Windows was eventually killed off by Microsoft. Shane Peter Voss wrote: Microsoft Updates 64-bit Windows XP Preview Editions http://www.eweek.com/article2/0,1759,1637471,00.asp Peter --- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED] --- Incoming mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.742 / Virus Database: 495 - Release Date: 8/19/2004 --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.742 / Virus Database: 495 - Release Date: 8/19/2004 --- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] SOTA 50 GB memory for Windows
I'm looking for price performance (access time) for: 1) Cached RAID This will be useless for runtime VM or pseudo-VM purposes. RAID cache isolates the application from write burst bottlenecks when syncing disks (e.g. checkpointing transaction logs), but that's about it. For flatter I/O patterns, you'll lose 3-4 orders of magnitude access time over non-cached main memory and it won't be appreciably faster than raw spindle. Wrong tool for the application. 2) RAM disks Functionally workable, but very expensive. It is much cheaper per GB to buy the biggest RAM chips you can find and put them on the motherboard. The primary advantage is that you can scale it to very large sizes while only losing somewhere around an order of magnitude versus main core if done well. 3) Internal RAM (using 64 bit architecture?) The best performing, and relatively cheap too. You can slap 32 GB of RAM in an off-the-shelf Opteron system for not much money. The biggest problem is finding motherboards with loads of memory slots and the fact that there is a hard upper bound on how much memory a given system will support. 4) other Nothing I can think of that will work with Windows. There are other performant and cost-effective options for Linux/Unix systems. A compromise might be to max out system RAM within reason (e.g. using 2GB DIMMs), and then using RAM disks on a fast HBA to get the rest of your capacity. All of this will require a 64-bit OS to be efficient. j. andrew rogers --- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED]
RE: [agi] SOTA 50 GB memory for Windows
Thanks Andrew. I didn't realize that RAID cache doesn't help on reads (like RAM disks do). Just how expensive is a high-performance 50GB RAM disk system? Off hand, anyone know progress/ETA on Intel EM64T for .net laguages (c#) ? Also, what Windows compatible machines offer the most RAM ? (Dell seems to max out at 8Gb) Peter -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of J. Andrew Rogers Sent: Friday, August 20, 2004 2:12 PM To: [EMAIL PROTECTED] Subject: Re: [agi] SOTA 50 GB memory for Windows I'm looking for price performance (access time) for: 1) Cached RAID This will be useless for runtime VM or pseudo-VM purposes. RAID cache isolates the application from write burst bottlenecks when syncing disks (e.g. checkpointing transaction logs), but that's about it. For flatter I/O patterns, you'll lose 3-4 orders of magnitude access time over non-cached main memory and it won't be appreciably faster than raw spindle. Wrong tool for the application. 2) RAM disks Functionally workable, but very expensive. It is much cheaper per GB to buy the biggest RAM chips you can find and put them on the motherboard. The primary advantage is that you can scale it to very large sizes while only losing somewhere around an order of magnitude versus main core if done well. 3) Internal RAM (using 64 bit architecture?) The best performing, and relatively cheap too. You can slap 32 GB of RAM in an off-the-shelf Opteron system for not much money. The biggest problem is finding motherboards with loads of memory slots and the fact that there is a hard upper bound on how much memory a given system will support. 4) other Nothing I can think of that will work with Windows. There are other performant and cost-effective options for Linux/Unix systems. A compromise might be to max out system RAM within reason (e.g. using 2GB DIMMs), and then using RAM disks on a fast HBA to get the rest of your capacity. All of this will require a 64-bit OS to be efficient. j. andrew rogers --- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED] --- Incoming mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.740 / Virus Database: 494 - Release Date: 8/16/2004 --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.740 / Virus Database: 494 - Release Date: 8/16/2004 --- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED]
RE: [agi] SOTA 50 GB memory for Windows
I didn't realize that RAID cache doesn't help on reads (like RAM disks do). Yeah, a lot of people have never really thought about it much. I've worked with database servers for years though, where we actually tuned that type of hardware. The main difference is that a write doesn't return a block to the application, so it can return immediately without physically writing to disk, just putting the blocks in the RAM cache until it has some spare iops to burn or the cache becomes full. For reads though, you have to block until you have physically pulled the block off the disk so that you have something to return. RAID controllers do support predictive read-ahead caching, but that only makes a difference if you have sequential access patterns e.g. streaming large files. Otherwise, it has to block until it pulls data off the spindle every time because it doesn't know what you'll ask for next (unless you get very lucky and the data is in cache). So it is primarily good for streaming large files (e.g. video editing) or buffering write bursts (e.g. database servers). Just how expensive is a high-performance 50GB RAM disk system? Expect to pay ~$2 per MB on the cheap end of things. Or for 50GB, about $100k. Using cheap machines maxed with RAM and RDMA fabrics or similar is a cheaper way to do this for roughly equivalent performance, but you won't be able to do this on Windows -- this is one of those arenas where Linux excels. cheers, j. andrew rogers --- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] SOTA 50 GB memory for Windows
Peter, In terms of hardware as far as I know the biggest PC style hardware you can buy supports 32 GB of RAM. For example you can by PCs this big from people like www.penguincomputing.com However there isn't a 64 bit version of Windows on the market nor will there be for some time. Thus your only option is to run something like Linux if you want to have all this data being accessed by code directly in RAM. Given that your core system is C# this could be a bit of a problem. I think this is perhaps the first issue you would need to sort out before thinking about hardware --- the hardware exists alright but with Windows you can't make use of it, at least in terms of having it directly in RAM. As for more indirect solutions like RAM disks... I think you would loose at least a factor of ten in speed compared to simply accessing system memory directly as you would need to go through the file system and out to an external device with RAM in it pretending to be a disk. If I remember correctly the fastest disk interfaces you can get for a PC are in the order of 100 MB per second while system RAM is more like a few GB per second. So if you need really high speed then RAM is perhaps your only option I think. Shane Peter Voss wrote: What are the best options for large amounts of fast memory for Windows-based systems? I'm looking for price performance (access time) for: 1) Cached RAID 2) RAM disks 3) Internal RAM (using 64 bit architecture?) 4) other Thanks for any info. Peter --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.740 / Virus Database: 494 - Release Date: 8/16/2004 --- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED] --- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED]
RE: [agi] SOTA 50 GB memory for Windows
Microsoft Updates 64-bit Windows XP Preview Editions http://www.eweek.com/article2/0,1759,1637471,00.asp Peter -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Shane . However there isn't a 64 bit version of Windows on the market nor will there be for some time. Thus your only option is to run something like Linux if you want to have all this data being accessed by code directly in RAM --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.740 / Virus Database: 494 - Release Date: 8/16/2004 --- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED]
Re: [agi] SOTA 50 GB memory for Windows
Hi. Given that your core system is C# this could be a bit of a problem. Just to put my 2c, if you should have the idea to try .NET under linux, better first do some tests. In my experience the linux .NET runtime (mono), although almost fully compatible, is about 5-10 time slower than that on Windows and does not handle over 1GB of memory even if it is there (on 32 bit system at least). Does anyone know how flash drives perform in such setting, can they be used as a slower alternative to RAM ? - lk --- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED]
