Re: [PERFORM] RAID card recommendation
On 12/1/09 6:08 PM, Karl Denninger k...@denninger.net wrote: Scott Carey wrote: On 11/24/09 11:13 AM, Scott Marlowe scott.marl...@gmail.com mailto:scott.marl...@gmail.com wrote: They get good reviews as well. Both manufacturers have their star performers, and their utility or work group class controllers. For what you're doing the areca 12xx or 3ware 95xx series should do fine. -1 to 3ware's SATA solutions 3ware 95xx and 96xx had performance somewhere between PERC 5 (horrid) and PERC 6 (mediocre) when I tested them with large SATA drives with RAID 10. Haven't tried raid 6 or 5. Haven't tried the SA model that supports SAS. When a competing card (Areca or Adaptec) gets 3x the sequential throughput on an 8 disk RAID 10 and only catches up to be 60% the speed after heavy tuning of readahead value, there's something wrong. Random access throughput doesn't suffer like that however -- but its nice when the I/O can sequential scan faser than postgres can read the tuples. What operating system? I am running under FreeBSD with 96xx series and am getting EXCELLENT performance. Under Postgres 8.4.x on identical hardware except for the disk controller, I am pulling a literal 3x the iops on the same disks that I do with the Adaptec (!) I DID note that under Linux the same hardware was a slug. Hm... Linux, Centos 5.3. Drivers/OS can certainly make a big difference. -- Karl -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] RAID card recommendation
On 12/1/09 6:49 PM, Greg Smith g...@2ndquadrant.com wrote: Scott Carey wrote: 3ware 95xx and 96xx had performance somewhere between PERC 5 (horrid) and PERC 6 (mediocre) when I tested them with large SATA drives with RAID 10. Haven't tried raid 6 or 5. Haven't tried the SA model that supports SAS The only models I've tested and recommended lately are exactly those though. The 9690SA is the earliest 3ware card I've mentioned as seeming to have reasonable performance. The 95XX cards are certainly much slower than similar models from, say, Areca. I've never had one of the earlier 96XX models to test. Now you've got me wondering what the difference between the earlier and current 96XX models really is. 9650 was made by 3Ware, essentially a PCIe version of the 9550. The 9690SA was from some sort of acquisition/merger. They are not the same product line at all. 3Ware, IIRC, has its roots in ATA and SATA RAID. I gave up on them after the 9650 and 9550 experiences (on Linux) though. -- Greg Smith2ndQuadrant Baltimore, MD PostgreSQL Training, Services and Support g...@2ndquadrant.com www.2ndQuadrant.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] RAID card recommendation
Scott Carey wrote: On 12/1/09 6:49 PM, Greg Smith g...@2ndquadrant.com wrote: Scott Carey wrote: 3ware 95xx and 96xx had performance somewhere between PERC 5 (horrid) and PERC 6 (mediocre) when I tested them with large SATA drives with RAID 10. Haven't tried raid 6 or 5. Haven't tried the SA model that supports SAS The only models I've tested and recommended lately are exactly those though. The 9690SA is the earliest 3ware card I've mentioned as seeming to have reasonable performance. The 95XX cards are certainly much slower than similar models from, say, Areca. I've never had one of the earlier 96XX models to test. Now you've got me wondering what the difference between the earlier and current 96XX models really is. 9650 was made by 3Ware, essentially a PCIe version of the 9550. The 9690SA was from some sort of acquisition/merger. They are not the same product line at all. 3Ware, IIRC, has its roots in ATA and SATA RAID. I gave up on them after the 9650 and 9550 experiences (on Linux) though. My experience under FreeBSD: 1. The Adaptecs suck. 1/3rd to 1/2 the performance of 2. The 9650s 3ware boards, which under FreeBSD are quite fast. 3. However, the Areca 1680-IX is UNBELIEVABLY fast. Ridiculously so in fact. I have a number of 9650s in service and have been happy with them under FreeBSD. Under Linux, however, they bite in comparison. The Areca 1680 is not cheap. However, it comes with out-of-band management (IP-KVM, direct SMTP and SNMP connectivity, etc) which is EXTREMELY nice, especially for colocated machines where you need a way in if things go horribly wrong. One warning: I have had problems with the Areca under FreeBSD if you set up a passthrough (e.g. JBOD) disc, delete it from the config while running and then either accidentally touch the device nodes OR try to use FreeBSD's camcontrol utility to tell it to pick up driver changes. Either is a great way to panic the machine. As such for RAID it's fine but use care if you need to be able to swap NON-RAID disks while the machine is operating (e.g. for backup purposes - run a dump then dismount it and pull the carrier) - it is dangerous to attempt this (the 3Ware card does NOT have this problem.) I am trying to figure out exactly what provokes this and if I can get around it at this point (in the lab of course!) No experience with the 9690 3Wares as of yet. -- Karl attachment: karl.vcf -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] RAID card recommendation
Scott Carey wrote: 9650 was made by 3Ware, essentially a PCIe version of the 9550. The 9690SA was from some sort of acquisition/merger. They are not the same product line at all. 3ware became a division of AMCC, which was then bought by LSI. The 9590SA came out while they were a part of AMCC. I was under the impression that the differences between the 9650 and the 9690SA were mainly related to adding SAS support, which was sort of a bridge addition rather than a fundamental change in the design of the card. You'll often see people refer to 9650/9690 as if they're the same card; they may never run the same firmware. They certainly always get firmware updates at the same time, and as part of the same download package. Another possibility for the difference between Scott's experience and mine is that I've only evaluated those particular cards recently, and there seems to be evidence that 3ware did some major firmware overhauls in late 2008, i.e. http://unix.derkeiler.com/Mailing-Lists/FreeBSD/performance/2008-10/msg5.html Let me try to summarize where things are at a little more clearly, with the data accumulated during this long thread: -Areca: Usually the fastest around. Management tools are limited enough that you really want the version with the on-board management NIC. May require some testing to find a good driver version. -3ware: Performance on current models not as good as Areca, but with a great set of management tools (unless you're using SAS) and driver reliability. Exact magnitude of the performance gap with Areca is somewhat controversial and may depend on OS--FreeBSD performance might be better than Linux in particular. Older 3ware cards were really slow. One of these days I need to wrangle up enough development cash to buy current Areca and 3ware cards, an Intel SSD, and disappear into the lab (already plenty of drives here) until I've sorted this all out to my satisfaction. -- Greg Smith2ndQuadrant Baltimore, MD PostgreSQL Training, Services and Support g...@2ndquadrant.com www.2ndQuadrant.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] RAID card recommendation
Greg Smith wrote: Let me try to summarize where things are at a little more clearly, with the data accumulated during this long thread: -Areca: Usually the fastest around. Management tools are limited enough that you really want the version with the on-board management NIC. May require some testing to find a good driver version. -3ware: Performance on current models not as good as Areca, but with a great set of management tools (unless you're using SAS) and driver reliability. Exact magnitude of the performance gap with Areca is somewhat controversial and may depend on OS--FreeBSD performance might be better than Linux in particular. Older 3ware cards were really slow. One of these days I need to wrangle up enough development cash to buy current Areca and 3ware cards, an Intel SSD, and disappear into the lab (already plenty of drives here) until I've sorted this all out to my satisfaction. ... and do I hear you saying that no other vendor is worth considering? Just how far off are they? Thanks, Craig -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] RAID card recommendation
Greg Smith wrote: Scott Carey wrote: 9650 was made by 3Ware, essentially a PCIe version of the 9550. The 9690SA was from some sort of acquisition/merger. They are not the same product line at all. 3ware became a division of AMCC, which was then bought by LSI. The 9590SA came out while they were a part of AMCC. I was under the impression that the differences between the 9650 and the 9690SA were mainly related to adding SAS support, which was sort of a bridge addition rather than a fundamental change in the design of the card. You'll often see people refer to 9650/9690 as if they're the same card; they may never run the same firmware. They certainly always get firmware updates at the same time, and as part of the same download package. Another possibility for the difference between Scott's experience and mine is that I've only evaluated those particular cards recently, and there seems to be evidence that 3ware did some major firmware overhauls in late 2008, i.e. http://unix.derkeiler.com/Mailing-Lists/FreeBSD/performance/2008-10/msg5.html Let me try to summarize where things are at a little more clearly, with the data accumulated during this long thread: -Areca: Usually the fastest around. Management tools are limited enough that you really want the version with the on-board management NIC. May require some testing to find a good driver version. -3ware: Performance on current models not as good as Areca, but with a great set of management tools (unless you're using SAS) and driver reliability. Exact magnitude of the performance gap with Areca is somewhat controversial and may depend on OS--FreeBSD performance might be better than Linux in particular. Older 3ware cards were really slow. One of these days I need to wrangle up enough development cash to buy current Areca and 3ware cards, an Intel SSD, and disappear into the lab (already plenty of drives here) until I've sorted this all out to my satisfaction. Most common SSDs will NOT come up on the 3ware cards at present. Not sure why as of yet - I've tried several. Not had the time to screw with them on the ARECA cards yet. -- Karl attachment: karl.vcf -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] RAID card recommendation
Craig James wrote: ... and do I hear you saying that no other vendor is worth considering? Just how far off are they? I wasn't trying to summarize every possible possibility, just the complicated ones there's some debate over. What else is OK besides Areca and 3ware? HP's P800 is good, albeit not so easy to buy unless you're getting an HP system. The LSI Megaraid stuff and its close relative the Dell PERC6 are OK for some apps too; my intense hatred of Dell usually results in my forgetting about them. (As an example, http://en.wikipedia.org/wiki/ATX#Issues_with_Dell_power_supplies documents what I consider the worst design decision ever made by a PC manufacturer) I don't think any of the other vendors on the market are viable for a Linux system due to driver issues and general low quality, which includes Adaptec, Promise, Highpoint, and all the motherboard Fake RAID stuff from Silicon Image, Intel, Via, etc. I don't feel there's any justification for using those products instead of using a simple SATA controller and Linux software RAID in a PostgreSQL context. -- Greg Smith2ndQuadrant Baltimore, MD PostgreSQL Training, Services and Support g...@2ndquadrant.com www.2ndQuadrant.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] RAID card recommendation
Karl Denninger wrote: Most common SSDs will NOT come up on the 3ware cards at present. Not sure why as of yet - I've tried several. Right, and they're being rather weasly at http://www.3ware.com/kb/Article.aspx?id=15470 talking about it too. Not had the time to screw with them on the ARECA cards yet. I know the situation there is much better, like: http://hothardware.com/News/24-Samsung-SSDs-Linked-Together-for-2GBSec/ Somebody at Newegg has said they got their Areca 1680 working with one of the Intel X-25 drives, but wasn't impressed by the write performance of the result. Makes me wonder if the Areca card is messing with the write cache of the drive. -- Greg Smith2ndQuadrant Baltimore, MD PostgreSQL Training, Services and Support g...@2ndquadrant.com www.2ndQuadrant.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] RAID card recommendation
Greg Smith wrote: Craig James wrote: ... and do I hear you saying that no other vendor is worth considering? Just how far off are they? I wasn't trying to summarize every possible possibility, just the complicated ones there's some debate over. What else is OK besides Areca and 3ware? HP's P800 is good, albeit not so easy to buy unless you're getting an HP system. The LSI Megaraid stuff and its close relative the Dell PERC6 are OK for some apps too; my intense hatred of Dell usually results in my forgetting about them. (As an example, http://en.wikipedia.org/wiki/ATX#Issues_with_Dell_power_supplies documents what I consider the worst design decision ever made by a PC manufacturer) I don't think any of the other vendors on the market are viable for a Linux system due to driver issues and general low quality, which includes Adaptec, Promise, Highpoint, and all the motherboard Fake RAID stuff from Silicon Image, Intel, Via, etc. I don't feel there's any justification for using those products instead of using a simple SATA controller and Linux software RAID in a PostgreSQL context. The LSI Megaraid (and Intel's repackaging of it, among others) is reasonably good under FreeBSD. Performance is slightly worse than the 3ware 95xx series boards, but not materially so. Their CLI interface is interesting (it drops a log file in the working directly BY DEFAULT unless you tell it otherwise, among other things.) -- Karl attachment: karl.vcf -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] RAID card recommendation
On 11/24/09 11:13 AM, Scott Marlowe scott.marl...@gmail.com wrote: They get good reviews as well. Both manufacturers have their star performers, and their utility or work group class controllers. For what you're doing the areca 12xx or 3ware 95xx series should do fine. -1 to 3ware's SATA solutions 3ware 95xx and 96xx had performance somewhere between PERC 5 (horrid) and PERC 6 (mediocre) when I tested them with large SATA drives with RAID 10. Haven't tried raid 6 or 5. Haven't tried the SA model that supports SAS. When a competing card (Areca or Adaptec) gets 3x the sequential throughput on an 8 disk RAID 10 and only catches up to be 60% the speed after heavy tuning of readahead value, there's something wrong. Random access throughput doesn't suffer like that however -- but its nice when the I/O can sequential scan faser than postgres can read the tuples. As far as drives go we've been really happy with WD of late, they make large enterprise class SATA drives that don't pull a lot of power (green series) and fast SATA drives that pull a bit more but are faster (black series). We've used both and are quite happy with each. We use a pair of blacks to build slony read slaves and they're very fast, with write speeds of ~100MB/second and read speeds double that in linux under sw RAID-1 -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] RAID card recommendation
Scott Carey wrote: On 11/24/09 11:13 AM, Scott Marlowe scott.marl...@gmail.com wrote: They get good reviews as well. Both manufacturers have their star performers, and their utility or work group class controllers. For what you're doing the areca 12xx or 3ware 95xx series should do fine. -1 to 3ware's SATA solutions 3ware 95xx and 96xx had performance somewhere between PERC 5 (horrid) and PERC 6 (mediocre) when I tested them with large SATA drives with RAID 10. Haven't tried raid 6 or 5. Haven't tried the SA model that supports SAS. When a competing card (Areca or Adaptec) gets 3x the sequential throughput on an 8 disk RAID 10 and only catches up to be 60% the speed after heavy tuning of readahead value, there's something wrong. Random access throughput doesn't suffer like that however -- but its nice when the I/O can sequential scan faser than postgres can read the tuples. What operating system? I am running under FreeBSD with 96xx series and am getting EXCELLENT performance. Under Postgres 8.4.x on identical hardware except for the disk controller, I am pulling a literal 3x the iops on the same disks that I do with the Adaptec (!) I DID note that under Linux the same hardware was a slug. Hm... -- Karl attachment: karl.vcf -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] RAID card recommendation
Scott Carey wrote: 3ware 95xx and 96xx had performance somewhere between PERC 5 (horrid) and PERC 6 (mediocre) when I tested them with large SATA drives with RAID 10. Haven't tried raid 6 or 5. Haven't tried the SA model that supports SAS The only models I've tested and recommended lately are exactly those though. The 9690SA is the earliest 3ware card I've mentioned as seeming to have reasonable performance. The 95XX cards are certainly much slower than similar models from, say, Areca. I've never had one of the earlier 96XX models to test. Now you've got me wondering what the difference between the earlier and current 96XX models really is. -- Greg Smith2ndQuadrant Baltimore, MD PostgreSQL Training, Services and Support g...@2ndquadrant.com www.2ndQuadrant.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] RAID card recommendation
Steve Crawford wrote: Greg Smith wrote: Jochen Erwied wrote: - Promise Technology Supertrak ES4650 + additional BBU I've never seen a Promise controller that had a Linux driver you would want to rely on under any circumstances... +1 I haven't tried Promise recently, but last time I did I determined that they got the name because they Promise the Linux driver for your card will be available real-soon-now. One more data point, it's not confidence inspiring that google turns up Promise Technologies customers that are quite vocal about suing them. http://www.carbonite.com/blog/post/2009/03/Further-clarification-on-our-lawsuit-against-Promise-Technologies.aspx -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] RAID card recommendation
--- On Tue, 24/11/09, Scott Marlowe scott.marl...@gmail.com wrote: Jochen Erwied joc...@pgsql-performance.erwied.eu wrote: Since I'm currently looking at upgrading my own database server, maybe some of the experts can give a comment on one of the following controllers: - Promise Technology Supertrak ES4650 + additional BBU - Adaptec RAID 5405 SGL/256 SATA/SAS + additional BBU - Adaptec RAID 5405Z SGL/512 SATA/SAS My personal favourite currently is the 5405Z, since it does not require regular battery replacements and because it has 512MB of cache. Have you searched the -performance archives for references to them? I'm not that familiar with Adaptec RAID controllers. Not requiring a battery check / replacement is nice. We've been running Adaptec 5805s for the past year and I've been pretty happy, I think they have the same dual core IOP348 as the Areca 1680s. I've a bunch of 5805Zs on my desk ready to go in some new servers too (that means more perc6 cards to chuck on my smash pile) and I'm excited to see how they go; I feer the unknown a bit though, and I'm not sure the sight big capacitors is reassuruing me... Only problem I've seen is one controller periodically report it's too hot, but I suspect that may be something to do with the server directly above it having fanless power supplies. -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] RAID card recommendation
Greg Smith wrote: Jochen Erwied wrote: - Promise Technology Supertrak ES4650 + additional BBU - Adaptec RAID 5405 SGL/256 SATA/SAS + additional BBU - Adaptec RAID 5405Z SGL/512 SATA/SAS I've never seen a Promise controller that had a Linux driver you would want to rely on under any circumstances...Easier to just buy from a company that has always cared about good Linux support, like 3ware. +1 I haven't tried Promise recently, but last time I did I determined that they got the name because they Promise the Linux driver for your card will be available real-soon-now. Actually got strung along for a couple months before calling my supplier and telling him to swap it out for a 3ware. The 3ware just works. I currently have a couple dozen Linux servers, including some PostgreSQL machines, running the 3ware cards. Cheers, Steve -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
[PERFORM] RAID card recommendation
We're about to purchase a new server to store some of our old databases, and I was wondering if someone could advise me on a RAID card. We want to make a 6-drive SATA RAID array out of 2TB drives, and it will be RAID 5 or 6 because there will be zero write traffic. The priority is stuffing as much storage into a small 2U rack as possible, with performance less important. We will be running Debian Linux. People have mentioned Areca as making good RAID controllers. We're looking at the Areca ARC-1220 PCI-Express x8 SATA II as a possibility. Does anyone have an opinion on whether it is a turkey or a star? Another possibility is a 3-ware card of some description. Thanks in advance, Matthew -- Now you see why I said that the first seven minutes of this section will have you looking for the nearest brick wall to beat your head against. This is why I do it at the end of the lecture - so I can run. -- Computer Science lecturer -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] RAID card recommendation
Matthew Wakeling wrote: We're about to purchase a new server to store some of our old databases, and I was wondering if someone could advise me on a RAID card. We want to make a 6-drive SATA RAID array out of 2TB drives, and it will be RAID 5 or 6 because there will be zero write traffic. The priority is stuffing as much storage into a small 2U rack as possible, with performance less important. We will be running Debian Linux. People have mentioned Areca as making good RAID controllers. We're looking at the Areca ARC-1220 PCI-Express x8 SATA II as a possibility. Does anyone have an opinion on whether it is a turkey or a star? Another possibility is a 3-ware card of some description. Do you actually need a RAID card at all? It's just another point of failure: the Linux software raid (mdadm) is pretty good. Also, be very wary of RAID5 for an array that size. It is highly probable that, if one disk has failed, then during the recovery process, you may lose a second disk. The unrecoverable error rate on standard disks is about 1 in 10^14 bits; your disk array is 10^11 bits in size... We got bitten by this Richard -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] RAID card recommendation
On Nov 24, 2009, at 9:23 AM, Matthew Wakeling wrote: We're about to purchase a new server to store some of our old databases, and I was wondering if someone could advise me on a RAID card. We want to make a 6-drive SATA RAID array out of 2TB drives, and it will be RAID 5 or 6 because there will be zero write traffic. The priority is stuffing as much storage into a small 2U rack as possible, with performance less important. We will be running Debian Linux. People have mentioned Areca as making good RAID controllers. We're looking at the Areca ARC-1220 PCI-Express x8 SATA II as a possibility. Does anyone have an opinion on whether it is a turkey or a star? We've used that card and have been quite happy with it. Looking through the release notes for firmware upgrades can be pretty worrying (you needed to fix what?!), but we never experienced any problems ourselves, and its not like 3ware release notes are any different. But the main benefits of a RAID card are a write cache and easy hot swap. It sounds like you don't need a write cache. Can you be happy with the kernel's hotswap ability?
Re: [PERFORM] RAID card recommendation
- Richard Neill rn...@cam.ac.uk escreveu: Matthew Wakeling wrote: We're about to purchase a new server to store some of our old databases, and I was wondering if someone could advise me on a RAID card. We want to make a 6-drive SATA RAID array out of 2TB drives, and it will be RAID 5 or 6 because there will be zero write traffic. The priority is stuffing as much storage into a small 2U rack as possible, with performance less important. We will be running Debian Linux. People have mentioned Areca as making good RAID controllers. We're looking at the Areca ARC-1220 PCI-Express x8 SATA II as a possibility. Does anyone have an opinion on whether it is a turkey or a star? Another possibility is a 3-ware card of some description. Do you actually need a RAID card at all? It's just another point of failure: the Linux software raid (mdadm) is pretty good. Also, be very wary of RAID5 for an array that size. It is highly probable that, if one disk has failed, then during the recovery process, you may lose a second disk. The unrecoverable error rate on standard disks is about 1 in 10^14 bits; your disk array is 10^11 bits in size... We got bitten by this Richard Linux kernel software RAID is fully supported in Debian Lenny, is quite cheap to implement and powerful. I would avoid SATA disks but it's just me. SAS controllers and disks are expensive but worth every penny spent on them. Prefer RAID 1+0 over RAID 5 not only because of the risk of failure of a second disk, but I have 3 cases of performance issues caused by RAID 5. It's said that performance is not the problem but think twice because a good application tends to scale fast to several users. Of course, keep a good continuous backup strategy of your databases and don't trust just the mirroring of disks in a RAID fashion. Flavio Henrique A. Gurgel Consultor -- 4Linux tel. 55-11-2125.4765 fax. 55-11-2125.4777 www.4linux.com.br -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] RAID card recommendation
On Tue, Nov 24, 2009 at 10:23 AM, Matthew Wakeling matt...@flymine.org wrote: We're about to purchase a new server to store some of our old databases, and I was wondering if someone could advise me on a RAID card. We want to make a 6-drive SATA RAID array out of 2TB drives, and it will be RAID 5 or 6 because there will be zero write traffic. The priority is stuffing as much storage into a small 2U rack as possible, with performance less important. We will be running Debian Linux. People have mentioned Areca as making good RAID controllers. We're looking at the Areca ARC-1220 PCI-Express x8 SATA II as a possibility. Does anyone have an opinion on whether it is a turkey or a star? We run a 12xx series on our office server in RAID-6 over 8 1TB 7200RPM server class SATA drives. Our production server runs the 1680 on top of 16 15k5 seagates in RAID-10. The performance difference between these two are enormous. Things that take minutes on the production server can take hours on the office server. Production handles 1.5Million users, office handles 20 or 30 users. I've been really happy with the reliability of the 12xx card here at work. 100% uptime for a year, that machine goes down for kernel updates and only that. But it's not worked that hard all day everyday, so I can't compare its reliability with production in RAID-10 which has had one drive fail the week it was delivered and none since in 400+days. We have two hot spares there. Another possibility is a 3-ware card of some description. They get good reviews as well. Both manufacturers have their star performers, and their utility or work group class controllers. For what you're doing the areca 12xx or 3ware 95xx series should do fine. As far as drives go we've been really happy with WD of late, they make large enterprise class SATA drives that don't pull a lot of power (green series) and fast SATA drives that pull a bit more but are faster (black series). We've used both and are quite happy with each. We use a pair of blacks to build slony read slaves and they're very fast, with write speeds of ~100MB/second and read speeds double that in linux under sw RAID-1 -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] RAID card recommendation
Since I'm currently looking at upgrading my own database server, maybe some of the experts can give a comment on one of the following controllers: - Promise Technology Supertrak ES4650 + additional BBU - Adaptec RAID 5405 SGL/256 SATA/SAS + additional BBU - Adaptec RAID 5405Z SGL/512 SATA/SAS My personal favourite currently is the 5405Z, since it does not require regular battery replacements and because it has 512MB of cache. Since my server only has room for four disks, I'd choose the following one: - Seagate Cheetah 15K.6 147GB SAS Drives would be organized as RAID-0 for fast access, I do not need terabytes of storage. The database currently is about 150 GB in size (including indexes), the main table having a bit less than 1 billion rows (maximum will be about 2 billion) and getting about 10-20 million updates per day, so update speed is critical. Currently the database is running on a mdadm raid-0 with four S-ATA drives (7.2k rpm), which was ok when the database was half this size... Operating System is Gentoo Linux 2.6.31-r1 on a Fujitsu Siemens Primergy 200 S2 (2xXEON @ 1.6 GHz) with 4 GB of RAM (which also would be increased to its maximum of 8 GB during the above update) -- Jochen Erwied | home: joc...@erwied.eu +49-208-38800-18, FAX: -19 Sauerbruchstr. 17 | work: j...@mbs-software.de +49-2151-7294-24, FAX: -50 D-45470 Muelheim | mobile: jochen.erw...@vodafone.de +49-173-5404164 -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] RAID card recommendation
On Tue, Nov 24, 2009 at 12:28 PM, Jochen Erwied joc...@pgsql-performance.erwied.eu wrote: Since I'm currently looking at upgrading my own database server, maybe some of the experts can give a comment on one of the following controllers: - Promise Technology Supertrak ES4650 + additional BBU - Adaptec RAID 5405 SGL/256 SATA/SAS + additional BBU - Adaptec RAID 5405Z SGL/512 SATA/SAS My personal favourite currently is the 5405Z, since it does not require regular battery replacements and because it has 512MB of cache. Have you searched the -performance archives for references to them? I'm not that familiar with Adaptec RAID controllers. Not requiring a battery check / replacement is nice. Since my server only has room for four disks, I'd choose the following one: - Seagate Cheetah 15K.6 147GB SAS We use the older gen 15k.5 and have been very happy with them. Nowadays it seems the fastest Seagates and Hitachis own the market for super fast drives. Drives would be organized as RAID-0 for fast access, I do not need terabytes of storage. So, you're willing (or forced by economics) to suffer downtime due to drive failure every so often. The database currently is about 150 GB in size (including indexes), the main table having a bit less than 1 billion rows (maximum will be about 2 billion) and getting about 10-20 million updates per day, so update speed is critical. So, assuming this means an 8 hour work day for ~20M rows, you're looking at around 700 per second. Currently the database is running on a mdadm raid-0 with four S-ATA drives (7.2k rpm), which was ok when the database was half this size... Operating System is Gentoo Linux 2.6.31-r1 on a Fujitsu Siemens Primergy 200 S2 (2xXEON @ 1.6 GHz) with 4 GB of RAM (which also would be increased to its maximum of 8 GB during the above update) I'd definitely test the heck out of whatever RAID card you're buying to make sure it performs well enough. For some loads and against some HW RAID cards, SW RAID might be the winner. Another option might be a JBOD box attached to the machine that holds 12 or so 2.5 15k like the hitachi ultrastar 147G 2.5 drives. This sounds like a problem you need to be able to throw a lot of drives at at one time. Is it likely to grow much after this? -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] RAID card recommendation
Gurgel, Flavio escribió: - Richard Neill rn...@cam.ac.uk escreveu: Matthew Wakeling wrote: We're about to purchase a new server to store some of our old databases, and I was wondering if someone could advise me on a RAID card. We want to make a 6-drive SATA RAID array out of 2TB drives, and it will be RAID 5 or 6 because there will be zero write traffic. The priority is stuffing as much storage into a small 2U rack as possible, with performance less important. We will be running Debian Linux. People have mentioned Areca as making good RAID controllers. We're looking at the Areca ARC-1220 PCI-Express x8 SATA II as a possibility. Does anyone have an opinion on whether it is a turkey or a star? Another possibility is a 3-ware card of some description. Do you actually need a RAID card at all? It's just another point of failure: the Linux software raid (mdadm) is pretty good. Also, be very wary of RAID5 for an array that size. It is highly probable that, if one disk has failed, then during the recovery process, you may lose a second disk. The unrecoverable error rate on standard disks is about 1 in 10^14 bits; your disk array is 10^11 bits in size... We got bitten by this Richard Linux kernel software RAID is fully supported in Debian Lenny, is quite cheap to implement and powerful. I would avoid SATA disks but it's just me. SAS controllers and disks are expensive but worth every penny spent on them. Prefer RAID 1+0 over RAID 5 not only because of the risk of failure of a second disk, but I have 3 cases of performance issues caused by RAID 5. It's said that performance is not the problem but think twice because a good application tends to scale fast to several users. Of course, keep a good continuous backup strategy of your databases and don't trust just the mirroring of disks in a RAID fashion. Flavio Henrique A. Gurgel Consultor -- 4Linux tel. 55-11-2125.4765 fax. 55-11-2125.4777 www.4linux.com.br Do you expose that performance issued caused by RAID 5? Because this is one of our solutions here on my country to save the data of our PostgreSQL database. Which model do you recommend ? RAID 0,RAID 1, RAID 5 or RAID 10? -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] RAID card recommendation
On Tue, Nov 24, 2009 at 1:37 PM, Ing. Marcos Ortiz Valmaseda mlor...@uci.cu wrote: Do you expose that performance issued caused by RAID 5? Because this is one of our solutions here on my country to save the data of our PostgreSQL database. Which model do you recommend ? RAID 0,RAID 1, RAID 5 or RAID 10? RAID-1 or RAID-10 are the default, mostly safe choices. For disposable dbs, RAID-0 is fine. For very large dbs with very little writing and mostly reading and on a budget, RAID-6 is ok. In most instances I never recommend RAID-5 anymore. -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] RAID card recommendation
Tuesday, November 24, 2009, 9:05:28 PM you wrote: Have you searched the -performance archives for references to them? I'm not that familiar with Adaptec RAID controllers. Not requiring a battery check / replacement is nice. Either I searched for the wrong terms, or there isn't really that much reference on RAID-controllers on this list. Aberdeen is menthioned once and looks interesting, but I didn't find a reseller in Germany. As far as I see from the list, Promise and Adaptec both seem to be not too bad choices. So, you're willing (or forced by economics) to suffer downtime due to drive failure every so often. I haven't experienced any downtime due to a disk failure for quite a while now (call me lucky), although I had a really catastrophic experience with a RAID-5 some time ago (1 drive crashed, the second one during rebuild :-() But for this application losing one day of updates is not a big deal, and downtime isn't either. It's a long running project of mine, with growing storage needs, but not with 100% of integrity or uptime. So, assuming this means an 8 hour work day for ~20M rows, you're looking at around 700 per second. It's an automated application running 24/7, so I require 'only' about 200-250 updates per second. I'd definitely test the heck out of whatever RAID card you're buying to make sure it performs well enough. For some loads and against some HW RAID cards, SW RAID might be the winner. Well, I haven't got so much opportunities to test out different kind of hardware, so I have to rely on experience or reports. Another option might be a JBOD box attached to the machine that holds 12 or so 2.5 15k like the hitachi ultrastar 147G 2.5 drives. This sounds like a problem you need to be able to throw a lot of drives at at one time. Is it likely to grow much after this? JBOD in an external casing would be an alternative, especially when using an external case. And no, the database will not grow too much after reaching its final size. But looking at the prices for anything larger than 4+1 drives in an external casing is not funny at all :-( -- Jochen Erwied | home: joc...@erwied.eu +49-208-38800-18, FAX: -19 Sauerbruchstr. 17 | work: j...@mbs-software.de +49-2151-7294-24, FAX: -50 D-45470 Muelheim | mobile: jochen.erw...@vodafone.de +49-173-5404164 -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] RAID card recommendation
- Scott Marlowe scott.marl...@gmail.com escreveu: On Tue, Nov 24, 2009 at 1:37 PM, Ing. Marcos Ortiz Valmaseda mlor...@uci.cu wrote: Do you expose that performance issued caused by RAID 5? Because this is one of our solutions here on my country to save the data of our PostgreSQL database. Which model do you recommend ? RAID 0,RAID 1, RAID 5 or RAID 10? RAID-1 or RAID-10 are the default, mostly safe choices. For disposable dbs, RAID-0 is fine. For very large dbs with very little writing and mostly reading and on a budget, RAID-6 is ok. In most instances I never recommend RAID-5 anymore. I would never recommend RAID-5 for database customers (any database system), some of the current ones are using it and the worst nightmares in disk performance are related to RAID-5. As Scott said, RAID-1 is safe, RAID-0 is fast (and accept more request load too), RAID-10 is a great combination of both worlds. Flavio Henrique A. Gurgel Consultor -- 4Linux tel. 55-11-2125.4765 fax. 55-11-2125.4777 www.4linux.com.br -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] RAID card recommendation
On Tue, Nov 24, 2009 at 1:59 PM, Jochen Erwied joc...@pgsql-performance.erwied.eu wrote: Tuesday, November 24, 2009, 9:05:28 PM you wrote: Have you searched the -performance archives for references to them? I'm not that familiar with Adaptec RAID controllers. Not requiring a battery check / replacement is nice. Either I searched for the wrong terms, or there isn't really that much reference on RAID-controllers on this list. Aberdeen is menthioned once and looks interesting, but I didn't find a reseller in Germany. As far as I see from the list, Promise and Adaptec both seem to be not too bad choices. Aberdeen is the builder I use. They'll put any card in you want (within reason) including our preference here, Areca. Perhaps you meant Areca? So, assuming this means an 8 hour work day for ~20M rows, you're looking at around 700 per second. It's an automated application running 24/7, so I require 'only' about 200-250 updates per second. Oh, much better. A decent hardware RAID controller with battery backed cache could handle that load with a pair of spinning 15k drives in RAID-1 probably. Another option might be a JBOD box attached to the machine that holds 12 or so 2.5 15k like the hitachi ultrastar 147G 2.5 drives. This sounds like a problem you need to be able to throw a lot of drives at at one time. Is it likely to grow much after this? JBOD in an external casing would be an alternative, especially when using an external case. And no, the database will not grow too much after reaching its final size. Yeah, if it's not gonna grow a lot more after the 2B rows, then you probably won't need an external case. -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] RAID card recommendation
The problem with RAID-5 or RAID-6 is not the normal speed operation, it's the degraded performance when there is a drive failure. This includes read-only scenarios. A DB server getting any kind of real use will effectively appear to be down to client apps if it loses a drive from that RAID set. Basically, think of RAID-5/6 as RAID-0 but with much slower writes, and a way to recover the data without going to backup tapes if there is a disc loss. It is NOT a solution for staying up in case of a failure. Presumably, there is a business reason that you're thinking of using RAID-5/6 with hardware RAID and maybe a hot spare, rather than software RAID-0 which would save you 2-3 spindles of formatted capacity, plus the cost of the RAID card. Whatever that reason is, it's also a reason to use RAID-10. If you absolutely need it to fit in 2U of rack space, you can get a 2U server with a bunch of 2.5 spindles and with 24x 500GB SATA you can get the same formatted size with RAID-10; or you can use an external SAS expander to put additional 3.5 drives in another enclosure. If we're taking rackmount server RAID card votes, I've had good experiences with the LSI under Linux. Cheers Dave On Tue, Nov 24, 2009 at 11:23 AM, Matthew Wakeling matt...@flymine.orgwrote: We're about to purchase a new server to store some of our old databases, and I was wondering if someone could advise me on a RAID card. We want to make a 6-drive SATA RAID array out of 2TB drives, and it will be RAID 5 or 6 because there will be zero write traffic. The priority is stuffing as much storage into a small 2U rack as possible, with performance less important. We will be running Debian Linux. People have mentioned Areca as making good RAID controllers. We're looking at the Areca ARC-1220 PCI-Express x8 SATA II as a possibility. Does anyone have an opinion on whether it is a turkey or a star? Another possibility is a 3-ware card of some description. Thanks in advance, Matthew -- Now you see why I said that the first seven minutes of this section will have you looking for the nearest brick wall to beat your head against. This is why I do it at the end of the lecture - so I can run. -- Computer Science lecturer -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] RAID card recommendation
Tuesday, November 24, 2009, 10:34:00 PM you wrote: Aberdeen is the builder I use. They'll put any card in you want (within reason) including our preference here, Areca. Perhaps you meant Areca? I knew Areca only for their internal arrays (which one of our customers uses for his 19 systems), but did not know they manufacture their own controllers. Added the ARC-1212+BBU to my wishlist :-) -- Jochen Erwied | home: joc...@erwied.eu +49-208-38800-18, FAX: -19 Sauerbruchstr. 17 | work: j...@mbs-software.de +49-2151-7294-24, FAX: -50 D-45470 Muelheim | mobile: jochen.erw...@vodafone.de +49-173-5404164 -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] RAID card recommendation
Jochen Erwied wrote: Tuesday, November 24, 2009, 10:34:00 PM you wrote: Aberdeen is the builder I use. They'll put any card in you want (within reason) including our preference here, Areca. Perhaps you meant Areca? I knew Areca only for their internal arrays (which one of our customers uses for his 19" systems), but did not know they manufacture their own controllers. Added the ARC-1212+BBU to my wishlist :-) For what it's worth I'm using the Adaptec 5445Z on my new server (don't have Postgre running on it) and have been happy with it. For storage on my servers I use http://www.pc-pitstop.com/sas_cables_enclosures/scsase16.asp which has an Areca ARC-8020 expander in it. With the 5445Z I use the 4 internal ports for a fast RAID0 "working array" with 450G Seagate 15k6 drives and the external goes to the 16 drive enclosure through the expander. With 16 drives you have a lot of possibilities for configuring arrays. I have another server with an Adaptec 52445 (don't have Postgre running on it either) connected to two of the 16 drive enclosures and am happy with it. I'm running Postgre on my workstation that has an Adaptec 52445 hooked up to two EnhanceBox-E8MS (http://www.enhance-tech.com/products/desktop/E8_Series.html). I have 8 ST373455SS drives in my tower and 8 in the EnhanceBox so my database is running off 16 drives in RAID5. Everyone complains about RAID5 but it works for me in my situation. Very very rarely am I waiting on the disks when running queries. The other EnhanceBox has 8 ST31000640SS drives in RAID5 just for backup images. All 24 drives run off the 52445 and again, I've been satisfied with it. I've also been happy with the Enhance Technology products. Sorry for being so long but just wanted to put a plug in for the Adaptec cards and let you know about the external options. The 5 series cards are a huge improvement over the 3 series. I had a 3805 and wasn't that impressed. It's actually sitting on my shelf now collecting dust. Bob
Re: [PERFORM] RAID card recommendation
Matthew Wakeling wrote: People have mentioned Areca as making good RAID controllers. We're looking at the Areca ARC-1220 PCI-Express x8 SATA II as a possibility. Does anyone have an opinion on whether it is a turkey or a star? Performance should be OK but not great compared with some of the newer alternatives (this design is a few years old now). The main issue I've had with this series of cards is that the command-line tools are very hit or miss. See http://notemagnet.blogspot.com/2008/08/linux-disk-failures-areca-is-not-so.html for a long commentary about the things I was disappointed by on the similar ARC-1210 once I actually ran into a drive failure on one. As Scott points out there, they have other cards with a built-in management NIC that allows an alternate management path, and I believe those have better performance too. Another possibility is a 3-ware card of some description. I've put a fair number of 9690SA cards in systems with little to complain about. Performance was reasonable as long as you make sure to tweak the read-ahead: http://www.3ware.com/kb/article.aspx?id=11050 Ignore most of the rest of their advice on that page though--for example, increasing vm.dirty_background_ratio and vm.dirty_ratio is an awful idea for PostgreSQL use, where if anything you want to decrease the defaults. Also, while they claim you can connect SAS drives to these cards, they don't support sending SMART commands to them and support seemed pretty limited overall for them. Stick with plain on SATA ones. -- Greg Smith2ndQuadrant Baltimore, MD PostgreSQL Training, Services and Support g...@2ndquadrant.com www.2ndQuadrant.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] RAID card recommendation
Scott Marlowe wrote: As far as drives go we've been really happy with WD of late, they make large enterprise class SATA drives that don't pull a lot of power (green series) and fast SATA drives that pull a bit more but are faster (black series). Be careful to note the caveat that you need their *enterprise class* drives. When you run into an error on their regular consumer drives, they get distracted for a while trying to cover the whole thing up, in a way that's exactly the opposite of the behavior you want for a RAID configuration. I have a regular consumer WD drive that refuses to admit that it has a problem such that I can RMA it, but that always generates an error if I rewrite the whole drive. The behavior of the firmware is downright shameful. As cheap consumer drives go, I feel like WD has pulled ahead of everybody else on performance and possibly even actual reliability, but the error handling of their firmware is so bad I'm still using Seagate drives--when those fail, as least they're honest about it. -- Greg Smith2ndQuadrant Baltimore, MD PostgreSQL Training, Services and Support g...@2ndquadrant.com www.2ndQuadrant.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] RAID card recommendation
Jochen Erwied wrote: - Promise Technology Supertrak ES4650 + additional BBU - Adaptec RAID 5405 SGL/256 SATA/SAS + additional BBU - Adaptec RAID 5405Z SGL/512 SATA/SAS I've never seen a Promise controller that had a Linux driver you would want to rely on under any circumstances. Adaptec used to have seriously bad Linux drivers too. I've gotten the impression they've cleaned up their act considerably the last few years, but they've been on my list of hardware to shun for so long I haven't bothered investigating. Easier to just buy from a company that has always cared about good Linux support, like 3ware. In any case, driver quality is what you want to research before purchasing any of these; doesn't matter how fast the cards are if they crash or corrupt your data. What I like to do is look at what companies who sell high-quality production servers with Linux preinstalled and see what hardware they include. You can find a list of vendors people here like at http://wiki.postgresql.org/wiki/SCSI_vs._IDE/SATA_Disks#Helpful_vendors_of_SATA_RAID_systems -- Greg Smith2ndQuadrant Baltimore, MD PostgreSQL Training, Services and Support g...@2ndquadrant.com www.2ndQuadrant.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
Re: [PERFORM] RAID card recommendation
On Tue, Nov 24, 2009 at 7:35 PM, Greg Smith g...@2ndquadrant.com wrote: Scott Marlowe wrote: As far as drives go we've been really happy with WD of late, they make large enterprise class SATA drives that don't pull a lot of power (green series) and fast SATA drives that pull a bit more but are faster (black series). Be careful to note the caveat that you need their *enterprise class* drives. When you run into an error on their regular consumer drives, they get distracted for a while trying to cover the whole thing up, in a way that's exactly the opposite of the behavior you want for a RAID configuration. I have a regular consumer WD drive that refuses to admit that it has a problem such that I can RMA it, but that always generates an error if I rewrite the whole drive. The behavior of the firmware is downright shameful. As cheap consumer drives go, I feel like WD has pulled ahead of everybody else on performance and possibly even actual reliability, but the error handling of their firmware is so bad I'm still using Seagate drives--when those fail, as least they're honest about it. When I inquired earlier this summer about using the consumer WDs in a new server I was told rather firmly by my sales guy uhm, no. They put the enterprise drives through the wringer before he said they seemed ok. They have been great, both green and black series. For what they are, big SATA drives in RAID-6 or RAID-10 they're quite good. Moderate to quite good performers at a reasonable price. -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
