Re: [CentOS] SATA vs. SAS
On Wednesday 22 August 2007, Rajeev R Veedu wrote: ... > Option 1) 2 servers each having 2.0TB raid disk with SAS drives, 2GB ram > and standard other features. > > Option 2) 4 No servers with 1TB each with 2GB ram and standard other > features. Late comment on this thread. Really short comment actually: SAS vs. SATA is not the issue, raid controller A vs. B is. /Peter signature.asc Description: This is a digitally signed message part. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] SATA vs. SAS
Binary drivers from Dell? The HBA that connects to the MD3000 is just an mptsas driver which is part of the stock kernel, but you can download the latest version from Dell's website as a dkms source package. Thank you Ross for the information. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
RE: [CentOS] SATA vs. SAS
From: Peter Arremann >On Wednesday 22 August 2007, Bowie Bailey wrote: > Peter Arremann wrote: >> > On the other hand, data reliability is another issue. >> Why do you say that SATA arrays are less reliable? >Not all drive support cache flushes and handle them correctly - even with NCQ. >Same for some older controllers also have some issues too. >Doesn't show up as a hardware error but as filesystem inconsistency after a >crash. >As I wrote, we haven't had issues yet either. But sun, sgi, ibm and others are >fairly conservative - sun says they still only ships 500GB disks in their >x4500 for that reason. EMC and IBM are shipping Seagate Barracuda ES 750GB drives now. Just bought and installed two CLARiiON CX3-10c's with two DAE3's each, full of 750GB SATA II drives (the interesting thing is that the DAE is still 4Gb/s FC; the SATA carriers have an emulex bridge board translating the FC-AL to SATA II on the carrier; the DAE's are FC all the way). The IBM DS4200 is available with SATA II. I chose EMC due to software features and VMware support 'stuff' even though it was quite a bit more $$ per TB. We have two 20TB systems at this point. Performance is excellent, at least according to bonnie++. I expected random access to suffer due to the 7200 RPM drives (versus what 15K drives would have been), and it did. Block writes from a CentOS 4 VM through ESX's multipathing through two Qlogic 4Gb/s PCIe 4x FC controllers was 125MB/s or so, RAID5 5 drive RAID groups and 1.95TB LUNs. EMC and IBM both made it clear that they consider SATA second tier well below FC; but FC is, of course, much more expensive. -- Lamar Owen Chief Information Officer Pisgah Astronomical Research Institute 1 PARI Drive Rosman, NC 28772 828-862-5554 www.pari.edu ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] SATA vs. SAS
On Wednesday 22 August 2007, Bowie Bailey wrote: > Peter Arremann wrote: > > On the other hand, data reliability is another issue. We have tons of > > sata based disk arrays and have had no issues yet (because our systems > > are all on UPS and multiple circuits) but if you don't have > > infrastructure like that, you are more likely to lose data on a sata > > based system... > > Why do you say that SATA arrays are less reliable? I have used both > SATA and SCSI raid and have had drive failures on both. Recovery from > the failures seems to be more a matter of the raid implementation than > the interface type. Not all drive support cache flushes and handle them correctly - even with NCQ. Same for some older controllers also have some issues too. Doesn't show up as a hardware error but as filesystem inconsistency after a crash. As I wrote, we haven't had issues yet either. But sun, sgi, ibm and others are fairly conservative - sun says they still only ships 500GB disks in their x4500 for that reason. Peter. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
RE: [CentOS] SATA vs. SAS
> -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Feizhou > Sent: Wednesday, August 22, 2007 10:54 AM > To: CentOS mailing list > Subject: Re: [CentOS] SATA vs. SAS > > > >>>> Option 1) 2 servers each having 2.0TB raid disk with SAS > >>>> drives, 2GB ram and standard other features. > >>> If going down this road, why not look into getting one of > >>> those fancy new storage enclosures where the RAID is built > >>> into the enclosure and can allow 2 servers to simultaneously > >>> access the arrays with full battery backed write-back cache? > >>> > >> What is available for Linux in this department? > > > > I'm testing out an MD3000 from Dell. It can allow 4 hosts with > > single 4x serial paths, or 2 hosts with redundant 4x serial > > and can chain up to 2 MD1000s off it for up to 45 spindles. > > > > It was 2 RAID controllers in 2 EMMs with 512MB BBU write-back > > that is synchronized between them which act as redundant RAID > > controllers. Ships with 4 plain-jane 2 path SAS controllers > > for host systems. > > > > Downside, right now, it currently only supports SAS drives, > > they hope to have a SAS/SATA firmware update maybe by year-end. > > Binary drivers from Dell? The HBA that connects to the MD3000 is just an mptsas driver which is part of the stock kernel, but you can download the latest version from Dell's website as a dkms source package. -Ross __ This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] SATA vs. SAS
Option 1) 2 servers each having 2.0TB raid disk with SAS drives, 2GB ram and standard other features. If going down this road, why not look into getting one of those fancy new storage enclosures where the RAID is built into the enclosure and can allow 2 servers to simultaneously access the arrays with full battery backed write-back cache? What is available for Linux in this department? I'm testing out an MD3000 from Dell. It can allow 4 hosts with single 4x serial paths, or 2 hosts with redundant 4x serial and can chain up to 2 MD1000s off it for up to 45 spindles. It was 2 RAID controllers in 2 EMMs with 512MB BBU write-back that is synchronized between them which act as redundant RAID controllers. Ships with 4 plain-jane 2 path SAS controllers for host systems. Downside, right now, it currently only supports SAS drives, they hope to have a SAS/SATA firmware update maybe by year-end. Binary drivers from Dell? ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] SATA vs. SAS
Bowie Bailey wrote: Feizhou wrote: Option 1) 2 servers each having 2.0TB raid disk with SAS drives, 2GB ram and standard other features. If going down this road, why not look into getting one of those fancy new storage enclosures where the RAID is built into the enclosure and can allow 2 servers to simultaneously access the arrays with full battery backed write-back cache? What is available for Linux in this department? http://www.coraid.com/ These are relatively inexpensive network storage boxes using SATA drives and AoE (ATA over Ethernet) connection to the server. Drivers build easily on CentOS. Very interesting. Just make sure you get drives that are on their compatible list. This may not be as much of an issue now, but I had some problems with drives being marked bad and dropped out of the array when I first set up mine a couple of years ago. This was caused by firmware incompatibilities and fixed by upgrading the drive firmware. Once this was fixed, everything has worked flawlessly since. Thank you for the link and the heads up! ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
RE: [CentOS] SATA vs. SAS
> -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Feizhou > Sent: Wednesday, August 22, 2007 9:54 AM > To: CentOS mailing list > Subject: Re: [CentOS] SATA vs. SAS > > > >> Option 1) 2 servers each having 2.0TB raid disk with SAS > >> drives, 2GB ram and standard other features. > > > > If going down this road, why not look into getting one of > > those fancy new storage enclosures where the RAID is built > > into the enclosure and can allow 2 servers to simultaneously > > access the arrays with full battery backed write-back cache? > > > > What is available for Linux in this department? I'm testing out an MD3000 from Dell. It can allow 4 hosts with single 4x serial paths, or 2 hosts with redundant 4x serial and can chain up to 2 MD1000s off it for up to 45 spindles. It was 2 RAID controllers in 2 EMMs with 512MB BBU write-back that is synchronized between them which act as redundant RAID controllers. Ships with 4 plain-jane 2 path SAS controllers for host systems. Downside, right now, it currently only supports SAS drives, they hope to have a SAS/SATA firmware update maybe by year-end. -Ross __ This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
RE: [CentOS] SATA vs. SAS
Feizhou wrote: > > > Option 1) 2 servers each having 2.0TB raid disk with SAS > > > drives, 2GB ram and standard other features. > > > > If going down this road, why not look into getting one of > > those fancy new storage enclosures where the RAID is built > > into the enclosure and can allow 2 servers to simultaneously > > access the arrays with full battery backed write-back cache? > > > > What is available for Linux in this department? http://www.coraid.com/ These are relatively inexpensive network storage boxes using SATA drives and AoE (ATA over Ethernet) connection to the server. Drivers build easily on CentOS. Just make sure you get drives that are on their compatible list. This may not be as much of an issue now, but I had some problems with drives being marked bad and dropped out of the array when I first set up mine a couple of years ago. This was caused by firmware incompatibilities and fixed by upgrading the drive firmware. Once this was fixed, everything has worked flawlessly since. -- Bowie ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] SATA vs. SAS
Bowie Bailey wrote: > > Recovery from > the failures seems to be more a matter of the raid implementation than > the interface type. > My 'day job' is with a data recovery firm. I cannot agree more fervently with Bowie's comment above. ~Ray ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
RE: [CentOS] SATA vs. SAS
Peter Arremann wrote: > > On the other hand, data reliability is another issue. We have tons of > sata based disk arrays and have had no issues yet (because our systems > are all on UPS and multiple circuits) but if you don't have > infrastructure like that, you are more likely to lose data on a sata > based system... Why do you say that SATA arrays are less reliable? I have used both SATA and SCSI raid and have had drive failures on both. Recovery from the failures seems to be more a matter of the raid implementation than the interface type. -- Bowie ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] SATA vs. SAS
Option 1) 2 servers each having 2.0TB raid disk with SAS drives, 2GB ram and standard other features. If going down this road, why not look into getting one of those fancy new storage enclosures where the RAID is built into the enclosure and can allow 2 servers to simultaneously access the arrays with full battery backed write-back cache? What is available for Linux in this department? ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
RE: [CentOS] SATA vs. SAS
> -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Rajeev R Veedu > Sent: Wednesday, August 22, 2007 2:52 AM > To: 'CentOS mailing list' > Subject: [CentOS] SATA vs. SAS > > I have 8 WD SATA HDD with raid ready (3mbps) hard disks on a > 8 port 3ware controller.(on raid 5.) Does anyone have a > comparison on SATA raid and SAS raid disk. As you know SAS > disk are very expensive and I would like to know from experts > in the list who could suggest which of the following would be > the best. As a lot of people have probably said the answer depends on your workload. If the workload is mostly small random io I would go with 15K SAS configured into a raid10. If the workload is mostly fairly large sequential reads/writes (file server) then I would probably go with SATA 7200 RAID5/6 or RAID50/60. If doing databases set the chunk size to the maximum size of a data dump i/o (usually 1MB) so each dump i/o hits a separate spindle, for random it doesn't really matter cause it's random, you just want the fastest access time money can afford. For file services you will have to gauge the chunk size by the type of files, mostly small, small chunks, mostly large, then larger chunks, 64K is the standard middle-of the road here. > Option 1) 2 servers each having 2.0TB raid disk with SAS > drives, 2GB ram and standard other features. If going down this road, why not look into getting one of those fancy new storage enclosures where the RAID is built into the enclosure and can allow 2 servers to simultaneously access the arrays with full battery backed write-back cache? > Option 2) 4 No servers with 1TB each with 2GB ram and > standard other features. I don't know if I follow you here... > If Data files (mostly AutoCAD Drawings of size 5MB to 50MB) > are distributed as per the above options do you think which > could perform better?. As you know the price of SATA disk is > much cheaper than the SAS disk and we could nearly by 4 > servers for that money. If that's the workload, then save the money and go SATA in a RAID50 (two RAID5's striped), say a 4-spindle/4-spindle if you could go to say 10-12 disks then do a 5-spindle/5-spindle with 1 or 2 hotspares. > Probability work disturbed by a server crash is low in the > second case but I am not sure about the comparison on performance. Look at a shared storage solution so you can have the storage fail-over in the even of a server crash rather then replicate it. > I would appreciate if you could spread some thought in this > regards, and apologize if this is out of topic. > > > > Best regards, > > > > Rajeev > > > > __ This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] SATA vs. SAS
On Wednesday 22 August 2007, Rajeev R Veedu wrote: > I have 8 WD SATA HDD with raid ready (3mbps) hard disks on a 8 port 3ware > controller.(on raid 5.) Does anyone have a comparison on SATA raid and SAS > raid disk. As you know SAS disk are very expensive and I would like to know > from experts in the list who could suggest which of the following would be > the best. I got a 4 port 9650 with 4 750GB Seagate drives in raid 0+1 on it for data storage, and a 8port LSI LSI00110 with 4 36GB 15K rpm disks, again raid 0+1, for the OS and swap. When I take a ext3 filesystem and run bonnie++ then the SATA solution is about 60% faster for sequential reads. However, for random access, the SAS solution is about 3 times faster. So - as always in this world - the answer depends on your usage pattern. > Option 1) 2 servers each having 2.0TB raid disk with SAS drives, 2GB ram > and standard other features. > > Option 2) 4 No servers with 1TB each with 2GB ram and standard other > features. > > If Data files (mostly AutoCAD Drawings of size 5MB to 50MB) are distributed > as per the above options do you think which could perform better?. As you > know the price of SATA disk is much cheaper than the SAS disk and we could > nearly by 4 servers for that money. Its been a while since I did AutoCAD but anyway - why only 2GB? As for the IO, AutoCAD (assuming you have enough ram so your system doesn't swap) doesn't do a lot of i/o - and if so, its mostly sequential. So, without having tried it, my guess is that you will not see much of a difference either way. I'd go with the 4 servers. On the other hand, data reliability is another issue. We have tons of sata based disk arrays and have had no issues yet (because our systems are all on UPS and multiple circuits) but if you don't have infrastructure like that, you are more likely to lose data on a sata based system... I personally would still go sata. Peter. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
