Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-15 Thread Pasi Kärkkäinen
On Fri, Jan 15, 2010 at 11:02:51AM +0100, Peter Kjellstrom wrote: > On Thursday 14 January 2010, Pasi Kärkkäinen wrote: > > On Thu, Jan 14, 2010 at 08:14:52PM +, Karanbir Singh wrote: > ... > > > Maybe its just bad luck here :) > > > > I remember a story about two similar HP proliants.. same mo

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-15 Thread Benjamin Donnachie
2010/1/15 Peter Kjellstrom : > IMO the most likely reason for one server working and not another one would be > HP shipping (or bounce-your-servers-around-the-globe as I like to call it)... Sadly that problem does not seem unique to HP. Ben ___ CentOS m

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-15 Thread Peter Kjellstrom
On Thursday 14 January 2010, Pasi Kärkkäinen wrote: > On Thu, Jan 14, 2010 at 08:14:52PM +, Karanbir Singh wrote: ... > > Maybe its just bad luck here :) > > I remember a story about two similar HP proliants.. same model number, > ordered the same day, same hardware configuration etc.. > > The

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-15 Thread Peter Kjellstrom
On Thursday 14 January 2010, John R Pierce wrote: > Karanbir Singh wrote: > > My main issue with that kit is that the linux drivers are very basic, > > lack most management capabilities and fail often with obscure issues. We certainly don't see a high frequency of obscure-cciss-issues. But since n

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-14 Thread Pasi Kärkkäinen
On Thu, Jan 14, 2010 at 08:14:52PM +, Karanbir Singh wrote: > On 01/12/2010 03:51 PM, nate wrote: > > I've used HP/cciss on a couple hundred systems over the past 7 years, > > can only recall 2 issues, both around a drive failing the controller > > didn't force the drive off line, and there was

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-14 Thread John R Pierce
Karanbir Singh wrote: > My main issue with that kit is that the linux drivers are very basic, > lack most management capabilities and fail often with obscure issues. > And, as Peter pointed out already, they are not really exposing a proper > scsi interface, but modeled around a really old ata s

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-14 Thread Karanbir Singh
On 01/12/2010 03:51 PM, nate wrote: > I've used HP/cciss on a couple hundred systems over the past 7 years, > can only recall 2 issues, both around a drive failing the controller > didn't force the drive off line, and there was no way to force it > off line using the command line tool, so had to go

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-14 Thread Pasi Kärkkäinen
On Thu, Jan 14, 2010 at 08:07:43PM +, Karanbir Singh wrote: > On 01/12/2010 10:43 AM, John Doe wrote: > > On the other hand, here, we have around 30 HP servers. > > Some DL360/380/180 G5/G6 with CentOS 4/5 and, > > in 2 years, only 3 drives failed... That's it; no other problems... > > Drives

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-14 Thread Karanbir Singh
On 01/12/2010 10:43 AM, John Doe wrote: > On the other hand, here, we have around 30 HP servers. > Some DL360/380/180 G5/G6 with CentOS 4/5 and, > in 2 years, only 3 drives failed... That's it; no other problems... Drives is hardly the issue - most of them are going to be seagate anyway. My main

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-13 Thread Chan Chung Hang Christopher
> On the machine where I had the problem I had to run memtest86 more than a day > to > finally catch it. Then after replacing the RAM and fsck'ing the volume, I > still > had mysterious problems about once a month until I realized that the disks > are > accessed alternately and the fsck pas

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-13 Thread Les Mikesell
Peter Kjellstrom wrote: >>> >> Please tell more about your hardware and software. What distro? What >> kernel? What disk controller? What disks? > > Both of my data-points are several years old so most of the details are lost > in the fog-of-lost-memories... > > Both were on desktop class hardwa

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-13 Thread Les Mikesell
Christopher Chan wrote: Funny you should mention software RAID1... I've seen two instances of that >>> getting silently out-of-sync and royally screwing things up beyond all >>> repair. >>> >>> Maybe this thread has gone on long enough now? >>> >> Not yet :) >> >> Please tell more about your

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-13 Thread Pasi Kärkkäinen
On Wed, Jan 13, 2010 at 11:43:35AM +0100, Peter Kjellstrom wrote: > On Wednesday 13 January 2010, Pasi Kärkkäinen wrote: > > On Wed, Jan 13, 2010 at 01:05:39AM +0100, Peter Kjellstrom wrote: > > > On Tuesday 12 January 2010, Les Mikesell wrote: > > > > On 1/12/2010 10:39 AM, Peter Kjellstrom wrote:

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-13 Thread Peter Kjellstrom
On Wednesday 13 January 2010, Pasi Kärkkäinen wrote: > On Wed, Jan 13, 2010 at 01:05:39AM +0100, Peter Kjellstrom wrote: > > On Tuesday 12 January 2010, Les Mikesell wrote: > > > On 1/12/2010 10:39 AM, Peter Kjellstrom wrote: > > > > ... > > > > > >>> ...that said, it's not much worse than the comp

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-12 Thread Christopher Chan
Pasi Kärkkäinen wrote: > On Wed, Jan 13, 2010 at 01:05:39AM +0100, Peter Kjellstrom wrote: >> On Tuesday 12 January 2010, Les Mikesell wrote: >>> On 1/12/2010 10:39 AM, Peter Kjellstrom wrote: >> ... >> ...that said, it's not much worse than the competetion, storage simply >> sucks ;-(

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-12 Thread Pasi Kärkkäinen
On Wed, Jan 13, 2010 at 01:05:39AM +0100, Peter Kjellstrom wrote: > On Tuesday 12 January 2010, Les Mikesell wrote: > > On 1/12/2010 10:39 AM, Peter Kjellstrom wrote: > ... > > >>> ...that said, it's not much worse than the competetion, storage simply > > >>> sucks ;-( > > >> > > >> So you are sayi

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-12 Thread Les Mikesell
On 1/12/2010 6:05 PM, Peter Kjellstrom wrote: > > ... > ...that said, it's not much worse than the competetion, storage simply > sucks ;-( So you are saying people dole out huge amounts of money for rubbish? That the software raid people were and have always been right? >>> >

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-12 Thread Peter Kjellstrom
On Tuesday 12 January 2010, Les Mikesell wrote: > On 1/12/2010 10:39 AM, Peter Kjellstrom wrote: ... > >>> ...that said, it's not much worse than the competetion, storage simply > >>> sucks ;-( > >> > >> So you are saying people dole out huge amounts of money for rubbish? > >> That the software rai

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-12 Thread Drew
>> ...that said, it's not much worse than the competetion, storage simply >> sucks ;-( > > So you are saying people dole out huge amounts of money for rubbish? > That the software raid people were and have always been right? Depends what the software raid people were saying. :) Hardware & Softwar

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-12 Thread Keith Keller
On Tue, Jan 12, 2010 at 09:01:42AM +0100, Peter Kjellstrom wrote: > > Is that supposed to be a joke? 3ware has certainly had their fair share of > stability problems (drive time-outs, bbu-problems, inconsistent > behaviour, ...) and monitoring wise they suck (imho). Do you like tw_cli? I don't

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-12 Thread Tom Georgoulias
On 01/12/2010 12:20 PM, JohnS wrote: > > On Mon, 2010-01-11 at 16:16 -0500, Tom Georgoulias wrote: >> CentOS 5.4 x86_64 works fine on the x4540s, I've installed it myself and >> didn't have to do anything special to see and use all of the disks. >> >> In my testing, the IO was faster and the stora

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-12 Thread JohnS
On Mon, 2010-01-11 at 16:16 -0500, Tom Georgoulias wrote: > > CentOS 5.4 x86_64 works fine on the x4540s, I've installed it myself and > didn't have to do anything special to see and use all of the disks. > > In my testing, the IO was faster and the storage easier to administer > with when us

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-12 Thread Les Mikesell
On 1/12/2010 10:39 AM, Peter Kjellstrom wrote: > Which is why I specifically said 'performance wise' as respects 3ware. I don't remember anything bad about 3ware stability wise or monitoring wise. >>> >>> Is that supposed to be a joke? 3ware has certainly had their fair share >>> of

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-12 Thread Peter Kjellstrom
On Tuesday 12 January 2010, Chan Chung Hang Christopher wrote: > >> Which is why I specifically said 'performance wise' as respects 3ware. I > >> don't remember anything bad about 3ware stability wise or monitoring > >> wise. > > > > Is that supposed to be a joke? 3ware has certainly had their fair

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-12 Thread Karanbir Singh
On 12/01/10 12:22, Rainer Duffner wrote: > Which is probably the reason why the ZFS-folks are trying to move as > much intelligence out of the HBA into the OS. not something that is really working - given that I've seen centos stock with a few hba's easily our perform raid-z - with better reliabil

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-12 Thread nate
Pasi Kärkkäinen wrote: > And I've been running DL380 and DL360 G3/G4 servers for years without > problems.. with CentOS and Xen.. using cciss local storage. :) I've used HP/cciss on a couple hundred systems over the past 7 years, can only recall 2 issues, both around a drive failing the controlle

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-12 Thread Pasi Kärkkäinen
On Tue, Jan 12, 2010 at 09:41:19AM +, Karanbir Singh wrote: > On 12/01/10 00:02, Christopher Chan wrote: > >> problems mostly centered around management and performance issues. the > >> world is littered with stores of cciss fail > > Really? Man, I have been given this spanking new HP DL370 G6

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-12 Thread Benjamin Donnachie
2010/1/12 Chan Chung Hang Christopher : > Boy, a Tyan or Supermicro solution is looking better by the minute for > the new server I plan to get the school for its library server and other > uses. If only Supermicro had a local distributor...I have not had a good > look at their solutions yet becaus

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-12 Thread Chan Chung Hang Christopher
Per Qvindesland wrote: > Hi > > Appologies I have not been following the thread here so am just > wondering if you have a MSA, EVA, XP left hand san or if this is just > storage that sits on the server with samba share? also what link is > between fc or ethernet. If you are asking me, then there

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-12 Thread Rainer Duffner
Am 12.01.2010 09:01, schrieb Peter Kjellstrom: > > Is that supposed to be a joke? 3ware has certainly had their fair share of > stability problems (drive time-outs, bbu-problems, inconsistent > behaviour, ...) and monitoring wise they suck (imho). Do you like tw_cli? > Enjoying the fact that "sh

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-12 Thread Chan Chung Hang Christopher
Benjamin Donnachie wrote: > 2010/1/12 Chan Chung Hang Christopher : >> Eeek! That thing will be hosting the school's vle. Looks like I better >> memorize the after hours password for HP support. > > I have had lots[1] of problems lately with DIMMs becoming defective in > six month old G5 HPs. Cou

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-12 Thread Chan Chung Hang Christopher
>> Which is why I specifically said 'performance wise' as respects 3ware. I >> don't remember anything bad about 3ware stability wise or monitoring wise. > > Is that supposed to be a joke? 3ware has certainly had their fair share of > stability problems (drive time-outs, bbu-problems, inconsiste

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-12 Thread Benjamin Donnachie
2010/1/12 Chan Chung Hang Christopher : > Eeek! That thing will be hosting the school's vle. Looks like I better > memorize the after hours password for HP support. I have had lots[1] of problems lately with DIMMs becoming defective in six month old G5 HPs. Could just be bad luck or maybe just pu

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-12 Thread Per Qvindesland
Hi Appologies I have not been following the thread here so am just wondering if you have a MSA, EVA, XP left hand san or if this is just storage that sits on the server with samba share? also what link is between fc or ethernet. Regards Per Qvindesland At Tisdag, 12-01-2010 on 11:57 "Chan Chung

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-12 Thread Chan Chung Hang Christopher
Karanbir Singh wrote: > On 12/01/10 00:02, Christopher Chan wrote: >>> problems mostly centered around management and performance issues. the >>> world is littered with stores of cciss fail >> Really? Man, I have been given this spanking new HP DL370 G6 and running >> Centos 5.4 on it... > > I'v

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-12 Thread John Doe
From: Karanbir Singh > On 12/01/10 00:02, Christopher Chan wrote: > >> problems mostly centered around management and performance issues. the > >> world is littered with stores of cciss fail > > Really? Man, I have been given this spanking new HP DL370 G6 and running > > Centos 5.4 on it... > I'

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-12 Thread Peter Kjellstrom
On Tuesday 12 January 2010, John R Pierce wrote: > Karanbir Singh wrote: > > On 12/01/10 00:02, Christopher Chan wrote: > >>> problems mostly centered around management and performance issues. the > >>> world is littered with stores of cciss fail > >> > >> Really? Man, I have been given this spanki

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-12 Thread John R Pierce
Karanbir Singh wrote: > On 12/01/10 00:02, Christopher Chan wrote: > >>> problems mostly centered around management and performance issues. the >>> world is littered with stores of cciss fail >>> >> Really? Man, I have been given this spanking new HP DL370 G6 and running >> Centos 5.4 o

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-12 Thread Karanbir Singh
On 12/01/10 00:02, Christopher Chan wrote: >> problems mostly centered around management and performance issues. the >> world is littered with stores of cciss fail > Really? Man, I have been given this spanking new HP DL370 G6 and running > Centos 5.4 on it... I've got a couple of DL380's at one

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-12 Thread Peter Kjellstrom
On Tuesday 12 January 2010, Christopher Chan wrote: > Keith Keller wrote: > > On Tue, Jan 12, 2010 at 08:07:17AM +0800, Christopher Chan wrote: > >> I see that the Areca driver has finally made it into the mainline Linux > >> kernel. But I wonder how things have improved from this particular case.

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-11 Thread Christopher Chan
Keith Keller wrote: > On Tue, Jan 12, 2010 at 08:07:17AM +0800, Christopher Chan wrote: >> I see that the Areca driver has finally made it into the mainline Linux >> kernel. But I wonder how things have improved from this particular case. >> >> http://notemagnet.blogspot.com/2008/08/linux-disk-fai

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-11 Thread Keith Keller
On Tue, Jan 12, 2010 at 08:07:17AM +0800, Christopher Chan wrote: > > I see that the Areca driver has finally made it into the mainline Linux > kernel. But I wonder how things have improved from this particular case. > > http://notemagnet.blogspot.com/2008/08/linux-disk-failures-areca-is-not-so.

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-11 Thread Christopher Chan
Bent Terp wrote: > On Wed, Jan 6, 2010 at 10:35 PM, Boris Epstein wrote: >> some storage servers to run under Linux - most likely CentOS. The storage >> volume would be in the range specified: 8-15 TB. Any recommendations as far >> as hardware? > > I'm kind of partial to Areca raid controllers, y

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-11 Thread Christopher Chan
Karanbir Singh wrote: > On 01/08/2010 05:28 PM, R-Elists wrote: >> what is wrong or what problems are you referring to with cciss please ? >> > > problems mostly centered around management and performance issues. the > world is littered with stores of cciss fail > > Really? Man, I have been gi

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-11 Thread Tom Georgoulias
On 01/11/2010 09:42 AM, Rainer Duffner wrote: > Am 11.01.2010 15:26, schrieb Pasi Kärkkäinen: >> X4540 uses LSI SATA controllers, that are supported. >> > > > Indeed: > > http://www.sun.com/servers/x64/x4540/os.jsp > > 5.3+ is needed. > > Of course, for a true Solaris-admin, this would be a big wa

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-11 Thread William Warren
On 1/11/2010 1:33 PM, Les Mikesell wrote: > On 1/11/2010 11:38 AM, John R Pierce wrote: > >> Pasi Kärkkäinen wrote: >> >>> It seems X4500 (not available anymore) had Marvell SATA controllers, that >>> are not supported with RHEL5. >>> >>> >> And those marvell controllers caused ma

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-11 Thread Les Mikesell
On 1/11/2010 11:38 AM, John R Pierce wrote: > Pasi Kärkkäinen wrote: >> It seems X4500 (not available anymore) had Marvell SATA controllers, that >> are not supported with RHEL5. >> > > And those marvell controllers caused major grief for Sun, especially > when Solaris added support for NCQ somewhe

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-11 Thread Bent Terp
On Wed, Jan 6, 2010 at 10:35 PM, Boris Epstein wrote: > some storage servers to run under Linux - most likely CentOS. The storage > volume would be in the range specified: 8-15 TB. Any recommendations as far > as hardware? I'm kind of partial to Areca raid controllers, you can get up to 24 ports,

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-11 Thread John R Pierce
Pasi Kärkkäinen wrote: > It seems X4500 (not available anymore) had Marvell SATA controllers, that > are not supported with RHEL5. > And those marvell controllers caused major grief for Sun, especially when Solaris added support for NCQ somewhere in there. under heavy IO workloads, the contro

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-11 Thread Rainer Duffner
Am 11.01.2010 15:26, schrieb Pasi Kärkkäinen: > > It seems X4500 (not available anymore) had Marvell SATA controllers, that > are not supported with RHEL5. > > X4540 uses LSI SATA controllers, that are supported. > Indeed: http://www.sun.com/servers/x64/x4540/os.jsp 5.3+ is needed. Of cours

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-11 Thread Pasi Kärkkäinen
On Mon, Jan 11, 2010 at 03:00:41PM +0200, Pasi Kärkkäinen wrote: > On Fri, Jan 08, 2010 at 12:33:39PM +0100, Rainer Duffner wrote: > > Karanbir Singh schrieb: > > > On 01/08/2010 01:58 AM, Christopher Chan wrote: > > > > > >>> the thumpers make for decent backup or vtl type roles, not so much fo

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-11 Thread Peter Kjellstrom
On Monday 11 January 2010, Karanbir Singh wrote: > On 01/08/2010 05:28 PM, R-Elists wrote: > > what is wrong or what problems are you referring to with cciss please ? > > problems mostly centered around management and performance issues. the > world is littered with stores of cciss fail I would ce

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-11 Thread Pasi Kärkkäinen
On Fri, Jan 08, 2010 at 12:33:39PM +0100, Rainer Duffner wrote: > Karanbir Singh schrieb: > > On 01/08/2010 01:58 AM, Christopher Chan wrote: > > > >>> the thumpers make for decent backup or vtl type roles, not so much for > >>> online high density storage. > >>> > >> I wonder how much th

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-11 Thread Karanbir Singh
On 01/08/2010 05:28 PM, R-Elists wrote: > what is wrong or what problems are you referring to with cciss please ? > problems mostly centered around management and performance issues. the world is littered with stores of cciss fail -- Karanbir Singh London, UK| http://www.karan.org/ | t

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-10 Thread JohnS
On Sat, 2010-01-09 at 07:14 -0800, nate wrote: > JohnS wrote: > > > Interesting link for info there. I found [1] and at the bottom of the > > page there is like tidbits of info in PDFs of the different models. Any > > idea where I could get more info than that, like data sheets and case > > st

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-09 Thread nate
JohnS wrote: > Interesting link for info there. I found [1] and at the bottom of the > page there is like tidbits of info in PDFs of the different models. Any > idea where I could get more info than that, like data sheets and case > studies. Not online at least, note the "Confidential" stuff a

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-08 Thread JohnS
On Fri, 2010-01-08 at 17:09 -0800, nate wrote: > > Using 15K RPM drives I can tell you that a 3PAR T400(very well > versed in their products, fast easy to use) can do 25.6 Gbits/second > (3.2 gigabytes/second) sustained throughput. 640 drives, 48GB data > cache. > > If you were starting out at

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-08 Thread JohnS
On Fri, 2010-01-08 at 17:53 -0800, John R Pierce wrote: > >> Your ROI of 5 minutes doesn't make any sense to me. > >> > > > > Ok, Job submission and completion is what I am getting at. > > > > ROI generally refers to the time an expense takes to pay off.Like, > if buying $X worth of

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-08 Thread John R Pierce
>> Your ROI of 5 minutes doesn't make any sense to me. >> > > Ok, Job submission and completion is what I am getting at. > ROI generally refers to the time an expense takes to pay off.Like, if buying $X worth of capital equipment will generate savings or additional income of $x over

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-08 Thread nate
JohnS wrote: > > On Fri, 2010-01-08 at 16:08 -0800, nate wrote: >> JohnS wrote: >> >> > Currently using the older model of this one [1] @ 4GB/s on the fiber. >> >> You sound pretty confused, there's no way in hell a Fujitsu DX440 >> is going to sustain 4 gigabytes/second, maybe 4 Gigabits/second >>

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-08 Thread JohnS
On Fri, 2010-01-08 at 16:08 -0800, nate wrote: > JohnS wrote: > > > Currently using the older model of this one [1] @ 4GB/s on the fiber. > > You sound pretty confused, there's no way in hell a Fujitsu DX440 > is going to sustain 4 gigabytes/second, maybe 4 Gigabits/second > (~500MB/s) > G Bits

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-08 Thread JohnS
On Fri, 2010-01-08 at 15:43 -0800, John R Pierce wrote: > JohnS wrote: > > Just asking is the fiber ports BiDirectional or Directional or can they > > support a Bond that is BiDirectional of 4GB/s or can they be trunked > > into 16GB/s? Bidirectional. I need about 24 GB/s banwidth sustained, >

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-08 Thread nate
JohnS wrote: > Currently using the older model of this one [1] @ 4GB/s on the fiber. You sound pretty confused, there's no way in hell a Fujitsu DX440 is going to sustain 4 gigabytes/second, maybe 4 Gigabits/second (~500MB/s) > Thats with BiDirectional, both links at 4 GB/s. Were looking for >

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-08 Thread JohnS
On Fri, 2010-01-08 at 15:23 -0800, nate wrote: > JohnS wrote: > > > Just asking is the fiber ports BiDirectional or Directional or can they > > support a Bond that is BiDirectional of 4GB/s or can they be trunked > > into 16GB/s? Bidirectional. I need about 24 GB/s banwidth sustained, > > yes

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-08 Thread John R Pierce
JohnS wrote: > Just asking is the fiber ports BiDirectional or Directional or can they > support a Bond that is BiDirectional of 4GB/s or can they be trunked > into 16GB/s? Bidirectional. I need about 24 GB/s banwidth sustained, > yes per second. Also what type of sparse file I/O you get . I

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-08 Thread nate
JohnS wrote: > Just asking is the fiber ports BiDirectional or Directional or can they > support a Bond that is BiDirectional of 4GB/s or can they be trunked > into 16GB/s? Bidirectional. I need about 24 GB/s banwidth sustained, > yes per second. Also what type of sparse file I/O you get . I

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-08 Thread JohnS
On Fri, 2010-01-08 at 14:36 -0800, nate wrote: > Les Mikesell wrote: > > On 1/8/2010 11:41 AM, nate wrote: > >> Ray Van Dolson wrote: > >> > >>> Out of curiosity, any idea what a full cabinet of one of these runs? > >> > >> Over $1M pretty easily, probably close/more than $2M. > > > > I think you

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-08 Thread nate
Les Mikesell wrote: > On 1/8/2010 11:41 AM, nate wrote: >> Ray Van Dolson wrote: >> >>> Out of curiosity, any idea what a full cabinet of one of these runs? >> >> Over $1M pretty easily, probably close/more than $2M. > > I think you are confusing it with something else. Somewhere I saw that > thes

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-08 Thread Matty
On Thu, Jan 7, 2010 at 11:25 AM, Boris Epstein wrote: > > > On Thu, Jan 7, 2010 at 11:09 AM, Matty wrote: >> >> On Thu, Jan 7, 2010 at 8:08 AM, Chan Chung Hang Christopher >> wrote: >> > John Doe wrote: >> >> From: Boris Epstein >> >>> This is not directly related to CentOS but still: we are tr

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-08 Thread Les Mikesell
On 1/8/2010 11:41 AM, nate wrote: > Ray Van Dolson wrote: > >> Out of curiosity, any idea what a full cabinet of one of these runs? > > Over $1M pretty easily, probably close/more than $2M. I think you are confusing it with something else. Somewhere I saw that these list around $400k for 80TB -

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-08 Thread nate
Ray Van Dolson wrote: > Out of curiosity, any idea what a full cabinet of one of these runs? Over $1M pretty easily, probably close/more than $2M. nate ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-08 Thread Ray Van Dolson
On Fri, Jan 08, 2010 at 11:06:10AM -0600, Les Mikesell wrote: > On 1/8/2010 10:09 AM, nate wrote: > > m.r...@5-cent.us wrote: > > > >> Dunno why you say that. Lessee, both google and maybe amazon run Linux; > >> meanwhile, AT&T, where I worked for a couple of years, Trustwave, a root > >> CA that I

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-08 Thread R-Elists
> Karanbir Singh wrote: snip > > Good question, they are after all ( the Sun 45xx's ) just > opteron box's with a mostly standard build. Finding a CentOS > compatible ( drivers pre-included, and not crap like cciss ) > would not be too hard. > > Who wants to offer up a machine to test on :)

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-08 Thread Les Mikesell
On 1/8/2010 10:09 AM, nate wrote: > m.r...@5-cent.us wrote: > >> Dunno why you say that. Lessee, both google and maybe amazon run Linux; >> meanwhile, AT&T, where I worked for a couple of years, Trustwave, a root >> CA that I worked for earlier this year, and here at the US NIH, we run >> Linux. >

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-08 Thread nate
m.r...@5-cent.us wrote: > Dunno why you say that. Lessee, both google and maybe amazon run Linux; > meanwhile, AT&T, where I worked for a couple of years, Trustwave, a root > CA that I worked for earlier this year, and here at the US NIH, we run > Linux. Since this is a storage thread.. back in 2

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-08 Thread Alpin Iolaire McKinnon
I suggest you get a second-hand Sun X4500 if you're feeling cheap, http://www.sun.com/servers/x64/x4500/specs.xml. 48x 500G will do you nicely with some MD RAID. Or you can go for the newer X4540 if you're feeling flush. Regards, Iolaire On 06/01/2010 22:35, Boris Epstein wrote: Hello everyo

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-08 Thread m . roth
> Warren Young wrote: >> On 1/6/2010 2:35 PM, Boris Epstein wrote: >>> we are trying to set >>> up some storage servers to run under Linux > Serious system administrators are not Linux fans I don't think. I tend Dunno why you say that. Lessee, both google and maybe amazon run Linux; meanwhile, AT

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-08 Thread James Bensley
A bit OT but you did you ever see http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how-to-build-cheap-cloud-storage/ -- Regards, James ;) Pablo Picasso - "Computers are useless. They can only give you answers." - http://www.brainyquote.com/quotes/authors/p/pablo_picasso.html _

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-08 Thread Chan Chung Hang Christopher
> " Maximum 3.5" hot-swap drives density 36x (24 front + 12 rear) HDD bays" > > http://www.supermicro.com/products/chassis/4U/847/SC847A-R1400.cfm > > Did anybody else think "WTF?" when you saw that picture? > > I have seen crazy stuff, but that one is pretty high-up on the list > > Doesn'

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-08 Thread Eero Volotinen
Quoting Rainer Duffner : > > " Maximum 3.5" hot-swap drives density 36x (24 front + 12 rear) HDD bays" > > http://www.supermicro.com/products/chassis/4U/847/SC847A-R1400.cfm > > Did anybody else think "WTF?" when you saw that picture? > > I have seen crazy stuff, but that one is pretty high-up on

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-08 Thread Rainer Duffner
Karanbir Singh schrieb: > On 01/08/2010 01:58 AM, Christopher Chan wrote: > >>> the thumpers make for decent backup or vtl type roles, not so much for >>> online high density storage. >>> >> I wonder how much that would change with a bbu NVRAM card for an >> external journal for ext4 and

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-08 Thread Karanbir Singh
On 01/08/2010 01:58 AM, Christopher Chan wrote: >> the thumpers make for decent backup or vtl type roles, not so much for >> online high density storage. > I wonder how much that would change with a bbu NVRAM card for an > external journal for ext4 and the disks on md. Unless one cannot add a > bbu

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-08 Thread Rainer Duffner
Christopher Chan schrieb: >> cause when I did - the x45xx's/zfs were between 18 to 20% slower on disk >> i/o alone compared with a supermicro box with dual areca 1220/xfs. >> >> the thumpers make for decent backup or vtl type roles, not so much for >> online high density storage. >> > > > Sp

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-08 Thread Tru Huynh
On Fri, Jan 08, 2010 at 12:49:30PM +0800, Christopher Chan wrote: > Warren Young wrote: > > On 1/7/2010 6:01 PM, Christopher Chan wrote: ... zfs on *solaris *bsd is getting off topic, if you need to fight, please take that somewhere else. Thanks, Tru -- Tru Huynh (mirrors, CentOS-3 i386/x86_64

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-07 Thread Christopher Chan
Warren Young wrote: > On 1/7/2010 6:01 PM, Christopher Chan wrote: >>> I'm not recommending OpenSolaris on purpose. >> Serious system administrators are not Linux fans I don't think. > > I think I must have been sent back in time, say to 1997 or so, because I > can't possibly be reading this in 2

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-07 Thread Warren Young
On 1/7/2010 6:01 PM, Christopher Chan wrote: > >> I'm not recommending OpenSolaris on purpose. > > Serious system administrators are not Linux fans I don't think. I think I must have been sent back in time, say to 1997 or so, because I can't possibly be reading this in 2010. I base this on the f

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-07 Thread Christopher Chan
> cause when I did - the x45xx's/zfs were between 18 to 20% slower on disk > i/o alone compared with a supermicro box with dual areca 1220/xfs. > > the thumpers make for decent backup or vtl type roles, not so much for > online high density storage. Speaking of thumpers and Supermicro, it loo

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-07 Thread Christopher Chan
Karanbir Singh wrote: > On 01/08/2010 12:53 AM, John R Pierce wrote: >> Christopher Chan wrote: >>> Yes, the Sun Fire X4540 uses software raid but not necessarily zfs...if >>> you install another operating system that is not Solaris or OpenSolaris, >>> it won't be zfs. >>> >> the thing to note on t

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-07 Thread Karanbir Singh
On 01/08/2010 12:53 AM, John R Pierce wrote: > Christopher Chan wrote: >> Yes, the Sun Fire X4540 uses software raid but not necessarily zfs...if >> you install another operating system that is not Solaris or OpenSolaris, >> it won't be zfs. >> > > the thing to note on the Thumper (X4540), each of

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-07 Thread Karanbir Singh
On 01/08/2010 01:01 AM, Christopher Chan wrote: >> That puts you right on the edge of workability with 32-bit hardware. >> ext3's limit on 32-bit is 8 TB, and you can push it to 16 TB by >> switching to XFS or JFS. Best to use 64-bit hardware if you can. > > Probably XFS if you want data guarantee

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-07 Thread Christopher Chan
Warren Young wrote: > On 1/6/2010 2:35 PM, Boris Epstein wrote: >> we are trying to set >> up some storage servers to run under Linux > > You should also consider FreeBSD 8.0, which has the newest version of > ZFS up and running stably on it. I use Linux for most server tasks, but > for big sto

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-07 Thread John R Pierce
Christopher Chan wrote: > Yes, the Sun Fire X4540 uses software raid but not necessarily zfs...if > you install another operating system that is not Solaris or OpenSolaris, > it won't be zfs. > the thing to note on the Thumper (X4540), each of those 48 SATA drives has its own channel to the

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-07 Thread Christopher Chan
John Doe wrote: > Yes, the Sun Fire Xs are costly... > Here, 35k euros for 48 x 1TB by example, or 22k for 48 x 500GB... > Our 12TB HP is around 6k. So 12k for almost the same as the 22k > But if you use 1TB disks on the Sun, you end up using half the Us (and save > some power) in your bay; which

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-07 Thread Warren Young
On 1/6/2010 2:35 PM, Boris Epstein wrote: > > we are trying to set > up some storage servers to run under Linux You should also consider FreeBSD 8.0, which has the newest version of ZFS up and running stably on it. I use Linux for most server tasks, but for big storage, Linux just doesn't have

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-07 Thread John Doe
Yes, the Sun Fire Xs are costly... Here, 35k euros for 48 x 1TB by example, or 22k for 48 x 500GB... Our 12TB HP is around 6k. So 12k for almost the same as the 22k But if you use 1TB disks on the Sun, you end up using half the Us (and save some power) in your bay; which might be nice if you are

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-07 Thread Boris Epstein
On Thu, Jan 7, 2010 at 11:09 AM, Matty wrote: > On Thu, Jan 7, 2010 at 8:08 AM, Chan Chung Hang Christopher > wrote: > > John Doe wrote: > >> From: Boris Epstein > >>> This is not directly related to CentOS but still: we are trying to set > up some storage servers to run under Linux - most like

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-07 Thread Matty
On Thu, Jan 7, 2010 at 8:08 AM, Chan Chung Hang Christopher wrote: > John Doe wrote: >> From: Boris Epstein >>> This is not directly related to CentOS but still: we are trying to set up >>> some storage servers to run under Linux - most likely CentOS. The storage >>> volume would be in the rang

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-07 Thread Benjamin Donnachie
2010/1/7 Karanbir Singh : > I've had 2 drobo's at work - and i can assure you that it is essentially > a wasted device. I agree with this. We had a Drobo on loan for a while, I found it sluggish and detested the way it over-reports its free space. Couldn't wait to hand it back. Ben

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-07 Thread Karanbir Singh
On 01/07/2010 03:28 AM, earl ramirez wrote: > You can have a look at this, I don't know what your budget is like > > http://www.drobo.com/Products/drobopro/index.php > > I have a drobo and it worked off the bat with a few linux distros > I've had 2 drobo's at work - and i can assure you that it is

Re: [CentOS] 8-15 TB storage: any recommendations?

2010-01-07 Thread Eero Volotinen
Quoting Chan Chung Hang Christopher : > John Doe wrote: >> From: Boris Epstein >>> This is not directly related to CentOS but still: we are trying to >>> set up some storage servers to run under Linux - most likely >>> CentOS. The storage volume would be in the range specified: 8-15 >>>

  1   2   >