Re: large ide raid system
James Manning wrote: > > [ Tuesday, January 11, 2000 ] Thomas Davis wrote: > > ---Sequential Output ---Sequential Input-- > > --Random-- > > -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- > > --Seeks--- > > MachineMB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU > > /sec %CPU > > pdsfdv10 1024 14076 85.1 18487 24.3 12089 35.8 20182 83.0 63064 69.8 > > 344.4 7.1 > > hmmm ok... > > Any chance I could talk you into running the tiobench.pl from > http://www.iki.fi/miku/tiotest/ (after "make" to build tiotest)? > > I'd love to see what it puts out vs. bonnie on such a system. > [root@pdsfdv10 tiotest-0.16]# ./tiobench.pl Found memory size of 255.94140625 MB Now running ./tiotest -t 1 -f 510 -s 4000 -b 4096 -d . -T -W Size is MB, BlkSz is Bytes, Read and Write are MB/sec, Seeks are Seeks/sec MachineDirectory Size(MB) BlkSz Threads Read Write Seeks --- --- - --- - -- --- --- . 510 4096 1 63.197 16.073 185.185 Now running ./tiotest -t 2 -f 255 -s 2000 -b 4096 -d . -T -W . 510 4096 2 62.347 15.366 304.183 Now running ./tiotest -t 4 -f 127 -s 1000 -b 4096 -d . -T -W . 510 4096 4 67.285 15.070 528.402 Now running ./tiotest -t 8 -f 63 -s 500 -b 4096 -d . -T -W . 510 4096 8 52.610 14.797 803.213 [root@pdsfdv10 tiotest-0.16]# ./tiobench.pl --threads 16 Found memory size of 255.94140625 MB Now running ./tiotest -t 16 -f 31 -s 250 -b 4096 -d . -T -W Size is MB, BlkSz is Bytes, Read and Write are MB/sec, Seeks are Seeks/sec MachineDirectory Size(MB) BlkSz Threads Read Write Seeks --- --- - --- - -- --- --- . 510 4096 16 33.514 14.806 1302.93 [root@pdsfdv10 tiotest-0.16]# ./tiobench.pl --threads 32 Found memory size of 255.94140625 MB Now running ./tiotest -t 32 -f 15 -s 125 -b 4096 -d . -T -W Size is MB, BlkSz is Bytes, Read and Write are MB/sec, Seeks are Seeks/sec MachineDirectory Size(MB) BlkSz Threads Read Write Seeks --- --- - --- - -- --- --- . 510 4096 32 27.491 13.445 1851.85 [root@pdsfdv10 tiotest-0.16]# ./tiobench.pl --threads 64 Found memory size of 255.94140625 MB Now running ./tiotest -t 64 -f 7 -s 62 -b 4096 -d . -T -W Size is MB, BlkSz is Bytes, Read and Write are MB/sec, Seeks are Seeks/sec MachineDirectory Size(MB) BlkSz Threads Read Write Seeks --- --- - --- - -- --- --- . 510 4096 64 42.667 13.211 2110.63 [root@pdsfdv10 tiotest-0.16]# ./tiobench.pl --threads 128 Found memory size of 255.94140625 MB Now running ./tiotest -t 128 -f 3 -s 31 -b 4096 -d . -T -W Size is MB, BlkSz is Bytes, Read and Write are MB/sec, Seeks are Seeks/sec MachineDirectory Size(MB) BlkSz Threads Read Write Seeks --- --- - --- - -- --- --- . 510 4096 128 44.548 13.627 2511.39 [root@pdsfdv10 tiotest-0.16]# ./tiobench.pl --threads 256 Found memory size of 255.94140625 MB Now running ./tiotest -t 256 -f 1 -s 15 -b 4096 -d . -T -W Size is MB, BlkSz is Bytes, Read and Write are MB/sec, Seeks are Seeks/sec MachineDirectory Size(MB) BlkSz Threads Read Write Seeks --- --- - --- - -- --- --- . 510 4096 256 36.941 12.580 4042.10 (I was playing around at the end..) -- +-- Thomas Davis| PDSF Project Leader [EMAIL PROTECTED] | (510) 486-4524 | "Only a petabyte of data this year?"
Re: Ribbon Cabling (was Re: large ide raid system)
$horse='dead'; &beat($horse); [ Wednesday, January 12, 2000 ] Bohumil Chalupa wrote: > ,,Termination`` means nothing else then a resistance at the end of the > cable (each pair) that is equivalent to the cable impedance. And the > impedance depends on the cable geometry (and material, of course). > So IMHO you can't divide the flat cable to the pairs (unless they're > twisted pairs) or even single wires without an impedance change > of the cable section involved. Impedance depends on the cable the signal is going through and the distance to its return path (which you agree to above). The distributed RLC model of transmission lines (longer than 3 inches, so I'm not trusting lumped :) is based on properties of the cable itself, though. Now, fast signals (GHz signals skimming only along the top of a microstrip, for instance) need a very-close signal return path (PCB traces that need to be closer to the power plane under/over them, as impedance rises with distance from return path) hence the need to keep the pairs together, but with the pairs kept together each cable is equi-distant to its return path both before and after the splitting, so impedance is not affected. If someone splits out the individual wires, you're absolutely right, but that's not what we're advocating :) Anyway, there's enough cables that come off the manufacturing line not meeting spec (and still get used) that even if messing with the ribbon had integrity issues, that doesn't the cable wouldn't still work :) James -- Miscellaneous Engineer --- IBM Netfinity Performance Development
Re: large ide raid system
Jan Edler wrote: > > It all depends on your minimum acceptable performance level. > I know my master/slave test setup couldn't keep up with fast ethernet > (10 MByte/s). I don't remember if it was >1 Mbyte/s or not. > Fastethernet is 12mb/sec, Ethernet is 1.2mb/sec. My 4way IDE based, 2 channels (ie, master/slave, master/slave) built using IBM 16gb Ultra33 drives in RAID0 are capable of about 25mb/sec across the raid. Adding a Promise 66 card, changing to all masters, got the numbers up into the 30's range (I don't have them at the moment.. hmm..) > I was also wondering about the reliability of using slaves. > Does anyone know about the likelihood of a single failed drive > bringing down the whole master/slave pair? Since I have tended to > stay away from slaves, for performance reasons, I don't know > how they influence reliability. Maybe it's ok. > When the slave fail, the master goes down. My experience has been, when _ANY_ IDE drive fails, it takes down the whole channel. Master or slave. The kernel just gives fits.. -- +-- Thomas Davis| PDSF Project Leader [EMAIL PROTECTED] | (510) 486-4524 | "Only a petabyte of data this year?"
Re: [FAQ-answer] Re: soft RAID5 + journalled FS + power failure =problems ?
Hi, On Wed, 12 Jan 2000 07:21:17 -0500 (EST), Ingo Molnar <[EMAIL PROTECTED]> said: > On Wed, 12 Jan 2000, Gadi Oxman wrote: >> As far as I know, we took care not to poke into the buffer cache to >> find clean buffers -- in raid5.c, the only code which does a find_buffer() >> is: > yep, this is still the case. OK, that's good to know. > Especially the reconstruction code is a rathole. Unfortunately > blocking reconstruction if b_count == 0 is not acceptable because > several filesystems (such as ext2fs) keep metadata caches around > (eg. the block group descriptors in the ext2fs case) which have > b_count == 1 for a longer time. That's not a problem: we don't need reconstruction to interact with the buffer cache at all. Ideally, what I'd like to see the reconstruction code do is to: * lock a stripe * read a new copy of that stripe locally * recalc parity and write back whatever disks are necessary for the stripe * unlock the stripe so that the data never goes through the buffer cache at all, but that the stripe is locked with respect to other IOs going on below the level of ll_rw_block (remember there may be IOs coming in to ll_rw_block which are not from the buffer cache, eg. swap or journal IOs). > We are '100% journal-safe' if power fails during resync. Except for the fact that resync isn't remotely journal-safe in the first place, yes. :-) --Stephen
Re: [FAQ-answer] Re: soft RAID5 + journalled FS + power failure = problems ?
Hi, On Tue, 11 Jan 2000 16:41:55 -0600, "Mark Ferrell" <[EMAIL PROTECTED]> said: > Perhaps I am confused. How is it that a power outage while attached > to the UPS becomes "unpredictable"? One of the most common ways to get an outage while on a UPS is somebody tripping over, or otherwise removing, the cable between the UPS and the computer. How exactly is that predictable? Just because you reduce the risk of unexpected power outage doesn't mean we can ignore the possibility. --Stephen
DPT PM3754U2 RaidV Millennium and Slackware.
Help! I am trying to get the DPT PM3754U2 Raid controller to work in an X86 dual PIII500 system. The only Linux I have seen support for is Red Hat. Has anyone out there possibly hacked the source from DPT to create a boot disk that will work with Slackware? Any help is much appreciated. Jon Preston BBS&G Systems Integration [EMAIL PROTECTED]
Re: RAID0 problem
[ Wednesday, January 12, 2000 ] Andre Cruz wrote: > mkraid: aborted, see the syslog and /proc/mdstat for potential clues. Which kernel? which raid patch? which raidtools? James -- Miscellaneous Engineer --- IBM Netfinity Performance Development
Re: raid-2.2.14-B1.14 doesn't patch 2.2.14 properly
[ Wednesday, January 12, 2000 ] Scott Thomson wrote: > Am I missing something here? > The source has just been freshly untarred from linux-2.2.14.tgz > This is just the the first prompt. It goes on and on... > > patching file `linux/init/main.c' > Hunk #1 FAILED at 19. > Hunk #2 FAILED at 488. > Hunk #3 FAILED at 928. > Hunk #4 FAILED at 1426. > 4 out of 4 hunks FAILED -- saving rejects to linux/init/main.c.rej > The next patch would create the file > `linux/include/linux/raid/linear.h', > which already exists! Assume -R? [n] Hmm works fine from here... maybe rm -rf linux or try checking your .tar.gz file: # md5sum linux-2.2.14.tar.gz 7dde751afbe82cfeaecb9dce3da8e992 linux-2.2.14.tar.gz [root@jmm /tmp]# tar xzf linux-2.2.14.tar.gz [root@jmm /tmp]# cd linux [root@jmm linux]# patch -p1 < ../raid-2.2.14-B1 patching file `init/main.c' Hunk #2 succeeded at 497 (offset 9 lines). Hunk #3 succeeded at 931 (offset 3 lines). Hunk #4 succeeded at 1435 (offset 9 lines). patching file `include/linux/raid/linear.h' patching file `include/linux/raid/hsm_p.h' patching file `include/linux/raid/md.h' patching file `include/linux/raid/md_compatible.h' [snip] James -- Miscellaneous Engineer --- IBM Netfinity Performance Development
Re: [FAQ-answer] Re: soft RAID5 + journalled FS + power failure =problems ?
"Stephen C. Tweedie" wrote: > Ideally, what I'd like to see the reconstruction code do is to: > > * lock a stripe > * read a new copy of that stripe locally > * recalc parity and write back whatever disks are necessary for the stripe > * unlock the stripe > > so that the data never goes through the buffer cache at all, but that > the stripe is locked with respect to other IOs going on below the level > of ll_rw_block (remember there may be IOs coming in to ll_rw_block which > are not from the buffer cache, eg. swap or journal IOs). > > > We are '100% journal-safe' if power fails during resync. > > Except for the fact that resync isn't remotely journal-safe in the first > place, yes. :-) > > --Stephen Sorry for my ignorance I got a little confused by this post: Ingo said we are 100% journal-safe, you said the contrary, can you or Ingo please explain us in which situation (power-loss) running linux-raid+ journaled FS we risk a corrupted filesystem ? I am interested what happens if the power goes down while you write heavily to a ext3/reiserfs (journaled FS) on soft-raid5 array. After the reboot if all disk remain intact physically, will we only lose the data that was being written, or is there a possibility to end up in a corrupted filesystem which could more damages in future ? (or do we need to wait for the raid code in 2.3 ?) sorry for re-asking that question, but I am still confused. regards, Benno.
Re: raid-2.2.14-B1.14 doesn't patch 2.2.14 properly
Scott Thomson wrote: > > Am I missing something here? > The source has just been freshly untarred from linux-2.2.14.tgz > This is just the the first prompt. It goes on and on... > > patching file `linux/init/main.c' > Hunk #1 FAILED at 19. > Hunk #2 FAILED at 488. > Hunk #3 FAILED at 928. > Hunk #4 FAILED at 1426. > 4 out of 4 hunks FAILED -- saving rejects to linux/init/main.c.rej > The next patch would create the file `linux/include/linux/raid/linear.h', > which already exists! Assume -R? [n] Two things: The patch that I initially downloaded from Mingo's site had line breaks between what should have been wrapped lines. This caused the patch to fail much like what you are describing. I am not sure where or how the bogus line breaks were introduced. I downloaded the patch again last night and it did apply cleanly to a pristine 2.2.14 source tree. The patch I downloaded was "raid-2.2.14-B1". -- Rob Riggs Technical Staff Tummy.com, Ltd. http://www.tummy.com/
RE: Ribbon Cabling (was Re: large ide raid system)
Title: RE: Ribbon Cabling (was Re: large ide raid system) You may be thinking of differential SCSI which uses a balanced (and twisted) pair for each data and signal line. In the old days, there was only one flavor of differential, and it was popular at least on Hewlett-Packard 800 series systems (which uses round SCSI cables). Now, in addition to the old differential, there is something called "low voltage differential" (LVD) also known in marketing hype as "Ultra2". LVD cables look like a "ribbon" cable but have the signal pairs twisted. The last ones I bought from Adaptec were high quality, but expensive! Best I remember, they were almost $100 for a 4 device cable. But then again, they have active terminators built on the end of the cable (LVD drives don't have built-in terminators). -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Tuesday, January 11, 2000 10:11 PM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: Re: Ribbon Cabling (was Re: large ide raid system) > From [EMAIL PROTECTED] Tue Jan 11 21:44:29 2000 > > On Tue, 11 Jan 2000, Gregory Leblanc wrote: > > If you cut the cable > > lengthwise (no, don't cut the wires) between wires (don't break the > > insulation on the wires themselves, just the connecting plastic) you can > > get your cables to be 1/4 the normal width (up until you get to the > > connector). > > I don't know about IDE, but I'm pretty sure that's a big no-no for SCSI > cables. The alternating conductors in the ribbon cable are sig, gnd, sig, > gnd, sig, etc. And it's electrically important (for proper impedance and > noise and cross-talk rejection) that they stay that way. > > I think the same is probably true for the schmancy UDMA66 cables too... Back in the day 8-) high end SCSI ribbon cables consisted of twisted pairs between the connectors so it was really easy to deform the cable to fit through tight spots. Now, all I seem to find is the cheap ribbon cable that's excreted from nameless companies in developing countries where their ideas of quality control differ vastly from mine. 8-) Either I'm really unlucky or the quality of ribbon cabling in general is in decline...sigh. And I agree with the idea that slicing up the ribbon cable is probably not going to work. Cheers, Chris -- Christopher Mauritz [EMAIL PROTECTED]
Re: [FAQ-answer] Re: soft RAID5 + journalled FS + power failure =problems ?
On Wed, 12 Jan 2000, Gadi Oxman wrote: > As far as I know, we took care not to poke into the buffer cache to > find clean buffers -- in raid5.c, the only code which does a find_buffer() > is: yep, this is still the case. (Sorry Stephen, my bad.) We will have these problems once we try to eliminate the current copying overhead. Nevertheless there are bad (illegal) interactions between the RAID code and the buffer cache, i'm cleaning up this for 2.3 right now. Especially the reconstruction code is a rathole. Unfortunately blocking reconstruction if b_count == 0 is not acceptable because several filesystems (such as ext2fs) keep metadata caches around (eg. the block group descriptors in the ext2fs case) which have b_count == 1 for a longer time. If both power and a disk fails at once then we still might get local corruption for partially written RAID5 stripes. If either power or a disk fails, then the Linux RAID5 code is safe wrt. journalling, because it behaves like an ordinary disk. We are '100% journal-safe' if power fails during resync. We are also 100% journal-safe if power fails during reconstruction of failed disk or in degraded mode. the 2.3 buffer-cache enhancements i wrote ensure that 'cache snooping' and adding to the buffer-cache can be done safely by 'external' cache managers. I also added means to do atomic IO operations which in fact are several underlying IO operations - without the need of allocating a separate bh. The RAID code uses these facilities now. Ingo
RAID0 problem
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 [root@lx12pc164 /]# mkraid --really-force /dev/md0 DESTROYING the contents of /dev/md0 in 5 seconds, Ctrl-C if unsure! handling MD device /dev/md0 analyzing super-block disk 0: /dev/hdc1, 1306336kB, raid superblock at 1306240kB disk 1: /dev/hdd2, 1183392kB, raid superblock at 1183296kB mkraid: aborted, see the syslog and /proc/mdstat for potential clues. [root@lx12pc164 /]# tail /var/log/messages Jan 12 10:18:03 localhost sshd2[373]: Daemon is running. Jan 12 10:18:03 localhost keytable: Loading keymap: Jan 12 10:18:04 localhost keytable: Loading /usr/lib/kbd/keymaps/i386/qwerty/pt-latin1.kmap.gz Jan 12 10:18:04 localhost keytable: Loading system font: Jan 12 10:18:04 localhost rc: Starting keytable succeeded Jan 12 10:18:05 localhost gpm: gpm startup succeeded Jan 12 10:18:05 localhost linuxconf: Linuxconf final setup Jan 12 10:18:08 localhost rc: Starting linuxconf succeeded Jan 12 10:18:12 localhost PAM_pwdb[440]: (login) session opened for user root by LOGIN(uid=0) Jan 12 10:27:13 localhost PAM_pwdb[441]: (login) session opened for user cruza by LOGIN(uid=0) [root@lx12pc164 /]# cat /proc/mdstat Personalities : [2 raid0] read_ahead not set md0 : inactive md1 : inactive md2 : inactive md3 : inactive [root@lx12pc164 /]# Can anyone tell me what's the problem? thanks - --- Andre Cruz [EMAIL PROTECTED] -BEGIN PGP SIGNATURE- Version: GnuPG v1.0.0 (SunOS) Comment: For info see http://www.gnupg.org iEYEARECAAYFAjh8WusACgkQW9f2ds85q7ldNwCdEKNUWdvzS7MQYplDCa8SKeOX HZMAniuC0mFFiLvGQwjnkv1RnJi8BRNQ =q3kO -END PGP SIGNATURE- # # RAID0 setup: # raiddev /dev/md0 raid-level0 nr-raid-disks 2 persistent-superblock 1 chunk-size4 device/dev/hdc1 raid-disk 0 device/dev/hdd2 raid-disk 1
Re: Mylex Ultra3 Cards
In article <[EMAIL PROTECTED]>, Jon Doyle wrote:
>Has anyone used the new ExtremeRAID or AccelRAID 352 on Linux
>with the DAC960 drivers? I run SuSE 6.3 on the Ultra2 and UW
>DAC, AccelRAD 250, and ExtremeRAID 1164, but thought I might
>ask before diving into the new gear.
ExtremeRAID seems very good, far better than the equivalent AMI
product. The ExtremeRAID you can boot off so you can use it for
your root FS. The only things that its missing is native linux tools
for reconfiguring it without having to take the machine down and go
into the BIOS. The other thing is some sort of monitoring that alerts
you when a failure of some type occurs. My solution to this was to
write the following script which emails you the status files if
a fault is detected. Just change the email address and start it
up from cron periodically
#!/usr/bin/perl -w
my $statfile = "/proc/rd/status";
my $curfile = "/proc/rd/c0/current_status";
my $initfile = "/proc/rd/c0/initial_status";
open(STATUS, "<$statfile");
$_ = ;
if ($_ ne "OK\n") {
my $report = "Status: ";
$report .= $_;
$report .= "Current status:\n";
open (CUR, "<$curfile");
while ($_ = ) {
$report .= $_;
}
close(CUR);
open (INIT, "<$initfile");
$report .= "Initial status:\n";
while ($_ = ) {
$report .= $_;
}
close(INIT);
open (MAIL, "|/bin/mail someone\@somewhere.com");
print MAIL "$report";
}
close(STATUS);
--
Chris Good - Dialog Corp. The Westbrook Centre, Milton Rd, Cambridge UK
Phone: 01223 715006 Mobile: 07801 788997
http://www.dialog.com
looking for pre-configured linux-compatible RAID servers
Hello, Buying a Mylex card and installing it on our Linux server seems easy enough, but just for information are there any complete systems resellers (like Dell for ex.) that integrate Linux-compatible RAID controllers in their offerings? And a terribly newbie-revealing question: if we install an AcceleRAID-250 do we need any other SCSI controller at all? It seems not, but I just wanted to make sure... Thanks in advance. -- Louis-David Mitterrand - [EMAIL PROTECTED] - http://www.aparima.com
Re: [FAQ-answer] Re: soft RAID5 + journalled FS + power fai
On 11 Jan 00 at 22:24, Stephen C. Tweedie wrote: > The race I'm concerned about could occur when the raid driver wants to > compute parity for a stripe and finds some of the blocks are present, > and clean, in the buffer cache. Raid assumes that those buffers > represent what is on disk, naturally enough. So, it uses them to > calculate parity without rereading all of the disk blocks in the stripe. > The trouble is that the standard practice in the kernel, when modifying > a buffer, is to make the change and _then_ mark the buffer dirty. If > you hit that window, then the raid driver will find a buffer which > doesn't match what is on disk, and will compute parity from that buffer > rather than from the on-disk contents. Hi Stephen, I did not follow this thread (on -fsdevel) too close (and I never looked into RAID code, so I should shut up), but... can you confirm that after buffer with data is finally marked dirty, parity is recomputed anyway? So that window is really small and same problems occurs every moment when you wrote data, but did not wrote parity yet? Thanks, Petr Vandrovec [EMAIL PROTECTED]
Re: Mylex Ultra3 Cards
I have the eXtremeRAID 1164 card and have been using it for over 6 months now under linux with no problems at all. It's performance is very very good too. -- Brian D. Haymore University of Utah Center for High Performance Computing 155 South 1452 East RM 405 Salt Lake City, Ut 84112-0190 Email: [EMAIL PROTECTED] - Phone: (801) 585-1755 - Fax: (801) 585-5366 On Tue, 11 Jan 2000, Jon Doyle wrote: > Has anyone used the new ExtremeRAID or AccelRAID 352 on Linux > with the DAC960 drivers? I run SuSE 6.3 on the Ultra2 and UW > DAC, AccelRAD 250, and ExtremeRAID 1164, but thought I might > ask before diving into the new gear. > > > Regards, > > Jon > > Jon R. Doyle > Systems Administrator > Document Solutions, Inc. > 1611 Telegraph Avenue Ste. 1010 > Oakland, Ca. 94612 > 510-986-0250 >
Re: sw raid 0 - performance problems
On Wed, 12 Jan 2000, Peter Palfrader aka Weasel wrote: > Hi Gurus! > > I've set up raid 0 on my box half a year ago and back then everything > worked fine. Recently I made a benchmark again and was pretty > disappointed by the results I got. > > I have a P II 400, 128 Megs of RAM, and 2 IBM DRVS 09V (10,000 RPM; > 6.3 ms; 9.1 Gig) on a AHA 2940 U2W. Those harddisks are the only > devices on this SCSI controller. > > > On a single hard disk on a partition at the very end of the hd (ext2) > bonnie reports the following: > ---Sequential Output ---Sequential Input-- --Random-- > -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks--- > MachineMB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU > 500 6068 85.0 6784 9.0 4196 11.4 6418 79.9 17241 15.0 104.5 1.0 > > > Now a raid0 system should IMHO be while not twice as fast at least > considerable faster (at least with sequention input) but bonnie > reports: > ---Sequential Output ---Sequential Input-- --Random-- > -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks--- > MachineMB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU > 500 6203 87.1 12428 16.3 7231 19.9 6443 80.4 25411 24.7 122.3 2.3 > > tested on a 2 partition raid 0 with both partitions being located at > the very start of the hard disks. > > When I set this up 5 or 6 months ago it was considerable faster > (approx. 35 Megs/sec). A friend of mine has a similar setup and also > gets about 35 Megs/sec. > If you haven't changed your kernel or disk setup, the only thing which comes to my mind is that before your disk was empty and the bonnie test file fit in nicely in order. Now if your disk is full, there isn't so much sequential blocks free, so it will be allocated a little here and there resulting a little poorer performance. But if you just made the array then i am puzzled. If you really want to have a fast system (more than just for bonnie) pay close attention to seektimes. > > I'm running a plain 2.2.14 but the results are no different than with > a 2.2.10 or 2.2.12. > > > Do you have any pointers what might be wrong/what I could try to > improve speed? > If i understood correctly you use plain 2.2.14 without newest raidpatches ? -- Mika <[EMAIL PROTECTED]>
Re: [FAQ-answer] Re: soft RAID5 + journalled FS + power failure = problems ?
James Manning wrote: > [ Tuesday, January 11, 2000 ] Benno Senoner wrote: > > The problem is that power outages are unpredictable even in presence > > of UPSes therefore it is important to have some protection against > > power losses. > > I gotta ask dying power supply? cord getting ripped out? > Most ppl run serial lines (of course :) and with powerd they > get nice shutdowns :) > > Just wanna make sure I'm understanding you... > > James > -- > Miscellaneous Engineer --- IBM Netfinity Performance Development yep, obviously the UPS has a serial line to shut down the machine nicely before a failure, but it happened to me that the serial cable was disconnected and the power outage lasted SEVERAL hours during a weekend , where no one was in the machine room (of an ISP). you know murphy's law ... :-) But I am mainly interested in the power-failure-protection in the case where you want to setup a workstation with a reliable disk array (soft raid5), and do not have always an UPS handy, you will loose the file that was being written, but the important thing is that the disk array remains in a safe state , just like a single disk + journaled FS. Sthephen Tweedie said that this is possible (by fixing the remaining races in the RAID code), if these problems will be fixed sometime, then our fears of a corrupted soft-RAID array in the case of a power-failure on a machine without UPS will completely go away. cheers, Benno.
How to set chunk-size with RH6.1
Hello How does one set the chunk-size with RH6.1, when installing? On my box it always sets it to 64 for my raid 5, but I want it to be 32. Is there any way to change this? I assume, that it is not possible to change this on an active raid system? Holger
Re: Ribbon Cabling (was Re: large ide raid system)
On Tue, 11 Jan 2000, James Manning wrote: > > > If you cut the cable > > > lengthwise (no, don't cut the wires) between wires (etc.) > > > > I don't know about IDE, but I'm pretty sure that's a big no-no for SCSI > > cables. The alternating conductors in the ribbon cable are sig, gnd, sig, > > gnd, sig, etc. And it's electrically important (for proper impedance and > > noise and cross-talk rejection) that they stay that way. > > > > I think the same is probably true for the schmancy UDMA66 cables too... > > So just check with a cable spec and make sure you're not separating a > data signal from its ground return path. Throw some mag rings around > the thing if you want, but since we're (hopefully) terminated properly > (no reflection) the crosstalk issues aren't huge... they suffer more > through the LC matrix of connector adaptors than this split would cause :) ,,Termination`` means nothing else then a resistance at the end of the cable (each pair) that is equivalent to the cable impedance. And the impedance depends on the cable geometry (and material, of course). So IMHO you can't divide the flat cable to the pairs (unless they're twisted pairs) or even single wires without an impedance change of the cable section involved. BoChal.
Re: sw raid 0 - performance problems
[ Tuesday, January 11, 2000 ] Peter Palfrader aka Weasel wrote: > I'm running a plain 2.2.14 but the results are no different than with > a 2.2.10 or 2.2.12. > > Do you have any pointers what might be wrong/what I could try to > improve speed? Hmmm... try using the "new" RAID (0.90) by patching your kernel with http://people.redhat.com/mingo/raid-2.2.14-B1 then using the newer raidtools at http://www.us.kernel.org/pub/linux/daemons/raid/alpha/raidtools-19990824-0.90.tar.gz you can do an mkraid --upgrade I'm not sure the perf will be much diff (this is just raid0, right?), but it'll be interesting to find out. James -- Miscellaneous Engineer --- IBM Netfinity Performance Development
Re: Ribbon Cabling (was Re: large ide raid system)
-BEGIN PGP SIGNED MESSAGE- On 11-Jan-2000 James Manning wrote: > [ Tuesday, January 11, 2000 ] Andy Poling wrote: >> On Tue, 11 Jan 2000, Gregory Leblanc wrote: >> > If you cut the cable >> > lengthwise (no, don't cut the wires) between wires (don't break the >> > insulation on the wires themselves, just the connecting plastic) you can >> > get your cables to be 1/4 the normal width (up until you get to the >> > connector). >> >> I don't know about IDE, but I'm pretty sure that's a big no-no for SCSI >> cables. The alternating conductors in the ribbon cable are sig, gnd, sig, >> gnd, sig, etc. And it's electrically important (for proper impedance and >> noise and cross-talk rejection) that they stay that way. >> >> I think the same is probably true for the schmancy UDMA66 cables too... > > So just check with a cable spec and make sure you're not separating a > data signal from its ground return path. Throw some mag rings around > the thing if you want, but since we're (hopefully) terminated properly > (no reflection) the crosstalk issues aren't huge... they suffer more > through the LC matrix of connector adaptors than this split would cause :) > > James > -- > Miscellaneous Engineer --- IBM Netfinity Performance Development Have a look at old 3M cables (used in most old suns and all old decstations). They have all the wires separated. And they work at least up to SCSI2. I also thought that the sig/gnd/sig/gnd was mandatory but these cables prove that there is another way to do it at least in some cases. My $0.02 - -- Anton R. Ivanov IP Engineer Level3 Communications RIPE: ARI2-RIPE E-Mail: Anton Ivanov <[EMAIL PROTECTED]> @*** Segal's Law *** A man with one watch knows what time it is; a man with two watches is never sure. - -- -BEGIN PGP SIGNATURE- Version: GnuPG v1.0.0 (GNU/Linux) Comment: For info see http://www.gnupg.org iQEVAwUBOHxGyylWAw/bM84zAQE5Swf+PUSlcf0gX+l1gUZTn/fsSN1Q+cO+kA6M Z5v9X/83mD0KOV8IAo5YRY9+E7BAIBaD5+rXgyFSWYdeIvewI0C9mTjSlliwv1ZN 1goBHvL5tqsIz21v/cbx/veW+zoSssHrj/ufm9GI2dXAIzdIA2YQ3BzZ60w6YLdH Pben/18W/KXNKuEqyEkBpRKJyXiLx6NBt2iM9qlMCfJHAd8KWv2ruqDv9v55aJOX e1HCKNxFHHuO951JjV4zzb+rlhD6lqGsw2EtN77228qGs1uKUkktAviTmRduHtzJ kTAr9YOu1T3/apUIFOjmZHtrjWqZWjVZ8lqT/iKF2HvcHsFCwqJJPg== =84Gh -END PGP SIGNATURE-
raid-2.2.14-B1.14 doesn't patch 2.2.14 properly
Am I missing something here?
The source has just been freshly untarred from linux-2.2.14.tgz
This is just the the first prompt. It goes on and on...
patching file `linux/init/main.c'
Hunk #1 FAILED at 19.
Hunk #2 FAILED at 488.
Hunk #3 FAILED at 928.
Hunk #4 FAILED at 1426.
4 out of 4 hunks FAILED -- saving rejects to linux/init/main.c.rej
The next patch would create the file `linux/include/linux/raid/linear.h',
which already exists! Assume -R? [n]
main.c.rej is as follows
***
*** 19,24
#include
#include
#include
#include
#include
#include
--- 19,25
#include
#include
#include
+ #include
#include
#include
#include
***
*** 487,493
#ifdef CONFIG_BLK_DEV_FD
{ "fd", 0x0200 },
#endif
- #ifdef CONFIG_MD_BOOT
{ "md", 0x0900 },
#endif
#ifdef CONFIG_BLK_DEV_XD
--- 488,494
#ifdef CONFIG_BLK_DEV_FD
{ "fd", 0x0200 },
#endif
+ #if CONFIG_MD_BOOT || CONFIG_AUTODETECT_RAID
{ "md", 0x0900 },
#endif
#ifdef CONFIG_BLK_DEV_XD
***
*** 927,932
#ifdef CONFIG_MD_BOOT
{ "md=", md_setup},
#endif
#ifdef CONFIG_ADBMOUSE
{ "adb_buttons=", adb_mouse_setup },
#endif
--- 928,936
#ifdef CONFIG_MD_BOOT
{ "md=", md_setup},
#endif
+ #if CONFIG_BLK_DEV_MD
+ { "raid=", raid_setup},
+ #endif
#ifdef CONFIG_ADBMOUSE
{ "adb_buttons=", adb_mouse_setup },
#endif
***
*** 1422,1427
while (pid != wait(&i));
if (MAJOR(real_root_dev) != RAMDISK_MAJOR
|| MINOR(real_root_dev) != 0) {
error = change_root(real_root_dev,"/initrd");
if (error)
printk(KERN_ERR "Change root to /initrd: "
--- 1426,1434
while (pid != wait(&i));
if (MAJOR(real_root_dev) != RAMDISK_MAJOR
|| MINOR(real_root_dev) != 0) {
+ #ifdef CONFIG_BLK_DEV_MD
+ autodetect_raid();
+ #endif
error = change_root(real_root_dev,"/initrd");
if (error)
printk(KERN_ERR "Change root to /initrd: "
Scott Thomson
sw raid 0 - performance problems
Hi Gurus! I've set up raid 0 on my box half a year ago and back then everything worked fine. Recently I made a benchmark again and was pretty disappointed by the results I got. I have a P II 400, 128 Megs of RAM, and 2 IBM DRVS 09V (10,000 RPM; 6.3 ms; 9.1 Gig) on a AHA 2940 U2W. Those harddisks are the only devices on this SCSI controller. On a single hard disk on a partition at the very end of the hd (ext2) bonnie reports the following: ---Sequential Output ---Sequential Input-- --Random-- -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks--- MachineMB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU 500 6068 85.0 6784 9.0 4196 11.4 6418 79.9 17241 15.0 104.5 1.0 Now a raid0 system should IMHO be while not twice as fast at least considerable faster (at least with sequention input) but bonnie reports: ---Sequential Output ---Sequential Input-- --Random-- -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks--- MachineMB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU 500 6203 87.1 12428 16.3 7231 19.9 6443 80.4 25411 24.7 122.3 2.3 tested on a 2 partition raid 0 with both partitions being located at the very start of the hard disks. When I set this up 5 or 6 months ago it was considerable faster (approx. 35 Megs/sec). A friend of mine has a similar setup and also gets about 35 Megs/sec. The chunk size is 8k but I've have also tried a size of 256k. It does not make any real difference. I've though of an interrupt problem but this controller does have an IRQ of its own. I've even tested this without my 2nd controller and without the nic with no better results. I'm running a plain 2.2.14 but the results are no different than with a 2.2.10 or 2.2.12. Do you have any pointers what might be wrong/what I could try to improve speed? If I gave too little info please say so together with what is missing :) Thanks in advance Peter Palfrader -- Weaselhttp://www.cosy.sbg.ac.at/~ppalfrad/ PGP/GPG encrypted messages prefered. See my site or finger -l ppalfrad -- Yes means No and No means Yes. Delete all files [Y]? PGP signature
Re: [FAQ-answer] Re: soft RAID5 + journalled FS + power failure = problems ?
Perhaps I am confused. How is it that a power outage while attached to the UPS becomes "unpredictable"? We run a Dell PowerEdge 2300/400 using Linux software raid and the system monitors it's own UPS. When power failure occures the system will bring itself down to a minimal state (runlevel 1) after the batteries are below 50% .. and once below 15% it will shutdown which turns off the UPS. When power comes back on the UPS fires up and the system resumes as normal. Addmitedly this wont prevent issues like god reaching out and slapping my system via lightning or something, nor will it resolve issues where someone decides to grab the power cable and swing around on it severing the connection from the UPS to the system .. but for the most part it has thus far prooven to be a fairly decent configuration. Benno Senoner wrote: > > "Stephen C. Tweedie" wrote: > > (...) > > > > > 3) The soft-raid backround rebuild code reads and writes through the > >buffer cache with no synchronisation at all with other fs activity. > >After a crash, this background rebuild code will kill the > >write-ordering attempts of any journalling filesystem. > > > >This affects both ext3 and reiserfs, under both RAID-1 and RAID-5. > > > > Interaction 3) needs a bit more work from the raid core to fix, but it's > > still not that hard to do. > > > > So, can any of these problems affect other, non-journaled filesystems > > too? Yes, 1) can: throughout the kernel there are places where buffers > > are modified before the dirty bits are set. In such places we will > > always mark the buffers dirty soon, so the window in which an incorrect > > parity can be calculated is _very_ narrow (almost non-existant on > > non-SMP machines), and the window in which it will persist on disk is > > also very small. > > > > This is not a problem. It is just another example of a race window > > which exists already with _all_ non-battery-backed RAID-5 systems (both > > software and hardware): even with perfect parity calculations, it is > > simply impossible to guarantee that an entire stipe update on RAID-5 > > completes in a single, atomic operation. If you write a single data > > block and its parity block to the RAID array, then on an unexpected > > reboot you will always have some risk that the parity will have been > > written, but not the data. On a reboot, if you lose a disk then you can > > reconstruct it incorrectly due to the bogus parity. > > > > THIS IS EXPECTED. RAID-5 isn't proof against multiple failures, and the > > only way you can get bitten by this failure mode is to have a system > > failure and a disk failure at the same time. > > > > > > > --Stephen > > thank you very much for these clear explanations, > > Last doubt: :-) > Assume all RAID code - FS interaction problems get fixed, > since a linux soft-RAID5 box has no battery backup, > does this mean that we will loose data > ONLY if there is a power failure AND successive disk failure ? > If we loose the power and then after reboot all disks remain intact > can the RAID layer reconstruct all information in a safe way ? > > The problem is that power outages are unpredictable even in presence > of UPSes therefore it is important to have some protection against > power losses. > > regards, > Benno.
Re: Ribbon Cabling (was Re: large ide raid system)
> From [EMAIL PROTECTED] Tue Jan 11 21:44:29 2000 > > On Tue, 11 Jan 2000, Gregory Leblanc wrote: > > If you cut the cable > > lengthwise (no, don't cut the wires) between wires (don't break the > > insulation on the wires themselves, just the connecting plastic) you can > > get your cables to be 1/4 the normal width (up until you get to the > > connector). > > I don't know about IDE, but I'm pretty sure that's a big no-no for SCSI > cables. The alternating conductors in the ribbon cable are sig, gnd, sig, > gnd, sig, etc. And it's electrically important (for proper impedance and > noise and cross-talk rejection) that they stay that way. > > I think the same is probably true for the schmancy UDMA66 cables too... Back in the day 8-) high end SCSI ribbon cables consisted of twisted pairs between the connectors so it was really easy to deform the cable to fit through tight spots. Now, all I seem to find is the cheap ribbon cable that's excreted from nameless companies in developing countries where their ideas of quality control differ vastly from mine. 8-) Either I'm really unlucky or the quality of ribbon cabling in general is in decline...sigh. And I agree with the idea that slicing up the ribbon cable is probably not going to work. Cheers, Chris -- Christopher Mauritz [EMAIL PROTECTED]
Re: Proper settings for fstab
[ Tuesday, January 11, 2000 ] Gregory Leblanc wrote: > I've managed to create a RAID stripe set (RAID 0) out of a pair of > SCSI2-W (20MB/Sec) drives, and it looks happy. I'd like to mount some > part of my filesystem to this new device, but when I add it to fstab in > an out-of-the way location with 1 2 following that entry is fstab, it > always has errors on boot. They are usually something about attempting > to read thus-and-such block caused a short read. fsck'ing that drive by > hand generally doesn't find any errors, although every third or fourth > time something will turn up (same error, respond ignore, then fix a > couple of minor errors). Any ideas on how to track this down? Thanks, If the partitions involved are type "fd" and the autostart flag is turned on in the kernel config, they should be available by the time sysinit is running through and gets to fsck'ing Do you see "autorun" messages on boot? James -- Miscellaneous Engineer --- IBM Netfinity Performance Development
Re: [FAQ-answer] Re: soft RAID5 + journalled FS + power failure =problems ?
- Original Message - From: "Benno Senoner" <[EMAIL PROTECTED]> To: "Stephen C. Tweedie" <[EMAIL PROTECTED]> Cc: "Linux RAID" <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>; "Ingo Molnar" <[EMAIL PROTECTED]> Sent: Tuesday, January 11, 2000 1:17 PM Subject: Re: [FAQ-answer] Re: soft RAID5 + journalled FS + power failure =problems ? -- much snippage here > > The problem is that power outages are unpredictable even in presence > of UPSes therefore it is important to have some protection against > power losses. > > regards, > Benno. > > I run an MGE UPS on my RH6.1 box running RAID 1, they have software for Linux that communicates with the UPS and performs an orderly system shutdown if the box goes on battery and stays on battery for a given (user selectable) length of time. I have tested and verified that this actually works, it's a Good Thing(tm). I did have to cut one pin on the standard RS-232 cable that came the UPS for use on the Linux box, and download the software and install (scripted, easy...) bwilling
Re: large ide raid system
> Getting back to the discussion of Hardware vs. Software raid... > Can someone say *definitively* *where* the raid-5 code is being run on a > *current* Raidzone product? Originally, it was an "md" process running > on the system cpu. Currently I'm not so sure. The SmartCan *does* have > its own BIOS, so there is *some* intelligence there, but what exactly is > the division of responsibility here... >From a recent email exchange with [EMAIL PROTECTED] of consensys, makers of raidzone: BG> Does the raidzone product for linux use hardware or software raid? RZ> It is in the firmware of the unit. RZ> RZ> Everything is our own raid. eg. it is not RAID tools in Linux. BG> [please clarify] RZ> Our raid is firmware. This means that its both hardware and software. RZ> RZ> Most people are interested in a RAID5 configuration. The parity is RZ> calculated on the CPU of the mother board. Our raid is as fast as anyone RZ> else's raid hardware or software. There is a great misnomer regarding RZ> raid today. Basically in the old days of 100 MHz CPU's there was a RZ> performance issuer with calculating the parity on the CPU. Today that is RZ> not true and many of the PC magazines reflect this in their comments. RZ> There are lots of left over cycles to calculate the parity. RZ> RZ> Of course this is not the only thing the affects speed. Other issues that RZ> make our units fast is the PCI bus which is 133Mbs and DMA directly to RZ> drives. It is however, still unclear whether it's safe to run reiserfs on a raidzone. I have a question about that out to Colin. Brian
Re: [FAQ-answer] Re: soft RAID5 + journalled FS + power failure = problems ?
Hi, On Wed, 12 Jan 2000 00:12:55 +0200 (IST), Gadi Oxman <[EMAIL PROTECTED]> said: > Stephen, I'm afraid that there are some misconceptions about the > RAID-5 code. I don't think so --- I've been through this with Ingo --- but I appreciate your feedback since I'm getting inconsistent advise here! Please let me explain... > In an early pre-release version of the RAID code (more than two years > ago?), which didn't protect against that race, we indeed saw locked > buffers changing under us from the point in which we computed the > parity till the point in which they were actually written to the disk, > leading to a corrupted parity. That is not the race. The race has nothing at all to do with buffers changing while they are being used for parity: that's a different problem, long ago fixed by copying the buffers. The race I'm concerned about could occur when the raid driver wants to compute parity for a stripe and finds some of the blocks are present, and clean, in the buffer cache. Raid assumes that those buffers represent what is on disk, naturally enough. So, it uses them to calculate parity without rereading all of the disk blocks in the stripe. The trouble is that the standard practice in the kernel, when modifying a buffer, is to make the change and _then_ mark the buffer dirty. If you hit that window, then the raid driver will find a buffer which doesn't match what is on disk, and will compute parity from that buffer rather than from the on-disk contents. > 1. n dirty blocks are scheduled for a stripe write. That's not the race. The problem occurs when only one single dirty block is scheduled for a write, and we need to find the contents of the rest of the stripe to compute parity. > Point (2) is also incorrect; we have taken care *not* to peek into > the buffer cache to find clean buffers and use them for parity > calculations. We make no such assumptions. Not according to Ingo --- can we get a definitive answer on this, please? Many thanks, Stephen
Re: [FAQ-answer] Re: soft RAID5 + journalled FS + power failure = problems ?
"Stephen C. Tweedie" wrote: > > Hi, > > On Tue, 11 Jan 2000 15:03:03 +0100, mauelsha > <[EMAIL PROTECTED]> said: > > >> THIS IS EXPECTED. RAID-5 isn't proof against multiple failures, and the > >> only way you can get bitten by this failure mode is to have a system > >> failure and a disk failure at the same time. > > > To try to avoid this kind of problem some brands do have additional > > logging (to disk which is slow for sure or to NVRAM) in place, which > > enables them to at least recognize the fault to avoid the > > reconstruction of invalid data or even enables them to recover the > > data by using redundant copies of it in NVRAM + logging information > > what could be written to the disks and what not. > > Absolutely: the only way to avoid it is to make the data+parity updates > atomic, either in NVRAM or via transactions. I'm not aware of any > software RAID solutions which do such logging at the moment: do you know > of any? > AFAIK Veritas only does the first part of what i mentioned above (invalid on disk data recognition). They do logging by default for RAID5 volumes and optionaly also for RAID1 volumes. In the RAID5 (with logging) case they can figure out if an n-1 disk write took place and can rebuild the data. In case an n-m (1 < m < n) took place they can therefore at least recognize the desaster ;-) In the RAID1 (with logging) scenario they are able to recognize, which of the n mirrors have actual data and which ones don't to deliver the actual data to the user and to try to make the other mirrors consistent. But because it's a software solution without any NVRAM support they can't handle the data redundancy case. Heinz
Mylex Ultra3 Cards
Has anyone used the new ExtremeRAID or AccelRAID 352 on Linux with the DAC960 drivers? I run SuSE 6.3 on the Ultra2 and UW DAC, AccelRAD 250, and ExtremeRAID 1164, but thought I might ask before diving into the new gear. Regards, Jon Jon R. Doyle Systems Administrator Document Solutions, Inc. 1611 Telegraph Avenue Ste. 1010 Oakland, Ca. 94612 510-986-0250
Re: Ribbon Cabling (was Re: large ide raid system)
[ Tuesday, January 11, 2000 ] Andy Poling wrote: > On Tue, 11 Jan 2000, Gregory Leblanc wrote: > > If you cut the cable > > lengthwise (no, don't cut the wires) between wires (don't break the > > insulation on the wires themselves, just the connecting plastic) you can > > get your cables to be 1/4 the normal width (up until you get to the > > connector). > > I don't know about IDE, but I'm pretty sure that's a big no-no for SCSI > cables. The alternating conductors in the ribbon cable are sig, gnd, sig, > gnd, sig, etc. And it's electrically important (for proper impedance and > noise and cross-talk rejection) that they stay that way. > > I think the same is probably true for the schmancy UDMA66 cables too... So just check with a cable spec and make sure you're not separating a data signal from its ground return path. Throw some mag rings around the thing if you want, but since we're (hopefully) terminated properly (no reflection) the crosstalk issues aren't huge... they suffer more through the LC matrix of connector adaptors than this split would cause :) James -- Miscellaneous Engineer --- IBM Netfinity Performance Development
Re: optimising raid performance
>dual PII-500 >> single 233 :) Don't underestimate the StrongArm, if the mylex firmware writers have half a brain I think you'd be suprised. >s/w raid 1 over 2 6-disk h/w raid0's is what I meant to ask for >I trust the strongarm to handle raid0, but that's about it :) Ok, will give it a go on thursday afternoon when I can get my hands on the kit. >what stripe/chunk sizes are you using in the raid? My exp. has been >smaller is better down to 4k, although I'm not sure why :) We're currently using 8k but with our load then if I can go smaller I will do. Is there any merit in using -R on mke2fs if we're doing raid1? Chris Good - Technical Architect - Webtop.com -- Chris Good - Dialog Corp. The Westbrook Centre, Milton Rd, Cambridge UK Phone: 01223 715000 Fax: 01223 715001 http://www.dialog.com
