Re: howto know if raid is really working?
In article <[EMAIL PROTECTED]>, Harland Christofferson <[EMAIL PROTECTED]> wrote: >At Thursday, 16 December 2004, you wrote: > >>Thanks for the reply, but I still haven't cleared my doubt. >>I'll cut the relevant part of the reply (relevant for what I'm trying >to >>understand) and comment at the end: >> >*snip* > >you mention mirroring partions. i have the entire disk mirrored, >not the individual partions. is it better to mirror individual partions? The current read-balancing code in RAID1 works better on whole-disk raid, since different MD devices don't know anything about eachother. Mike. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
raid Re: howto know if raid is really working?
On Wed, 15 Dec 2004, Harland Christofferson wrote: > >> development:/etc# cat raidtab > >> raiddev /dev/md0 > >> raid-level 1 > >> nr-raid-disks 2 notice ... level 1 and number of devices > mdadm --stop /dev/md0; mdadm -C /dev/md0 -l1 -n2 /dev/hda /dev/hdc not good ... you only use "-C" the first time you should NOT have to specify /dev/ stuff since you created /etc/mtab and/or put it in the right location that mdadm will find it you should be using: mdadm create mdadm assemble mdadm stop mdadm start mdadm examine .. etc .. c ya alvin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: howto know if raid is really working?
At Wednesday, 15 December 2004, Alvin Oga <[EMAIL PROTECTED] com> wrote: >hi ya harland > >On Wed, 15 Dec 2004, Harland Christofferson wrote: > >> development:/etc# cat raidtab >> raiddev /dev/md0 >> raid-level 1 >> nr-raid-disks 2 >> nr-spare-disks 0 >> chunk-size 4 >> persistent-superblock 1 >> device /dev/hda >> raid-disk 0 >> device /dev/hdc >> raid-disk 1 > >you need to use /dev/hda1 and /dev/hdc1 > >or whatever your corresponding partition is for your setup > >/dev/hda1 / >/dev/hda2 /tmp >/dev/hda3 /var >/dev/hda5 /usr >/dev/hda6 swap >/dev/hda7 /home > >. gets extremely tiresome for creating /dev/md devices > for sw raid > >use google to find other raidtab examples for >config with multiple partitions > >c ya >alvin > okay ... it looks like using the whole disk was not the way to go as it looks like the only time the disks are synced is when i mdadm --stop /dev/md0; mdadm -C /dev/md0 -l1 -n2 /dev/hda /dev/hdc some of the reading i have googled leads me to believe that i have to start from scratch. the following commands aren't working out for me: mdadm -C /dev/md0 -l1 -n2 /dev/hda1 /dev/hdc1 mdadm -C /dev/md1 -l1 -n2 /dev/hda3 /dev/hdc3 mdadm -C /dev/md2 -l1 -n2 /dev/hda6 /dev/hdc6 mdadm -C /dev/md3 -l1 -n2 /dev/hda7 /dev/hdc7 mdadm -C /dev/md4 -l1 -n2 /dev/hda8 /dev/hdc8 mdadm -C /dev/md5 -l1 -n2 /dev/hda5 /dev/hdc5 -- Zero Crossings, Inc. -- Embedded and Digital Signal Processing Systems http://www.zerocrossings.com/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: howto know if raid is really working?
hi ya harland On Wed, 15 Dec 2004, Harland Christofferson wrote: > development:/etc# cat raidtab > raiddev /dev/md0 > raid-level 1 > nr-raid-disks 2 > nr-spare-disks 0 > chunk-size 4 > persistent-superblock 1 > device /dev/hda > raid-disk 0 > device /dev/hdc > raid-disk 1 you need to use /dev/hda1 and /dev/hdc1 or whatever your corresponding partition is for your setup /dev/hda1 / /dev/hda2 /tmp /dev/hda3 /var /dev/hda5 /usr /dev/hda6 swap /dev/hda7 /home gets extremely tiresome for creating /dev/md devices for sw raid use google to find other raidtab examples for config with multiple partitions c ya alvin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: howto know if raid is really working?
hi ya joao On Thu, 16 Dec 2004, Joao Clemente wrote: > Image you're using software raid and 1 disk fails. You somehow get > alerted and , AFAIK, you > 1 - shutdown the machine (ok, this if you don't have a hot-swap system) > 2 - remove the failed disk > 3 - insert a new, "fresh-from-the-store" disk for sanity ... i always fdisk the new disk to be the same as the remaining disk > 4 - power-up and sw raid will mirror the good disk onto the new disk depending on size of your disk ( data ), it can take a day .. if you continue to write data, while is mirroring, yo risk losing everything ... if the idea of mirroring was so that you can operate, 24x7x365, than you should be using a complete server in NYC and a complete server in LA ... having 2 disks on one system is an oxymoronic way to (try) guarantee 24x7x365 operation with zero downtime IDE is NOT hot swappable SATA disk tries to be hotswappable by looking like a scsi disk ... SCA scsi disks is hot swappable but is NOT cheap in terms of the same sized 1TB of 4x 300GB ( $300ea ) IDE disk array vs lots of expensive hotswap scsi disks to create 1TB of space > Now, if this was a hardware raid solution, yes I believe the array will > self-contruct again. sw raid, when PROPERLY created will also resysnc/self-construct again all by itself > My question is if, with these steps you'll have a > software RAID system resync'ing the array... or you need to do extra > steps like: no extra step is supposed to be needed except to take the old disk out and plug in a new one - power down would depend on if its ide or sata or scsi and how the disk is mounted > 5 - partition the disk with same partition layout as the removed one probably a good idea ... to keep it the same as before even if your enw disk is bigger than before - use the xtra (unused) space for something else > and only after this step the array can re-construct .. What's your > experience on this? no problems with sw raid .. hw raid isn't worth a penny .. ie .. throw it away .. but if you got a real raid controller for say $10K or $20K where that's all the company makes is raid controllers, than those hw raid controllers does work as advertized - pc/pci based hw raid is a disaster waiting to happen - hopefaully data is backed up on other systems where last weeks data is NOT overwritten by this weeks suspect/corrupt new data which you find out is corrupted 2 weeks in the future c ya alvin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
RE: howto know if raid is really working?
On Wed, 15 Dec 2004, Harland Christofferson wrote: > but, what if i write a file to /dev/hda6 (/home), then i mount /dev/hdc6 > to /mnt/hdc6 ... should i see the newly written file in the same > place on /mnt/hdc6 as was written on /dev/hda6 ? given that raid is confusing ... do NOT, do NOT mount the raid partitions separately ... it will probably corrupt your raid data raid works on timestamps on the inodes and other temp files to figure out to copy data from hda to hdc or hdc to hda - which every one has the "supposdely, newer" data mirroring is NOT identical on both disk -- you can have a working software raid with a 10GB disk and a 300GB disks .. but, you'd be wasting 290GB of the 300GB, so you'd probably be using that for something else instead of sw-raid c ya alvin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: howto know if raid is really working?
At Wednesday, 15 December 2004, you wrote: >On Thu, 16 Dec 2004, Joao Clemente wrote: > >> Ok, Alvin, as far as I understand when you said at the top "for sanity >> ... i always fdisk the new disk to be the same as the remaining disk" >> you mean limiting it's size, as you say here at last few lines, right? > >i manually partition the new disk, because i do not trust that >the sw raid mirroring will partition the right way > >> My doubt is: If you DONT do this (and, following my steps, you CANT >> fdisk unless you power the machine first :-) what will happen then? > >you cannot fdisk once /dev/mdxxx is created > >you create the sw raid by: > fdisk /dev/hda ... > fdisk /dev/hdc ... > > ( if you do NOT partition it ... i think sw raid uses > ( the whole disk as 1 giant partiton .. i always partition it > > mdadd /dev/md0 /dev/hda1 /dev/hdc1 > mdadd /dev/md1 /dev/hda2 /dev/hdc2 > ... > > mke2fs -j /dev/md0 > mke2fs -j /dev/md1 > ... > > mount /dev/md0 / > mount /dev/md1 /home Oh, I think I see what I am doing incorrectly, I still have mounted /dev/hda1 /home ... not /dev/md0 /home Also, my raidtab is: development:/etc# cat raidtab raiddev /dev/md0 raid-level 1 nr-raid-disks 2 nr-spare-disks 0 chunk-size 4 persistent-superblock 1 device /dev/hda raid-disk 0 device /dev/hdc raid-disk 1 > ... > > fix /etc/mtab or wherever the equivalent file is saved > >> Supose you have your disks with 3 partitions each, {hd?1, hd?2, hda?3} >> from wich you have your 3 software raid partitions {md0, md1, md2}. > >good > >> One disk fails. You put a new one. New as "out-of-the-shop", no >> partitions, no filesystem. What happens? Will the partitions be >> generated? > >yes ... if you trust the system to do it for you > >> Or you need do setup {hdx1, hdx2, hdx3} on the new disk, >> before software raid resyncs the disks? > >i prefer to manually fdisk the new disk so that i dont count >on the sw code to do it right or wrong > >> Or this is not the way to do it? > >trail and error ... > >i've never had a problem when i fdisk it manually first >and it also tells me i can write the disk, at least partition it > >> From what I remember reading, HW RAID handles "disks", SW RAID handles >> "partitions". > >i think you can make /dev/hda1 and /dev/hdc1 as one "whole disk" > >even "hw raid" will have at least one partition > >if you use oracle .. they will use raw disks .. no partitions > >> If you replace a disk in a HW RAID, the new disk will be >> copied and be equal to the older ones. > >not necessarily... > >but than again, most people do not mix and match different sized >disks when replacing the dead one > >and in sw raid .. you can have the dead 40GB disks replaced by >a 300GB disks and everythign will still work properly > and have a spare (unused) 260GB on the new disk > >> Mapping this to SW RAID makes the >> sentence like this: "If you replace a PARTITION in a SW RAID, the new >> PARTITION will be copied and be equal to the older ones". > >for the mirrored and used portion of the raid .. > >> So what >> happens if the disk has no partitions? > >sw raid will partition the new disk for you >and format it and merge it into the raid array and sync >the data onto the new disk > >c ya >alvin > >-- -- Zero Crossings, Inc. -- Embedded and Digital Signal Processing Systems http://www.zerocrossings.com/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: howto know if raid is really working?
On Thu, 16 Dec 2004, Joao Clemente wrote: > Ok, Alvin, as far as I understand when you said at the top "for sanity > ... i always fdisk the new disk to be the same as the remaining disk" > you mean limiting it's size, as you say here at last few lines, right? i manually partition the new disk, because i do not trust that the sw raid mirroring will partition the right way > My doubt is: If you DONT do this (and, following my steps, you CANT > fdisk unless you power the machine first :-) what will happen then? you cannot fdisk once /dev/mdxxx is created you create the sw raid by: fdisk /dev/hda ... fdisk /dev/hdc ... ( if you do NOT partition it ... i think sw raid uses ( the whole disk as 1 giant partiton .. i always partition it mdadd /dev/md0 /dev/hda1 /dev/hdc1 mdadd /dev/md1 /dev/hda2 /dev/hdc2 ... mke2fs -j /dev/md0 mke2fs -j /dev/md1 ... mount /dev/md0 / mount /dev/md1 /home ... fix /etc/mtab or wherever the equivalent file is saved > Supose you have your disks with 3 partitions each, {hd?1, hd?2, hda?3} > from wich you have your 3 software raid partitions {md0, md1, md2}. good > One disk fails. You put a new one. New as "out-of-the-shop", no > partitions, no filesystem. What happens? Will the partitions be > generated? yes ... if you trust the system to do it for you > Or you need do setup {hdx1, hdx2, hdx3} on the new disk, > before software raid resyncs the disks? i prefer to manually fdisk the new disk so that i dont count on the sw code to do it right or wrong > Or this is not the way to do it? trail and error ... i've never had a problem when i fdisk it manually first and it also tells me i can write the disk, at least partition it > From what I remember reading, HW RAID handles "disks", SW RAID handles > "partitions". i think you can make /dev/hda1 and /dev/hdc1 as one "whole disk" even "hw raid" will have at least one partition if you use oracle .. they will use raw disks .. no partitions > If you replace a disk in a HW RAID, the new disk will be > copied and be equal to the older ones. not necessarily... but than again, most people do not mix and match different sized disks when replacing the dead one and in sw raid .. you can have the dead 40GB disks replaced by a 300GB disks and everythign will still work properly and have a spare (unused) 260GB on the new disk > Mapping this to SW RAID makes the > sentence like this: "If you replace a PARTITION in a SW RAID, the new > PARTITION will be copied and be equal to the older ones". for the mirrored and used portion of the raid .. > So what > happens if the disk has no partitions? sw raid will partition the new disk for you and format it and merge it into the raid array and sync the data onto the new disk c ya alvin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: howto know if raid is really working?
At Thursday, 16 December 2004, you wrote: >Thanks for the reply, but I still haven't cleared my doubt. >I'll cut the relevant part of the reply (relevant for what I'm trying to >understand) and comment at the end: > *snip* you mention mirroring partions. i have the entire disk mirrored, not the individual partions. is it better to mirror individual partions? -- Zero Crossings, Inc. -- Embedded and Digital Signal Processing Systems http://www.zerocrossings.com/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: howto know if raid is really working?
Harland Christofferson wrote: [snip] but, what if i write a file to /dev/hda6 (/home), then i mount /dev/hdc6 to /mnt/hdc6 ... should i see the newly written file in the same place on /mnt/hdc6 as was written on /dev/hda6 ? Yo Harland, you're getting all messed up For what I've understood, you have /dev/hda6 and /dev/hdc6 together as a RAID1 partition (/dev/mdx, wich is monted at /home) Example: - Power down, take hdc out, power up. - Write a file to /home. - Power down, insert hdc, power up. - Let raid resync. - Power down, take hda out, power up. (now only the second disk is there) - You should see the file in /home. It has been resynced earlyer. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: howto know if raid is really working?
Thanks for the reply, but I still haven't cleared my doubt. I'll cut the relevant part of the reply (relevant for what I'm trying to understand) and comment at the end: Alvin Oga wrote: On Thu, 16 Dec 2004, Joao Clemente wrote: Image you're using software raid and 1 disk fails. You somehow get alerted and , AFAIK, you 1 - shutdown the machine (ok, this if you don't have a hot-swap system) 2 - remove the failed disk 3 - insert a new, "fresh-from-the-store" disk for sanity ... i always fdisk the new disk to be the same as the remaining disk 4 - power-up and sw raid will mirror the good disk onto the new disk [snip] Now, if this was a hardware raid solution, yes I believe the array will self-contruct again. sw raid, when PROPERLY created will also resysnc/self-construct again all by itself My question is if, with these steps you'll have a software RAID system resync'ing the array... or you need to do extra steps like: no extra step is supposed to be needed except to take the old disk out and plug in a new one [snip] 5 - partition the disk with same partition layout as the removed one probably a good idea ... to keep it the same as before even if your enw disk is bigger than before - use the xtra (unused) space for something else Ok, Alvin, as far as I understand when you said at the top "for sanity ... i always fdisk the new disk to be the same as the remaining disk" you mean limiting it's size, as you say here at last few lines, right? My doubt is: If you DONT do this (and, following my steps, you CANT fdisk unless you power the machine first :-) what will happen then? Supose you have your disks with 3 partitions each, {hd?1, hd?2, hda?3} from wich you have your 3 software raid partitions {md0, md1, md2}. One disk fails. You put a new one. New as "out-of-the-shop", no partitions, no filesystem. What happens? Will the partitions be generated? Or you need do setup {hdx1, hdx2, hdx3} on the new disk, before software raid resyncs the disks? Or this is not the way to do it? From what I remember reading, HW RAID handles "disks", SW RAID handles "partitions". If you replace a disk in a HW RAID, the new disk will be copied and be equal to the older ones. Mapping this to SW RAID makes the sentence like this: "If you replace a PARTITION in a SW RAID, the new PARTITION will be copied and be equal to the older ones". So what happens if the disk has no partitions? Thanks Joao Clemente -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
RE: howto know if raid is really working?
At Wednesday, 15 December 2004, Alvin Oga <[EMAIL PROTECTED] com> wrote: >On Wed, 15 Dec 2004, Harland Christofferson wrote: > >> I am curious about how to know if it is working. If cat-ting /proc/mdstat >> is knowing all about the raid array, then so-be-it. > >when you think its working .. > >power down .. pull one disk out ... > power up .. write a 2GB or 4GB file on the degraded array > and wait for it to finish, and power down > >stick the first disk back in and see that when it powers up, >that it will resync the new 4GB file onto the 2nd disk > >do the same with each of the other disk pulled out .. > >if it always resync ... its working > >if you have to touch the keyboard to make it work, >than its NOT working > only "cat /proc/mdstat" is allowed > any other raid commands says the raid is not working > >c ya >alvin > but, what if i write a file to /dev/hda6 (/home), then i mount /dev/hdc6 to /mnt/hdc6 ... should i see the newly written file in the same place on /mnt/hdc6 as was written on /dev/hda6 ? -- Zero Crossings, Inc. -- Embedded and Digital Signal Processing Systems http://www.zerocrossings.com/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: howto know if raid is really working?
At Wednesday, 15 December 2004, Alvin Oga <[EMAIL PROTECTED] com> wrote: >hi ya joao > >On Thu, 16 Dec 2004, Joao Clemente wrote: > >> Image you're using software raid and 1 disk fails. You somehow get >> alerted and , AFAIK, you >> 1 - shutdown the machine (ok, this if you don't have a hot-swap system) >> 2 - remove the failed disk >> 3 - insert a new, "fresh-from-the-store" disk > >for sanity ... i always fdisk the new disk to be the same as the >remaining disk > >> 4 - power-up > >and sw raid will mirror the good disk onto the new disk > >depending on size of your disk ( data ), it can take a day .. > >if you continue to write data, while is mirroring, yo risk losing >everything ... > >if the idea of mirroring was so that you can operate, 24x7x365, >than you should be using a complete server in NYC and a complete server >in LA ... > >having 2 disks on one system is an oxymoronic way to (try) guarantee >24x7x365 operation with zero downtime > >IDE is NOT hot swappable > >SATA disk tries to be hotswappable by looking like a scsi disk ... > >SCA scsi disks is hot swappable but is NOT cheap in terms of >the same sized 1TB of 4x 300GB ( $300ea ) IDE disk array >vs lots of expensive hotswap scsi disks to create 1TB of space > >> Now, if this was a hardware raid solution, yes I believe the array will >> self-contruct again. > >sw raid, when PROPERLY created will also resysnc/self-construct >again all by itself > >> My question is if, with these steps you'll have a >> software RAID system resync'ing the array... or you need to do extra >> steps like: > >no extra step is supposed to be needed except to take the old >disk out and plug in a new one > - power down would depend on if its ide or sata or scsi > and how the disk is mounted > >> 5 - partition the disk with same partition layout as the removed one > >probably a good idea ... to keep it the same as before >even if your enw disk is bigger than before > - use the xtra (unused) space for something else > >> and only after this step the array can re-construct .. What's your >> experience on this? > >no problems with sw raid .. > >hw raid isn't worth a penny .. ie .. throw it away .. > >but if you got a real raid controller for say $10K or $20K >where that's all the company makes is raid controllers, than >those hw raid controllers does work as advertized >- pc/pci based hw raid is a disaster waiting to happen > > - hopefaully data is backed up > on other systems where last weeks data is NOT > overwritten by this weeks suspect/corrupt new data > which you find out is corrupted 2 weeks in the future > >c ya >alvin > >-- >To UNSUBSCRIBE, email to [EMAIL PROTECTED] >with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED] debian.org > it's all so confusing. -- Zero Crossings, Inc. -- Embedded and Digital Signal Processing Systems http://www.zerocrossings.com/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: howto know if raid is really working?
Alvin Oga wrote: if you have to touch the keyboard to make it work, than its NOT working only "cat /proc/mdstat" is allowed any other raid commands says the raid is not working c ya alvin Hi Alvin. I'm just wondering this: Is this true for software RAID aswell? My doubt is about sofware raid vs disk partitions.. Image you're using software raid and 1 disk fails. You somehow get alerted and , AFAIK, you 1 - shutdown the machine (ok, this if you don't have a hot-swap system) 2 - remove the failed disk 3 - insert a new, "fresh-from-the-store" disk 4 - power-up Now, if this was a hardware raid solution, yes I believe the array will self-contruct again. My question is if, with these steps you'll have a software RAID system resync'ing the array... or you need to do extra steps like: 5 - partition the disk with same partition layout as the removed one and only after this step the array can re-construct .. What's your experience on this? Thanks Joao Clemente -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
RE: howto know if raid is really working?
On Wed, 15 Dec 2004, Harland Christofferson wrote: > I am curious about how to know if it is working. If cat-ting /proc/mdstat > is knowing all about the raid array, then so-be-it. when you think its working .. power down .. pull one disk out ... power up .. write a 2GB or 4GB file on the degraded array and wait for it to finish, and power down stick the first disk back in and see that when it powers up, that it will resync the new 4GB file onto the 2nd disk do the same with each of the other disk pulled out .. if it always resync ... its working if you have to touch the keyboard to make it work, than its NOT working only "cat /proc/mdstat" is allowed any other raid commands says the raid is not working c ya alvin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
RE: howto know if raid is really working?
At Thursday, 16 December 2004, "Steven Jones" <[EMAIL PROTECTED] nz> wrote: >mdstat shows quite well that all the disks are on line, or not, it is >simple and effective IMHO. > >When you mount one side of the mirror you may actually be corrupting it. >possibly preventing it being written to. > >If you are going to do that, go to single user mode, unmount the mirror >device, mount one/both sides in read only mode and look at the files. > >I must admit this is novel, I have never come across someone not trusting >a mirror before > >regards > >Thing > I am curious about how to know if it is working. If cat-ting /proc/mdstat is knowing all about the raid array, then so-be-it. cheers! -- Zero Crossings, Inc. -- Embedded and Digital Signal Processing Systems http://www.zerocrossings.com/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]