Re: gmirror problem on 5.3-R i386 (SOLVED)
On Thu, Jan 13, 2005 at 09:51:03AM +0100, Christian Hiris wrote: Hash: SHA1 On Thursday 13 January 2005 03:54, Doug Poland wrote: I'm having a problem with gmirror. It would seem that I cannot add my first disk to the mirror after booting off the second drive. My current status is a degraded mirror and I cannot see any of the partitions on disk one. # dd if=/dev/zero of=/dev/ad4 bs=512 count=79 You probably destroyed your slice table here. The dd command only makes sense if you insert whole disks (ie. /dev/ad4 and /dev/ad6) as providers. Otherwise you need to create (or already have) a valid slice table on the disk, which enables gmirror to locate and insert your slice. One of my problems was incorrect partition sizes when I created the bsdlabel for /dev/mirror/gm0s1. I didn't really know what I was doing nor did I understand the consequences of my actions ;) (!) Before you start to correct your gmirror setup, please read all of my comments, because to me it looks like you run a mirror of ad6 and not ad6s1. I'm not sure why gmirror label shows that but it's definately misleading. Since this is a brand-new box with nothing but a minimal install, I decided to re-install 5.3-R. This time I got the install right and gmirror reads: Geom name: gm0s1 State: COMPLETE Components: 2 Balance: round-robin Slice: 4096 Flags: NONE SyncID: 1 ID: 3524221455 Providers: 1. Name: mirror/gm0s1 Mediasize: 80026329088 (75G) Sectorsize: 512 Mode: r5w5e1 Consumers: 1. Name: ad4 Mediasize: 80026361856 (75G) Sectorsize: 512 Mode: r5w5e2 State: ACTIVE Priority: 0 Flags: DIRTY SyncID: 1 ID: 2031344187 2. Name: ad6 Mediasize: 80026361856 (75G) Sectorsize: 512 Mode: r5w5e2 State: ACTIVE Priority: 0 Flags: DIRTY SyncID: 1 ID: 3116345061 Geom name: gm0s1.sync I'm going to include the steps I took, with one deviation from the published doc I referenced, and ask a couple more questions. 1 dd if=/dev/zero of=/dev/ad6 bs=512 count=79 2 fdisk -v -B -I /dev/ad6 3 4 gmirror label -v -n -b round-robin gm0s1 /dev/ad6s1 5 gmirror load 6 7 bsdlabel -w -B /dev/mirror/gm0s1 8 bsdlabel -e /dev/mirror/gm0s1 9 10 newfs -U /dev/mirror/gm0s1a 11 mount /dev/mirror/gm0s1a /mnt 12 dump -L -0 -f- / | ( cd /mnt ; restore -r -v -f- ) 13 newfs -U /dev/mirror/gm0s1d 14 mount /dev/mirror/gm0s1d /mnt/var 15 dump -L -0 -f- /var | ( cd /mnt/var ; restore -r -v -f- ) 16 newfs -U /dev/mirror/gm0s1e 17 mount /dev/mirror/gm0s1e /mnt/tmp 18 dump -L -0 -f- /tmp | ( cd /mnt/tmp ; restore -r -v -f- ) 19 newfs -U /dev/mirror/gm0s1f 20 mount /dev/mirror/gm0s1f /mnt/usr 21 dump -L -0 -f- /usr | ( cd /mnt/usr ; restore -r -v -f- ) 22 23 cp -p /mnt/etc/fstab /mnt/etc/fstab.orig 24 sed -e 's/dev\/ad4s1/dev\/mirror\/gm0s1/g' /mnt/etc/fstab.orig /mnt/etc/fstab 25 echo 'swapoff=YES' /mnt/etc/rc.conf 26 echo 'geom_mirror_load=YES' /mnt/boot/loader.conf 27 echo 1:ad(6,a)/boot/loader /boot.config 28 29 shutdown -r now 30 31 dd if=/dev/zero of=/dev/ad4 bs=512 count=79 32 fdisk -v -B -I /dev/ad4 33 gmirror configure -a gm0s1 34 gmirror insert gm0s1 /dev/ad4s1 35 sh -c 'while [ .`gmirror list | grep SYNCHRONIZING` != . ]; do sleep 1; done' 36 37 shutdown -r now On line 32, I deviated from the instructions on: http://people.freebsd.org/~rse/mirror/ The published command was: size=`fdisk ad6 | grep ', size ' | head -1 | sed -e 's;^.*size \([0-9]*\).*$;\1;'` (echo p 1 165 63 $size; echo a 1) | fdisk -v -B -f- -i /dev/ad4 That command returned: improperly placed quotes (). I set the value of $size manually but it was some small number like 5. I wrote it to ad4 but the command on line 33 puked. So I simply issued the command shown on line 32. Question: On line 27 we issue the command to... # instruct boot stage 2 loader on first disk to boot # with the boot stage 3 loader from the second disk # (mainly because BIOS might not allow easy booting from second ATA disk # or at least requires manual intervention on the console) So how do I get rid of that boot.config file? Should I get rid of it? Question: On line 25 we set swapoff=YES. Is this necessary if I'm going to run 5-STABLE? What are the implications of having swap on a mirror like this? Thanks for your help! -- Regards, Doug ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: gmirror problem on 5.3-R i386 (SOLVED)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Friday 14 January 2005 20:43, Doug Poland wrote: On Thu, Jan 13, 2005 at 09:51:03AM +0100, Christian Hiris wrote: On Thursday 13 January 2005 03:54, Doug Poland wrote: I'm having a problem with gmirror. It would seem that I cannot add my first disk to the mirror after booting off the second drive. My current status is a degraded mirror and I cannot see any of the partitions on disk one. # dd if=/dev/zero of=/dev/ad4 bs=512 count=79 You probably destroyed your slice table here. The dd command only makes sense if you insert whole disks (ie. /dev/ad4 and /dev/ad6) as providers. Otherwise you need to create (or already have) a valid slice table on the disk, which enables gmirror to locate and insert your slice. One of my problems was incorrect partition sizes when I created the bsdlabel for /dev/mirror/gm0s1. I didn't really know what I was doing nor did I understand the consequences of my actions ;) :) (!) Before you start to correct your gmirror setup, please read all of my comments, because to me it looks like you run a mirror of ad6 and not ad6s1. I'm not sure why gmirror label shows that but it's definately misleading. Since this is a brand-new box with nothing but a minimal install, I decided to re-install 5.3-R. This time I got the install right and gmirror reads: This looks like gmirror in fact uses the the whole disk. The difference between the providers and the consumers length is 63 sectors, which is the first slice's offset (see fdisk below). Geom name: gm0s1 State: COMPLETE Components: 2 Balance: round-robin Slice: 4096 Flags: NONE SyncID: 1 ID: 3524221455 Providers: 1. Name: mirror/gm0s1 Mediasize: 80026329088 (75G) Sectorsize: 512 Mode: r5w5e1 Consumers: 1. Name: ad4 Mediasize: 80026361856 (75G) Sectorsize: 512 Mode: r5w5e2 State: ACTIVE Priority: 0 Flags: DIRTY SyncID: 1 ID: 2031344187 2. Name: ad6 Mediasize: 80026361856 (75G) Sectorsize: 512 Mode: r5w5e2 State: ACTIVE Priority: 0 Flags: DIRTY SyncID: 1 ID: 3116345061 Geom name: gm0s1.sync I'm going to include the steps I took, with one deviation from the published doc I referenced, and ask a couple more questions. 1 dd if=/dev/zero of=/dev/ad6 bs=512 count=79 2 fdisk -v -B -I /dev/ad6 3 4 gmirror label -v -n -b round-robin gm0s1 /dev/ad6s1 5 gmirror load 6 7 bsdlabel -w -B /dev/mirror/gm0s1 8 bsdlabel -e /dev/mirror/gm0s1 9 10 newfs -U /dev/mirror/gm0s1a 11 mount /dev/mirror/gm0s1a /mnt 12 dump -L -0 -f- / | ( cd /mnt ; restore -r -v -f- ) 13 newfs -U /dev/mirror/gm0s1d 14 mount /dev/mirror/gm0s1d /mnt/var 15 dump -L -0 -f- /var | ( cd /mnt/var ; restore -r -v -f- ) 16 newfs -U /dev/mirror/gm0s1e 17 mount /dev/mirror/gm0s1e /mnt/tmp 18 dump -L -0 -f- /tmp | ( cd /mnt/tmp ; restore -r -v -f- ) 19 newfs -U /dev/mirror/gm0s1f 20 mount /dev/mirror/gm0s1f /mnt/usr 21 dump -L -0 -f- /usr | ( cd /mnt/usr ; restore -r -v -f- ) 22 23 cp -p /mnt/etc/fstab /mnt/etc/fstab.orig 24 sed -e 's/dev\/ad4s1/dev\/mirror\/gm0s1/g' /mnt/etc/fstab.orig /mnt/etc/fstab 25 echo 'swapoff=YES' /mnt/etc/rc.conf 26 echo 'geom_mirror_load=YES' /mnt/boot/loader.conf 27 echo 1:ad(6,a)/boot/loader /boot.config 28 29 shutdown -r now 30 31 dd if=/dev/zero of=/dev/ad4 bs=512 count=79 32 fdisk -v -B -I /dev/ad4 33 gmirror configure -a gm0s1 34 gmirror insert gm0s1 /dev/ad4s1 35 sh -c 'while [ .`gmirror list | grep SYNCHRONIZING` != . ]; do sleep 1; done' 36 37 shutdown -r now On line 32, I deviated from the instructions on: http://people.freebsd.org/~rse/mirror/ The published command was: size=`fdisk ad6 | grep ', size ' | head -1 | sed -e 's;^.*size \([0-9]*\).*$;\1;'` (echo p 1 165 63 $size; echo a 1) | fdisk -v -B -f- -i /dev/ad4 That command returned: improperly placed quotes (). I set the value of $size manually but it was some small number like 5. I wrote it to ad4 but the command on line 33 puked. So I simply issued the command shown on line 32. I ever handled this by sysinstall. The above commands simply sets up the first slice (1) to a FreeBSD-type slice (165) starting at offset (63) with length ($size) of the first slice of ad6 and sets the active flag on the first slice (1). The command 'fdisk ad6' shows the size of the slice(s) on ad6. The same length should be used for ad4. I tried to run the above commands under /bin/sh: # size=`fdisk ad6 | grep ', size ' | head -1 | sed -e 's;^.*size \([0-9]*\).*$;\1;'` # (echo p 1 165 63 $size; echo a 1) p 1 165 63 240107427 a 1 # (echo p 1 165 63 $size; echo a 1) | fdisk -v -t -B -f -i mirror/mirror0 *** Working on device /dev/mirror/mirror0 *** So all the commands seem to work properly. Did you run this under /bin/sh? Question: On line 27 we issue the command to... # instruct boot stage 2 loader on first disk to boot # with the boot
Re: gmirror problem on 5.3-R i386 (SOLVED)
On Sat, Jan 15, 2005 at 12:12:28AM +0100, Christian Hiris wrote: On Friday 14 January 2005 20:43, Doug Poland wrote: size=`fdisk ad6 | grep ', size ' | head -1 | sed -e 's;^.*size \([0-9]*\).*$;\1;'` (echo p 1 165 63 $size; echo a 1) | fdisk -v -B -f- -i /dev/ad4 That command returned: improperly placed quotes (). I set the value of $size manually but it was some small number like 5. I wrote it to ad4 but the command on line 33 puked. So I simply issued the command shown on line 32. I ever handled this by sysinstall. The above commands simply sets up the first slice (1) to a FreeBSD-type slice (165) starting at offset (63) with length ($size) of the first slice of ad6 and sets the active flag on the first slice (1). The command 'fdisk ad6' shows the size of the slice(s) on ad6. The same length should be used for ad4. I tried to run the above commands under /bin/sh: # size=`fdisk ad6 | grep ', size ' | head -1 | sed -e 's;^.*size \([0-9]*\).*$;\1;'` # (echo p 1 165 63 $size; echo a 1) p 1 165 63 240107427 a 1 # (echo p 1 165 63 $size; echo a 1) | fdisk -v -t -B -f -i mirror/mirror0 *** Working on device /dev/mirror/mirror0 *** So all the commands seem to work properly. Did you run this under /bin/sh? Yes, I tried it under both tcsh and sh. I didn't take it apart the same way you did however. Also, when size came up as 5 (24MB) I knew something wasn't write and didn't pursue that further. What did the command: fdisk -v -B -I /dev/ad4 do? It would seem my mirror is correct and consistent. Question: On line 27 we issue the command to... # instruct boot stage 2 loader on first disk to boot # with the boot stage 3 loader from the second disk # (mainly because BIOS might not allow easy booting from second ATA disk # or at least requires manual intervention on the console) So how do I get rid of that boot.config file? Should I get rid of it? If you have a modern machine the BIOS (hopefully) can boot from every harddisks that has a partition/slice on it with the active flag set. I for myself use a bootmanager on every gmirror disk. You can install it with boot0cfg(8) or sysinstall (I only would use boot0cfg, if your mirror already has been set up). So I can safely remove /boot.config? Otherwise wouldn't the boot stage 2 loader always then load boot stage 3 off disk 2? Question: On line 25 we set swapoff=YES. Is this necessary if I'm going to run 5-STABLE? What are the implications of having swap on a mirror like this? No, if you run stable you don't need this: http://lists.freebsd.org/pipermail/cvs-all/2005-January/099589.html http://lists.freebsd.org/pipermail/cvs-all/2005-January/099590.html On 5-STABLE systems, younger than the cvsdate of the above commits, the /etc/rc.d/swap1 script runs on system shutdown and ensures that (eventually mirrored) swap devices are stopped before gmirror shuts down. Otherwise the mirror will brake. This is the reason why you need to set swapoff=YES. The recommend method to reboot your gmirrored system is 'shutdown -r now' instead of 'reboot'. Great, thanks for all your help. -- Regards, Doug ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: gmirror problem on 5.3-R i386 (SOLVED)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Saturday 15 January 2005 04:52, Doug Poland wrote: On Sat, Jan 15, 2005 at 12:12:28AM +0100, Christian Hiris wrote: On Friday 14 January 2005 20:43, Doug Poland wrote: Yes, I tried it under both tcsh and sh. I didn't take it apart the same way you did however. Also, when size came up as 5 (24MB) I knew something wasn't write and didn't pursue that further. What did the command: fdisk -v -B -I /dev/ad4 do? It would seem my mirror is correct and consistent. The option -B initializes the bootcode in sector 0, option -I creates one slice that covers the whole disk (man fdisk). Question: On line 27 we issue the command to... # instruct boot stage 2 loader on first disk to boot # with the boot stage 3 loader from the second disk # (mainly because BIOS might not allow easy booting from second ATA disk # or at least requires manual intervention on the console) So how do I get rid of that boot.config file? Should I get rid of it? If you have a modern machine the BIOS (hopefully) can boot from every harddisks that has a partition/slice on it with the active flag set. I for myself use a bootmanager on every gmirror disk. You can install it with boot0cfg(8) or sysinstall (I only would use boot0cfg, if your mirror already has been set up). So I can safely remove /boot.config? Otherwise wouldn't the boot stage 2 loader always then load boot stage 3 off disk 2? Yes, I would remove it. I think, in case, that disk ad6 breaks, the system won't boot w/o manual interaction. In general, it's the best, if you do some real life testing by pulling powercables off the drives. Normally you should be able to replace a disk on a running system. Assuming that disks are connected to different channels: Replace the broken disk by a fresh, clean one, run 'atacontrol attach channel#' (or maybe 'atacontrol reinit channel#) and there you go. It's even possible to swap disks around between controller cards on a running machine. I tested this out on cheap Promise TX2 cards - worked like a charm :) Cheers, ch - -- Christian Hiris [EMAIL PROTECTED] | OpenPGP KeyID 0x3BCA53BE OpenPGP-Key at hkp://wwwkeys.eu.pgp.net and http://pgp.mit.edu -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.6 (FreeBSD) iD8DBQFB6Mbx09WjGjvKU74RAhHuAJ0VW2YuQC4vndOYM+nMQALNs4c/QQCfdOfb wFtdG02YOq+hS5qCzSjugpY= =YShc -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
