Re: gmirror per partition. Was: Re: gmirror gm0 destroyed on shutdown; GPT corrupt
# gmirror label -vb round-robin root /dev/da0p2 gmirror: Can't store metadata on /dev/da0p2: Operation not permitted. isn't that partition accessed by other process or mounted? should it not be mounted? yes it should not, no matter what architecture. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: gmirror per partition. Was: Re: gmirror gm0 destroyed on shutdown; GPT corrupt
On Wed, Jul 01, 2009 at 10:00:54PM +0200, Wojciech Puchar wrote: > >> It's better to use gmirror per partition. > > > > Like this? > > > > # gmirror label -vb round-robin root /dev/da0p2 > > gmirror: Can't store metadata on /dev/da0p2: Operation not permitted. > isn't that partition accessed by other process or mounted? should it not be mounted? Sorry, I was just following the handbook, but I now understand it is incorrect when it comes to ia64. many thanks anton > ___ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org" -- Anton Shterenlikht Room 2.6, Queen's Building Mech Eng Dept Bristol University University Walk, Bristol BS8 1TR, UK Tel: +44 (0)117 928 8233 Fax: +44 (0)117 929 4423 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: gmirror per partition. Was: Re: gmirror gm0 destroyed on shutdown; GPT corrupt
It's better to use gmirror per partition. Like this? # gmirror label -vb round-robin root /dev/da0p2 gmirror: Can't store metadata on /dev/da0p2: Operation not permitted. isn't that partition accessed by other process or mounted? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
gmirror per partition. Was: Re: gmirror gm0 destroyed on shutdown; GPT corrupt
On Thu, Jun 25, 2009 at 09:41:13AM -0700, Marcel Moolenaar wrote: > > On Jun 25, 2009, at 4:02 AM, Anton Shterenlikht wrote: > > dev_taste(DEV,mirror/gm0) > > g_part_taste(PART,mirror/gm0) > > > > GEOM: mirror/gm0: the secondary GPT table is corrupt or invalid. > > GEOM: mirror/gm0: using the primary only -- recovery suggested. > > ^^^ > > You created the mirror after the GPT, which means you destroyed > the GPT backup header. gmirror uses the last sector on the disk > for metadata and that by itself is a cause for various problems. > > It's better to use gmirror per partition. Like this? # gmirror label -vb round-robin root /dev/da0p2 gmirror: Can't store metadata on /dev/da0p2: Operation not permitted. # I've read some boot disk gmirror examples, e.g. http://people.freebsd.org/~rse/mirror however, all examples I've seen are for i386, talking about MBR, fdisk and bsdlabel, so these are not directly applicable to ia64. Application of gvinum for boot disk on ia64 is not clear either. It seems gvinum section of the handbook, 21.9, is also based on i386. Please advise many thanks anton -- Anton Shterenlikht Room 2.6, Queen's Building Mech Eng Dept Bristol University University Walk, Bristol BS8 1TR, UK Tel: +44 (0)117 928 8233 Fax: +44 (0)117 929 4423 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: gmirror gm0 destroyed on shutdown; GPT corrupt
Hi, On 28.06.2009, at 10:49, Pawel Jakub Dawidek wrote: I for one never put mirror on already partitioned disk. Although it is sometimes safe to use the last sector. Gjournal already looks for UFS and if UFS is in place, it figures out if the last sector is in use - it isn't if partition size is not multiple of UFS block size. does this actually also mean that gmirror used on a partition (eg mirroring two partitions of two different disks) is not recommended and is going to write its metadata always on the last sector of the disk, instead of the last sector of the partition? regards, Lorenzo ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: gmirror gm0 destroyed on shutdown; GPT corrupt
On Thu, Jun 25, 2009 at 09:41:13AM -0700, Marcel Moolenaar wrote: > > On Jun 25, 2009, at 4:02 AM, Anton Shterenlikht wrote: > > dev_taste(DEV,mirror/gm0) > > g_part_taste(PART,mirror/gm0) > > > > GEOM: mirror/gm0: the secondary GPT table is corrupt or invalid. > > GEOM: mirror/gm0: using the primary only -- recovery suggested. > > ^^^ > > You created the mirror after the GPT, which means you destroyed > the GPT backup header. gmirror uses the last sector on the disk > for metadata and that by itself is a cause for various problems. So, gmirror cannot be used on ia64 to mirror the boot disk? Because on ia64 the last sector always contains secondary GPT. I take it the RAID1 section, 19.4, in FBSD user manual, was written with i386 or alpha architecture in mind. > It's better to use gmirror per partition. how? Is it in the manual? any link? > > #echo 'geom_mirror_load="YES"' >> /boot/loader.conf > > Is /boot a symlink for /efi/boot? yes, lrwxr-xr-x 1 root wheel 8 Jun 25 10:44 boot -> efi/boot > > And when the system is rebooted, there is no /dev/mirror anymore. > > You could run into a race condition between GPT and gmirror and > GPT winning (again the result of gmirror using the last sector > on a disk for metadata). > > Alternatively, make sure gmirror got loaded at boot. # kldstat Id Refs AddressSize Name 13 0xe400 ff9c08 kernel 21 0xe4ffa000 3c830geom_mirror.ko # It's not that I desperately need to mirror a boot disk, it just that gmirror looked so easy in the manual, I wanted to give it a go. Perhaps I can just do a block copy to the second disk, say once a day, and have it as a backup. Could you also possibly comment on gvinum on ia64? many thanks anton -- Anton Shterenlikht Room 2.6, Queen's Building Mech Eng Dept Bristol University University Walk, Bristol BS8 1TR, UK Tel: +44 (0)117 928 8233 Fax: +44 (0)117 929 4423 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: gmirror gm0 destroyed on shutdown; GPT corrupt
Ivan Voras wrote: Yes, it would be cleaner to implement but it would also make the mirrored devices unbootable. But maybe the class of users needing the functionality is smaller now. Most dedicated server providers can't afford to use hardware RAID systems because that would drastically increase the price of a single system; yet many customers want mirroring. Looks too complicated and fragile. Maybe there's a need for metadata-less automatic mirrors in some way, by storing the configuration somewhere else, possibly in /etc. This might be dangerous in some cases. Imagine booting with two drives swapped; such a configuration might lead to data corruption on a volume which was enumerated incorrectly or swapped. -- Kamigishi Rei KREI-RIPE ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: gmirror gm0 destroyed on shutdown; GPT corrupt
2009/6/28 Marcel Moolenaar : > Using the last sector is not only flawed because it creates a race > condition, it's flawed in the assumption that you can always make > a geom part of a mirror by storing meta-data on the geom without > causing corruption. This whole idea of using the last sector was > so that a fully partitioned disk with data could be turned into a > mirrored disk. A neat idea, but hardly the basis for a generic > mirroring implementation when it silently corrupts a disk. > > I think it's better to change gmirror to use the first sector on the > provider. Yes, it would be cleaner to implement but it would also make the mirrored devices unbootable. But maybe the class of users needing the functionality is smaller now. > This never creates a race condition and as such, you don't > need to invent a priority scheme, that has it's own set of flaws on > top of it. The only downside is that it's not easy to make a fully > partitioned and populated disk part of a mirror: one would need to > move the data forward one sector to free the first sector. This we > can actually do by inserting a GEOM that does it while I/O is still > ongoing. The good thing is: we need a class that does exactly this > for implementing the "move" verb in gpart. Looks too complicated and fragile. Maybe there's a need for metadata-less automatic mirrors in some way, by storing the configuration somewhere else, possibly in /etc. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: gmirror gm0 destroyed on shutdown; GPT corrupt
On Sat, Jun 27, 2009 at 06:20:49PM -0700, Marcel Moolenaar wrote: > Using the last sector is not only flawed because it creates a race > condition, it's flawed in the assumption that you can always make > a geom part of a mirror by storing meta-data on the geom without > causing corruption. This whole idea of using the last sector was > so that a fully partitioned disk with data could be turned into a > mirrored disk. A neat idea, but hardly the basis for a generic > mirroring implementation when it silently corrupts a disk. This wasn't the idea:) People started putting gmirror on top of partitioned disk, because it was easier/simpler/faster than creating mirror, partitioning and copying the data. I for one never put mirror on already partitioned disk. Although it is sometimes safe to use the last sector. Gjournal already looks for UFS and if UFS is in place, it figures out if the last sector is in use - it isn't if partition size is not multiple of UFS block size. > I think it's better to change gmirror to use the first sector on the > provider. This never creates a race condition and as such, you don't > need to invent a priority scheme, that has it's own set of flaws on > top of it. The only downside is that it's not easy to make a fully > partitioned and populated disk part of a mirror: one would need to > move the data forward one sector to free the first sector. This we > can actually do by inserting a GEOM that does it while I/O is still > ongoing. The good thing is: we need a class that does exactly this > for implementing the "move" verb in gpart. There were two reasons to use the last sector instead of first: 1. You want to be able to boot from gmirror. If all your data will be moved forward your boot sectors and kernel will be harder to find. 2. For recovery reasons you may want to turn off gmirror and still be able to access your data. Note that gmirror can handle the case where disk, slice and partition share the same last sector - it simply stores provider size in its metadata, so once it gets disk for tasting it detects its too big and ignores it, then slice will be given for tasting, but it also has larger size than expected and will be ignored as well. Finally partition will be tasted and gmirror configured. -- Pawel Jakub Dawidek http://www.wheel.pl p...@freebsd.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! pgpXtFT4O58hK.pgp Description: PGP signature
Re: gmirror gm0 destroyed on shutdown; GPT corrupt
On Jun 27, 2009, at 4:15 AM, Ivan Voras wrote: Marcel Moolenaar wrote: On Jun 25, 2009, at 4:02 AM, Anton Shterenlikht wrote: dev_taste(DEV,mirror/gm0) g_part_taste(PART,mirror/gm0) GEOM: mirror/gm0: the secondary GPT table is corrupt or invalid. GEOM: mirror/gm0: using the primary only -- recovery suggested. ^^^ You created the mirror after the GPT, which means you destroyed the GPT backup header. gmirror uses the last sector on the disk for metadata and that by itself is a cause for various problems. It's better to use gmirror per partition. Or create the GPT partition inside the gmirror device - then the GPT backup table will be at last_sector-1, but... You could run into a race condition between GPT and gmirror and GPT winning (again the result of gmirror using the last sector on a disk for metadata). unfortunately this could still happen, and will lead to the same error if GPT is tasted first, since it is embedded in the first sector and will assume the whole drive is available to GPT, and will then proceed to not find its backup data in the last sector. It looks to me like GEOM classes should have a "priority" field for tasting. Any objections to that idea? Using the last sector is not only flawed because it creates a race condition, it's flawed in the assumption that you can always make a geom part of a mirror by storing meta-data on the geom without causing corruption. This whole idea of using the last sector was so that a fully partitioned disk with data could be turned into a mirrored disk. A neat idea, but hardly the basis for a generic mirroring implementation when it silently corrupts a disk. I think it's better to change gmirror to use the first sector on the provider. This never creates a race condition and as such, you don't need to invent a priority scheme, that has it's own set of flaws on top of it. The only downside is that it's not easy to make a fully partitioned and populated disk part of a mirror: one would need to move the data forward one sector to free the first sector. This we can actually do by inserting a GEOM that does it while I/O is still ongoing. The good thing is: we need a class that does exactly this for implementing the "move" verb in gpart. In other words: Solving the problem that putting the metadata in the first sector creates, can and will be re-used in implementing the gpart "move partition" feature. I doubt anyone will complain that the creation of a mirror brings with it a few hours of disk activity that does not inhibit normal operation... -- Marcel Moolenaar xcl...@mac.com ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: gmirror gm0 destroyed on shutdown; GPT corrupt
On Jun 25, 2009, at 4:02 AM, Anton Shterenlikht wrote: dev_taste(DEV,mirror/gm0) g_part_taste(PART,mirror/gm0) GEOM: mirror/gm0: the secondary GPT table is corrupt or invalid. GEOM: mirror/gm0: using the primary only -- recovery suggested. ^^^ You created the mirror after the GPT, which means you destroyed the GPT backup header. gmirror uses the last sector on the disk for metadata and that by itself is a cause for various problems. It's better to use gmirror per partition. #echo 'geom_mirror_load="YES"' >> /boot/loader.conf Is /boot a symlink for /efi/boot? GEOM_MIRROR: Device gm0 destroyed. ^ This is normal. And when the system is rebooted, there is no /dev/mirror anymore. You could run into a race condition between GPT and gmirror and GPT winning (again the result of gmirror using the last sector on a disk for metadata). Alternatively, make sure gmirror got loaded at boot. FYI, -- Marcel Moolenaar xcl...@mac.com ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
gmirror gm0 destroyed on shutdown; GPT corrupt
FreeBSD 8.0-CURRENT-200906 ia64, fresh installation Following the handbook, section 19.1 RAID1 - mirroring, I'm trying to use gmirror with 2 identical scsi disks: da0 at mpt0 bus 0 target 0 lun 0 da0: Fixed Direct Access SCSI-3 device da0: 160.000MB/s transfers (80.000MHz, offset 63, 16bit) da0: Command Queueing Enabled da0: 17366MB (35566478 512 byte sectors: 255H 63S/T 2213C) da1 at mpt0 bus 0 target 1 lun 0 da1: Fixed Direct Access SCSI-3 device da1: 160.000MB/s transfers (80.000MHz, offset 63, 16bit) da1: Command Queueing Enabled da1: 17366MB (35566478 512 byte sectors: 255H 63S/T 2213C) # sysctl kern.geom.debugflags=17 kern.geom.debugflags: 0 -> 17 # gmirror label -vb round-robin gm0 /dev/da0 Metadata value stored on /dev/da0. Done. # gmirror load #g_modevent(MIRROR, LOAD) g_post_event_x(0xe4b8eb10, 0xe00010686e40, 2, 262144) g_load_class(MIRROR) g_mirror_taste(MIRROR, acd0t01) acd0: FAILURE - READ_BIG MEDIUM ERROR asc=0x02 ascq=0x00 g_detach(0xe000106eb580) g_destroy_consumer(0xe000106eb580) g_destroy_geom(0xe00010724800(mirror:taste)) g_mirror_taste(MIRROR, acd0) acd0: FAILURE - READ_BIG MEDIUM ERROR asc=0x02 ascq=0x00 g_detach(0xe000106eb700) g_destroy_consumer(0xe000106eb700) g_destroy_geom(0xe00010724a00(mirror:taste)) g_mirror_taste(MIRROR, da2p6) g_detach(0xe000106eb880) g_destroy_consumer(0xe000106eb880) g_destroy_geom(0xe00010724c00(mirror:taste)) g_mirror_taste(MIRROR, da2p5) g_detach(0xe000106eba80) g_destroy_consumer(0xe000106eba80) g_destroy_geom(0xe00010724c00(mirror:taste)) g_mirror_taste(MIRROR, da2p4) g_detach(0xe000106ebc00) g_destroy_consumer(0xe000106ebc00) g_destroy_geom(0xe00010630700(mirror:taste)) g_mirror_taste(MIRROR, da2p3) g_detach(0xe00010738000) g_destroy_consumer(0xe00010738000) g_destroy_geom(0xe00010724e00(mirror:taste)) g_mirror_taste(MIRROR, da2p2) g_detach(0xe00010738180) g_destroy_consumer(0xe00010738180) g_destroy_geom(0xe0001075f400(mirror:taste)) g_mirror_taste(MIRROR, da2p1) g_detach(0xe00010620e80) g_destroy_consumer(0xe00010620e80) g_destroy_geom(0xe00010724a00(mirror:taste)) g_mirror_taste(MIRROR, da0p6) g_detach(0xe000106e9580) g_destroy_consumer(0xe000106e9580) g_destroy_geom(0xe00010724800(mirror:taste)) g_mirror_taste(MIRROR, da0p5) g_detach(0xe00010739700) g_destroy_consumer(0xe00010739700) g_destroy_geom(0xe000108f8f00(mirror:taste)) g_mirror_taste(MIRROR, da0p4) g_detach(0xe00010739680) g_destroy_consumer(0xe00010739680) g_destroy_geom(0xe000108f8d00(mirror:taste)) g_mirror_taste(MIRROR, da0p3) g_detach(0xe00010739800) g_destroy_consumer(0xe00010739800) g_destroy_geom(0xe0001072ce00(mirror:taste)) g_mirror_taste(MIRROR, da0p2) g_detach(0xe00010739780) g_destroy_consumer(0xe00010739780) g_destroy_geom(0xe000108f8700(mirror:taste)) g_mirror_taste(MIRROR, da0p1) g_detach(0xe000106e9680) g_destroy_consumer(0xe000106e9680) g_destroy_geom(0xe0001072cb00(mirror:taste)) g_mirror_taste(MIRROR, da2) g_detach(0xe000106a4900) g_destroy_consumer(0xe000106a4900) g_destroy_geom(0xe0001075fa00(mirror:taste)) g_mirror_taste(MIRROR, da1) g_detach(0xe000106ebc80) g_destroy_consumer(0xe000106ebc80) g_destroy_geom(0xe00010630b00(mirror:taste)) g_mirror_taste(MIRROR, da0) g_detach(0xe000106e8400) g_destroy_consumer(0xe000106e8400) g_destroy_geom(0xe0001072c800(mirror:taste)) g_post_event_x(0xe4b861c0, 0xe000108f9000, 2, 0) ref 0xe000108f9000 ref 0xe00010760800 GEOM_MIRROR: Device mirror/gm0 launched (1/1). g_mirror_taste(MIRROR, ufsid/4a3fa1b76cb317b5) g_detach(0xe000106a4780) g_destroy_consumer(0xe000106a4780) g_destroy_geom(0xe000108fa300(mirror:taste)) g_mirror_taste(MIRROR, ufsid/4a3fa1b69c522d30) g_detach(0xe000106e9900) g_destroy_consumer(0xe000106e9900) g_destroy_geom(0xe000108fac00(mirror:taste)) g_mirror_taste(MIRROR, ufsid/4a3fa1b751514347) g_detach(0xe000106ebb00) g_destroy_consumer(0xe000106ebb00) g_destroy_geom(0xe0001075f300(mirror:taste)) g_mirror_taste(MIRROR, ufsid/4a3fa1b5e5003da2) g_detach(0xe000106e9880) g_destroy_consumer(0xe000106e9880) g_destroy_geom(0xe00010763600(mirror:taste)) g_mirror_taste(MIRROR, iso9660/FreeBSD_Install) acd0: FAILURE - READ_BIG MEDIUM ERROR asc=0x02 ascq=0x00 g_detach(0xe00010738080) g_destroy_consumer(0xe00010738080) g_destroy_geom(0xe0001081dd00(mirror:taste)) g_mirror_taste(MIRROR, mirror/gm0) g_detach(0xe000106e9980) g_destroy_consumer(0xe000106e9980) g_destroy_geom(0xe00010763600(mirror:taste)) dev_taste(DEV,mirror/gm0) g_part_taste(PART,mirror/gm0) GEOM: mirror/gm0: the secondary GPT table is corrupt or invalid. GEOM: mirror/gm0: using the primary only -- recovery suggested. ^^^ g_post_event_x(0xe4b861c0, 0xe000108fa400, 2, 0) ref 0
