Re: good/best practices for gmirror and gjournal on a pair of disks?
George Hartzell wrote: Adam McDougall writes: > George Hartzell wrote: > >[...] > > - I've read in the gjournal man page that when it is "... configured > > on top of gmirror(8) or graid3(8) providers, it also keeps them in > > a consistent state..." I've been trying to figure out if this > > simply falls out of how gjournal works or if there's explicity > > collusion with gmirror/graid3 but can't come up with a > > satisfactory explanation. Can someone walk me through it? > > > > Since I'm only gjournal'ing a portion of the underlying gmirror > > device I assume that I don't get this benefit? > >[...] > [...] > I decided to journal /usr /var /tmp and leave / as a standard UFS > partition because it is so small, fsck doesn't take long anyway and > hopefully doesn't get written to enough to cause damage by an abrupt > reboot. Because I'm not journaling the root partition, I chose to > ignore the possibility of gjournal marking the mirror clean. Sudden > reboots don't happen enough on servers for me to care. And all my > servers got abruptly rebooted this sunday and they all came up fine :) > [...] So you're confirming my belief that setting up gjournal on a bsdlabel'ed partition of a gmirror does *not* provide the consistency guarantee and that I should leave autosynchronization enabled. Right? g. I forgot to address that. I think to gain that, you have to (re)label the mirror using -F (see man gmirror). I believe without using -F, the mirrors will still be synced (but probably don't need to). Otherwise, look for initial mail list announcements (freebsd-current?) of gjournal which may explain. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: good/best practices for gmirror and gjournal on a pair of disks?
Adam McDougall writes: > George Hartzell wrote: > >[...] > > - I've read in the gjournal man page that when it is "... configured > > on top of gmirror(8) or graid3(8) providers, it also keeps them in > > a consistent state..." I've been trying to figure out if this > > simply falls out of how gjournal works or if there's explicity > > collusion with gmirror/graid3 but can't come up with a > > satisfactory explanation. Can someone walk me through it? > > > > Since I'm only gjournal'ing a portion of the underlying gmirror > > device I assume that I don't get this benefit? > >[...] > [...] > I decided to journal /usr /var /tmp and leave / as a standard UFS > partition because it is so small, fsck doesn't take long anyway and > hopefully doesn't get written to enough to cause damage by an abrupt > reboot. Because I'm not journaling the root partition, I chose to > ignore the possibility of gjournal marking the mirror clean. Sudden > reboots don't happen enough on servers for me to care. And all my > servers got abruptly rebooted this sunday and they all came up fine :) > [...] So you're confirming my belief that setting up gjournal on a bsdlabel'ed partition of a gmirror does *not* provide the consistency guarantee and that I should leave autosynchronization enabled. Right? g. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: good/best practices for gmirror and gjournal on a pair of disks?
Adam McDougall writes: > [...] > I believe gjournal uses 1G for journal (2x512) which seemed to be > sufficient on all of the systems where I have used the default, but I > quickly found that using a smaller journal is a bad idea and leads to > panics that I was unable to avoid with tuning. Considering 1G was such > a close value, I chose to go several times above the default journal > size (disk is cheap and I want to be sure) but I ran into problems using > gjournal label -s (size) rejecting my sizes or wrapping the value around > to something too low. [...] I also stumbled on this and was unable to find any mention of it in the pr database. One of my todo items is to make sure I'm not messing up somehow, dig further into the PR db for an existing report, and file one if I can't find one? I tried -s 2147483648 and it was found to be "too small". A quick read of the source led me to find that jsize is an intmax_t and that gctl_get_intmax() should be returning an intmax_t and that intmax_ ought to be an __int64_t (I'm on amd64), which left me confused. Has anyone else seen/reported a problem with gjournal -s and values > 1G? g. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: good/best practices for gmirror and gjournal on a pair of disks?
George Hartzell wrote: I've been running many of my systems for some time now using gmirror on a pair of identical disks, as described by Ralf at: http://people.freebsd.org/~rse/mirror/ Each disk has single slice that covers almost all of the disk. These slices are combined into the gmirror device (gm0), which is then carved up by bsdlabel into gm0a (/), gm0b (swap), gm0d (/var), gm0e (/tmp), and gm0f (/usr). My latest machine is using Seagate 1TB disks so I thought I should add gjournal to the mix to avoid ugly fsck's if/when the machine doesn't shut down cleanly. I ended up just creating a gm0f.journal and using it for /usr, which basically seems to be working. I'm left with a couple of questions though: - I've read in the gjournal man page that when it is "... configured on top of gmirror(8) or graid3(8) providers, it also keeps them in a consistent state..." I've been trying to figure out if this simply falls out of how gjournal works or if there's explicity collusion with gmirror/graid3 but can't come up with a satisfactory explanation. Can someone walk me through it? Since I'm only gjournal'ing a portion of the underlying gmirror device I assume that I don't get this benefit? - I've also read in the gjournal man page "... that sync(2) and fsync(2) system calls do not work as expected anymore." Does this invalidate any of the assumptions made by various database packages such as postgresql, sqlite, berkeley db, etc about if/when/whether their data is safely on the disk? - What's the cleanest gjournal adaptation of rse's two-disk-mirror-everything setup that would be able to avoid tedious gmirror sync's. The best I've come up with is to do two slices per disk, combine the slices into a pair of gmirror devices, bsdlabel the first into gm0a (/), gm0b (swap), gm0d (/var) and gm0e (/tmp) and bsdlabel the second into a gm1f which gets a gjournal device. Alternatively, would it work and/or make sense to give each disk a single slice, combine them into a gmirror, put a gjournal on top of that, then use bsdlabel to slice it up into partitions? Is anyone using gjournal and gmirror for all of the system on a pair of disks in some other configuration? Thanks, g. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]" I am pasting below the instructions I would use to convert a recently installed system with only / (root) and swap to be using gmirror+gjournal. It is in mediawiki markup format so it could be pasted into one if desired. I based my gmirror steps on the instructions from http://people.freebsd.org/~rse/mirror/ so thats why some of the words sound familiar. I also have similar instructions for setting up a gmirrored da0s1a and da0s1b alongside a zfs mirror containing the rest. I decided to journal /usr /var /tmp and leave / as a standard UFS partition because it is so small, fsck doesn't take long anyway and hopefully doesn't get written to enough to cause damage by an abrupt reboot. Because I'm not journaling the root partition, I chose to ignore the possibility of gjournal marking the mirror clean. Sudden reboots don't happen enough on servers for me to care. And all my servers got abruptly rebooted this sunday and they all came up fine :) I believe gjournal uses 1G for journal (2x512) which seemed to be sufficient on all of the systems where I have used the default, but I quickly found that using a smaller journal is a bad idea and leads to panics that I was unable to avoid with tuning. Considering 1G was such a close value, I chose to go several times above the default journal size (disk is cheap and I want to be sure) but I ran into problems using gjournal label -s (size) rejecting my sizes or wrapping the value around to something too low. As a workaround I chose to use a separate partition for each journal. I quickly ran out of partitions in a bsd disklabel so I decided to partition each disk into two slices; the first for data and the second for journals. This also made it easier to line up disk devices so they made more sense as a pair, for example: gm0s1d(data) + gm0s2d(journal) = /usr. I will note that if you accidentally put a gjournal label in the 'wrong' spot on your disk, you might make a tough situation for yourself getting rid of it. I have had plenty of times where I applied a gjournal label, discovered something unideal with it, but every time I did 'gjournal stop foo' the label would automatically get detected as a child of a different part of the disk because it could be seen and I could not unload it. That is part of why I use -h for gjournal label, and use slices+partitions, and the first partition is at offset 16, some of which may have been for gmirror's sake too. ==Softwa
good/best practices for gmirror and gjournal on a pair of disks?
I've been running many of my systems for some time now using gmirror on a pair of identical disks, as described by Ralf at: http://people.freebsd.org/~rse/mirror/ Each disk has single slice that covers almost all of the disk. These slices are combined into the gmirror device (gm0), which is then carved up by bsdlabel into gm0a (/), gm0b (swap), gm0d (/var), gm0e (/tmp), and gm0f (/usr). My latest machine is using Seagate 1TB disks so I thought I should add gjournal to the mix to avoid ugly fsck's if/when the machine doesn't shut down cleanly. I ended up just creating a gm0f.journal and using it for /usr, which basically seems to be working. I'm left with a couple of questions though: - I've read in the gjournal man page that when it is "... configured on top of gmirror(8) or graid3(8) providers, it also keeps them in a consistent state..." I've been trying to figure out if this simply falls out of how gjournal works or if there's explicity collusion with gmirror/graid3 but can't come up with a satisfactory explanation. Can someone walk me through it? Since I'm only gjournal'ing a portion of the underlying gmirror device I assume that I don't get this benefit? - I've also read in the gjournal man page "... that sync(2) and fsync(2) system calls do not work as expected anymore." Does this invalidate any of the assumptions made by various database packages such as postgresql, sqlite, berkeley db, etc about if/when/whether their data is safely on the disk? - What's the cleanest gjournal adaptation of rse's two-disk-mirror-everything setup that would be able to avoid tedious gmirror sync's. The best I've come up with is to do two slices per disk, combine the slices into a pair of gmirror devices, bsdlabel the first into gm0a (/), gm0b (swap), gm0d (/var) and gm0e (/tmp) and bsdlabel the second into a gm1f which gets a gjournal device. Alternatively, would it work and/or make sense to give each disk a single slice, combine them into a gmirror, put a gjournal on top of that, then use bsdlabel to slice it up into partitions? Is anyone using gjournal and gmirror for all of the system on a pair of disks in some other configuration? Thanks, g. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
