Re: gjournal panic 7.0-RC1
Chris wrote: Came back to see box had rebooted itself from a journal related panic. panic: Journal overflow (joffset=49905408 active=499691355136 inactive=4990$ cpuid = 0 AFAIK this means that the journal is too small for your machine - try doubling it until there are no more panics. If so, this is the same class of errors as ZFS (some would call it tuning errors), only this time the space reserved for the on-disk journal is too small, and the fast drives fill it up before data can be transfered from the journal to the data area. signature.asc Description: OpenPGP digital signature
Re: gjournal panic 7.0-RC1
On Sun, Feb 03, 2008 at 09:35:44PM +0100, Ivan Voras wrote: Chris wrote: Came back to see box had rebooted itself from a journal related panic. panic: Journal overflow (joffset=49905408 active=499691355136 inactive=4990$ cpuid = 0 AFAIK this means that the journal is too small for your machine - try doubling it until there are no more panics. If so, this is the same class of errors as ZFS (some would call it tuning errors), only this time the space reserved for the on-disk journal is too small, and the fast drives fill it up before data can be transfered from the journal to the data area. Is there something stopping gjournal from temporarily blocking writes to the journal to allow it to flush the journaled data to the provider? Thanks, Gary ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: gjournal panic 7.0-RC1
Ivan Voras wrote: Chris wrote: Came back to see box had rebooted itself from a journal related panic. panic: Journal overflow (joffset=49905408 active=499691355136 inactive=4990$ cpuid = 0 AFAIK this means that the journal is too small for your machine - try doubling it until there are no more panics. If so, this is the same class of errors as ZFS (some would call it tuning errors), only this time the space reserved for the on-disk journal is too small, and the fast drives fill it up before data can be transfered from the journal to the data area. I did some experimentation with gjournal a few weeks ago to determine how I might partition a new server, as well as how large to make my journals and where. I did find that for the computers I have tested so far, a 1 gig (default size) journal seems to be sufficient, but half of that or less is asking for trouble and I could not find any workarounds to reduce the chances of panic when I was already stuck with a too-small journal I created a while ago. I also found the -s parameter is vague in that it does not say what units it accepts (appears to be bytes) and I *could not* get it to make a journal inside a data partition any bigger than somewhere around 1.7 gigs. Some values of -s seemed to wrap around to a smaller number, while other values gave errors about being too small (when they weren't) or invalid size. The only way I could force a journal size 2G or larger was to make a separate partition for journal. On the server I was setting up, I decided to make my (journaled) data partitions da0s1d,e,f and the journals da0s2d,e,f. I'm just getting this out there to the list because I don't have time to debug it further. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: gjournal panic 7.0-RC1
I did some experimentation with gjournal a few weeks ago to determine how I might partition a new server, as well as how large to make my journals and where. I did find that for the computers I have tested so far, a 1 gig (default size) journal seems to be sufficient, but half of that or less is asking for trouble and I could not find any workarounds to reduce the chances of panic when I was already stuck with a too-small journal I created a while ago. I also found the -s parameter is vague in that it does not say what units it accepts (appears to be bytes) and I *could not* get it to make a journal inside a data partition any bigger than somewhere around 1.7 gigs. Some values of -s seemed to wrap around to a smaller number, while other values gave errors about being too small (when they weren't) or invalid size. The only way I could force a journal size 2G or larger was to make a separate partition for journal. On the server I was setting up, I decided to make my (journaled) data partitions da0s1d,e,f and the journals da0s2d,e,f. I'm just getting this out there to the list because I don't have time to debug it further. thanks for this info I have spent some 8 hours on this today and it seems the documentation for this is somewhat lacking all information required, the server is handling large 50meg files, I have an identical server that hasnt beetouched by gjournal and even after I had setup gjournal properly so no more errors I then found the write speeds were pretty poor, with hd load very high in vmstat. My initial impressions of gjournal is complicated to setup, unstable and slow write speeds however I have not given up yet and may play with it a little more tommorow. The errors that were appearing are these. ad4: FAILURE - WRITE_DMA48 status=51READY,DSC,ERROR error=10NID_NOT_FOUND LBA=662870719 This is not hd failure but occurs when the gjournal was enabled on the initial newfs but has no actual journal location. I stopped these errors after the proper configuration but of course had that single panic. If the only advantage of journaling is to avoid slow fsck's then I may decide I can live without it, the real attraction to me was been able to use the much glamorised async which is what made me so shocked when write speeds were low. Chris ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: gjournal panic 7.0-RC1
AFAIK this means that the journal is too small for your machine - try doubling it until there are no more panics. If so, this is the same class of errors as ZFS (some would call it tuning errors), only this time the space reserved for the on-disk journal is too small, and the fast drives fill it up before data can be transfered from the journal to the data area. To double it is to do another newfs and start from scratch again or can I somehow increase the size without losing the data on the drive? Does a larger journal incease write speeds as I am finding them very poor around 60% of a sync + soft updates drive. Chris ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: gjournal panic 7.0-RC1
Gary Palmer wrote: On Sun, Feb 03, 2008 at 09:35:44PM +0100, Ivan Voras wrote: If so, this is the same class of errors as ZFS (some would call it tuning errors), only this time the space reserved for the on-disk journal is too small, and the fast drives fill it up before data can be transfered from the journal to the data area. Is there something stopping gjournal from temporarily blocking writes to the journal to allow it to flush the journaled data to the provider? I've done something like that in the past, but I don't know if Pawel's gjournal has this implemented. I feel that a good solution could be to somehow pause the file system at the VFS layer not to generate requests that would result in IO writes - this could (in theory...) help with ZFS. signature.asc Description: OpenPGP digital signature
Re: gjournal panic 7.0-RC1
Chris wrote: If the only advantage of journaling is to avoid slow fsck's then I may decide I can live without it, the real attraction to me was been able to use the much glamorised async which is what made me so shocked when write speeds were low. If I understood this thread correctly, the impression of poor performance is based on a configuration where both the journal and the data are on the same physical drive. Intuitively, this will likely penalize any transaction on the volume, read or write, since you're asking the drive to not only accumulate a queue of information to the journal in one region of the disk but also to flush that data in idle time to a region in the data space on that same disk at a significant seek-length away. I would think that journaling on one drive and storing the resultant data-set on another would improve performance enormously (reduced seek-lengths) and more so if they were 1) high-rpm drives (less rotational latency) and 2) on different buses (no bus/controller contention), Michael ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: gjournal panic 7.0-RC1
If I understood this thread correctly, the impression of poor performance is based on a configuration where both the journal and the data are on the same physical drive. Intuitively, this will likely penalize any transaction on the volume, read or write, since you're asking the drive to not only accumulate a queue of information to the journal in one region of the disk but also to flush that data in idle time to a region in the data space on that same disk at a significant seek-length away. I would think that journaling on one drive and storing the resultant data-set on another would improve performance enormously (reduced seek-lengths) and more so if they were 1) high-rpm drives (less rotational latency) and 2) on different buses (no bus/controller contention), Michael Yes I have suspected this, there is 2 physical drives in the machine so this would be possible, if its possible to swap the journals round so they journaling for each other I will give it a go tommorow. They both sata 300 drives. Chris ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: gjournal panic 7.0-RC1
Michael Butler wrote: I would think that journaling on one drive and storing the resultant data-set on another would improve performance enormously (reduced seek-lengths) and more so if they were 1) high-rpm drives (less rotational latency) and 2) on different buses (no bus/controller contention), There are (very near) limits to what you can do with such a setup: the drive that holds the external journal needs to be much faster than the data drive, since it will become the main bottleneck in IO. It has to be faster mostly in sequential IO, seeks are only present when transferring journal data to the main drives while under simultaneous write IO from the file system. Ideally, the journal drive would have to deliver at least twice as sequential IO as the main drive to maximize the potential performance. Thus, using a conveniently small medium as an USB flash drive is not very useful (the high seek rate will remain unused and sequential performance is generally lower than regular drives) I've done some benchmarking. The setup is: three 7.5k RPM drives, two in RAID0, one for the journal. Here's a summary of the results: UFS+SU: bonnie++: writes: 102 MB/s, rewrites: 47 MB/s, reads: 103 MB/s postmark: 110 trans/s UFS+GJ: bonnie++: writes: 35 MB/s, rewrites: 22 MB/s, reads: 99 MB/s postmark: 123 trans/s UFS+GJ-detached: bonnie++: writes: 46 MB/s, rewrites: 36 MB/s, reads: 100 MB/s postmark: 263 trans/s Postmark is configured to have a bias for writing a lot of small files, and benefits a lot from the detached journal. Margins of errors are around +/- 3 MB/s for bonnie++ and around 15 trans/s for postmark. For comparison, here are the results for Linux 2.6.23, regular ext3: bonnie++: writes: 105 MB/s, rewrites: 52 MB/s, reads: 128 MB/s postmark: 173 trans/s signature.asc Description: OpenPGP digital signature
Re: gjournal panic 7.0-RC1
Chris wrote: AFAIK this means that the journal is too small for your machine - try doubling it until there are no more panics. If so, this is the same class of errors as ZFS (some would call it tuning errors), only this time the space reserved for the on-disk journal is too small, and the fast drives fill it up before data can be transfered from the journal to the data area. To double it is to do another newfs and start from scratch again or can I somehow increase the size without losing the data on the drive? If you use an external journal, you can re-create the journal keep the data, if you use an inline journal on the same drive as the file system (the default configuration), you need to re-create both the journal and the file system (newfs). Does a larger journal incease write speeds as I am finding them very poor around 60% of a sync + soft updates drive. No. An external journal could help you to increase the performance. signature.asc Description: OpenPGP digital signature
