Re: Anyone seeing lots of "Check tree block failed" and other errors with latest kernel?

2012-10-29 Thread Richard W.M. Jones
On Thu, Oct 11, 2012 at 07:26:28AM -0400, Chris Mason wrote: > On Thu, Oct 11, 2012 at 01:28:21AM -0600, Richard W.M. Jones wrote: > > Well the bad news is that the bug happened again overnight, even > > though we were definitely using btrfs-progs with the 6eba90029 patch > > added, _and_ it was do

Re: Anyone seeing lots of "Check tree block failed" and other errors with latest kernel?

2012-10-11 Thread Chris Mason
On Thu, Oct 11, 2012 at 01:28:21AM -0600, Richard W.M. Jones wrote: > Well the bad news is that the bug happened again overnight, even > though we were definitely using btrfs-progs with the 6eba90029 patch > added, _and_ it was doing a sync + fsync between the mkfs and the > mount. This is good ju

Re: Anyone seeing lots of "Check tree block failed" and other errors with latest kernel?

2012-10-11 Thread Richard W.M. Jones
Well the bad news is that the bug happened again overnight, even though we were definitely using btrfs-progs with the 6eba90029 patch added, _and_ it was doing a sync + fsync between the mkfs and the mount. Here is the log: modprobe btrfs [ 15.716610] Btrfs loaded grep ^[[:space:]]*btrfs$ /proc

Re: Anyone seeing lots of "Check tree block failed" and other errors with latest kernel?

2012-10-10 Thread Richard W.M. Jones
On Wed, Oct 10, 2012 at 03:41:13PM -0400, Chris Mason wrote: > On Wed, Oct 10, 2012 at 01:38:53PM -0600, Richard W.M. Jones wrote: > > On Wed, Oct 10, 2012 at 08:38:08AM -0400, Chris Mason wrote: > > > On Tue, Oct 09, 2012 at 03:00:12AM -0600, David Sterba wrote: > > > > On Tue, Oct 09, 2012 at 08:

Re: Anyone seeing lots of "Check tree block failed" and other errors with latest kernel?

2012-10-10 Thread Chris Mason
On Wed, Oct 10, 2012 at 01:38:53PM -0600, Richard W.M. Jones wrote: > On Wed, Oct 10, 2012 at 08:38:08AM -0400, Chris Mason wrote: > > On Tue, Oct 09, 2012 at 03:00:12AM -0600, David Sterba wrote: > > > On Tue, Oct 09, 2012 at 08:33:57AM +0100, Richard W.M. Jones wrote: > > > > On Tue, Oct 09, 2012

Re: Anyone seeing lots of "Check tree block failed" and other errors with latest kernel?

2012-10-10 Thread Richard W.M. Jones
On Wed, Oct 10, 2012 at 08:38:08AM -0400, Chris Mason wrote: > On Tue, Oct 09, 2012 at 03:00:12AM -0600, David Sterba wrote: > > On Tue, Oct 09, 2012 at 08:33:57AM +0100, Richard W.M. Jones wrote: > > > On Tue, Oct 09, 2012 at 08:20:02AM +0100, Richard W.M. Jones wrote: > > > > On Mon, Oct 08, 2012

Re: Anyone seeing lots of "Check tree block failed" and other errors with latest kernel?

2012-10-10 Thread Chris Mason
On Tue, Oct 09, 2012 at 03:00:12AM -0600, David Sterba wrote: > On Tue, Oct 09, 2012 at 08:33:57AM +0100, Richard W.M. Jones wrote: > > On Tue, Oct 09, 2012 at 08:20:02AM +0100, Richard W.M. Jones wrote: > > > On Mon, Oct 08, 2012 at 08:00:51PM -0400, Chris Mason wrote: > > > > Ok, what's a rough i

Re: Anyone seeing lots of "Check tree block failed" and other errors with latest kernel?

2012-10-10 Thread Richard W.M. Jones
On Mon, Oct 08, 2012 at 10:22:30PM +0100, Richard W.M. Jones wrote: > Adding sync() + fsync of each /dev/sd* device after the mkfs command > does appear to fix the problem. > > However it's a little bit difficult to know for sure because I might > just be changing the timing of things by adding th

Re: Anyone seeing lots of "Check tree block failed" and other errors with latest kernel?

2012-10-09 Thread Richard W.M. Jones
On Tue, Oct 09, 2012 at 11:16:57AM +0200, David Sterba wrote: > On Tue, Oct 09, 2012 at 08:20:02AM +0100, Richard W.M. Jones wrote: > > On my local machine, I'm reproducing this with what Fedora calls > > 3.7.0-0.rc0.git2.4.fc19.x86_64 (note I found an unrelated but very > > serious bug in this ker

Re: Anyone seeing lots of "Check tree block failed" and other errors with latest kernel?

2012-10-09 Thread David Sterba
On Tue, Oct 09, 2012 at 08:20:02AM +0100, Richard W.M. Jones wrote: > On my local machine, I'm reproducing this with what Fedora calls > 3.7.0-0.rc0.git2.4.fc19.x86_64 (note I found an unrelated but very > serious bug in this kernel: > http://marc.info/?l=linux-kernel&m=134973394826408&w=2 ) And i

Re: Anyone seeing lots of "Check tree block failed" and other errors with latest kernel?

2012-10-09 Thread David Sterba
On Tue, Oct 09, 2012 at 08:33:57AM +0100, Richard W.M. Jones wrote: > On Tue, Oct 09, 2012 at 08:20:02AM +0100, Richard W.M. Jones wrote: > > On Mon, Oct 08, 2012 at 08:00:51PM -0400, Chris Mason wrote: > > > Ok, what's a rough idea of the mainline git equiv of the buggy kernel? > > > > On my loca

Re: Anyone seeing lots of "Check tree block failed" and other errors with latest kernel?

2012-10-09 Thread Richard W.M. Jones
On Tue, Oct 09, 2012 at 08:20:02AM +0100, Richard W.M. Jones wrote: > On Mon, Oct 08, 2012 at 08:00:51PM -0400, Chris Mason wrote: > > On Mon, Oct 08, 2012 at 03:22:30PM -0600, Richard W.M. Jones wrote: > > > > > > I have now reproduced this bug locally. > > > > > > Adding sync() + fsync of each

Re: Anyone seeing lots of "Check tree block failed" and other errors with latest kernel?

2012-10-09 Thread Richard W.M. Jones
On Mon, Oct 08, 2012 at 08:00:51PM -0400, Chris Mason wrote: > On Mon, Oct 08, 2012 at 03:22:30PM -0600, Richard W.M. Jones wrote: > > > > I have now reproduced this bug locally. > > > > Adding sync() + fsync of each /dev/sd* device after the mkfs command > > does appear to fix the problem. > >

Re: Anyone seeing lots of "Check tree block failed" and other errors with latest kernel?

2012-10-08 Thread Chris Mason
On Mon, Oct 08, 2012 at 03:22:30PM -0600, Richard W.M. Jones wrote: > > I have now reproduced this bug locally. > > Adding sync() + fsync of each /dev/sd* device after the mkfs command > does appear to fix the problem. > > However it's a little bit difficult to know for sure because I might > ju

Re: Anyone seeing lots of "Check tree block failed" and other errors with latest kernel?

2012-10-08 Thread Richard W.M. Jones
On Mon, Oct 08, 2012 at 04:15:14PM +0100, Richard W.M. Jones wrote: > On Mon, Oct 08, 2012 at 11:04:19AM -0400, Chris Mason wrote: > > On Mon, Oct 08, 2012 at 08:57:30AM -0600, Richard W.M. Jones wrote: > > > On Mon, Oct 08, 2012 at 10:27:57AM -0400, Chris Mason wrote: > > > > On Mon, Oct 08, 2012

Re: Anyone seeing lots of "Check tree block failed" and other errors with latest kernel?

2012-10-08 Thread Richard W.M. Jones
On Mon, Oct 08, 2012 at 06:42:27PM +0200, David Sterba wrote: > On Mon, Oct 08, 2012 at 04:15:14PM +0100, Richard W.M. Jones wrote: > > Unfortunately I'm struggling to reproduce this outside of our build > > system (Koji). I will keep you informed if I do manage to reproduce > > it locally. Addin

Re: Anyone seeing lots of "Check tree block failed" and other errors with latest kernel?

2012-10-08 Thread David Sterba
On Mon, Oct 08, 2012 at 04:15:14PM +0100, Richard W.M. Jones wrote: > Unfortunately I'm struggling to reproduce this outside of our build > system (Koji). I will keep you informed if I do manage to reproduce > it locally. Adding fsync /dev/sda1 was also my first instinct :-) Have you updated the

Re: Anyone seeing lots of "Check tree block failed" and other errors with latest kernel?

2012-10-08 Thread Chris Mason
On Mon, Oct 08, 2012 at 09:15:14AM -0600, Richard W.M. Jones wrote: > On Mon, Oct 08, 2012 at 11:04:19AM -0400, Chris Mason wrote: > > On Mon, Oct 08, 2012 at 08:57:30AM -0600, Richard W.M. Jones wrote: > > > On Mon, Oct 08, 2012 at 10:27:57AM -0400, Chris Mason wrote: > > > > On Mon, Oct 08, 2012

Re: Anyone seeing lots of "Check tree block failed" and other errors with latest kernel?

2012-10-08 Thread Richard W.M. Jones
On Mon, Oct 08, 2012 at 11:04:19AM -0400, Chris Mason wrote: > On Mon, Oct 08, 2012 at 08:57:30AM -0600, Richard W.M. Jones wrote: > > On Mon, Oct 08, 2012 at 10:27:57AM -0400, Chris Mason wrote: > > > On Mon, Oct 08, 2012 at 08:16:42AM -0600, Richard W.M. Jones wrote: > > > > > > > > I'm tracking

Re: Anyone seeing lots of "Check tree block failed" and other errors with latest kernel?

2012-10-08 Thread Chris Mason
On Mon, Oct 08, 2012 at 08:57:30AM -0600, Richard W.M. Jones wrote: > On Mon, Oct 08, 2012 at 10:27:57AM -0400, Chris Mason wrote: > > On Mon, Oct 08, 2012 at 08:16:42AM -0600, Richard W.M. Jones wrote: > > > > > > I'm tracking this bug here: > > > > > > https://bugzilla.redhat.com/show_bug.cgi?i

Re: Anyone seeing lots of "Check tree block failed" and other errors with latest kernel?

2012-10-08 Thread Richard W.M. Jones
On Mon, Oct 08, 2012 at 10:27:57AM -0400, Chris Mason wrote: > On Mon, Oct 08, 2012 at 08:16:42AM -0600, Richard W.M. Jones wrote: > > > > I'm tracking this bug here: > > > > https://bugzilla.redhat.com/show_bug.cgi?id=863978 > > > > Since approx. last week I'm seeing lots of failures in btrfs.

Re: Anyone seeing lots of "Check tree block failed" and other errors with latest kernel?

2012-10-08 Thread Chris Mason
On Mon, Oct 08, 2012 at 08:16:42AM -0600, Richard W.M. Jones wrote: > > I'm tracking this bug here: > > https://bugzilla.redhat.com/show_bug.cgi?id=863978 > > Since approx. last week I'm seeing lots of failures in btrfs. The > common factor seems to be that the filesystem is created (mkfs.btrfs

Anyone seeing lots of "Check tree block failed" and other errors with latest kernel?

2012-10-08 Thread Richard W.M. Jones
I'm tracking this bug here: https://bugzilla.redhat.com/show_bug.cgi?id=863978 Since approx. last week I'm seeing lots of failures in btrfs. The common factor seems to be that the filesystem is created (mkfs.btrfs /dev/sda1) and then it is immediately used -- eg. mounted or some btrfs subtool