Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-12-06 Thread Moffett, Kyle D
Hello again! I know it's been ages, but I finally got some time to get that patch tested out and try additional debugging. On Sep 01, 2011, at 11:17, Jan Kara wrote: On Tue 30-08-11 19:26:22, Moffett, Kyle D wrote: On Aug 30, 2011, at 18:12, Jan Kara wrote: I can still trigger it on my VM

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-09-01 Thread Jan Kara
On Tue 30-08-11 19:26:22, Moffett, Kyle D wrote: On Aug 30, 2011, at 18:12, Jan Kara wrote: I can still trigger it on my VM snapshot very easily, so if you have anything you think I should test I would be very happy to give it a shot. OK, so in the meantime I found a bug in

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-08-30 Thread Jan Kara
Hi, On Fri 26-08-11 16:03:32, Moffett, Kyle D wrote: Ping? Any more ideas for debugging this issue? Sorry for not getting to you earlier. I can still trigger it on my VM snapshot very easily, so if you have anything you think I should test I would be very happy to give it a shot. OK,

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-08-30 Thread Moffett, Kyle D
On Aug 30, 2011, at 18:12, Jan Kara wrote: On Fri 26-08-11 16:03:32, Moffett, Kyle D wrote: Ping? Any more ideas for debugging this issue? Sorry for not getting to you earlier. That's ok, I have a workaround so it's been on my back burner for a while. I can still trigger it on my VM

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-08-26 Thread Moffett, Kyle D
Ping? Any more ideas for debugging this issue? I can still trigger it on my VM snapshot very easily, so if you have anything you think I should test I would be very happy to give it a shot. On Jun 24, 2011, at 16:51, Kyle Moffett wrote: On Jun 24, 2011, at 16:02, Jan Kara wrote: On Fri

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-06-28 Thread Jan Kara
On Mon 27-06-11 23:21:17, Moffett, Kyle D wrote: On Jun 27, 2011, at 12:01, Ted Ts'o wrote: On Mon, Jun 27, 2011 at 05:30:11PM +0200, Lukas Czerner wrote: I've found some. So although data=journal users are minority, there are some. That being said I agree with you we should do something

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-06-28 Thread Ben Hutchings
On Tue, 2011-06-28 at 11:36 +0200, Jan Kara wrote: On Mon 27-06-11 23:21:17, Moffett, Kyle D wrote: [...] Please correct me if this is horribly horribly wrong: no journal: Nothing is journalled + Very fast. + Works well for filesystems that are mkfsed on every boot - Have

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-06-28 Thread Ted Ts'o
My basic impression is that the use of data=journalled can help reduce the risk (slightly) of serious corruption to some kinds of databases when the application does not provide appropriate syncs or journalling on its own (IE: such as text-based Wiki database files). Yes, although if the

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-06-28 Thread Moffett, Kyle D
On Jun 28, 2011, at 10:16, Ted Ts'o wrote: My basic impression is that the use of data=journalled can help reduce the risk (slightly) of serious corruption to some kinds of databases when the application does not provide appropriate syncs or journalling on its own (IE: such as text-based Wiki

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-06-28 Thread Moffett, Kyle D
This is really helpful to me, but it's deviated a bit from solving the original bug. Based on the last log that I generated showing that the error occurs in journal_stop(), what else should I be testing? Further discussion of the exact behavior of data-journalling below: On Jun 28, 2011, at

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-06-28 Thread Jan Kara
On Tue 28-06-11 14:30:55, Moffett, Kyle D wrote: This is really helpful to me, but it's deviated a bit from solving the original bug. Based on the last log that I generated showing that the error occurs in journal_stop(), what else should I be testing? I was looking at it for a while but so

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-06-28 Thread Moffett, Kyle D
On Jun 28, 2011, at 18:57, Jan Kara wrote: On Tue 28-06-11 14:30:55, Moffett, Kyle D wrote: On Jun 28, 2011, at 05:36, Jan Kara wrote: Well, direct IO is atomic in data=journal the same way as in data=ordered. It can happen only half of direct IO write is done when you hit power button at the

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-06-27 Thread Lukas Czerner
On Fri, 24 Jun 2011, Jan Kara wrote: On Fri 24-06-11 11:03:52, Moffett, Kyle D wrote: On Jun 24, 2011, at 09:46, Jan Kara wrote: On Thu 23-06-11 16:19:08, Moffett, Kyle D wrote: Besides which, line 534 in the Debian 2.6.32 kernel I am using is this one:

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-06-27 Thread Amir Goldstein
On Mon, Jun 27, 2011 at 2:16 PM, Lukas Czerner lczer...@redhat.com wrote: On Fri, 24 Jun 2011, Jan Kara wrote: On Fri 24-06-11 11:03:52, Moffett, Kyle D wrote: On Jun 24, 2011, at 09:46, Jan Kara wrote: On Thu 23-06-11 16:19:08, Moffett, Kyle D wrote: Besides which, line 534 in the

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-06-27 Thread Jan Kara
On Mon 27-06-11 13:16:50, Lukas Czerner wrote: On Fri, 24 Jun 2011, Jan Kara wrote: On Fri 24-06-11 11:03:52, Moffett, Kyle D wrote: On Jun 24, 2011, at 09:46, Jan Kara wrote: On Thu 23-06-11 16:19:08, Moffett, Kyle D wrote: Besides which, line 534 in the Debian 2.6.32 kernel I am

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-06-27 Thread Lukas Czerner
On Mon, 27 Jun 2011, Jan Kara wrote: On Mon 27-06-11 13:16:50, Lukas Czerner wrote: On Fri, 24 Jun 2011, Jan Kara wrote: On Fri 24-06-11 11:03:52, Moffett, Kyle D wrote: On Jun 24, 2011, at 09:46, Jan Kara wrote: On Thu 23-06-11 16:19:08, Moffett, Kyle D wrote: Besides

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-06-27 Thread Ted Ts'o
On Mon, Jun 27, 2011 at 05:30:11PM +0200, Lukas Czerner wrote: I've found some. So although data=journal users are minority, there are some. That being said I agree with you we should do something about it - either state that we want to fully support data=journal - and then we should

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-06-27 Thread Jan Kara
On Mon 27-06-11 12:01:40, Ted Tso wrote: On Mon, Jun 27, 2011 at 05:30:11PM +0200, Lukas Czerner wrote: I've found some. So although data=journal users are minority, there are some. That being said I agree with you we should do something about it - either state that we want to fully

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-06-27 Thread Moffett, Kyle D
On Jun 27, 2011, at 12:01, Ted Ts'o wrote: On Mon, Jun 27, 2011 at 05:30:11PM +0200, Lukas Czerner wrote: I've found some. So although data=journal users are minority, there are some. That being said I agree with you we should do something about it - either state that we want to fully support

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-06-24 Thread Jan Kara
On Thu 23-06-11 16:19:08, Moffett, Kyle D wrote: On Jun 23, 2011, at 16:55, Sean Ryle wrote: Maybe I am wrong here, but shouldn't the cast be to (unsigned long) or to (sector_t)? Line 534 of commit.c: jbd_debug(4, JBD: got buffer %llu (%p)\n,

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-06-24 Thread Moffett, Kyle D
On Jun 24, 2011, at 09:46, Jan Kara wrote: On Thu 23-06-11 16:19:08, Moffett, Kyle D wrote: Besides which, line 534 in the Debian 2.6.32 kernel I am using is this one: J_ASSERT(commit_transaction-t_nr_buffers = commit_transaction-t_outstanding_credits); Hmm, OK, so we've used

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-06-24 Thread Jan Kara
On Fri 24-06-11 11:03:52, Moffett, Kyle D wrote: On Jun 24, 2011, at 09:46, Jan Kara wrote: On Thu 23-06-11 16:19:08, Moffett, Kyle D wrote: Besides which, line 534 in the Debian 2.6.32 kernel I am using is this one: J_ASSERT(commit_transaction-t_nr_buffers =

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-06-24 Thread Kyle Moffett
On Jun 24, 2011, at 16:02, Jan Kara wrote: On Fri 24-06-11 11:03:52, Moffett, Kyle D wrote: On Jun 24, 2011, at 09:46, Jan Kara wrote: On Thu 23-06-11 16:19:08, Moffett, Kyle D wrote: Besides which, line 534 in the Debian 2.6.32 kernel I am using is this one:

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-06-23 Thread Moffett, Kyle D
Hello again everyone, I'm in the middle of doing some software testing on a pre-production clone of this system using some modified software configurations and a testing-only data volume, and I've managed to trigger this panic again. The trigger was exactly the same; I had a bunch of queued

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-06-23 Thread Sean Ryle
Maybe I am wrong here, but shouldn't the cast be to (unsigned long) or to (sector_t)? Line 534 of commit.c: jbd_debug(4, JBD: got buffer %llu (%p)\n, (unsigned long long)bh-b_blocknr, bh-b_data); Line 64 of buffer_head.h: sector_t

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-06-23 Thread Ted Ts'o
On Thu, Jun 23, 2011 at 01:32:48PM -0500, Moffett, Kyle D wrote: Ted, since this new iteration has no customer data, passwords, keys, or any other private data, I'm going to try to get approval to release an exact EC2 image of this system for you to test with, including the fake data volume

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-06-23 Thread Moffett, Kyle D
On Jun 23, 2011, at 16:55, Sean Ryle wrote: Maybe I am wrong here, but shouldn't the cast be to (unsigned long) or to (sector_t)? Line 534 of commit.c: jbd_debug(4, JBD: got buffer %llu (%p)\n, (unsigned long long)bh-b_blocknr,

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-04-05 Thread Moffett, Kyle D
On Apr 04, 2011, at 20:15, Ted Ts'o wrote: On Mon, Apr 04, 2011 at 09:24:28AM -0500, Moffett, Kyle D wrote: Unfortunately it was not a trivial process to install Debian squeeze onto an EC2 instance; it took a couple ugly Perl scripts, a patched Debian-Installer, and several manual

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-04-05 Thread Ted Ts'o
On Tue, Apr 05, 2011 at 10:30:11AM -0500, Moffett, Kyle D wrote: Couple of questions which might give me some clues: (a) was this a natively formatted ext4 file system, or a ext3 file system which was later converted to ext4? All the filesystems were formatted like this using

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-04-04 Thread Moffett, Kyle D
On Apr 02, 2011, at 22:02, Ted Ts'o wrote: Sorry for not following up sooner. Are you still able to reproduce this failure? If I set up an identical Debian stable instance on EC-2, am I likely to reproduce it myself? Do you have a package list or EC2 base image I can use as a starting

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-04-04 Thread Moffett, Kyle D
On Apr 04, 2011, at 10:24, Moffett, Kyle D wrote: On Apr 02, 2011, at 22:02, Ted Ts'o wrote: Sorry for not following up sooner. Are you still able to reproduce this failure? If I set up an identical Debian stable instance on EC-2, am I likely to reproduce it myself? Do you have a package

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-04-04 Thread Ted Ts'o
On Mon, Apr 04, 2011 at 09:24:28AM -0500, Moffett, Kyle D wrote: Unfortunately it was not a trivial process to install Debian squeeze onto an EC2 instance; it took a couple ugly Perl scripts, a patched Debian-Installer, and several manual post-install-but-before-reboot steps (like fixing up

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-04-02 Thread Ted Ts'o
Hi Kyle, Sorry for not following up sooner. Are you still able to reproduce this failure? If I set up an identical Debian stable instance on EC-2, am I likely to reproduce it myself? Do you have a package list or EC2 base image I can use as a starting point? Thanks,

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-03-01 Thread Kyle Moffett
Package: linux-2.6 Version: 2.6.32-30 Severity: important I'm getting a repeatable BUG from ext4, which seems to be caused by Postfix processing its mail queue. The specific filesystem block device that has the problem seems to be dm-13, which on this boot is the logical volume containing the

Bug#615998: linux-image-2.6.32-5-xen-amd64: Repeatable kernel BUG at fs/jbd2/commit.c:534 from Postfix on ext4

2011-03-01 Thread Moffett, Kyle D
Whoops, looks like the Debian bug-tracker lost the CC list somehow. I believe I've got all the CCs re-added, sorry for any duplicate emails. On Mar 01, 2011, at 11:52, Kyle Moffett wrote: Package: linux-2.6 Version: 2.6.32-30 Severity: important I'm getting a repeatable BUG from ext4,