Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-19 Thread Chris Mason
Quoting Jens Axboe (2013-04-19 09:32:50) > > > > No wonder this thing crashes. Chris, can't the original bio carry > > bbio in bi_private and let end_bio_extent_readpage() free the bbio > > instead of abusing bi_bdev like this? > > Ugh, wtf. > > Chris, time for a swim in the bay :-) Yeah, I

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-19 Thread Jens Axboe
On Thu, Apr 18 2013, Tejun Heo wrote: > (cc'ing btrfs people) > > On Fri, Apr 19, 2013 at 11:33:20AM +0800, Wanlong Gao wrote: > > RIP: 0010:[] [] > > ftrace_raw_event_block_bio_complete+0x73/0xf0 > ... > > [] bio_endio+0x80/0x90 > > [] btrfs_end_bio+0xf6/0x190 [btrfs] > > []

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-19 Thread Jens Axboe
On Thu, Apr 18 2013, Tejun Heo wrote: > On Thu, Apr 18, 2013 at 10:57:54PM -0700, Tejun Heo wrote: > > No wonder this thing crashes. Chris, can't the original bio carry > > bbio in bi_private and let end_bio_extent_readpage() free the bbio > > instead of abusing bi_bdev like this? > > BTW, I

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-19 Thread Chris Mason
Quoting Tejun Heo (2013-04-19 01:57:54) > > Ewweehh > > No wonder this thing crashes. Chris, can't the original bio carry > bbio in bi_private and let end_bio_extent_readpage() free the bbio > instead of abusing bi_bdev like this? Yes, we can definitely carry bbio up

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-19 Thread Jan Schmidt
On Fri, April 19, 2013 at 07:57 (+0200), Tejun Heo wrote: > (cc'ing btrfs people) > > On Fri, Apr 19, 2013 at 11:33:20AM +0800, Wanlong Gao wrote: >> RIP: 0010:[] [] >> ftrace_raw_event_block_bio_complete+0x73/0xf0 > ... >> [] bio_endio+0x80/0x90 >> [] btrfs_end_bio+0xf6/0x190 [btrfs] >> []

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-19 Thread Wanlong Gao
On 04/19/2013 02:17 PM, Tejun Heo wrote: > On Thu, Apr 18, 2013 at 10:57:54PM -0700, Tejun Heo wrote: >> No wonder this thing crashes. Chris, can't the original bio carry >> bbio in bi_private and let end_bio_extent_readpage() free the bbio >> instead of abusing bi_bdev like this? > > BTW, I

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-19 Thread Tejun Heo
On Thu, Apr 18, 2013 at 10:57:54PM -0700, Tejun Heo wrote: > No wonder this thing crashes. Chris, can't the original bio carry > bbio in bi_private and let end_bio_extent_readpage() free the bbio > instead of abusing bi_bdev like this? BTW, I think it's a bit too late to fix this properly from

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-19 Thread Tejun Heo
On Thu, Apr 18, 2013 at 10:57:54PM -0700, Tejun Heo wrote: No wonder this thing crashes. Chris, can't the original bio carry bbio in bi_private and let end_bio_extent_readpage() free the bbio instead of abusing bi_bdev like this? BTW, I think it's a bit too late to fix this properly from

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-19 Thread Wanlong Gao
On 04/19/2013 02:17 PM, Tejun Heo wrote: On Thu, Apr 18, 2013 at 10:57:54PM -0700, Tejun Heo wrote: No wonder this thing crashes. Chris, can't the original bio carry bbio in bi_private and let end_bio_extent_readpage() free the bbio instead of abusing bi_bdev like this? BTW, I think it's a

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-19 Thread Jan Schmidt
On Fri, April 19, 2013 at 07:57 (+0200), Tejun Heo wrote: (cc'ing btrfs people) On Fri, Apr 19, 2013 at 11:33:20AM +0800, Wanlong Gao wrote: RIP: 0010:[812484d3] [812484d3] ftrace_raw_event_block_bio_complete+0x73/0xf0 ... [811b6c10] bio_endio+0x80/0x90

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-19 Thread Chris Mason
Quoting Tejun Heo (2013-04-19 01:57:54) Ewweehh No wonder this thing crashes. Chris, can't the original bio carry bbio in bi_private and let end_bio_extent_readpage() free the bbio instead of abusing bi_bdev like this? Yes, we can definitely carry bbio up higher

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-19 Thread Jens Axboe
On Thu, Apr 18 2013, Tejun Heo wrote: On Thu, Apr 18, 2013 at 10:57:54PM -0700, Tejun Heo wrote: No wonder this thing crashes. Chris, can't the original bio carry bbio in bi_private and let end_bio_extent_readpage() free the bbio instead of abusing bi_bdev like this? BTW, I think it's a

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-19 Thread Jens Axboe
On Thu, Apr 18 2013, Tejun Heo wrote: (cc'ing btrfs people) On Fri, Apr 19, 2013 at 11:33:20AM +0800, Wanlong Gao wrote: RIP: 0010:[812484d3] [812484d3] ftrace_raw_event_block_bio_complete+0x73/0xf0 ... [811b6c10] bio_endio+0x80/0x90 [a0790d26]

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-19 Thread Chris Mason
Quoting Jens Axboe (2013-04-19 09:32:50) No wonder this thing crashes. Chris, can't the original bio carry bbio in bi_private and let end_bio_extent_readpage() free the bbio instead of abusing bi_bdev like this? Ugh, wtf. Chris, time for a swim in the bay :-) Yeah, I can't really

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread Tejun Heo
(cc'ing btrfs people) On Fri, Apr 19, 2013 at 11:33:20AM +0800, Wanlong Gao wrote: > RIP: 0010:[] [] > ftrace_raw_event_block_bio_complete+0x73/0xf0 ... > [] bio_endio+0x80/0x90 > [] btrfs_end_bio+0xf6/0x190 [btrfs] > [] bio_endio+0x3d/0x90 > [] req_bio_endio+0xa3/0xe0 Ugh In

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread Wanlong Gao
On 04/18/2013 10:30 PM, Jens Axboe wrote: > On Thu, Apr 18 2013, Wanlong Gao wrote: >> On 04/18/2013 09:35 PM, Jens Axboe wrote: >>> On Thu, Apr 18 2013, Wanlong Gao wrote: > A bio is always fully initialized, regardless of which internal > allocator it came from. If people are doing

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread srost...@gmail.com
Could it be a module not recompiled with a newer kernel? -- Steve Jens Axboe wrote: > >Hmm dunno. It happens right after we've completed the bio, which >touches >a lot of fields too. bi_bdev sits between bi_next (which we definitely >used) and bi_flags. > >But adding slab use-after-free

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread Jens Axboe
On Thu, Apr 18 2013, Linus Torvalds wrote: > On Thu, Apr 18, 2013 at 11:13 AM, Jens Axboe wrote: > > On Thu, Apr 18 2013, Tejun Heo wrote: > >> On Thu, Apr 18, 2013 at 10:39:00AM -0700, Jens Axboe wrote: > >> > > >> > Yep, thanks Linus for that hint... Must be someone abusing it for a > >> > flag

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread Linus Torvalds
On Thu, Apr 18, 2013 at 11:13 AM, Jens Axboe wrote: > On Thu, Apr 18 2013, Tejun Heo wrote: >> On Thu, Apr 18, 2013 at 10:39:00AM -0700, Jens Axboe wrote: >> > >> > Yep, thanks Linus for that hint... Must be someone abusing it for a >> > flag field post submission? Crazy. >> >> Let's hope that's

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread Jens Axboe
On Thu, Apr 18 2013, Tejun Heo wrote: > On Thu, Apr 18, 2013 at 10:39:00AM -0700, Jens Axboe wrote: > > > If I'm not mistaken, it seems like we have a bio which has 1 as its > > > ->bi_bdev. Ummm no idea whatsoever how that can happen tho right > > > now. Looking into it. > > > > Yep,

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread Tejun Heo
On Thu, Apr 18, 2013 at 10:39:00AM -0700, Jens Axboe wrote: > > If I'm not mistaken, it seems like we have a bio which has 1 as its > > ->bi_bdev. Ummm no idea whatsoever how that can happen tho right > > now. Looking into it. > > Yep, thanks Linus for that hint... Must be someone abusing

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread Tejun Heo
Hello, Wanlong. On Thu, Apr 18, 2013 at 10:45:10PM +0800, Wanlong Gao wrote: > OK, but I should capture it tomorrow morning because this remote machine has > already panicked > and need hard reboot. Can you please apply the following patch when you try the next time and report the kernel log?

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread Jens Axboe
On Thu, Apr 18 2013, Tejun Heo wrote: > Hello, > > On Thu, Apr 18, 2013 at 09:08:34AM -0700, Linus Torvalds wrote: > > On Thu, Apr 18, 2013 at 7:30 AM, Jens Axboe wrote: > > > > > > Could you capture the full dump? I'd like to see what rcx was, and that > > > seems to have scrolled off. > > > >

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread Tejun Heo
Hello, On Thu, Apr 18, 2013 at 09:08:34AM -0700, Linus Torvalds wrote: > On Thu, Apr 18, 2013 at 7:30 AM, Jens Axboe wrote: > > > > Could you capture the full dump? I'd like to see what rcx was, and that > > seems to have scrolled off. > > You don't need it. The %cr2 value shows the address of

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread Linus Torvalds
On Thu, Apr 18, 2013 at 7:30 AM, Jens Axboe wrote: > > Could you capture the full dump? I'd like to see what rcx was, and that > seems to have scrolled off. You don't need it. The %cr2 value shows the address of the bad access, and it's 0x0001. Linus -- To unsubscribe from

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread Wanlong Gao
On 04/18/2013 10:30 PM, Jens Axboe wrote: > On Thu, Apr 18 2013, Wanlong Gao wrote: >> On 04/18/2013 09:35 PM, Jens Axboe wrote: >>> On Thu, Apr 18 2013, Wanlong Gao wrote: > A bio is always fully initialized, regardless of which internal > allocator it came from. If people are doing

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread Jens Axboe
On Thu, Apr 18 2013, Wanlong Gao wrote: > On 04/18/2013 09:35 PM, Jens Axboe wrote: > > On Thu, Apr 18 2013, Wanlong Gao wrote: > >>> A bio is always fully initialized, regardless of which internal > >>> allocator it came from. If people are doing private kmallocs, then they > >>> better be using

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread Wanlong Gao
On 04/18/2013 09:35 PM, Jens Axboe wrote: > On Thu, Apr 18 2013, Wanlong Gao wrote: >>> A bio is always fully initialized, regardless of which internal >>> allocator it came from. If people are doing private kmallocs, then they >>> better be using bio_init() as well. >>> >>> Wanlong, would it be

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread Jens Axboe
On Thu, Apr 18 2013, Wanlong Gao wrote: > > A bio is always fully initialized, regardless of which internal > > allocator it came from. If people are doing private kmallocs, then they > > better be using bio_init() as well. > > > > Wanlong, would it be possible to get a full dmesg on boot see I

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread Jens Axboe
On Wed, Apr 17 2013, Steven Rostedt wrote: > On Wed, 2013-04-17 at 16:36 +0800, Wanlong Gao wrote: > > Hi Tj, all, > > > > I can get kernel panic on my machine with kernel 3.9.0-rc7-4-gbb33db7 every > > time > > after booting for several minutes. > > > > Here attached the panic picture and the

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread Jens Axboe
On Wed, Apr 17 2013, Steven Rostedt wrote: On Wed, 2013-04-17 at 16:36 +0800, Wanlong Gao wrote: Hi Tj, all, I can get kernel panic on my machine with kernel 3.9.0-rc7-4-gbb33db7 every time after booting for several minutes. Here attached the panic picture and the config. I'm

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread Jens Axboe
On Thu, Apr 18 2013, Wanlong Gao wrote: A bio is always fully initialized, regardless of which internal allocator it came from. If people are doing private kmallocs, then they better be using bio_init() as well. Wanlong, would it be possible to get a full dmesg on boot see I can see

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread Wanlong Gao
On 04/18/2013 09:35 PM, Jens Axboe wrote: On Thu, Apr 18 2013, Wanlong Gao wrote: A bio is always fully initialized, regardless of which internal allocator it came from. If people are doing private kmallocs, then they better be using bio_init() as well. Wanlong, would it be possible to get a

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread Jens Axboe
On Thu, Apr 18 2013, Wanlong Gao wrote: On 04/18/2013 09:35 PM, Jens Axboe wrote: On Thu, Apr 18 2013, Wanlong Gao wrote: A bio is always fully initialized, regardless of which internal allocator it came from. If people are doing private kmallocs, then they better be using bio_init() as

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread Wanlong Gao
On 04/18/2013 10:30 PM, Jens Axboe wrote: On Thu, Apr 18 2013, Wanlong Gao wrote: On 04/18/2013 09:35 PM, Jens Axboe wrote: On Thu, Apr 18 2013, Wanlong Gao wrote: A bio is always fully initialized, regardless of which internal allocator it came from. If people are doing private kmallocs,

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread Linus Torvalds
On Thu, Apr 18, 2013 at 7:30 AM, Jens Axboe ax...@kernel.dk wrote: Could you capture the full dump? I'd like to see what rcx was, and that seems to have scrolled off. You don't need it. The %cr2 value shows the address of the bad access, and it's 0x0001. Linus -- To

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread Tejun Heo
Hello, On Thu, Apr 18, 2013 at 09:08:34AM -0700, Linus Torvalds wrote: On Thu, Apr 18, 2013 at 7:30 AM, Jens Axboe ax...@kernel.dk wrote: Could you capture the full dump? I'd like to see what rcx was, and that seems to have scrolled off. You don't need it. The %cr2 value shows the

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread Jens Axboe
On Thu, Apr 18 2013, Tejun Heo wrote: Hello, On Thu, Apr 18, 2013 at 09:08:34AM -0700, Linus Torvalds wrote: On Thu, Apr 18, 2013 at 7:30 AM, Jens Axboe ax...@kernel.dk wrote: Could you capture the full dump? I'd like to see what rcx was, and that seems to have scrolled off.

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread Tejun Heo
Hello, Wanlong. On Thu, Apr 18, 2013 at 10:45:10PM +0800, Wanlong Gao wrote: OK, but I should capture it tomorrow morning because this remote machine has already panicked and need hard reboot. Can you please apply the following patch when you try the next time and report the kernel log? It

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread Tejun Heo
On Thu, Apr 18, 2013 at 10:39:00AM -0700, Jens Axboe wrote: If I'm not mistaken, it seems like we have a bio which has 1 as its -bi_bdev. Ummm no idea whatsoever how that can happen tho right now. Looking into it. Yep, thanks Linus for that hint... Must be someone abusing it for a

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread Jens Axboe
On Thu, Apr 18 2013, Tejun Heo wrote: On Thu, Apr 18, 2013 at 10:39:00AM -0700, Jens Axboe wrote: If I'm not mistaken, it seems like we have a bio which has 1 as its -bi_bdev. Ummm no idea whatsoever how that can happen tho right now. Looking into it. Yep, thanks Linus for

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread Linus Torvalds
On Thu, Apr 18, 2013 at 11:13 AM, Jens Axboe ax...@kernel.dk wrote: On Thu, Apr 18 2013, Tejun Heo wrote: On Thu, Apr 18, 2013 at 10:39:00AM -0700, Jens Axboe wrote: Yep, thanks Linus for that hint... Must be someone abusing it for a flag field post submission? Crazy. Let's hope that's

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread Jens Axboe
On Thu, Apr 18 2013, Linus Torvalds wrote: On Thu, Apr 18, 2013 at 11:13 AM, Jens Axboe ax...@kernel.dk wrote: On Thu, Apr 18 2013, Tejun Heo wrote: On Thu, Apr 18, 2013 at 10:39:00AM -0700, Jens Axboe wrote: Yep, thanks Linus for that hint... Must be someone abusing it for a flag

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread srost...@gmail.com
Could it be a module not recompiled with a newer kernel? -- Steve Jens Axboe ax...@kernel.dk wrote: Hmm dunno. It happens right after we've completed the bio, which touches a lot of fields too. bi_bdev sits between bi_next (which we definitely used) and bi_flags. But adding slab

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread Wanlong Gao
On 04/18/2013 10:30 PM, Jens Axboe wrote: On Thu, Apr 18 2013, Wanlong Gao wrote: On 04/18/2013 09:35 PM, Jens Axboe wrote: On Thu, Apr 18 2013, Wanlong Gao wrote: A bio is always fully initialized, regardless of which internal allocator it came from. If people are doing private kmallocs,

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-18 Thread Tejun Heo
(cc'ing btrfs people) On Fri, Apr 19, 2013 at 11:33:20AM +0800, Wanlong Gao wrote: RIP: 0010:[812484d3] [812484d3] ftrace_raw_event_block_bio_complete+0x73/0xf0 ... [811b6c10] bio_endio+0x80/0x90 [a0790d26] btrfs_end_bio+0xf6/0x190 [btrfs]

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-17 Thread Steven Rostedt
On Wed, 2013-04-17 at 16:36 +0800, Wanlong Gao wrote: > Hi Tj, all, > > I can get kernel panic on my machine with kernel 3.9.0-rc7-4-gbb33db7 every > time > after booting for several minutes. > > Here attached the panic picture and the config. > I'm sorry that the panic log is not completed. >

Re: [BUG REPORT] Kernel panic on 3.9.0-rc7-4-gbb33db7

2013-04-17 Thread Steven Rostedt
On Wed, 2013-04-17 at 16:36 +0800, Wanlong Gao wrote: Hi Tj, all, I can get kernel panic on my machine with kernel 3.9.0-rc7-4-gbb33db7 every time after booting for several minutes. Here attached the panic picture and the config. I'm sorry that the panic log is not completed. But