adding aufs-users list. ---------- Forwarded message ---------- From: Thayumanavar Sachithanantham <[email protected]> Date: Fri, Aug 6, 2010 at 7:50 PM Subject: Re: question on branch deletion in aufs - reg. To: [email protected]
Hello Okajima, I don't have some info that you had requested. Info which i can provide is: All of CONFIG_AUFS* option is set. The kernel is 2.6.27.45 SLES kernel with aufs ( recent ver as taken git for 2.6.27) pulled into it. I do not have the .config and /sys related info. under the kernel for which the crash occurred. i have 4 branchs /dev/sd* disks formatted with reiserfs filesystem on it. The issue doesn't get reproduce frequently. The test cases involves creating parallel threads ( that does file creation (using dd), mkdir, unlink and rename) and a parallel script that tries do branch deletion contiuously in a loop until we succed to delete a branch ( when no EBUSY comes up). Info. regarding the system it is a 8 CPU X86_64 (2127 Mhz) with 16 GB memory. Here is the backtrace and diassembly for the crash: [ from diassembly it appear that deref of h_inode->i_count in au_h_iptr led to crash). ( Thank for the quick response to my earlier mail.) PID: 22513 TASK: ffff88046296e840 CPU: 3 COMMAND: "mount" #0 [ffff8803c284f960] crash_kexec at ffffffff80267322 #1 [ffff8803c284fa30] __die at ffffffff8049add1 #2 [ffff8803c284fa50] do_page_fault at ffffffff8049c7b1 #3 [ffff8803c284fbb0] error_exit at ffffffff8049a1d9 [exception RIP: au_h_iptr+69] RIP: ffffffffa02a777d RSP: ffff8803c284fc60 RFLAGS: 00010286 RAX: ffffffff0000001d RBX: 00000000ffffffff RCX: 0000000000000000 RDX: ffffffffffffffe0 RSI: 00000000ffffffff RDI: ffff8803ea4850c0 RBP: ffff8801735f9080 R8: ffff8801735f9080 R9: 0000000000000000 R10: 0000000000000001 R11: 0000000000000000 R12: 00000000735f9002 R13: 0000000000000008 R14: ffff8801735f90c8 R15: ffff8801735f9080 ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018 #4 [ffff8803c284fc60] au_refresh_hinode_self at ffffffffa02a817b #5 [ffff8803c284fd00] aufs_remount_fs at ffffffffa028f8a8 #6 [ffff8803c284fdd0] __do_remount_sb at ffffffff802b1670 #7 [ffff8803c284fe00] do_remount at ffffffff802c5b55 #8 [ffff8803c284fe50] do_mount at ffffffff802c78c7 #9 [ffff8803c284ff30] sys_mount at ffffffff802c79e4 #10 [ffff8803c284ff80] system_call_fastpath at ffffffff8020bfcb RIP: 00007fbcb5f7f88a RSP: 00007fffc4a7a938 RFLAGS: 00010202 RAX: 00000000000000a5 RBX: ffffffff8020bfcb RCX: ffffffffc0ed0020 RDX: 000000000061aec0 RSI: 000000000061aea0 RDI: 000000000061ae80 RBP: 000000000061ae80 R8: 000000000061af80 R9: 00007fffc4a7abdc R10: ffffffffc0ed0020 R11: 0000000000000202 R12: 00007fffc4a7abdc R13: 00007fffc4a7af68 R14: 00000000c0ed0020 R15: 00007fffc4a7ab80 ORIG_RAX: 00000000000000a5 CS: 0033 SS: 002b crash> dis au_h_iptr 0xffffffffa02a7738 <au_h_iptr>: lea 0xffffffffffffffb8(%rdi),%r8 0xffffffffa02a773c <au_h_iptr+4>: mov 0xfffffffffffffff0(%rdi),%rdi 0xffffffffa02a7740 <au_h_iptr+8>: xor %ecx,%ecx 0xffffffffa02a7742 <au_h_iptr+10>: mov %rcx,%rax 0xffffffffa02a7745 <au_h_iptr+13>: mov %sil,%dl 0xffffffffa02a7748 <au_h_iptr+16>: test %rdi,%rdi 0xffffffffa02a774b <au_h_iptr+19>: cmovne %r8,%rax 0xffffffffa02a774f <au_h_iptr+23>: cmpl $0x0,0x28(%rax) 0xffffffffa02a7753 <au_h_iptr+27>: jg 0xffffffffa02a775f 0xffffffffa02a7755 <au_h_iptr+29>: cmpl $0x0,0x2c(%rax) 0xffffffffa02a7759 <au_h_iptr+33>: jg 0xffffffffa02a775f 0xffffffffa02a775b <au_h_iptr+35>: ud2a 0xffffffffa02a775d <au_h_iptr+37>: jmp 0xffffffffa02a775d 0xffffffffa02a775f <au_h_iptr+39>: xor %eax,%eax 0xffffffffa02a7761 <au_h_iptr+41>: test %rdi,%rdi 0xffffffffa02a7764 <au_h_iptr+44>: movsbq %dl,%rdx 0xffffffffa02a7768 <au_h_iptr+48>: cmovne %r8,%rax 0xffffffffa02a776c <au_h_iptr+52>: shl $0x5,%rdx 0xffffffffa02a7770 <au_h_iptr+56>: mov 0x38(%rax),%rax 0xffffffffa02a7774 <au_h_iptr+60>: mov (%rax,%rdx,1),%rax 0xffffffffa02a7778 <au_h_iptr+64>: test %rax,%rax 0xffffffffa02a777b <au_h_iptr+67>: je 0xffffffffa02a7787 0xffffffffa02a777d <au_h_iptr+69>: cmpl $0x0,0x48(%rax) 0xffffffffa02a7781 <au_h_iptr+73>: jg 0xffffffffa02a7787 0xffffffffa02a7783 <au_h_iptr+75>: ud2a 0xffffffffa02a7785 <au_h_iptr+77>: jmp 0xffffffffa02a7785 0xffffffffa02a7787 <au_h_iptr+79>: retq On Fri, Aug 6, 2010 at 7:19 PM, <[email protected]> wrote: > > Hello Thayumanavar, > > Thayumanavar Sachithanantham: >> In AUFS branch deletion code paths, it seems that s_inodes of the aufs >> superblock is walked through without taking the inode_lock. Is it a >> safe to walk through this list and working on it without holding the >> inode_lock? Rest of places in the kernel where s_inodes list is > > sb->s_umount should protect it in vfs and I don't think inode_lock is > necessary. > > >> Sometimes on case of branch deletion on one of branch succeeds when >> some i/o happening on aufs mount point, sometimes i seem to hit upon a >> crash in the code path au_h_iptr.( via au_refresh_hinode_self code) >> where h_inode contain some invalid kernel virtual address. >> Also i see that au_ii could return NULL. > > That may be another bug in aufs. > Will you give me these info? > > (from the aufs README) > - /proc/mounts (instead of the output of mount(8)) > - /sys/module/aufs/* > - /sys/fs/aufs/* (if you have them) > - /debug/aufs/* (if you have them) > - linux kernel version > if your kernel is not plain, for example modified by distributor, > the url where i can download its source is necessary too. > - aufs version which was printed at loading the module or booting the > system, instead of the date you downloaded. > - configuration (define/undefine CONFIG_AUFS_xxx) > - kernel configuration or /proc/config.gz (if you have it) > - behaviour which you think to be incorrect > - actual operation, reproducible one is better > - mailto: aufs-users at lists.sourceforge.net > > > J. R. Okajima > ------------------------------------------------------------------------------ This SF.net email is sponsored by Make an app they can't live without Enter the BlackBerry Developer Challenge http://p.sf.net/sfu/RIM-dev2dev
