Hello J. R., >---------------------------------------- >When you have any problems or strange behaviour in aufs, please let me >know with: >- /proc/mounts (instead of the output of mount(8)) rootfs / rootfs rw 0 0 none /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 none /proc proc rw,nosuid,nodev,noexec,relatime 0 0 none /dev devtmpfs rw,relatime,size=120924k,nr_inodes=30231,mode=755 0 0 none /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 /dev/disk/by-uuid/8b080e2d-33d4-42cd-945f-0b5845132273 / ext4 rw,relatime,errors=remount-ro,barrier=1,data=ordered 0 0 none /sys/fs/fuse/connections fusectl rw,relatime 0 0 none /sys/kernel/debug debugfs rw,relatime 0 0 none /sys/kernel/security securityfs rw,relatime 0 0 none /dev/shm tmpfs rw,nosuid,nodev,relatime 0 0 none /var/run tmpfs rw,nosuid,relatime,mode=755 0 0 none /var/lock tmpfs rw,nosuid,nodev,noexec,relatime 0 0 none /lib/init/rw tmpfs rw,nosuid,relatime,mode=755 0 0 rpc_pipefs /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 curlftpfs#ftp://root:r...@172.20.20.120/ /mnt/test fuse rw,nosuid,nodev,relatime,user_id=0,group_id=0 0 0 none /mnt/aufs aufs rw,relatime,si=5949b626,udba=none,rdcache=90,rdblk=102400,rdhash=30720,sum 0 0
>- /sys/module/aufs/* ls /sys/module/aufs/ holders initstate notes parameters refcnt sections srcversion version cat /sys/module/aufs/srcversion DAA70C879577CB4901122FF cat /sys/module/aufs/version 2-standalone.tree-20091207 cat /sys/module/aufs/parameters/brs 1 r...@vmebre:~/vmebk/perl# cat /sys/module/aufs/parameters/nwkq 4 ls /sys/fs/aufs/si_5949b626/ br0 br13 br18 br22 br27 br31 br36 br40 br45 br5 br54 br59 br63 br68 br72 br77 br81 br1 br14 br19 br23 br28 br32 br37 br41 br46 br50 br55 br6 br64 br69 br73 br78 br9 br10 br15 br2 br24 br29 br33 br38 br42 br47 br51 br56 br60 br65 br7 br74 br79 xi_path br11 br16 br20 br25 br3 br34 br39 br43 br48 br52 br57 br61 br66 br70 br75 br8 br12 br17 br21 br26 br30 br35 br4 br44 br49 br53 br58 br62 br67 br71 br76 br80 >- linux kernel version Linux vmebre 2.6.32-21-generic-pae #32-Ubuntu SMP Fri Apr 16 09:39:35 UTC 2010 i686 GNU/Linux (Stock ubuntu 10.04 LTS kernel) >- configuration (define/undefine CONFIG_AUFS_xxx) CONFIG_AUFS_FS=m CONFIG_AUFS_BRANCH_MAX_127=y CONFIG_AUFS_HINOTIFY=y CONFIG_AUFS_BR_RAMFS=y CONFIG_AUFS_BR_FUSE=y CONFIG_AUFS_POLL=y CONFIG_AUFS_BDEV_LOOP=y >And as you might know, every access to linux filesystem includes >"revalidating" a file. When you access aufs, it calls the "revalidate" >function in branch filesystems. If the function in your "slower" WAN >connection costs high, then what you should concentrate is the branch >filesystem instead of aufs. >So I'd ask you how could you confirm that "aufs re-reads each branch on >every ls"? >I mean, could you see the readdir operation via WAN connection, or is it >the revalidate operation? The WAN Connection is basically an FTP connection mapped via curlftpfs. For a simple test, /mnt/test mounts just one FTP server. aufs then unifys subirs within that mountpoint. What i can see is that the ftp server gets and answers requests (from aufs through curlftpfs) for each branch, when doing an ls /mnt/aufs which is the mountpoint of aufs (and therefore the top-level dir) To be more precise, it is not one request but two per branch. I am not sure wether this is bc of revalidating or readdir operations or both. Maybe the first call is lookup, the second getattr, so both :-). My assumption is that aufs walks through all branches to see if an upper branch masks a lower branch, before responding to ls (or any other file/directory operation). (which is intended behavoiur) If the above is correct, the following formular should give the number of request to the underlying filesystem when using ls (readdir): Requests = nrBranch * lookupAttrCalls * nrOfEntries Example: ls /mnt/aufs using with just one subdir (mnt/aufs/1) and no files: nrBranch = 2 nrOfEntries = 1 # assume only one subdir or one file for simplicity lookupAttrCalls = 2 # lookup+getattr Requests = 2 x 1 x 2 = 4 Example: ls /mnt/aufs/1 with mnt/aufs/1/1/ (one more subdir, no files): requests = 2 x 1 x 2 (/mnt/aufs/1) + 2 x 1 x 2 (/mnt/aufs/1/1) = 8 Example: ls /mnt/aufs/1/1 with mnt/aufs/1/1/1/ (one more subdir, no files): requests = 2 x 1 x 2 (/mnt/aufs/1) + 2 x 1 x 2 (/mnt/aufs/1/1) + 2 x 1 x 2 (/mnt/aufs/1/1) = 12 Now assume a top level direcory has 50 entries, each subdir has also 50 entries and we have 10 branches with a similar directory structure: ls /mnt/aufs/ (contains dir 1/..50/) Requests = 10 x 50 x 2 = 1000 ls /mnt/aufs/1 (dir 1/ contains 1/..50/) Requests = 10 x 50 x 2 + 10 x 50 x 2 = 2000 Same calculation with 80 branches: ls /mnt/aufs/ (contains dir 1..50) Requests = 80 x 50 x 2 = 8000 ls /mnt/aufs/1 (dir 1/ contains 1..50) Requests = 80 x 50 x 2 + 10 x 50 x 2 = 16000 file open/read will require less requests, of cause. That would explain the ftp requests i can see. I thought that increasing the cache (rdcache, rdhash, rdblk, udba=none) would help, but it seems those settings do not take effect in this particular case. If the above seems to be correct, my question would be if there is an option to cache branch information or something similar to avoid re-reading branches (readdir), since the entire mountPoint can be considered RO (even the first branch if needed) and no udba is in effect. Best regards Wolfgang Alper ___ Altrust Consulting GmbH In der Horst 43 D-32278 Kirchlengern Germany ------------------------------------------------------------------------------ This SF.net email is sponsored by Sprint What will you do first with EVO, the first 4G phone? Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
