time delay before changes become visible

2007-06-18 Thread Michael Towers

We are getting a strange 'lag' behaviour using aufs (well it seems
strange to us). It's happening in a live CD environment using squashfs.

After certain file-system operations - apparently where a large number
of files get added or updated, but exactly when it happens and when not
is not clear - it takes a long time, sometimes more than a minute,
before the updates are visible in the aufs mount (although they are
immediately visible in the write branch).

Here is a report of how it manifests itself - it is unfortunately very
specific to
Archlinux so it might not help you much ...

It seems like there is still something wrong with aufs lag time.
Here's the problem:

- I made a live cd using your latest larch scripts.
- I booted into the live cd and did a pacman -Sl and pacman lists all
packages in the repos, about 3925 packages.
- I did a pacman -Syu and all my packages were up to date. Immediately
after that I do a pacman -Sl again and nothing is listed anymore.
- I wait about 1 minute and do a pacman -Sl and the 3925 packages are
listed again.

As you can see below, in essence each root directory has its own aufs
mount with a base branch and an overlay branch (both in squashfs) plus a
write branch in tmpfs. In the example where the problem arose, the many
files and directories were in /var/lib/pacman subdirectories.



aufs version 20070611 (but also other versions, I think an earlier one
was worse)
kernel 2.6.21.5



Here is how the aufs module was built (Archlinux PKGBUILD, I hope it's
clear enough):

pkgname=aufs
_kerndir=2.6.21-ARCH
pkgver=2.6.21cvs
pkgrel=2
pkgdesc=A unification file system ('another unionfs')
arch=(i686)
url=http://aufs.sourceforge.net;
depends=('kernel26=2.6.21' 'kernel26=2.6.22')
_cvsroot=:pserver:anonymous:@aufs.cvs.sourceforge.net:/cvsroot/aufs
_cvsmod=aufs

build() {
  msg Connecting to $_cvsmod.sourceforge.net CVS server
  cvs -d$_cvsroot checkout $_cvsmod

  msg CVS checkout done or server timeout
  msg Starting make...

  cp -r $_cvsmod $_cvsmod-build
  cd $_cvsmod-build
  make KDIR=/lib/modules/${_kerndir}/build -f local.mk || return 1
  #mkdir -p ${startdir}/pkg/sbin
  #install -m 500 -p mount.aufs umount.aufs auplink aulchown
${startdir}/pkg/sbin
  install -D -m 644 -p aufs.ko
${startdir}/pkg/lib/modules/${_kerndir}/kernel/fs/aufs/aufs.ko
}



/proc/mounts:

rootfs / rootfs rw 0 0
tmpfs / tmpfs rw 0 0
/dev/sr0 /.livesys/livecd iso9660 ro 0 0
/dev/loop0 /.livesys/base squashfs ro 0 0
/dev/loop1 /.livesys/etc squashfs ro 0 0
/dev/loop0 /bin squashfs ro 0 0
/dev/loop0 /lib squashfs ro 0 0
/dev/loop0 /sbin squashfs ro 0 0
/dev/loop0 /usr squashfs ro 0 0
/dev/loop1 /etc squashfs ro 0 0
none /proc proc rw 0 0
/dev/loop2 /.livesys/system squashfs ro 0 0
/dev/loop3 /.livesys/overlay squashfs ro 0 0
aufs /bin aufs
rw,xino=/.livesys/.bin_w/.aufs.xino,br:/.livesys/.bin_w=rw:/.livesys/base/bin=ro:/.livesys/system/bin=ro 


0 0
aufs /boot aufs
rw,xino=/.livesys/.boot_w/.aufs.xino,br:/.livesys/.boot_w=rw:/.livesys/system/boot=ro 


0 0
aufs /etc aufs
rw,xino=/.livesys/.etc_w/.aufs.xino,br:/.livesys/.etc_w=rw:/.livesys/etc/etc=ro:/.livesys/system/etc=ro 


0 0
aufs /home aufs
rw,xino=/.livesys/.home_w/.aufs.xino,br:/.livesys/.home_w=rw:/.livesys/system/home=ro 


0 0
aufs /lib aufs
rw,xino=/.livesys/.lib_w/.aufs.xino,br:/.livesys/.lib_w=rw:/.livesys/base/lib=ro:/.livesys/system/lib=ro 


0 0
aufs /opt aufs
rw,xino=/.livesys/.opt_w/.aufs.xino,br:/.livesys/.opt_w=rw:/.livesys/system/opt=ro 


0 0
aufs /root aufs
rw,xino=/.livesys/.root_w/.aufs.xino,br:/.livesys/.root_w=rw:/.livesys/overlay/root=ro+wh:/.livesys/system/root=ro 


0 0
aufs /sbin aufs
rw,xino=/.livesys/.sbin_w/.aufs.xino,br:/.livesys/.sbin_w=rw:/.livesys/base/sbin=ro:/.livesys/system/sbin=ro 


0 0
aufs /srv aufs
rw,xino=/.livesys/.srv_w/.aufs.xino,br:/.livesys/.srv_w=rw:/.livesys/system/srv=ro 


0 0
aufs /usr aufs
rw,xino=/.livesys/.usr_w/.aufs.xino,br:/.livesys/.usr_w=rw:/.livesys/base/usr=ro:/.livesys/system/usr=ro 


0 0
aufs /var aufs
rw,xino=/.livesys/.var_w/.aufs.xino,br:/.livesys/.var_w=rw:/.livesys/system/var=ro 


0 0
aufs /larch aufs
rw,xino=/.livesys/.larch_w/.aufs.xino,br:/.livesys/.larch_w=rw:/.livesys/overlay/larch=ro 


0 0
none /sys sysfs rw 0 0
none /dev ramfs rw 0 0
none /proc/bus/usb usbfs rw 0 0
none /dev/pts devpts rw 0 0
none /dev/shm tmpfs rw 0 0



/sys/fs/aufs/* attached as aufs.tar.gz



Best wishes,
mt




aufs.tar.gz
Description: GNU Zip compressed data
-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/

Re: time delay before changes become visible

2007-06-18 Thread sfjro

Hello Michael,

Michael Towers:
 - I made a live cd using your latest larch scripts.
 - I booted into the live cd and did a pacman -Sl and pacman lists all
 packages in the repos, about 3925 packages.
 - I did a pacman -Syu and all my packages were up to date. Immediately
 after that I do a pacman -Sl again and nothing is listed anymore.
 - I wait about 1 minute and do a pacman -Sl and the 3925 packages are
 listed again.

Thank you for your report.
Are there any message left in your syslog?
Does your packman access only aufs, or a branch filesystem directly?
And please send me the output of 'strace pacman -Sl' which didn't show
the list. I need to know which systemcall returned what error.


 aufs version 20070611 (but also other versions, I think an earlier one
 was worse)

What was worse?


Junjiro Okajima

-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/