Hello,
I found tmpfs usage grow unexpectedly on my systems until crash when it
reaches 100%. The system use aufs to unite tmpfs and squashfs. After
long investigation I am able to reproduce it now. 4 blocks are allocated each
time a file is created via aufs (in addition to the file size), but the blocks
are never freed. This does not happen when a file is created directly on the
tmpfs.
Example session:
Creating the aufs (/rootfs_ro contains my squashfs image):
[root@test ~]# cd /tmp
[root@test /tmp]# mkdir rw
[root@test /tmp]# mkdir mnt
[root@test /tmp]# mount -t tmpfs none /tmp/rw
[root@test /tmp]# mount -t aufs -n -o "br:/tmp/rw:/rootfs_ro/usr" none
"/tmp/mnt"
Size and usage stats of the aufs:
[root@test /tmp]# df /tmp/mnt ; du -skx /tmp/mnt
Filesystem 1K-blocks Used Available Use% Mounted on
none 252908 24 252884 1% /tmp/mnt
272101 /tmp/mnt
Size and the usage stats of the tmpfs:
[root@test /tmp]# df /tmp/rw ; du -skx /tmp/rw
Filesystem 1K-blocks Used Available Use% Mounted on
none 252908 56 252852 1% /tmp/rw
0 /tmp/rw
Please note how 'df' and 'du' output does not match. I can understand
some bookkeeping data being accounted by df and not by du and I don't
care if the difference is small and does not increase…
Now, let's create a file on the aufs:
[root@test /tmp]# dd if=/dev/zero of=/tmp/mnt/test bs=1024 count=100
100+0 records in
100+0 records out
102400 bytes (102 kB) copied, 0.00262438 s, 39.0 MB/s
[root@test /tmp]# df /tmp/rw ; du -skx /tmp/rw
Filesystem 1K-blocks Used Available Use% Mounted on
none 252908 160 252748 1% /tmp/rw
100 /tmp/rw
'du' output has been increased by 100 blocks, as expected. 'df' output
by 104 blocks. No problem if the extra 4 blocks can be recovered.
Let's try…
[root@test /tmp]# rm /tmp/mnt/test
[root@test /tmp]# df /tmp/rw ; du -skx /tmp/rw
Filesystem 1K-blocks Used Available Use% Mounted on
none 252908 60 252848 1% /tmp/rw
0 /tmp/rw
'du' output is back to 0, but the extra 4 blocks reported by 'df' still
in use…
Let's try again…
[root@test /tmp]# dd if=/dev/zero of=/tmp/mnt/test bs=1024 count=100
100+0 records in
100+0 records out
102400 bytes (102 kB) copied, 0.00162909 s, 62.9 MB/s
[root@test /tmp]# df /tmp/rw ; du -skx /tmp/rw
Filesystem 1K-blocks Used Available Use% Mounted on
none 252908 164 252744 1% /tmp/rw
100 /tmp/rw
[root@test /tmp]# rm /tmp/mnt/test
[root@test /tmp]# df /tmp/rw ; du -skx /tmp/rw
Filesystem 1K-blocks Used Available Use% Mounted on
none 252908 64 252844 1% /tmp/rw
0 /tmp/rw
Another 4 blocks lost… And it will happen each time a file is created
and removed.
Is it tmpfs or aufs? Let's try the same directly on the tmpfs:
[root@test /tmp]# dd if=/dev/zero of=/tmp/rw/test bs=1024 count=100
100+0 records in
100+0 records out
102400 bytes (102 kB) copied, 0.00175222 s, 58.4 MB/s
[root@test /tmp]# df /tmp/rw ; du -skx /tmp/rw
Filesystem 1K-blocks Used Available Use% Mounted on
none 252908 164 252744 1% /tmp/rw
100 /tmp/rw
Both 'du' and 'df' output has been increased by 100 blocks only…
[root@test /tmp]# rm /tmp/rw/test
[root@test /tmp]# df /tmp/rw ; du -skx /tmp/rw
Filesystem 1K-blocks Used Available Use% Mounted on
none 252908 64 252844 1% /tmp/rw
0 /tmp/rw
…and the 100 blocks where recovered after the file has been removed.
So, it seems aufs allocates some 'invisible' data on the tmpfs and never
frees it.
I am using Linux kernel 3.6.6 and aufs3.6. I had the same problem with
kernels 3.4.6 and 3.4.17 with aufs3.4.
Greets,
Jacek
------------------------------------------------------------------------------
LogMeIn Central: Instant, anywhere, Remote PC access and management.
Stay in control, update software, and manage PCs from one command center
Diagnose problems and improve visibility into emerging IT issues
Automate, monitor and manage. Do more in less time with Central
http://p.sf.net/sfu/logmein12331_d2d
