On Mon, Jan 12, 2015 at 4:35 PM, <[email protected]> wrote:
>
> Daniel Smedegaard Buus:
>> Righty-O, I'll do that now then :)
>> (*EDIT*: Actually, considering the discoveries below, do you still
>> need me to do this? AFAICT, the behavior is pretty clear now, and I
>> don't think this is a bug as such, just a limitation, and perhaps all
>> that's needed is a descriptive error message when a mount such as mine
>> fails.)
>
> Probably you are right.
> Those behaviours were due to the fact that aufs XINO files cannot be put
> on zfs nor btrfs I think. But there still mysteries are left. Why there
> were not messages left in kernel log files?
Yesterday, before leaving for responsibilites, I was trying to set up
autofs to automatically mount my aufs /Archive folder, but couldn't
get it working (is aufs not compatible with autofs? I could get other
indirect fses such as a tmpfs mounted just fine, but aufs not...).
In trying to figure out why, I tried enabling both "verbose" and
"debug" logging in autofs, but here also there were zero messages in
any log files — I'm wondering if it might be some system-wide problem
or rsyslog setting, if that's at all possible.
> For the XINO file,
> - first aufs tries creating in on the first writable branch.
> - When aufs finds that the branch fs doesn't have abilities for XINO,
> aufs tries /tmp.
> - When /tmp doesn't have abilities for XINO either, aufs rejects
> mounting and returns an error with messaging
> "xino doesn't support /tmp/brabra(fs-type)"
>
> So for the cases,
> - "mount aufs with many RO zfs"
> + there is no writable branch
> + aufs tries /tmp and finds it is btrfs
> + then aufs should produce the msg "xino doesn't support
> /tmp/brabra(btrfs)"
> - "mount aufs with two btrfs simply"
> + aufs tries the first writable branch, and finds it is btrfs
> + same behaviour should happen
> In both cases, aufs rejects to be mounted in the end.
>
Cool, that's what I thought, and that explains why it worked on my old
installation where /tmp was on ext4 :)
>
>> When I has a writable AUFS working on my old installation, I could
>> write files to it, and they ended up where expected =E2=80=94 is it only th=
>> e
>> XINO file that requires specific filesystem?
>
> Internally some filesystems doesn't maintain several inode attributes
> which are necessary for XINO files. I've checked all filesystems in
> mainline (at that time), and registered those who are not usable for
> XINO into fs/aufs/fstype.h. At mounting aufs, aufs checks those bad FS
> for XINO and returns an error if it mathces.
>
Perhaps for some reason the code isn't recognizing btrfs and zfs and
so I never get that type of message? Or maybe it's due to the old
aufs-tools package in Ubuntu?
>
> Basically every FS should have the inode numbers consistently. Which
> means if you see inum is 10 for fileA, then you should see inum10 again
> even if the inode of fileA has been expired and discarded from the
> system cache.
> Because these inode numbers are indexed by its value in XINO, the
> larger inum on the branch fs will make XINO file bigger. But in most
> cases, XINO file is a sparse file which has many "holes" in it (the disk
> blocks are not allocated for the hole).
>
> So, for your case, as long as zfs has the consistent inode numbers, the
> size of XINO will not grow forever. The size of XINO depends upon the
> largest inum on your branch.
>
> Does it grow eternally?
I understand now, thank you :) And as for the 1.8+ M inodes in my aufs
mount, the XINO will grow to ~19MB and then stay at that size. (I
eventually decided on storing XINO in /run, btw)
>
> Yeah, I am just hoping to be a good uncle. :-)
>
Ha ha :D :D
>
>> Another observation: I was testing what happens when I put the xino
>> file on a very small tmpfs. I noticed that running a find command to
>> list every file and folder in my aufs mount (~1.8 M files) would use
>> about 19 MB on the xino file, so I set the tmpfs size to 5 MB and ran
>> a find on it. The result was that find finished, exited with a 0, yet
>> it only found 465 k files. There were no messages or errors in the
>> logs og dmesg. Shouldn't there be some kind of warning? It basically
>
> Yes, of course.
> Again an error message is lost. I don't know why. Aufs should produce
> "write failed (%zd)\n" and return EIO.
>
>
> We could make it clear why you could not mount aufs. The reason seems
> due to xino path. It is OK. But I don't know why any msgs were not left.
>
> I'd ask you these things.
> - check the setting of the syslog daemon. where does the
> kern.{err,warning} messages go?
Oh, this is embarrassing :( ... I kind of thought that the more
explicit error messages were supposed to be printed on the TTY. I
looked at rsyslog's conf, and kern.* goes to kern.log. I just tailed
kern.log while trying to mount without an explicit xino path, and
kern.log shows,
[60666.886619] aufs au_xino_create:774:mount[10155]: xino doesn't
support /tmp/.aufs.xino(btrfs)
I don't know how I missed that before. I think maybe I've just been
f-tailing /var/log/messages, wrongfully assuming that
/var/log/messages would contain all the log statements that dmesg and
friends do. And still expecting to see error messages in my terminal
:/ So sorry!
> - re-build aufs with CONFIG_AUFS_DEBUG and try the commands in my
> previous mail.
>
Btw, should I use the source code tar bundle from kernel.org, or
should I use the Ubuntu mainline kernel PPA?
> And I need to know the detail of small tmpfs for XINO.
> Is it something like this? I could not reproduce the problem.
>
> $ sudo mount -t tmpfs -osize=64k none /tmp/x
> $ sudo mount -t aufs -o br=/tmp/x:/usr none /tmp/w
> aufs test_add:268:mount[3422]: uid/gid/perm /usr 0/0/0755, 0/0/01777
> $ sudo find /tmp/w | wc -l
> aufs au_xino_do_write:420:find[3432]: I/O Error, write failed (-28)
> aufs au_xino_write:456:find[3432]: I/O Error, write failed (-5)
> 358
>
# mkdir /dummy
# mount -t tmpfs -o size=1M tmpfs /dummy
# mount -t aufs -o ro,xino=/dummy/xino,br=<a lot of =ro folders on
zfs> none /Archive
# find /Archive
Okay — again I'm an idiot. /var/log/kern.log DOES print a bunch of,
Jan 13 11:15:15 disneyland kernel: [61206.125170] aufs
au_xino_do_write:433:find[10698]: I/O Error, write failed (-28)
— although find still exits with a 0 (perhaps it's not possible to let
the aufs error bubble up to find, so that it fails?).
Seems aufs is behaving just as expected, it's just me being a moron :/
>
> I could find a bug.
> Internally au_set_h_iptr() got an error from XINO (which produces "I/O
> Error, write failed (-28)"), but simply ignores.
> I might think such situation is very rare and set "void" as the type of
> au_set_h_iptr().
> Hmm, to fix or not to fix, that is the question...
>
> I looked closer again, found the real cause and could fix this bug (only
> this bug). If you want, try this patch which will be included in next
> Monday release.
>
Okay, that's cool :) So even though I was a misunderstanding dummy,
something good came out of it!
Do you have any idea why aufs won't mount with autofs? I have a direct
map using tmpfs which works:
/Archive -fstype=tmpfs,size=20M none
Doing an ls on /Archive instantly mounts it. However,
/Archive
-fstype=aufs,ro,xino=/run/xino,br=/titanic/Downloads/Processed=ro:/titanic/Volumes/XD001=ro
none
does not. It just,
# ll /Archive/
ls: cannot access /Archive/: No such file or directory
It takes a few seconds before the error message is given. No messages
appear in dmesg, messages, or kern.log. Weird thing is, mount and df
do not show /Archive as being mounted, but the first time I do "umount
/Archive" afterwards it will not say that Archive is not mounted.
Second time it will. Strange.
I thought it might be the ':' separator in the branch options, so I
did one with just
/Archive -fstype=aufs,ro,xino=/run/xino,br=/titanic/Downloads/Processed=ro
none
— but it's the same.
Well, anyway, I know I'm bugging you a lot here, so I'll lay off :)
Thank you for all your help, you've been very kind :)
Daniel
------------------------------------------------------------------------------
New Year. New Location. New Benefits. New Data Center in Ashburn, VA.
GigeNET is offering a free month of service with a new server in Ashburn.
Choose from 2 high performing configs, both with 100TB of bandwidth.
Higher redundancy.Lower latency.Increased capacity.Completely compliant.
http://p.sf.net/sfu/gigenet
