Re: Subdirectory creation on snapshot
On 2012-05-14, at 5:08 AM, David Sterba wrote: > On Wed, May 09, 2012 at 10:38:30AM -0700, Brendan Smithyman wrote: >> If I understand you correctly, this would be the case with nested >> subvolumes; i.e., if subvolume A is exists within the directory tree >> subvolume B, and B is snapshotted. I expected this, and it sounds >> totally consistent with my understanding of how btrfs subvolumes work. >> However, the behaviour I'm seeing seems to be a different thing, so I >> just want to double-check: > > I've read again your first post and indeed it's not the empty subvol > issue. > >> In my case I am executing the "btrfs subvolume snapshot @working >> newsnapshot" command (or something like it). The "@working" subvolume >> exists in the filesystem root, and does not contain any other >> subvolumes within its own subdirectory tree. In the new subvolume, >> "newsnapshot", there is an entry called "@working" that is identified >> as inode number 2 as you say. But this isn't due to a subvolume in >> the directory tree of the original "@working", since it still happens, >> e.g., if it is the only subvolume on the system (apart from the root, >> of course). > > So I followed the steps to reproduce it with a 3.4 kernel, but I don't > see the duplicated @working anywhere. > > # btrfs subvol create @working > # ls @working > src/ > # btrfs subvol snap @working 2012-05-14 > # btrfs subvol snap @working test > # ls test/ > src/ > > # btrfs subvol list . > ID 258 top level 5 path @working > ID 259 top level 5 path 2012-05-14 > ID 260 top level 5 path test > > >> The naive assumption is that (excepting nested subvolumes), the >> snapshot should be indistinguishable from the original. Additionally, >> I'm a bit perplexed by the behaviour on some of my volumes and not >> others. > > Not a consistent behaviour then, so there's another factor. > > Please run stat on the test/@working directory, if it's the ino == 2 > (empty subvol) or not. It is inode == 2. > > Do you do lot's of snapshots on the fs, or are there lots of data plus > the filesystem is ~80% full? I have a theory, that this can somehow > interact with background subvolume deletion: if a subvolume is deleted > from directory hierarchy, but just scheduled for deletion, reusing it's > name could be incorrectly taken as part of the dir hierarchy to snapshot > and thus the extra '@working' is created. Not a ton; I may have had half a dozen at one point while testing a backup script, but they were all essentially identical. Probably < 10 in the history of the filesystem (it's quite new). Data use is maybe 60% of the raw drive capacity IIRC; still lots of unallocated space for new chunks. > > In your environment this would mean that there was a subvolume 'test', > then deleted, still in the queue for actual deletion. Slow snapshot > deletion may come from large number of them and this is even slower when > the filesystem is fragmented or near full. > >> It's not a big deal, and I'm happy to take your word for it >> (or look at the code, if you'd be willing to point me in the right >> direction; I'm not averse to learning). I just wanted to double-check >> that we're talking about the same thing. > > Seems more like a bug, let's narrow down the conditions before we look > into the code. I just replied to your other email about the existing bug. Thanks, Brendan > > > thanks, > david smime.p7s Description: S/MIME cryptographic signature
Re: Subdirectory creation on snapshot
On 2012-05-14, at 9:27 AM, David Sterba wrote: > On Mon, May 14, 2012 at 02:08:52PM +0200, David Sterba wrote: >> Seems more like a bug, let's narrow down the conditions before we look >> into the code. > > And already known bug (with a proposed fix, cc'd author): > > http://www.spinics.net/lists/linux-btrfs/msg15195.html > > Quoting: > > # mkfs.btrfs /dev/sda1 > # mount /dev/sda1 /mnt > # mkdir /mnt/1 > # cd /mnt/1 > # btrfs subvolume snapshot /mnt snap0 > # ll /mnt/1 > total 0 > drwxr-xr-x 1 root root 10 Jun 30 15:01 1 > ^^^ > # ll /mnt/1/snap0/ > total 0 > drwxr-xr-x 1 root root 10 Jun 30 15:01 1 > ^^^ > It is also 10, but... > # ll /mnt/1/snap0/1 > total 0 > [None] > # cd /mnt/1/snap0/1/snap0 > [Enter a unexisted directory successfully] > --- > > Though there's a difference, that in your listing the @working (here > it'd be snap0) is present. > > > david Thanks David, Yes, it's looking similar, but perhaps not identical. I checked, and the "@working" directory inside the new snapshot has inode 2. The phantom directory shows up on ls, and can be removed by rmdir. However: since I sent the first email, I moved one of the volumes to btrfs RAID1 from btrfs single on top of md raid. I recreated the filesystem (mkfs.btrfs -L reaper -m raid1 -d raid1 /dev/sdc1 /dev/sdd1) at that time, and restored data from a backup. The new filesystem is not displaying the issues from before; it is now behaving in-line with the SSD I mentioned in my first post, which was similarly set up from the command line with mkfs.btrfs. The disks that *are* still showing the subdirectory creation issue were both converted from ext4 (using old tools). So perhaps that's a direction to explore. Thanks for the help. Cheers, Brendan smime.p7s Description: S/MIME cryptographic signature
Re: Subdirectory creation on snapshot
Thanks David, If I understand you correctly, this would be the case with nested subvolumes; i.e., if subvolume A is exists within the directory tree subvolume B, and B is snapshotted. I expected this, and it sounds totally consistent with my understanding of how btrfs subvolumes work. However, the behaviour I'm seeing seems to be a different thing, so I just want to double-check: In my case I am executing the "btrfs subvolume snapshot @working newsnapshot" command (or something like it). The "@working" subvolume exists in the filesystem root, and does not contain any other subvolumes within its own subdirectory tree. In the new subvolume, "newsnapshot", there is an entry called "@working" that is identified as inode number 2 as you say. But this isn't due to a subvolume in the directory tree of the original "@working", since it still happens, e.g., if it is the only subvolume on the system (apart from the root, of course). The naive assumption is that (excepting nested subvolumes), the snapshot should be indistinguishable from the original. Additionally, I'm a bit perplexed by the behaviour on some of my volumes and not others. It's not a big deal, and I'm happy to take your word for it (or look at the code, if you'd be willing to point me in the right direction; I'm not averse to learning). I just wanted to double-check that we're talking about the same thing. I appreciate the help! Cheers, Brendan On 2012-05-09, at 9:58 AM, David Sterba wrote: > On Mon, May 07, 2012 at 05:10:08PM -0700, Brendan Smithyman wrote: >> I'm experiencing some odd-seeming behaviour with btrfs on Ubuntu >> 12.04, using the Ubuntu x86-64 generic 3.2.0-24 kernel and btrfs-tools >> 0.19+20120328-1~precise1 (backported from the current Debian version >> using Ubuntu's backportpackage). When I snapshot a subvolume on some >> of my drives, it creates an empty directory inside the snapshot with >> the same name as the original subvolume. Example case and details >> below: > > This is known and it's not a problem, though I was surprised when I had > first seen it myself. Snapshotting is not recursive, the case of > file->file, directory->directory is straightforward, and when a > subvolume is encountered, a new file sub-type is created, it's > identified by BTRFS_EMPTY_SUBVOL_DIR_OBJECTID internally, so it's a kind > of a "stub" subvolume. It is identified by inode number 2 in stat > output. The object cannot be modified and just sits there. > > > david smime.p7s Description: S/MIME cryptographic signature
Subdirectory creation on snapshot
Hi All, I'm experiencing some odd-seeming behaviour with btrfs on Ubuntu 12.04, using the Ubuntu x86-64 generic 3.2.0-24 kernel and btrfs-tools 0.19+20120328-1~precise1 (backported from the current Debian version using Ubuntu's backportpackage). When I snapshot a subvolume on some of my drives, it creates an empty directory inside the snapshot with the same name as the original subvolume. Example case and details below: root@zeus:/mnt/reaper-all# ls 2012-05-03 @working root@zeus:/mnt/reaper-all# ls @working documents games MeganMusic misc-docs photos vm Downloads isosMeganPhotos Network Trash Folder Temporary Items workdir DropboxiTunes MeganUBC otherphotos videos root@zeus:/mnt/reaper-all# btrfs subvolume snapshot @working test Create a snapshot of '@working' in './test' root@zeus:/mnt/reaper-all# ls test/ documents games MeganMusic misc-docs photos vm Downloads isosMeganPhotos Network Trash Folder Temporary Items workdir DropboxiTunes MeganUBC otherphotos videos @working root@zeus:/mnt/reaper-all# ls test/@working/ root@zeus:/mnt/reaper-all# btrfs subvolume list . ID 257 top level 5 path @working ID 258 top level 5 path 2012-05-03 ID 263 top level 5 path test The preceding volume is mounted with "rw,compress" on top of LVM on md raid1, and I get the same behaviour from another volume running directly on a gpt partition. In both cases, the volumes are 1 TB btrfs with default creation parameters, and were converted from ext4 using btrfs-convert (with the Ubuntu 0.19+20100601 version, before switching btrfs-tools to version 0.19+20120328). As far as I am able to tell, both filesystems are healthy. The drives are exported to some Mac OS systems via netatalk, which leads to some odd directories, but AFAIK shouldn't affect much else. The subdirectory under the snapshot is just that (i.e., a directory and not a subvolume). I don't see this behaviour on either the "@" or "@home" subvolumes of the system SSD (conforming to the Ubuntu btrfs layout), which are mounted "rw,noatime,discard,subvol=@" and "rw,noatime,discard,subvol=@home". However, the btrfs volume on the SSD was built without btrfs-convert. It's an Intel 520 with Sandforce controller, which is why I'm not using -o compress in this case. I haven't confirmed one way or the other if it is an issue of -o compress, but can do a test reboot with different options if that will help. Could anyone shed some light on what's going on? If it's simply an issue with out of date btrfs-progs, I can either live with it or upgrade. However, I'd rather not track the bleeding edge on this system if I can avoid it. Thanks! All the best, Brendan Smithyman smime.p7s Description: S/MIME cryptographic signature
