from: Hendra Soetjahja
Salutations linux http://aruizendaal.nl/leave.php?happen=mprgvgeyvskesmthupr hend...@yahoo.com linux Sent from my iPhone -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: mount btrfs takes 30 minutes, btrfs check runs out of memory
I'll try without autodefrag anyways tomorrow just to make sure. And then file a bug report too with however it decides to behave. Vincent -Original Message- From: "Duncan" <1i5t5.dun...@cox.net> Sent: Thursday, August 13, 2015 20:30 To: linux-btrfs@vger.kernel.org Subject: Re: mount btrfs takes 30 minutes, btrfs check runs out of memory Chris Murphy posted on Thu, 13 Aug 2015 17:19:41 -0600 as excerpted: > Well I think others have suggested 3000 snapshots and quite a few things > will get very slow. But then also you have autodefrag and I forget the > interaction of this with many snapshots since the snapshot aware defrag > code was removed. Autodefrag shouldn't have any snapshots mount-time-related interaction, with snapshot-aware-defrag disabled. The interaction between defrag (auto or not) and snapshots will be additional data space usage, since with snapshot-aware disabled, defrag only works with the current copy, thus forcing it to COW the extents elsewhere while not freeing the old extents as they're still referenced by the snapshots, but it shouldn't affect mount-time. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: mount btrfs takes 30 minutes, btrfs check runs out of memory
I have 2 snapshots a few days apart for incrementally backing up the volume but that's it. I'll try without autodefrag tomorrow. Vincent -Original Message- From: "Chris Murphy" Sent: Thursday, August 13, 2015 19:19 To: "Btrfs BTRFS" Subject: Re: mount btrfs takes 30 minutes, btrfs check runs out of memory On Thu, Aug 13, 2015 at 4:38 PM, Vincent Olivier wrote: > Hi, > > I think I might be having this problem too. 12 x 4TB RAID10 (original makefs, > not converted from ext or whatnot). Says it has ~6TiB left. Centos 7. Dual > Xeon CPU. 32GB RAM. ELRepo Kernel 4.1.5. Fstab options: > noatime,autodefrag,compress=zlib,space_cache,nossd,noauto,x-systemd.automount Well I think others have suggested 3000 snapshots and quite a few things will get very slow. But then also you have autodefrag and I forget the interaction of this with many snapshots since the snapshot aware defrag code was removed. I'd say file a bug with the full details of the hardware from the ground up to the Btrfs file system. And include as an attachment, dmesg with sysrq+t during this "hang". Usually I see t asked if there's just slowness/delays, and w if there's already a kernel message saying there's a blocked task for 120 seconds. -- Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: mount btrfs takes 30 minutes, btrfs check runs out of memory
Chris Murphy posted on Thu, 13 Aug 2015 17:19:41 -0600 as excerpted: > Well I think others have suggested 3000 snapshots and quite a few things > will get very slow. But then also you have autodefrag and I forget the > interaction of this with many snapshots since the snapshot aware defrag > code was removed. Autodefrag shouldn't have any snapshots mount-time-related interaction, with snapshot-aware-defrag disabled. The interaction between defrag (auto or not) and snapshots will be additional data space usage, since with snapshot-aware disabled, defrag only works with the current copy, thus forcing it to COW the extents elsewhere while not freeing the old extents as they're still referenced by the snapshots, but it shouldn't affect mount-time. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
lack of scrub error data
Below is the result of testing a corrupted filesystem. What's going on here? The kernel message log and the btrfs output don't tell me how many errors there were. Also the data is RAID-0 (the default for a filesystem created with 2 devices) so if this was in a data area it should have lost data, but no data was lost so apparently not. Debian kernel 4.0.8-2 and btrfs-tools 4.0-2. # cp -r /lib /mnt/tmp # btrfs scrub start -B /mnt/tmp scrub done for c0992a26-3c7f-47b0-b8de-878dfe197469 scrub started at Fri Aug 14 10:17:53 2015 and finished after 0 seconds total bytes scrubbed: 372.82MiB with 0 errors # dd if=/dev/zero of=/dev/loop0 bs=1024k count=20 seek=100 20+0 records in 20+0 records out 20971520 bytes (21 MB) copied, 0.0267078 s, 785 MB/s # btrfs scrub start -B /mnt/tmp scrub done for c0992a26-3c7f-47b0-b8de-878dfe197469 scrub started at Fri Aug 14 10:19:15 2015 and finished after 0 seconds total bytes scrubbed: 415.04MiB with 0 errors WARNING: errors detected during scrubbing, corrected. # btrfs scrub start -B /mnt/tmp scrub done for c0992a26-3c7f-47b0-b8de-878dfe197469 scrub started at Fri Aug 14 10:20:25 2015 and finished after 0 seconds total bytes scrubbed: 415.04MiB with 0 errors # btrfs fi df /mnt/tmp Data, RAID0: total=800.00MiB, used=400.23MiB System, RAID1: total=8.00MiB, used=16.00KiB System, single: total=4.00MiB, used=0.00B Metadata, RAID1: total=400.00MiB, used=7.39MiB Metadata, single: total=8.00MiB, used=0.00B GlobalReserve, single: total=16.00MiB, used=0.00B # btrfs device stats /mnt/tmp [/dev/loop0].write_io_errs 0 [/dev/loop0].read_io_errs0 [/dev/loop0].flush_io_errs 0 [/dev/loop0].corruption_errs 0 [/dev/loop0].generation_errs 0 [/dev/loop1].write_io_errs 0 [/dev/loop1].read_io_errs0 [/dev/loop1].flush_io_errs 0 [/dev/loop1].corruption_errs 0 [/dev/loop1].generation_errs 0 # diff -ru /lib /mnt/tmp/lib # -- My Main Blog http://etbe.coker.com.au/ My Documents Bloghttp://doc.coker.com.au/ -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RAID0 wrong (raw) device?
I would have been surprised if any generic file system copes well with being mounted in several locations at once, DRBD appears to fight really hard to avoid that happening :) And yeah I'm doing the second thing, I've successfully switched which of the servers is active a few times with no ill effect (I would expect scrub to give me some significant warnings if one of the disks was a couple of months out of date) so I'm presuming that DRBD copes reasonably well or I've been very lucky. Either that luck is very deterministic, DRBD copes correctly, or I've been very very lucky. Very very lucky doesn't sound likely. On Fri, Aug 14, 2015 at 8:54 AM, Hugo Mills wrote: > On Fri, Aug 14, 2015 at 08:32:46AM +1000, Gareth Pye wrote: >> On Thu, Aug 13, 2015 at 9:44 PM, Austin S Hemmelgarn >> wrote: >> > 3. See the warnings about doing block level copies and LVM snapshots of >> > BTRFS volumes, the same applies to using it on DRBD currently as well (with >> > the possible exception of remote DRBD nodes (ie, ones without a local copy >> > of the backing store) (in this case, we need to blacklist backing devices >> > for stacked storage (I think the same issue may be present with BTRFS on a >> > MD based RAID1 set). >> >> >> I've been using BTRFS on top of DRBD for several years now, what >> specifically am I meant to avoid? >> >> I have 6 drives mirrored across a local network, this is done with DRBD. >> At any one time only a single server has the 6 drives mounted with btrfs. >> Is this a ticking time bomb? > >There are two things which are potentially worrisome here: > > - Having the same filesystem mounted on more than one machine at a >time (which you're not doing). > > - Having one or more of the DRBD backing store devices present on the >same machine that the DRBD filesystem is mounted on (which you may >be doing). > >Of these, the first is definitely going to be dangerous. The second > may or may not be, depending on how well DRBD copes with direct writes > to its backing store, and how lucky you are about the kernel > identifying the right devices to use for the FS. > >Hugo. > > -- > Hugo Mills | "Big data" doesn't just mean increasing the font > hugo@... carfax.org.uk | size. > http://carfax.org.uk/ | > PGP: E2AB1DE4 | -- Gareth Pye - blog.cerberos.id.au Level 2 MTG Judge, Melbourne, Australia "Dear God, I would like to file a bug report" -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Oddness with phantom device replacing real device.
On Thu, 13 Aug 2015 09:55:10 + Hugo Mills wrote: > On Thu, Aug 13, 2015 at 01:33:22PM +1000, David Seikel wrote: > > I don't actually think that this is a BTRFS problem, but it's > > showing symptoms within BTRFS, and I have no other clues, so maybe > > the BTRFS experts can help me figure out what is actually going > > wrong. > > > > I'm a sysadmin working for a company that does scientific modelling. > > They have many TBs of data. We use two servers running Ubuntu > > 14.04 LTS to backup all of this data. One of them includes 16 > > spinning rust disks hooked to a RAID controller running in JBOD > > mode (in other words, as far as Linux is concerned, they are just > > 16 ordinary disks). They are /dev/sdc to /dev/sdr, all being used > > as a single BTRFS file system. > > > > I have been having no end of trouble with this system recently. > > Keep in mind that due to the huge amount of data we deal with, doing > > anything takes a long time. So "recently" means "in the last > > several months". > > > > My latest attempt to beat some sense into this server was to > > upgrade it to the latest officially backported kernel from Ubuntu, > > and compile my own copy of btrfs-progs from source code (latest > > release from github). Then I recreated the 16 disk BTRFS file > > system, and started the backup software running again, from > > scratch. The next day, /dev/sdc has vanished, to be replaced be a > > phantom /dev/sds. There's no such disk as /dev/sds. /dev/sds is > > now included in the BTRFS file system replacing /dev/sdc. In /dev > > sdc does indeed vanish, and sds does indeed appear. This was > > happening before. /dev/sds then starts to fill up with errors, > > since no such disk actually exists. > >Sounds like the kind of behaviour when the disk has vanished from > the system for long enough to drop out and be recreated by the > driver. The renaming may (possibly) be down to a poor error-handling > path in btrfs -- we see this happening on USB sometimes, where the > original device node is still hung on to by the FS on a hardware > error, and so when the device comes back it's given a different name. So that part may need some fixing in BTRFS. > > I don't know what is actually causing the problem. The disks are > > in a hot swap backplane, and if I actually pulled sdc out, then it > > would still be listed as part of the BTRFS file system, wouldn't it? > >With btrfs fi show, no, you'd get ** some devices missing ** in the > output. Which is different from what I'm getting. > > If I > > then where to plug some new disk into the same spot, it would not be > > recognised as part of the file system? > >Correct... Unless the device had a superblock with the same UUID in > it (like, say, the new device is just the old one reappearing > again). In that case, udev would trigger a btrfs dev scan, and the > "new" device would rejoin the FS -- probably a little out of date, but > that would be caught by checksums and be fixed if you have redundancy > in the storage. But btrfs is thinking it's a different device, hence all the errors as it gets confused. > > So assuming that the RAID > > controller is getting confused and thinking that sdc has been > > pulled, then replaced by sds, it should not be showing up as part > > of the BTRFS file system? Or maybe there's a signature on sdc that > > BTRFS notices makes it part of the file system, even though BTRFS > > is now confused about it's location? > >See above. > > > After a reboot, sdc returns and sds is gone again. > >Expected. > > > The RAID controller has recently been replaced, but there where > > similar problems with the old one as well. A better model of RAID > > controller was chosen this time. > > > > I've also not been able to complete a scrub on this system recently. > > The really odd thing is that I get messages that the scrub has > > aborted, yet the scrub continues, then much later (days later) the > > scrub causes a kernel panic. The "aborted" happens some random > > time into the scrub, but usually in the early part of the scrub. > > Mind you, if BTRFS is completely confused due to a problem > > elsewhere, then maybe this can be excused. > >I think that means that it's aborting on one device but continuing > on all the others. Ah, would be useful for scrub to say so, and point out which device/s got aborted. > > The other backup server is almost identical, though it has less > > disks in the array. It doesn't have any issues with the BTRFS file > > system. > > > > Can any one help shed some light on this please? Hopefully some > > "quick" things to try, given my definition of "recently" above means > > that most things take days or weeks, or even months for me to try. > > > > I have attached the usual debugging info requested. This is after > > the bogus sds replaces sdc. > > > >The first thing would be to check your system logs for signs of > hardware problems (ATA error
Re: mount btrfs takes 30 minutes, btrfs check runs out of memory
On Thu, Aug 13, 2015 at 4:38 PM, Vincent Olivier wrote: > Hi, > > I think I might be having this problem too. 12 x 4TB RAID10 (original makefs, > not converted from ext or whatnot). Says it has ~6TiB left. Centos 7. Dual > Xeon CPU. 32GB RAM. ELRepo Kernel 4.1.5. Fstab options: > noatime,autodefrag,compress=zlib,space_cache,nossd,noauto,x-systemd.automount Well I think others have suggested 3000 snapshots and quite a few things will get very slow. But then also you have autodefrag and I forget the interaction of this with many snapshots since the snapshot aware defrag code was removed. I'd say file a bug with the full details of the hardware from the ground up to the Btrfs file system. And include as an attachment, dmesg with sysrq+t during this "hang". Usually I see t asked if there's just slowness/delays, and w if there's already a kernel message saying there's a blocked task for 120 seconds. -- Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: trim not working and irreparable errors from btrfsck
On Thu, Aug 13, 2015 at 3:23 AM, Marc Joliet wrote: > Speaking as a user, since "fstrim -av" still always outputs 0 bytes trimmed > on my system: what's the status of this? Did anybody ever file a bug report? Since I'm not having this problem with my SSD, I'm not in a position to provide any meaningful information for such a report. The bug should whether this problem is reproducible with ext4 and XFS on the same device, and the complete details of the stacking (if this is not the full device or partition of it; e.g. if LVM, md, or encryption is between fs and physical device). And also the bug should include full dmesg as attachment, and strace of the fstrim command that results in 0 bytes trimmed. And probably separate bugs for each make/model of SSD, with the bug including make/model and firmware version. Right now I think there's no status because a.) no bug report and b.) not enough information. -- Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Major qgroup regression in 4.2?
Hi I was looking at qgroups in linux 4.2 and noticed that the code to handle subvolume deletion was removed and replaced with a comment: /* * TODO: Modify related function to add related node/leaf to * dirty_extent_root, * for later qgroup accounting. * * Current, this function does nothing. */ The commit in question is 0ed4792af0e8346cb670b4bc540df7594f4b2020, "btrfs: qgroup: Switch to new extent-oriented qgroup mechanism." Sure enough, I can reproduce an inconsistency by just removing a subvolume with more than 1 level. A script to reproduce this is attached to my e-mail (it's from the last time I fixed subvolume delete). This does not happen with Kernel 4.1. I also found the following in an e-mail from Qu a few weeks ago on the list: "And we still don't have a good idea to fix the snapshot deletion bug. (My patchset can only handle snapshot with up to 2 levels. With higher level, the qgroup number will still be wrong until related node/leaves are all COWed)" http://www.spinics.net/lists/linux-btrfs/msg45724.html So did we just commit another qgroup rewrite while knowingly re-introducing a major regression without a plan to fix it, or am I crazy? If there *is* a plan to make this all work again, can I please hear it? The comment mentions something about adding those nodes to a dirty_extent_root. Why wasn't that done? Thanks, --Mark #! /bin/bash SCRATCH_MNT="/btrfs" SCRATCH_DEV=/dev/vdb1 BTRFS_UTIL_PROG=btrfs echo "format, mount fs" mkfs.btrfs -f $SCRATCH_DEV mount -t btrfs $SCRATCH_DEV $SCRATCH_MNT # This always reproduces level 1 trees maxfiles=100 echo "create file set" # Make a bunch of small files in a directory. This is designed to expand # the filesystem tree to something more than zero levels. mkdir $SCRATCH_MNT/files for i in `seq -w 0 $maxfiles`; do dd status=none if=/dev/zero of=$SCRATCH_MNT/files/file$i bs=4096 count=4 done # create a snapshot of what we just did $BTRFS_UTIL_PROG fi sy $SCRATCH_MNT $BTRFS_UTIL_PROG su sna $SCRATCH_MNT $SCRATCH_MNT/snap1 mv $SCRATCH_MNT/snap1/files $SCRATCH_MNT/snap1/old # same thing as before but on the snapshot. this way we can generate # some exclusively owned tree nodes. echo "create file set on snapshot" mkdir $SCRATCH_MNT/snap1/files for i in `seq -w 0 $maxfiles`; do dd status=none if=/dev/zero of=$SCRATCH_MNT/snap1/files/file$i bs=4096 count=4 done SECS=30 echo "sleep for $SECS seconds" # Enable qgroups now that we have our filesystem prepared. This # will kick off a scan which we will have to wait for below. $BTRFS_UTIL_PROG qu en $SCRATCH_MNT sleep $SECS echo "unmount, remount fs to clear cache" umount $SCRATCH_MNT mount -t btrfs $SCRATCH_DEV $SCRATCH_MNT SECS=45 echo "delete snapshot, sleep for $SECS seconds" # Ok, delete the snapshot we made previously. Since btrfs drop # snapshot is a delayed action with no way to force it, we have to # impose another sleep here. $BTRFS_UTIL_PROG su de $SCRATCH_MNT/snap1 sleep $SECS echo "unmount" umount $SCRATCH_MNT # generate a qgroup report and look for inconsistent groups $BTRFS_UTIL_PROG check --qgroup-report $SCRATCH_DEV exit -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RAID0 wrong (raw) device?
On Fri, Aug 14, 2015 at 08:32:46AM +1000, Gareth Pye wrote: > On Thu, Aug 13, 2015 at 9:44 PM, Austin S Hemmelgarn > wrote: > > 3. See the warnings about doing block level copies and LVM snapshots of > > BTRFS volumes, the same applies to using it on DRBD currently as well (with > > the possible exception of remote DRBD nodes (ie, ones without a local copy > > of the backing store) (in this case, we need to blacklist backing devices > > for stacked storage (I think the same issue may be present with BTRFS on a > > MD based RAID1 set). > > > I've been using BTRFS on top of DRBD for several years now, what > specifically am I meant to avoid? > > I have 6 drives mirrored across a local network, this is done with DRBD. > At any one time only a single server has the 6 drives mounted with btrfs. > Is this a ticking time bomb? There are two things which are potentially worrisome here: - Having the same filesystem mounted on more than one machine at a time (which you're not doing). - Having one or more of the DRBD backing store devices present on the same machine that the DRBD filesystem is mounted on (which you may be doing). Of these, the first is definitely going to be dangerous. The second may or may not be, depending on how well DRBD copes with direct writes to its backing store, and how lucky you are about the kernel identifying the right devices to use for the FS. Hugo. -- Hugo Mills | "Big data" doesn't just mean increasing the font hugo@... carfax.org.uk | size. http://carfax.org.uk/ | PGP: E2AB1DE4 | signature.asc Description: Digital signature
Re: mount btrfs takes 30 minutes, btrfs check runs out of memory
Hi, I think I might be having this problem too. 12 x 4TB RAID10 (original makefs, not converted from ext or whatnot). Says it has ~6TiB left. Centos 7. Dual Xeon CPU. 32GB RAM. ELRepo Kernel 4.1.5. Fstab options: noatime,autodefrag,compress=zlib,space_cache,nossd,noauto,x-systemd.automount Sometimes (not all the time) when I cd or ls the mount point it will not return within 5 minutes (I never let it run more than 5 minutes before rebooting) and I reboot and then it takes between 10-30s. Well as I'm writing this it's already been more than 10 minutes. I don't have the problem when I mount manually without the "noauto,x-systemd.automount" options. Can anyone help ? Thanks. Vincent -Original Message- From: "Austin S Hemmelgarn" Sent: Wednesday, August 5, 2015 07:30 To: "John Ettedgui" Cc: "Qu Wenruo" , "btrfs" Subject: Re: mount btrfs takes 30 minutes, btrfs check runs out of memory On 2015-08-04 13:36, John Ettedgui wrote: > On Tue, Aug 4, 2015 at 4:28 AM, Austin S Hemmelgarn > wrote: >> On 2015-08-04 00:58, John Ettedgui wrote: >>> >>> On Mon, Aug 3, 2015 at 8:01 PM, Qu Wenruo wrote: Although the best practice is staying away from such converted fs, either using pure, newly created btrfs, or convert back to ext* before any balance. >>> Unfortunately I don't have enough hard drive space to do a clean >>> btrfs, so my only way to use btrfs for that partition was a >>> conversion. >> >> If you could get your hands on a decent sized flash drive (32G or more), you >> could do an incremental conversion offline. The steps would look something >> like this: >> >> 1. Boot the system into a LiveCD or something similar that doesn't need to >> run from your regular root partition (SystemRescueCD would be my personal >> recommendation, although if you go that way, make sure to boot the >> alternative kernel, as it's a lot newer then the standard ones). >> 2. Plug in the flash drive, format it as BTRFS. >> 3. Mount both your old partition and the flash drive somewhere. >> 4. Start copying files from the old partition to the flash drive. >> 5. When you hit ENOSPC on the flash drive, unmount the old partition, shrink >> it down to the minimum size possible, and create a new partition in the free >> space produced by doing so. >> 6. Add the new partition to the BTRFS filesystem on the flash drive. >> 7. Repeat steps 4-6 until you have copied everything. >> 8. Wipe the old partition, and add it to the BTRFS filesystem. >> 9. Run a full balance on the new BTRFS filesystem. >> 10. Delete the partition from step 5 that is closest to the old partition >> (via btrfs device delete), then resize the old partition to fill the space >> that the deleted partition took up. >> 11. Repeat steps 9-10 until the only remaining partitions in the new BTRFS >> filesystem are the old one and the flash drive. >> 12. Delete the flash drive from the BTRFS filesystem. >> >> This takes some time and coordination, but it does work reliably as long as >> you are careful (I've done it before on multiple systems). >> >> > I suppose I could do that even without the flash as I have some free > space anyway, but moving Tbs of data with Gbs of free space will take > days, plus the repartitioning. It'd probably be easier to start with a > 1Tb drive or something. > Is this currently my best bet as conversion is not as good as I thought? > > I believe my other 2 partitions also come from conversion, though I > may have rebuilt them later from scratch. > > Thank you! > John > Yeah, you're probably better off getting a TB disk and starting with that. In theory it is possible to automate the process, but I would advise against that if at all possible, it's a lot easier to recover from an error if you're doing it manually. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RAID0 wrong (raw) device?
On Thu, Aug 13, 2015 at 9:44 PM, Austin S Hemmelgarn wrote: > 3. See the warnings about doing block level copies and LVM snapshots of > BTRFS volumes, the same applies to using it on DRBD currently as well (with > the possible exception of remote DRBD nodes (ie, ones without a local copy > of the backing store) (in this case, we need to blacklist backing devices > for stacked storage (I think the same issue may be present with BTRFS on a > MD based RAID1 set). I've been using BTRFS on top of DRBD for several years now, what specifically am I meant to avoid? I have 6 drives mirrored across a local network, this is done with DRBD. At any one time only a single server has the 6 drives mounted with btrfs. Is this a ticking time bomb? -- Gareth Pye - blog.cerberos.id.au Level 2 MTG Judge, Melbourne, Australia "Dear God, I would like to file a bug report" -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: weird qgroup behavior, possible bug? btrfs-progs 4.1.2 and 4.1.5 kernel.
On Thu, Aug 13, 2015 at 11:22 AM, Suman Chakravartula wrote: > Hi, > > I use qgroups for subvolumes in Rockstor and have been noticing this > behavior for a while(at least from 3.18 days). The behavior is that I > get "Disk quota exceeded" errors before even hitting 70% usage. Here's > a simple demonstration of the problem. > > [root@rock-dev ~]# btrfs fi show singlepool > Label: 'singlepool' uuid: 77eb22bf-5f07-4f7e-83af-da183ceccd4d > Total devices 1 FS bytes used 224.00KiB > devid1 size 3.00GiB used 276.00MiB path /dev/sdab > > btrfs-progs v4.1.2 > [root@rock-dev ~]# btrfs subvol list -p /mnt2/singlepool/ > ID 257 gen 9 parent 5 top level 5 path singleshare1 > [root@rock-dev ~]# btrfs qgroup show -pcre /mnt2/singlepool > qgroupid rfer excl max_rfer max_excl parent child > -- - > 0/5 16.00KiB 16.00KiB none none --- --- > 0/25716.00KiB 16.00KiB none none 2015/1 --- > 2015/1 16.00KiB 16.00KiB 1.00GiB none --- 0/257 > > As you can see, the subvolume is part of the 2015/1 qgroup with a 1GiB > max_rfer limit. Now I start writing to the it. > > [root@rock-dev ~]# dd if=/dev/urandom of=/mnt2/singleshare1/file.1 > bs=1M count=256; sync > 256+0 records in > 256+0 records out > 268435456 bytes (268 MB) copied, 15.5902 s, 17.2 MB/s > [root@rock-dev ~]# btrfs fi df /mnt2/singlepool/ > Data, single: total=328.00MiB, used=256.12MiB > System, single: total=4.00MiB, used=16.00KiB > Metadata, single: total=264.00MiB, used=432.00KiB > GlobalReserve, single: total=16.00MiB, used=0.00B > [root@rock-dev ~]# btrfs qgroup show -pcre /mnt2/singlepool > qgroupid rfer excl max_rfer max_excl parent child > -- - > 0/5 16.00KiB 16.00KiB none none --- --- > 0/257 256.02MiB256.02MiB none none 2015/1 --- > 2015/1 256.02MiB256.02MiB 1.00GiB none --- 0/257 > [root@rock-dev ~]# dd if=/dev/urandom of=/mnt2/singleshare1/file.2 > bs=1M count=256; sync > 256+0 records in > 256+0 records out > 268435456 bytes (268 MB) copied, 15.7899 s, 17.0 MB/s > [root@rock-dev ~]# btrfs fi df /mnt2/singlepool/ > Data, single: total=648.00MiB, used=512.19MiB > System, single: total=4.00MiB, used=16.00KiB > Metadata, single: total=264.00MiB, used=688.00KiB > GlobalReserve, single: total=16.00MiB, used=0.00B > [root@rock-dev ~]# btrfs qgroup show -pcre /mnt2/singlepool > qgroupid rfer excl max_rfer max_excl parent child > -- - > 0/5 16.00KiB 16.00KiB none none --- --- > 0/257 512.02MiB512.02MiB none none 2015/1 --- > 2015/1 512.02MiB512.02MiB 1.00GiB none --- 0/257 > > Ok, so far so good. I've written 2 files, 512MiB in total and it's > reflected accurately in the qgroup accounting. Now, I write a 3rd > 256MiB file. > > [root@rock-dev ~]# dd if=/dev/urandom of=/mnt2/singleshare1/file.3 > bs=1M count=256; sync > 256+0 records in > 256+0 records out > 268435456 bytes (268 MB) copied, 15.6963 s, 17.1 MB/s > [root@rock-dev ~]# sync > [root@rock-dev ~]# btrfs fi df /mnt2/singlepool/ > Data, single: total=968.00MiB, used=768.25MiB > System, single: total=4.00MiB, used=16.00KiB > Metadata, single: total=264.00MiB, used=944.00KiB > GlobalReserve, single: total=16.00MiB, used=0.00B > [root@rock-dev ~]# btrfs qgroup show -pcre /mnt2/singlepool > qgroupid rfer excl max_rfer max_excl parent child > -- - > 0/5 16.00KiB 16.00KiB none none --- --- > 0/257 662.00MiB662.00MiB none none 2015/1 --- > 2015/1 662.00MiB662.00MiB 1.00GiB none --- 0/257 > > I find it odd that the usage shows 662.00Mib, I was expecting to see > 768MiB(512+256). Is this because the third 256MiB file was compressed > to 150MiB while the first two files were not compressible? Or is it > something else, and if so, what is it? > > If I attempt to write another 256MiB file, i get this error: > > [root@rock-dev ~]# dd if=/dev/urandom of=/mnt2/singleshare1/file.4 > bs=1M count=256; sync > dd: failed to open ‘/mnt2/singleshare1/file.4’: Disk quota exceeded > > If I try to remove one of the three files written successfully, i get > the same error: > > [root@rock-dev ~]# cd /mnt2/singleshare2/ > [root@rock-dev singleshare2]# ls > file.1 file.2 file.3 > [root@rock-dev singleshare2]# ls -l > total 786432 > -rw-r--r-- 1 root root 268435456 Aug 13 10:49 file.1 > -rw-r--r-- 1 root root 268435456 Aug 13 10:50 file.2 > -rw-r--r-- 1 root root 268435456 Aug 13 10:50 file.3 > [root@rock-dev singleshar
weird qgroup behavior, possible bug? btrfs-progs 4.1.2 and 4.1.5 kernel.
Hi,
I use qgroups for subvolumes in Rockstor and have been noticing this
behavior for a while(at least from 3.18 days). The behavior is that I
get "Disk quota exceeded" errors before even hitting 70% usage. Here's
a simple demonstration of the problem.
[root@rock-dev ~]# btrfs fi show singlepool
Label: 'singlepool' uuid: 77eb22bf-5f07-4f7e-83af-da183ceccd4d
Total devices 1 FS bytes used 224.00KiB
devid1 size 3.00GiB used 276.00MiB path /dev/sdab
btrfs-progs v4.1.2
[root@rock-dev ~]# btrfs subvol list -p /mnt2/singlepool/
ID 257 gen 9 parent 5 top level 5 path singleshare1
[root@rock-dev ~]# btrfs qgroup show -pcre /mnt2/singlepool
qgroupid rfer excl max_rfer max_excl parent child
-- -
0/5 16.00KiB 16.00KiB none none --- ---
0/25716.00KiB 16.00KiB none none 2015/1 ---
2015/1 16.00KiB 16.00KiB 1.00GiB none --- 0/257
As you can see, the subvolume is part of the 2015/1 qgroup with a 1GiB
max_rfer limit. Now I start writing to the it.
[root@rock-dev ~]# dd if=/dev/urandom of=/mnt2/singleshare1/file.1
bs=1M count=256; sync
256+0 records in
256+0 records out
268435456 bytes (268 MB) copied, 15.5902 s, 17.2 MB/s
[root@rock-dev ~]# btrfs fi df /mnt2/singlepool/
Data, single: total=328.00MiB, used=256.12MiB
System, single: total=4.00MiB, used=16.00KiB
Metadata, single: total=264.00MiB, used=432.00KiB
GlobalReserve, single: total=16.00MiB, used=0.00B
[root@rock-dev ~]# btrfs qgroup show -pcre /mnt2/singlepool
qgroupid rfer excl max_rfer max_excl parent child
-- -
0/5 16.00KiB 16.00KiB none none --- ---
0/257 256.02MiB256.02MiB none none 2015/1 ---
2015/1 256.02MiB256.02MiB 1.00GiB none --- 0/257
[root@rock-dev ~]# dd if=/dev/urandom of=/mnt2/singleshare1/file.2
bs=1M count=256; sync
256+0 records in
256+0 records out
268435456 bytes (268 MB) copied, 15.7899 s, 17.0 MB/s
[root@rock-dev ~]# btrfs fi df /mnt2/singlepool/
Data, single: total=648.00MiB, used=512.19MiB
System, single: total=4.00MiB, used=16.00KiB
Metadata, single: total=264.00MiB, used=688.00KiB
GlobalReserve, single: total=16.00MiB, used=0.00B
[root@rock-dev ~]# btrfs qgroup show -pcre /mnt2/singlepool
qgroupid rfer excl max_rfer max_excl parent child
-- -
0/5 16.00KiB 16.00KiB none none --- ---
0/257 512.02MiB512.02MiB none none 2015/1 ---
2015/1 512.02MiB512.02MiB 1.00GiB none --- 0/257
Ok, so far so good. I've written 2 files, 512MiB in total and it's
reflected accurately in the qgroup accounting. Now, I write a 3rd
256MiB file.
[root@rock-dev ~]# dd if=/dev/urandom of=/mnt2/singleshare1/file.3
bs=1M count=256; sync
256+0 records in
256+0 records out
268435456 bytes (268 MB) copied, 15.6963 s, 17.1 MB/s
[root@rock-dev ~]# sync
[root@rock-dev ~]# btrfs fi df /mnt2/singlepool/
Data, single: total=968.00MiB, used=768.25MiB
System, single: total=4.00MiB, used=16.00KiB
Metadata, single: total=264.00MiB, used=944.00KiB
GlobalReserve, single: total=16.00MiB, used=0.00B
[root@rock-dev ~]# btrfs qgroup show -pcre /mnt2/singlepool
qgroupid rfer excl max_rfer max_excl parent child
-- -
0/5 16.00KiB 16.00KiB none none --- ---
0/257 662.00MiB662.00MiB none none 2015/1 ---
2015/1 662.00MiB662.00MiB 1.00GiB none --- 0/257
I find it odd that the usage shows 662.00Mib, I was expecting to see
768MiB(512+256). Is this because the third 256MiB file was compressed
to 150MiB while the first two files were not compressible? Or is it
something else, and if so, what is it?
If I attempt to write another 256MiB file, i get this error:
[root@rock-dev ~]# dd if=/dev/urandom of=/mnt2/singleshare1/file.4
bs=1M count=256; sync
dd: failed to open ‘/mnt2/singleshare1/file.4’: Disk quota exceeded
If I try to remove one of the three files written successfully, i get
the same error:
[root@rock-dev ~]# cd /mnt2/singleshare2/
[root@rock-dev singleshare2]# ls
file.1 file.2 file.3
[root@rock-dev singleshare2]# ls -l
total 786432
-rw-r--r-- 1 root root 268435456 Aug 13 10:49 file.1
-rw-r--r-- 1 root root 268435456 Aug 13 10:50 file.2
-rw-r--r-- 1 root root 268435456 Aug 13 10:50 file.3
[root@rock-dev singleshare2]# rm file.3
rm: remove regular file ‘file.3’? y
rm: cannot remove ‘file.3’: Disk quota exceeded
I was able to reproduce this behavior with other raid profiles as well
and recently, some of Rockstor users have also confirmed it with
subvolume sizes smal
Re: RAID0 wrong (raw) device?
On 08/13/2015 10:55 PM, Ulli Horlacher wrote: On Thu 2015-08-13 (14:02), Ulli Horlacher wrote: On Thu 2015-08-13 (15:34), anand jain wrote: root@toy02:~# df -T /data Filesystem Type 1K-blocks Used Available Use% Mounted on /dev/sdb btrfs 3906909856 140031696 3765056176 4% /data root@toy02:~# btrfs filesystem show /data Label: data uuid: 411af13f-6cae-4f03-99dc-5941acb3135b Total devices 2 FS bytes used 129.81GiB devid3 size 1.82TiB used 67.03GiB path /dev/drbd2 devid4 size 1.82TiB used 67.03GiB path /dev/sdb Btrfs v3.12 ==> btrfs shows the wrong (raw) device /dev/sdb instead of /dev/drbd3 ! Don't be too alarmed by that, progs do a bit of user land fabrication (wrong). kernel may /may-not be using sdb. try -m option. It is really weird: meanwhile (without any mount change or even reboot) I get: root@toy02:~# btrfs filesystem show Label: data uuid: 411af13f-6cae-4f03-99dc-5941acb3135b Total devices 2 FS bytes used 106.51GiB devid3 size 1.82TiB used 82.03GiB path /dev/drbd2 devid4 size 1.82TiB used 82.03GiB path /dev/drbd3 And now, after a reboot: root@toy02:~/bin# btrfs filesystem show Label: data uuid: 411af13f-6cae-4f03-99dc-5941acb3135b Total devices 2 FS bytes used 119.82GiB devid3 size 1.82TiB used 82.03GiB path /dev/drbd2 devid4 size 1.82TiB used 82.03GiB path /dev/sde GRMPF! pls use 'btrfs fi show -m' and just ignore no option or -d if fs is mounted, as -m reads from the kernel. at mount you could assemble correct set of devices using mount -o device option. Thanks, -Anand -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RAID0 wrong (raw) device?
On Thu 2015-08-13 (14:02), Ulli Horlacher wrote: > On Thu 2015-08-13 (15:34), anand jain wrote: > > > > root@toy02:~# df -T /data > > > Filesystem Type 1K-blocks Used Available Use% Mounted on > > > /dev/sdb btrfs 3906909856 140031696 3765056176 4% /data > > > > > > root@toy02:~# btrfs filesystem show /data > > > Label: data uuid: 411af13f-6cae-4f03-99dc-5941acb3135b > > > Total devices 2 FS bytes used 129.81GiB > > > devid3 size 1.82TiB used 67.03GiB path /dev/drbd2 > > > devid4 size 1.82TiB used 67.03GiB path /dev/sdb > > > > > > Btrfs v3.12 > > > > > > ==> btrfs shows the wrong (raw) device /dev/sdb instead of /dev/drbd3 ! > > > > Don't be too alarmed by that, progs do a bit of user land fabrication > > (wrong). kernel may /may-not be using sdb. try -m option. > > It is really weird: meanwhile (without any mount change or even reboot) I > get: > > root@toy02:~# btrfs filesystem show > Label: data uuid: 411af13f-6cae-4f03-99dc-5941acb3135b > Total devices 2 FS bytes used 106.51GiB > devid3 size 1.82TiB used 82.03GiB path /dev/drbd2 > devid4 size 1.82TiB used 82.03GiB path /dev/drbd3 And now, after a reboot: root@toy02:~/bin# btrfs filesystem show Label: data uuid: 411af13f-6cae-4f03-99dc-5941acb3135b Total devices 2 FS bytes used 119.82GiB devid3 size 1.82TiB used 82.03GiB path /dev/drbd2 devid4 size 1.82TiB used 82.03GiB path /dev/sde GRMPF! -- Ullrich Horlacher Server und Virtualisierung Rechenzentrum IZUS/TIK E-Mail: horlac...@tik.uni-stuttgart.de Universitaet Stuttgart Tel:++49-711-68565868 Allmandring 30aFax:++49-711-682357 70550 Stuttgart (Germany) WWW:http://www.tik.uni-stuttgart.de/ REF:<20150813120211.ga24...@rus.uni-stuttgart.de> -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 2/3] xfstests: btrfs: test device replace, with EIO on the src dev
So the test always fails due to a mismatch with this expected golden output: -Label: none uuid: +Label: none uuid: Total devices FS bytes used devid size used path SCRATCH_DEV devid size used path /dev/mapper/error-test The extra space after "uuid:" comes from _filter_uuid: sed -e "s/\(uuid[ :=]\+\) *[0-9a-f-][0-9a-f-]*/\1 /ig" Which gets \1 with the string "uuid: " and then does "uuid: " + " " + "". We need to either to fix the golden output here or _filter_uuid (and make sure it doesn't break any other tests using it directly or indirectly such as yours). commit a092363bbdfa6cc6e44353136bae9d3aa81baae2 (PATCH 3/3 V6] xfs: test changing UUID on V5 superblock) caused the regression. btrfs/006 is affected as well. Thanks Anand -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 1/3] xfstests: btrfs: add functions to create dm-error device
+# this test requires the device mapper error target
+#
+_require_dmerror()
+{
+_require_command "$DMSETUP_PROG" dmsetup
+
+$DMSETUP_PROG targets | grep error >/dev/null 2>&1
+if [ $? -eq 0 ]
+then
+ :
+else
+ _notrun "This test requires dm error support"
+fi
Why not just:
[ $? -ne 0 ] && _notrun "This test requires dm error support"
The empty branch doesn't make much sense.
yes.
Also, please indent the body of this function using an 8 spaces tab,
which is the official style for fstests (just look at the surrounding
functions for example).
not sure how this got slipped. I am fixing it.
Other than that, it looks good to me. You can add:
Reviewed-by: Filipe Manana
Thanks.
Anand
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 3/3] xfstests: btrfs: test device delete with EIO on src dev
Hi Filipe, Any reason to not run here fsstress like the test from patch 2? Doing the device delete with a non-empty fs is a lot more interesting and can help find bugs and regressions in the future. You are reviewing v2. fsstress was added in v4. Also this test needs the latest btrfs-progs and kernel patch Not patch, but patch set instead. I was looking for a patch with that title and didn't found any, only a cover letter with the subject: "[PATCH 0/8 v2] device delete by devid". under title [PATCH] device delete by devid will update comments. When this patch is not found in the btrfs-progs, this test will not run. However when the require patch is not found in the kernel it will fail gracefully. What's the status of these patches (both kernel and btrfs-progs)? I've noticed they've been around since april and no feedback on the mailing list. Thanks, Anand -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RAID0 wrong (raw) device?
On Thu 2015-08-13 (07:44), Austin S Hemmelgarn wrote: > 2. Be _VERY_ careful using BTRFS on top of _ANY_ kind of shared storage. > Most non-clustered filesystems will have issues if multiply mounted, > but in almost all cases I've personally seen, it _WILL_ cause > irreparable damage to a BTRFS filesystem (we really need to do something > like ext4's MMP in BTRFS). Same with ZFS: it has also no MMP and when you mount a shared block device twice you will destroy it irreparable. Kids, do not try this at home - I have done so ;-) -- Ullrich Horlacher Server und Virtualisierung Rechenzentrum IZUS/TIK E-Mail: horlac...@tik.uni-stuttgart.de Universitaet Stuttgart Tel:++49-711-68565868 Allmandring 30aFax:++49-711-682357 70550 Stuttgart (Germany) WWW:http://www.tik.uni-stuttgart.de/ REF:<55cc830d.2070...@gmail.com> -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RAID0 wrong (raw) device?
On Thu 2015-08-13 (15:34), anand jain wrote: > > root@toy02:~# df -T /data > > Filesystem Type 1K-blocks Used Available Use% Mounted on > > /dev/sdb btrfs 3906909856 140031696 3765056176 4% /data > > > > root@toy02:~# btrfs filesystem show /data > > Label: data uuid: 411af13f-6cae-4f03-99dc-5941acb3135b > > Total devices 2 FS bytes used 129.81GiB > > devid3 size 1.82TiB used 67.03GiB path /dev/drbd2 > > devid4 size 1.82TiB used 67.03GiB path /dev/sdb > > > > Btrfs v3.12 > > > > ==> btrfs shows the wrong (raw) device /dev/sdb instead of /dev/drbd3 ! > > Don't be too alarmed by that, progs do a bit of user land fabrication > (wrong). kernel may /may-not be using sdb. try -m option. It is really weird: meanwhile (without any mount change or even reboot) I get: root@toy02:~# btrfs filesystem show Label: data uuid: 411af13f-6cae-4f03-99dc-5941acb3135b Total devices 2 FS bytes used 106.51GiB devid3 size 1.82TiB used 82.03GiB path /dev/drbd2 devid4 size 1.82TiB used 82.03GiB path /dev/drbd3 ==> no more /dev/sdb ! BUT: root@toy02:~# df -T /data Filesystem Type 1K-blocks Used Available Use% Mounted on /dev/sdb btrfs 3906909856 111822636 3793208212 3% /data root@toy02:~# mount | grep /data /dev/sdb on /data type btrfs (rw) root@toy02:~# grep /data /proc/mounts /dev/drbd2 /data btrfs rw,relatime,space_cache 0 0 And still, Linux sees 3 HGST devices (there are real 2 drives): root@toy02:~# hdparm -I /dev/sdb | grep Number: Model Number: HGST HUS724020ALA640 Serial Number: PN2134P5G2P2AX root@toy02:~# hdparm -I /dev/sdd | grep Number: Model Number: HGST HUS724020ALA640 Serial Number: PN2134P5G2P2XX root@toy02:~# hdparm -I /dev/sde | grep Number: Model Number: HGST HUS724020ALA640 Serial Number: PN2134P5G2P2AX -- Ullrich Horlacher Server und Virtualisierung Rechenzentrum IZUS/TIK E-Mail: horlac...@tik.uni-stuttgart.de Universitaet Stuttgart Tel:++49-711-68565868 Allmandring 30aFax:++49-711-682357 70550 Stuttgart (Germany) WWW:http://www.tik.uni-stuttgart.de/ REF:<55cc488d.4020...@oracle.com> -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RAID0 wrong (raw) device?
On Wed 2015-08-12 (11:03), Chris Murphy wrote: > On Wed, Aug 12, 2015 at 7:07 AM, Ulli Horlacher > wrote: > > > /dev/sdb and /dev/sde are in reality the same physical disk! > > When does all of this confusion happen? Is it already confused before > mkfs or only after mkfs or only after mount? I have not looked closely before the mkfs.btrfs, because I noticed no problems. > I would find out what instigates it, wipe all signatures from everything, > reboot, start from scratch This is not an option for me, because this server (despite its name) is a production system. -- Ullrich Horlacher Server und Virtualisierung Rechenzentrum IZUS/TIK E-Mail: horlac...@tik.uni-stuttgart.de Universitaet Stuttgart Tel:++49-711-68565868 Allmandring 30aFax:++49-711-682357 70550 Stuttgart (Germany) WWW:http://www.tik.uni-stuttgart.de/ REF: -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RAID0 wrong (raw) device?
A couple of observations: 1. BTRFS currently has no knowledge of multipath or anything like that. In theory it should work fine as long as the multiple device instances all point to the same storage directly (including having identical block addresses), but we still need to add proper handling for it. 2. Be _VERY_ careful using BTRFS on top of _ANY_ kind of shared storage. Most non-clustered filesystems will have issues if multiply mounted, but in almost all cases I've personally seen, it _WILL_ cause irreparable damage to a BTRFS filesystem (we really need to do something like ext4's MMP in BTRFS). 3. See the warnings about doing block level copies and LVM snapshots of BTRFS volumes, the same applies to using it on DRBD currently as well (with the possible exception of remote DRBD nodes (ie, ones without a local copy of the backing store) (in this case, we need to blacklist backing devices for stacked storage (I think the same issue may be present with BTRFS on a MD based RAID1 set). smime.p7s Description: S/MIME Cryptographic Signature
Re: bedup --defrag freezing
On 2015-08-12 15:30, Chris Murphy wrote: On Wed, Aug 12, 2015 at 12:44 PM, Konstantin Svist wrote: On 08/06/2015 04:10 AM, Austin S Hemmelgarn wrote: On 2015-08-05 17:45, Konstantin Svist wrote: Hi, I've been running btrfs on Fedora for a while now, with bedup --defrag running in a night-time cronjob. Last few runs seem to have gotten stuck, without possibility of even killing the process (kill -9 doesn't work) -- all I could do is hard power cycle. Did something change recently? Is bedup simply too out of date? What should I use to de-duplicate across snapshots instead? Etc.? AFAIK, bedup hasn't been actively developed for quite a while (I'm actually kind of surprised it runs with the newest btrfs-progs). Personally, I'd suggest using duperemove (https://github.com/markfasheh/duperemove) Thanks, good to know. Tried duperemove -- it looks like it builds a database of its own checksums every time it runs... why won't it use BTRFS internal checksums for fast rejection? Would run a LOT faster... I think the reason is duperremove does extent based deduplication. Where Btrfs checksums are 4KiB block based, not extent based. And so many 4KiB CRC32C checksums would need to be in memory, that could be kinda expensive. And also, I don't know if CRC32C checksums have essentially no practical chance of collision. If it's really rare, rather than "so improbable as to be impossible" then you could end up with "really rare" corruption where incorrect deduplication happens. Yeah, duperemove doesn't use them because of the memory limitations. Theoretically it's possible to take the the CRC checksums of the individual blocks and then combine them to get a checksum of the blocks as a whold, but it really isn't worth it for that (it would take just about as long as the current hashing. As for the collision properties of CRC32C, it's actually almost trivial to construct collisions. The reason that it is used in BTRFS is because there is a functional guarantee that any single bit error in a block _will_ result in a different CRC, and most larger errors will also. In other words, the usage of CRC32C in BTRFS is for error detection and because it's ridiculously fast on all modern processors. As far as the possibility of incorrect deduplication, the kernel does a bytewise comparison of the extents submitted before actually deduplicating them, so there's no chance (barring hardware issues and/or external influence from a ill-intentioned third-party) of it happening. Because of this, you could theoretically just call the ioctl on every possible combination of extents in the FS, but that would take a ridiculous amount of time (especially because calls involving the same byte ranges get internally serialized by the kernel), which is why we have programs like duperemove (while the hashing has to read all the data too, it's still a lot faster than just comparing all of it directly). There was a patch late last year I think to re-introduce sha256 hash as the checksum, but as far as I know it's not in btrfs-progs yet. I forget if that's file, extent or block based. I'm pretty sure that that patch never made it into the kernel (the original one was for the kernel, not the userspace programs, and it never got brought in because the argument for it (better protection against malicious intent) was inherently invalid for the usage of checksums in BTRFS (if someone can rewrite your data arbitrarily on disk, they can do so for the checksums also)), and that it was block based (and as such less useful for deduplication than the CRC32C that we are currently using). smime.p7s Description: S/MIME Cryptographic Signature
Re: Oddness with phantom device replacing real device.
On Thu, Aug 13, 2015 at 01:33:22PM +1000, David Seikel wrote: > I don't actually think that this is a BTRFS problem, but it's showing > symptoms within BTRFS, and I have no other clues, so maybe the BTRFS > experts can help me figure out what is actually going wrong. > > I'm a sysadmin working for a company that does scientific modelling. > They have many TBs of data. We use two servers running Ubuntu 14.04 LTS > to backup all of this data. One of them includes 16 spinning rust > disks hooked to a RAID controller running in JBOD mode (in other words, > as far as Linux is concerned, they are just 16 ordinary disks). They > are /dev/sdc to /dev/sdr, all being used as a single BTRFS file system. > > I have been having no end of trouble with this system recently. Keep > in mind that due to the huge amount of data we deal with, doing > anything takes a long time. So "recently" means "in the last several > months". > > My latest attempt to beat some sense into this server was to upgrade it > to the latest officially backported kernel from Ubuntu, and compile my > own copy of btrfs-progs from source code (latest release from github). > Then I recreated the 16 disk BTRFS file system, and started the backup > software running again, from scratch. The next day, /dev/sdc has > vanished, to be replaced be a phantom /dev/sds. There's no such disk > as /dev/sds. /dev/sds is now included in the BTRFS file system > replacing /dev/sdc. In /dev sdc does indeed vanish, and sds does > indeed appear. This was happening before. /dev/sds then starts to > fill up with errors, since no such disk actually exists. Sounds like the kind of behaviour when the disk has vanished from the system for long enough to drop out and be recreated by the driver. The renaming may (possibly) be down to a poor error-handling path in btrfs -- we see this happening on USB sometimes, where the original device node is still hung on to by the FS on a hardware error, and so when the device comes back it's given a different name. > I don't know what is actually causing the problem. The disks are in a > hot swap backplane, and if I actually pulled sdc out, then it would > still be listed as part of the BTRFS file system, wouldn't it? With btrfs fi show, no, you'd get ** some devices missing ** in the output. > If I > then where to plug some new disk into the same spot, it would not be > recognised as part of the file system? Correct... Unless the device had a superblock with the same UUID in it (like, say, the new device is just the old one reappearing again). In that case, udev would trigger a btrfs dev scan, and the "new" device would rejoin the FS -- probably a little out of date, but that would be caught by checksums and be fixed if you have redundancy in the storage. > So assuming that the RAID > controller is getting confused and thinking that sdc has been pulled, > then replaced by sds, it should not be showing up as part of the BTRFS > file system? Or maybe there's a signature on sdc that BTRFS notices > makes it part of the file system, even though BTRFS is now confused > about it's location? See above. > After a reboot, sdc returns and sds is gone again. Expected. > The RAID controller has recently been replaced, but there where similar > problems with the old one as well. A better model of RAID controller > was chosen this time. > > I've also not been able to complete a scrub on this system recently. > The really odd thing is that I get messages that the scrub has aborted, > yet the scrub continues, then much later (days later) the scrub causes > a kernel panic. The "aborted" happens some random time into the scrub, > but usually in the early part of the scrub. Mind you, if BTRFS is > completely confused due to a problem elsewhere, then maybe this can be > excused. I think that means that it's aborting on one device but continuing on all the others. > The other backup server is almost identical, though it has less disks > in the array. It doesn't have any issues with the BTRFS file system. > > Can any one help shed some light on this please? Hopefully some > "quick" things to try, given my definition of "recently" above means > that most things take days or weeks, or even months for me to try. > > I have attached the usual debugging info requested. This is after the > bogus sds replaces sdc. > The first thing would be to check your system logs for signs of hardware problems (ATA errors). This sounds a lot like you've got a dodgy disk that needs to be replaced. Hugo. -- Hugo Mills | A gentleman doesn't do damage unless he's paid for hugo@... carfax.org.uk | it. http://carfax.org.uk/ | PGP: E2AB1DE4 |Juri Papay signature.asc Description: Digital signature
Re: [PATCH v2] fstests: generic/018: expand "write backwards sync but contiguous" to test regression in btrfs
On Thu, Aug 13, 2015 at 10:43 AM, Filipe David Manana wrote: > On Thu, Aug 13, 2015 at 9:47 AM, Liu Bo wrote: >> Btrfs has a problem when defraging a file which has a large fragment'ed >> range, >> it'd leave the tail extent as a seperate extent instead of merging it with >> previous extents. >> >> This makes generic/018 recognize the above regression. >> >> Meanwhile, I find that in the case of 'write backwards sync but contiguous", >> ext4 doesn't produce fragments like btrfs and xfs, so I modify 018.out a >> little >> bit to let ext4 pass. >> >> Moreover, I follow Filipe's suggestion to filter xfs_io's output in order to >> check these writes actually succeed. >> >> Signed-off-by: Liu Bo > > Reviewed-by: Filipe Manana > > The lines with XFS_IO_PROG are now wider than 80 characters (the echo > command lines were already wider than 80 too). But other than that all > looks good to me. > Test fails with the btrfs kernel fix, test passes with the fix applied Err, typo, should have been "test fails without the btrfs kernel fix..." > and ext4/xfs continue to pass here. > Thanks. > >> --- >> v2: fix typo in title, s/expend/expand/g >> >> tests/generic/018 | 16 ++-- >> tests/generic/018.out | 198 >> +- >> 2 files changed, 203 insertions(+), 11 deletions(-) >> >> diff --git a/tests/generic/018 b/tests/generic/018 >> index d97bb88..3693874 100755 >> --- a/tests/generic/018 >> +++ b/tests/generic/018 >> @@ -68,28 +68,24 @@ $XFS_IO_PROG -f -c "truncate 1m" $fragfile >> _defrag --before 0 --after 0 $fragfile >> >> echo "Contiguous file:" | tee -a $seqres.full >> -$XFS_IO_PROG -f -c "pwrite -b $((4 * bsize)) 0 $((4 * bsize))" $fragfile \ >> - > /dev/null >> +$XFS_IO_PROG -f -c "pwrite -b $((4 * bsize)) 0 $((4 * bsize))" $fragfile | >> _filter_xfs_io >> _defrag --before 1 --after 1 $fragfile >> >> echo "Write backwards sync, but contiguous - should defrag to 1 extent" | >> tee -a $seqres.full >> -for i in `seq 9 -1 0`; do >> - $XFS_IO_PROG -fs -c "pwrite -b $bsize $((i * bsize)) $bsize" >> $fragfile \ >> - > /dev/null >> +for i in `seq 64 -1 0`; do >> + $XFS_IO_PROG -fd -c "pwrite -b $bsize $((i * bsize)) $bsize" >> $fragfile | _filter_xfs_io >> done >> -_defrag --before 10 --after 1 $fragfile >> +_defrag --after 1 $fragfile >> >> echo "Write backwards sync leaving holes - defrag should do nothing" | tee >> -a $seqres.full >> for i in `seq 31 -2 0`; do >> - $XFS_IO_PROG -fs -c "pwrite -b $bsize $((i * bsize)) $bsize" >> $fragfile \ >> - > /dev/null >> + $XFS_IO_PROG -fs -c "pwrite -b $bsize $((i * bsize)) $bsize" >> $fragfile | _filter_xfs_io >> done >> _defrag --before 16 --after 16 $fragfile >> >> echo "Write forwards sync leaving holes - defrag should do nothing" | tee >> -a $seqres.full >> for i in `seq 0 2 31`; do >> - $XFS_IO_PROG -fs -c "pwrite -b $bsize $((i * bsize)) $bsize" >> $fragfile \ >> - > /dev/null >> + $XFS_IO_PROG -fs -c "pwrite -b $bsize $((i * bsize)) $bsize" >> $fragfile | _filter_xfs_io >> done >> _defrag --before 16 --after 16 $fragfile >> >> diff --git a/tests/generic/018.out b/tests/generic/018.out >> index 5f265d1..0886a9a 100644 >> --- a/tests/generic/018.out >> +++ b/tests/generic/018.out >> @@ -6,14 +6,210 @@ Sparse file (no blocks): >> Before: 0 >> After: 0 >> Contiguous file: >> +wrote 16384/16384 bytes at offset 0 >> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) >> Before: 1 >> After: 1 >> Write backwards sync, but contiguous - should defrag to 1 extent >> -Before: 10 >> +wrote 4096/4096 bytes at offset 262144 >> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) >> +wrote 4096/4096 bytes at offset 258048 >> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) >> +wrote 4096/4096 bytes at offset 253952 >> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) >> +wrote 4096/4096 bytes at offset 249856 >> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) >> +wrote 4096/4096 bytes at offset 245760 >> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) >> +wrote 4096/4096 bytes at offset 241664 >> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) >> +wrote 4096/4096 bytes at offset 237568 >> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) >> +wrote 4096/4096 bytes at offset 233472 >> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) >> +wrote 4096/4096 bytes at offset 229376 >> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) >> +wrote 4096/4096 bytes at offset 225280 >> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) >> +wrote 4096/4096 bytes at offset 221184 >> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) >> +wrote 4096/4096 bytes at offset 217088 >> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/s
Re: trim not working and irreparable errors from btrfsck
Am Sun, 21 Jun 2015 07:21:03 + schrieb Paul Jones : > > -Original Message- > > From: Lutz Euler [mailto:lutz.eu...@freenet.de] > > Sent: Sunday, 21 June 2015 12:11 AM > > To: Christian; Paul Jones; Austin S Hemmelgarn > > Cc: linux-btrfs@vger.kernel.org > > Subject: RE: trim not working and irreparable errors from btrfsck > > > > Hi Christian, Paul and Austin, > > > > Christian wrote: > > > However, fstrim still gives me "0 B (0 bytes) trimmed, so that may be > > > another problem. Is there a way to check if trim works? > > > > Paul wrote: > > > I've got the same problem. I've got 2 SSDs with 2 partitions in RAID1, > > > fstrim always works on the 2nd partition but not the first. There are > > > no errors on either filesystem that I know of, but the first one is > > > root so I can't take it offline to run btrfs check. > > > > Austin wrote: > > > I'm seeing the same issue here, but with a Crucial brand SSD. > > > Somewhat interestingly, I don't see any issues like this with BTRFS on > > > top of LVM's thin-provisioning volumes, or with any other filesystems, > > > so I think it has something to do with how BTRFS is reporting unused > > > space or how it is submitting the discard requests. > > > > Probably you all suffer from the same problem I had a few years ago. > > It is a bug in how btrfs implements fstrim. > > > > To check whether you are a victim of this bug simply run: > > > > # btrfs-debug-tree /dev/whatever | grep 'FIRST_CHUNK_TREE > > CHUNK_ITEM' > > > > where /dev/whatever is a device of your filesystem, and interrupt after the > > first several output lines with C-c. (Officially the filesystem should be > > unmounted when running btrfs-debug-tree, but that is not necessary as we > > only read from it and the relevant data doesn't change very often.) > > > > You get something like: > > > > item 2 key (FIRST_CHUNK_TREE CHUNK_ITEM 0) > > item 3 key (FIRST_CHUNK_TREE CHUNK_ITEM 12947816448) > > item 4 key (FIRST_CHUNK_TREE CHUNK_ITEM 14021558272) > > ... > > > > (This output is from an old version of btrfs-progs. I understand newer > > version > > are more verbose, but you should nevertheless easily be able to interpret > > the output). > > > > If the first number different from 0 (here, the 12947816448) is larger than > > the > > sum of the sizes of the devices the filesystem consists of, bingo. > > > > This has been discussed already in the past and there is a patch. > > > > Please see for the patch: > > http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg40618.html > > > > and for the background: > > http://comments.gmane.org/gmane.comp.file-systems.btrfs/15597 > > > > Kind regards, > > > > Lutz Euler > > I tried the test and the numbers I was getting seemed reasonable, however I > went ahead and applied the patch anyway. Trim now works correctly! > > Thanks, > Paul. > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in Speaking as a user, since "fstrim -av" still always outputs 0 bytes trimmed on my system: what's the status of this? Did anybody ever file a bug report? There was also that other thread, "fstrim not working on one of three BTRFS filesystems", that also never went anywhere. I take it from this that my SSD has been running untrimmed for quite a while now? (FWIW, queued trim is blocked by my kernel (it's "forced_unqueued"), but fstrim should still start an unqueued trim, right?) # uname -a Linux thetick 4.1.4-gentoo #1 SMP PREEMPT Tue Aug 4 21:58:41 CEST 2015 x86_64 AMD Athlon(tm) 64 X2 Dual Core Processor 4200+ AuthenticAMD GNU/Linux # btrfs --version btrfs-progs v4.1.2 # btrfs filesystem show Label: 'MARCEC_ROOT' uuid: 0267d8b3-a074-460a-832d-5d5fd36bae64 Total devices 1 FS bytes used 56.59GiB devid1 size 107.79GiB used 69.03GiB path /dev/sda1 Label: 'MARCEC_STORAGE' uuid: 472c9290-3ff2-4096-9c47-0612d3a52cef Total devices 2 FS bytes used 597.75GiB devid1 size 931.51GiB used 600.03GiB path /dev/sdc devid2 size 931.51GiB used 600.03GiB path /dev/sdb Label: 'MARCEC_BACKUP' uuid: f97b3cda-15e8-418b-bb9b-235391ef2a38 Total devices 1 FS bytes used 807.59GiB devid1 size 976.56GiB used 837.06GiB path /dev/sdd2 btrfs-progs v4.1.2 # btrfs filesystem df / Data, single: total=65.00GiB, used=54.83GiB System, single: total=32.00MiB, used=16.00KiB Metadata, single: total=4.00GiB, used=1.76GiB GlobalReserve, single: total=512.00MiB, used=0.00B Greetings -- Marc Joliet -- "People who think they know everything really annoy those of us who know we don't" - Bjarne Stroustrup pgpe5zrE9jGKM.pgp Description: Digitale Signatur von OpenPGP
Re: [PATCH v2] fstests: generic/018: expand "write backwards sync but contiguous" to test regression in btrfs
On Thu, Aug 13, 2015 at 9:47 AM, Liu Bo wrote: > Btrfs has a problem when defraging a file which has a large fragment'ed range, > it'd leave the tail extent as a seperate extent instead of merging it with > previous extents. > > This makes generic/018 recognize the above regression. > > Meanwhile, I find that in the case of 'write backwards sync but contiguous", > ext4 doesn't produce fragments like btrfs and xfs, so I modify 018.out a > little > bit to let ext4 pass. > > Moreover, I follow Filipe's suggestion to filter xfs_io's output in order to > check these writes actually succeed. > > Signed-off-by: Liu Bo Reviewed-by: Filipe Manana The lines with XFS_IO_PROG are now wider than 80 characters (the echo command lines were already wider than 80 too). But other than that all looks good to me. Test fails with the btrfs kernel fix, test passes with the fix applied and ext4/xfs continue to pass here. Thanks. > --- > v2: fix typo in title, s/expend/expand/g > > tests/generic/018 | 16 ++-- > tests/generic/018.out | 198 > +- > 2 files changed, 203 insertions(+), 11 deletions(-) > > diff --git a/tests/generic/018 b/tests/generic/018 > index d97bb88..3693874 100755 > --- a/tests/generic/018 > +++ b/tests/generic/018 > @@ -68,28 +68,24 @@ $XFS_IO_PROG -f -c "truncate 1m" $fragfile > _defrag --before 0 --after 0 $fragfile > > echo "Contiguous file:" | tee -a $seqres.full > -$XFS_IO_PROG -f -c "pwrite -b $((4 * bsize)) 0 $((4 * bsize))" $fragfile \ > - > /dev/null > +$XFS_IO_PROG -f -c "pwrite -b $((4 * bsize)) 0 $((4 * bsize))" $fragfile | > _filter_xfs_io > _defrag --before 1 --after 1 $fragfile > > echo "Write backwards sync, but contiguous - should defrag to 1 extent" | > tee -a $seqres.full > -for i in `seq 9 -1 0`; do > - $XFS_IO_PROG -fs -c "pwrite -b $bsize $((i * bsize)) $bsize" > $fragfile \ > - > /dev/null > +for i in `seq 64 -1 0`; do > + $XFS_IO_PROG -fd -c "pwrite -b $bsize $((i * bsize)) $bsize" > $fragfile | _filter_xfs_io > done > -_defrag --before 10 --after 1 $fragfile > +_defrag --after 1 $fragfile > > echo "Write backwards sync leaving holes - defrag should do nothing" | tee > -a $seqres.full > for i in `seq 31 -2 0`; do > - $XFS_IO_PROG -fs -c "pwrite -b $bsize $((i * bsize)) $bsize" > $fragfile \ > - > /dev/null > + $XFS_IO_PROG -fs -c "pwrite -b $bsize $((i * bsize)) $bsize" > $fragfile | _filter_xfs_io > done > _defrag --before 16 --after 16 $fragfile > > echo "Write forwards sync leaving holes - defrag should do nothing" | tee -a > $seqres.full > for i in `seq 0 2 31`; do > - $XFS_IO_PROG -fs -c "pwrite -b $bsize $((i * bsize)) $bsize" > $fragfile \ > - > /dev/null > + $XFS_IO_PROG -fs -c "pwrite -b $bsize $((i * bsize)) $bsize" > $fragfile | _filter_xfs_io > done > _defrag --before 16 --after 16 $fragfile > > diff --git a/tests/generic/018.out b/tests/generic/018.out > index 5f265d1..0886a9a 100644 > --- a/tests/generic/018.out > +++ b/tests/generic/018.out > @@ -6,14 +6,210 @@ Sparse file (no blocks): > Before: 0 > After: 0 > Contiguous file: > +wrote 16384/16384 bytes at offset 0 > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > Before: 1 > After: 1 > Write backwards sync, but contiguous - should defrag to 1 extent > -Before: 10 > +wrote 4096/4096 bytes at offset 262144 > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > +wrote 4096/4096 bytes at offset 258048 > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > +wrote 4096/4096 bytes at offset 253952 > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > +wrote 4096/4096 bytes at offset 249856 > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > +wrote 4096/4096 bytes at offset 245760 > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > +wrote 4096/4096 bytes at offset 241664 > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > +wrote 4096/4096 bytes at offset 237568 > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > +wrote 4096/4096 bytes at offset 233472 > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > +wrote 4096/4096 bytes at offset 229376 > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > +wrote 4096/4096 bytes at offset 225280 > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > +wrote 4096/4096 bytes at offset 221184 > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > +wrote 4096/4096 bytes at offset 217088 > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > +wrote 4096/4096 bytes at offset 212992 > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > +wrote 4096/4096 bytes at offset 208896 > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > +wrote 4096/4096 bytes at offset 204800
Re: Deleted files cause btrfs-send to fail
Am Thu, 13 Aug 2015 08:29:19 + (UTC) schrieb Duncan <1i5t5.dun...@cox.net>: > Marc Joliet posted on Thu, 13 Aug 2015 09:05:41 +0200 as excerpted: > > > Here's the actual output now, obtained via btrfs-progs 4.0.1 from an > > initramfs emergency shell: > > > > checking extents checking free space cache checking fs roots root 5 > > inode 8338813 errors 2000, link count wrong > > unresolved ref dir 26699 index 50500 namelen 4 name root > > filetype 0 errors 3, no dir item, no dir index > > root 5 inode 8338814 errors 2000, link count wrong > > unresolved ref dir 26699 index 50502 namelen 6 name marcec > > filetype 0 errors 3, no dir item, no dir index > > root 5 inode 8338815 errors 2000, link count wrong > > unresolved ref dir 26699 index 50504 namelen 6 name systab > > filetype 0 errors 3, no dir item, no dir index > > root 5 inode 8710030 errors 2000, link count wrong > > unresolved ref dir 26699 index 59588 namelen 6 name marcec > > filetype 0 errors 3, no dir item, no dir index > > root 5 inode 8710031 errors 2000, link count wrong > > unresolved ref dir 26699 index 59590 namelen 4 name root > > filetype 0 errors 3, no dir item, no dir index > > Checking filesystem on /dev/sda1 UUID: > > 0267d8b3-a074-460a-832d-5d5fd36bae64 found 63467610172 bytes used err is > > 1 total csum bytes: 59475016 total tree bytes: 1903411200 total fs tree > > bytes: 1691504640 total extent tree bytes: 130322432 btree space waste > > bytes: 442495212 file data blocks allocated: 555097092096 > > referenced 72887840768 > > btrfs-progs v4.0.1 > > > > Again: is this fixable? > > FWIW, root 5 (which you asked about upthread) is the main filesystem > root. So all these appear to be on the main filesystem, not on snapshots/ > subvolumes. OK > As for the problem itself, noting that I'm not a dev, just a user/admin > following the list, I believe... > > There was a recent bug (early 4.0 or 4.1, IDR which) that (as I recall > understanding it) would fail to decrement link count and would thus leave > unnamed inodes hanging around in directories with no way to delete them. > That looks very much like what you're seeing. Now that you mention it, I think I remember seeing that patch (series?). > The bug has indeed been > fixed in current, and a current btrfs check should fix it, but I don't > believe that v4.0.1 userspace from the initramfs is new enough to have > that fix. The 4.1.2 userspace on your main system (from the first post) > is current and should fix it, I believe, however. I have updated the initramfs in the meantime. (Funny: I *just* started using one, mainly to be able to use btrfstune on /, but now I have a genuine necessity for it.) > But if it's critical, you may wish to wait and have someone else confirm > that before acting on it, just in case I have it wrong. I can wait until tonight, at least. The FS still mounts, and it's just the root subvolume that's affected; running btrfs-send on the /home subvolume still works. Greetings -- Marc Joliet -- "People who think they know everything really annoy those of us who know we don't" - Bjarne Stroustrup pgpXLNONJFRwt.pgp Description: Digitale Signatur von OpenPGP
[PATCH v2] fstests: generic/018: expand "write backwards sync but contiguous" to test regression in btrfs
Btrfs has a problem when defraging a file which has a large fragment'ed range, it'd leave the tail extent as a seperate extent instead of merging it with previous extents. This makes generic/018 recognize the above regression. Meanwhile, I find that in the case of 'write backwards sync but contiguous", ext4 doesn't produce fragments like btrfs and xfs, so I modify 018.out a little bit to let ext4 pass. Moreover, I follow Filipe's suggestion to filter xfs_io's output in order to check these writes actually succeed. Signed-off-by: Liu Bo --- v2: fix typo in title, s/expend/expand/g tests/generic/018 | 16 ++-- tests/generic/018.out | 198 +- 2 files changed, 203 insertions(+), 11 deletions(-) diff --git a/tests/generic/018 b/tests/generic/018 index d97bb88..3693874 100755 --- a/tests/generic/018 +++ b/tests/generic/018 @@ -68,28 +68,24 @@ $XFS_IO_PROG -f -c "truncate 1m" $fragfile _defrag --before 0 --after 0 $fragfile echo "Contiguous file:" | tee -a $seqres.full -$XFS_IO_PROG -f -c "pwrite -b $((4 * bsize)) 0 $((4 * bsize))" $fragfile \ - > /dev/null +$XFS_IO_PROG -f -c "pwrite -b $((4 * bsize)) 0 $((4 * bsize))" $fragfile | _filter_xfs_io _defrag --before 1 --after 1 $fragfile echo "Write backwards sync, but contiguous - should defrag to 1 extent" | tee -a $seqres.full -for i in `seq 9 -1 0`; do - $XFS_IO_PROG -fs -c "pwrite -b $bsize $((i * bsize)) $bsize" $fragfile \ - > /dev/null +for i in `seq 64 -1 0`; do + $XFS_IO_PROG -fd -c "pwrite -b $bsize $((i * bsize)) $bsize" $fragfile | _filter_xfs_io done -_defrag --before 10 --after 1 $fragfile +_defrag --after 1 $fragfile echo "Write backwards sync leaving holes - defrag should do nothing" | tee -a $seqres.full for i in `seq 31 -2 0`; do - $XFS_IO_PROG -fs -c "pwrite -b $bsize $((i * bsize)) $bsize" $fragfile \ - > /dev/null + $XFS_IO_PROG -fs -c "pwrite -b $bsize $((i * bsize)) $bsize" $fragfile | _filter_xfs_io done _defrag --before 16 --after 16 $fragfile echo "Write forwards sync leaving holes - defrag should do nothing" | tee -a $seqres.full for i in `seq 0 2 31`; do - $XFS_IO_PROG -fs -c "pwrite -b $bsize $((i * bsize)) $bsize" $fragfile \ - > /dev/null + $XFS_IO_PROG -fs -c "pwrite -b $bsize $((i * bsize)) $bsize" $fragfile | _filter_xfs_io done _defrag --before 16 --after 16 $fragfile diff --git a/tests/generic/018.out b/tests/generic/018.out index 5f265d1..0886a9a 100644 --- a/tests/generic/018.out +++ b/tests/generic/018.out @@ -6,14 +6,210 @@ Sparse file (no blocks): Before: 0 After: 0 Contiguous file: +wrote 16384/16384 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) Before: 1 After: 1 Write backwards sync, but contiguous - should defrag to 1 extent -Before: 10 +wrote 4096/4096 bytes at offset 262144 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 258048 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 253952 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 249856 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 245760 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 241664 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 237568 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 233472 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 229376 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 225280 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 221184 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 217088 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 212992 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 208896 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 204800 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 200704 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 196608 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 192512 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 188416 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 184320 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/
[PATCH] fstests: generic/018: expend "write backwards sync but contiguous" to test regression in btrfs
Btrfs has a problem when defraging a file which has a large fragment'ed range, it'd leave the tail extent as a seperate extent instead of merging it with previous extents. This makes generic/018 recognize the above regression. Meanwhile, I find that in the case of 'write backwards sync but contiguous", ext4 doesn't produce fragments like btrfs and xfs, so I modify 018.out a little bit to let ext4 pass. Moreover, I follow Filipe's suggestion to filter xfs_io's output in order to check these writes actually succeed. Signed-off-by: Liu Bo --- tests/generic/018 | 16 ++-- tests/generic/018.out | 198 +- 2 files changed, 203 insertions(+), 11 deletions(-) diff --git a/tests/generic/018 b/tests/generic/018 index d97bb88..3693874 100755 --- a/tests/generic/018 +++ b/tests/generic/018 @@ -68,28 +68,24 @@ $XFS_IO_PROG -f -c "truncate 1m" $fragfile _defrag --before 0 --after 0 $fragfile echo "Contiguous file:" | tee -a $seqres.full -$XFS_IO_PROG -f -c "pwrite -b $((4 * bsize)) 0 $((4 * bsize))" $fragfile \ - > /dev/null +$XFS_IO_PROG -f -c "pwrite -b $((4 * bsize)) 0 $((4 * bsize))" $fragfile | _filter_xfs_io _defrag --before 1 --after 1 $fragfile echo "Write backwards sync, but contiguous - should defrag to 1 extent" | tee -a $seqres.full -for i in `seq 9 -1 0`; do - $XFS_IO_PROG -fs -c "pwrite -b $bsize $((i * bsize)) $bsize" $fragfile \ - > /dev/null +for i in `seq 64 -1 0`; do + $XFS_IO_PROG -fd -c "pwrite -b $bsize $((i * bsize)) $bsize" $fragfile | _filter_xfs_io done -_defrag --before 10 --after 1 $fragfile +_defrag --after 1 $fragfile echo "Write backwards sync leaving holes - defrag should do nothing" | tee -a $seqres.full for i in `seq 31 -2 0`; do - $XFS_IO_PROG -fs -c "pwrite -b $bsize $((i * bsize)) $bsize" $fragfile \ - > /dev/null + $XFS_IO_PROG -fs -c "pwrite -b $bsize $((i * bsize)) $bsize" $fragfile | _filter_xfs_io done _defrag --before 16 --after 16 $fragfile echo "Write forwards sync leaving holes - defrag should do nothing" | tee -a $seqres.full for i in `seq 0 2 31`; do - $XFS_IO_PROG -fs -c "pwrite -b $bsize $((i * bsize)) $bsize" $fragfile \ - > /dev/null + $XFS_IO_PROG -fs -c "pwrite -b $bsize $((i * bsize)) $bsize" $fragfile | _filter_xfs_io done _defrag --before 16 --after 16 $fragfile diff --git a/tests/generic/018.out b/tests/generic/018.out index 5f265d1..0886a9a 100644 --- a/tests/generic/018.out +++ b/tests/generic/018.out @@ -6,14 +6,210 @@ Sparse file (no blocks): Before: 0 After: 0 Contiguous file: +wrote 16384/16384 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) Before: 1 After: 1 Write backwards sync, but contiguous - should defrag to 1 extent -Before: 10 +wrote 4096/4096 bytes at offset 262144 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 258048 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 253952 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 249856 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 245760 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 241664 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 237568 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 233472 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 229376 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 225280 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 221184 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 217088 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 212992 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 208896 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 204800 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 200704 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 196608 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 192512 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 188416 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 184320 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 4096/4096 bytes at offset 180224 +XXX Bytes, X
4.2-rc6: kernel BUG at fs/btrfs/inode.c:3230
Seen today: [150110.712196] [ cut here ] [150110.776995] kernel BUG at fs/btrfs/inode.c:3230! [150110.841067] invalid opcode: [#1] SMP [150110.904472] Modules linked in: dm_mod netconsole ipt_REJECT nf_reject_ipv4 xt_multiport iptable_filter ip_tables x_tables cpufreq_userspace cpufreq_stats cpufreq_powersave cpufreq_conservative bonding usbhid ext2 coretemp loop ehci_pci ehci_hcd sb_edac i2c_i801 usbcore edac_core i2c_core usb_common ipmi_si shpchp ipmi_msghandler button btrfs lzo_compress raid0 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq raid1 md_mod sg sd_mod ixgbe(O) i40e(O) vxlan ip6_udp_tunnel udp_tunnel ahci ptp aacraid libahci pps_core [150111.236617] CPU: 27 PID: 5015 Comm: btrfs-cleaner Tainted: G O 4.2-rc6 #1 [150111.305690] Hardware name: Supermicro X10DRH/X10DRH-IT, BIOS 1.0c 02/18/2015 [150111.376185] task: 88104ec19d50 ti: 8810328bc000 task.ti: 8810328bc000 [150111.446596] RIP: 0010:[] [] btrfs_orphan_add+0x153/0x230 [btrfs] [150111.518909] RSP: 0018:8810328bfbf8 EFLAGS: 00010286 [150111.591117] RAX: ffe4 RBX: 88104bc0d000 RCX: 881052942000 [150111.664479] RDX: dc9e RSI: 0004 RDI: 881052942138 [150111.737600] RBP: 8810328bfc38 R08: 60ef80001fd0 R09: 880ec4c0f3f0 [150111.810923] R10: a01e1527 R11: ea0030914100 R12: 880a127477d0 [150111.884381] R13: 0001 R14: 880d4c35fcc0 R15: 0001 [150111.957062] FS: () GS:88107fd6() knlGS: [150112.031233] CS: 0010 DS: ES: CR0: 80050033 [150112.104347] CR2: 01262000 CR3: 01c14000 CR4: 001407e0 [150112.178463] Stack: [150112.251776] 8810328bfc38 a020ce17 8810538aa000 880d4c35fcc0 [150112.327874] 8810538aa000 880ec4c0f3f0 880a127477d0 881037d26600 [150112.404005] 8810328bfcd8 a01b3e06 88104cf54c00 880d4c35fcc0 [150112.480156] Call Trace: [150112.555499] [] ? lookup_free_space_inode+0x47/0xf0 [btrfs] [150112.632958] [] btrfs_remove_block_group+0x246/0x930 [btrfs] [150112.709655] [] btrfs_remove_chunk+0x6a7/0x950 [btrfs] [150112.785134] [] btrfs_delete_unused_bgs+0x329/0x350 [btrfs] [150112.860907] [] cleaner_kthread+0x196/0x200 [btrfs] [150112.936807] [] ? btree_read_extent_buffer_pages.constprop.128+0x110/0x110 [btrfs] [150113.014224] [] kthread+0xc9/0xe0 [150113.090363] [] ? kthread_worker_fn+0x150/0x150 [150113.165866] [] ret_from_fork+0x58/0x90 [150113.241276] [] ? kthread_worker_fn+0x150/0x150 [150113.316909] Code: 0f 1f 84 00 00 00 00 00 49 8b 54 24 38 eb 8d 66 0f 1f 84 00 00 00 00 00 4c 89 e6 4c 89 f7 e8 c5 05 fe ff 85 c0 0f 84 53 ff ff ff <0f> 0b 0f 1f 00 be 07 00 00 00 48 89 df e8 fb 02 fe ff 48 85 c0 [150113.476216] RIP [] btrfs_orphan_add+0x153/0x230 [btrfs] [150113.553676] RSP [150113.630868] ---[ end trace d01d42d1600e0787 ]--- -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Deleted files cause btrfs-send to fail
Marc Joliet posted on Thu, 13 Aug 2015 09:05:41 +0200 as excerpted: > Here's the actual output now, obtained via btrfs-progs 4.0.1 from an > initramfs emergency shell: > > checking extents checking free space cache checking fs roots root 5 > inode 8338813 errors 2000, link count wrong > unresolved ref dir 26699 index 50500 namelen 4 name root > filetype 0 errors 3, no dir item, no dir index > root 5 inode 8338814 errors 2000, link count wrong > unresolved ref dir 26699 index 50502 namelen 6 name marcec > filetype 0 errors 3, no dir item, no dir index > root 5 inode 8338815 errors 2000, link count wrong > unresolved ref dir 26699 index 50504 namelen 6 name systab > filetype 0 errors 3, no dir item, no dir index > root 5 inode 8710030 errors 2000, link count wrong > unresolved ref dir 26699 index 59588 namelen 6 name marcec > filetype 0 errors 3, no dir item, no dir index > root 5 inode 8710031 errors 2000, link count wrong > unresolved ref dir 26699 index 59590 namelen 4 name root > filetype 0 errors 3, no dir item, no dir index > Checking filesystem on /dev/sda1 UUID: > 0267d8b3-a074-460a-832d-5d5fd36bae64 found 63467610172 bytes used err is > 1 total csum bytes: 59475016 total tree bytes: 1903411200 total fs tree > bytes: 1691504640 total extent tree bytes: 130322432 btree space waste > bytes: 442495212 file data blocks allocated: 555097092096 > referenced 72887840768 > btrfs-progs v4.0.1 > > Again: is this fixable? FWIW, root 5 (which you asked about upthread) is the main filesystem root. So all these appear to be on the main filesystem, not on snapshots/ subvolumes. As for the problem itself, noting that I'm not a dev, just a user/admin following the list, I believe... There was a recent bug (early 4.0 or 4.1, IDR which) that (as I recall understanding it) would fail to decrement link count and would thus leave unnamed inodes hanging around in directories with no way to delete them. That looks very much like what you're seeing. The bug has indeed been fixed in current, and a current btrfs check should fix it, but I don't believe that v4.0.1 userspace from the initramfs is new enough to have that fix. The 4.1.2 userspace on your main system (from the first post) is current and should fix it, I believe, however. But if it's critical, you may wish to wait and have someone else confirm that before acting on it, just in case I have it wrong. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RAID0 wrong (raw) device?
root@toy02:~# df -T /data Filesystem Type 1K-blocks Used Available Use% Mounted on /dev/sdb btrfs 3906909856 140031696 3765056176 4% /data root@toy02:~# btrfs filesystem show /data Label: data uuid: 411af13f-6cae-4f03-99dc-5941acb3135b Total devices 2 FS bytes used 129.81GiB devid3 size 1.82TiB used 67.03GiB path /dev/drbd2 devid4 size 1.82TiB used 67.03GiB path /dev/sdb Btrfs v3.12 ==> btrfs shows the wrong (raw) device /dev/sdb instead of /dev/drbd3 ! Don't be too alarmed by that, progs do a bit of user land fabrication (wrong). kernel may /may-not be using sdb. try -m option. just in case if it didn't, Then use mount -o devices / btrfs dev scan option to provide the desired dev path to the kernel. Thanks, Anand -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
4.2-rc6: cp reflink call trace
I'm getting the following trace on a daily basis when stacking a lot of cp --reflink commands. Somethinkg like: File a 80GB cp --reflink=always a b modify b cp --reflink=always b c modify c cp --reflink=always c d modify d ... [57623.099897] INFO: task cp:1319 blocked for more than 120 seconds. [57623.179331] Tainted: GW O 4.2-rc6 #1 [57623.259794] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [57623.343106] cp D 88085fcd1cc0 0 1319 1237 0x0002 [57623.425986] 880040127798 0002 88002459 00011cc0 [57623.507907] 880040127fd8 00011cc0 880854018000 88002459 [57623.588682] 880040127778 8800401278b8 8800401278c0 7fff [57623.669714] Call Trace: [57623.747413] [] schedule+0x29/0x70 [57623.821280] [] schedule_timeout+0x1fc/0x260 [57623.894211] [] ? __queue_delayed_work+0xaa/0x1a0 [57623.967373] [] ? try_to_grab_pending+0x102/0x150 [57624.042693] [] wait_for_completion+0x96/0x100 [57624.113736] [] ? try_to_wake_up+0x310/0x310 [57624.185449] [] writeback_inodes_sb_nr+0x86/0xb0 [57624.255713] [] flush_space+0x408/0x480 [btrfs] [57624.321342] [] ? btrfs_get_alloc_profile+0x30/0x40 [btrfs] [57624.389582] [] ? can_overcommit+0x5f/0x100 [btrfs] [57624.454281] [] reserve_metadata_bytes+0x206/0x490 [btrfs] [57624.517303] [] ? btrfs_clear_lock_blocking_rw+0x54/0x100 [btrfs] [57624.587297] [] btrfs_block_rsv_add+0x38/0x60 [btrfs] [57624.656038] [] ? btrfs_search_slot+0x89b/0xa10 [btrfs] [57624.722189] [] start_transaction+0x473/0x5c0 [btrfs] [57624.794538] [] btrfs_start_transaction+0x1b/0x20 [btrfs] [57624.865341] [] btrfs_clone+0x5b4/0x1070 [btrfs] [57624.935456] [] btrfs_ioctl_clone+0x47f/0x4f0 [btrfs] [57625.004768] [] ? do_last.isra.62+0x286/0xce0 [57625.076043] [] btrfs_ioctl+0x18e1/0x28f0 [btrfs] [57625.144157] [] ? path_openat+0xbe/0x600 [57625.209512] [] ? from_kgid+0x12/0x20 [57625.275033] [] ? from_kgid_munged+0xe/0x20 [57625.340483] [] ? acct_account_cputime+0x1c/0x20 [57625.403169] [] do_vfs_ioctl+0x439/0x500 [57625.463798] [] ? vtime_account_user+0x44/0x60 [57625.520728] [] ? syscall_trace_enter_phase1+0xf8/0x160 [57625.578309] [] SyS_ioctl+0x4c/0x90 [57625.635960] [] ? int_check_syscall_exit_work+0x34/0x3d [57625.694317] [] system_call_fastpath+0x12/0x17 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Deleted files cause btrfs-send to fail
Am Thu, 13 Aug 2015 00:34:19 +0200 schrieb Marc Joliet : [...] > Since this is the root file system, I haven't gotten a copy of the actual > output > of "btrfs check", though I have run it from an initramfs rescue shell. The > output I saw there was much like the following (taken from an Email by Roman > Mamedov from 2014-12-28): > > root 22730 inode 6236418 errors 2000, link count wrong unresolved ref dir > 105512 index 586340 namelen 48 name [redacted].dat.bak filetype 0 errors 3, > no dir item, no dir index > > Only in my case, it's "root 5" and "root 4" (I think), and the file names > (and other file system specifics) are of course different. I definitely saw > "errors 2000" (I take it that's supposed to be an error code?). [...] Here's the actual output now, obtained via btrfs-progs 4.0.1 from an initramfs emergency shell: checking extents checking free space cache checking fs roots root 5 inode 8338813 errors 2000, link count wrong unresolved ref dir 26699 index 50500 namelen 4 name root filetype 0 errors 3, no dir item, no dir index root 5 inode 8338814 errors 2000, link count wrong unresolved ref dir 26699 index 50502 namelen 6 name marcec filetype 0 errors 3, no dir item, no dir index root 5 inode 8338815 errors 2000, link count wrong unresolved ref dir 26699 index 50504 namelen 6 name systab filetype 0 errors 3, no dir item, no dir index root 5 inode 8710030 errors 2000, link count wrong unresolved ref dir 26699 index 59588 namelen 6 name marcec filetype 0 errors 3, no dir item, no dir index root 5 inode 8710031 errors 2000, link count wrong unresolved ref dir 26699 index 59590 namelen 4 name root filetype 0 errors 3, no dir item, no dir index Checking filesystem on /dev/sda1 UUID: 0267d8b3-a074-460a-832d-5d5fd36bae64 found 63467610172 bytes used err is 1 total csum bytes: 59475016 total tree bytes: 1903411200 total fs tree bytes: 1691504640 total extent tree bytes: 130322432 btree space waste bytes: 442495212 file data blocks allocated: 555097092096 referenced 72887840768 btrfs-progs v4.0.1 Again: is this fixable? -- Marc Joliet -- "People who think they know everything really annoy those of us who know we don't" - Bjarne Stroustrup pgpO1kk8QiHHv.pgp Description: Digitale Signatur von OpenPGP
