Re: ENOSPC while creating snapshot
On 5.3.2016 06:34, Duncan wrote: Chris Murphy posted on Fri, 04 Mar 2016 19:46:34 -0700 as excerpted: On Fri, Mar 4, 2016 at 4:16 PM, Martin Mlynář wrote: [Mount options line split/wrapped for followup] rw,noatime,nodatasum,nodatacow,ssd,discard,space_cache, enospc_debug,commit=900,subvolid=5,subvol=/ Most likely unrelated but commit time of 15 minutes? Umm, OK why? I'm trying to reduce writes to my ssd. This will not reduce writes. It will only delay them. And it increases the chance of data loss by a lot. AFAIK, to the extent that temporary files are created and then deleted again, within that 900 seconds aka 15 minutes, it will indeed reduce writes. This can be the case for the build-tmp location for people running build- from-sources distros such as gentoo, for instance, as many packages will be built and tmp-installed to that build-tmp, before being quick-merged to the live system and the work deleted from build-tmp in well under 15 minutes, at least on today's reasonably powerful quad-core-plus systems. Tho on gentoo, the recommendation for those with the memory available is to point that build-tmp at a tmpfs instead of a permanent-storage backed filesystem of any sort. And in general, for those without the memory to support build-tmp in tmpfs, a 15-minute commit time isn't going to help either, because if they have enough memory to avoid flushing to free up memory for that full 15 minutes, they obviously have enough memory to store all those files that would be in tmpfs if they'd have simply pointed build-tmp at that, instead. Another use-case is laptops and other mobile systems with enough memory to cache the normal working set, is to power down the storage devices for as long as possible between powerups. However, the heavy power usage there is normally on spinning up the disk and/or keeping it spinning, and SSDs obviously aren't subject to that. While some small amount of power may still be saved by powering down the SSD, I expect it to be pretty small, and the writes are going to take the same amount of power no matter when they're done. In either case, 15 minute commit times are rather extreme, because as has been pointed out, that's potentially 15 minutes of lost work should the system crash before those writes are completed, and losing 15 minutes worth of work is well beyond the acceptable risk level for most people. 5 minutes, much more common, 10 minutes, not so common but you'll fine people doing it. 15 minutes, pretty rare, I expect. The point being, yes, there are use-cases where 15 minute commit times makes sense. But the given reason, a bare wish to reduce writes to the ssd, without support such as one of the above use-cases or something similar, really doesn't make sense, at least on its face. I'll agree with other posters on that. Thank you for valuable insight, all of you. It is battery backed-up laptop so I thought it should work well. I've met no problems since when I've set up this few years ago. To be hones I even forgot I've got this set up :) I'll lower the value, you're right, that 15 minutes are pretty extreme. -- Martin -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: ENOSPC while creating snapshot
On Sat, Mar 5, 2016 at 2:09 AM, Duncan <1i5t5.dun...@cox.net> wrote: > Roman Mamedov posted on Sat, 05 Mar 2016 03:49:10 +0500 as excerpted: > >> As you use the nodatacow mount option, this seems to be another case of >> http://www.spinics.net/lists/linux-btrfs/msg51276.html >> http://www.spinics.net/lists/linux-btrfs/msg51819.html >> >> and is fixed by https://patchwork.kernel.org/patch/7967161/ >> >> Unfortunately the bug is known since the start of the 4.4 series and the >> patch is out for 2 months, but it didn't get included into even 4.4.4 >> released recently. You have to apply it by yourself and recompile the >> kernel. > > Tho AFAIK it should be in 4.5, which is getting close to release, so if > you prefer to run 4.5-rcX to applying the patch yourself, that should > work as well. No, it's not in any 4.5-rc. It's in the integration branch for 4.6, however. > > -- > Duncan - List replies preferred. No HTML msgs. > "Every nonfree program has a lord, a master -- > and if you use the program, he is your master." Richard Stallman > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- Filipe David Manana, "Reasonable men adapt themselves to the world. Unreasonable men adapt the world to themselves. That's why all progress depends on unreasonable men." -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: ENOSPC while creating snapshot
On 5.3.2016 11:46, Filipe Manana wrote: On Sat, Mar 5, 2016 at 2:09 AM, Duncan <1i5t5.dun...@cox.net> wrote: Roman Mamedov posted on Sat, 05 Mar 2016 03:49:10 +0500 as excerpted: As you use the nodatacow mount option, this seems to be another case of http://www.spinics.net/lists/linux-btrfs/msg51276.html http://www.spinics.net/lists/linux-btrfs/msg51819.html and is fixed by https://patchwork.kernel.org/patch/7967161/ Unfortunately the bug is known since the start of the 4.4 series and the patch is out for 2 months, but it didn't get included into even 4.4.4 released recently. You have to apply it by yourself and recompile the kernel. Tho AFAIK it should be in 4.5, which is getting close to release, so if you prefer to run 4.5-rcX to applying the patch yourself, that should work as well. No, it's not in any 4.5-rc. It's in the integration branch for 4.6, however. Thank you all for your help. Now I'm on 4.5.0-rc6-mainline with pointed patch and issue seems to be resolved. Thank you for your time! -- Martin -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: balance hangs and starts again on reboot
Hi Chris, I apologize for not being able to deliver logs in the way you might find them more helpful. On Fri, Mar 04, 2016 at 12:08:10PM -0700, Chris Murphy wrote: > On Fri, Mar 4, 2016 at 10:31 AM, Marc Haber > wrote: > > I have another btrfs on the same host that has no the no space left on > > device balance issue, but on another disk. On this btrfs, it seems > > like a balance process is stuck, with a lot of hanging kernel > > threads. After a reboot, when I mount the filesystem, the balance > > immediately starts again. btrfs balance cancel just hangs around with > > no visible reaction for hours. > > > > Log appended. Is there rescue? > > The log is made much more useful if you can sysrq+w while the blocked > task is happening; and then dmesg or journalctl -k to get the results > into a file for attachment to avoid the annoying MUA wrapping. This list has repeatedly eaten log attachments without giving any indication why. I had assumed that attachments are disallowed here, and am taking careful attention that inserted logs are not wrapped on my side. The list archives (http://www.spinics.net/lists/linux-btrfs/msg52663.html) show that my efforts not to cause wrapping on my side were actually successful. What is the most helpful way to include logs? Pastebinning them would probably reduce the list archives' usefulness due to pastebin expiring, attaching doesn't work (see above), and including them causes "annoying MUA wrapping". I do only have 24 years of e-mail experience, so I'm a clueless newbie, maybe one can give advice how to do that properly. I'm going to try the sysrq+w thing next time things happen. Greetings Marc -- - Marc Haber | "I don't trust Computers. They | Mailadresse im Header Leimen, Germany| lose things."Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: balance hangs and starts again on reboot
On Fri, Mar 04, 2016 at 07:09:39PM +0100, Holger Hoffstätte wrote: > On 03/04/16 18:31, Marc Haber wrote: > > I have another btrfs on the same host that has no the no space left on > > device balance issue, but on another disk. On this btrfs, it seems > > like a balance process is stuck, with a lot of hanging kernel > > threads. After a reboot, when I mount the filesystem, the balance > > immediately starts again. btrfs balance cancel just hangs around with > > no visible reaction for hours. > > > > Log appended. Is there rescue? > > Can't offer much help other than to recommend to *always* mount with > -o skip_balance, which IMHO should have been the default behaviour > from the beginning. That's an important hint. The btrfs balance cancel has worked over night though. > Then try to balance in small increments. -dusage=5 and incrementing? Or what do you mean with "in small increments"? Greetings Marc -- - Marc Haber | "I don't trust Computers. They | Mailadresse im Header Leimen, Germany| lose things."Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Again, no space left on device while rebalancing and recipe doesnt work
Hi, I have not seen this message coming back to the mailing list. Was it again too long? I have pastebinned the log at http://paste.debian.net/412118/ On Tue, Mar 01, 2016 at 08:51:32PM +, Duncan wrote: > There has been something bothering me about this thread that I wasn't > quite pinning down, but here it is. > > If you look at the btrfs fi df/usage numbers, data chunk total vs. used > are very close to one another (113 GiB total, 112.77 GiB used, single > profile, assuming GiB data chunks, that's only a fraction of a single > data chunk unused), so balance would seem to be getting thru them just > fine. Where would you see those numbers? I have those, pre-balance: Mar 2 20:28:01 fan root: Data, single: total=77.00GiB, used=76.35GiB Mar 2 20:28:01 fan root: System, DUP: total=32.00MiB, used=48.00KiB Mar 2 20:28:01 fan root: Metadata, DUP: total=86.50GiB, used=2.11GiB Mar 2 20:28:01 fan root: GlobalReserve, single: total=512.00MiB, used=0.00B > But there's a /huge/ spread between total vs. used metadata (32 GiB > total, under 4 GiB used, clearly _many_ empty or nearly empty chunks), > implying that has not been successfully balanced in quite some time, if > ever. This is possible, yes. > So I'd surmise the problem is in metadata, not in data. > > Which would explain why balancing data works fine, but a whole-filesystem > balance doesn't, because it's getting stuck on the metadata, not the data. > > Now the balance metadata filters include system as well, by default, and > the -mprofiles=dup and -sprofiles=dup balances finished, apparently > without error, which throws a wrench into my theory. Also finishes without changing things, post-balance: Mar 2 21:55:37 fan root: Data, single: total=77.00GiB, used=76.36GiB Mar 2 21:55:37 fan root: System, DUP: total=32.00MiB, used=80.00KiB Mar 2 21:55:37 fan root: Metadata, DUP: total=99.00GiB, used=2.11GiB Mar 2 21:55:37 fan root: GlobalReserve, single: total=512.00MiB, used=0.00B Wait, Metadata used actually _grew_??? > But while we have the btrfs fi df from before the attempt with the > profiles filters, we don't have the same output from after. s We now have everything. New log attached. > > I'd like to remove unused snapshots and keep the number of them to 4 > > digits, as a workaround. > > I'll strongly second that recommendation. Btrfs is known to have > snapshot scaling issues at 10K snapshots and above. My strong > recommendation is to limit snapshots per filesystem to 3000 or less, with > a target of 2000 per filesystem or less if possible, and an ideal of 1000 > per filesystem or less if it's practical to keep it to that, which it > should be with thinning, if you're only snapshotting 1-2 subvolumes, but > may not be if you're snapshotting more. I'm snapshotting /home every 10 minutes, the filesystem that I have been posting logs from has about 400 snapshots, and snapshot cleanup works fine. The slow snapshot removal is a different filesystem on the same host which is on a rotating rust HDD, and is much bigger. > By 3000 snapshots per filesystem, you'll be beginning to notice slowdowns > in some btrfs maintenance commands if you're sensitive to it, tho it's > still at least practical to work with, and by 10K, it's generally > noticeable by all, at least once they thin down to 2K or so, as it's > suddenly faster again! Above 100K, some btrfs maintenance commands slow > to a crawl and doing that sort of maintenance really becomes impractical > enough that it's generally easier to backup what you need to and blow > away the filesystem to start again with a new one, than it is to try to > recover the existing filesystem to a workable state, given that > maintenance can at that point take days to weeks. Ouch. This shold not be the case, or btrfs subvolume snapshot should at least emit a warning. It is not good that it is so easy to get a filesystem into a state this bad. > So 5-digits of snapshots on a filesystem is definitely well outside of > the recommended range, to the point that in some cases, particularly > approaching 6-digits of snapshots, it'll be more practical to simply > ditch the filesystem and start over, than to try to work with it any > longer. Just don't do it; setup your thinning schedule so your peak is > 3000 snapshots per filesystem or under, and you won't have that problem > to worry about. =:^) That needs to be documented prominently. Ths ZFS fanbois will love that. > Oh, and btrfs quota management exacerbates the scaling issues > dramatically. If you're using btrfs quotas Am not, thankfully. Greetings Marc -- - Marc Haber | "I don't trust Computers. They | Mailadresse im Header Leimen, Germany| lose things."Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421 -- To unsubscribe from this list: send the line "
Re: Again, no space left on device while rebalancing and recipe doesnt work
On Thu, Mar 03, 2016 at 02:28:36AM +0200, Dāvis Mosāns wrote: > I've same issue, 4.4.3 kernel on Arch Linux > > $ sudo btrfs fi show /mnt/fs/ > Label: 'fs' uuid: a3c66d25-2c25-40e5-a827-5f7e5208e235 > Total devices 1 FS bytes used 396.94GiB > devid1 size 435.76GiB used 435.76GiB path /dev/sdi2 > > $ sudo btrfs fi df /mnt/fs/ > Data, single: total=416.70GiB, used=390.62GiB > System, DUP: total=32.00MiB, used=96.00KiB > Metadata, DUP: total=9.50GiB, used=6.32GiB > GlobalReserve, single: total=512.00MiB, used=0.00B > > $ sudo btrfs fi usage /mnt/fs/ > Overall: > Device size: 435.76GiB > Device allocated:435.76GiB > Device unallocated:1.00MiB > Device missing: 0.00B > Used:403.26GiB > Free (estimated): 26.07GiB (min: 26.07GiB) > Data ratio: 1.00 > Metadata ratio: 2.00 > Global reserve: 512.00MiB (used: 0.00B) > > Data,single: Size:416.70GiB, Used:390.62GiB >/dev/sdi2 416.70GiB > > Metadata,DUP: Size:9.50GiB, Used:6.32GiB >/dev/sdi2 19.00GiB > > System,DUP: Size:32.00MiB, Used:96.00KiB >/dev/sdi2 64.00MiB > > Unallocated: >/dev/sdi2 1.00MiB http://paste.ubuntu.com/15292589/ has another log of mine with btrfs fi usage calls as well, just in case this helps. Greetings Marc -- - Marc Haber | "I don't trust Computers. They | Mailadresse im Header Leimen, Germany| lose things."Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: balance hangs and starts again on reboot
On 03/05/16 15:17, Marc Haber wrote: >> Then try to balance in small increments. > > -dusage=5 and incrementing? Or what do you mean with "in small > increments"? Exactly, yes. Sorry for not being more clear. FWIW I've been balancing a lot recently (both for stress testing and cleaning up a few filesystems) and have never run into this particular stall, but only ever do filtered balances. Also I wouldn't be surprised at all if this is yet another problem where md does something in a way that btrfs doesn' expect, and things go wrong. -h -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: balance hangs and starts again on reboot
On Sat, Mar 5, 2016 at 7:12 AM, Marc Haber wrote: > What is the most helpful way to include logs? Attach as a text file. If they're too big and get rejected, then it depends. If I'm pretty sure it's a bug, I open a bug report on bugzilla.kernel.org and attach there, then URL in list. If I'm not sure, I put the text file on google drive or dropbox and post public URL here. > including them > causes "annoying MUA wrapping". It can be either sending or receiving client that does this. I have no doubt gmail does it sending and receiving because I always have this problem. > I do only have 24 years of e-mail > experience, so I'm a clueless newbie, maybe one can give advice how to > do that properly. I've given up with pasting kernel messages inline. -- Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: balance hangs and starts again on reboot
On Sat, Mar 05, 2016 at 04:38:57PM +0100, Holger Hoffstätte wrote: > On 03/05/16 15:17, Marc Haber wrote: > >> Then try to balance in small increments. > > > > -dusage=5 and incrementing? Or what do you mean with "in small > > increments"? > > Exactly, yes. Sorry for not being more clear. So you would recommend something along for nr in $(seq 5 5 100); do btrfs balance start -dusage=$nr $FS done right? Won't this take ages longer than a straight unfiltered balance? > FWIW I've been balancing a lot recently (both for stress testing and > cleaning up a few filesystems) and have never run into this particular > stall, but only ever do filtered balances. Also I wouldn't be surprised > at all if this is yet another problem where md does something in a way > that btrfs doesn' expect, and things go wrong. md as in the Linux Software RAID? That's not in the game here, it's a single SATA hard disk. Greetings Marc -- - Marc Haber | "I don't trust Computers. They | Mailadresse im Header Leimen, Germany| lose things."Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: balance hangs and starts again on reboot
On 03/05/16 18:25, Marc Haber wrote: > On Sat, Mar 05, 2016 at 04:38:57PM +0100, Holger Hoffstätte wrote: >> On 03/05/16 15:17, Marc Haber wrote: Then try to balance in small increments. >>> >>> -dusage=5 and incrementing? Or what do you mean with "in small >>> increments"? >> >> Exactly, yes. Sorry for not being more clear. > > So you would recommend something along > > for nr in $(seq 5 5 100); do > btrfs balance start -dusage=$nr $FS > done > > right? Except for the 100 part, which seems pointless. Maybe more like 10,20..80 max. If that doesn't help you are probably out of space anyway. > Won't this take ages longer than a straight unfiltered balance? Touching less stuff conditionally is pretty much guaranteed to be faster than unconditionally rewriting everything, and less likely to end up out of space since you garbage-collect the smallest chunks first, freeing up a larger one and so on. > md as in the Linux Software RAID? That's not in the game here, it's a > single SATA hard disk. I thought your df output contained md or something. If not, all the better. -h -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 12/12] block: test fallocate for block devices
I'm not sure xfstests is the right fit, as it does not test a file system, but rather block devices. If people think it should go into xfstests we should at least not add it to the default group, but just to a new bdev group. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Again, no space left on device while rebalancing and recipe doesnt work
I can't tell what this btrfs-balance script is doing because not every btrfs balance command is in the log. It may be doing something not advisable or suboptimal or unexpected that along with some other bug is causing this to happen Metadata,DUP: Size:107.00GiB, Used:2.11GiB I'd try to use -musage filter alone, in whatever increments work. So try 0. Then 5. If 5 fails, try 2. Increment until size is not much more than 2-3x used. Something is happening with the usage of this file system that's out of the ordinary. This is the first time I've seen such a large amount of unused metadata allocation. And then for it not only fail to balance, but for the allocation amount to increase is a first. So understanding the usage is important to figuring out what's happening. I'd file a bug and include as much information on how the fs got into this state as possible. And also if possible make a btrfs-image using the proper flags to blot out the filenames for privacy. And what btrfs-progs tools were used to create this file system. Etc. The alternative if this can't be fixed, is to recreate the filesystem because there's no practical way yet to migrate so many snapshots to a new file system. Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Again, no space left on device while rebalancing and recipe doesnt work
On Sat, Mar 05, 2016 at 12:34:09PM -0700, Chris Murphy wrote:
> I can't tell what this btrfs-balance script is doing because not every
> btrfs balance command is in the log.
It is. I wrote it to produce reproducible logs.
[1/499]mh@fan:~$ cat btrfs-balance
#!/bin/bash
FS="/mnt/fanbtr"
showdf() {
logger -- btrfs fi df $FS
btrfs fi df $FS 2>&1 | logger
logger -- btrfs fi show /
btrfs fi show / | logger
logger -- btrfs fi usage /
btrfs fi usage / | logger
}
logger -- BEGIN btrfs-balance script
showdf
btrfs balance start $FS 2>&1 | logger
showdf
logger -- BEGIN btrfs balance start -dprofiles=single $FS
btrfs balance start -dprofiles=single $FS 2>&1 | logger
showdf
logger -- BEGIN btrfs balance start -mprofiles=dup $FS
btrfs balance start -mprofiles=dup $FS 2>&1 | logger
showdf
logger -- BEGIN btrfs balance start --force -sprofiles=dup $FS
btrfs balance start --force -sprofiles=dup $FS 2>&1 | logger
showdf
logger -- BEGIN btrfs balance start $FS
btrfs balance start $FS 2>&1 | logger
showdf
logger -- END btrfs-balance script
[2/500]mh@fan:~$
I see. The logger -- BEGIN is missing for the very first command. My
bad.
> Something is happening with the usage of this file system that's out
> of the ordinary. This is the first time I've seen such a large amount
> of unused metadata allocation. And then for it not only fail to
> balance, but for the allocation amount to increase is a first.
It is just a root filesystem of a workstation running Debian Linux, in
daily use, with daily snapshots of the system, and
ten-minute-increment snapshots of /home, with no cleanup happening for
a few months.
> So understanding the usage is important to figuring out what's
> happening. I'd file a bug and include as much information on how the
> fs got into this state as possible. And also if possible make a
> btrfs-image using the proper flags to blot out the filenames for
> privacy.
That would btrfs-image -s?
> And what btrfs-progs tools were used to create this file system. Etc.
The file system is at least two years old, I do not remember, which
version of btrfs-tools was in Debian unstable back then. Is this
information somewhere in the filesystem label? How do I obtain this one?
> The alternative if this can't be fixed, is to recreate the filesystem
> because there's no practical way yet to migrate so many snapshots to a
> new file system.
I am now back to a mid three-digit number of snapshots.
Greetings
Marc
--
-
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany| lose things."Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 11/12] xfs: remove NOCOW_FL testing from test
Looks fine, Reviewed-by: Christoph Hellwig -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 08/12] xfs/122: define _GNU_SOURCE when compiling test program
Looks fine, Reviewed-by: Christoph Hellwig -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 09/12] xfs/122: support rmapxbt
Looks fine, Reviewed-by: Christoph Hellwig -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 01/12] xfs/207: fix golden output to match FS_IOC_FSSETXATTR hoist
Looks good, Reviewed-by: Christoph Hellwig -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 02/12] xfs: test copy-on-write leftover recovery
Looks good, Reviewed-by: Christoph Hellwig -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 03/12] reflink: fix fragmentation tests to work on >4k block size filesystems
Looks fine, Reviewed-by: Christoph Hellwig -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 04/12] xfs/23[3-6]: don't source common/xfs, it doesn't exist
On Fri, Mar 04, 2016 at 04:37:43PM -0800, Darrick J. Wong wrote: > Don't source common/xfs, since it doesn't (yet) exist. Heh. Wonder how they ever worked before.. (they probably didn't, and I didn't notice as I never ran with rmapbt) Reviewed-by: Christoph Hellwig -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/12] xfs/206: fix output when mkfs knows about reflink
Looks fine, Reviewed-by: Christoph Hellwig -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 06/12] xfs/030: fix output on newer filesystems
Still fails for me: --- tests/xfs/030.out 2016-03-03 07:55:58.556427678 + +++ /root/xfstests/results//xfs/030.out.bad 2016-03-05 20:20:17.561433837 + @@ -231,8 +231,6 @@ bad agbno AGBNO in agfl, agno 0 bad agbno AGBNO in agfl, agno 0 bad agbno AGBNO in agfl, agno 0 -bad agbno AGBNO in agfl, agno 0 -bad agbno AGBNO in agfl, agno 0 - found root inode chunk Phase 3 - for each AG... - scan and clear agi unlinked lists... -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 07/12] xfs/073: fix output
Fixes one warning for my config, but adds two new ones (see below). Looks like we need to do some better feature detection. --- tests/xfs/073.out 2016-03-03 07:55:58.556427678 + +++ /root/xfstests/results//xfs/073.out.bad 2016-03-05 20:21:07.368100504 + @@ -1,6 +1,4 @@ QA output created by 073 -warning: finobt not supported without CRC support, disabled. -warning: rmapbt not supported without CRC support, disabled. warning: reflink not supported without CRC support, disabled. meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks data = bsize=XXX blocks=XXX, imaxpct=PCT @@ -38,6 +36,7 @@ unmounting and removing new image === copying scratch device to single target, large ro device +warning: reflink not supported without CRC support, disabled. meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks data = bsize=XXX blocks=XXX, imaxpct=PCT = sunit=XXX swidth=XXX, unwritten=X @@ -51,6 +50,7 @@ checking new image mounting new image on loopback comparing new image files to old +File /mnt/test/23382.source_dir/xfstests/dev is a block special file while file /mnt/test/23382.loop/xfstests/dev is a block special file comparing new image directories to old comparing new image geometry to old unmounting and removing new image -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 10/12] xfs: test per-ag allocation accounting during truncate-caused refcountbt expansion
Looks fine, Reviewed-by: Christoph Hellwig -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 13/12] xfs/209: filter scratch dir properly
Signed-off-by: Christoph Hellwig diff --git a/tests/xfs/209 b/tests/xfs/209 index cecd9c7..9bf1f12 100755 --- a/tests/xfs/209 +++ b/tests/xfs/209 @@ -73,7 +73,7 @@ echo "Check cowextsize settings" seq 1 2 | while read nr; do seq 1 4 | while read nnr; do file="$testdir/dir-$nr/file-$nnr" - $XFS_IO_PROG -c "cowextsize" $file + $XFS_IO_PROG -c "cowextsize" $file | _filter_scratch done done diff --git a/tests/xfs/209.out b/tests/xfs/209.out index 109af34..b97fa96 100644 --- a/tests/xfs/209.out +++ b/tests/xfs/209.out @@ -3,11 +3,11 @@ Format and mount Set extsz and cowextsz on directory Create a fake tree structure Check cowextsize settings -[1048576] /opt/test-209/dir-1/file-1 -[1048576] /opt/test-209/dir-1/file-2 -[1048576] /opt/test-209/dir-1/file-3 -[1048576] /opt/test-209/dir-1/file-4 -[1048576] /opt/test-209/dir-2/file-1 -[1048576] /opt/test-209/dir-2/file-2 -[1048576] /opt/test-209/dir-2/file-3 -[1048576] /opt/test-209/dir-2/file-4 +[1048576] SCRATCH_MNT/test-209/dir-1/file-1 +[1048576] SCRATCH_MNT/test-209/dir-1/file-2 +[1048576] SCRATCH_MNT/test-209/dir-1/file-3 +[1048576] SCRATCH_MNT/test-209/dir-1/file-4 +[1048576] SCRATCH_MNT/test-209/dir-2/file-1 +[1048576] SCRATCH_MNT/test-209/dir-2/file-2 +[1048576] SCRATCH_MNT/test-209/dir-2/file-3 +[1048576] SCRATCH_MNT/test-209/dir-2/file-4 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
List missing files on a degraded read-only btrfs mount
I have a btrfs filesystem spanning 3 drives. The metadata is using raid1 (mirroring), but the data is using single, so no mirroring or parity just spanning. For example: mkfs.btrfs -m raid1 -d single /dev/sda /dev/sdb /dev/sdc One of the drives, /dev/sdb, had a hardware failure before I could replace it. I'm able to mount the filesystem in read-only using: mount -o degraded,ro /dev/sda /mnt/data But how do I list the files that were on the failed drive? -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[check] set metadata extent size of tree block extents
When scanning extents, we didn't set num_bytes when visiting a tree
block extent. On the corrupted filesystem I was trying to fix, this
caused an extent to have its size guessed as zero, so we'd compute end
as start-1, which caused us to hit insert_state's BUG_ON(end
---
cmds-check.c |5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/cmds-check.c b/cmds-check.c
index 0165fba..e563354 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -5208,9 +5208,10 @@ static int process_extent_item(struct btrfs_root *root,
ei = btrfs_item_ptr(eb, slot, struct btrfs_extent_item);
refs = btrfs_extent_refs(eb, ei);
- if (btrfs_extent_flags(eb, ei) & BTRFS_EXTENT_FLAG_TREE_BLOCK)
+ if (btrfs_extent_flags(eb, ei) & BTRFS_EXTENT_FLAG_TREE_BLOCK) {
metadata = 1;
- else
+ num_bytes = root->leafsize;
+ } else
metadata = 0;
add_extent_rec(extent_cache, NULL, 0, key.objectid, num_bytes,
--
Alexandre Oliva, freedom fighterhttp://FSFLA.org/~lxoliva/
You must be the change you wish to see in the world. -- Gandhi
Be Free! -- http://FSFLA.org/ FSF Latin America board member
Free Software Evangelist|Red Hat Brasil GNU Toolchain Engineer
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: balance hangs and starts again on reboot
Holger Hoffstätte posted on Sat, 05 Mar 2016 16:38:57 +0100 as excerpted: > On 03/05/16 15:17, Marc Haber wrote: >>> Then try to balance in small increments. >> >> -dusage=5 and incrementing? Or what do you mean with "in small >> increments"? > > Exactly, yes. Sorry for not being more clear. > > FWIW I've been balancing a lot recently (both for stress testing and > cleaning up a few filesystems) and have never run into this particular > stall, but only ever do filtered balances. Also I wouldn't be surprised > at all if this is yet another problem where md does something in a way > that btrfs doesn' expect, and things go wrong. What I thought you meant was the drange= or vrange= filters, changing the range to eventually cover the entire filesystem. That should work too, tho I've never actually used it myself and I suspect I'd have to play around with the ranges a bit to figure out what numbers I should actually be supplying, as the filter descriptions in the manpage are somewhat vague on this point. (Anyone who knows where to actually find those numbers to plug in and/or has useful examples, feel free to consider this an invitation to elucidate. =:^) -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Again, no space left on device while rebalancing and recipe doesnt work
Marc Haber posted on Sat, 05 Mar 2016 21:09:09 +0100 as excerpted: > On Sat, Mar 05, 2016 at 12:34:09PM -0700, Chris Murphy wrote: >> Something is happening with the usage of this file system that's out of >> the ordinary. This is the first time I've seen such a large amount of >> unused metadata allocation. And then for it not only fail to balance, >> but for the allocation amount to increase is a first. > > It is just a root filesystem of a workstation running Debian Linux, in > daily use, with daily snapshots of the system, and ten-minute-increment > snapshots of /home, with no cleanup happening for a few months. > >> So understanding the usage is important to figuring out what's >> happening. I'd file a bug and include as much information on how the >> fs got into this state as possible. And also if possible make a >> btrfs-image using the proper flags to blot out the filenames for >> privacy. Now you're homing in on what I picked up on. There's something very funky about that metadata, 100+ GiB of metadata total, only just over 2 GiB metadata used, and attempts to balance it don't help with the spread between the two at all, only increasing the total metadata, if anything, but still seem to complete without error. There's gotta be some sort of bug going on there, and I'd /bet/ it's the same one that's keeping full balances from working, as well. OK, this question's out of left field, but it's the only thing (well, /almost/ only, see below) I've seen do anything /remotely/ like that: Was the filesystem originally created as a convert from ext*, using btrfs- convert? If so, was the ext2_saved or whatever subvolume removed, and a successful defrag and balance completed at that time? Because as I said, problems due to faulty conversion from ext* have been the one thing repeatedly reported to trigger balance behavior and spreads between total and used that balance doesn't fix, like this. Tho AFAIK there was in addition a very narrow timeframe in which a bug in mkfs.btrfs would create invalid btrfs'. That was with btrfs-progs 4.1.1, released in July 2015, with an urgent bugfix release 4.1.2 in the same month to fix the problem, so the timeframe was days or weeks. Btrfs created with that buggy mkfs.btrfs were known to have some pretty wild behavior as well, with the recommendation being to simply blow them up and recreate them with a mkfs.btrfs from a btrfs-progs without the bug, as the btrfs created by the bugged version were simply too bugged out to reliably fix, and might well appear to work fine for awhile, until BOOM! If there's a chance the filesystem in question was created by that bugged mkfs.btrfs, don't even try to fix it, just get what you can off it and recreate with a mkfs.btrfs without that bug, ASAP. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: List missing files on a degraded read-only btrfs mount
Justin Madru posted on Sat, 05 Mar 2016 12:32:31 -0800 as excerpted: > I have a btrfs filesystem spanning 3 drives. The metadata is using raid1 > (mirroring), but the data is using single, so no mirroring or parity > just spanning. For example: > > mkfs.btrfs -m raid1 -d single /dev/sda /dev/sdb /dev/sdc > > One of the drives, /dev/sdb, had a hardware failure before I could > replace it. I'm able to mount the filesystem in read-only using: > > mount -o degraded,ro /dev/sda /mnt/data > > But how do I list the files that were on the failed drive? I don't believe there's a simple, admin-level command, to list only the files that happened to be on a particular drive. What you /can/ do is just do a global copy to somewhere else, and unaffected files will copy, while affected ones will fail. The other alternative is to use btrfs restore on the unmounted filesystem, restoring the files that are possible to some other location. Note that by default, the restored files will be written as root, using umask, with symlinks skipped, but on reasonably recent btrfs- progs, restore has options that allow you to restore metadata such as ownership/perms/times, and symlinks, if you wish. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
