On 2018-01-09 23:38, Duncan wrote:
Graham Cobb posted on Mon, 08 Jan 2018 18:17:13 +0000 as excerpted:
On 08/01/18 16:34, Austin S. Hemmelgarn wrote:
Ideally, I think it should be as generic as reasonably possible,
possibly something along the lines of:
A: While not strictly necessary, running regular filtered balances (for
example `btrfs balance start -dusage=50 -dlimit=2 -musage=50
-mlimit=4`,
see `man btrfs-balance` for more info on what the options mean) can
help keep a volume healthy by mitigating the things that typically
cause ENOSPC errors. Full balances by contrast are long and expensive
operations, and should be done only as a last resort.
That recommendation is similar to what I do and it works well for my use
case. I would recommend it to anyone with my usage, but cannot say how
well it would work for other uses. In my case, I run balances like that
once a week: some weeks nothing happens, other weeks 5 or 10 blocks may
get moved.
Why 50% usage, and why the rather low limits?
OK, so it rarely makes sense to go over 50% usage when the intent of the
balance is to return chunks to the unallocated pool, because at 50% the
payback ratio is one free chunk for two processed and it gets worse after
that and MUCH worse after ~67-75%, where the ratios are 1:3 and 1:4
respectively, but why so high especially for a suggested scheduled/
routine command?
Largely because that's what I use myself, and I know it works reliably.
In my case, I use a large number of small filesystems, don't delete very
large amounts of data very often, and run the command daily, so it's not
very likely that a large number of chunks are going to be below half
full, and therefore it made sense for me to just limit it to a small
number of half full chunks so that it completes quickly.
I'd suggest a rather lower usage value, say 20/25/34%, for favorable
payback ratios of 5:1, 4:1, and 3:1. That should be reasonable for a
generic recommendation for scheduled/routine balances. If that's not
enough, people can do more manually or increase the values from the
generic recommendation for their specific use-case.
That's probably a good idea, though I'd likely go for about 25% as a
generic recommendation (much lower, and you're not likely to process any
chunks at all most of the time since BTRFS will back-fill things, much
higher and the ratio becomes rather unfavorable).
And I'd suggest either no limits or (for kernels that can handle it,
4.4+, which at this point is everything within our recommended support
range of the last two LTSs, thus now 4.9 earliest, anyway) range-limits,
say 2..20, so it won't bother if there's less than enough to clear at
least one chunk within the usage target (but see the observed behavior
change noted below), but will do more than the low 2-4 in the above
suggested limits if there is. With the lower usage= values, processing
should take less time per chunk, and if there's no more that fit the
usage filter it won't use the higher range anyway, so the limit can and
should be higher.
Good point on the limits too, though I would say that we should probably
comment specifically on the fact that you need 4.4 or newer for the
range support (there are still people dealing with much older kernels
out there, think of embedded life-cycles for example).
Meanwhile, for any recommendation of balance, I'd suggest also mentioning
the negative effect that enabled quotas have on balance times, probably
with a link to a fuller discussion where I'd suggest disabling them due
to the scaling issues if the use-case doesn't require them, and if that's
not possible due to the use-case, to at least consider temporarily
disabling quotas before doing a balance so as to speed it up, after which
they can be enabled again. (I'm not sure if a manual quota rescan is
required to update them at that point, or not. I don't use quotas here
or I'd test.)
Also a good point!
And an additional observation...
I'm on ssd here and run many rather small independent btrfs instead of
fewer larger ones, so I'm used to keeping an eye on usage, tho I've never
found the need to schedule balances, partly because on ssd with
relatively small btrfs, balances are fast enough they're not a problem to
do "while I wait".
In my case, they're pretty darn fast too, I just don't like having to
remember to run them by hand (that is the main appeal for automation
after all).
And I've definitely noticed an effect since the ssd option stopped using
the 2 MiB spreading algorithm in 4.14. In particular, while chunk usage
was generally stable before that and I only occasionally needed to run
balance to clear out empty chunks, now, balance with the usage filter
will apparently actively fill in empty space in existing chunks, so while
previously a usage-filtered balance that only rewrote one chunk didn't
actually free anything, simply allocating a new chunk to replace the one
it freed, so at least two chunks needed rewritten to actually free space
back to unallocated...
Now, usage-filtered rewrites of only a single chunk routinely frees the
allocated space, because it writes that small bit of data in the freed
chunk into existing free space in other chunks.
At least I /presume/ that new balance-usage behavior is due to the ssd
changes. Maybe it's due to other patches. Either way, it's an
interesting and useful change. =:^)
I'm pretty sure it's due to the 'ssd' option change. The way it was
coded previously made the allocator rather averse to back-filling free
space, and balance just sends stuff back through the allocator again
(other than the filtering, that is quite literally all it does), so a
change to the allocator's behavior will change balance behavior too.
Regardless, this is also a good point that should probably be added to
the FAQ. Given this, it might also be worth recommending that people
with SSD's who upgraded to 4.14 should run a much more aggressive
filtered balance (thinking 50% usage and no limit filter) to repack
things a bit more efficiently.
Overall, I'm starting to think that the best option here is to update
the FAQ entry, and then have netdata's help text point to the FAQ entry
instead of trying to contain the same info.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
