Robert White posted on Sun, 02 Nov 2014 03:08:46 -0800 as excerpted: > Confusing bit, for example, from wiki > > [QUOTE] > If you are getting out of space errors due to metadata being full, try > > btrfs balance start -v -dusage=0 /mnt/btrfs [/QUOTE] > > Combined with "Balances only block groups with usage under the given > percentage. " > > Which I was reading -dusage=0 means don't bother with data chunks and > (and so just fix the metadata), otherwise the mention of using a -d > filter to affect metadata is perverse. > > Blarg... I mean just... blarg... > > But now I know. 8-)
If metadata is full and there's no unallocated space left from which to create new metadata chunks, then balancing metadata wouldn't do any good anyway. Which is why you balance data chunks in that case. The typical scenario is this. Someone creates a btrfs and starts using it, creating files, deleting files, but over time, tending to create more files than they delete, so the space starts to fill up. As they do so, btrfs allocates new data and metadata chunks on demand from the unallocated space. Btrfs allocation and usage happens in two steps, unallocated space gets allocated to chunkspace, either data or metadata, and then that allocated chunkspace gets actually used for file data or metadata, depending on the chunk type. Data chunks are 1 GiB each by default, while metadata chunks default to a quarter GiB each. The critical bit to understand here is that while btrfs can automatically allocate both chunks and actual usage on demand, when it frees space, it can only automatically free actual usage, not the allocated chunks. And it can't switch chunks from one type to the other. To free the chunks back to unallocated so they can again be allocated on-demand to data or metadata as necessary, one must run a balance, which rewrites the chunks, consolidating as it goes, thereby freeing the excess allocated chunks if actual usage fits into less chunks than were previously allocated. Picking up our typical scenario... Then they delete a bunch of files, often the bigger ones, but the data tends to be much bigger than the metadata, so deleting these files frees up a lot of data chunk space but only a relatively little metadata chunk space. Then they go writing files again, but on average smaller ones. These smaller files take up less data space but the same amount of metadata space, so without a manual balance to reclaim allocated but mostly empty chunks, the limited metadata space freed by that big deletion gets filled up faster than the data space, and suddenly, people are getting ENOSPC errors when df says there's LOTS of space, because all that space is taken up by mostly empty data chunks, leaving no room to write new metadata chunks. The scenario is similar to that of ext* running out of inodes (a type of metadata, after all) since it preallocates them at mkfs time, except that over time, the default number of inodes at a particular ext* filesystem size has been bumped up so that this seldom happens in practice any more. But btrfs stores quite a bit more metadata per file, including checksums, and for small files, perhaps the entire file including the data, in which case it won't actually have a data extent, so oversizing btrfs metadata by a similar amount would mean wasting MUCH more space for the typical user. And btrfs can automatically allocate data and metadata chunks on demand -- the catch is that it can't automatically unallocate chunks on demand[1], a balance is required for that, nor can it switch usage types on chunks once allocated. In that scenario, it's metadata that's out, but to fix it you have to balance data, returning unused but allocated data chunks back to the unallocated space pool, so they can be allocated as metadata. Which is why/how the -d (data) filter affects -m (metadata) -- by freeing mostly (or with the suggested -dusage=0, entirely[2]) empty data chunks back to unallocated so they can be reallocated as metadata chunks. So call it perverse if you want to, but it's an entirely logical perversion![3] =:^) Meanwhile, it's also possible, altho less common, to run into the opposite situation, out of data space, with metadata space left. That's actually rather interesting, as you can create files and sometimes even write just a small bit of content into them, since small files are entirely stored within the metadata leaf and don't require a data allocation. But as soon as you try to write anything of any significant size (a few KiB) to the new file, it'll ENOSPC when it tries to allocate a data extent and can't. --- [1] Yet. There's patches circulating that once thru discussion and merged, should let btrfs automatically handle at least the normal cases of data/metadata chunk imbalance. [2] If there's actual data in a chunk, a balance must have at least enough space left in ordered to create at least one more chunk, so as to be able to do the rewrite. But with a bit of luck, there's at least one chunk that's entirely empty, in which case usage=0 will free it without actually requiring space to create a new chunk to rewrite into, since there's nothing to rewrite. That's why the usage=0. If you're unlucky and there's no entirely empty chunks available for the balance to simply delete, then the usage=0 won't help. That's where the suggestion to temporarily add another device of at least a few gigs comes in, the idea being to give balance enough room to rewrite a few chunks on the new device, thereby freeing the space they would have used on the original device(s). Assuming an over-allocation, the balance should correct the problem, leaving enough space on the original device(s) so there's room to transfer the chunks back to the original device(s) using btrfs device delete <tmp-device>, and hopefully still leave some unallocated space left after that. [3] Sort of like the (in)famous MS Windows perversion of having to hit the start button to stop... -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
