Qu Wenruo posted on Thu, 19 May 2016 09:33:19 +0800 as excerpted: > Graham Cobb wrote on 2016/05/18 14:29 +0100: >> Hi, >> >> I have a 6TB btrfs filesystem I created last year (about 60% used). It >> is my main data disk for my home server so it gets a lot of usage >> (particularly mail). I do frequent snapshots (using btrbk) so I have a >> lot of snapshots (about 1500 now, although it was about double that >> until I cut back the retention times recently). > > Even at 1500, it's still quite large, especially when they are all > snapshots. > > The biggest problem of large amount of snapshots is, it will make any > backref walk operation very slow. (O(n^3)~O(n^4)) > This includes: btrfs qgroup and balance, even fiemap (recently submitted > patch will solve fiemap problem though) > > The btrfs design ensures snapshot creation fast, but that comes with the > cost of backref walk. > > > So, unless some super huge rework, I would prefer to keep the number of > snapshots to a small amount, or avoid balance/qgroup.
Qu and Graham, As you may have seen on my previous posts, my normal snapshots recommendation is to try to keep under 250-300 per subvolume, and definitely under 3000 max, 2000 preferably, and 1000 if being conservative, per filesystem, thus allowing snapshotting of 6-8 subvolumes per filesystem before hitting the filesystem cap, due to scaling issues like the above that are directly related to number of snapshots. Also, recognizing that the btrfs quota code dramatically compounds the scaling issues, as well as because of the btrfs quota functionality still never actually working fully correctly on btrfs, I recommend turning it off unless it's definitely and specifically known to be needed, and if it's actually needed, I recommend strong consideration be given to use of a more mature filesystem where quotas are known to work reliably without the scaling issues they present on btrfs. So to Graham, are these 1.5K snapshots all of the same subvolume, or split into snapshots of several subvolumes? If it's all of the same subvolume or of only 2-3 subvolumes, you still have some work to do in terms of getting down to recommended snapshot levels. Also, if you have quotas on and don't specifically need them, try turning them off and see if that alone makes it workable. It's worth noting that a reasonable snapshot thinning program can help quite a bit here, letting you still keep a reasonable retention, and that 250-300 snapshots per subvolume fits very well within that model. Consider, if you're starting with say hourly snapshots, a year or even three months out, are you really going to care what specific hourly snapshot you retrieve a file from, or would daily or weekly snapshots do just as well and actually make finding an appropriate snapshot easier as there's less to go thru? Generally speaking, most people starting with hourly snapshots can delete every other snapshot, thinning by at least half, within a day or two, and those doing snapshots even more frequently can thin down to at least hourly within hours even, since if you haven't noticed a mistaken deletion or whatever within a few hours, chances are good that recovery from hourly snapshots is more than practical, and if you haven't noticed it within a day or two, recovery from say two-hourly or six-hourly snapshots will be fine. Similarly, a week out, most people can thin to twice-daily or daily snapshots, and by 4 weeks out, perhaps to Monday/ Wednesday/Friday snapshots. By 13 weeks (one quarter) out, weekly snapshots are often fine, and by six months (26 weeks) out, thinning to quarterly (13-week) snapshots may be practical. If not, it certainly should be within a year, tho well before a year is out, backups to separate media should have taken over allowing the oldest snapshots be dropped, finally reclaiming the space they were keeping locked up. And primarily to Qu... Is that 2K snapshots overall filesystem cap recommendation still too high, even if per-subvolume snapshots are limited to 300-ish? Or is the real problem per-subvolume snapshots, and as long as snapshots are limited to 300ish per subvolume, for people who have gone subvolume mad and have say 50 separate subvolumes being snapshotted (perhaps not too unreasonable in a VM context with each VM on its own subvolume), if a 300ish cap per subvolume is maintained, the 15K total snapshots per filesystem should still work reasonably well, so I should be able to drop the overall filesystem cap recommendation and simply recommend a per- subvolume snapshot cap of a few hundred? -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
