Tomokhov Alexander posted on Sun, 21 Aug 2016 21:59:36 +0300 as excerpted: > Btrfs wiki FAQ gives a link to example Python script: > https://github.com/stsquad/scripts/blob/master/uncow.py > > But such a crucial and fundamental tool must exist in stock btrfs-progs. > Filesystem with CoW technology at it's core must provide user sufficient > control over CoW aspects. Running 3rd-party or manually written scripts > for filesystem properties/metadata manipulation is not convenient, not > safe and definitely not the way it must be done.
Why? No script or dedicated tool needed as it's a simple enough process. Simply: 1. chattr +C (that being nocow) the containing directory. Then either: 2a. mv the file to and from another filesystem, so it's actually created new in the directory and thus inherits the nocow attribute at file creation, or 2b. mv out and then cp the file back into place with --reflink=never, again, forcing the file to be created new in the directory, so it inherits the nocow attribute at creation, OR (replacing both steps above) Create the empty file (using touch or similar), set it nocow, and use cat srcfile >> destfile style redirection to fill it, so the file again gets the nocow attribute set before it has content, but allowing you to set only the file nocow, without setting the containing directory nocow. Of course there's no exception here to the general case, if you're doing the same thing to a whole bunch of files, setting up a script to do it may be more efficient than doing it to each one manually one by one, and a script could be useful there, but that's a general rule, nothing exceptional for btrfs nocow, and a script or fancy tool isn't actually required, regardless. The point being, cow is the default case, and should work /reasonably/ well in most cases, certainly well enough so that normal people doing normal things shouldn't need to worry about it. The only people who will need to worry about it, therefore, are people worried about the last bit of optimization possible to various corner-case use-cases that don't match default assumptions very well, and it's precisely these sorts of people that are /technical/ enough to be able to understand both why they might want nocow (and what the positives and negatives are going to be), and how to actually get it. > Also is it possible (at least in theory) to "uncow" files being > currently opened in-place? Without the trickery with creation & renaming > of files or directories. So that running "chattr +C" on a file would be > sufficient. If possible, is it going to be implemented? It's software. Of course it's possible, tho it's also possible the negatives make it not worth the trouble. If the implementation simply creates a new file and does a --reflink=never cp in the background when the nocow file attribute is set, it may not be worth it. As to whether it'll ultimately be implemented, I don't know as I'm not a dev. But even if it is ultimately implemented, it might well be five years out or longer, because there's simply way more "it'd be nice" btrfs- related ideas out there than there are devs working on implementations, and projecting more than five years out in a software development environment like the Linux kernel doesn't make a lot of sense, so five years out or longer is likely, but beyond that, nobody really knows. Add to that the fact that a lot of existing btrfs features took rather longer to implement and stabilize than originally projected, and... Meanwhile, if you aren't already, be aware that the basic concepts of snapshots locking in references to existing extents as unchangeable (snapshots being a btrfs feature that depends on its cow nature), and nocow setting files as rewrite-in-place, basically can't work with each other at the concept level, because once a snapshot is taken, extents referenced by that snapshot really /cannot/ change until that snapshot is deleted, something that can only be true if the extents are copy-on-write. To work around this btrfs makes the nocow attribute weaker than the snapshot locking a particular extent in place, using a process sometimes referred to as cow-once or cow1. When a change is written to a(n otherwise) nocow file that has been snapshotted and thus has its extents locked in place, the block of the file that is changed will have to be cowed, despite the nocow attribute. However, the nocow attribute is kept, and any further changes to that block will be rewritten to the new location it was copied to without further cowing of that block, of course until the next snapshot locks that location in place as well, at which further writes to the same block will of course cow it once more, to a third location, which will again remain in place until yet another snapshot... So snapshotting a nocow file means it's no longer absolutely nocow, it's now cow1. If the rewrite rate is higher than the snapshot rate, the nocow should still have some effect, but where the snapshot rate is higher than the rewrite rate, the effect will be as if the file wasn't nocow at all, because each snapshot will lock in the file as it then exists. It is for this reason that the recommendation is to keep nocow files in a dedicated subvolume, so snapshots to the parent subvolume exclude the nocow files, and to minimize snapshotting of the nocow subvolume. As long as snapshotting is occurring at all, however, there will be some fragmentation over time, but by combining a limited snapshotting rate with a reasonable defrag schedule targeting the nowcow files as well, fragmentation should at least remain under control. And of course nocow has other negatives as well. Setting nocow turns off both compression (if you otherwise have compression on) and checksumming, thus killing two other major btrfs features, including its realtime file integrity validation via checksumming. So there are definitely negatives to nocow that must be weighed before setting it. But it's worth keeping in mind that all of these features are really practical due to cow in the first place, the reason other filesystems don't tend to have them, and that while there is definitely a tradeoff to cow vs. nocow, setting nocow doesn't turn off features you'd have in conventional filesystems anyway, so it's not as if you're losing features you'd have if you weren't using btrfs with its cow by default, in the first place. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
