Re: Will Btrfs have an official command to "uncow" existing files?

2016-08-22 Thread Tomokhov Alexander
Thanks for the in-depth answer.

Well, "simple enough process" is still a sequence of steps which must be 
carefully done. In a proper order with correct parameters depending on 
environment. It's work with data, data which can be invaluable.
No, really, I'm not a beginner user. I use Arch Linux everyday from 2009, 
program in different languages and so on. But even a few ordered manual 
commands for changing file attribute involving quite dangerous "mv" and "cp" 
(overwrite case) is something very suspicious.

"2a" - step depends on whether another filesystem simply exists, better if it's 
free, has enough space, supports same file permission features, etc. Requires 
time to figure out these conditions, not suitable for all systems.

"2b" - step with "mv out". Out to where? What if the file with the same name 
already exists in a destination directory you "mv out". Not reliable. Ok, need 
to create temporary directory. Where, how to call it then - involves 
conditional checks performed by user.

Similarly creation of empty file should also satisfy the condition that it's 
name is unique in the directory.

Additionally all existing ways of "uncow" require manual free space check 
beforehand. User must control and monitor if the file is currently not opened. 
I'm sure I missed something else.
These all are problems that are unrelated to file attributes itself, but user 
must think of them for some reason. An official specialized tool could 
automatically track all these conditions, perform the right sequence of actions 
and report to user results.

Yes I do take into consideration that there are situations when "uncow" cannot 
be actually applied to a file for the reasons you described. No snapshots atm 
in my case and, for example, I have firefox sqlite database file with 900+ 
extents on a rotational disk. I wouldn't say it's noticeable, but at least 
desire the number of extents not to increase further so that I won't notice it 
ever. I admit that Btrfs may defragment it, but may not. Sometimes we need a 
more controllable approach.

22.08.2016, 05:00, "Duncan" <1i5t5.dun...@cox.net>:
> Tomokhov Alexander posted on Sun, 21 Aug 2016 21:59:36 +0300 as excerpted:
>
>>  Btrfs wiki FAQ gives a link to example Python script:
>>  https://github.com/stsquad/scripts/blob/master/uncow.py
>>
>>  But such a crucial and fundamental tool must exist in stock btrfs-progs.
>>  Filesystem with CoW technology at it's core must provide user sufficient
>>  control over CoW aspects. Running 3rd-party or manually written scripts
>>  for filesystem properties/metadata manipulation is not convenient, not
>>  safe and definitely not the way it must be done.
>
> Why? No script or dedicated tool needed as it's a simple enough process.
>
> Simply:
>
> 1. chattr +C (that being nocow) the containing directory.
>
> Then either:
>
> 2a. mv the file to and from another filesystem, so it's actually created
> new in the directory and thus inherits the nocow attribute at file
> creation,
>
> or
>
> 2b. mv out and then cp the file back into place with --reflink=never,
> again, forcing the file to be created new in the directory, so it
> inherits the nocow attribute at creation,
>
> OR (replacing both steps above)
>
> Create the empty file (using touch or similar), set it nocow, and use cat
> srcfile >> destfile style redirection to fill it, so the file again gets
> the nocow attribute set before it has content, but allowing you to set
> only the file nocow, without setting the containing directory nocow.
>
> Of course there's no exception here to the general case, if you're doing
> the same thing to a whole bunch of files, setting up a script to do it
> may be more efficient than doing it to each one manually one by one, and
> a script could be useful there, but that's a general rule, nothing
> exceptional for btrfs nocow, and a script or fancy tool isn't actually
> required, regardless.
>
> The point being, cow is the default case, and should work /reasonably/
> well in most cases, certainly well enough so that normal people doing
> normal things shouldn't need to worry about it. The only people who will
> need to worry about it, therefore, are people worried about the last bit
> of optimization possible to various corner-case use-cases that don't
> match default assumptions very well, and it's precisely these sorts of
> people that are /technical/ enough to be able to understand both why they
> might want nocow (and what the positives and negatives are going to be),
> and how to actually get it.
>
>>  Also is it possible (at least in theory) to "uncow" files being
>>  currently opened in-place? Without the trickery with cr

Re: Will Btrfs have an official command to "uncow" existing files?

2016-08-22 Thread Tomokhov Alexander
Oh, didn't know that XFS is going to have many of Btrfs features and continues 
to evolve. Thank you for the answer.

22.08.2016, 23:14, "Jeff Mahoney" <je...@suse.com>:
> On 8/21/16 2:59 PM, Tomokhov Alexander wrote:
>>  Btrfs wiki FAQ gives a link to example Python script: 
>> https://github.com/stsquad/scripts/blob/master/uncow.py
>>
>>  But such a crucial and fundamental tool must exist in stock btrfs-progs. 
>> Filesystem with CoW technology at it's core must provide user sufficient 
>> control over CoW aspects. Running 3rd-party or manually written scripts for 
>> filesystem properties/metadata manipulation is not convenient, not safe and 
>> definitely not the way it must be done.
>>
>>  Also is it possible (at least in theory) to "uncow" files being currently 
>> opened in-place? Without the trickery with creation & renaming of files or 
>> directories. So that running "chattr +C" on a file would be sufficient. If 
>> possible, is it going to be implemented?
>
> XFS is looking to do this via fallocate using a flag that all file
> systems can choose to honor. Once that lands, it would make sense for
> btrfs to use it as well. The idea is that when you pass the flag in, we
> examine the range and CoW anything that has a refcount != 1.
>
> That code hasn't landed yet though. The last time I saw it posted was
> June. I don't speak with knowledge of the integration plan, but it
> might just be queued up for the next merge window now that the reverse
> mapping patches have landed in 4.8.
>
> -Jeff
>
> --
> Jeff Mahoney
> SUSE Labs
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Will Btrfs have an official command to "uncow" existing files?

2016-08-21 Thread Tomokhov Alexander
Btrfs wiki FAQ gives a link to example Python script: 
https://github.com/stsquad/scripts/blob/master/uncow.py

But such a crucial and fundamental tool must exist in stock btrfs-progs. 
Filesystem with CoW technology at it's core must provide user sufficient 
control over CoW aspects. Running 3rd-party or manually written scripts for 
filesystem properties/metadata manipulation is not convenient, not safe and 
definitely not the way it must be done.

Also is it possible (at least in theory) to "uncow" files being currently 
opened in-place? Without the trickery with creation & renaming of files or 
directories. So that running "chattr +C" on a file would be sufficient. If 
possible, is it going to be implemented?
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html