Thanks for the replies! I'll clear up the unknowns.
The nocow attribute was set on the folder prior to any data being written. So based on your understanding the files are truly nocow and must therefore be not compressed (an inherited attribute from the subvolume). I think it's safe to say the file wasn't skipped if the number of extents changed? So if it isn't really compressed, and it wasn't skipped, is there any reason to still think filefrag is confused about the results? Total used space is about 1T and free space is approximately 9t. Training-flat.vmdk is 60g Thanks again, Donald On Fri, Jul 10, 2015 at 11:30 PM, Duncan <1i5t5.dun...@cox.net> wrote: > Donald Pearson posted on Fri, 10 Jul 2015 15:57:46 -0500 as excerpted: > >> If I'm reading this right, my most fragmented file >> (Training-flat.vmdk) is now almost 3x more fragmented? > > [snip to context for brevity] > >> # filefrag /mnt2/tank/virtual_machines/virtual_machines/Training/* >> /mnt2/tank/virtual_machines/virtual_machines/Training/Training-flat.vmdk: >> 1444 extents found > >> # lsattr /mnt2/tank/virtual_machines/virtual_machines/Training/ >> --------c------C >> /mnt2/tank/virtual_machines/virtual_machines/Training/Training-flat.vmdk > >> # ./btrfs fi defragment -vr /mnt2/tank/virtual_machines >> /mnt2/tank/virtual_machines/virtual_machines/Training/Training-flat.vmdk >> btrfs-progs v4.1 > >> # filefrag /mnt2/tank/virtual_machines/virtual_machines/Training/* >> /mnt2/tank/virtual_machines/virtual_machines/Training/Training-flat.vmdk: >> 4090 extents found > > FWIW RE the lsattr results, the chattr manpage says c=compressed, C=nocow > (disabling copy-on-write). > > The combination of the compressed and nocow attributes _seriously_ > complicates the picture because their interaction is unknown, > given the information provided. > > 1) As the chattr manpage warns, on btrfs the C/nocow attribute should > be set on new/empty files. If set on a file with existing data blocks, > when nocow stability takes effect is undefined. > > The sequence of events in your post doesn't detail when the file was > marked nocow, so we basically haven't a clue whether that nocow > attribute has actually taken effect and the file stabilized, or not. > > 2) On btrfs, nocow has the effect of disabling both compression and > checksumming. This is necessary because btrfs takes advantage of > cow so it has to calculate checksum only on newly written blocks, > not the whole previous extent, and because newly written data may > compress more or less effectively than the old, changing the > compressed size, no problem with cow since the new data is written > elsewhere, a serious problem with nocow, since now you have new data > that's likely of a different size trying to fit into the same space > as the old data. > > So nocow would ordinarily override and disable compression, but > due to issue #1, we don't know whether nocow is actually on or not, > so we don't know if compression is actually disabled or not. > > 3) Filefrag doesn't know about btrfs compression, which works in > 128 KiB blocks, and counts each 128 KiB block as a separarate extent. > > Due to #2, we don't know if the file is actually compressed or not, > and you didn't list the size of the file either, but if it /is/ > compressed, we'd expect filefrag to report one fragment per 128 KiB, > 1/8 MiB, so divide those 4090 reported extents by 8... 511+ MB, > basically half a GiB. > > If that's about the size of the file, then we can gather from > the evidence that: > > a) The file isn't actually nocow as that would have disabled compression. > > b) The file had the compression and nocow attributes set after creation, > so nocow didn't apply, and the defrag actually triggered the compression > of a previously uncompressed file, thereby causing filefrag to report > individual 128 KiB compression blocks as extents. > > If the file is *NOT* about half a GiB in size, then something else > happened. > > Among other things, your existing free space may be so fragmented that > defrag couldn't find sufficiently large blocks of space to reduce > fragmentation, so it actually got _more_ fragmented. > > It's also worth noting that by default, defrag skips what it considers > "large" extents (IIRC 256 KiB... or was it 256 MiB?). Use the -t option > set to something like 2G, to tell it to defrag everything. (Given btrfs' > nominal data chunk size of 1 GiB, above that isn't going to do a lot of > good in any case, and don't use values above 3G as there's an integer > overflow bug in btrfs-progs earlier than the just released 4.1.1, which > effectively makes it very small, instead.) > > Whatever the case, having both the c/compression and C/nocow attributes > on those files isn't a good idea, since one or the other will be > a lie. And if you really want nocow, as indicated above, the file > must be created with nocow. The easiest way to do that is to set nocow > on the directory the files will be created in. For existing files, after > the dir they will be placed in is set nocow, copy them into place in such > a way that the new copy is actually created. (The easiest two ways to > ensure that are (a) copy/move across a filesystem boundary, or (b) cat > the file into place using redirection: cat src > dest.) > > -- > Duncan - List replies preferred. No HTML msgs. > "Every nonfree program has a lord, a master -- > and if you use the program, he is your master." Richard Stallman > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
