On Mon, Sep 18, 2017 at 7:11 PM, Timofey Titovets <nefelim...@gmail.com> wrote: > 2017-09-18 16:28 GMT+03:00 shally verma <shallyvermacav...@gmail.com>: >> On Mon, Sep 18, 2017 at 1:56 PM, Timofey Titovets <nefelim...@gmail.com> >> wrote: >>> 2017-09-18 10:36 GMT+03:00 shally verma <shallyvermacav...@gmail.com>: >>>> Hi >>>> >>>> I wanted to test btrfs compression using fio command but somehow >>>> during fio writes, I don't see code taking route of compression blocks >>>> where as If I do a copy to btrfs compression enabled mount point then >>>> I can easily see code falling through compression.c. >>>> >>>> Here's how I do my setup >>>> >>>> 1. mkfs.btrfs /dev/sdb1 >>>> 2. mount -t btrfs -o compress=zlib,compress-force /dev/sdb1 /mnt >>>> 3. cp <some large test file> /mnt >>>> 4. dmesg shows print staments from compression.c and zlib.c confirming >>>> compression routine was invoked during write >>>> 5. now, copy back from btrfs mount point to home directory also shows >>>> decompress call invokation >>>> >>>> Now, try same with fio commands: >>>> >>>> fio command >>>> >>>> fio --directory=/mnt/ --numjobs=1 --direct=0 --buffered=1 >>>> --ioengine=libaio --group_reporting --bs=64k --rw=write --iodepth=128 >>>> --name=test --size=10G --runtime=180 --time_based >>>> >>>> But it seems to write uncompressed data. >>>> >>>> Any help here? what's missing? >>>> >>>> Thanks >>>> Shally >>>> -- >>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in >>>> the body of a message to majord...@vger.kernel.org >>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>> >>> 1. mount -t btrfs -o compress=zlib,compress-force -> compress-force=zlib >>> 2. Tune fio to generate compressible data >>> >> How do I "tune" fio to generate data. I had assumed once compression >> is enabled on btrfs any system fwrite call will simply compress data >> into it .Isn't it so? >> Can you share fio command that I can test? >> Thanks >> Shally >>> >>> -- >>> Have a nice day, >>> Timofey. > > That useless to compress uncompressible data. > Also, as you enable compress, not compress-force > So after first uncompressible write btrfs just stop trying compress that file. > > From man fio: > buffer_compress_percentage=int > If this is set, then fio will attempt to provide I/O > buffer content (on WRITEs) that compresses to the specified level. Fio > does this by providing a mix of random data and a fixed pattern. The > fixed pattern is either > zeros, or the pattern specified by buffer_pattern. If > the pattern option is used, it might skew the compression ratio > slightly. Note that this is per block size unit, for file/disk wide > compression level that matches this > setting, you'll also want to set refill_buffers. > > buffer_compress_chunk=int > See buffer_compress_percentage. This setting allows fio > to manage how big the ranges of random data and zeroed data is. > Without this set, fio will provide buffer_compress_percentage of > blocksize random data, followed by > the remaining zeroed. With this set to some chunk size > smaller than the block size, fio can alternate random and zeroed data > throughout the I/O buffer. > > Good luck :)
Now. I did following: 1. mount -t btrfs -o compress-force=zlib /dev/sdb1 mnt 2. fio --directory=mnt/ --numjobs=1 --direct=0 --buffered=1 --bs=64k --rw=write --iodepth=128 --name=test --size=1G --buffer_compress_percentage=100 --buffer_pattern=0xFF --refill_buffer --ioengine=libaio 1GN file written uncompressed. Here no compression invoked (though compress-force=zlib) 3. cp mnt/test ./ --> copy back fio generated test file from btrfs mount point to local drive 4. hex dump test file (all FFs) -- confirmed that data is compressible no random data. 5. cp test mnt/ --> now, copy same test again back to mount point (reverse of step 3) . Now, here I see during copying compression is invoked. I am using kernel 4.9 and compress-foce is said to be working for kernel > 2.13 from wiki ... so I wonder what's so special with cp command which is not happening during fio writes??? Thanks Shally > -- > Have a nice day, > Timofey. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
