This may be my ignorance, but I thought all modern unix filesystems created sparse files in this way?
-----Original Message----- From: Stuart Anderson <[EMAIL PROTECTED]> Date: Mon, 14 Apr 2008 15:45:03 To:Luke Scharf <[EMAIL PROTECTED]> Cc:zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] Confused by compressratio On Mon, Apr 14, 2008 at 05:22:03PM -0400, Luke Scharf wrote: > Stuart Anderson wrote: > >On Mon, Apr 14, 2008 at 09:59:48AM -0400, Luke Scharf wrote: > > > >>Stuart Anderson wrote: > >> > >>>As an artificial test, I created a filesystem with compression enabled > >>>and ran "mkfile 1g" and the reported compressratio for that filesystem > >>>is 1.00x even though this 1GB file only uses only 1kB. > >>> > >>> > >>ZFS seems to treat files filled with zeroes as sparse files, regardless > >>of whether or not compression is enabled. Try "dd if=/dev/urandom > >>of=1g.dat bs=1024 count=1048576" to create a file that won't exhibit > >>this behavior. Creating this file is a lot slower than writing zeroes > >>(mostly due to the speed of the urandom device), but ZFS won't treat it > >>like a sparse file, and it won't compress very well either. > >> > > > >However, I am still trying to reconcile the compression ratio as > >reported by compressratio vs the ratio of file sizes to disk blocks > >used (whether or not ZFS is creating sparse files). > > > > Can you describe the data you're storing a bit? Any big disk images? > Understanding the "mkfile" case would be a start, but the initial filesystem that started my confusion is one that has a number of ~50GByte mysql database files as well as a number of application code files. Here is another simple test to avoid any confusion/bugs related to NULL character sequeneces being compressed to nothing versus being treated as sparse files. In particular, a 2GByte file full of the output of /bin/yes: >zfs create export-cit/compress >cd /export/compress >/bin/df -k . Filesystem kbytes used avail capacity Mounted on export-cit/compress 1704858624 55 1261199742 1% /export/compress >zfs get compression export-cit/compress NAME PROPERTY VALUE SOURCE export-cit/compress compression on inherited from export-cit >/bin/yes | head -1073741824 > yes.dat >/bin/ls -ls yes.dat 185017 -rw-r--r-- 1 root root 2147483648 Apr 14 15:31 yes.dat >/bin/df -k . Filesystem kbytes used avail capacity Mounted on export-cit/compress 1704858624 92563 1261107232 1% /export/compress >zfs get compressratio export-cit/compress NAME PROPERTY VALUE SOURCE export-cit/compress compressratio 28.39x - So compressratio reports 28.39, but the ratio of file size to used disk for the only regular file on this filesystem, i.e., excluding the initial 55kB allocated for the "empty" filesystem is: 2147483648 / (185017 * 512) = 22.67 Calculated another way from "zfs list" for the entire filesystem: >zfs list /export/compress NAME USED AVAIL REFER MOUNTPOINT export-cit/compress 90.4M 1.17T 90.4M /export/compress is 2GB/90.4M = 2048 / 90.4 = 22.65 That still leaves me puzzled what the precise definition of compressratio is? Thanks. --- Stuart Anderson [EMAIL PROTECTED] http://www.ligo.caltech.edu/~anderson _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
