ZFS Compression Oddities with Amanda
I'm attempting to commit 7 TB of text to tape. It's presently stored in a natively gzip-9 compressed zvol, weighing in at 1.7 TB. My holding area is 5 TB, and is set to a native gzip-5 compression. The functional difference between gzip-5 and gzip-9 is not very much: Level 9 compression has a 4-8% advantage over level 5. The entire DLE (taken as files, not a snapshot) should fit quite comfortably in my holding area. It didn't! I watched my holding area balloon to 4 TB and keep right on going, as if it wasn't compressing at all. Is there any scenario in which this might happen? Would you recommend against a setup like the one I've described? I'm happy to offer any details needed, but this is probably a good start: Source: NAMEPROPERTY VALUE SOURCE zulu01/keyRepo type filesystem - zulu01/keyRepo creation Tue Jan 8 13:48 2013 - zulu01/keyRepo used 1.74T - zulu01/keyRepo available 6.60T - zulu01/keyRepo referenced1.74T - zulu01/keyRepo compressratio 4.65x - zulu01/keyRepo mounted yes- zulu01/keyRepo quota none default zulu01/keyRepo reservation none default zulu01/keyRepo recordsize128K default zulu01/keyRepo mountpoint/tank/datastoredefault zulu01/keyRepo sharenfs offdefault zulu01/keyRepo checksum on default zulu01/keyRepo compression gzip-9 local Hold: NAME PROPERTY VALUE SOURCE hold type filesystem - hold creation Fri Dec 7 10:34 2012 - hold used 257M - hold available 5.35T - hold referenced243M - hold compressratio 1.00x - hold mounted no - hold quota none default hold reservation none default hold recordsize128K default hold mountpoint/hold default hold sharenfs offdefault hold checksum on default hold compression gzip-5 local -- Guy Sisalli gsisa...@mynetwatchman.com IT Operations Manager myNetWatchman.com Atlanta, GA +1.631.708.8346
Re: ZFS Compression Oddities with Amanda
Guy, Are you also using amanda software compression? Using compression on the holding disk is probably just a waste of CPU as the data is compressed once and decompressed once, but if it the only way it can fit in the holding disk. If you use amanda software compression, then the data is compressed on holding disk and on tape. Jean-Louis On 05/01/2013 01:10 PM, Guy Sisalli wrote: I'm attempting to commit 7 TB of text to tape. It's presently stored in a natively gzip-9 compressed zvol, weighing in at 1.7 TB. My holding area is 5 TB, and is set to a native gzip-5 compression. The functional difference between gzip-5 and gzip-9 is not very much: Level 9 compression has a 4-8% advantage over level 5. The entire DLE (taken as files, not a snapshot) should fit quite comfortably in my holding area. It didn't! I watched my holding area balloon to 4 TB and keep right on going, as if it wasn't compressing at all. Is there any scenario in which this might happen? Would you recommend against a setup like the one I've described? I'm happy to offer any details needed, but this is probably a good start: Source: NAMEPROPERTY VALUE SOURCE zulu01/keyRepo type filesystem - zulu01/keyRepo creation Tue Jan 8 13:48 2013 - zulu01/keyRepo used 1.74T - zulu01/keyRepo available 6.60T - zulu01/keyRepo referenced1.74T - zulu01/keyRepo compressratio 4.65x - zulu01/keyRepo mounted yes- zulu01/keyRepo quota none default zulu01/keyRepo reservation none default zulu01/keyRepo recordsize128K default zulu01/keyRepo mountpoint/tank/datastoredefault zulu01/keyRepo sharenfs offdefault zulu01/keyRepo checksum on default zulu01/keyRepo compression gzip-9 local Hold: NAME PROPERTY VALUE SOURCE hold type filesystem - hold creation Fri Dec 7 10:34 2012 - hold used 257M - hold available 5.35T - hold referenced243M - hold compressratio 1.00x - hold mounted no - hold quota none default hold reservation none default hold recordsize128K default hold mountpoint/hold default hold sharenfs offdefault hold checksum on default hold compression gzip-5 local
Re: ZFS Compression Oddities with Amanda
I'd thought for both ZFS and the newer (LTO) tape drives that HW-compression was determined on a block by block basic (if enabled) so that expansion of data would not occur. Granted, this does nothing to help with on CPU usage, but I'd thought it did save, rather, preserve, storage volume. On Wed, May 01, 2013 at 01:41:35PM -0400, Jean-Louis Martineau wrote: Guy, Are you also using amanda software compression? Using compression on the holding disk is probably just a waste of CPU as the data is compressed once and decompressed once, but if it the only way it can fit in the holding disk. If you use amanda software compression, then the data is compressed on holding disk and on tape. Jean-Louis On 05/01/2013 01:10 PM, Guy Sisalli wrote: I'm attempting to commit 7 TB of text to tape. It's presently stored in a natively gzip-9 compressed zvol, weighing in at 1.7 TB. My holding area is 5 TB, and is set to a native gzip-5 compression. The functional difference between gzip-5 and gzip-9 is not very much: Level 9 compression has a 4-8% advantage over level 5. The entire DLE (taken as files, not a snapshot) should fit quite comfortably in my holding area. It didn't! I watched my holding area balloon to 4 TB and keep right on going, as if it wasn't compressing at all. Is there any scenario in which this might happen? Would you recommend against a setup like the one I've described? I'm happy to offer any details needed, but this is probably a good start: Source: NAMEPROPERTY VALUE SOURCE zulu01/keyRepo type filesystem - zulu01/keyRepo creation Tue Jan 8 13:48 2013 - zulu01/keyRepo used 1.74T - zulu01/keyRepo available 6.60T - zulu01/keyRepo referenced1.74T - zulu01/keyRepo compressratio 4.65x - zulu01/keyRepo mounted yes- zulu01/keyRepo quota none default zulu01/keyRepo reservation none default zulu01/keyRepo recordsize128K default zulu01/keyRepo mountpoint/tank/datastoredefault zulu01/keyRepo sharenfs offdefault zulu01/keyRepo checksum on default zulu01/keyRepo compression gzip-9 local Hold: NAME PROPERTY VALUE SOURCE hold type filesystem - hold creation Fri Dec 7 10:34 2012 - hold used 257M - hold available 5.35T - hold referenced243M - hold compressratio 1.00x - hold mounted no - hold quota none default hold reservation none default hold recordsize128K default hold mountpoint/hold default hold sharenfs offdefault hold checksum on default hold compression gzip-5 local --- Brian R Cuttler brian.cutt...@wadsworth.org Computer Systems Support(v) 518 486-1697 Wadsworth Center(f) 518 473-6384 NYS Department of HealthHelp Desk 518 473-0773
