Re: Packing Methods
Packing the same data more than once sometimes results in a larger file than you would have if it only had been packed once. Jim Alain Benveniste wrote: Ce message est au format MIME. Comme votre programme de lecture de courriers ne comprend pas ce format, il se peut que tout ou une partie de ce message soit illisible. --B_3323942706_366709 Content-type: text/plain; charset="ISO-8859-1" Content-transfer-encoding: quoted-printable Richard, You have the same questions I had when I started to put in place our DR solution. We also have 3590E drives and I never tried to remove the hard drive default compaction. I don=B9t see a reason for that. Now choosing software compaction is a must if you have enough cpu to do the work for backup and ALSO for restore. For us we can spend more time to use software compaction because we know that we have enough cpu to do the work at restor= e time offsite. The gain at restore justifies to take more time at backup processing. It=B9s true too that software compaction takes less tapes than with no compaction. If you have many dasds to backup and a time constraint to restore i would suggest you to both use hard and software compactions. Our idea is to say that when we restore in a DR test the cpu is used ONLY for restore. Why not fully using it ! Regards Alain Benveniste =20 Le 29/04/09 20:46, =AB=A0Schuh, Richard=A0=BB rsc...@visa.com a =E9crit=A0: We are working on a DR process. I notice that the defaults for a Hidro ba= ckup include the PACK option which tells Hidro to pack, or condense in some fashion, its output. The output is being written to 3590E drives. It appe= ars that there are three choices we can make for condensing the data: softwar= e only, hardware only, or a combination of the two (uncompacted was purpose= ly omitted from the list). Which is likely to give the best results? Does software compaction produce consistently lower output volumes than lettin= g the drive do it? Is there anything to be gained by using both h/w and s/w? Obviously, software compaction costs in terms of cpu time. The question i= s, is it worth the time spent? =20 Regards,=20 Richard Schuh=20 =20 =20 =20 =20 --B_3323942706_366709 Content-type: text/html; charset="ISO-8859-1" Content-transfer-encoding: quoted-printable HTML HEAD TITLERe: Packing Methods/TITLE /HEAD BODY FONT FACE=3D"Verdana, Helvetica, Arial"SPAN STYLE=3D'font-size:12.0px'Richa= rd,BR BR You have the same questions I had when I started to put nbsp;in place our = DR solution. We also have 3590E drives and I never tried to remove the hard = drive default compaction. I don#8217;t see a reason for that. Now choosing = software compaction is a must if you have enough cpu to do the work for back= up and ALSO for restore. For us we can spend more time to use software compa= ction because we know that we have enough cpu to do the work at restore time= offsite. The gain at restore justifies to take more time at backup processi= ng. It#8217;s true too that software compaction takes less tapes than with = no compaction.BR If you have many dasds to backup and a time constraint to restore i would s= uggest you to both use hard and software compactions. Our idea is to say tha= t when we restore in a DR test the cpu is used ONLY for restore. Why not ful= ly using it !BR BR RegardsBR Alain Benveniste nbsp;nbsp;nbsp;BR BR Le 29/04/09 20:46, laquo;=A0Schuh, Richard=A0raquo; lt;rsc...@visa.comgt; a= eacute;crit=A0:BR BR /SPAN/FONTBLOCKQUOTESPAN STYLE=3D'font-size:12.0px'FONT FACE=3D"Arial"= We are working on a DR process. I notice that the defaults for a Hidro back= up include the PACK option which tells Hidro to pack, or condense in some fa= shion, its output. The output is being written to 3590E drives. It appears t= hat there are three choices we can make for condensing the data: software on= ly, hardware only, or a combination of the two (uncompacted was purposely om= itted from the list). Which is likely to give the best results? Does softwar= e compaction produce consistently lower output volumes than letting the driv= e do it? Is there anything to be gained by using both h/w and s/w? Obviously= , software compaction costs in terms of cpu time. The question is, is it wor= th the time spent?BR nbsp;BR Regards, BR Richard Schuh BR nbsp;BR nbsp;BR nbsp;BR /FONTFONT FACE=3D"Verdana, Helvetica, Arial"BR /FONT/SPAN/BLOCKQUOTESPAN STYLE=3D'font-size:12.0px'FONT FACE=3D"Verda= na, Helvetica, Arial"BR /FONT/SPAN /BODY /HTML --B_3323942706_366709-- -- Jim Bohnsack Cornell University (972) 596-6377 home/office (972) 342-5823 cell jab...@cornell.edu
Re: Packing Methods
-Original Message- From: The IBM z/VM Operating System [mailto:ib...@listserv.uark.edu] On Behalf Of Tom Duerbusch Sent: Friday, May 01, 2009 10:21 AM To: IBMVM@LISTSERV.UARK.EDU Subject: Re: Packing Methods The reason to turn off 3590E hardware compression, is if you having problems feeding the beast. You get faster backups if you can keep the drive going. An IBM 3590E writes to tape at 14 MBs. If hardware compression is turned on and you are getting 3:1 compression, then you have to feed the controller at 3*14 or 42 MBs. If you can, you have the best of both worlds, speed and amount of data on a single tape. If you can't then you need to choose speed vs high data storage. We have no problems feeding the beast. We are writing to a remote SL3000 across the wire, a high speed wire to be sure, but one that is shared by other systems doing similar things. The network people have determined that our workload will not strain the capacity, and they appear to be correct. A backup of our larger VTAPE (VSSI version) library takes between 2.5 and 3 hours when the output is to local tape drives (Ficon connected) and only a few minutes more, less than 15, when writing to the remote site which is 1500 miles away. Doing software compression, only makes sense to me if you are going over slow (escon) channels. Given all that In DR tests, less tapes seems to be a good thing. With the current high-capacity, high-speed drives, the number of tapes is well under control. The VTAPE library backup mentioned above requires only a single tape to back a 26 (3390-03) volume library that typically runs 80% or more occupancy. The DR site has better, faster, hardware then most of our shops. (everything is ficon) You may buy less MIPS but you can't buy less I/O thruput. It takes more CPU to compress data then it takes to expand the data. i.e. worry about your own MIPS. We have the same kind of h/w as the DR site. It is an LPAR carved out of a system at our second continental datacenter. We may on occasion, for a short while, have better equipment than it. The prudent path is usually to install the new hardware on the VM development system before implementing it on the production systems. We get the devices and processors before that are deemed acceptable as replacements for the hardware used by TPF. In general, I don't see the benefit in software compression, when hardware compression is available. But if you tested the difference in your site, and you have come to a software compression conclusion, more power to you. Each site has a different set of concerns. The original question had as much to do with is there anything to be gained using both types of compression because the defaults for both Hidro and the tape drives is to compress the data or, if not, which was likely to be better at doing the compression. It would appear that h/w alone is the answer. That is, it is if the SL3000 does the compaction prior to sending the data across the wire. Tom Duerbusch THD Consulting Alain Benveniste a.benveni...@free.fr 4/30/2009 6:25 AM Richard, You have the same questions I had when I started to put in place our DR solution. We also have 3590E drives and I never tried to remove the hard drive default compaction. I don¹t see a reason for that. Now choosing software compaction is a must if you have enough cpu to do the work for backup and ALSO for restore. For us we can spend more time to use software compaction because we know that we have enough cpu to do the work at restore time offsite. The gain at restore justifies to take more time at backup processing. It¹s true too that software compaction takes less tapes than with no compaction. If you have many dasds to backup and a time constraint to restore i would suggest you to both use hard and software compactions. Our idea is to say that when we restore in a DR test the cpu is used ONLY for restore. Why not fully using it ! Regards Alain Benveniste Le 29/04/09 20:46, « Schuh, Richard » rsc...@visa.com a écrit : We are working on a DR process. I notice that the defaults for a Hidro backup include the PACK option which tells Hidro to pack, or condense in some fashion, its output. The output is being written to 3590E drives. It appears that there are three choices we can make for condensing the data: software only, hardware only, or a combination of the two (uncompacted was purposely omitted from the list). Which is likely to give the best results? Does software compaction produce consistently lower output volumes than letting the drive do it? Is there anything to be gained by using both h/w and s/w? Obviously, software compaction costs in terms of cpu time. The question is, is it worth the time spent? Regards, Richard Schuh
Re: Packing Methods
Ok so you can feed the beast, when running software and hardware compression. Good. The software compression is able to read/compress/send down the channel at a rate of 14 MBs. The hardware compression isn't going to do much. Normally, compressing a compressed file makes the file grow slightly. So the 14 MBs keeps the tape moving. You might be able to send data down a little faster until the tape controller is full, then you will be throttled down to native tape drive speed. If you turned off software compression, then you need to send to the drive at a rate near 42 MBs (3:1 compression) in order to keep the tape moving. That may or may not be doable at your shop, with your current hardware. You're ficon, which can do it, but I've never experienced distance communication at or near the rated speed. Always less. How much less and does it make a difference? 14 MBs is about 112 mbs. 42 MBs is 336 mbs. And how many transmissions are happening over that line at the same time? All factors feed into keeping the tape heads spinning. I would still test turning off software compression and see what you get. Buying back mainframe CPU cycles is normally a good thing. But if they would just go to waste anyway, and software compression is the only thing that can keep the drive going, then that is a good use of mainframe cycles. However, leaving on hardware compression, really doesn't have much of a cost. If you later add in encryption, the rule to follow is to compress first then encrypt. If you encrypt first, you will have a much larger file as compression will have no effect. We were specing out SAN attached IBM 3592 tape drives. Couldn't be done within budget as most of the network was 1 gb switches, and the 3592, to be driven, needed 2 gb switch (to operate one drive) and 4 gb switch to operate both drives. The SAN disk space, which is fairly old, needed to be upgraded to support the higher transfer rate. (Basically, cheaper to replace the SAN, but no budget for that.) Tom Duerbusch THD Consulting Schuh, Richard rsc...@visa.com 5/4/2009 12:30 PM -Original Message- From: The IBM z/VM Operating System [mailto:ib...@listserv.uark.edu] On Behalf Of Tom Duerbusch Sent: Friday, May 01, 2009 10:21 AM To: IBMVM@LISTSERV.UARK.EDU Subject: Re: Packing Methods The reason to turn off 3590E hardware compression, is if you having problems feeding the beast. You get faster backups if you can keep the drive going. An IBM 3590E writes to tape at 14 MBs. If hardware compression is turned on and you are getting 3:1 compression, then you have to feed the controller at 3*14 or 42 MBs. If you can, you have the best of both worlds, speed and amount of data on a single tape. If you can't then you need to choose speed vs high data storage. We have no problems feeding the beast. We are writing to a remote SL3000 across the wire, a high speed wire to be sure, but one that is shared by other systems doing similar things. The network people have determined that our workload will not strain the capacity, and they appear to be correct. A backup of our larger VTAPE (VSSI version) library takes between 2.5 and 3 hours when the output is to local tape drives (Ficon connected) and only a few minutes more, less than 15, when writing to the remote site which is 1500 miles away. Doing software compression, only makes sense to me if you are going over slow (escon) channels. Given all that In DR tests, less tapes seems to be a good thing. With the current high-capacity, high-speed drives, the number of tapes is well under control. The VTAPE library backup mentioned above requires only a single tape to back a 26 (3390-03) volume library that typically runs 80% or more occupancy. The DR site has better, faster, hardware then most of our shops. (everything is ficon) You may buy less MIPS but you can't buy less I/O thruput. It takes more CPU to compress data then it takes to expand the data. i.e. worry about your own MIPS. We have the same kind of h/w as the DR site. It is an LPAR carved out of a system at our second continental datacenter. We may on occasion, for a short while, have better equipment than it. The prudent path is usually to install the new hardware on the VM development system before implementing it on the production systems. We get the devices and processors before that are deemed acceptable as replacements for the hardware used by TPF. In general, I don't see the benefit in software compression, when hardware compression is available. But if you tested the difference in your site, and you have come to a software compression conclusion, more power to you. Each site has a different set of concerns. The original question had as much to do with is there anything to be gained using both types of compression because the defaults for both Hidro and the tape drives is to compress the data or, if not, which was likely
Re: Packing Methods
The reason to turn off 3590E hardware compression, is if you having problems feeding the beast. You get faster backups if you can keep the drive going. An IBM 3590E writes to tape at 14 MBs. If hardware compression is turned on and you are getting 3:1 compression, then you have to feed the controller at 3*14 or 42 MBs. If you can, you have the best of both worlds, speed and amount of data on a single tape. If you can't then you need to choose speed vs high data storage. Doing software compression, only makes sense to me if you are going over slow (escon) channels. Given all that In DR tests, less tapes seems to be a good thing. The DR site has better, faster, hardware then most of our shops. (everything is ficon) You may buy less MIPS but you can't buy less I/O thruput. It takes more CPU to compress data then it takes to expand the data. i.e. worry about your own MIPS. In general, I don't see the benefit in software compression, when hardware compression is available. But if you tested the difference in your site, and you have come to a software compression conclusion, more power to you. Each site has a different set of concerns. Tom Duerbusch THD Consulting Alain Benveniste a.benveni...@free.fr 4/30/2009 6:25 AM Richard, You have the same questions I had when I started to put in place our DR solution. We also have 3590E drives and I never tried to remove the hard drive default compaction. I don¹t see a reason for that. Now choosing software compaction is a must if you have enough cpu to do the work for backup and ALSO for restore. For us we can spend more time to use software compaction because we know that we have enough cpu to do the work at restore time offsite. The gain at restore justifies to take more time at backup processing. It¹s true too that software compaction takes less tapes than with no compaction. If you have many dasds to backup and a time constraint to restore i would suggest you to both use hard and software compactions. Our idea is to say that when we restore in a DR test the cpu is used ONLY for restore. Why not fully using it ! Regards Alain Benveniste Le 29/04/09 20:46, « Schuh, Richard » rsc...@visa.com a écrit : We are working on a DR process. I notice that the defaults for a Hidro backup include the PACK option which tells Hidro to pack, or condense in some fashion, its output. The output is being written to 3590E drives. It appears that there are three choices we can make for condensing the data: software only, hardware only, or a combination of the two (uncompacted was purposely omitted from the list). Which is likely to give the best results? Does software compaction produce consistently lower output volumes than letting the drive do it? Is there anything to be gained by using both h/w and s/w? Obviously, software compaction costs in terms of cpu time. The question is, is it worth the time spent? Regards, Richard Schuh
Re: Packing Methods
Richard, You have the same questions I had when I started to put in place our DR solution. We also have 3590E drives and I never tried to remove the hard drive default compaction. I don¹t see a reason for that. Now choosing software compaction is a must if you have enough cpu to do the work for backup and ALSO for restore. For us we can spend more time to use software compaction because we know that we have enough cpu to do the work at restore time offsite. The gain at restore justifies to take more time at backup processing. It¹s true too that software compaction takes less tapes than with no compaction. If you have many dasds to backup and a time constraint to restore i would suggest you to both use hard and software compactions. Our idea is to say that when we restore in a DR test the cpu is used ONLY for restore. Why not fully using it ! Regards Alain Benveniste Le 29/04/09 20:46, « Schuh, Richard » rsc...@visa.com a écrit : We are working on a DR process. I notice that the defaults for a Hidro backup include the PACK option which tells Hidro to pack, or condense in some fashion, its output. The output is being written to 3590E drives. It appears that there are three choices we can make for condensing the data: software only, hardware only, or a combination of the two (uncompacted was purposely omitted from the list). Which is likely to give the best results? Does software compaction produce consistently lower output volumes than letting the drive do it? Is there anything to be gained by using both h/w and s/w? Obviously, software compaction costs in terms of cpu time. The question is, is it worth the time spent? Regards, Richard Schuh
Re: Packing Methods
In our situation, the tapes are being written to a remote SL3000 located in another of our datacenters. They will be read by the same tape units that wrote them because the DR site is an LPAR in that same center. We have tested the process and are well within the limits established for us. The main consideration we have regarding the time is the impact to the users during the backup. We have a global user community and the overnight (for us in the Americas) period is sometimes as busy as is our daytime shift. Regards, Richard Schuh From: The IBM z/VM Operating System [mailto:ib...@listserv.uark.edu] On Behalf Of Alain Benveniste Sent: Thursday, April 30, 2009 4:25 AM To: IBMVM@LISTSERV.UARK.EDU Subject: Re: Packing Methods Richard, You have the same questions I had when I started to put in place our DR solution. We also have 3590E drives and I never tried to remove the hard drive default compaction. I don't see a reason for that. Now choosing software compaction is a must if you have enough cpu to do the work for backup and ALSO for restore. For us we can spend more time to use software compaction because we know that we have enough cpu to do the work at restore time offsite. The gain at restore justifies to take more time at backup processing. It's true too that software compaction takes less tapes than with no compaction. If you have many dasds to backup and a time constraint to restore i would suggest you to both use hard and software compactions. Our idea is to say that when we restore in a DR test the cpu is used ONLY for restore. Why not fully using it ! Regards Alain Benveniste Le 29/04/09 20:46, « Schuh, Richard » rsc...@visa.com a écrit : We are working on a DR process. I notice that the defaults for a Hidro backup include the PACK option which tells Hidro to pack, or condense in some fashion, its output. The output is being written to 3590E drives. It appears that there are three choices we can make for condensing the data: software only, hardware only, or a combination of the two (uncompacted was purposely omitted from the list). Which is likely to give the best results? Does software compaction produce consistently lower output volumes than letting the drive do it? Is there anything to be gained by using both h/w and s/w? Obviously, software compaction costs in terms of cpu time. The question is, is it worth the time spent? Regards, Richard Schuh
Re: Packing Methods
If your main concern is impact on users and you have a speedy link between the sites, then by all means use hardware compression only. If your link is slow or there is other traffic on it, you may consider software compression and then muzzle the backup application using priorities etc. I personally prefer hardware only, wherever possible. Although software+hardware compression produces less tapes, both backups and restores may still take longer regardless of CPU availability (YMMV). Also, if you are prepared to waste CPU cycles just because you can (e.g. nothing else is running), be mindful that job schedules may change... Ivica
