AW: Client Compression question
Hi Kevin, at the moment i have no speed problem. But sometimes i do not have enough tapeunits.(i share the units with mvs). If the primary diskpool is too small and the available drives are all used by tsm (migration, db backup,dedicated backups to tape of very big files.) in the past i often went into trouble. Migration from disk to tape disk needs about 3-4 drives to handle incoming data. Therefore i decided to work with client compression (more data fits on disk pools). I noticed, that mainly there is no degraded speed (sometimes i get 8-10MB/sec during backup for that NAS Node). It seems that the search for files limits at the moment the backup-speed. BTW i am happy with 8-10MB/sec Can you please more explain your hint regarding device class? I do not find the format parameter, i only get Device Class Name: 3590RZ Device Access Strategy: Sequential Storage Pool Count: 2 Last Update by (administrator): PPOHS Last Update Date/Time: 13.12.2001 18:09:14 Device Type: 3590 Maximum Capacity (MB): Estimated Capacity (MB): 15.360,0 Dataset Name Prefix: ADSM Mount Limit: 4 Mount Retention (min): 1 Label Type: IBMSL Expiration Date: Mount Wait (min): 120 Unit Name: MAGRZ Volser: Compression: Yes Protection: Automatic Retention: Server Name: Retry Period: Retry Interval: do you mean Device Type? With regards, Stefan Holzwarth > -Ursprüngliche Nachricht- > Von: Kevin [mailto:[EMAIL PROTECTED]] > Gesendet: Mittwoch, 9. Januar 2002 21:12 > An: [EMAIL PROTECTED] > Betreff: Re: Client Compression question > > > Hi Stefan, > > Using compression will really slow down the backup - mostly in backup > mode, but in restore as well. If you set the format parameter in the > device class to drive (instead of a fixed number) you will > get both the > benefit of a dynamic device class with regard to number of drives (for > additional drive installations and drive failure(s)) and you > will enable > automatic hardware compression. This compression is better than the > software based compression, however when the system is queried, the > number returned will reflect raw data sizes, not compressed > data sizes. > Client side (software based) compression or server side > (software based > compression) will work in conjunction with library based hardware > compression however you will not receive any compression benefit by > running both and the numbers returned upon query will not reflect the > actual size of data on the tape. In some cases an already compressed > file has taken 15 minutes while software based compression tried to > re-compress it. The impact is not small. > > Remember, this is based on the device class specification of > format=drive, and may not be applicable to all drives, but only those > with compression capabilities. We currently use LTO based native fiber > drives. > > Choosing one method of compression over the other will improve > performance and specifying hardware compression while forcing off > software compression should really improve performance. > > Another neat performance tuning trick (as I'm sure you > already know) is > to backup to a hard drive initially, and then force migration (set the > pool to migrate hi=0 lo=0 in a script, and then set it back to normal > again after an hour or so) to de-stage the data off the hard disk and > onto tape. This allows multiple sessions (sessions are not limited to > number of available tape drives) and hard disk is generally > faster as a > storage pool. > > Let me know if this helps, > > Kevin. > > -Original Message- > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]] > On Behalf Of > Stefan Holzwarth > Sent: Wednesday, January 09, 2002 6:56 AM > To: [EMAIL PROTECTED] > Subject: AW: Client Compression question > > We are using client compression for backup of a 1,5 TByte NAS > system.(F760) > (reason is to have apparently larger disk storage pools) > Backup is done using 5 different nodes on the same machine (Dual 1 GHz > P3) > to have the opportiunity to restore with 5 sessions. > During backup i can see about 3-4 MByte per second on the > network to the > TSM-server on MVS. > Restoretests for the netapp system seem to have a limit with this > compressed > data at 8-9 MB per second. (only if tapes 3590-B a filled well) > At this speed (can be observed over a longer period) TSM > client and NAS > system have nearly 100% CPU load. > The data a typica
Re: AW: Client Compression question
Stefan, Are those backup/restore speeds you have quoted the total for all 5 nodes or are they what you see for each individual session. Is your network 100 megabits or gigabit. regards John Stefan Holzwarth <[EMAIL PROTECTED]> on 01/09/2002 11:56:23 AM Please respond to "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]> To: [EMAIL PROTECTED] cc:(bcc: John Naylor/HAV/SSE) Subject: AW: Client Compression question ** The information in this E-Mail is confidential and may be legally privileged. It may not represent the views of Scottish and Southern Energy plc. It is intended solely for the addressees. Access to this E-Mail by anyone else is unauthorised. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. Any unauthorised recipient should advise the sender immediately of the error in transmission. Scottish Hydro-Electric, Southern Electric, SWALEC and S+S are trading names of the Scottish and Southern Energy Group. ** We are using client compression for backup of a 1,5 TByte NAS system.(F760) (reason is to have apparently larger disk storage pools) Backup is done using 5 different nodes on the same machine (Dual 1 GHz P3) to have the opportiunity to restore with 5 sessions. During backup i can see about 3-4 MByte per second on the network to the TSM-server on MVS. Restoretests for the netapp system seem to have a limit with this compressed data at 8-9 MB per second. (only if tapes 3590-B a filled well) At this speed (can be observed over a longer period) TSM client and NAS system have nearly 100% CPU load. The data a typical userdata - many small and some large Regards, Stefan Holzwarth > -Ursprüngliche Nachricht- > Von: Sotonyi Attila [mailto:[EMAIL PROTECTED]] > Gesendet: Mittwoch, 9. Januar 2002 12:29 > An: [EMAIL PROTECTED] > Betreff: Re: Client Compression question > > > HI, > > With many small files this performance is not quite bad, but I think > you should tuning you system. What kind of library you're using? Is > compression set to enabled at drive and client level? If yes do not > use this. > > Üdv, > Sótonyi Attila mailto:[EMAIL PROTECTED] Certified TSM 4.1 admin MÁV Informatika Kft. Tel.:06(1)457-9372 > Hi. > I have been testing Novell backups and restores using a Compaq proliant 8500 as > the client > box. > We have been seeing ok performance of approx 10 gb an hour for backups and > restores, > of 1 million objects. > This applies both to Netware compressed and Netware uncompressed volumes. > However when I try TSM client compression on a backup of a netware uncompressed > volume, the performance drops to 1 gb. an hour. > It is not a resource problem the compaq cpu is hardly stirring, but it is > definitely the TSM compression that is taking up the time (TSM tracing shows it > as 89% of the total time) > I always believed in the past that slow performance with client compression was > down to > lack of cpu on the client box, but there is cpu to spare with this > configuration. > My question is :- > Is 1 gb an hour the best performance I that I can expect using TSM compression > or can anyone think of reasons why that is all I am achieving ? > thanks > ** > The information in this E-Mail is confidential and may be legally > privileged. It may not represent the views of Scottish and Southern > Energy plc. > It is intended solely for the addressees. Access to this E-Mail by > anyone else is unauthorised. If you are not the intended recipient, > any disclosure, copying, distribution or any action taken or omitted > to be taken in reliance on it, is prohibited and may be unlawful. > Any unauthorised recipient should advise the sender immediately of > the error in transmission. > Scottish Hydro-Electric, Southern Electric, SWALEC and S+S > are trading names of the Scottish and Southern Energy Group. > **
AW: Client Compression question
We are using client compression for backup of a 1,5 TByte NAS system.(F760) (reason is to have apparently larger disk storage pools) Backup is done using 5 different nodes on the same machine (Dual 1 GHz P3) to have the opportiunity to restore with 5 sessions. During backup i can see about 3-4 MByte per second on the network to the TSM-server on MVS. Restoretests for the netapp system seem to have a limit with this compressed data at 8-9 MB per second. (only if tapes 3590-B a filled well) At this speed (can be observed over a longer period) TSM client and NAS system have nearly 100% CPU load. The data a typical userdata - many small and some large Regards, Stefan Holzwarth > -Ursprüngliche Nachricht- > Von: Sotonyi Attila [mailto:[EMAIL PROTECTED]] > Gesendet: Mittwoch, 9. Januar 2002 12:29 > An: [EMAIL PROTECTED] > Betreff: Re: Client Compression question > > > HI, > > With many small files this performance is not quite bad, but I think > you should tuning you system. What kind of library you're using? Is > compression set to enabled at drive and client level? If yes do not > use this. > > Üdv, > Sótonyi Attila mailto:[EMAIL PROTECTED] Certified TSM 4.1 admin MÁV Informatika Kft. Tel.:06(1)457-9372 > Hi. > I have been testing Novell backups and restores using a Compaq proliant 8500 as > the client > box. > We have been seeing ok performance of approx 10 gb an hour for backups and > restores, > of 1 million objects. > This applies both to Netware compressed and Netware uncompressed volumes. > However when I try TSM client compression on a backup of a netware uncompressed > volume, the performance drops to 1 gb. an hour. > It is not a resource problem the compaq cpu is hardly stirring, but it is > definitely the TSM compression that is taking up the time (TSM tracing shows it > as 89% of the total time) > I always believed in the past that slow performance with client compression was > down to > lack of cpu on the client box, but there is cpu to spare with this > configuration. > My question is :- > Is 1 gb an hour the best performance I that I can expect using TSM compression > or can anyone think of reasons why that is all I am achieving ? > thanks > ** > The information in this E-Mail is confidential and may be legally > privileged. It may not represent the views of Scottish and Southern > Energy plc. > It is intended solely for the addressees. Access to this E-Mail by > anyone else is unauthorised. If you are not the intended recipient, > any disclosure, copying, distribution or any action taken or omitted > to be taken in reliance on it, is prohibited and may be unlawful. > Any unauthorised recipient should advise the sender immediately of > the error in transmission. > Scottish Hydro-Electric, Southern Electric, SWALEC and S+S > are trading names of the Scottish and Southern Energy Group. > **
