Re: Improving TSM performance - memory related setting
We have experienced this phenomenon as well at 5.3.5.2 on a Solaris TSM
Server.
The system has 32GB memory, 6 CPU.
We used 20GB for bufferpool and verified that there was no paging going on.
Performance was terrible! As a measure, expiration took about a week. All
other processes, sessions, queries were similarly bad.
Reduced buffpool to about 1/2 GB and performance is now great. Expiration
now runs in about 3 hours. Other measures improved dramatically as well.
Current DB stats:
Available Space (MB): 204,800
Assigned Capacity (MB): 202,860
Maximum Extension (MB): 1,940
Maximum Reduction (MB): 75,212
Page Size (bytes): 4,096
Total Usable Pages: 51,932,160
Used Pages: 32,695,965
Pct Util: 63.0
Max. Pct Util: 63.0
Physical Volumes: 16
Buffer Pool Pages: 131,072
Total Buffer Requests: 306,981,255
Cache Hit Pct.: 99.72
Cache Wait Pct.: 0.00
Robert R. Price
TSM Administrator
Computer Sciences Corporation
Phone: 412-374-3247
Fax: 412-374-6371
[EMAIL PROTECTED]
Computer Sciences Corporation
Registered Office: 3170 Fairview Park Drive, Falls Church, Virginia 22042,
USA
Registered in Nevada, USA No: C-489-59
-
This is a PRIVATE message. If you are not the intended recipient, please
delete without copying and kindly advise us by e-mail of the mistake in
delivery.
NOTE: Regardless of content, this e-mail shall not operate to bind CSC to
any order or other contract unless pursuant to explicit written agreement
or government initiative expressly permitting the use of e-mail for such
purpose.
-
Matthew Glanville
To
Sent by: "ADSM: ADSM-L@VM.MARIST.EDU
Dist Stor cc
Manager"
<[EMAIL PROTECTED] Subject
.EDU> Re: Improving TSM performance -
memory related setting
04/07/2008 09:43
AM
Please respond to
"ADSM: Dist Stor
Manager"
<[EMAIL PROTECTED]
.EDU>
Just a warning, don't necessarily go to 1/2 or 1/8th of your total
physical memory..
If your server has 64 GB of memory, 8 GB (1/8th) for BUFPOOLSIZE is
probably too high. I would keep it below 1 GB unless you prove to
yourself with some testing that it is helping speed up the backups or
restores. Just don't look at database cache hit rate.
I think there's some inefficiency in how TSM database cache works, as it
may have to search through that cached memory to find a hit. When you are
caching 8 GB, that takes some time to search through all that.
You'll see this as the dsmserv processes CPU use will increase the larger
the BUFPOOLSIZE is. If it takes longer to search through that memory than
it does to read the page from disk, you are hurting performance not
helping it by caching in memory.
Maybe DB2 TSM in version 6 will help this issue.
I tried to put a problem in for it a few years back, but couldn't get
beyond level 2 since no one understood what I was talking about, they kept
thinking the server was slow due to the operating system using it's own
page file to make up for the large memory setting, even though the 64 bit
server had 32 GB of physical memory... Ahh well. I can use that other 30
GB for something else.
or is it fixed now?
Matt G.
"ADSM: Dist Stor Manager" wrote on 04/04/2008
03:40:56 AM:
> Bufpoolsize should be set between 1/2 and 1/8 of the among of the
> total physical memory, if I remember well, that is, what best
> practices recommend.
> Also if you modify this parameter you will have to take a look at
> the percentage hit cache (q db f=d) that can be increase (modifying
> the bufpolsize), or perhaps decrease if the setting is incorrect.
> Percentage hit cache must be above 99 %.
>
> I will also recommend to take a look at the performance tuning
> guide, that will let you know more about many settings that can
> increase the performance of your TSM server.
>
> Regards,
> Bernaldo.
>
>
>
> - Mensaje original
> De: Paul Dudley <[EMAIL PROTECTED]>
> Para: A
Re: Improving TSM performance - memory related setting
I would caution against speculating about TSM database cache algorithms and BUFPoolsize value effects without supporting evidence. I would not expect its use of database records in memory to be substantially different than its indexing of records on disk, where I doubt that there would be a lot of "searching through memory". Performance degradation with elevated BUFPoolsize levels is traditionally due to paging, as the TSM Performance Tuning Guide explains in its recommendations for values. (There are supplementary Technotes.) Operating system memory utilization is always a complex topic, which also changes as each OS evolves. Having a large amount of memory is wonderful; but one can't assume that the OS is going to apportion its utilization as we might expect. An OS typically computes sizes for various reserved areas at boot time, according to the ingredients defined to the system, and hardware discovered. The amount of memory left for application use may be substantially less than total memory. Environmental values under which a server process is started can also limit how much memory it is allowed to work with. So, be sure to pursue analysis where you perceive performance characteristics you don't expect. Enlist your OS specialist as necessary. Richard Sims at Boston University
Re: Improving TSM performance - memory related setting
What version did you last see the problem? I'm interested as I'm about to put in a new TSM server, and I was contemplating which version to go to. See Ya' Howard > -Original Message- > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf > Of Matthew Glanville > Sent: Monday, April 07, 2008 8:44 AM > To: ADSM-L@VM.MARIST.EDU > Subject: Re: [ADSM-L] Improving TSM performance - memory related > setting > > Just a warning, don't necessarily go to 1/2 or 1/8th of your total > physical memory.. > > If your server has 64 GB of memory, 8 GB (1/8th) for BUFPOOLSIZE is > probably too high. I would keep it below 1 GB unless you prove to > yourself with some testing that it is helping speed up the backups or > restores. Just don't look at database cache hit rate. > > I think there's some inefficiency in how TSM database cache works, as > it > may have to search through that cached memory to find a hit. When you > are > caching 8 GB, that takes some time to search through all that. > > You'll see this as the dsmserv processes CPU use will increase the > larger > the BUFPOOLSIZE is. If it takes longer to search through that memory > than > it does to read the page from disk, you are hurting performance not > helping it by caching in memory. > > Maybe DB2 TSM in version 6 will help this issue. > I tried to put a problem in for it a few years back, but couldn't get > beyond level 2 since no one understood what I was talking about, they > kept > thinking the server was slow due to the operating system using it's own > page file to make up for the large memory setting, even though the 64 > bit > server had 32 GB of physical memory... Ahh well. I can use that other > 30 > GB for something else. > > or is it fixed now? > Matt G. > > > "ADSM: Dist Stor Manager" wrote on 04/04/2008 > 03:40:56 AM: > > > Bufpoolsize should be set between 1/2 and 1/8 of the among of the > > total physical memory, if I remember well, that is, what best > > practices recommend. > > Also if you modify this parameter you will have to take a look at > > the percentage hit cache (q db f=d) that can be increase (modifying > > the bufpolsize), or perhaps decrease if the setting is incorrect. > > Percentage hit cache must be above 99 %. > > > > I will also recommend to take a look at the performance tuning > > guide, that will let you know more about many settings that can > > increase the performance of your TSM server. > > > > Regards, > > Bernaldo. > > > > > > > > - Mensaje original > > De: Paul Dudley <[EMAIL PROTECTED]> > > Para: ADSM-L@VM.MARIST.EDU > > Enviado: viernes, 4 de abril, 2008 5:44:02 > > Asunto: Re: [ADSM-L] Improving TSM performance - memory related > setting > > > > Currently the BUFPOOLSIZE is set to: > > > > BufPoolSize: 122,880 K > > > > The server has 3 Gb of memory. Is this something I can increase? > > > > Regards > > Paul Dudley > > > > Senior IT Systems Administrator > > ANL IT Operations Dept. > > ANL Container Line > > [EMAIL PROTECTED] > > 03-9257-0603 > > http://www.anl.com.au > > > > > > > > > -Original Message- > > > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On > > > Behalf Of Skylar Thompson > > > Sent: Friday, 4 April 2008 1:05 PM > > > To: ADSM-L@VM.MARIST.EDU > > > Subject: Re: [ADSM-L] Improving TSM performance - memory related > > setting > > > > > > Paul Dudley wrote: > > > > If I recall correctly there is a TSM system setting that relates > to > > the > > > > memory available on the server, and you can check and adjust this > > > > setting to improve performance of your TSM server. > > > > > > > > > > > > > > > > Can anyone remind me which setting it is? > > > > > > > > > > > > > > Are you thinking of BUFPOOLSIZE and LOGPOOLSIZE? The former > controls > > > the > > > size of the database buffer pool (bounded only by the physical > memory > > of > > > the machine), and the latter the size of the temporary transaction > > space > > > in the recovery log (up to 8192 pages IIRC). > > > > > > -- > > > -- Skylar Thompson ([EMAIL PROTECTED]) > > > -- Genome Sciences Department, System Administrator > > > -- Foege Building S048, (206)-685-7354 > > > -- University of Washington School of Medicine
Re: Improving TSM performance - memory related setting
Just a warning, don't necessarily go to 1/2 or 1/8th of your total physical memory.. If your server has 64 GB of memory, 8 GB (1/8th) for BUFPOOLSIZE is probably too high. I would keep it below 1 GB unless you prove to yourself with some testing that it is helping speed up the backups or restores. Just don't look at database cache hit rate. I think there's some inefficiency in how TSM database cache works, as it may have to search through that cached memory to find a hit. When you are caching 8 GB, that takes some time to search through all that. You'll see this as the dsmserv processes CPU use will increase the larger the BUFPOOLSIZE is. If it takes longer to search through that memory than it does to read the page from disk, you are hurting performance not helping it by caching in memory. Maybe DB2 TSM in version 6 will help this issue. I tried to put a problem in for it a few years back, but couldn't get beyond level 2 since no one understood what I was talking about, they kept thinking the server was slow due to the operating system using it's own page file to make up for the large memory setting, even though the 64 bit server had 32 GB of physical memory... Ahh well. I can use that other 30 GB for something else. or is it fixed now? Matt G. "ADSM: Dist Stor Manager" wrote on 04/04/2008 03:40:56 AM: > Bufpoolsize should be set between 1/2 and 1/8 of the among of the > total physical memory, if I remember well, that is, what best > practices recommend. > Also if you modify this parameter you will have to take a look at > the percentage hit cache (q db f=d) that can be increase (modifying > the bufpolsize), or perhaps decrease if the setting is incorrect. > Percentage hit cache must be above 99 %. > > I will also recommend to take a look at the performance tuning > guide, that will let you know more about many settings that can > increase the performance of your TSM server. > > Regards, > Bernaldo. > > > > - Mensaje original > De: Paul Dudley <[EMAIL PROTECTED]> > Para: ADSM-L@VM.MARIST.EDU > Enviado: viernes, 4 de abril, 2008 5:44:02 > Asunto: Re: [ADSM-L] Improving TSM performance - memory related setting > > Currently the BUFPOOLSIZE is set to: > > BufPoolSize: 122,880 K > > The server has 3 Gb of memory. Is this something I can increase? > > Regards > Paul Dudley > > Senior IT Systems Administrator > ANL IT Operations Dept. > ANL Container Line > [EMAIL PROTECTED] > 03-9257-0603 > http://www.anl.com.au > > > > > -Original Message- > > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On > > Behalf Of Skylar Thompson > > Sent: Friday, 4 April 2008 1:05 PM > > To: ADSM-L@VM.MARIST.EDU > > Subject: Re: [ADSM-L] Improving TSM performance - memory related > setting > > > > Paul Dudley wrote: > > > If I recall correctly there is a TSM system setting that relates to > the > > > memory available on the server, and you can check and adjust this > > > setting to improve performance of your TSM server. > > > > > > > > > > > > Can anyone remind me which setting it is? > > > > > > > > > > Are you thinking of BUFPOOLSIZE and LOGPOOLSIZE? The former controls > > the > > size of the database buffer pool (bounded only by the physical memory > of > > the machine), and the latter the size of the temporary transaction > space > > in the recovery log (up to 8192 pages IIRC). > > > > -- > > -- Skylar Thompson ([EMAIL PROTECTED]) > > -- Genome Sciences Department, System Administrator > > -- Foege Building S048, (206)-685-7354 > > -- University of Washington School of Medicine
Re: Improving TSM performance - memory related setting
Check your "SELFTUNEBUFpoolsize" setting. If its set to no, change it to yes. Then the BUFPOOLSIZE just becomes an "initial" size setting that the TSM server can adjust upward as it needs to. Our BUFPOOLSIZE is set to 262144 right now which I think for 5.2.9 is close to the max, if not so. I'll have to go back and dig that up again. See Ya' Howard > -Original Message- > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf > Of Paul Dudley > Sent: Thursday, April 03, 2008 10:44 PM > To: ADSM-L@VM.MARIST.EDU > Subject: Re: [ADSM-L] Improving TSM performance - memory related > setting > > Currently the BUFPOOLSIZE is set to: > > BufPoolSize: 122,880 K > > The server has 3 Gb of memory. Is this something I can increase? > > Regards > Paul Dudley > > Senior IT Systems Administrator > ANL IT Operations Dept. > ANL Container Line > [EMAIL PROTECTED] > 03-9257-0603 > http://www.anl.com.au > > > > > -Original Message- > > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On > > Behalf Of Skylar Thompson > > Sent: Friday, 4 April 2008 1:05 PM > > To: ADSM-L@VM.MARIST.EDU > > Subject: Re: [ADSM-L] Improving TSM performance - memory related > setting > > > > Paul Dudley wrote: > > > If I recall correctly there is a TSM system setting that relates to > the > > > memory available on the server, and you can check and adjust this > > > setting to improve performance of your TSM server. > > > > > > > > > > > > Can anyone remind me which setting it is? > > > > > > > > > > Are you thinking of BUFPOOLSIZE and LOGPOOLSIZE? The former controls > > the > > size of the database buffer pool (bounded only by the physical memory > of > > the machine), and the latter the size of the temporary transaction > space > > in the recovery log (up to 8192 pages IIRC). > > > > -- > > -- Skylar Thompson ([EMAIL PROTECTED]) > > -- Genome Sciences Department, System Administrator > > -- Foege Building S048, (206)-685-7354 > > -- University of Washington School of Medicine > > > > > > ANL DISCLAIMER > This e-mail and any file attached is confidential, and intended solely > to the named addressees. Any unauthorised dissemination or use is > strictly prohibited. If you received this e-mail in error, please > immediately notify the sender by return e-mail from your system. Please > do not copy, use or make reference to it for any purpose, or disclose > its contents to any person.
Re: Improving TSM performance - memory related setting
Bufpoolsize should be set between 1/2 and 1/8 of the among of the total physical memory, if I remember well, that is, what best practices recommend. Also if you modify this parameter you will have to take a look at the percentage hit cache (q db f=d) that can be increase (modifying the bufpolsize), or perhaps decrease if the setting is incorrect. Percentage hit cache must be above 99 %. I will also recommend to take a look at the performance tuning guide, that will let you know more about many settings that can increase the performance of your TSM server. Regards, Bernaldo. - Mensaje original De: Paul Dudley <[EMAIL PROTECTED]> Para: ADSM-L@VM.MARIST.EDU Enviado: viernes, 4 de abril, 2008 5:44:02 Asunto: Re: [ADSM-L] Improving TSM performance - memory related setting Currently the BUFPOOLSIZE is set to: BufPoolSize: 122,880 K The server has 3 Gb of memory. Is this something I can increase? Regards Paul Dudley Senior IT Systems Administrator ANL IT Operations Dept. ANL Container Line [EMAIL PROTECTED] 03-9257-0603 http://www.anl.com.au > -Original Message- > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On > Behalf Of Skylar Thompson > Sent: Friday, 4 April 2008 1:05 PM > To: ADSM-L@VM.MARIST.EDU > Subject: Re: [ADSM-L] Improving TSM performance - memory related setting > > Paul Dudley wrote: > > If I recall correctly there is a TSM system setting that relates to the > > memory available on the server, and you can check and adjust this > > setting to improve performance of your TSM server. > > > > > > > > Can anyone remind me which setting it is? > > > > > > Are you thinking of BUFPOOLSIZE and LOGPOOLSIZE? The former controls > the > size of the database buffer pool (bounded only by the physical memory of > the machine), and the latter the size of the temporary transaction space > in the recovery log (up to 8192 pages IIRC). > > -- > -- Skylar Thompson ([EMAIL PROTECTED]) > -- Genome Sciences Department, System Administrator > -- Foege Building S048, (206)-685-7354 > -- University of Washington School of Medicine ANL DISCLAIMER This e-mail and any file attached is confidential, and intended solely to the named addressees. Any unauthorised dissemination or use is strictly prohibited. If you received this e-mail in error, please immediately notify the sender by return e-mail from your system. Please do not copy, use or make reference to it for any purpose, or disclose its contents to any person. __ ¿Con Mascota por primera vez? Sé un mejor Amigo. Entra en Yahoo! Respuestas http://es.answers.yahoo.com/info/welcome
Re: Improving TSM performance - memory related setting
Currently the BUFPOOLSIZE is set to: BufPoolSize: 122,880 K The server has 3 Gb of memory. Is this something I can increase? Regards Paul Dudley Senior IT Systems Administrator ANL IT Operations Dept. ANL Container Line [EMAIL PROTECTED] 03-9257-0603 http://www.anl.com.au > -Original Message- > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On > Behalf Of Skylar Thompson > Sent: Friday, 4 April 2008 1:05 PM > To: ADSM-L@VM.MARIST.EDU > Subject: Re: [ADSM-L] Improving TSM performance - memory related setting > > Paul Dudley wrote: > > If I recall correctly there is a TSM system setting that relates to the > > memory available on the server, and you can check and adjust this > > setting to improve performance of your TSM server. > > > > > > > > Can anyone remind me which setting it is? > > > > > > Are you thinking of BUFPOOLSIZE and LOGPOOLSIZE? The former controls > the > size of the database buffer pool (bounded only by the physical memory of > the machine), and the latter the size of the temporary transaction space > in the recovery log (up to 8192 pages IIRC). > > -- > -- Skylar Thompson ([EMAIL PROTECTED]) > -- Genome Sciences Department, System Administrator > -- Foege Building S048, (206)-685-7354 > -- University of Washington School of Medicine ANL DISCLAIMER This e-mail and any file attached is confidential, and intended solely to the named addressees. Any unauthorised dissemination or use is strictly prohibited. If you received this e-mail in error, please immediately notify the sender by return e-mail from your system. Please do not copy, use or make reference to it for any purpose, or disclose its contents to any person.
Re: Improving TSM performance - memory related setting
Paul Dudley wrote: If I recall correctly there is a TSM system setting that relates to the memory available on the server, and you can check and adjust this setting to improve performance of your TSM server. Can anyone remind me which setting it is? Are you thinking of BUFPOOLSIZE and LOGPOOLSIZE? The former controls the size of the database buffer pool (bounded only by the physical memory of the machine), and the latter the size of the temporary transaction space in the recovery log (up to 8192 pages IIRC). -- -- Skylar Thompson ([EMAIL PROTECTED]) -- Genome Sciences Department, System Administrator -- Foege Building S048, (206)-685-7354 -- University of Washington School of Medicine
Improving TSM performance - memory related setting
If I recall correctly there is a TSM system setting that relates to the memory available on the server, and you can check and adjust this setting to improve performance of your TSM server. Can anyone remind me which setting it is? Regards Paul Dudley Senior IT Systems Administrator ANL IT Operations Dept. ANL Container Line [EMAIL PROTECTED] ANL DISCLAIMER This e-mail and any file attached is confidential, and intended solely to the named addressees. Any unauthorised dissemination or use is strictly prohibited. If you received this e-mail in error, please immediately notify the sender by return e-mail from your system. Please do not copy, use or make reference to it for any purpose, or disclose its contents to any person.
