TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Fast]

TSM Log Pinning problem ( 85% Pct Util) is definitely related to TSM Clients 
with slow Network data transfer rates.  Look for ANR0483W near the ANR2997W and 
ANR2998W and you may find the TSM Client causing the log is pinning.  I agree 
TSM 5.5, database and log all play roles in this issue.  100BaseT Duplex 
mismatch between ethernet adapter and switch caused slow network data transrfer 
rate.  I have been rebuilding log files and database files using mirrors every 
three - six months until I switch TSM Clients over to a new TSM 6.2 Server.
I have opened several PMR's on database, log utilization and clients.   I 
recommend investigating Ethernet adapter and switch on TSM Clients with slow 
network data transfer rates.

+--
|This was sent by lf...@ohio.gov via Backup Central.
|Forward SPAM to ab...@backupcentral.com.
+--

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Druckenmiller, David]

Did this issue ever get resolved?  I've noticed the same thing for my 5.5.0 
server and have been trying to isolate the issue, but to no avail.

Thanks

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Dave 
Canan
Sent: Friday, May 13, 2011 11:52 AM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Andy Raibeck and I have been monitoring this thread for the past few weeks
and have been having numerous discussions about what we need to do to
address these problems. In addition, we have noted the many PMRs coming into
the IBM support queue, and many of these PMRs are now on my queue. Andy will
be assisting me along with other support people as needed.

As part of this analysis we will be doing, we have a new questionnaire
(titled Items to Gather for a PMR with Log Pinning Condition for TSM V5)
that customers will be filling out that ask several questions regarding your
environment. We understand that this involves extra work to gather this
information, but there can be many different areas that can cause log
pinning conditions, so this information is needed. In addition, there will
be a script provided (named serverperf5a.pl) that will help us gather
additional data. Both of these will be provided to you by support. When
these PMRs are now opened, please make sure that level 1 support adds the
keyword LOGPIN55 to the PMR. This will allow Andy and I to quickly find all
the PMRs being worked for this issue.

Eric, your PMR is now one that I have on my queue (or I will shortly today).
We will be contacting you to work the PMR.

Dave Canan
IBM ATS TSM Performance
ddcananATUSDOTIBMDOTCOM
916-723-2410

On Thu, May 12, 2011 at 7:53 AM, Loon, EJ van - SPLXO eric-van.l...@klm.com
 wrote:

 Hi Rick!
 You are running in normal mode. In this mode the recovery log only
 contains uncommited transactions. Don't you agree that in this case, the
 log should NEVER reach 90%? It should not even reach something like 40%!
 I don't want to have to implement extra monitoring to keep TSM from
 crashing. It should just work. Like it did when we were running 5.3...
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines





 
 Van: ADSM: Dist Stor Manager [ADSM-L@VM.MARIST.EDU] namens Loon, EJ van
 -
 SPLXO [eric-van.l...@klm.com]
 Verzonden: donderdag 12 mei 2011 11:11
 Aan: ADSM-L@VM.MARIST.EDU
 Onderwerp: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 Hi Paul!
 We are already running 5.5.5.2 and the log is still filling up, even
 after switching from rollforward to normal mode.
 Management currently is questioning whether TSM is the right product for
 the future. Although I'm a big fan of TSM for 15 years, I'm really in
 doubt too...
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines


 On Wed, May 11, 2011 at 9:03 AM, Loon, EJ van - SPLXO
 eric-van.l...@klm.com
  wrote:

  Hi Robert!
  Thanks you very much for your reply! Several others on this list
  reported this behavior and (as far as I know) three other users opened
 a
  PMR too. I hope they have more luck, because I'm stuck. Level 2 keeps
 on
  saying that the log keeps on growing because of slow running client
  sessions. Indeed I see slow running client sessions, but they are
 slowed
  down by the fact that TSM is delaying all transactions because the log
  is used for more that 80% during a large part of the backup window!
 Now
 
 
  From:   Loon, EJ van - SPLXO eric-van.l...@klm.com
  To: ADSM-L@VM.MARIST.EDU
  Date:   05/11/2011 02:05 AM
  Subject:Re: [ADSM-L] TSM Recovery log is pinning since upgrade
  to
  5.5.5.0 code
  Sent by:ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU
 
 
 
  Hi TSM-ers!
  Here is a small follow up on my PMR about the recovery log
 utilization.
  I'm at TSM level 2, still trying to convince them that there is
  something broken in the TSM server code. To convince them, I have
  changed the logmode to normal on one of my servers. I created a graph
  (through TSMManager) which shows the recovery log utilization during
  last night's client backup window and is doesn't differ much from the
  night before, with logmode rollforward. When running in normal mode,
 TSM
  should only use the recovery log for uncommitted transactions, so
  utilization should be very low. My log is 12 Gb and the backuptrigger
  value (75%) was still hit twice!
  This clearly shows that there is something wrong with TSM, let's hope
 I
  can convince Level 2 too, so my case gets forwarded to the lab.
  I'll keep you guys posted!
  Kind regards,
  Eric van Loon
  KLM Royal Dutch Airlines
  
  For information, services and offers, please visit our web site:
  http://www.klm.com. This e-mail and any attachment may contain
  confidential and privileged material intended for the addressee only.
 If
  you are not the addressee, you

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Dave Canan]

We are working several PMRs now for several customers seeing this issue. We are 
recommending certain levels of the server code and client code for customers 
seeing this issue. On the server side, we are recommending 5.5.5.0 (which you 
or at), or if your environment does a lot of customized reporting with selects, 
then we recommend 5.5.5.2. For the client side, the question is whether you 
have windows 7 or windows 2008 in your environment. If so, our first choice for 
client levels for those machines is 6.1.4.2. 

I am out this week on vacation, but if Andy is monitoring the thread maybe he 
could add some additional input here. In each of these levels we have 
identified potential APARs that could create possible log pinning issues. We 
are still pursuing other issues as well. If you are seeing log pinning 
problems, we encourage you to open a PMR with IBM software support.

Dave Canan
Advanced Technical Support
TSM Performance
ddca...@us.ibm.com

Sent from my iPad

On Jun 1, 2011, at 7:01 AM, Druckenmiller, David druc...@mail.amc.edu wrote:

 Did this issue ever get resolved?  I've noticed the same thing for my 5.5.0 
 server and have been trying to isolate the issue, but to no avail.
 
 Thanks
 
 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Dave 
 Canan
 Sent: Friday, May 13, 2011 11:52 AM
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 
 code
 
 Andy Raibeck and I have been monitoring this thread for the past few weeks
 and have been having numerous discussions about what we need to do to
 address these problems. In addition, we have noted the many PMRs coming into
 the IBM support queue, and many of these PMRs are now on my queue. Andy will
 be assisting me along with other support people as needed.
 
 As part of this analysis we will be doing, we have a new questionnaire
 (titled Items to Gather for a PMR with Log Pinning Condition for TSM V5)
 that customers will be filling out that ask several questions regarding your
 environment. We understand that this involves extra work to gather this
 information, but there can be many different areas that can cause log
 pinning conditions, so this information is needed. In addition, there will
 be a script provided (named serverperf5a.pl) that will help us gather
 additional data. Both of these will be provided to you by support. When
 these PMRs are now opened, please make sure that level 1 support adds the
 keyword LOGPIN55 to the PMR. This will allow Andy and I to quickly find all
 the PMRs being worked for this issue.
 
 Eric, your PMR is now one that I have on my queue (or I will shortly today).
 We will be contacting you to work the PMR.
 
 Dave Canan
 IBM ATS TSM Performance
 ddcananATUSDOTIBMDOTCOM
 916-723-2410
 
 On Thu, May 12, 2011 at 7:53 AM, Loon, EJ van - SPLXO eric-van.l...@klm.com
 wrote:
 
 Hi Rick!
 You are running in normal mode. In this mode the recovery log only
 contains uncommited transactions. Don't you agree that in this case, the
 log should NEVER reach 90%? It should not even reach something like 40%!
 I don't want to have to implement extra monitoring to keep TSM from
 crashing. It should just work. Like it did when we were running 5.3...
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines
 
 
 
 
 
 
 Van: ADSM: Dist Stor Manager [ADSM-L@VM.MARIST.EDU] namens Loon, EJ van
 -
 SPLXO [eric-van.l...@klm.com]
 Verzonden: donderdag 12 mei 2011 11:11
 Aan: ADSM-L@VM.MARIST.EDU
 Onderwerp: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code
 
 Hi Paul!
 We are already running 5.5.5.2 and the log is still filling up, even
 after switching from rollforward to normal mode.
 Management currently is questioning whether TSM is the right product for
 the future. Although I'm a big fan of TSM for 15 years, I'm really in
 doubt too...
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines
 
 
 On Wed, May 11, 2011 at 9:03 AM, Loon, EJ van - SPLXO
 eric-van.l...@klm.com
 wrote:
 
 Hi Robert!
 Thanks you very much for your reply! Several others on this list
 reported this behavior and (as far as I know) three other users opened
 a
 PMR too. I hope they have more luck, because I'm stuck. Level 2 keeps
 on
 saying that the log keeps on growing because of slow running client
 sessions. Indeed I see slow running client sessions, but they are
 slowed
 down by the fact that TSM is delaying all transactions because the log
 is used for more that 80% during a large part of the backup window!
 Now
 
 
 From:   Loon, EJ van - SPLXO eric-van.l...@klm.com
 To: ADSM-L@VM.MARIST.EDU
 Date:   05/11/2011 02:05 AM
 Subject:Re: [ADSM-L] TSM Recovery log is pinning since upgrade
 to
 5.5.5.0 code
 Sent by:ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU
 
 
 
 Hi TSM-ers!
 Here is a small follow up on my PMR about the recovery log
 utilization.
 I'm at TSM level 2, still trying

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Prather, Wanda]

Thanks Dave; 
If the Windows 2008 systems are already running 6.2.2, do we need to drop back 
to 6.1.4.2?

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Dave 
Canan
Sent: Wednesday, June 01, 2011 12:25 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code

We are working several PMRs now for several customers seeing this issue. We are 
recommending certain levels of the server code and client code for customers 
seeing this issue. On the server side, we are recommending 5.5.5.0 (which you 
or at), or if your environment does a lot of customized reporting with selects, 
then we recommend 5.5.5.2. For the client side, the question is whether you 
have windows 7 or windows 2008 in your environment. If so, our first choice for 
client levels for those machines is 6.1.4.2. 

I am out this week on vacation, but if Andy is monitoring the thread maybe he 
could add some additional input here. In each of these levels we have 
identified potential APARs that could create possible log pinning issues. We 
are still pursuing other issues as well. If you are seeing log pinning 
problems, we encourage you to open a PMR with IBM software support.

Dave Canan
Advanced Technical Support
TSM Performance
ddca...@us.ibm.com

Sent from my iPad

On Jun 1, 2011, at 7:01 AM, Druckenmiller, David druc...@mail.amc.edu wrote:

 Did this issue ever get resolved?  I've noticed the same thing for my 5.5.0 
 server and have been trying to isolate the issue, but to no avail.
 
 Thanks
 
 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf 
 Of Dave Canan
 Sent: Friday, May 13, 2011 11:52 AM
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 
 5.5.5.0 code
 
 Andy Raibeck and I have been monitoring this thread for the past few 
 weeks and have been having numerous discussions about what we need to 
 do to address these problems. In addition, we have noted the many PMRs 
 coming into the IBM support queue, and many of these PMRs are now on 
 my queue. Andy will be assisting me along with other support people as needed.
 
 As part of this analysis we will be doing, we have a new questionnaire 
 (titled Items to Gather for a PMR with Log Pinning Condition for TSM 
 V5) that customers will be filling out that ask several questions 
 regarding your environment. We understand that this involves extra 
 work to gather this information, but there can be many different areas 
 that can cause log pinning conditions, so this information is needed. 
 In addition, there will be a script provided (named serverperf5a.pl) 
 that will help us gather additional data. Both of these will be 
 provided to you by support. When these PMRs are now opened, please 
 make sure that level 1 support adds the keyword LOGPIN55 to the PMR. 
 This will allow Andy and I to quickly find all the PMRs being worked for this 
 issue.
 
 Eric, your PMR is now one that I have on my queue (or I will shortly today).
 We will be contacting you to work the PMR.
 
 Dave Canan
 IBM ATS TSM Performance
 ddcananATUSDOTIBMDOTCOM
 916-723-2410
 
 On Thu, May 12, 2011 at 7:53 AM, Loon, EJ van - SPLXO 
 eric-van.l...@klm.com
 wrote:
 
 Hi Rick!
 You are running in normal mode. In this mode the recovery log only 
 contains uncommited transactions. Don't you agree that in this case, 
 the log should NEVER reach 90%? It should not even reach something like 40%!
 I don't want to have to implement extra monitoring to keep TSM from 
 crashing. It should just work. Like it did when we were running 5.3...
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines
 
 
 
 
 
 
 Van: ADSM: Dist Stor Manager [ADSM-L@VM.MARIST.EDU] namens Loon, EJ 
 van
 -
 SPLXO [eric-van.l...@klm.com]
 Verzonden: donderdag 12 mei 2011 11:11
 Aan: ADSM-L@VM.MARIST.EDU
 Onderwerp: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 
 code
 
 Hi Paul!
 We are already running 5.5.5.2 and the log is still filling up, even 
 after switching from rollforward to normal mode.
 Management currently is questioning whether TSM is the right product 
 for the future. Although I'm a big fan of TSM for 15 years, I'm 
 really in doubt too...
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines
 
 
 On Wed, May 11, 2011 at 9:03 AM, Loon, EJ van - SPLXO 
 eric-van.l...@klm.com
 wrote:
 
 Hi Robert!
 Thanks you very much for your reply! Several others on this list 
 reported this behavior and (as far as I know) three other users 
 opened
 a
 PMR too. I hope they have more luck, because I'm stuck. Level 2 
 keeps
 on
 saying that the log keeps on growing because of slow running client 
 sessions. Indeed I see slow running client sessions, but they are
 slowed
 down by the fact that TSM is delaying all transactions because the 
 log is used for more that 80% during a large part of the backup window

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Cheung, Richard]

Hi. I had a server like that - log filled up extremely quickly, multiple log 
pins etc.  

I ended up actually doing a full DB DUMP, DB RELOAD, as it turned out to be 
something corrupted within my database + logs. It has been generally running 
fine ever since.  

I do agree, this is something related to workload for sure, but also do look at 
the health of your database and logs - this may be pointing to possible issues 
within them. 



-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Loon, 
EJ van - SPLXO
Sent: Monday, 16 May 2011 11:49 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Hi Richard!
 So is 5.5.5.0 also having this issue with the log pinning?
Yes. As soon as the log hist 80% TSM starts delaying transactions. In
this level a bug was fixed which caused the log to fill too soon after
it reached 80%. I'm running 5.5.5.2 and it's still filling to 80%
multiple times a night.
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Cheung, Richard
Sent: zondag 15 mei 2011 04:02
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Hi

The worry about this is I had a PMR opened for this issue on my servers,
which are currently at 5.5.4. I was advised it's a known issue, refer to
APAR IC64438, and the recommended fix is to go to 5.5.5.0

So is 5.5.5.0 also having this issue with the log pinning?

I also believed this was happening in early versions of TSM v6.. But the
later builds are okay - can anyone confirm?


Richard Cheung | Server Engineer | Information Systems | Santos Limited
Santos Centre | 60 Flinders Street Adelaide SA 5000 Australia
Ph +61 8 8116 7125 | 7 Fax +61 8 8116 5073 |   richard.che...@santos.com


-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Dave Canan
Sent: Saturday, 14 May 2011 1:22 AM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to
5.5.5.0 code

Andy Raibeck and I have been monitoring this thread for the past few
weeks and have been having numerous discussions about what we need to do
to address these problems. In addition, we have noted the many PMRs
coming into the IBM support queue, and many of these PMRs are now on my
queue. Andy will be assisting me along with other support people as
needed.

As part of this analysis we will be doing, we have a new questionnaire
(titled Items to Gather for a PMR with Log Pinning Condition for TSM
V5) that customers will be filling out that ask several questions
regarding your environment. We understand that this involves extra work
to gather this information, but there can be many different areas that
can cause log pinning conditions, so this information is needed. In
addition, there will be a script provided (named serverperf5a.pl) that
will help us gather additional data. Both of these will be provided to
you by support. When these PMRs are now opened, please make sure that
level 1 support adds the keyword LOGPIN55 to the PMR. This will allow
Andy and I to quickly find all the PMRs being worked for this issue.

Eric, your PMR is now one that I have on my queue (or I will shortly
today).
We will be contacting you to work the PMR.

Dave Canan
IBM ATS TSM Performance
ddcananATUSDOTIBMDOTCOM
916-723-2410

On Thu, May 12, 2011 at 7:53 AM, Loon, EJ van - SPLXO
eric-van.l...@klm.com
 wrote:

 Hi Rick!
 You are running in normal mode. In this mode the recovery log only
 contains uncommited transactions. Don't you agree that in this case,
 the log should NEVER reach 90%? It should not even reach something
like 40%!
 I don't want to have to implement extra monitoring to keep TSM from
 crashing. It should just work. Like it did when we were running 5.3...
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf
 Of Richard Rhodes
 Sent: donderdag 12 mei 2011 15:04
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 I have a cron job that checks log utilization every 15m.  If the log
 goes over 90% the scripts displays a bunch of stuff (q sess, q proc, q
 log, show logpin, etc),  runs a show logpin cancel,  displays the
 stuff again,
 and finally emails me. (We run in normal mode.)   It's been very
 effective
 in preventing a log full condition.

 rick






 From:   Bos, Karel karel@atosorigin.com
 To: ADSM-L@VM.MARIST.EDU
 Date:   05/12/2011 06:11 AM
 Subject:Re: TSM Recovery log is pinning since upgrade to
5.5.5.0
 code
 Sent by:ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



 Hi,

 Dunno if u already looked at these options: THROUGHPUTDATATHRESHOLD
 and THROUGHPUTTIMETHRESHOLD. After a couple of crashes due to log
 filling up

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Mueller, Ken]

So perhaps the issue folks are seeing isn't directly caused by a
languishing log pin, but that something in the 5.5.5.0 upgrade made a
questionable change to the database causing corruption/inconsistency?
Or some housekeeping operation changed in 5.5.5.0 that is generating
more/larger transactions or activity log entries, causing the log to
fill quicker than before, thus exposing these previously innocuous log
pins?  Just thinking out loud...

Ken Mueller
Chief Technology Officer
Magna Carta Companies

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Cheung, Richard
Sent: Tuesday, May 17, 2011 3:19 AM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code


Hi. I had a server like that - log filled up extremely quickly, multiple
log pins etc.  

I ended up actually doing a full DB DUMP, DB RELOAD, as it turned out to
be something corrupted within my database + logs. It has been generally
running fine ever since.  

I do agree, this is something related to workload for sure, but also do
look at the health of your database and logs - this may be pointing to
possible issues within them. 



-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Loon, EJ van - SPLXO
Sent: Monday, 16 May 2011 11:49 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to
5.5.5.0 code

Hi Richard!
 So is 5.5.5.0 also having this issue with the log pinning?
Yes. As soon as the log hist 80% TSM starts delaying transactions. In
this level a bug was fixed which caused the log to fill too soon after
it reached 80%. I'm running 5.5.5.2 and it's still filling to 80%
multiple times a night. Kind regards, Eric van Loon KLM Royal Dutch
Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Cheung, Richard
Sent: zondag 15 mei 2011 04:02
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Hi

The worry about this is I had a PMR opened for this issue on my servers,
which are currently at 5.5.4. I was advised it's a known issue, refer to
APAR IC64438, and the recommended fix is to go to 5.5.5.0

So is 5.5.5.0 also having this issue with the log pinning?

I also believed this was happening in early versions of TSM v6.. But the
later builds are okay - can anyone confirm?


Richard Cheung | Server Engineer | Information Systems | Santos Limited
Santos Centre | 60 Flinders Street Adelaide SA 5000 Australia
Ph +61 8 8116 7125 | 7 Fax +61 8 8116 5073 |   richard.che...@santos.com


-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Dave Canan
Sent: Saturday, 14 May 2011 1:22 AM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to
5.5.5.0 code

Andy Raibeck and I have been monitoring this thread for the past few
weeks and have been having numerous discussions about what we need to do
to address these problems. In addition, we have noted the many PMRs
coming into the IBM support queue, and many of these PMRs are now on my
queue. Andy will be assisting me along with other support people as
needed.

As part of this analysis we will be doing, we have a new questionnaire
(titled Items to Gather for a PMR with Log Pinning Condition for TSM
V5) that customers will be filling out that ask several questions
regarding your environment. We understand that this involves extra work
to gather this information, but there can be many different areas that
can cause log pinning conditions, so this information is needed. In
addition, there will be a script provided (named serverperf5a.pl) that
will help us gather additional data. Both of these will be provided to
you by support. When these PMRs are now opened, please make sure that
level 1 support adds the keyword LOGPIN55 to the PMR. This will allow
Andy and I to quickly find all the PMRs being worked for this issue.

Eric, your PMR is now one that I have on my queue (or I will shortly
today). We will be contacting you to work the PMR.

Dave Canan
IBM ATS TSM Performance
ddcananATUSDOTIBMDOTCOM
916-723-2410

On Thu, May 12, 2011 at 7:53 AM, Loon, EJ van - SPLXO
eric-van.l...@klm.com
 wrote:

 Hi Rick!
 You are running in normal mode. In this mode the recovery log only 
 contains uncommited transactions. Don't you agree that in this case, 
 the log should NEVER reach 90%? It should not even reach something
like 40%!
 I don't want to have to implement extra monitoring to keep TSM from 
 crashing. It should just work. Like it did when we were running 5.3...

 Kind regards, Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf 
 Of Richard Rhodes
 Sent: donderdag 12 mei 2011 15:04
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Loon, EJ van - SPLXO]

Hi Richard!
 So is 5.5.5.0 also having this issue with the log pinning?
Yes. As soon as the log hist 80% TSM starts delaying transactions. In
this level a bug was fixed which caused the log to fill too soon after
it reached 80%. I'm running 5.5.5.2 and it's still filling to 80%
multiple times a night.
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Cheung, Richard
Sent: zondag 15 mei 2011 04:02
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Hi

The worry about this is I had a PMR opened for this issue on my servers,
which are currently at 5.5.4. I was advised it's a known issue, refer to
APAR IC64438, and the recommended fix is to go to 5.5.5.0

So is 5.5.5.0 also having this issue with the log pinning?

I also believed this was happening in early versions of TSM v6.. But the
later builds are okay - can anyone confirm?


Richard Cheung | Server Engineer | Information Systems | Santos Limited
Santos Centre | 60 Flinders Street Adelaide SA 5000 Australia
Ph +61 8 8116 7125 | 7 Fax +61 8 8116 5073 |   richard.che...@santos.com


-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Dave Canan
Sent: Saturday, 14 May 2011 1:22 AM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to
5.5.5.0 code

Andy Raibeck and I have been monitoring this thread for the past few
weeks and have been having numerous discussions about what we need to do
to address these problems. In addition, we have noted the many PMRs
coming into the IBM support queue, and many of these PMRs are now on my
queue. Andy will be assisting me along with other support people as
needed.

As part of this analysis we will be doing, we have a new questionnaire
(titled Items to Gather for a PMR with Log Pinning Condition for TSM
V5) that customers will be filling out that ask several questions
regarding your environment. We understand that this involves extra work
to gather this information, but there can be many different areas that
can cause log pinning conditions, so this information is needed. In
addition, there will be a script provided (named serverperf5a.pl) that
will help us gather additional data. Both of these will be provided to
you by support. When these PMRs are now opened, please make sure that
level 1 support adds the keyword LOGPIN55 to the PMR. This will allow
Andy and I to quickly find all the PMRs being worked for this issue.

Eric, your PMR is now one that I have on my queue (or I will shortly
today).
We will be contacting you to work the PMR.

Dave Canan
IBM ATS TSM Performance
ddcananATUSDOTIBMDOTCOM
916-723-2410

On Thu, May 12, 2011 at 7:53 AM, Loon, EJ van - SPLXO
eric-van.l...@klm.com
 wrote:

 Hi Rick!
 You are running in normal mode. In this mode the recovery log only
 contains uncommited transactions. Don't you agree that in this case,
 the log should NEVER reach 90%? It should not even reach something
like 40%!
 I don't want to have to implement extra monitoring to keep TSM from
 crashing. It should just work. Like it did when we were running 5.3...
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf
 Of Richard Rhodes
 Sent: donderdag 12 mei 2011 15:04
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 I have a cron job that checks log utilization every 15m.  If the log
 goes over 90% the scripts displays a bunch of stuff (q sess, q proc, q
 log, show logpin, etc),  runs a show logpin cancel,  displays the
 stuff again,
 and finally emails me. (We run in normal mode.)   It's been very
 effective
 in preventing a log full condition.

 rick






 From:   Bos, Karel karel@atosorigin.com
 To: ADSM-L@VM.MARIST.EDU
 Date:   05/12/2011 06:11 AM
 Subject:Re: TSM Recovery log is pinning since upgrade to
5.5.5.0
 code
 Sent by:ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



 Hi,

 Dunno if u already looked at these options: THROUGHPUTDATATHRESHOLD
 and THROUGHPUTTIMETHRESHOLD. After a couple of crashes due to log
 filling up, a couple of years back, I standard added them to all my
 clients (with low values to only kill the slowest/hung ones) and
 untill now never had any issues with the V5.5 and below log space
 under normal backup loads.

 Log filling only happend when some sys admin made some changes to the
 biggest file servers we had but even this was handled like it should
 be done by ITSM.

 Regards,

 Karel



 
 Van: ADSM: Dist Stor Manager [ADSM-L@VM.MARIST.EDU] namens Loon, EJ
 van
 -
 SPLXO [eric-van.l...@klm.com]
 Verzonden: donderdag 12 mei 2011 11:11
 Aan: ADSM-L@VM.MARIST.EDU
 Onderwerp: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0
 code

 Hi Paul!
 We

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Mark Mooney]

Rookie Question, What is Log Pinning?

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Loon, 
EJ van - SPLXO
Sent: Monday, May 16, 2011 10:19 AM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Hi Richard!
 So is 5.5.5.0 also having this issue with the log pinning?
Yes. As soon as the log hist 80% TSM starts delaying transactions. In this 
level a bug was fixed which caused the log to fill too soon after it reached 
80%. I'm running 5.5.5.2 and it's still filling to 80% multiple times a night.
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of 
Cheung, Richard
Sent: zondag 15 mei 2011 04:02
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Hi

The worry about this is I had a PMR opened for this issue on my servers, which 
are currently at 5.5.4. I was advised it's a known issue, refer to APAR 
IC64438, and the recommended fix is to go to 5.5.5.0

So is 5.5.5.0 also having this issue with the log pinning?

I also believed this was happening in early versions of TSM v6.. But the later 
builds are okay - can anyone confirm?


Richard Cheung | Server Engineer | Information Systems | Santos Limited Santos 
Centre | 60 Flinders Street Adelaide SA 5000 Australia
Ph +61 8 8116 7125 | 7 Fax +61 8 8116 5073 |   richard.che...@santos.com


-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Dave 
Canan
Sent: Saturday, 14 May 2011 1:22 AM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to
5.5.5.0 code

Andy Raibeck and I have been monitoring this thread for the past few weeks and 
have been having numerous discussions about what we need to do to address these 
problems. In addition, we have noted the many PMRs coming into the IBM support 
queue, and many of these PMRs are now on my queue. Andy will be assisting me 
along with other support people as needed.

As part of this analysis we will be doing, we have a new questionnaire (titled 
Items to Gather for a PMR with Log Pinning Condition for TSM
V5) that customers will be filling out that ask several questions regarding 
your environment. We understand that this involves extra work to gather this 
information, but there can be many different areas that can cause log pinning 
conditions, so this information is needed. In addition, there will be a script 
provided (named serverperf5a.pl) that will help us gather additional data. Both 
of these will be provided to you by support. When these PMRs are now opened, 
please make sure that level 1 support adds the keyword LOGPIN55 to the PMR. 
This will allow Andy and I to quickly find all the PMRs being worked for this 
issue.

Eric, your PMR is now one that I have on my queue (or I will shortly today).
We will be contacting you to work the PMR.

Dave Canan
IBM ATS TSM Performance
ddcananATUSDOTIBMDOTCOM
916-723-2410

On Thu, May 12, 2011 at 7:53 AM, Loon, EJ van - SPLXO eric-van.l...@klm.com
 wrote:

 Hi Rick!
 You are running in normal mode. In this mode the recovery log only
 contains uncommited transactions. Don't you agree that in this case,
 the log should NEVER reach 90%? It should not even reach something
like 40%!
 I don't want to have to implement extra monitoring to keep TSM from
 crashing. It should just work. Like it did when we were running 5.3...
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf
 Of Richard Rhodes
 Sent: donderdag 12 mei 2011 15:04
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 I have a cron job that checks log utilization every 15m.  If the log
 goes over 90% the scripts displays a bunch of stuff (q sess, q proc, q
 log, show logpin, etc),  runs a show logpin cancel,  displays the
 stuff again,
 and finally emails me. (We run in normal mode.)   It's been very
 effective
 in preventing a log full condition.

 rick






 From:   Bos, Karel karel@atosorigin.com
 To: ADSM-L@VM.MARIST.EDU
 Date:   05/12/2011 06:11 AM
 Subject:Re: TSM Recovery log is pinning since upgrade to
5.5.5.0
 code
 Sent by:ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



 Hi,

 Dunno if u already looked at these options: THROUGHPUTDATATHRESHOLD
 and THROUGHPUTTIMETHRESHOLD. After a couple of crashes due to log
 filling up, a couple of years back, I standard added them to all my
 clients (with low values to only kill the slowest/hung ones) and
 untill now never had any issues with the V5.5 and below log space
 under normal backup loads.

 Log filling only happend when some sys admin made some changes to the
 biggest file servers we had but even this was handled like it should
 be done by ITSM.

 Regards,

 Karel

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Loon, EJ van - SPLXO]

Hi Dave!
Thank you VERY VERY much for your help
I'm indeed contacted by your colleague, I filled the document and the
script is currently collecting data. I'm absolutely willing to do the
extra work, if it helps IBM fixing this issue, no problem!
Thanks again!
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Dave Canan
Sent: vrijdag 13 mei 2011 17:52
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Andy Raibeck and I have been monitoring this thread for the past few
weeks
and have been having numerous discussions about what we need to do to
address these problems. In addition, we have noted the many PMRs coming
into
the IBM support queue, and many of these PMRs are now on my queue. Andy
will
be assisting me along with other support people as needed.

As part of this analysis we will be doing, we have a new questionnaire
(titled Items to Gather for a PMR with Log Pinning Condition for TSM
V5)
that customers will be filling out that ask several questions regarding
your
environment. We understand that this involves extra work to gather this
information, but there can be many different areas that can cause log
pinning conditions, so this information is needed. In addition, there
will
be a script provided (named serverperf5a.pl) that will help us gather
additional data. Both of these will be provided to you by support. When
these PMRs are now opened, please make sure that level 1 support adds
the
keyword LOGPIN55 to the PMR. This will allow Andy and I to quickly find
all
the PMRs being worked for this issue.

Eric, your PMR is now one that I have on my queue (or I will shortly
today).
We will be contacting you to work the PMR.

Dave Canan
IBM ATS TSM Performance
ddcananATUSDOTIBMDOTCOM
916-723-2410

On Thu, May 12, 2011 at 7:53 AM, Loon, EJ van - SPLXO
eric-van.l...@klm.com
 wrote:

 Hi Rick!
 You are running in normal mode. In this mode the recovery log only
 contains uncommited transactions. Don't you agree that in this case,
the
 log should NEVER reach 90%? It should not even reach something like
40%!
 I don't want to have to implement extra monitoring to keep TSM from
 crashing. It should just work. Like it did when we were running 5.3...
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf
Of
 Richard Rhodes
 Sent: donderdag 12 mei 2011 15:04
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 I have a cron job that checks log utilization every 15m.  If the log
 goes
 over 90% the scripts displays a bunch of stuff (q sess, q proc, q log,
 show logpin, etc),  runs a show logpin cancel,  displays the stuff
 again,
 and finally emails me. (We run in normal mode.)   It's been very
 effective
 in preventing a log full condition.

 rick






 From:   Bos, Karel karel@atosorigin.com
 To: ADSM-L@VM.MARIST.EDU
 Date:   05/12/2011 06:11 AM
 Subject:Re: TSM Recovery log is pinning since upgrade to
5.5.5.0
 code
 Sent by:ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



 Hi,

 Dunno if u already looked at these options: THROUGHPUTDATATHRESHOLD
and
 THROUGHPUTTIMETHRESHOLD. After a couple of crashes due to log filling
 up,
 a couple of years back, I standard added them to all my clients (with
 low
 values to only kill the slowest/hung ones) and untill now never had
any
 issues with the V5.5 and below log space under normal backup loads.

 Log filling only happend when some sys admin made some changes to the
 biggest file servers we had but even this was handled like it should
be
 done by ITSM.

 Regards,

 Karel



 
 Van: ADSM: Dist Stor Manager [ADSM-L@VM.MARIST.EDU] namens Loon, EJ
van
 -
 SPLXO [eric-van.l...@klm.com]
 Verzonden: donderdag 12 mei 2011 11:11
 Aan: ADSM-L@VM.MARIST.EDU
 Onderwerp: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0
code

 Hi Paul!
 We are already running 5.5.5.2 and the log is still filling up, even
 after switching from rollforward to normal mode.
 Management currently is questioning whether TSM is the right product
for
 the future. Although I'm a big fan of TSM for 15 years, I'm really in
 doubt too...
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf
Of
 Paul Fielding
 Sent: woensdag 11 mei 2011 20:05
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 Hi folks, if this has already been commented on, my apologies, I
hvaen't
 been following closely but just noticed this thread.

 We were experiencing log pinned issues after upgrading to 5.5.0.0 code
 at
 one of my client sites.  What we were finding was that, occasionally,
 I'd
 get up in the morning and look

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Schneider, Jim]

Answer to rookie question:

All files backed up create an entry in the transaction log.  When the
file backup is complete the entry is eligible to be written to the
database.  TSM only writes the oldest entry in the transaction log.  If
a file backup takes a long time it can become the oldest entry in the
log.  Any new entries for backed-up files are stored in the log until
the long-running file backup is complete.  This is called log pinning.

If the backup for one file takes too long the transaction log can become
more than 80% full, at which time TSM slows down its backup process to
allow transaction log processing to complete.  If the transaction log
becomes completely full, TSM processing stops.

Hope this helps,
Jim Schneider

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@vm.marist.edu] On Behalf Of
Mark Mooney
Sent: Monday, May 16, 2011 11:23 AM
To: ADSM-L@vm.marist.edu
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to
5.5.5.0 code

Rookie Question, What is Log Pinning?

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Loon, EJ van - SPLXO
Sent: Monday, May 16, 2011 10:19 AM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Hi Richard!
 So is 5.5.5.0 also having this issue with the log pinning?
Yes. As soon as the log hist 80% TSM starts delaying transactions. In
this level a bug was fixed which caused the log to fill too soon after
it reached 80%. I'm running 5.5.5.2 and it's still filling to 80%
multiple times a night.
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Cheung, Richard
Sent: zondag 15 mei 2011 04:02
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Hi

The worry about this is I had a PMR opened for this issue on my servers,
which are currently at 5.5.4. I was advised it's a known issue, refer to
APAR IC64438, and the recommended fix is to go to 5.5.5.0

So is 5.5.5.0 also having this issue with the log pinning?

I also believed this was happening in early versions of TSM v6.. But the
later builds are okay - can anyone confirm?


Richard Cheung | Server Engineer | Information Systems | Santos Limited
Santos Centre | 60 Flinders Street Adelaide SA 5000 Australia
Ph +61 8 8116 7125 | 7 Fax +61 8 8116 5073 |   richard.che...@santos.com


-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Dave Canan
Sent: Saturday, 14 May 2011 1:22 AM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to
5.5.5.0 code

Andy Raibeck and I have been monitoring this thread for the past few
weeks and have been having numerous discussions about what we need to do
to address these problems. In addition, we have noted the many PMRs
coming into the IBM support queue, and many of these PMRs are now on my
queue. Andy will be assisting me along with other support people as
needed.

As part of this analysis we will be doing, we have a new questionnaire
(titled Items to Gather for a PMR with Log Pinning Condition for TSM
V5) that customers will be filling out that ask several questions
regarding your environment. We understand that this involves extra work
to gather this information, but there can be many different areas that
can cause log pinning conditions, so this information is needed. In
addition, there will be a script provided (named serverperf5a.pl) that
will help us gather additional data. Both of these will be provided to
you by support. When these PMRs are now opened, please make sure that
level 1 support adds the keyword LOGPIN55 to the PMR. This will allow
Andy and I to quickly find all the PMRs being worked for this issue.

Eric, your PMR is now one that I have on my queue (or I will shortly
today).
We will be contacting you to work the PMR.

Dave Canan
IBM ATS TSM Performance
ddcananATUSDOTIBMDOTCOM
916-723-2410

On Thu, May 12, 2011 at 7:53 AM, Loon, EJ van - SPLXO
eric-van.l...@klm.com
 wrote:

 Hi Rick!
 You are running in normal mode. In this mode the recovery log only
 contains uncommited transactions. Don't you agree that in this case,
 the log should NEVER reach 90%? It should not even reach something
like 40%!
 I don't want to have to implement extra monitoring to keep TSM from
 crashing. It should just work. Like it did when we were running 5.3...
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf
 Of Richard Rhodes
 Sent: donderdag 12 mei 2011 15:04
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 I have a cron job that checks log utilization every 15m.  If the log
 goes over 90% the scripts displays a bunch of stuff (q sess, q proc, q
 log, show logpin, etc),  runs

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Andrew Raibeck]

Hi Eric (and others following this thread),

Just to clarify: we are not aware of any code defects in the TSM 5.5.5
server that would contribute to recovery log pin situations. Rather, it is
likely related to workload. But having said that, we do not dismiss any
possibility out of hand.

Some things that can pin the recovery log:

- Windows 2008, Windows 2008 R2, Windows 7 and Windows Vista system state
backups using the 6.2 client can cause the recovery log to pin during the
object assignment phase of the backup

- NDMP backups that span more than one tape (the pin occurs at the time the
first tape starts to rewind and another tape is needed)

- BACKUP IMAGE operations that span more than one tape (similar situation
as NDMP)

- Data Protection for * products, typically with large transactions that
pin the log until the transaction is committed

Of course there can be other reasons as well. It may very well be that
others who responds me too to this post, could be experiencing the
recovery log pin for different reasons.

Or we might actually uncover a code-related issue... it all remains to be
determined.

Best regards,

Andy Raibeck
IBM Software Group
Tivoli Storage Manager Client Product Development
Level 3 Team Lead
Internal Notes e-mail: Andrew Raibeck/Hartford/IBM@IBMUS
Internet e-mail: stor...@us.ibm.com

IBM Tivoli Storage Manager support web page:
http://www.ibm.com/support/entry/portal/Overview/Software/Tivoli/Tivoli_Storage_Manager

ADSM: Dist Stor Manager ADSM-L@vm.marist.edu wrote on 2011-05-16
10:14:15:

 From: Loon, EJ van - SPLXO eric-van.l...@klm.com
 To: ADSM-L@vm.marist.edu
 Date: 2011-05-16 13:13
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code
 Sent by: ADSM: Dist Stor Manager ADSM-L@vm.marist.edu

 Hi Dave!
 Thank you VERY VERY much for your help
 I'm indeed contacted by your colleague, I filled the document and the
 script is currently collecting data. I'm absolutely willing to do the
 extra work, if it helps IBM fixing this issue, no problem!
 Thanks again!
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
 Dave Canan
 Sent: vrijdag 13 mei 2011 17:52
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 Andy Raibeck and I have been monitoring this thread for the past few
 weeks
 and have been having numerous discussions about what we need to do to
 address these problems. In addition, we have noted the many PMRs coming
 into
 the IBM support queue, and many of these PMRs are now on my queue. Andy
 will
 be assisting me along with other support people as needed.

 As part of this analysis we will be doing, we have a new questionnaire
 (titled Items to Gather for a PMR with Log Pinning Condition for TSM
 V5)
 that customers will be filling out that ask several questions regarding
 your
 environment. We understand that this involves extra work to gather this
 information, but there can be many different areas that can cause log
 pinning conditions, so this information is needed. In addition, there
 will
 be a script provided (named serverperf5a.pl) that will help us gather
 additional data. Both of these will be provided to you by support. When
 these PMRs are now opened, please make sure that level 1 support adds
 the
 keyword LOGPIN55 to the PMR. This will allow Andy and I to quickly find
 all
 the PMRs being worked for this issue.

 Eric, your PMR is now one that I have on my queue (or I will shortly
 today).
 We will be contacting you to work the PMR.

 Dave Canan
 IBM ATS TSM Performance
 ddcananATUSDOTIBMDOTCOM
 916-723-2410

 On Thu, May 12, 2011 at 7:53 AM, Loon, EJ van - SPLXO
 eric-van.l...@klm.com
  wrote:

  Hi Rick!
  You are running in normal mode. In this mode the recovery log only
  contains uncommited transactions. Don't you agree that in this case,
 the
  log should NEVER reach 90%? It should not even reach something like
 40%!
  I don't want to have to implement extra monitoring to keep TSM from
  crashing. It should just work. Like it did when we were running 5.3...
  Kind regards,
  Eric van Loon
  KLM Royal Dutch Airlines
 
  -Original Message-
  From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf
 Of
  Richard Rhodes
  Sent: donderdag 12 mei 2011 15:04
  To: ADSM-L@VM.MARIST.EDU
  Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code
 
  I have a cron job that checks log utilization every 15m.  If the log
  goes
  over 90% the scripts displays a bunch of stuff (q sess, q proc, q log,
  show logpin, etc),  runs a show logpin cancel,  displays the stuff
  again,
  and finally emails me. (We run in normal mode.)   It's been very
  effective
  in preventing a log full condition.
 
  rick
 
 
 
 
 
 
  From:   Bos, Karel karel@atosorigin.com
  To: ADSM-L@VM.MARIST.EDU
  Date:   05/12/2011 06:11 AM
  Subject:Re: TSM Recovery log

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Mark Mooney]

Thank you Gentlemen, Much Appreciated!

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of 
Schneider, Jim
Sent: Monday, May 16, 2011 1:52 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Answer to rookie question:

All files backed up create an entry in the transaction log.  When the file 
backup is complete the entry is eligible to be written to the database.  TSM 
only writes the oldest entry in the transaction log.  If a file backup takes a 
long time it can become the oldest entry in the log.  Any new entries for 
backed-up files are stored in the log until the long-running file backup is 
complete.  This is called log pinning.

If the backup for one file takes too long the transaction log can become more 
than 80% full, at which time TSM slows down its backup process to allow 
transaction log processing to complete.  If the transaction log becomes 
completely full, TSM processing stops.

Hope this helps,
Jim Schneider

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@vm.marist.edu] On Behalf Of Mark 
Mooney
Sent: Monday, May 16, 2011 11:23 AM
To: ADSM-L@vm.marist.edu
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to
5.5.5.0 code

Rookie Question, What is Log Pinning?

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Loon, 
EJ van - SPLXO
Sent: Monday, May 16, 2011 10:19 AM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Hi Richard!
 So is 5.5.5.0 also having this issue with the log pinning?
Yes. As soon as the log hist 80% TSM starts delaying transactions. In this 
level a bug was fixed which caused the log to fill too soon after it reached 
80%. I'm running 5.5.5.2 and it's still filling to 80% multiple times a night.
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of 
Cheung, Richard
Sent: zondag 15 mei 2011 04:02
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Hi

The worry about this is I had a PMR opened for this issue on my servers, which 
are currently at 5.5.4. I was advised it's a known issue, refer to APAR 
IC64438, and the recommended fix is to go to 5.5.5.0

So is 5.5.5.0 also having this issue with the log pinning?

I also believed this was happening in early versions of TSM v6.. But the later 
builds are okay - can anyone confirm?


Richard Cheung | Server Engineer | Information Systems | Santos Limited Santos 
Centre | 60 Flinders Street Adelaide SA 5000 Australia
Ph +61 8 8116 7125 | 7 Fax +61 8 8116 5073 |   richard.che...@santos.com


-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Dave 
Canan
Sent: Saturday, 14 May 2011 1:22 AM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to
5.5.5.0 code

Andy Raibeck and I have been monitoring this thread for the past few weeks and 
have been having numerous discussions about what we need to do to address these 
problems. In addition, we have noted the many PMRs coming into the IBM support 
queue, and many of these PMRs are now on my queue. Andy will be assisting me 
along with other support people as needed.

As part of this analysis we will be doing, we have a new questionnaire (titled 
Items to Gather for a PMR with Log Pinning Condition for TSM
V5) that customers will be filling out that ask several questions regarding 
your environment. We understand that this involves extra work to gather this 
information, but there can be many different areas that can cause log pinning 
conditions, so this information is needed. In addition, there will be a script 
provided (named serverperf5a.pl) that will help us gather additional data. Both 
of these will be provided to you by support. When these PMRs are now opened, 
please make sure that level 1 support adds the keyword LOGPIN55 to the PMR. 
This will allow Andy and I to quickly find all the PMRs being worked for this 
issue.

Eric, your PMR is now one that I have on my queue (or I will shortly today).
We will be contacting you to work the PMR.

Dave Canan
IBM ATS TSM Performance
ddcananATUSDOTIBMDOTCOM
916-723-2410

On Thu, May 12, 2011 at 7:53 AM, Loon, EJ van - SPLXO eric-van.l...@klm.com
 wrote:

 Hi Rick!
 You are running in normal mode. In this mode the recovery log only
 contains uncommited transactions. Don't you agree that in this case,
 the log should NEVER reach 90%? It should not even reach something
like 40%!
 I don't want to have to implement extra monitoring to keep TSM from
 crashing. It should just work. Like it did when we were running 5.3...
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Paul Zarnowski]

Wanda,

Apologies if this was already mentioned (I haven't read this entire thread), 
but Expiration can have a dramatic impact when the log is pinned.  It does not 
in itself typically pin the log for long periods of time.  However, it can 
exacerbate the situation where the log is pinned, especially when you have 
logmode=rollforward.  The log is actually circular and has a head and a 
tail.  The oldest open transaction pins the tail of the log.  Any other 
updates, whether they remain open or not (for rollforward) will move the head 
forward.  When the head catches up to the tail, then the log utilization 
reaches 100%.  The faster the head moves in a situation where the tail is 
pinned, the faster your log utilization will near 100%.  Expiration is one 
thing which can move the head the fastest (IMHO).  When I see our logutil 
rising, the first thing I check is to see if expiration is running, and if so 
cancel it.  This will slow down the rate of increase in logutil.  It would be 
nice if this happened automatically, even before db writes are slowed down, but 
I don't believe it does.

This may not apply for logmode=normal, I'm not really sure.

..Paul

At 04:17 PM 5/15/2011, Prather, Wanda wrote:
Am I correct that scheduling of background processes is important as well?
Don't expiration, migration, backup stgpool also create log transactions?
I'm wondering if it is the housekeeping, rather than the number of client 
backups, that is sending my server over the edge.


--
Paul ZarnowskiPh: 607-255-4757
Manager, Storage Services Fx: 607-255-8521
719 Rhodes Hall, Ithaca, NY 14853-3801Em: p...@cornell.edu  

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[David E Ehresman]

 Andrew Raibeck stor...@us.ibm.com 5/16/2011 2:04 PM 
Some things that can pin the recovery log:

- NDMP backups that span more than one tape (the pin occurs at the
time the
first tape starts to rewind and another tape is needed)
At what point does the NDMP induced pinning get released?

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Andrew Raibeck]

 At what point does the NDMP induced pinning get released?

With NDMP, each file system is backed up as an image. Therefore each file
system is backed up in its own transaction. The transaction is committed
(pin released) when the backup for the file system completes.

If the backup does not span more than one tape then there is no pin for the
backup of that file system.

Best regards,

Andy Raibeck
IBM Software Group
Tivoli Storage Manager Client Product Development
Level 3 Team Lead
Internal Notes e-mail: Andrew Raibeck/Hartford/IBM@IBMUS
Internet e-mail: stor...@us.ibm.com

IBM Tivoli Storage Manager support web page:
http://www.ibm.com/support/entry/portal/Overview/Software/Tivoli/Tivoli_Storage_Manager

ADSM: Dist Stor Manager ADSM-L@vm.marist.edu wrote on 2011-05-16
15:53:31:

 From: David E Ehresman deehr...@louisville.edu
 To: ADSM-L@vm.marist.edu
 Date: 2011-05-16 15:59
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code
 Sent by: ADSM: Dist Stor Manager ADSM-L@vm.marist.edu

  Andrew Raibeck stor...@us.ibm.com 5/16/2011 2:04 PM 
 Some things that can pin the recovery log:
 
 - NDMP backups that span more than one tape (the pin occurs at the
 time the
 first tape starts to rewind and another tape is needed)
 At what point does the NDMP induced pinning get released?

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Prather, Wanda]

- Data Protection for * products, typically with large transactions that pin 
the log until the transaction is committed

I'm assuming that a transaction for MSSQL = 1 data base when doing the 
requisite full dump yes?
So in the case of ginormous SQL data bases that take hours to back up, what do 
you recommend to avoid log pinning?



-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Andrew 
Raibeck
Sent: Monday, May 16, 2011 2:04 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Hi Eric (and others following this thread),

Just to clarify: we are not aware of any code defects in the TSM 5.5.5 server 
that would contribute to recovery log pin situations. Rather, it is likely 
related to workload. But having said that, we do not dismiss any possibility 
out of hand.

Some things that can pin the recovery log:

- Windows 2008, Windows 2008 R2, Windows 7 and Windows Vista system state 
backups using the 6.2 client can cause the recovery log to pin during the 
object assignment phase of the backup

- NDMP backups that span more than one tape (the pin occurs at the time the 
first tape starts to rewind and another tape is needed)

- BACKUP IMAGE operations that span more than one tape (similar situation as 
NDMP)

- Data Protection for * products, typically with large transactions that pin 
the log until the transaction is committed

Of course there can be other reasons as well. It may very well be that others 
who responds me too to this post, could be experiencing the recovery log pin 
for different reasons.

Or we might actually uncover a code-related issue... it all remains to be 
determined.

Best regards,

Andy Raibeck
IBM Software Group
Tivoli Storage Manager Client Product Development Level 3 Team Lead Internal 
Notes e-mail: Andrew Raibeck/Hartford/IBM@IBMUS Internet e-mail: 
stor...@us.ibm.com

IBM Tivoli Storage Manager support web page:
http://www.ibm.com/support/entry/portal/Overview/Software/Tivoli/Tivoli_Storage_Manager

ADSM: Dist Stor Manager ADSM-L@vm.marist.edu wrote on 2011-05-16
10:14:15:

 From: Loon, EJ van - SPLXO eric-van.l...@klm.com
 To: ADSM-L@vm.marist.edu
 Date: 2011-05-16 13:13
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code 
 Sent by: ADSM: Dist Stor Manager ADSM-L@vm.marist.edu

 Hi Dave!
 Thank you VERY VERY much for your help
 I'm indeed contacted by your colleague, I filled the document and the 
 script is currently collecting data. I'm absolutely willing to do the 
 extra work, if it helps IBM fixing this issue, no problem!
 Thanks again!
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf 
 Of Dave Canan
 Sent: vrijdag 13 mei 2011 17:52
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 Andy Raibeck and I have been monitoring this thread for the past few 
 weeks and have been having numerous discussions about what we need to 
 do to address these problems. In addition, we have noted the many PMRs 
 coming into the IBM support queue, and many of these PMRs are now on 
 my queue. Andy will be assisting me along with other support people as 
 needed.

 As part of this analysis we will be doing, we have a new questionnaire 
 (titled Items to Gather for a PMR with Log Pinning Condition for TSM
 V5)
 that customers will be filling out that ask several questions 
 regarding your environment. We understand that this involves extra 
 work to gather this information, but there can be many different areas 
 that can cause log pinning conditions, so this information is needed. 
 In addition, there will be a script provided (named serverperf5a.pl) 
 that will help us gather additional data. Both of these will be 
 provided to you by support. When these PMRs are now opened, please 
 make sure that level 1 support adds the keyword LOGPIN55 to the PMR. 
 This will allow Andy and I to quickly find all the PMRs being worked 
 for this issue.

 Eric, your PMR is now one that I have on my queue (or I will shortly 
 today).
 We will be contacting you to work the PMR.

 Dave Canan
 IBM ATS TSM Performance
 ddcananATUSDOTIBMDOTCOM
 916-723-2410

 On Thu, May 12, 2011 at 7:53 AM, Loon, EJ van - SPLXO 
 eric-van.l...@klm.com
  wrote:

  Hi Rick!
  You are running in normal mode. In this mode the recovery log only 
  contains uncommited transactions. Don't you agree that in this case,
 the
  log should NEVER reach 90%? It should not even reach something like
 40%!
  I don't want to have to implement extra monitoring to keep TSM from 
  crashing. It should just work. Like it did when we were running 5.3...
  Kind regards,
  Eric van Loon
  KLM Royal Dutch Airlines
 
  -Original Message-
  From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On 
  Behalf
 Of
  Richard Rhodes
  Sent: donderdag 12 mei

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Zoltan Forray]

With Oracle TDP we are able to set the chunk/transaction size. By default it 
tries to do a whole database as a single transaction. We broke up the problem 
nodes into 2GB chunks.

Prather, Wanda wprat...@icfi.com wrote:

- Data Protection for * products, typically with large transactions that pin 
the log until the transaction is committed I'm assuming that a transaction 
for MSSQL = 1 data base when doing the requisite full dump yes? So in the case 
of ginormous SQL data bases that take hours to back up, what do you recommend 
to avoid log pinning? -Original Message- From: ADSM: Dist Stor Manager 
[mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Andrew Raibeck Sent: Monday, May 16, 
2011 2:04 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] TSM Recovery log is 
pinning since upgrade to 5.5.5.0 code Hi Eric (and others following this 
thread), Just to clarify: we are not aware of any code defects in the TSM 5.5.5 
server that would contribute to recovery log pin situations. Rather, it is 
likely related to workload. But having said that, we do not dismiss any 
possibility out of hand. Some things that can pin the recovery log: - Windows 
2008, Windows 2008 R2, Windows 7 and Windows Vista system state backups using 
the 6.2 client can cause the recovery log to pin during the object assignment 
phase of the backup - NDMP backups that span more than one tape (the pin occurs 
at the time the first tape starts to rewind and another tape is needed) - 
BACKUP IMAGE operations that span more than one tape (similar situation as 
NDMP) - Data Protection for * products, typically with large transactions that 
pin the log until the transaction is committed Of course there can be other 
reasons as well. It may very well be that others who responds me too to this 
post, could be experiencing the recovery log pin for different reasons. Or we 
might actually uncover a code-related issue... it all remains to be determined. 
Best regards, Andy Raibeck IBM Software Group Tivoli Storage Manager Client 
Product Development Level 3 Team Lead Internal Notes e-mail: Andrew 
Raibeck/Hartford/IBM@IBMUS Internet e-mail: stor...@us.ibm.com IBM Tivoli 
Storage Manager support web page: 
http://www.ibm.com/support/entry/portal/Overview/Software/Tivoli/Tivoli_Storage_Manager
 ADSM: Dist Stor Manager ADSM-L@vm.marist.edu wrote on 2011-05-16 10:14:15: 
 From: Loon, EJ van - SPLXO eric-van.l...@klm.com  To: 
ADSM-L@vm.marist.edu  Date: 2011-05-16 13:13  Subject: Re: TSM Recovery log 
is pinning since upgrade to 5.5.5.0 code  Sent by: ADSM: Dist Stor Manager 
ADSM-L@vm.marist.edu   Hi Dave!  Thank you VERY VERY much for your 
help  I'm indeed contacted by your colleague, I filled the document and 
the  script is currently collecting data. I'm absolutely willing to do the  
extra work, if it helps IBM fixing this issue, no problem!  Thanks again!  
Kind regards,  Eric van Loon  KLM Royal Dutch Airlines   -Original 
Message-  From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On 
Behalf  Of Dave Canan  Sent: vrijdag 13 mei 2011 17:52  To: 
ADSM-L@VM.MARIST.EDU  Subject: Re: TSM Recovery log is pinning since upgrade 
to 5.5.5.0 code   Andy Raibeck and I have been monitoring this thread for the 
past few  weeks and have been having numerous discussions about what we need 
to  do to address these problems. In addition, we have noted the many PMRs  
coming into the IBM support queue, and many of these PMRs are now on  my 
queue. Andy will be assisting me along with other support people as  needed.  
 As part of this analysis we will be doing, we have a new questionnaire  
(titled Items to Gather for a PMR with Log Pinning Condition for TSM  V5)  
that customers will be filling out that ask several questions  regarding your 
environment. We understand that this involves extra  work to gather this 
information, but there can be many different areas  that can cause log pinning 
conditions, so this information is needed.  In addition, there will be a 
script provided (named serverperf5a.pl)  that will help us gather additional 
data. Both of these will be  provided to you by support. When these PMRs are 
now opened, please  make sure that level 1 support adds the keyword LOGPIN55 
to the PMR.  This will allow Andy and I to quickly find all the PMRs being 
worked  for this issue.   Eric, your PMR is now one that I have on my queue 
(or I will shortly  today).  We will be contacting you to work the PMR.   
Dave Canan  IBM ATS TSM Performance  ddcananATUSDOTIBMDOTCOM  916-723-2410  
 On Thu, May 12, 2011 at 7:53 AM, Loon, EJ van - SPLXO  
eric-van.l...@klm.com   wrote:Hi Rick!   You are running in normal 
mode. In this mode the recovery log only   contains uncommited transactions. 
Don't you agree that in this case,  the   log should NEVER reach 90%? It 
should not even reach something like  40%!   I don't want to have to 
implement extra monitoring to keep TSM from   crashing. It should just work. 
Like it did when we were

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Prather, Wanda]

Hi Eric, I feel your pain.
I have exactly 1 customer that is having the same problem with log pinning, and 
I can't figure out why.

-In the past, we've all seen logpinning problems when a site is running in 
rollfoward mode with large, slow TDP clients coming in over the WAN.
-There are also issues with the Windows client from 6.0.0 through 6.1.2 that 
will cause the client to sleep for many hours in the middle of a backup, 
which I suspect can contribute to log pinning.

But this customer is running 5.5.4 in normal mode, they upgraded all the Win 
6.0 clients, all the clients come over GigE.
And yet they have problems with the log pinning.
Go figure.

One suspect is a large 64bit SQL TDP that backs up ~300GB per day.  I'm 
suspecting him because it is 1 large object.  But SHOW LOGPIN doesn't always 
finger that guy - it varies.  (I'm beginning to wonder if the code bug is in 
SHOW LOGPIN...)

Wanda

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Richard Sims]

Has anyone experiencing this problem performed a client trace (TSM or 
otherwise) of the apparent troublesome client?  Just wondering if it might 
illuminate something we can't see from external observation.  Given the nature 
of the problem, the overhead of a trace is unlikely to impair things.

Richard Sims

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Prather, Wanda]

Am I correct that scheduling of background processes is important as well?
Don't expiration, migration, backup stgpool also create log transactions?
I'm wondering if it is the housekeeping, rather than the number of client 
backups, that is sending my server over the edge.

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Zoltan 
Forray/AC/VCU
Sent: Friday, May 13, 2011 3:50 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code

When I went through this with Andy R, the only solutions we came up with
was:

1.  Redistribute the workload/schedules to smooth out the spikes of number of 
nodes connecting to perform backups 2.  Redistributed / moved nodes to another 
TSM server 3.  In some cases where we were able to identify nodes causing the 
pinning, which was usually Oracle TDP backups, we made changes to break up the 
backups into multiple smaller transactions/chunks

This eventually smoothed out the stair-stepping of the transaction load 
against the logs.

This basically accelerated / gave me an excuse to move nodes to my 6.1.4 server 
and any new nodes are create there (or our 6.2.x servers).
Zoltan Forray
TSM Software  Hardware Administrator
Virginia Commonwealth University
UCC/Office of Technology Services
zfor...@vcu.edu - 804-828-4807
Don't be a phishing victim - VCU and other reputable organizations will never 
use email to request that you reply with your password, social security number 
or confidential personal information. For more details visit 
http://infosecurity.vcu.edu/phishing.html



From:
Meadows Andrew andrew.mead...@hcahealthcare.com
To:
ADSM-L@VM.MARIST.EDU
Date:
05/13/2011 03:24 PM
Subject:
Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code Sent by:
ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



We have seen this for quite a while with our TSM servers/Clients. The only 
thing we have found that works as a work around is to point our clients to back 
up directly to tape. If you are able to find a resolution to this issue please 
include me on the resolution as well as I would rather not write backup data 
directly to tape during backups.

Thanks,

Andrew

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Dave 
Canan
Sent: Friday, May 13, 2011 10:52 AM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Andy Raibeck and I have been monitoring this thread for the past few weeks and 
have been having numerous discussions about what we need to do to address these 
problems. In addition, we have noted the many PMRs coming into the IBM support 
queue, and many of these PMRs are now on my queue. Andy will be assisting me 
along with other support people as needed.

As part of this analysis we will be doing, we have a new questionnaire (titled 
Items to Gather for a PMR with Log Pinning Condition for TSM V5) that 
customers will be filling out that ask several questions regarding your 
environment. We understand that this involves extra work to gather this 
information, but there can be many different areas that can cause log pinning 
conditions, so this information is needed. In addition, there will be a script 
provided (named serverperf5a.pl) that will help us gather additional data. Both 
of these will be provided to you by support. When these PMRs are now opened, 
please make sure that level 1 support adds the keyword LOGPIN55 to the PMR. 
This will allow Andy and I to quickly find all the PMRs being worked for this 
issue.

Eric, your PMR is now one that I have on my queue (or I will shortly today).
We will be contacting you to work the PMR.

Dave Canan
IBM ATS TSM Performance
ddcananATUSDOTIBMDOTCOM
916-723-2410

On Thu, May 12, 2011 at 7:53 AM, Loon, EJ van - SPLXO eric-van.l...@klm.com
 wrote:

 Hi Rick!
 You are running in normal mode. In this mode the recovery log only 
 contains uncommited transactions. Don't you agree that in this case, 
 the log should NEVER reach 90%? It should not even reach something like 40%!
 I don't want to have to implement extra monitoring to keep TSM from 
 crashing. It should just work. Like it did when we were running 5.3...
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf 
 Of Richard Rhodes
 Sent: donderdag 12 mei 2011 15:04
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 I have a cron job that checks log utilization every 15m.  If the log 
 goes over 90% the scripts displays a bunch of stuff (q sess, q proc, q 
 log, show logpin, etc),  runs a show logpin cancel,  displays the 
 stuff again,
 and finally emails me. (We run in normal mode.)   It's been very
 effective
 in preventing a log full condition.

 rick






 From:   Bos, Karel karel@atosorigin.com

Ang: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Daniel Sparrman]

Background operations shouldnt really be the reason for this, since most of 
them are actually faster on 6.X than 5.5. Expiration should be faster due to 
the ability to control resources for expiration, and copy / move processes 
should also be faster due to the performance upgrade of the database.
 
To find the issue, i'd be looking for clients who isnt behaving naturally. 
Check your timings (which clients aren't performing aswell as pre-upgrade) 
since that's usually the reason.
 
My 5 cents worth.
 
Best Regards



Daniel Sparrman
Exist i Stockholm AB
Växel: 08-754 98 00
Fax: 08-754 97 30
daniel.sparr...@exist.se
http://www.existgruppen.se
Posthusgatan 1 761 30 NORRTÄLJE



-ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU skrev: -


Till: ADSM-L@VM.MARIST.EDU
Från: Prather, Wanda wprat...@icfi.com
Sänt av: ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU
Datum: 05/15/2011 22:17
Ärende: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Am I correct that scheduling of background processes is important as well?
Don't expiration, migration, backup stgpool also create log transactions?
I'm wondering if it is the housekeeping, rather than the number of client 
backups, that is sending my server over the edge.

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Zoltan 
Forray/AC/VCU
Sent: Friday, May 13, 2011 3:50 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code

When I went through this with Andy R, the only solutions we came up with
was:

1.  Redistribute the workload/schedules to smooth out the spikes of number of 
nodes connecting to perform backups 2.  Redistributed / moved nodes to another 
TSM server 3.  In some cases where we were able to identify nodes causing the 
pinning, which was usually Oracle TDP backups, we made changes to break up the 
backups into multiple smaller transactions/chunks

This eventually smoothed out the stair-stepping of the transaction load 
against the logs.

This basically accelerated / gave me an excuse to move nodes to my 6.1.4 server 
and any new nodes are create there (or our 6.2.x servers).
Zoltan Forray
TSM Software  Hardware Administrator
Virginia Commonwealth University
UCC/Office of Technology Services
zfor...@vcu.edu - 804-828-4807
Don't be a phishing victim - VCU and other reputable organizations will never 
use email to request that you reply with your password, social security number 
or confidential personal information. For more details visit 
http://infosecurity.vcu.edu/phishing.html



From:
Meadows Andrew andrew.mead...@hcahealthcare.com
To:
ADSM-L@VM.MARIST.EDU
Date:
05/13/2011 03:24 PM
Subject:
Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code Sent by:
ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



We have seen this for quite a while with our TSM servers/Clients. The only 
thing we have found that works as a work around is to point our clients to back 
up directly to tape. If you are able to find a resolution to this issue please 
include me on the resolution as well as I would rather not write backup data 
directly to tape during backups.

Thanks,

Andrew

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Dave 
Canan
Sent: Friday, May 13, 2011 10:52 AM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Andy Raibeck and I have been monitoring this thread for the past few weeks and 
have been having numerous discussions about what we need to do to address these 
problems. In addition, we have noted the many PMRs coming into the IBM support 
queue, and many of these PMRs are now on my queue. Andy will be assisting me 
along with other support people as needed.

As part of this analysis we will be doing, we have a new questionnaire (titled 
Items to Gather for a PMR with Log Pinning Condition for TSM V5) that 
customers will be filling out that ask several questions regarding your 
environment. We understand that this involves extra work to gather this 
information, but there can be many different areas that can cause log pinning 
conditions, so this information is needed. In addition, there will be a script 
provided (named serverperf5a.pl) that will help us gather additional data. Both 
of these will be provided to you by support. When these PMRs are now opened, 
please make sure that level 1 support adds the keyword LOGPIN55 to the PMR. 
This will allow Andy and I to quickly find all the PMRs being worked for this 
issue.

Eric, your PMR is now one that I have on my queue (or I will shortly today).
We will be contacting you to work the PMR.

Dave Canan
IBM ATS TSM Performance
ddcananATUSDOTIBMDOTCOM
916-723-2410

On Thu, May 12, 2011 at 7:53 AM, Loon, EJ van - SPLXO eric-van.l...@klm.com
 wrote:

 Hi Rick!
 You are running in normal mode. In this mode the recovery log only 
 contains uncommited transactions

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Cheung, Richard]

Hi

The worry about this is I had a PMR opened for this issue on my servers, which 
are currently at 5.5.4. I was advised it's a known issue, refer to APAR 
IC64438, and the recommended fix is to go to 5.5.5.0

So is 5.5.5.0 also having this issue with the log pinning?

I also believed this was happening in early versions of TSM v6.. But the later 
builds are okay - can anyone confirm?


Richard Cheung | Server Engineer | Information Systems | Santos Limited
Santos Centre | 60 Flinders Street Adelaide SA 5000 Australia
Ph +61 8 8116 7125 | 7 Fax +61 8 8116 5073 |   richard.che...@santos.com


-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Dave 
Canan
Sent: Saturday, 14 May 2011 1:22 AM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Andy Raibeck and I have been monitoring this thread for the past few weeks and 
have been having numerous discussions about what we need to do to address these 
problems. In addition, we have noted the many PMRs coming into the IBM support 
queue, and many of these PMRs are now on my queue. Andy will be assisting me 
along with other support people as needed.

As part of this analysis we will be doing, we have a new questionnaire (titled 
Items to Gather for a PMR with Log Pinning Condition for TSM V5) that 
customers will be filling out that ask several questions regarding your 
environment. We understand that this involves extra work to gather this 
information, but there can be many different areas that can cause log pinning 
conditions, so this information is needed. In addition, there will be a script 
provided (named serverperf5a.pl) that will help us gather additional data. Both 
of these will be provided to you by support. When these PMRs are now opened, 
please make sure that level 1 support adds the keyword LOGPIN55 to the PMR. 
This will allow Andy and I to quickly find all the PMRs being worked for this 
issue.

Eric, your PMR is now one that I have on my queue (or I will shortly today).
We will be contacting you to work the PMR.

Dave Canan
IBM ATS TSM Performance
ddcananATUSDOTIBMDOTCOM
916-723-2410

On Thu, May 12, 2011 at 7:53 AM, Loon, EJ van - SPLXO eric-van.l...@klm.com
 wrote:

 Hi Rick!
 You are running in normal mode. In this mode the recovery log only
 contains uncommited transactions. Don't you agree that in this case,
 the log should NEVER reach 90%? It should not even reach something like 40%!
 I don't want to have to implement extra monitoring to keep TSM from
 crashing. It should just work. Like it did when we were running 5.3...
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf
 Of Richard Rhodes
 Sent: donderdag 12 mei 2011 15:04
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 I have a cron job that checks log utilization every 15m.  If the log
 goes over 90% the scripts displays a bunch of stuff (q sess, q proc, q
 log, show logpin, etc),  runs a show logpin cancel,  displays the
 stuff again,
 and finally emails me. (We run in normal mode.)   It's been very
 effective
 in preventing a log full condition.

 rick






 From:   Bos, Karel karel@atosorigin.com
 To: ADSM-L@VM.MARIST.EDU
 Date:   05/12/2011 06:11 AM
 Subject:Re: TSM Recovery log is pinning since upgrade to 5.5.5.0
 code
 Sent by:ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



 Hi,

 Dunno if u already looked at these options: THROUGHPUTDATATHRESHOLD
 and THROUGHPUTTIMETHRESHOLD. After a couple of crashes due to log
 filling up, a couple of years back, I standard added them to all my
 clients (with low values to only kill the slowest/hung ones) and
 untill now never had any issues with the V5.5 and below log space
 under normal backup loads.

 Log filling only happend when some sys admin made some changes to the
 biggest file servers we had but even this was handled like it should
 be done by ITSM.

 Regards,

 Karel



 
 Van: ADSM: Dist Stor Manager [ADSM-L@VM.MARIST.EDU] namens Loon, EJ
 van
 -
 SPLXO [eric-van.l...@klm.com]
 Verzonden: donderdag 12 mei 2011 11:11
 Aan: ADSM-L@VM.MARIST.EDU
 Onderwerp: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0
 code

 Hi Paul!
 We are already running 5.5.5.2 and the log is still filling up, even
 after switching from rollforward to normal mode.
 Management currently is questioning whether TSM is the right product
 for the future. Although I'm a big fan of TSM for 15 years, I'm really
 in doubt too...
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf
 Of Paul Fielding
 Sent: woensdag 11 mei 2011 20:05
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 Hi

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Loon, EJ van - SPLXO]

Hi Richard!
The log only contains all transactions (since the last database backup)
when running in rollforward mode. When running in normal mode, only
uncommitted transactions are written to the log. As soon as a
transaction is completed, the transaction is committed to the database
and thus removed from the log.
I agree that a slow running client backing up very large files can pin
the log for a very long time, but the log only contains transaction
data. Per backup file TSM stores 600 bytes of data in the database, so a
12 Gb recovery should be able to hold quite a lot of uncommitted client
transactions!
When sending smaller files: as soon as the txnbytelimit is reached, the
server commits the transaction, so slow backup sessions sending smaller
files don't necessarily pin the log.
Anyway, I'm monitoring logpinning on regular intervals and there is
little logpinning except for occasional TDP for SQL nodes with very
large databases.
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Richard Rhodes
Sent: donderdag 12 mei 2011 17:23
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

No . . . It's completely possible for the the log in normal mode to be
pinned and hit 100%.  My understanding is that the log contains ALL
transactions.  It's the checkpoint/rollback for info for the db.  An
open
tran (uncommitted?) pins the log. We get them every once in a while
when some server sends a big file very slowly - usually a remote site,
but
sometimes a local with something set wrong (my favorite is the
half-duplex
ethernet connection which runs at 50k/s).  But it's not just the log
being
pinned by a long running tran that is a concern, it's also the load on
the
TSM server for all the other backups that are running and their
transactions that determine how quickly a pinned log fills up.

Here is the log consumption for the past day for six of our tsm servers:
  tsm1   3gb
  tsm2   2gb
  tsm3   8gb
  tsm4  10gb
  tsm5   4gb
  tsm6   6gb

They all have 12gb of log space.

Back a few years ago, tsm1 was consuming over 20gb per day and we had
tons
of logpin problems, one time resulting in the log filling completely.
This
was when I put in the monitoring with the auto logpin cancel.It was
one of the many indications that we needed more instances.  It's also
why
we have a daily report of slow and long running backups so we can
identify
these problem servers.

Rick










From:   Loon, EJ van - SPLXO eric-van.l...@klm.com
To: ADSM-L@VM.MARIST.EDU
Date:   05/12/2011 10:54 AM
Subject:Re: TSM Recovery log is pinning since upgrade to 5.5.5.0
code
Sent by:ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



Hi Rick!
You are running in normal mode. In this mode the recovery log only
contains uncommited transactions. Don't you agree that in this case, the
log should NEVER reach 90%? It should not even reach something like 40%!
I don't want to have to implement extra monitoring to keep TSM from
crashing. It should just work. Like it did when we were running 5.3...
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Richard Rhodes
Sent: donderdag 12 mei 2011 15:04
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

I have a cron job that checks log utilization every 15m.  If the log
goes
over 90% the scripts displays a bunch of stuff (q sess, q proc, q log,
show logpin, etc),  runs a show logpin cancel,  displays the stuff
again,
and finally emails me. (We run in normal mode.)   It's been very
effective
in preventing a log full condition.

rick






From:   Bos, Karel karel@atosorigin.com
To: ADSM-L@VM.MARIST.EDU
Date:   05/12/2011 06:11 AM
Subject:Re: TSM Recovery log is pinning since upgrade to 5.5.5.0
code
Sent by:ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



Hi,

Dunno if u already looked at these options: THROUGHPUTDATATHRESHOLD and
THROUGHPUTTIMETHRESHOLD. After a couple of crashes due to log filling
up,
a couple of years back, I standard added them to all my clients (with
low
values to only kill the slowest/hung ones) and untill now never had any
issues with the V5.5 and below log space under normal backup loads.

Log filling only happend when some sys admin made some changes to the
biggest file servers we had but even this was handled like it should be
done by ITSM.

Regards,

Karel




Van: ADSM: Dist Stor Manager [ADSM-L@VM.MARIST.EDU] namens Loon, EJ van
-
SPLXO [eric-van.l...@klm.com]
Verzonden: donderdag 12 mei 2011 11:11
Aan: ADSM-L@VM.MARIST.EDU
Onderwerp: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Hi Paul!
We are already running 5.5.5.2 and the log is still filling up, even
after switching from rollforward

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Richard Rhodes]

 Hi Richard!
 The log only contains all transactions (since the last database backup)
 when running in rollforward mode. When running in normal mode, only
 uncommitted transactions are written to the log. As soon as a
 transaction is completed, the transaction is committed to the database
 and thus removed from the log.

We're probably saying the same thing ;-) . . . here is how
I understand the log to work.

My understanding is that the log is a logical sequential-circular file
with a head  (where new trans are written) and tail(the oldest
uncommitted tran). As trans are created they are written
to the head (uncommitted).  As trans complete (commited),
commits are written to the log.   Thus tsm knows what to do after a
crash - needing everything from head to tail to put the db
back into a consistent state.  When replaying the log committed trans can
be ignored,
and uncommitted trans can be rolled back.
The space used by committed trans
between the head and tail is _not_ reusable.  Usable space
is what is in front of the head up to the tail. As the oldest
tran commits, the tail is pulled forward.  If any one  tran remains
uncommitted long enough for the head to come all the way around
and but into the tail . . . you filled the log.  In a sense,
committed trans are like expired files on tape that need
reclamation before the space is reusable.

I thought the difference going to roll-forward mode is that
the log tail is pinned at the last backup.  The tail is not
moved up until a backup has completed.   While is normal mode
the tail  moves forward as the oldest tran commits.

In a sense, the log is always pinned, but the pinning tran is always
changing as trans commit.  So I see logpin _problem_ as being caused by
two issues:
1)  Some long running (slow) tran keeping the tail pinned.
2)  The rate new trans are added and move the head ever forward toward the
tail.

That's my understanding how it works . . . which could be very wrong!

Rick




-
The information contained in this message is intended only for the
personal and confidential use of the recipient(s) named above. If
the reader of this message is not the intended recipient or an
agent responsible for delivering it to the intended recipient, you
are hereby notified that you have received this document in error
and that any review, dissemination, distribution, or copying of
this message is strictly prohibited. If you have received this
communication in error, please notify us immediately, and delete
the original message.

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Paul Fielding]

Hi Eric,

Indeed there may be something broken, but you have to understand that from
IBM's perspective, they have fixed it - you go to TSM 6.2.  5.5 will
eventually be removed from support (whether we like it or not) and then
you'll be really stuck.

The reason I ask the DB sizes is on topic.   I would suggest that the
published formulas for how long the TSM upgrade takes are inaccurate.  I
have performed a number of TSM 5.5 to 6.2 upgrades, some on Windows, some on
AIX.

The smallest one had a 140GB database, it took 8.5h from start to finish.
The biggest one had a 350GB database, it took 26h from start to finish.

There are several factors that will impact how long your upgrade will take,
not just db size.  The horsepower of the TSM 6.2 box (whether upgrading in
place or upgrading to a new server), and TSM 6.2 disk layout are two big
ones.

The single best thing you can do, rather than assuming your upgrade will
take 3 days, is to test it.  Setup a test server, restore the 5.5 db, then
upgrade it.   If doing a network upgrade to new box (ideal situation), then
setup your new TSM 6.2 server, setup a test 5.5 server, restore the 5.5 db,
then do the network upgrade from the test box to your 6.2 server.  My
clients (understandably) want to have a realistic idea of how long the
upgrade will take, so I've insisted on doing a test upgrade on every upgrade
I've done so far.  In each case my production upgrade came to within 1-2h of
the timing that the test took.

I guess what I'm saying is that you're going to have some pain no matter
what you do, but the pain to upgrade may not be as bad as you think, and the
potential payoff is big

regards,

Paul


On Thu, May 12, 2011 at 9:05 AM, Loon, EJ van - SPLXO eric-van.l...@klm.com
 wrote:

 Hi Paul!
 I have 3 servers with 86, 100 and 129 gb databases. I haven't tried
 migrating, but I read a document somewhere which contained some figures
 about what performance one can expect from the migration process. At
 that time I roughly calculated 3 and a half days.
 Anyway, let's stay on the subject here: there is something broken in TSM
 (remember, I'm not the only one struggling with the recovery log size)
 and IBM should seriously look in to it and fix it. I never had any
 problems with the log when we were using 5.3!
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
 Paul Fielding
 Sent: donderdag 12 mei 2011 15:53
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 A couple of questions, Eric

 - how big is your DB
 - have you tried a test upgrade to an alternate box to see how long it
 will
 take?

 Indeed, I agree that you're going to have to do something, whether it's
 take
 the outage to go to V6 or start to migrate to another product.  However,
 if
 you are in a position to move to another product, then (worst case
 scenario), you're probably in a position to move to a clean V6 server
 without migrating.   The one thing you really can't do is stay where you
 are
 forever, at least not with support

 Paul

 On Thu, May 12, 2011 at 7:38 AM, David E Ehresman
 deehr...@louisville.eduwrote:

  Eric,
 
  Pointing out the obvious here, but if you really can't afford to move
  to TSM v6 then its time for you to start planning your move to
 something
  else.  Support for TSM 5 will be dropped sometime and I suspect the
 move
  from TSM to something else will take some planning.
 
  David
 
   Loon, EJ van - SPLXO eric-van.l...@klm.com 5/12/2011 8:56 AM
  
  Hi Steve!
  If it would be possible, I would have already done that, but migrating
  our servers from 5.5 to 6.2 would require multiple days of downtime.
  Simple not acceptable in our shop...
  Kind regards,
  Eric van Loon
  KLM Royal Dutch Airlines
 
  -Original Message-
  From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf
  Of
  Paul Fielding
  Sent: donderdag 12 mei 2011 14:33
  To: ADSM-L@VM.MARIST.EDU
  Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code
 
  You could also go to TSM 6.2, which changes to DB2 and changes the way
  recovery logs are dealt with, including allowing for much, much more
  log
 
 
  On Thu, May 12, 2011 at 5:53 AM, Steve Roder s...@buffalo.edu wrote:
 
   What's pinning your log?  Do a show logpinned, and then look at what
   that client is doing.  If it is a TDP, they are known for pinning
  the
   log and causing these kinds of issues.
  
   We see this issue also, and it is nearly always caused by an Oracle
  TDP
   or an Exchange TDP backup.
  
   Thanks,
   Steve
  
  
Hi Paul!
   We are already running 5.5.5.2 and the log is still filling up,
  even
   after switching from rollforward to normal mode.
   Management currently is questioning whether TSM is the right
  product
  for
   the future. Although I'm a big fan of TSM for 15 years, I'm really
  in
   doubt too

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Dave Canan]

Andy Raibeck and I have been monitoring this thread for the past few weeks
and have been having numerous discussions about what we need to do to
address these problems. In addition, we have noted the many PMRs coming into
the IBM support queue, and many of these PMRs are now on my queue. Andy will
be assisting me along with other support people as needed.

As part of this analysis we will be doing, we have a new questionnaire
(titled Items to Gather for a PMR with Log Pinning Condition for TSM V5)
that customers will be filling out that ask several questions regarding your
environment. We understand that this involves extra work to gather this
information, but there can be many different areas that can cause log
pinning conditions, so this information is needed. In addition, there will
be a script provided (named serverperf5a.pl) that will help us gather
additional data. Both of these will be provided to you by support. When
these PMRs are now opened, please make sure that level 1 support adds the
keyword LOGPIN55 to the PMR. This will allow Andy and I to quickly find all
the PMRs being worked for this issue.

Eric, your PMR is now one that I have on my queue (or I will shortly today).
We will be contacting you to work the PMR.

Dave Canan
IBM ATS TSM Performance
ddcananATUSDOTIBMDOTCOM
916-723-2410

On Thu, May 12, 2011 at 7:53 AM, Loon, EJ van - SPLXO eric-van.l...@klm.com
 wrote:

 Hi Rick!
 You are running in normal mode. In this mode the recovery log only
 contains uncommited transactions. Don't you agree that in this case, the
 log should NEVER reach 90%? It should not even reach something like 40%!
 I don't want to have to implement extra monitoring to keep TSM from
 crashing. It should just work. Like it did when we were running 5.3...
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
 Richard Rhodes
 Sent: donderdag 12 mei 2011 15:04
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 I have a cron job that checks log utilization every 15m.  If the log
 goes
 over 90% the scripts displays a bunch of stuff (q sess, q proc, q log,
 show logpin, etc),  runs a show logpin cancel,  displays the stuff
 again,
 and finally emails me. (We run in normal mode.)   It's been very
 effective
 in preventing a log full condition.

 rick






 From:   Bos, Karel karel@atosorigin.com
 To: ADSM-L@VM.MARIST.EDU
 Date:   05/12/2011 06:11 AM
 Subject:Re: TSM Recovery log is pinning since upgrade to 5.5.5.0
 code
 Sent by:ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



 Hi,

 Dunno if u already looked at these options: THROUGHPUTDATATHRESHOLD and
 THROUGHPUTTIMETHRESHOLD. After a couple of crashes due to log filling
 up,
 a couple of years back, I standard added them to all my clients (with
 low
 values to only kill the slowest/hung ones) and untill now never had any
 issues with the V5.5 and below log space under normal backup loads.

 Log filling only happend when some sys admin made some changes to the
 biggest file servers we had but even this was handled like it should be
 done by ITSM.

 Regards,

 Karel



 
 Van: ADSM: Dist Stor Manager [ADSM-L@VM.MARIST.EDU] namens Loon, EJ van
 -
 SPLXO [eric-van.l...@klm.com]
 Verzonden: donderdag 12 mei 2011 11:11
 Aan: ADSM-L@VM.MARIST.EDU
 Onderwerp: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 Hi Paul!
 We are already running 5.5.5.2 and the log is still filling up, even
 after switching from rollforward to normal mode.
 Management currently is questioning whether TSM is the right product for
 the future. Although I'm a big fan of TSM for 15 years, I'm really in
 doubt too...
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
 Paul Fielding
 Sent: woensdag 11 mei 2011 20:05
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 Hi folks, if this has already been commented on, my apologies, I hvaen't
 been following closely but just noticed this thread.

 We were experiencing log pinned issues after upgrading to 5.5.0.0 code
 at
 one of my client sites.  What we were finding was that, occasionally,
 I'd
 get up in the morning and look at the server to see it completely bogged
 down with a huge number of backups still attempting to run, but nothing
 was
 moving - the log was at 84% (over it's trigger), and it had been firing
 off
 db backups for the last 4h to no avail, the log wasn't getting low
 enough.

 A key symptom was that in the actlog we were seeing messages to the
 effect
 of Recovery log is at 84%, transactions will be delayed by 3ms (or
 something like that).

 In opening a ticket, it was found there was an issue in the 5.5.0.0 code
 where, when the recovery log gets too

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Meadows Andrew]

We have seen this for quite a while with our TSM servers/Clients. The only 
thing we have found that works as a work around is to point our clients to back 
up directly to tape. If you are able to find a resolution to this issue please 
include me on the resolution as well as I would rather not write backup data 
directly to tape during backups.

Thanks,

Andrew

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Dave 
Canan
Sent: Friday, May 13, 2011 10:52 AM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Andy Raibeck and I have been monitoring this thread for the past few weeks
and have been having numerous discussions about what we need to do to
address these problems. In addition, we have noted the many PMRs coming into
the IBM support queue, and many of these PMRs are now on my queue. Andy will
be assisting me along with other support people as needed.

As part of this analysis we will be doing, we have a new questionnaire
(titled Items to Gather for a PMR with Log Pinning Condition for TSM V5)
that customers will be filling out that ask several questions regarding your
environment. We understand that this involves extra work to gather this
information, but there can be many different areas that can cause log
pinning conditions, so this information is needed. In addition, there will
be a script provided (named serverperf5a.pl) that will help us gather
additional data. Both of these will be provided to you by support. When
these PMRs are now opened, please make sure that level 1 support adds the
keyword LOGPIN55 to the PMR. This will allow Andy and I to quickly find all
the PMRs being worked for this issue.

Eric, your PMR is now one that I have on my queue (or I will shortly today).
We will be contacting you to work the PMR.

Dave Canan
IBM ATS TSM Performance
ddcananATUSDOTIBMDOTCOM
916-723-2410

On Thu, May 12, 2011 at 7:53 AM, Loon, EJ van - SPLXO eric-van.l...@klm.com
 wrote:

 Hi Rick!
 You are running in normal mode. In this mode the recovery log only
 contains uncommited transactions. Don't you agree that in this case, the
 log should NEVER reach 90%? It should not even reach something like 40%!
 I don't want to have to implement extra monitoring to keep TSM from
 crashing. It should just work. Like it did when we were running 5.3...
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
 Richard Rhodes
 Sent: donderdag 12 mei 2011 15:04
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 I have a cron job that checks log utilization every 15m.  If the log
 goes
 over 90% the scripts displays a bunch of stuff (q sess, q proc, q log,
 show logpin, etc),  runs a show logpin cancel,  displays the stuff
 again,
 and finally emails me. (We run in normal mode.)   It's been very
 effective
 in preventing a log full condition.

 rick






 From:   Bos, Karel karel@atosorigin.com
 To: ADSM-L@VM.MARIST.EDU
 Date:   05/12/2011 06:11 AM
 Subject:Re: TSM Recovery log is pinning since upgrade to 5.5.5.0
 code
 Sent by:ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



 Hi,

 Dunno if u already looked at these options: THROUGHPUTDATATHRESHOLD and
 THROUGHPUTTIMETHRESHOLD. After a couple of crashes due to log filling
 up,
 a couple of years back, I standard added them to all my clients (with
 low
 values to only kill the slowest/hung ones) and untill now never had any
 issues with the V5.5 and below log space under normal backup loads.

 Log filling only happend when some sys admin made some changes to the
 biggest file servers we had but even this was handled like it should be
 done by ITSM.

 Regards,

 Karel



 
 Van: ADSM: Dist Stor Manager [ADSM-L@VM.MARIST.EDU] namens Loon, EJ van
 -
 SPLXO [eric-van.l...@klm.com]
 Verzonden: donderdag 12 mei 2011 11:11
 Aan: ADSM-L@VM.MARIST.EDU
 Onderwerp: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 Hi Paul!
 We are already running 5.5.5.2 and the log is still filling up, even
 after switching from rollforward to normal mode.
 Management currently is questioning whether TSM is the right product for
 the future. Although I'm a big fan of TSM for 15 years, I'm really in
 doubt too...
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
 Paul Fielding
 Sent: woensdag 11 mei 2011 20:05
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 Hi folks, if this has already been commented on, my apologies, I hvaen't
 been following closely but just noticed this thread.

 We were experiencing log pinned issues after upgrading to 5.5.0.0 code
 at
 one of my client sites.  What we were finding

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Zoltan Forray/AC/VCU]

When I went through this with Andy R, the only solutions we came up with
was:

1.  Redistribute the workload/schedules to smooth out the spikes of number
of nodes connecting to perform backups
2.  Redistributed / moved nodes to another TSM server
3.  In some cases where we were able to identify nodes causing the
pinning, which was usually Oracle TDP backups, we made changes to break up
the backups into multiple smaller transactions/chunks

This eventually smoothed out the stair-stepping of the transaction load
against the logs.

This basically accelerated / gave me an excuse to move nodes to my 6.1.4
server and any new nodes are create there (or our 6.2.x servers).
Zoltan Forray
TSM Software  Hardware Administrator
Virginia Commonwealth University
UCC/Office of Technology Services
zfor...@vcu.edu - 804-828-4807
Don't be a phishing victim - VCU and other reputable organizations will
never use email to request that you reply with your password, social
security number or confidential personal information. For more details
visit http://infosecurity.vcu.edu/phishing.html



From:
Meadows Andrew andrew.mead...@hcahealthcare.com
To:
ADSM-L@VM.MARIST.EDU
Date:
05/13/2011 03:24 PM
Subject:
Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code
Sent by:
ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



We have seen this for quite a while with our TSM servers/Clients. The only
thing we have found that works as a work around is to point our clients to
back up directly to tape. If you are able to find a resolution to this
issue please include me on the resolution as well as I would rather not
write backup data directly to tape during backups.

Thanks,

Andrew

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Dave Canan
Sent: Friday, May 13, 2011 10:52 AM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0
code

Andy Raibeck and I have been monitoring this thread for the past few weeks
and have been having numerous discussions about what we need to do to
address these problems. In addition, we have noted the many PMRs coming
into
the IBM support queue, and many of these PMRs are now on my queue. Andy
will
be assisting me along with other support people as needed.

As part of this analysis we will be doing, we have a new questionnaire
(titled Items to Gather for a PMR with Log Pinning Condition for TSM V5)
that customers will be filling out that ask several questions regarding
your
environment. We understand that this involves extra work to gather this
information, but there can be many different areas that can cause log
pinning conditions, so this information is needed. In addition, there will
be a script provided (named serverperf5a.pl) that will help us gather
additional data. Both of these will be provided to you by support. When
these PMRs are now opened, please make sure that level 1 support adds the
keyword LOGPIN55 to the PMR. This will allow Andy and I to quickly find
all
the PMRs being worked for this issue.

Eric, your PMR is now one that I have on my queue (or I will shortly
today).
We will be contacting you to work the PMR.

Dave Canan
IBM ATS TSM Performance
ddcananATUSDOTIBMDOTCOM
916-723-2410

On Thu, May 12, 2011 at 7:53 AM, Loon, EJ van - SPLXO
eric-van.l...@klm.com
 wrote:

 Hi Rick!
 You are running in normal mode. In this mode the recovery log only
 contains uncommited transactions. Don't you agree that in this case, the
 log should NEVER reach 90%? It should not even reach something like 40%!
 I don't want to have to implement extra monitoring to keep TSM from
 crashing. It should just work. Like it did when we were running 5.3...
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
 Richard Rhodes
 Sent: donderdag 12 mei 2011 15:04
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 I have a cron job that checks log utilization every 15m.  If the log
 goes
 over 90% the scripts displays a bunch of stuff (q sess, q proc, q log,
 show logpin, etc),  runs a show logpin cancel,  displays the stuff
 again,
 and finally emails me. (We run in normal mode.)   It's been very
 effective
 in preventing a log full condition.

 rick






 From:   Bos, Karel karel@atosorigin.com
 To: ADSM-L@VM.MARIST.EDU
 Date:   05/12/2011 06:11 AM
 Subject:Re: TSM Recovery log is pinning since upgrade to 5.5.5.0
 code
 Sent by:ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



 Hi,

 Dunno if u already looked at these options: THROUGHPUTDATATHRESHOLD and
 THROUGHPUTTIMETHRESHOLD. After a couple of crashes due to log filling
 up,
 a couple of years back, I standard added them to all my clients (with
 low
 values to only kill the slowest/hung ones) and untill now never had any
 issues with the V5.5 and below log space under normal backup loads

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Loon, EJ van - SPLXO]

Hi Paul!
We are already running 5.5.5.2 and the log is still filling up, even
after switching from rollforward to normal mode. 
Management currently is questioning whether TSM is the right product for
the future. Although I'm a big fan of TSM for 15 years, I'm really in
doubt too...
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Paul Fielding
Sent: woensdag 11 mei 2011 20:05
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Hi folks, if this has already been commented on, my apologies, I hvaen't
been following closely but just noticed this thread.

We were experiencing log pinned issues after upgrading to 5.5.0.0 code
at
one of my client sites.  What we were finding was that, occasionally,
I'd
get up in the morning and look at the server to see it completely bogged
down with a huge number of backups still attempting to run, but nothing
was
moving - the log was at 84% (over it's trigger), and it had been firing
off
db backups for the last 4h to no avail, the log wasn't getting low
enough.

A key symptom was that in the actlog we were seeing messages to the
effect
of Recovery log is at 84%, transactions will be delayed by 3ms (or
something like that).

In opening a ticket, it was found there was an issue in the 5.5.0.0 code
where, when the recovery log gets too full, it would start delaying
transactions in order to prevent the log from filling too quickly.
However,
the side effect was that the pinning transaction would also get delayed,
causing a bit of a never-ending loop.  Transactions keep getting
delayed,
pinned transaction never gets to finish, and everything would just grind
to
a halt.  I would have to halt the server and restart in order to get
things
back to normal.

IBM recognized this as an issue and recommended going to any 5.5.5.0
level
of code, where the problem was supposed to be fixed.  I installed
5.5.5.2,
and the problem has indeed gone away.   It was supposed to be fixed at
5.5.5.0, though perhaps they didn't quite get it done at that code level
as
they hoped?  I'd try installing 5.5.5.2 and see what happens

regards,

Paul


On Wed, May 11, 2011 at 9:03 AM, Loon, EJ van - SPLXO
eric-van.l...@klm.com
 wrote:

 Hi Robert!
 Thanks you very much for your reply! Several others on this list
 reported this behavior and (as far as I know) three other users opened
a
 PMR too. I hope they have more luck, because I'm stuck. Level 2 keeps
on
 saying that the log keeps on growing because of slow running client
 sessions. Indeed I see slow running client sessions, but they are
slowed
 down by the fact that TSM is delaying all transactions because the log
 is used for more that 80% during a large part of the backup window!
Now
 they refuse to help me, unless I buy a Passport Advantage
contract!!
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf
Of
 Robert Clark
 Sent: woensdag 11 mei 2011 16:05
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 I believe we've been seeing this problem as well.

 One night in the busiest backup period, I issued a q actlog
 begint=now-00:30, and got no results back but an error.

 I started dsmadmc -console on that server, and could see that the
 console
 output was most of an hour behind. (And the console output was
scrolling
 so fast that it could barely be read.)

 In that case, I think we determined that SQL LiteSpeed was set to some
 rediculously small transaction size, and this was causing way too many
 actlog entries.

 I think I also noted that the session number incremented by something
 like
 100,000 in an hour.

 Asking the users of SQL LiteSpeed to make some changes was enough to
 rememdy this problem, although we continue to fight with the logs
 getting
 full.

 Thanks,
 [RC]



 From:   Loon, EJ van - SPLXO eric-van.l...@klm.com
 To: ADSM-L@VM.MARIST.EDU
 Date:   05/11/2011 02:05 AM
 Subject:Re: [ADSM-L] TSM Recovery log is pinning since upgrade
 to
 5.5.5.0 code
 Sent by:ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



 Hi TSM-ers!
 Here is a small follow up on my PMR about the recovery log
utilization.
 I'm at TSM level 2, still trying to convince them that there is
 something broken in the TSM server code. To convince them, I have
 changed the logmode to normal on one of my servers. I created a graph
 (through TSMManager) which shows the recovery log utilization during
 last night's client backup window and is doesn't differ much from the
 night before, with logmode rollforward. When running in normal mode,
TSM
 should only use the recovery log for uncommitted transactions, so
 utilization should be very low. My log is 12 Gb and the backuptrigger
 value (75%) was still hit twice!
 This clearly shows that there is something

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Grigori Solonovitch]

Have you tried to resolve problem by SETOPT LOGWARNFULLPERCENT 95 to avoid 
Recovery log is at 84%, transactions will be delayed by 3ms for some time.
Maybe some slow sessions will be completed till log utilization reaches 95%?

Grigori G. Solonovitch


-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Loon, 
EJ van - SPLXO
Sent: Thursday, May 12, 2011 12:11 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Hi Paul!
We are already running 5.5.5.2 and the log is still filling up, even
after switching from rollforward to normal mode.
Management currently is questioning whether TSM is the right product for
the future. Although I'm a big fan of TSM for 15 years, I'm really in
doubt too...
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Paul Fielding
Sent: woensdag 11 mei 2011 20:05
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Hi folks, if this has already been commented on, my apologies, I hvaen't
been following closely but just noticed this thread.

We were experiencing log pinned issues after upgrading to 5.5.0.0 code
at
one of my client sites.  What we were finding was that, occasionally,
I'd
get up in the morning and look at the server to see it completely bogged
down with a huge number of backups still attempting to run, but nothing
was
moving - the log was at 84% (over it's trigger), and it had been firing
off
db backups for the last 4h to no avail, the log wasn't getting low
enough.

A key symptom was that in the actlog we were seeing messages to the
effect
of Recovery log is at 84%, transactions will be delayed by 3ms (or
something like that).

In opening a ticket, it was found there was an issue in the 5.5.0.0 code
where, when the recovery log gets too full, it would start delaying
transactions in order to prevent the log from filling too quickly.
However,
the side effect was that the pinning transaction would also get delayed,
causing a bit of a never-ending loop.  Transactions keep getting
delayed,
pinned transaction never gets to finish, and everything would just grind
to
a halt.  I would have to halt the server and restart in order to get
things
back to normal.

IBM recognized this as an issue and recommended going to any 5.5.5.0
level
of code, where the problem was supposed to be fixed.  I installed
5.5.5.2,
and the problem has indeed gone away.   It was supposed to be fixed at
5.5.5.0, though perhaps they didn't quite get it done at that code level
as
they hoped?  I'd try installing 5.5.5.2 and see what happens

regards,

Paul


On Wed, May 11, 2011 at 9:03 AM, Loon, EJ van - SPLXO
eric-van.l...@klm.com
 wrote:

 Hi Robert!
 Thanks you very much for your reply! Several others on this list
 reported this behavior and (as far as I know) three other users opened
a
 PMR too. I hope they have more luck, because I'm stuck. Level 2 keeps
on
 saying that the log keeps on growing because of slow running client
 sessions. Indeed I see slow running client sessions, but they are
slowed
 down by the fact that TSM is delaying all transactions because the log
 is used for more that 80% during a large part of the backup window!
Now
 they refuse to help me, unless I buy a Passport Advantage
contract!!
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf
Of
 Robert Clark
 Sent: woensdag 11 mei 2011 16:05
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 I believe we've been seeing this problem as well.

 One night in the busiest backup period, I issued a q actlog
 begint=now-00:30, and got no results back but an error.

 I started dsmadmc -console on that server, and could see that the
 console
 output was most of an hour behind. (And the console output was
scrolling
 so fast that it could barely be read.)

 In that case, I think we determined that SQL LiteSpeed was set to some
 rediculously small transaction size, and this was causing way too many
 actlog entries.

 I think I also noted that the session number incremented by something
 like
 100,000 in an hour.

 Asking the users of SQL LiteSpeed to make some changes was enough to
 rememdy this problem, although we continue to fight with the logs
 getting
 full.

 Thanks,
 [RC]



 From:   Loon, EJ van - SPLXO eric-van.l...@klm.com
 To: ADSM-L@VM.MARIST.EDU
 Date:   05/11/2011 02:05 AM
 Subject:Re: [ADSM-L] TSM Recovery log is pinning since upgrade
 to
 5.5.5.0 code
 Sent by:ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



 Hi TSM-ers!
 Here is a small follow up on my PMR about the recovery log
utilization.
 I'm at TSM level 2, still trying to convince them that there is
 something broken in the TSM server code

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Loon, EJ van - SPLXO]

Hi Grigori!
The log is filling up very rapidly. The fact that TSM starts delaying
transactions when reaching 80% prevents the log from filling up to 100%
most of the time. If I would change LOGWARNFULLPERCENT to an even higher
percentage, the 100% will be reached even faster. During the weekend,
when a lot of database clients are making full backups, TSM desperately
tries to lower the log by triggering incremental after incremental, but
it happened several times that the log filled up to 100% and I was paged
to fix it from home...
Yesterday we did some extensive LAN performance testing, to see if the
LAN adapter of the host is the bottleneck: we measured 128 Mbyte/sec, so
no bottleneck here!
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Grigori Solonovitch
Sent: donderdag 12 mei 2011 11:39
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Have you tried to resolve problem by SETOPT LOGWARNFULLPERCENT 95 to
avoid Recovery log is at 84%, transactions will be delayed by 3ms for
some time.
Maybe some slow sessions will be completed till log utilization reaches
95%?

Grigori G. Solonovitch


-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Loon, EJ van - SPLXO
Sent: Thursday, May 12, 2011 12:11 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to
5.5.5.0 code

Hi Paul!
We are already running 5.5.5.2 and the log is still filling up, even
after switching from rollforward to normal mode.
Management currently is questioning whether TSM is the right product for
the future. Although I'm a big fan of TSM for 15 years, I'm really in
doubt too...
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Paul Fielding
Sent: woensdag 11 mei 2011 20:05
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Hi folks, if this has already been commented on, my apologies, I hvaen't
been following closely but just noticed this thread.

We were experiencing log pinned issues after upgrading to 5.5.0.0 code
at
one of my client sites.  What we were finding was that, occasionally,
I'd
get up in the morning and look at the server to see it completely bogged
down with a huge number of backups still attempting to run, but nothing
was
moving - the log was at 84% (over it's trigger), and it had been firing
off
db backups for the last 4h to no avail, the log wasn't getting low
enough.

A key symptom was that in the actlog we were seeing messages to the
effect
of Recovery log is at 84%, transactions will be delayed by 3ms (or
something like that).

In opening a ticket, it was found there was an issue in the 5.5.0.0 code
where, when the recovery log gets too full, it would start delaying
transactions in order to prevent the log from filling too quickly.
However,
the side effect was that the pinning transaction would also get delayed,
causing a bit of a never-ending loop.  Transactions keep getting
delayed,
pinned transaction never gets to finish, and everything would just grind
to
a halt.  I would have to halt the server and restart in order to get
things
back to normal.

IBM recognized this as an issue and recommended going to any 5.5.5.0
level
of code, where the problem was supposed to be fixed.  I installed
5.5.5.2,
and the problem has indeed gone away.   It was supposed to be fixed at
5.5.5.0, though perhaps they didn't quite get it done at that code level
as
they hoped?  I'd try installing 5.5.5.2 and see what happens

regards,

Paul


On Wed, May 11, 2011 at 9:03 AM, Loon, EJ van - SPLXO
eric-van.l...@klm.com
 wrote:

 Hi Robert!
 Thanks you very much for your reply! Several others on this list
 reported this behavior and (as far as I know) three other users opened
a
 PMR too. I hope they have more luck, because I'm stuck. Level 2 keeps
on
 saying that the log keeps on growing because of slow running client
 sessions. Indeed I see slow running client sessions, but they are
slowed
 down by the fact that TSM is delaying all transactions because the log
 is used for more that 80% during a large part of the backup window!
Now
 they refuse to help me, unless I buy a Passport Advantage
contract!!
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf
Of
 Robert Clark
 Sent: woensdag 11 mei 2011 16:05
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 I believe we've been seeing this problem as well.

 One night in the busiest backup period, I issued a q actlog
 begint=now-00:30, and got no results back but an error.

 I started dsmadmc -console on that server, and could see that the
 console
 output

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Bos, Karel]

Hi,

Dunno if u already looked at these options: THROUGHPUTDATATHRESHOLD and 
THROUGHPUTTIMETHRESHOLD. After a couple of crashes due to log filling up, a 
couple of years back, I standard added them to all my clients (with low values 
to only kill the slowest/hung ones) and untill now never had any issues with 
the V5.5 and below log space under normal backup loads. 

Log filling only happend when some sys admin made some changes to the biggest 
file servers we had but even this was handled like it should be done by ITSM.

Regards,

Karel




Van: ADSM: Dist Stor Manager [ADSM-L@VM.MARIST.EDU] namens Loon, EJ van - SPLXO 
[eric-van.l...@klm.com]
Verzonden: donderdag 12 mei 2011 11:11
Aan: ADSM-L@VM.MARIST.EDU
Onderwerp: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Hi Paul!
We are already running 5.5.5.2 and the log is still filling up, even
after switching from rollforward to normal mode.
Management currently is questioning whether TSM is the right product for
the future. Although I'm a big fan of TSM for 15 years, I'm really in
doubt too...
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Paul Fielding
Sent: woensdag 11 mei 2011 20:05
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Hi folks, if this has already been commented on, my apologies, I hvaen't
been following closely but just noticed this thread.

We were experiencing log pinned issues after upgrading to 5.5.0.0 code
at
one of my client sites.  What we were finding was that, occasionally,
I'd
get up in the morning and look at the server to see it completely bogged
down with a huge number of backups still attempting to run, but nothing
was
moving - the log was at 84% (over it's trigger), and it had been firing
off
db backups for the last 4h to no avail, the log wasn't getting low
enough.

A key symptom was that in the actlog we were seeing messages to the
effect
of Recovery log is at 84%, transactions will be delayed by 3ms (or
something like that).

In opening a ticket, it was found there was an issue in the 5.5.0.0 code
where, when the recovery log gets too full, it would start delaying
transactions in order to prevent the log from filling too quickly.
However,
the side effect was that the pinning transaction would also get delayed,
causing a bit of a never-ending loop.  Transactions keep getting
delayed,
pinned transaction never gets to finish, and everything would just grind
to
a halt.  I would have to halt the server and restart in order to get
things
back to normal.

IBM recognized this as an issue and recommended going to any 5.5.5.0
level
of code, where the problem was supposed to be fixed.  I installed
5.5.5.2,
and the problem has indeed gone away.   It was supposed to be fixed at
5.5.5.0, though perhaps they didn't quite get it done at that code level
as
they hoped?  I'd try installing 5.5.5.2 and see what happens

regards,

Paul


On Wed, May 11, 2011 at 9:03 AM, Loon, EJ van - SPLXO
eric-van.l...@klm.com
 wrote:

 Hi Robert!
 Thanks you very much for your reply! Several others on this list
 reported this behavior and (as far as I know) three other users opened
a
 PMR too. I hope they have more luck, because I'm stuck. Level 2 keeps
on
 saying that the log keeps on growing because of slow running client
 sessions. Indeed I see slow running client sessions, but they are
slowed
 down by the fact that TSM is delaying all transactions because the log
 is used for more that 80% during a large part of the backup window!
Now
 they refuse to help me, unless I buy a Passport Advantage
contract!!
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf
Of
 Robert Clark
 Sent: woensdag 11 mei 2011 16:05
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 I believe we've been seeing this problem as well.

 One night in the busiest backup period, I issued a q actlog
 begint=now-00:30, and got no results back but an error.

 I started dsmadmc -console on that server, and could see that the
 console
 output was most of an hour behind. (And the console output was
scrolling
 so fast that it could barely be read.)

 In that case, I think we determined that SQL LiteSpeed was set to some
 rediculously small transaction size, and this was causing way too many
 actlog entries.

 I think I also noted that the session number incremented by something
 like
 100,000 in an hour.

 Asking the users of SQL LiteSpeed to make some changes was enough to
 rememdy this problem, although we continue to fight with the logs
 getting
 full.

 Thanks,
 [RC]



 From:   Loon, EJ van - SPLXO eric-van.l...@klm.com
 To: ADSM-L@VM.MARIST.EDU
 Date:   05/11/2011 02:05 AM
 Subject:Re: [ADSM-L] TSM Recovery

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Steve Roder]

What's pinning your log?  Do a show logpinned, and then look at what
that client is doing.  If it is a TDP, they are known for pinning the
log and causing these kinds of issues.

We see this issue also, and it is nearly always caused by an Oracle TDP
or an Exchange TDP backup.

Thanks,
Steve


Hi Paul!
We are already running 5.5.5.2 and the log is still filling up, even
after switching from rollforward to normal mode.
Management currently is questioning whether TSM is the right product for
the future. Although I'm a big fan of TSM for 15 years, I'm really in
doubt too...
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Paul Fielding
Sent: woensdag 11 mei 2011 20:05
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Hi folks, if this has already been commented on, my apologies, I hvaen't
been following closely but just noticed this thread.

We were experiencing log pinned issues after upgrading to 5.5.0.0 code
at
one of my client sites.  What we were finding was that, occasionally,
I'd
get up in the morning and look at the server to see it completely bogged
down with a huge number of backups still attempting to run, but nothing
was
moving - the log was at 84% (over it's trigger), and it had been firing
off
db backups for the last 4h to no avail, the log wasn't getting low
enough.

A key symptom was that in the actlog we were seeing messages to the
effect
of Recovery log is at 84%, transactions will be delayed by 3ms (or
something like that).

In opening a ticket, it was found there was an issue in the 5.5.0.0 code
where, when the recovery log gets too full, it would start delaying
transactions in order to prevent the log from filling too quickly.
However,
the side effect was that the pinning transaction would also get delayed,
causing a bit of a never-ending loop.  Transactions keep getting
delayed,
pinned transaction never gets to finish, and everything would just grind
to
a halt.  I would have to halt the server and restart in order to get
things
back to normal.

IBM recognized this as an issue and recommended going to any 5.5.5.0
level
of code, where the problem was supposed to be fixed.  I installed
5.5.5.2,
and the problem has indeed gone away.   It was supposed to be fixed at
5.5.5.0, though perhaps they didn't quite get it done at that code level
as
they hoped?  I'd try installing 5.5.5.2 and see what happens

regards,

Paul


On Wed, May 11, 2011 at 9:03 AM, Loon, EJ van - SPLXO
eric-van.l...@klm.com

wrote:
Hi Robert!
Thanks you very much for your reply! Several others on this list
reported this behavior and (as far as I know) three other users opened

a

PMR too. I hope they have more luck, because I'm stuck. Level 2 keeps

on

saying that the log keeps on growing because of slow running client
sessions. Indeed I see slow running client sessions, but they are

slowed

down by the fact that TSM is delaying all transactions because the log
is used for more that 80% during a large part of the backup window!

Now

they refuse to help me, unless I buy a Passport Advantage

contract!!

Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf

Of

Robert Clark
Sent: woensdag 11 mei 2011 16:05
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

I believe we've been seeing this problem as well.

One night in the busiest backup period, I issued a q actlog
begint=now-00:30, and got no results back but an error.

I started dsmadmc -console on that server, and could see that the
console
output was most of an hour behind. (And the console output was

scrolling

so fast that it could barely be read.)

In that case, I think we determined that SQL LiteSpeed was set to some
rediculously small transaction size, and this was causing way too many
actlog entries.

I think I also noted that the session number incremented by something
like
100,000 in an hour.

Asking the users of SQL LiteSpeed to make some changes was enough to
rememdy this problem, although we continue to fight with the logs
getting
full.

Thanks,
[RC]



From:   Loon, EJ van - SPLXOeric-van.l...@klm.com
To: ADSM-L@VM.MARIST.EDU
Date:   05/11/2011 02:05 AM
Subject:Re: [ADSM-L] TSM Recovery log is pinning since upgrade
to
5.5.5.0 code
Sent by:ADSM: Dist Stor ManagerADSM-L@VM.MARIST.EDU



Hi TSM-ers!
Here is a small follow up on my PMR about the recovery log

utilization.

I'm at TSM level 2, still trying to convince them that there is
something broken in the TSM server code. To convince them, I have
changed the logmode to normal on one of my servers. I created a graph
(through TSMManager) which shows the recovery log utilization during
last night's client backup window and is doesn't differ much from the
night before, with logmode

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Paul Fielding]

You could also go to TSM 6.2, which changes to DB2 and changes the way
recovery logs are dealt with, including allowing for much, much more log


On Thu, May 12, 2011 at 5:53 AM, Steve Roder s...@buffalo.edu wrote:

 What's pinning your log?  Do a show logpinned, and then look at what
 that client is doing.  If it is a TDP, they are known for pinning the
 log and causing these kinds of issues.

 We see this issue also, and it is nearly always caused by an Oracle TDP
 or an Exchange TDP backup.

 Thanks,
 Steve


  Hi Paul!
 We are already running 5.5.5.2 and the log is still filling up, even
 after switching from rollforward to normal mode.
 Management currently is questioning whether TSM is the right product for
 the future. Although I'm a big fan of TSM for 15 years, I'm really in
 doubt too...
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
 Paul Fielding
 Sent: woensdag 11 mei 2011 20:05
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 Hi folks, if this has already been commented on, my apologies, I hvaen't
 been following closely but just noticed this thread.

 We were experiencing log pinned issues after upgrading to 5.5.0.0 code
 at
 one of my client sites.  What we were finding was that, occasionally,
 I'd
 get up in the morning and look at the server to see it completely bogged
 down with a huge number of backups still attempting to run, but nothing
 was
 moving - the log was at 84% (over it's trigger), and it had been firing
 off
 db backups for the last 4h to no avail, the log wasn't getting low
 enough.

 A key symptom was that in the actlog we were seeing messages to the
 effect
 of Recovery log is at 84%, transactions will be delayed by 3ms (or
 something like that).

 In opening a ticket, it was found there was an issue in the 5.5.0.0 code
 where, when the recovery log gets too full, it would start delaying
 transactions in order to prevent the log from filling too quickly.
 However,
 the side effect was that the pinning transaction would also get delayed,
 causing a bit of a never-ending loop.  Transactions keep getting
 delayed,
 pinned transaction never gets to finish, and everything would just grind
 to
 a halt.  I would have to halt the server and restart in order to get
 things
 back to normal.

 IBM recognized this as an issue and recommended going to any 5.5.5.0
 level
 of code, where the problem was supposed to be fixed.  I installed
 5.5.5.2,
 and the problem has indeed gone away.   It was supposed to be fixed at
 5.5.5.0, though perhaps they didn't quite get it done at that code level
 as
 they hoped?  I'd try installing 5.5.5.2 and see what happens

 regards,

 Paul


 On Wed, May 11, 2011 at 9:03 AM, Loon, EJ van - SPLXO
 eric-van.l...@klm.com

 wrote:
 Hi Robert!
 Thanks you very much for your reply! Several others on this list
 reported this behavior and (as far as I know) three other users opened

 a

 PMR too. I hope they have more luck, because I'm stuck. Level 2 keeps

 on

 saying that the log keeps on growing because of slow running client
 sessions. Indeed I see slow running client sessions, but they are

 slowed

 down by the fact that TSM is delaying all transactions because the log
 is used for more that 80% during a large part of the backup window!

 Now

 they refuse to help me, unless I buy a Passport Advantage

 contract!!

 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf

 Of

 Robert Clark
 Sent: woensdag 11 mei 2011 16:05
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 I believe we've been seeing this problem as well.

 One night in the busiest backup period, I issued a q actlog
 begint=now-00:30, and got no results back but an error.

 I started dsmadmc -console on that server, and could see that the
 console
 output was most of an hour behind. (And the console output was

 scrolling

 so fast that it could barely be read.)

 In that case, I think we determined that SQL LiteSpeed was set to some
 rediculously small transaction size, and this was causing way too many
 actlog entries.

 I think I also noted that the session number incremented by something
 like
 100,000 in an hour.

 Asking the users of SQL LiteSpeed to make some changes was enough to
 rememdy this problem, although we continue to fight with the logs
 getting
 full.

 Thanks,
 [RC]



 From:   Loon, EJ van - SPLXOeric-van.l...@klm.com
 To: ADSM-L@VM.MARIST.EDU
 Date:   05/11/2011 02:05 AM
 Subject:Re: [ADSM-L] TSM Recovery log is pinning since upgrade
 to
 5.5.5.0 code
 Sent by:ADSM: Dist Stor ManagerADSM-L@VM.MARIST.EDU



 Hi TSM-ers!
 Here is a small follow up on my PMR about the recovery log

 utilization.

 I'm at TSM level 2, still trying

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Loon, EJ van - SPLXO]

Hi Steve!
Nothing. I have a show logpinned command scheduled at regular intervals
and I only see a SQL client pop up every now and then and that's because
you cannot specify a backup file size in the SQL client. 
If you see a Oracle client pinning your log, they probably haven't
specified a backup piece size (I believe it's called MAXPIECESIZE if I
remember correctly) in RMAN.
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: Steve Roder [mailto:s...@buffalo.edu] 
Sent: donderdag 12 mei 2011 13:53
To: Dist Stor Manager
Cc: Loon, EJ van - SPLXO
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to
5.5.5.0 code

What's pinning your log?  Do a show logpinned, and then look at what 
that client is doing.  If it is a TDP, they are known for pinning the 
log and causing these kinds of issues.

We see this issue also, and it is nearly always caused by an Oracle TDP 
or an Exchange TDP backup.

Thanks,
Steve

 Hi Paul!
 We are already running 5.5.5.2 and the log is still filling up, even
 after switching from rollforward to normal mode.
 Management currently is questioning whether TSM is the right product
for
 the future. Although I'm a big fan of TSM for 15 years, I'm really in
 doubt too...
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf
Of
 Paul Fielding
 Sent: woensdag 11 mei 2011 20:05
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 Hi folks, if this has already been commented on, my apologies, I
hvaen't
 been following closely but just noticed this thread.

 We were experiencing log pinned issues after upgrading to 5.5.0.0 code
 at
 one of my client sites.  What we were finding was that, occasionally,
 I'd
 get up in the morning and look at the server to see it completely
bogged
 down with a huge number of backups still attempting to run, but
nothing
 was
 moving - the log was at 84% (over it's trigger), and it had been
firing
 off
 db backups for the last 4h to no avail, the log wasn't getting low
 enough.

 A key symptom was that in the actlog we were seeing messages to the
 effect
 of Recovery log is at 84%, transactions will be delayed by 3ms (or
 something like that).

 In opening a ticket, it was found there was an issue in the 5.5.0.0
code
 where, when the recovery log gets too full, it would start delaying
 transactions in order to prevent the log from filling too quickly.
 However,
 the side effect was that the pinning transaction would also get
delayed,
 causing a bit of a never-ending loop.  Transactions keep getting
 delayed,
 pinned transaction never gets to finish, and everything would just
grind
 to
 a halt.  I would have to halt the server and restart in order to get
 things
 back to normal.

 IBM recognized this as an issue and recommended going to any 5.5.5.0
 level
 of code, where the problem was supposed to be fixed.  I installed
 5.5.5.2,
 and the problem has indeed gone away.   It was supposed to be fixed at
 5.5.5.0, though perhaps they didn't quite get it done at that code
level
 as
 they hoped?  I'd try installing 5.5.5.2 and see what happens

 regards,

 Paul


 On Wed, May 11, 2011 at 9:03 AM, Loon, EJ van - SPLXO
 eric-van.l...@klm.com
 wrote:
 Hi Robert!
 Thanks you very much for your reply! Several others on this list
 reported this behavior and (as far as I know) three other users
opened
 a
 PMR too. I hope they have more luck, because I'm stuck. Level 2 keeps
 on
 saying that the log keeps on growing because of slow running client
 sessions. Indeed I see slow running client sessions, but they are
 slowed
 down by the fact that TSM is delaying all transactions because the
log
 is used for more that 80% during a large part of the backup window!
 Now
 they refuse to help me, unless I buy a Passport Advantage
 contract!!
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf
 Of
 Robert Clark
 Sent: woensdag 11 mei 2011 16:05
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0
code

 I believe we've been seeing this problem as well.

 One night in the busiest backup period, I issued a q actlog
 begint=now-00:30, and got no results back but an error.

 I started dsmadmc -console on that server, and could see that the
 console
 output was most of an hour behind. (And the console output was
 scrolling
 so fast that it could barely be read.)

 In that case, I think we determined that SQL LiteSpeed was set to
some
 rediculously small transaction size, and this was causing way too
many
 actlog entries.

 I think I also noted that the session number incremented by something
 like
 100,000 in an hour.

 Asking the users of SQL LiteSpeed to make some changes was enough to
 rememdy this problem, although we continue to fight

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Richard Rhodes]

I have a cron job that checks log utilization every 15m.  If the log  goes
over 90% the scripts displays a bunch of stuff (q sess, q proc, q log,
show logpin, etc),  runs a show logpin cancel,  displays the stuff again,
and finally emails me. (We run in normal mode.)   It's been very effective
in preventing a log full condition.

rick






From:   Bos, Karel karel@atosorigin.com
To: ADSM-L@VM.MARIST.EDU
Date:   05/12/2011 06:11 AM
Subject:Re: TSM Recovery log is pinning since upgrade to 5.5.5.0
code
Sent by:ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



Hi,

Dunno if u already looked at these options: THROUGHPUTDATATHRESHOLD and
THROUGHPUTTIMETHRESHOLD. After a couple of crashes due to log filling up,
a couple of years back, I standard added them to all my clients (with low
values to only kill the slowest/hung ones) and untill now never had any
issues with the V5.5 and below log space under normal backup loads.

Log filling only happend when some sys admin made some changes to the
biggest file servers we had but even this was handled like it should be
done by ITSM.

Regards,

Karel




Van: ADSM: Dist Stor Manager [ADSM-L@VM.MARIST.EDU] namens Loon, EJ van -
SPLXO [eric-van.l...@klm.com]
Verzonden: donderdag 12 mei 2011 11:11
Aan: ADSM-L@VM.MARIST.EDU
Onderwerp: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Hi Paul!
We are already running 5.5.5.2 and the log is still filling up, even
after switching from rollforward to normal mode.
Management currently is questioning whether TSM is the right product for
the future. Although I'm a big fan of TSM for 15 years, I'm really in
doubt too...
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Paul Fielding
Sent: woensdag 11 mei 2011 20:05
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Hi folks, if this has already been commented on, my apologies, I hvaen't
been following closely but just noticed this thread.

We were experiencing log pinned issues after upgrading to 5.5.0.0 code
at
one of my client sites.  What we were finding was that, occasionally,
I'd
get up in the morning and look at the server to see it completely bogged
down with a huge number of backups still attempting to run, but nothing
was
moving - the log was at 84% (over it's trigger), and it had been firing
off
db backups for the last 4h to no avail, the log wasn't getting low
enough.

A key symptom was that in the actlog we were seeing messages to the
effect
of Recovery log is at 84%, transactions will be delayed by 3ms (or
something like that).

In opening a ticket, it was found there was an issue in the 5.5.0.0 code
where, when the recovery log gets too full, it would start delaying
transactions in order to prevent the log from filling too quickly.
However,
the side effect was that the pinning transaction would also get delayed,
causing a bit of a never-ending loop.  Transactions keep getting
delayed,
pinned transaction never gets to finish, and everything would just grind
to
a halt.  I would have to halt the server and restart in order to get
things
back to normal.

IBM recognized this as an issue and recommended going to any 5.5.5.0
level
of code, where the problem was supposed to be fixed.  I installed
5.5.5.2,
and the problem has indeed gone away.   It was supposed to be fixed at
5.5.5.0, though perhaps they didn't quite get it done at that code level
as
they hoped?  I'd try installing 5.5.5.2 and see what happens

regards,

Paul


On Wed, May 11, 2011 at 9:03 AM, Loon, EJ van - SPLXO
eric-van.l...@klm.com
 wrote:

 Hi Robert!
 Thanks you very much for your reply! Several others on this list
 reported this behavior and (as far as I know) three other users opened
a
 PMR too. I hope they have more luck, because I'm stuck. Level 2 keeps
on
 saying that the log keeps on growing because of slow running client
 sessions. Indeed I see slow running client sessions, but they are
slowed
 down by the fact that TSM is delaying all transactions because the log
 is used for more that 80% during a large part of the backup window!
Now
 they refuse to help me, unless I buy a Passport Advantage
contract!!
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf
Of
 Robert Clark
 Sent: woensdag 11 mei 2011 16:05
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 I believe we've been seeing this problem as well.

 One night in the busiest backup period, I issued a q actlog
 begint=now-00:30, and got no results back but an error.

 I started dsmadmc -console on that server, and could see that the
 console
 output was most of an hour behind. (And the console output was
scrolling
 so fast that it could barely be read

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Loon, EJ van - SPLXO]

Hi Steve!
If it would be possible, I would have already done that, but migrating
our servers from 5.5 to 6.2 would require multiple days of downtime.
Simple not acceptable in our shop...
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Paul Fielding
Sent: donderdag 12 mei 2011 14:33
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

You could also go to TSM 6.2, which changes to DB2 and changes the way
recovery logs are dealt with, including allowing for much, much more
log


On Thu, May 12, 2011 at 5:53 AM, Steve Roder s...@buffalo.edu wrote:

 What's pinning your log?  Do a show logpinned, and then look at what
 that client is doing.  If it is a TDP, they are known for pinning the
 log and causing these kinds of issues.

 We see this issue also, and it is nearly always caused by an Oracle
TDP
 or an Exchange TDP backup.

 Thanks,
 Steve


  Hi Paul!
 We are already running 5.5.5.2 and the log is still filling up, even
 after switching from rollforward to normal mode.
 Management currently is questioning whether TSM is the right product
for
 the future. Although I'm a big fan of TSM for 15 years, I'm really in
 doubt too...
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf
Of
 Paul Fielding
 Sent: woensdag 11 mei 2011 20:05
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0
code

 Hi folks, if this has already been commented on, my apologies, I
hvaen't
 been following closely but just noticed this thread.

 We were experiencing log pinned issues after upgrading to 5.5.0.0
code
 at
 one of my client sites.  What we were finding was that, occasionally,
 I'd
 get up in the morning and look at the server to see it completely
bogged
 down with a huge number of backups still attempting to run, but
nothing
 was
 moving - the log was at 84% (over it's trigger), and it had been
firing
 off
 db backups for the last 4h to no avail, the log wasn't getting low
 enough.

 A key symptom was that in the actlog we were seeing messages to the
 effect
 of Recovery log is at 84%, transactions will be delayed by 3ms (or
 something like that).

 In opening a ticket, it was found there was an issue in the 5.5.0.0
code
 where, when the recovery log gets too full, it would start delaying
 transactions in order to prevent the log from filling too quickly.
 However,
 the side effect was that the pinning transaction would also get
delayed,
 causing a bit of a never-ending loop.  Transactions keep getting
 delayed,
 pinned transaction never gets to finish, and everything would just
grind
 to
 a halt.  I would have to halt the server and restart in order to get
 things
 back to normal.

 IBM recognized this as an issue and recommended going to any 5.5.5.0
 level
 of code, where the problem was supposed to be fixed.  I installed
 5.5.5.2,
 and the problem has indeed gone away.   It was supposed to be fixed
at
 5.5.5.0, though perhaps they didn't quite get it done at that code
level
 as
 they hoped?  I'd try installing 5.5.5.2 and see what happens

 regards,

 Paul


 On Wed, May 11, 2011 at 9:03 AM, Loon, EJ van - SPLXO
 eric-van.l...@klm.com

 wrote:
 Hi Robert!
 Thanks you very much for your reply! Several others on this list
 reported this behavior and (as far as I know) three other users
opened

 a

 PMR too. I hope they have more luck, because I'm stuck. Level 2
keeps

 on

 saying that the log keeps on growing because of slow running client
 sessions. Indeed I see slow running client sessions, but they are

 slowed

 down by the fact that TSM is delaying all transactions because the
log
 is used for more that 80% during a large part of the backup window!

 Now

 they refuse to help me, unless I buy a Passport Advantage

 contract!!

 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On
Behalf

 Of

 Robert Clark
 Sent: woensdag 11 mei 2011 16:05
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0
code

 I believe we've been seeing this problem as well.

 One night in the busiest backup period, I issued a q actlog
 begint=now-00:30, and got no results back but an error.

 I started dsmadmc -console on that server, and could see that the
 console
 output was most of an hour behind. (And the console output was

 scrolling

 so fast that it could barely be read.)

 In that case, I think we determined that SQL LiteSpeed was set to
some
 rediculously small transaction size, and this was causing way too
many
 actlog entries.

 I think I also noted that the session number incremented by
something
 like
 100,000 in an hour.

 Asking the users of SQL LiteSpeed to make some changes was enough

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Strand, Neil B.]

Eric,
Regarding TSM as being the right product.  You may want to assess it
as part of your entire storage infrastructure and the services this
environment provides to the business and end users.

   Last year I had an issue that almost resulted in my having to recover
several (~30) TB of user data (many millions of files).  During the
initial phase of the problem, we initiated a TSM recovery to an
alternate device, just to get the recovery started in the event we
couldn't resolve the issue.  What did save my bacon was the data being
mirrored to another device that I brought online.  We were able to
resolve the issue and everything was returned to normal.

   This event caused me to really look hard at our backups.  Had we
really needed to recover all of that data from tape, we would have been
down or partially operational for a week or so.  I am using 18 TS1120
drives in a 10 frame dual robot TS3500 library and IBM P550 TSM servers
with 4Gb SAN connections  - not cutting edge but still pretty good
stuff.

   Currently, I do not rely on tape backup for large scale recovery. It
can be accomplished, but the RTO (for large amounts of data) is
unacceptable for normal business operations.  For NAS data, I use
snapshots for short term RPO (2weeks) with mirroring to protect from
disaster.  Tape recovery fits in between these two scenarios to provide
recovery for older data (more than 2 weeks old) and for long term
archives (7 years).  Production devices with SAN or local storage are
mirrored using the OS or DB tools and also backed up but the tape backup
is a secondary recovery mechanism to the OS  DB tool recovery and also
provides for long term archive.

   In general, as we become more virtualized and RTO requirements shrink
to minutes, the amount of data to recover grows to terabytes and the
number of files to recover is rounded to the nearest 1/10 of a million,
tape recovery becomes ineffective for large recovery scenarios and the
only alternative is snapshot or mirror recovery methods.  Tape still
offers a cost effective long term archive solution and can provide a
bridge for recovering older data that is not in a snapshot.  It also
provides an alternative storage device to store data and should not be
affected by OS or application bugs or malicious software - kind of a
failsafe device (assuming the backup processes are successful).
Eventually, we may go to a cloud based backup solution if it proves to
be operationally and cost effective and secure.

Cheers,
Neil Strand
Storage Engineer - Legg Mason
Baltimore, MD.
(410) 580-7491
Whatever you can do or believe you can, begin it.
Boldness has genius, power and magic.


-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Loon, EJ van - SPLXO
Sent: Thursday, May 12, 2011 12:11 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to
5.5.5.0 code

Hi Paul!
We are already running 5.5.5.2 and the log is still filling up, even
after switching from rollforward to normal mode.
Management currently is questioning whether TSM is the right product for
the future. Although I'm a big fan of TSM for 15 years, I'm really in
doubt too...
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines


IMPORTANT:  E-mail sent through the Internet is not secure. Legg Mason 
therefore recommends that you do not send any confidential or sensitive 
information to us via electronic mail, including social security numbers, 
account numbers, or personal identification numbers. Delivery, and or timely 
delivery of Internet mail is not guaranteed. Legg Mason therefore recommends 
that you do not send time sensitive 
or action-oriented messages to us via electronic mail.

This message is intended for the addressee only and may contain privileged or 
confidential information. Unless you are the intended recipient, you may not 
use, copy or disclose to anyone any information contained in this message. If 
you have received this message in error, please notify the author by replying 
to this message and then kindly delete the message. Thank you.

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[David E Ehresman]

Eric,

Pointing out the obvious here, but if you really can't afford to move
to TSM v6 then its time for you to start planning your move to something
else.  Support for TSM 5 will be dropped sometime and I suspect the move
from TSM to something else will take some planning.

David

 Loon, EJ van - SPLXO eric-van.l...@klm.com 5/12/2011 8:56 AM

Hi Steve!
If it would be possible, I would have already done that, but migrating
our servers from 5.5 to 6.2 would require multiple days of downtime.
Simple not acceptable in our shop...
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf
Of
Paul Fielding
Sent: donderdag 12 mei 2011 14:33
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

You could also go to TSM 6.2, which changes to DB2 and changes the way
recovery logs are dealt with, including allowing for much, much more
log


On Thu, May 12, 2011 at 5:53 AM, Steve Roder s...@buffalo.edu wrote:

 What's pinning your log?  Do a show logpinned, and then look at what
 that client is doing.  If it is a TDP, they are known for pinning
the
 log and causing these kinds of issues.

 We see this issue also, and it is nearly always caused by an Oracle
TDP
 or an Exchange TDP backup.

 Thanks,
 Steve


  Hi Paul!
 We are already running 5.5.5.2 and the log is still filling up,
even
 after switching from rollforward to normal mode.
 Management currently is questioning whether TSM is the right
product
for
 the future. Although I'm a big fan of TSM for 15 years, I'm really
in
 doubt too...
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On
Behalf
Of
 Paul Fielding
 Sent: woensdag 11 mei 2011 20:05
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0
code

 Hi folks, if this has already been commented on, my apologies, I
hvaen't
 been following closely but just noticed this thread.

 We were experiencing log pinned issues after upgrading to 5.5.0.0
code
 at
 one of my client sites.  What we were finding was that,
occasionally,
 I'd
 get up in the morning and look at the server to see it completely
bogged
 down with a huge number of backups still attempting to run, but
nothing
 was
 moving - the log was at 84% (over it's trigger), and it had been
firing
 off
 db backups for the last 4h to no avail, the log wasn't getting low
 enough.

 A key symptom was that in the actlog we were seeing messages to the
 effect
 of Recovery log is at 84%, transactions will be delayed by 3ms
(or
 something like that).

 In opening a ticket, it was found there was an issue in the 5.5.0.0
code
 where, when the recovery log gets too full, it would start delaying
 transactions in order to prevent the log from filling too quickly.
 However,
 the side effect was that the pinning transaction would also get
delayed,
 causing a bit of a never-ending loop.  Transactions keep getting
 delayed,
 pinned transaction never gets to finish, and everything would just
grind
 to
 a halt.  I would have to halt the server and restart in order to
get
 things
 back to normal.

 IBM recognized this as an issue and recommended going to any
5.5.5.0
 level
 of code, where the problem was supposed to be fixed.  I installed
 5.5.5.2,
 and the problem has indeed gone away.   It was supposed to be fixed
at
 5.5.5.0, though perhaps they didn't quite get it done at that code
level
 as
 they hoped?  I'd try installing 5.5.5.2 and see what happens

 regards,

 Paul


 On Wed, May 11, 2011 at 9:03 AM, Loon, EJ van - SPLXO
 eric-van.l...@klm.com

 wrote:
 Hi Robert!
 Thanks you very much for your reply! Several others on this list
 reported this behavior and (as far as I know) three other users
opened

 a

 PMR too. I hope they have more luck, because I'm stuck. Level 2
keeps

 on

 saying that the log keeps on growing because of slow running
client
 sessions. Indeed I see slow running client sessions, but they are

 slowed

 down by the fact that TSM is delaying all transactions because the
log
 is used for more that 80% during a large part of the backup
window!

 Now

 they refuse to help me, unless I buy a Passport Advantage

 contract!!

 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On
Behalf

 Of

 Robert Clark
 Sent: woensdag 11 mei 2011 16:05
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0
code

 I believe we've been seeing this problem as well.

 One night in the busiest backup period, I issued a q actlog
 begint=now-00:30, and got no results back but an error.

 I started dsmadmc -console on that server, and could see that the
 console
 output was most of an hour behind. (And the console output was

 scrolling

 so fast that it could barely

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Paul Fielding]

A couple of questions, Eric

- how big is your DB
- have you tried a test upgrade to an alternate box to see how long it will
take?

Indeed, I agree that you're going to have to do something, whether it's take
the outage to go to V6 or start to migrate to another product.  However, if
you are in a position to move to another product, then (worst case
scenario), you're probably in a position to move to a clean V6 server
without migrating.   The one thing you really can't do is stay where you are
forever, at least not with support

Paul

On Thu, May 12, 2011 at 7:38 AM, David E Ehresman
deehr...@louisville.eduwrote:

 Eric,

 Pointing out the obvious here, but if you really can't afford to move
 to TSM v6 then its time for you to start planning your move to something
 else.  Support for TSM 5 will be dropped sometime and I suspect the move
 from TSM to something else will take some planning.

 David

  Loon, EJ van - SPLXO eric-van.l...@klm.com 5/12/2011 8:56 AM
 
 Hi Steve!
 If it would be possible, I would have already done that, but migrating
 our servers from 5.5 to 6.2 would require multiple days of downtime.
 Simple not acceptable in our shop...
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf
 Of
 Paul Fielding
 Sent: donderdag 12 mei 2011 14:33
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 You could also go to TSM 6.2, which changes to DB2 and changes the way
 recovery logs are dealt with, including allowing for much, much more
 log


 On Thu, May 12, 2011 at 5:53 AM, Steve Roder s...@buffalo.edu wrote:

  What's pinning your log?  Do a show logpinned, and then look at what
  that client is doing.  If it is a TDP, they are known for pinning
 the
  log and causing these kinds of issues.
 
  We see this issue also, and it is nearly always caused by an Oracle
 TDP
  or an Exchange TDP backup.
 
  Thanks,
  Steve
 
 
   Hi Paul!
  We are already running 5.5.5.2 and the log is still filling up,
 even
  after switching from rollforward to normal mode.
  Management currently is questioning whether TSM is the right
 product
 for
  the future. Although I'm a big fan of TSM for 15 years, I'm really
 in
  doubt too...
  Kind regards,
  Eric van Loon
  KLM Royal Dutch Airlines
 
  -Original Message-
  From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On
 Behalf
 Of
  Paul Fielding
  Sent: woensdag 11 mei 2011 20:05
  To: ADSM-L@VM.MARIST.EDU
  Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0
 code
 
  Hi folks, if this has already been commented on, my apologies, I
 hvaen't
  been following closely but just noticed this thread.
 
  We were experiencing log pinned issues after upgrading to 5.5.0.0
 code
  at
  one of my client sites.  What we were finding was that,
 occasionally,
  I'd
  get up in the morning and look at the server to see it completely
 bogged
  down with a huge number of backups still attempting to run, but
 nothing
  was
  moving - the log was at 84% (over it's trigger), and it had been
 firing
  off
  db backups for the last 4h to no avail, the log wasn't getting low
  enough.
 
  A key symptom was that in the actlog we were seeing messages to the
  effect
  of Recovery log is at 84%, transactions will be delayed by 3ms
 (or
  something like that).
 
  In opening a ticket, it was found there was an issue in the 5.5.0.0
 code
  where, when the recovery log gets too full, it would start delaying
  transactions in order to prevent the log from filling too quickly.
  However,
  the side effect was that the pinning transaction would also get
 delayed,
  causing a bit of a never-ending loop.  Transactions keep getting
  delayed,
  pinned transaction never gets to finish, and everything would just
 grind
  to
  a halt.  I would have to halt the server and restart in order to
 get
  things
  back to normal.
 
  IBM recognized this as an issue and recommended going to any
 5.5.5.0
  level
  of code, where the problem was supposed to be fixed.  I installed
  5.5.5.2,
  and the problem has indeed gone away.   It was supposed to be fixed
 at
  5.5.5.0, though perhaps they didn't quite get it done at that code
 level
  as
  they hoped?  I'd try installing 5.5.5.2 and see what happens
 
  regards,
 
  Paul
 
 
  On Wed, May 11, 2011 at 9:03 AM, Loon, EJ van - SPLXO
  eric-van.l...@klm.com
 
  wrote:
  Hi Robert!
  Thanks you very much for your reply! Several others on this list
  reported this behavior and (as far as I know) three other users
 opened
 
  a
 
  PMR too. I hope they have more luck, because I'm stuck. Level 2
 keeps
 
  on
 
  saying that the log keeps on growing because of slow running
 client
  sessions. Indeed I see slow running client sessions, but they are
 
  slowed
 
  down by the fact that TSM is delaying all transactions because the
 log
  is used for more that 80% during a large

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Loon, EJ van - SPLXO]

Hi Rick!
You are running in normal mode. In this mode the recovery log only
contains uncommited transactions. Don't you agree that in this case, the
log should NEVER reach 90%? It should not even reach something like 40%!
I don't want to have to implement extra monitoring to keep TSM from
crashing. It should just work. Like it did when we were running 5.3...
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Richard Rhodes
Sent: donderdag 12 mei 2011 15:04
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

I have a cron job that checks log utilization every 15m.  If the log
goes
over 90% the scripts displays a bunch of stuff (q sess, q proc, q log,
show logpin, etc),  runs a show logpin cancel,  displays the stuff
again,
and finally emails me. (We run in normal mode.)   It's been very
effective
in preventing a log full condition.

rick






From:   Bos, Karel karel@atosorigin.com
To: ADSM-L@VM.MARIST.EDU
Date:   05/12/2011 06:11 AM
Subject:Re: TSM Recovery log is pinning since upgrade to 5.5.5.0
code
Sent by:ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



Hi,

Dunno if u already looked at these options: THROUGHPUTDATATHRESHOLD and
THROUGHPUTTIMETHRESHOLD. After a couple of crashes due to log filling
up,
a couple of years back, I standard added them to all my clients (with
low
values to only kill the slowest/hung ones) and untill now never had any
issues with the V5.5 and below log space under normal backup loads.

Log filling only happend when some sys admin made some changes to the
biggest file servers we had but even this was handled like it should be
done by ITSM.

Regards,

Karel




Van: ADSM: Dist Stor Manager [ADSM-L@VM.MARIST.EDU] namens Loon, EJ van
-
SPLXO [eric-van.l...@klm.com]
Verzonden: donderdag 12 mei 2011 11:11
Aan: ADSM-L@VM.MARIST.EDU
Onderwerp: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Hi Paul!
We are already running 5.5.5.2 and the log is still filling up, even
after switching from rollforward to normal mode.
Management currently is questioning whether TSM is the right product for
the future. Although I'm a big fan of TSM for 15 years, I'm really in
doubt too...
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Paul Fielding
Sent: woensdag 11 mei 2011 20:05
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Hi folks, if this has already been commented on, my apologies, I hvaen't
been following closely but just noticed this thread.

We were experiencing log pinned issues after upgrading to 5.5.0.0 code
at
one of my client sites.  What we were finding was that, occasionally,
I'd
get up in the morning and look at the server to see it completely bogged
down with a huge number of backups still attempting to run, but nothing
was
moving - the log was at 84% (over it's trigger), and it had been firing
off
db backups for the last 4h to no avail, the log wasn't getting low
enough.

A key symptom was that in the actlog we were seeing messages to the
effect
of Recovery log is at 84%, transactions will be delayed by 3ms (or
something like that).

In opening a ticket, it was found there was an issue in the 5.5.0.0 code
where, when the recovery log gets too full, it would start delaying
transactions in order to prevent the log from filling too quickly.
However,
the side effect was that the pinning transaction would also get delayed,
causing a bit of a never-ending loop.  Transactions keep getting
delayed,
pinned transaction never gets to finish, and everything would just grind
to
a halt.  I would have to halt the server and restart in order to get
things
back to normal.

IBM recognized this as an issue and recommended going to any 5.5.5.0
level
of code, where the problem was supposed to be fixed.  I installed
5.5.5.2,
and the problem has indeed gone away.   It was supposed to be fixed at
5.5.5.0, though perhaps they didn't quite get it done at that code level
as
they hoped?  I'd try installing 5.5.5.2 and see what happens

regards,

Paul


On Wed, May 11, 2011 at 9:03 AM, Loon, EJ van - SPLXO
eric-van.l...@klm.com
 wrote:

 Hi Robert!
 Thanks you very much for your reply! Several others on this list
 reported this behavior and (as far as I know) three other users opened
a
 PMR too. I hope they have more luck, because I'm stuck. Level 2 keeps
on
 saying that the log keeps on growing because of slow running client
 sessions. Indeed I see slow running client sessions, but they are
slowed
 down by the fact that TSM is delaying all transactions because the log
 is used for more that 80% during a large part of the backup window!
Now
 they refuse to help me, unless I buy a Passport Advantage
contract!!
 Kind regards

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Loon, EJ van - SPLXO]

Hi Paul! 
I have 3 servers with 86, 100 and 129 gb databases. I haven't tried
migrating, but I read a document somewhere which contained some figures
about what performance one can expect from the migration process. At
that time I roughly calculated 3 and a half days.
Anyway, let's stay on the subject here: there is something broken in TSM
(remember, I'm not the only one struggling with the recovery log size)
and IBM should seriously look in to it and fix it. I never had any
problems with the log when we were using 5.3!
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Paul Fielding
Sent: donderdag 12 mei 2011 15:53
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

A couple of questions, Eric

- how big is your DB
- have you tried a test upgrade to an alternate box to see how long it
will
take?

Indeed, I agree that you're going to have to do something, whether it's
take
the outage to go to V6 or start to migrate to another product.  However,
if
you are in a position to move to another product, then (worst case
scenario), you're probably in a position to move to a clean V6 server
without migrating.   The one thing you really can't do is stay where you
are
forever, at least not with support

Paul

On Thu, May 12, 2011 at 7:38 AM, David E Ehresman
deehr...@louisville.eduwrote:

 Eric,

 Pointing out the obvious here, but if you really can't afford to move
 to TSM v6 then its time for you to start planning your move to
something
 else.  Support for TSM 5 will be dropped sometime and I suspect the
move
 from TSM to something else will take some planning.

 David

  Loon, EJ van - SPLXO eric-van.l...@klm.com 5/12/2011 8:56 AM
 
 Hi Steve!
 If it would be possible, I would have already done that, but migrating
 our servers from 5.5 to 6.2 would require multiple days of downtime.
 Simple not acceptable in our shop...
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf
 Of
 Paul Fielding
 Sent: donderdag 12 mei 2011 14:33
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 You could also go to TSM 6.2, which changes to DB2 and changes the way
 recovery logs are dealt with, including allowing for much, much more
 log


 On Thu, May 12, 2011 at 5:53 AM, Steve Roder s...@buffalo.edu wrote:

  What's pinning your log?  Do a show logpinned, and then look at what
  that client is doing.  If it is a TDP, they are known for pinning
 the
  log and causing these kinds of issues.
 
  We see this issue also, and it is nearly always caused by an Oracle
 TDP
  or an Exchange TDP backup.
 
  Thanks,
  Steve
 
 
   Hi Paul!
  We are already running 5.5.5.2 and the log is still filling up,
 even
  after switching from rollforward to normal mode.
  Management currently is questioning whether TSM is the right
 product
 for
  the future. Although I'm a big fan of TSM for 15 years, I'm really
 in
  doubt too...
  Kind regards,
  Eric van Loon
  KLM Royal Dutch Airlines
 
  -Original Message-
  From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On
 Behalf
 Of
  Paul Fielding
  Sent: woensdag 11 mei 2011 20:05
  To: ADSM-L@VM.MARIST.EDU
  Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0
 code
 
  Hi folks, if this has already been commented on, my apologies, I
 hvaen't
  been following closely but just noticed this thread.
 
  We were experiencing log pinned issues after upgrading to 5.5.0.0
 code
  at
  one of my client sites.  What we were finding was that,
 occasionally,
  I'd
  get up in the morning and look at the server to see it completely
 bogged
  down with a huge number of backups still attempting to run, but
 nothing
  was
  moving - the log was at 84% (over it's trigger), and it had been
 firing
  off
  db backups for the last 4h to no avail, the log wasn't getting low
  enough.
 
  A key symptom was that in the actlog we were seeing messages to the
  effect
  of Recovery log is at 84%, transactions will be delayed by 3ms
 (or
  something like that).
 
  In opening a ticket, it was found there was an issue in the 5.5.0.0
 code
  where, when the recovery log gets too full, it would start delaying
  transactions in order to prevent the log from filling too quickly.
  However,
  the side effect was that the pinning transaction would also get
 delayed,
  causing a bit of a never-ending loop.  Transactions keep getting
  delayed,
  pinned transaction never gets to finish, and everything would just
 grind
  to
  a halt.  I would have to halt the server and restart in order to
 get
  things
  back to normal.
 
  IBM recognized this as an issue and recommended going to any
 5.5.5.0
  level
  of code, where the problem was supposed to be fixed.  I installed
  5.5.5.2,
  and the problem has

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Loon, EJ van - SPLXO]

Hi David!
We're already working on that. I hope to use TSM for that new
environment, but the way my PMR is handled by IBM support makes me
wonder if it's not wiser to start looking at other vendors...
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
David E Ehresman
Sent: donderdag 12 mei 2011 15:38
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Eric,

Pointing out the obvious here, but if you really can't afford to move
to TSM v6 then its time for you to start planning your move to something
else.  Support for TSM 5 will be dropped sometime and I suspect the move
from TSM to something else will take some planning.

David

 Loon, EJ van - SPLXO eric-van.l...@klm.com 5/12/2011 8:56 AM

Hi Steve!
If it would be possible, I would have already done that, but migrating
our servers from 5.5 to 6.2 would require multiple days of downtime.
Simple not acceptable in our shop...
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf
Of
Paul Fielding
Sent: donderdag 12 mei 2011 14:33
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

You could also go to TSM 6.2, which changes to DB2 and changes the way
recovery logs are dealt with, including allowing for much, much more
log


On Thu, May 12, 2011 at 5:53 AM, Steve Roder s...@buffalo.edu wrote:

 What's pinning your log?  Do a show logpinned, and then look at what
 that client is doing.  If it is a TDP, they are known for pinning
the
 log and causing these kinds of issues.

 We see this issue also, and it is nearly always caused by an Oracle
TDP
 or an Exchange TDP backup.

 Thanks,
 Steve


  Hi Paul!
 We are already running 5.5.5.2 and the log is still filling up,
even
 after switching from rollforward to normal mode.
 Management currently is questioning whether TSM is the right
product
for
 the future. Although I'm a big fan of TSM for 15 years, I'm really
in
 doubt too...
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On
Behalf
Of
 Paul Fielding
 Sent: woensdag 11 mei 2011 20:05
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0
code

 Hi folks, if this has already been commented on, my apologies, I
hvaen't
 been following closely but just noticed this thread.

 We were experiencing log pinned issues after upgrading to 5.5.0.0
code
 at
 one of my client sites.  What we were finding was that,
occasionally,
 I'd
 get up in the morning and look at the server to see it completely
bogged
 down with a huge number of backups still attempting to run, but
nothing
 was
 moving - the log was at 84% (over it's trigger), and it had been
firing
 off
 db backups for the last 4h to no avail, the log wasn't getting low
 enough.

 A key symptom was that in the actlog we were seeing messages to the
 effect
 of Recovery log is at 84%, transactions will be delayed by 3ms
(or
 something like that).

 In opening a ticket, it was found there was an issue in the 5.5.0.0
code
 where, when the recovery log gets too full, it would start delaying
 transactions in order to prevent the log from filling too quickly.
 However,
 the side effect was that the pinning transaction would also get
delayed,
 causing a bit of a never-ending loop.  Transactions keep getting
 delayed,
 pinned transaction never gets to finish, and everything would just
grind
 to
 a halt.  I would have to halt the server and restart in order to
get
 things
 back to normal.

 IBM recognized this as an issue and recommended going to any
5.5.5.0
 level
 of code, where the problem was supposed to be fixed.  I installed
 5.5.5.2,
 and the problem has indeed gone away.   It was supposed to be fixed
at
 5.5.5.0, though perhaps they didn't quite get it done at that code
level
 as
 they hoped?  I'd try installing 5.5.5.2 and see what happens

 regards,

 Paul


 On Wed, May 11, 2011 at 9:03 AM, Loon, EJ van - SPLXO
 eric-van.l...@klm.com

 wrote:
 Hi Robert!
 Thanks you very much for your reply! Several others on this list
 reported this behavior and (as far as I know) three other users
opened

 a

 PMR too. I hope they have more luck, because I'm stuck. Level 2
keeps

 on

 saying that the log keeps on growing because of slow running
client
 sessions. Indeed I see slow running client sessions, but they are

 slowed

 down by the fact that TSM is delaying all transactions because the
log
 is used for more that 80% during a large part of the backup
window!

 Now

 they refuse to help me, unless I buy a Passport Advantage

 contract!!

 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On
Behalf

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Richard Rhodes]

No . . . It's completely possible for the the log in normal mode to be
pinned and hit 100%.  My understanding is that the log contains ALL
transactions.  It's the checkpoint/rollback for info for the db.  An open
tran (uncommitted?) pins the log. We get them every once in a while
when some server sends a big file very slowly - usually a remote site, but
sometimes a local with something set wrong (my favorite is the half-duplex
ethernet connection which runs at 50k/s).  But it's not just the log being
pinned by a long running tran that is a concern, it's also the load on the
TSM server for all the other backups that are running and their
transactions that determine how quickly a pinned log fills up.

Here is the log consumption for the past day for six of our tsm servers:
  tsm1   3gb
  tsm2   2gb
  tsm3   8gb
  tsm4  10gb
  tsm5   4gb
  tsm6   6gb

They all have 12gb of log space.

Back a few years ago, tsm1 was consuming over 20gb per day and we had tons
of logpin problems, one time resulting in the log filling completely. This
was when I put in the monitoring with the auto logpin cancel.It was
one of the many indications that we needed more instances.  It's also why
we have a daily report of slow and long running backups so we can identify
these problem servers.

Rick










From:   Loon, EJ van - SPLXO eric-van.l...@klm.com
To: ADSM-L@VM.MARIST.EDU
Date:   05/12/2011 10:54 AM
Subject:Re: TSM Recovery log is pinning since upgrade to 5.5.5.0
code
Sent by:ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



Hi Rick!
You are running in normal mode. In this mode the recovery log only
contains uncommited transactions. Don't you agree that in this case, the
log should NEVER reach 90%? It should not even reach something like 40%!
I don't want to have to implement extra monitoring to keep TSM from
crashing. It should just work. Like it did when we were running 5.3...
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Richard Rhodes
Sent: donderdag 12 mei 2011 15:04
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

I have a cron job that checks log utilization every 15m.  If the log
goes
over 90% the scripts displays a bunch of stuff (q sess, q proc, q log,
show logpin, etc),  runs a show logpin cancel,  displays the stuff
again,
and finally emails me. (We run in normal mode.)   It's been very
effective
in preventing a log full condition.

rick






From:   Bos, Karel karel@atosorigin.com
To: ADSM-L@VM.MARIST.EDU
Date:   05/12/2011 06:11 AM
Subject:Re: TSM Recovery log is pinning since upgrade to 5.5.5.0
code
Sent by:ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



Hi,

Dunno if u already looked at these options: THROUGHPUTDATATHRESHOLD and
THROUGHPUTTIMETHRESHOLD. After a couple of crashes due to log filling
up,
a couple of years back, I standard added them to all my clients (with
low
values to only kill the slowest/hung ones) and untill now never had any
issues with the V5.5 and below log space under normal backup loads.

Log filling only happend when some sys admin made some changes to the
biggest file servers we had but even this was handled like it should be
done by ITSM.

Regards,

Karel




Van: ADSM: Dist Stor Manager [ADSM-L@VM.MARIST.EDU] namens Loon, EJ van
-
SPLXO [eric-van.l...@klm.com]
Verzonden: donderdag 12 mei 2011 11:11
Aan: ADSM-L@VM.MARIST.EDU
Onderwerp: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Hi Paul!
We are already running 5.5.5.2 and the log is still filling up, even
after switching from rollforward to normal mode.
Management currently is questioning whether TSM is the right product for
the future. Although I'm a big fan of TSM for 15 years, I'm really in
doubt too...
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Paul Fielding
Sent: woensdag 11 mei 2011 20:05
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Hi folks, if this has already been commented on, my apologies, I hvaen't
been following closely but just noticed this thread.

We were experiencing log pinned issues after upgrading to 5.5.0.0 code
at
one of my client sites.  What we were finding was that, occasionally,
I'd
get up in the morning and look at the server to see it completely bogged
down with a huge number of backups still attempting to run, but nothing
was
moving - the log was at 84% (over it's trigger), and it had been firing
off
db backups for the last 4h to no avail, the log wasn't getting low
enough.

A key symptom was that in the actlog we were seeing messages to the
effect
of Recovery log is at 84%, transactions will be delayed by 3ms (or
something like that).

In opening a ticket, it was found

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Loon, EJ van - SPLXO]

Hi TSM-ers!
Here is a small follow up on my PMR about the recovery log utilization.
I'm at TSM level 2, still trying to convince them that there is
something broken in the TSM server code. To convince them, I have
changed the logmode to normal on one of my servers. I created a graph
(through TSMManager) which shows the recovery log utilization during
last night's client backup window and is doesn't differ much from the
night before, with logmode rollforward. When running in normal mode, TSM
should only use the recovery log for uncommitted transactions, so
utilization should be very low. My log is 12 Gb and the backuptrigger
value (75%) was still hit twice!
This clearly shows that there is something wrong with TSM, let's hope I
can convince Level 2 too, so my case gets forwarded to the lab.
I'll keep you guys posted!
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

For information, services and offers, please visit our web site: 
http://www.klm.com. This e-mail and any attachment may contain confidential and 
privileged material intended for the addressee only. If you are not the 
addressee, you are notified that no part of the e-mail or any attachment may be 
disclosed, copied or distributed, and that any other action related to this 
e-mail or attachment is strictly prohibited, and may be unlawful. If you have 
received this e-mail by error, please notify the sender immediately by return 
e-mail, and delete this message. 

Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its 
employees shall not be liable for the incorrect or incomplete transmission of 
this e-mail or any attachments, nor responsible for any delay in receipt. 
Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch 
Airlines) is registered in Amstelveen, The Netherlands, with registered number 
33014286

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Robert Clark]

I believe we've been seeing this problem as well.

One night in the busiest backup period, I issued a q actlog
begint=now-00:30, and got no results back but an error.

I started dsmadmc -console on that server, and could see that the console
output was most of an hour behind. (And the console output was scrolling
so fast that it could barely be read.)

In that case, I think we determined that SQL LiteSpeed was set to some
rediculously small transaction size, and this was causing way too many
actlog entries.

I think I also noted that the session number incremented by something like
100,000 in an hour.

Asking the users of SQL LiteSpeed to make some changes was enough to
rememdy this problem, although we continue to fight with the logs getting
full.

Thanks,
[RC]



From:   Loon, EJ van - SPLXO eric-van.l...@klm.com
To: ADSM-L@VM.MARIST.EDU
Date:   05/11/2011 02:05 AM
Subject:Re: [ADSM-L] TSM Recovery log is pinning since upgrade to
5.5.5.0 code
Sent by:ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



Hi TSM-ers!
Here is a small follow up on my PMR about the recovery log utilization.
I'm at TSM level 2, still trying to convince them that there is
something broken in the TSM server code. To convince them, I have
changed the logmode to normal on one of my servers. I created a graph
(through TSMManager) which shows the recovery log utilization during
last night's client backup window and is doesn't differ much from the
night before, with logmode rollforward. When running in normal mode, TSM
should only use the recovery log for uncommitted transactions, so
utilization should be very low. My log is 12 Gb and the backuptrigger
value (75%) was still hit twice!
This clearly shows that there is something wrong with TSM, let's hope I
can convince Level 2 too, so my case gets forwarded to the lab.
I'll keep you guys posted!
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

For information, services and offers, please visit our web site:
http://www.klm.com. This e-mail and any attachment may contain
confidential and privileged material intended for the addressee only. If
you are not the addressee, you are notified that no part of the e-mail or
any attachment may be disclosed, copied or distributed, and that any other
action related to this e-mail or attachment is strictly prohibited, and
may be unlawful. If you have received this e-mail by error, please notify
the sender immediately by return e-mail, and delete this message.

Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its
employees shall not be liable for the incorrect or incomplete transmission
of this e-mail or any attachments, nor responsible for any delay in
receipt.
Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch
Airlines) is registered in Amstelveen, The Netherlands, with registered
number 33014286




U.S. BANCORP made the following annotations
-
Electronic Privacy Notice. This e-mail, and any attachments, contains 
information that is, or may be, covered by electronic communications privacy 
laws, and is also confidential and proprietary in nature. If you are not the 
intended recipient, please be advised that you are legally prohibited from 
retaining, using, copying, distributing, or otherwise disclosing this 
information in any manner. Instead, please reply to the sender that you have 
received this communication in error, and then immediately delete it. Thank you 
in advance for your cooperation.



-

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Loon, EJ van - SPLXO]

Hi Robert!
Thanks you very much for your reply! Several others on this list
reported this behavior and (as far as I know) three other users opened a
PMR too. I hope they have more luck, because I'm stuck. Level 2 keeps on
saying that the log keeps on growing because of slow running client
sessions. Indeed I see slow running client sessions, but they are slowed
down by the fact that TSM is delaying all transactions because the log
is used for more that 80% during a large part of the backup window! Now
they refuse to help me, unless I buy a Passport Advantage contract!!
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Robert Clark
Sent: woensdag 11 mei 2011 16:05
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

I believe we've been seeing this problem as well.

One night in the busiest backup period, I issued a q actlog
begint=now-00:30, and got no results back but an error.

I started dsmadmc -console on that server, and could see that the
console
output was most of an hour behind. (And the console output was scrolling
so fast that it could barely be read.)

In that case, I think we determined that SQL LiteSpeed was set to some
rediculously small transaction size, and this was causing way too many
actlog entries.

I think I also noted that the session number incremented by something
like
100,000 in an hour.

Asking the users of SQL LiteSpeed to make some changes was enough to
rememdy this problem, although we continue to fight with the logs
getting
full.

Thanks,
[RC]



From:   Loon, EJ van - SPLXO eric-van.l...@klm.com
To: ADSM-L@VM.MARIST.EDU
Date:   05/11/2011 02:05 AM
Subject:Re: [ADSM-L] TSM Recovery log is pinning since upgrade
to
5.5.5.0 code
Sent by:ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



Hi TSM-ers!
Here is a small follow up on my PMR about the recovery log utilization.
I'm at TSM level 2, still trying to convince them that there is
something broken in the TSM server code. To convince them, I have
changed the logmode to normal on one of my servers. I created a graph
(through TSMManager) which shows the recovery log utilization during
last night's client backup window and is doesn't differ much from the
night before, with logmode rollforward. When running in normal mode, TSM
should only use the recovery log for uncommitted transactions, so
utilization should be very low. My log is 12 Gb and the backuptrigger
value (75%) was still hit twice!
This clearly shows that there is something wrong with TSM, let's hope I
can convince Level 2 too, so my case gets forwarded to the lab.
I'll keep you guys posted!
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

For information, services and offers, please visit our web site:
http://www.klm.com. This e-mail and any attachment may contain
confidential and privileged material intended for the addressee only. If
you are not the addressee, you are notified that no part of the e-mail
or
any attachment may be disclosed, copied or distributed, and that any
other
action related to this e-mail or attachment is strictly prohibited, and
may be unlawful. If you have received this e-mail by error, please
notify
the sender immediately by return e-mail, and delete this message.

Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or
its
employees shall not be liable for the incorrect or incomplete
transmission
of this e-mail or any attachments, nor responsible for any delay in
receipt.
Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch
Airlines) is registered in Amstelveen, The Netherlands, with registered
number 33014286




U.S. BANCORP made the following annotations
-
Electronic Privacy Notice. This e-mail, and any attachments, contains
information that is, or may be, covered by electronic communications
privacy laws, and is also confidential and proprietary in nature. If you
are not the intended recipient, please be advised that you are legally
prohibited from retaining, using, copying, distributing, or otherwise
disclosing this information in any manner. Instead, please reply to the
sender that you have received this communication in error, and then
immediately delete it. Thank you in advance for your cooperation.



-

For information, services and offers, please visit our web site: 
http://www.klm.com. This e-mail and any attachment may contain confidential and 
privileged material intended for the addressee only. If you are not the 
addressee, you are notified that no part of the e-mail or any attachment may be 
disclosed, copied or distributed

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Paul Fielding]

Hi folks, if this has already been commented on, my apologies, I hvaen't
been following closely but just noticed this thread.

We were experiencing log pinned issues after upgrading to 5.5.0.0 code at
one of my client sites.  What we were finding was that, occasionally, I'd
get up in the morning and look at the server to see it completely bogged
down with a huge number of backups still attempting to run, but nothing was
moving - the log was at 84% (over it's trigger), and it had been firing off
db backups for the last 4h to no avail, the log wasn't getting low enough.

A key symptom was that in the actlog we were seeing messages to the effect
of Recovery log is at 84%, transactions will be delayed by 3ms (or
something like that).

In opening a ticket, it was found there was an issue in the 5.5.0.0 code
where, when the recovery log gets too full, it would start delaying
transactions in order to prevent the log from filling too quickly.  However,
the side effect was that the pinning transaction would also get delayed,
causing a bit of a never-ending loop.  Transactions keep getting delayed,
pinned transaction never gets to finish, and everything would just grind to
a halt.  I would have to halt the server and restart in order to get things
back to normal.

IBM recognized this as an issue and recommended going to any 5.5.5.0 level
of code, where the problem was supposed to be fixed.  I installed 5.5.5.2,
and the problem has indeed gone away.   It was supposed to be fixed at
5.5.5.0, though perhaps they didn't quite get it done at that code level as
they hoped?  I'd try installing 5.5.5.2 and see what happens

regards,

Paul


On Wed, May 11, 2011 at 9:03 AM, Loon, EJ van - SPLXO eric-van.l...@klm.com
 wrote:

 Hi Robert!
 Thanks you very much for your reply! Several others on this list
 reported this behavior and (as far as I know) three other users opened a
 PMR too. I hope they have more luck, because I'm stuck. Level 2 keeps on
 saying that the log keeps on growing because of slow running client
 sessions. Indeed I see slow running client sessions, but they are slowed
 down by the fact that TSM is delaying all transactions because the log
 is used for more that 80% during a large part of the backup window! Now
 they refuse to help me, unless I buy a Passport Advantage contract!!
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines

 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
 Robert Clark
 Sent: woensdag 11 mei 2011 16:05
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 I believe we've been seeing this problem as well.

 One night in the busiest backup period, I issued a q actlog
 begint=now-00:30, and got no results back but an error.

 I started dsmadmc -console on that server, and could see that the
 console
 output was most of an hour behind. (And the console output was scrolling
 so fast that it could barely be read.)

 In that case, I think we determined that SQL LiteSpeed was set to some
 rediculously small transaction size, and this was causing way too many
 actlog entries.

 I think I also noted that the session number incremented by something
 like
 100,000 in an hour.

 Asking the users of SQL LiteSpeed to make some changes was enough to
 rememdy this problem, although we continue to fight with the logs
 getting
 full.

 Thanks,
 [RC]



 From:   Loon, EJ van - SPLXO eric-van.l...@klm.com
 To: ADSM-L@VM.MARIST.EDU
 Date:   05/11/2011 02:05 AM
 Subject:Re: [ADSM-L] TSM Recovery log is pinning since upgrade
 to
 5.5.5.0 code
 Sent by:ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



 Hi TSM-ers!
 Here is a small follow up on my PMR about the recovery log utilization.
 I'm at TSM level 2, still trying to convince them that there is
 something broken in the TSM server code. To convince them, I have
 changed the logmode to normal on one of my servers. I created a graph
 (through TSMManager) which shows the recovery log utilization during
 last night's client backup window and is doesn't differ much from the
 night before, with logmode rollforward. When running in normal mode, TSM
 should only use the recovery log for uncommitted transactions, so
 utilization should be very low. My log is 12 Gb and the backuptrigger
 value (75%) was still hit twice!
 This clearly shows that there is something wrong with TSM, let's hope I
 can convince Level 2 too, so my case gets forwarded to the lab.
 I'll keep you guys posted!
 Kind regards,
 Eric van Loon
 KLM Royal Dutch Airlines
 
 For information, services and offers, please visit our web site:
 http://www.klm.com. This e-mail and any attachment may contain
 confidential and privileged material intended for the addressee only. If
 you are not the addressee, you are notified that no part of the e-mail
 or
 any attachment may be disclosed, copied or distributed

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Loon, EJ van - SPLXO]

Hi TSM-ers!
Just for your information: this weekend my log again was filling up to
95%. I monitored the growth of the log online and I witnessed that the
rate the log was filling up was very high. I tried timing it and it grew
with about 0.1% per 5 seconds!
I tried to prevent it from filling up to 100% by setting the logmode to
normal. I noticed that the log utilization immediately dropped to about
70%, but kept on growing.
From the TSM Administrator Guide: Logmode normal: The recovery log keeps
only uncommitted transactions, and its size is not affected by normal
mode.
When running in normal mode, the recovery log should be hardly utilized.
The fact that it keeps on growing, even after switching from rollforward
to normal mode, shows that for some reason the TSM server doesn't commit
the transactions properly.
I also added this to my PMR, let's hope I can convince IBM that there is
really something broken here.
I haven't seen him around on the list for a while, but in the early day
TSM server code guys like Dave Cannon were monitoring this list. It's a
pity they stopped doing this, fixing TSM issues were much easier in
those days...
Kind regards,
Eric van Loon

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Oscar Kolsteren
Sent: donderdag 21 april 2011 09:23
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Hi all,

 

I've seen exactly the same after upgrading to 5.5.5.2. I tried to
reschedule client and storage pool backups but still need to make at
least 2 incremental backups a night.

 

Raised PMR with IBM - 65823 999 866
https://www-946.ibm.com/support/servicerequest/problemDescriptionSelect
.action?userType=0srNumber=65823sourceAppl=XSRretainCountryCode=866b
ranch=999sourceNode=prNode5sourceTranId=558271303370048077draft=0 .

 

Hope this helps...

 

Best Regards,

 

Oscar Kolsteren

 

 

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Zoltan Forray/AC/VCU
Sent: 20 April 2011 17:38
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to
5.5.5.0 code

 

I went back in old emails and found mine from June 2010

 

PMR 93470,487,000

Zoltan Forray

TSM Software  Hardware Administrator

Virginia Commonwealth University

UCC/Office of Technology Services

zfor...@vcu.edu mailto:zfor...@vcu.edu  - 804-828-4807

Don't be a phishing victim - VCU and other reputable organizations will
never use email to request that you reply with your password, social
security number or confidential personal information. For more details
visit http://infosecurity.vcu.edu/phishing.html
http://infosecurity.vcu.edu/phishing.html 

 

 

 

From:

Loon, EJ van - SPLXO eric-van.l...@klm.com

To:

ADSM-L@VM.MARIST.EDU

Date:

04/20/2011 11:39 AM

Subject:

Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code
Sent by:

ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU

 

 

 

Hi Nancy!

Thanks for you PMR number, I will open one too tomorrow.

That's what support told Zoltan too, it's bull!!! We never had any
backup triggering when we were running 5.3, the problems started after
upgrading to 5.5, on all 3 servers. One of them cannot have a higher
load since we do not add additional clients on this server for more than
two years. The library is nearly full on this server.

Kind regards,

Eric van Loon

KLM Royal Dutch Airlines

 

-Original Message-

From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Nancy L Leugemors

Sent: woensdag 20 april 2011 16:20

To: ADSM-L@VM.MARIST.EDU

Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 

Eric,

 

My PMR is 32989,180.I too was told that my TSM server is very busy

and

maybe it's time to split it off into another instance of TSM.My log

pinned again yesterday on a Windows server this time where the other
times it was a database backups for different servers.

 

Please send out your  PMRs to the group so we can reference them in our
tickets.

 

Thanks everyone who replied, I will let you know the end result if there
is any :-)

 

Thank You,

 

 

 

Nancy Leugemors

Enterprise Systems

HealthNow, NY

716-887-7979

 

 

 

From:

Loon, EJ van - SPLXO eric-van.l...@klm.com

To:

ADSM-L@VM.MARIST.EDU

Date:

04/20/2011 04:50 AM

Subject:

Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code
Sent by:

ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU

 

 

 

Hi TSM-ers!

I too upgraded my servers to 5.5.5.2 and the log is still filling up to
fast. During last nights backup window, three incrementals were
triggered by the db backup trigger. My log is set to 12Gb with the
trigger at 75%.

I will open a PMR too for this and again, I urge everybody else too do
this too. I'm 100% convinced this excessive log utilization is a bug
introduced in the 5.5 code. I have never seen any db backup triggering
when we were

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Richard Rhodes]

We only run in normal mode, and I can assure you that in this mode it's
 possible to have log problems.  When it happens,  It's  usually caused by
a slow
running backup of a large object.  In other words, a single transaction
that runs a long time. We have monitoring scripts running that issue
 a show login cancel if our log reaches 90%.

Rick




From:   Loon, EJ van - SPLXO eric-van.l...@klm.com
To: ADSM-L@VM.MARIST.EDU
Date:   04/28/2011 05:02 AM
Subject:Re: TSM Recovery log is pinning since upgrade to 5.5.5.0
code
Sent by:ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



Hi TSM-ers!
Just for your information: this weekend my log again was filling up to
95%. I monitored the growth of the log online and I witnessed that the
rate the log was filling up was very high. I tried timing it and it grew
with about 0.1% per 5 seconds!
I tried to prevent it from filling up to 100% by setting the logmode to
normal. I noticed that the log utilization immediately dropped to about
70%, but kept on growing.
From the TSM Administrator Guide: Logmode normal: The recovery log keeps
only uncommitted transactions, and its size is not affected by normal
mode.
When running in normal mode, the recovery log should be hardly utilized.
The fact that it keeps on growing, even after switching from rollforward
to normal mode, shows that for some reason the TSM server doesn't commit
the transactions properly.
I also added this to my PMR, let's hope I can convince IBM that there is
really something broken here.
I haven't seen him around on the list for a while, but in the early day
TSM server code guys like Dave Cannon were monitoring this list. It's a
pity they stopped doing this, fixing TSM issues were much easier in
those days...
Kind regards,
Eric van Loon

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Oscar Kolsteren
Sent: donderdag 21 april 2011 09:23
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Hi all,



I've seen exactly the same after upgrading to 5.5.5.2. I tried to
reschedule client and storage pool backups but still need to make at
least 2 incremental backups a night.



Raised PMR with IBM - 65823 999 866
https://www-946.ibm.com/support/servicerequest/problemDescriptionSelect
.action?userType=0srNumber=65823sourceAppl=XSRretainCountryCode=866b
ranch=999sourceNode=prNode5sourceTranId=558271303370048077draft=0 .



Hope this helps...



Best Regards,



Oscar Kolsteren





-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Zoltan Forray/AC/VCU
Sent: 20 April 2011 17:38
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to
5.5.5.0 code



I went back in old emails and found mine from June 2010



PMR 93470,487,000

Zoltan Forray

TSM Software  Hardware Administrator

Virginia Commonwealth University

UCC/Office of Technology Services

zfor...@vcu.edu mailto:zfor...@vcu.edu  - 804-828-4807

Don't be a phishing victim - VCU and other reputable organizations will
never use email to request that you reply with your password, social
security number or confidential personal information. For more details
visit http://infosecurity.vcu.edu/phishing.html
http://infosecurity.vcu.edu/phishing.html







From:

Loon, EJ van - SPLXO eric-van.l...@klm.com

To:

ADSM-L@VM.MARIST.EDU

Date:

04/20/2011 11:39 AM

Subject:

Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code
Sent by:

ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU







Hi Nancy!

Thanks for you PMR number, I will open one too tomorrow.

That's what support told Zoltan too, it's bull!!! We never had any
backup triggering when we were running 5.3, the problems started after
upgrading to 5.5, on all 3 servers. One of them cannot have a higher
load since we do not add additional clients on this server for more than
two years. The library is nearly full on this server.

Kind regards,

Eric van Loon

KLM Royal Dutch Airlines



-Original Message-

From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Nancy L Leugemors

Sent: woensdag 20 april 2011 16:20

To: ADSM-L@VM.MARIST.EDU

Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code



Eric,



My PMR is 32989,180.I too was told that my TSM server is very busy

and

maybe it's time to split it off into another instance of TSM.My log

pinned again yesterday on a Windows server this time where the other
times it was a database backups for different servers.



Please send out your  PMRs to the group so we can reference them in our
tickets.



Thanks everyone who replied, I will let you know the end result if there
is any :-)



Thank You,







Nancy Leugemors

Enterprise Systems

HealthNow, NY

716-887-7979







From:

Loon, EJ van - SPLXO eric-van.l...@klm.com

To:

ADSM-L@VM.MARIST.EDU

Date:

04/20/2011 04:50 AM

Subject:

Re: [ADSM-L] TSM

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Loon, EJ van - SPLXO]

Hi Rick!
You are right, logpinning clients can cause the log to fill too, but I
saw the log grow in normal mode with no logpinning according to the show
logpinned command.
Also, if there is no logpinning, I would expect the log to drop to
nearly 0% when switching from rollforward to normal mode. It didn't, it
dropped from 97% to around 70%. So the log was filled for 70% with
uncommited transactions while there were no logpinning clients.
Something is really broken here...
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Richard Rhodes
Sent: donderdag 28 april 2011 13:17
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

We only run in normal mode, and I can assure you that in this mode it's
 possible to have log problems.  When it happens,  It's  usually caused
by
a slow
running backup of a large object.  In other words, a single transaction
that runs a long time. We have monitoring scripts running that issue
 a show login cancel if our log reaches 90%.

Rick




From:   Loon, EJ van - SPLXO eric-van.l...@klm.com
To: ADSM-L@VM.MARIST.EDU
Date:   04/28/2011 05:02 AM
Subject:Re: TSM Recovery log is pinning since upgrade to 5.5.5.0
code
Sent by:ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



Hi TSM-ers!
Just for your information: this weekend my log again was filling up to
95%. I monitored the growth of the log online and I witnessed that the
rate the log was filling up was very high. I tried timing it and it grew
with about 0.1% per 5 seconds!
I tried to prevent it from filling up to 100% by setting the logmode to
normal. I noticed that the log utilization immediately dropped to about
70%, but kept on growing.
From the TSM Administrator Guide: Logmode normal: The recovery log keeps
only uncommitted transactions, and its size is not affected by normal
mode.
When running in normal mode, the recovery log should be hardly utilized.
The fact that it keeps on growing, even after switching from rollforward
to normal mode, shows that for some reason the TSM server doesn't commit
the transactions properly.
I also added this to my PMR, let's hope I can convince IBM that there is
really something broken here.
I haven't seen him around on the list for a while, but in the early day
TSM server code guys like Dave Cannon were monitoring this list. It's a
pity they stopped doing this, fixing TSM issues were much easier in
those days...
Kind regards,
Eric van Loon

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Oscar Kolsteren
Sent: donderdag 21 april 2011 09:23
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Hi all,



I've seen exactly the same after upgrading to 5.5.5.2. I tried to
reschedule client and storage pool backups but still need to make at
least 2 incremental backups a night.



Raised PMR with IBM - 65823 999 866
https://www-946.ibm.com/support/servicerequest/problemDescriptionSelect
.action?userType=0srNumber=65823sourceAppl=XSRretainCountryCode=866b
ranch=999sourceNode=prNode5sourceTranId=558271303370048077draft=0 .



Hope this helps...



Best Regards,



Oscar Kolsteren





-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Zoltan Forray/AC/VCU
Sent: 20 April 2011 17:38
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to
5.5.5.0 code



I went back in old emails and found mine from June 2010



PMR 93470,487,000

Zoltan Forray

TSM Software  Hardware Administrator

Virginia Commonwealth University

UCC/Office of Technology Services

zfor...@vcu.edu mailto:zfor...@vcu.edu  - 804-828-4807

Don't be a phishing victim - VCU and other reputable organizations will
never use email to request that you reply with your password, social
security number or confidential personal information. For more details
visit http://infosecurity.vcu.edu/phishing.html
http://infosecurity.vcu.edu/phishing.html







From:

Loon, EJ van - SPLXO eric-van.l...@klm.com

To:

ADSM-L@VM.MARIST.EDU

Date:

04/20/2011 11:39 AM

Subject:

Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code
Sent by:

ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU







Hi Nancy!

Thanks for you PMR number, I will open one too tomorrow.

That's what support told Zoltan too, it's bull!!! We never had any
backup triggering when we were running 5.3, the problems started after
upgrading to 5.5, on all 3 servers. One of them cannot have a higher
load since we do not add additional clients on this server for more than
two years. The library is nearly full on this server.

Kind regards,

Eric van Loon

KLM Royal Dutch Airlines



-Original Message-

From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Nancy L Leugemors

Sent: woensdag 20 april 2011 16:20

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Loon, EJ van - SPLXO]

Hi TSM-ers!
Thanks for sending me you PMR numbers. I just opened one too and I
referred to your PMR's. Here is mine: 11000,211,788.
Luckily I'm using TSMManager over here, so I have two years of history
to prove that this problem is not a load or scalability issue. I have
one TSM server which hasn't grown during this two year period, because
the libraries are 95% full and no clients are added since.
Again, I urge everybody else who's seeing multiple database backup
triggering during the client backup window to open a PMR too, maybe
together we can convince IBM that there's something broken here.
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Oscar Kolsteren
Sent: donderdag 21 april 2011 09:23
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Hi all,

 

I've seen exactly the same after upgrading to 5.5.5.2. I tried to
reschedule client and storage pool backups but still need to make at
least 2 incremental backups a night.

 

Raised PMR with IBM - 65823 999 866
https://www-946.ibm.com/support/servicerequest/problemDescriptionSelect
.action?userType=0srNumber=65823sourceAppl=XSRretainCountryCode=866b
ranch=999sourceNode=prNode5sourceTranId=558271303370048077draft=0 .

 

Hope this helps...

 

Best Regards,

 

Oscar Kolsteren

 

 

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Zoltan Forray/AC/VCU
Sent: 20 April 2011 17:38
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to
5.5.5.0 code

 

I went back in old emails and found mine from June 2010

 

PMR 93470,487,000

Zoltan Forray

TSM Software  Hardware Administrator

Virginia Commonwealth University

UCC/Office of Technology Services

zfor...@vcu.edu mailto:zfor...@vcu.edu  - 804-828-4807

Don't be a phishing victim - VCU and other reputable organizations will
never use email to request that you reply with your password, social
security number or confidential personal information. For more details
visit http://infosecurity.vcu.edu/phishing.html
http://infosecurity.vcu.edu/phishing.html 

 

 

 

From:

Loon, EJ van - SPLXO eric-van.l...@klm.com

To:

ADSM-L@VM.MARIST.EDU

Date:

04/20/2011 11:39 AM

Subject:

Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code
Sent by:

ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU

 

 

 

Hi Nancy!

Thanks for you PMR number, I will open one too tomorrow.

That's what support told Zoltan too, it's bull!!! We never had any
backup triggering when we were running 5.3, the problems started after
upgrading to 5.5, on all 3 servers. One of them cannot have a higher
load since we do not add additional clients on this server for more than
two years. The library is nearly full on this server.

Kind regards,

Eric van Loon

KLM Royal Dutch Airlines

 

-Original Message-

From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Nancy L Leugemors

Sent: woensdag 20 april 2011 16:20

To: ADSM-L@VM.MARIST.EDU

Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 

Eric,

 

My PMR is 32989,180.I too was told that my TSM server is very busy

and

maybe it's time to split it off into another instance of TSM.My log

pinned again yesterday on a Windows server this time where the other
times it was a database backups for different servers.

 

Please send out your  PMRs to the group so we can reference them in our
tickets.

 

Thanks everyone who replied, I will let you know the end result if there
is any :-)

 

Thank You,

 

 

 

Nancy Leugemors

Enterprise Systems

HealthNow, NY

716-887-7979

 

 

 

From:

Loon, EJ van - SPLXO eric-van.l...@klm.com

To:

ADSM-L@VM.MARIST.EDU

Date:

04/20/2011 04:50 AM

Subject:

Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code
Sent by:

ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU

 

 

 

Hi TSM-ers!

I too upgraded my servers to 5.5.5.2 and the log is still filling up to
fast. During last nights backup window, three incrementals were
triggered by the db backup trigger. My log is set to 12Gb with the
trigger at 75%.

I will open a PMR too for this and again, I urge everybody else too do
this too. I'm 100% convinced this excessive log utilization is a bug
introduced in the 5.5 code. I have never seen any db backup triggering
when we were using the 5.3 code.

Nancy, could you please post your PMR number to the list, so I can refer
to it in my PMR?

Thanks!!!

Kind regards,

Eric van Loon

KLM Royal Dutch Airlines

 

-Original Message-

From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Nancy L Leugemors

Sent: maandag 18 april 2011 16:34

To: ADSM-L@VM.MARIST.EDU

Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 

Thanks to everyone who replied with the not so wonderful news on the

performance

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Oscar Kolsteren]

Hi all,

 

I've seen exactly the same after upgrading to 5.5.5.2. I tried to
reschedule client and storage pool backups but still need to make at
least 2 incremental backups a night.

 

Raised PMR with IBM - 65823 999 866
https://www-946.ibm.com/support/servicerequest/problemDescriptionSelect
.action?userType=0srNumber=65823sourceAppl=XSRretainCountryCode=866b
ranch=999sourceNode=prNode5sourceTranId=558271303370048077draft=0 .

 

Hope this helps...

 

Best Regards,

 

Oscar Kolsteren

 

 

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Zoltan Forray/AC/VCU
Sent: 20 April 2011 17:38
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to
5.5.5.0 code

 

I went back in old emails and found mine from June 2010

 

PMR 93470,487,000

Zoltan Forray

TSM Software  Hardware Administrator

Virginia Commonwealth University

UCC/Office of Technology Services

zfor...@vcu.edu mailto:zfor...@vcu.edu  - 804-828-4807

Don't be a phishing victim - VCU and other reputable organizations will
never use email to request that you reply with your password, social
security number or confidential personal information. For more details
visit http://infosecurity.vcu.edu/phishing.html
http://infosecurity.vcu.edu/phishing.html 

 

 

 

From:

Loon, EJ van - SPLXO eric-van.l...@klm.com

To:

ADSM-L@VM.MARIST.EDU

Date:

04/20/2011 11:39 AM

Subject:

Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code
Sent by:

ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU

 

 

 

Hi Nancy!

Thanks for you PMR number, I will open one too tomorrow.

That's what support told Zoltan too, it's bull!!! We never had any
backup triggering when we were running 5.3, the problems started after
upgrading to 5.5, on all 3 servers. One of them cannot have a higher
load since we do not add additional clients on this server for more than
two years. The library is nearly full on this server.

Kind regards,

Eric van Loon

KLM Royal Dutch Airlines

 

-Original Message-

From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Nancy L Leugemors

Sent: woensdag 20 april 2011 16:20

To: ADSM-L@VM.MARIST.EDU

Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 

Eric,

 

My PMR is 32989,180.I too was told that my TSM server is very busy

and

maybe it's time to split it off into another instance of TSM.My log

pinned again yesterday on a Windows server this time where the other
times it was a database backups for different servers.

 

Please send out your  PMRs to the group so we can reference them in our
tickets.

 

Thanks everyone who replied, I will let you know the end result if there
is any :-)

 

Thank You,

 

 

 

Nancy Leugemors

Enterprise Systems

HealthNow, NY

716-887-7979

 

 

 

From:

Loon, EJ van - SPLXO eric-van.l...@klm.com

To:

ADSM-L@VM.MARIST.EDU

Date:

04/20/2011 04:50 AM

Subject:

Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code
Sent by:

ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU

 

 

 

Hi TSM-ers!

I too upgraded my servers to 5.5.5.2 and the log is still filling up to
fast. During last nights backup window, three incrementals were
triggered by the db backup trigger. My log is set to 12Gb with the
trigger at 75%.

I will open a PMR too for this and again, I urge everybody else too do
this too. I'm 100% convinced this excessive log utilization is a bug
introduced in the 5.5 code. I have never seen any db backup triggering
when we were using the 5.3 code.

Nancy, could you please post your PMR number to the list, so I can refer
to it in my PMR?

Thanks!!!

Kind regards,

Eric van Loon

KLM Royal Dutch Airlines

 

-Original Message-

From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Nancy L Leugemors

Sent: maandag 18 april 2011 16:34

To: ADSM-L@VM.MARIST.EDU

Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

 

Thanks to everyone who replied with the not so wonderful news on the

performance/recovery log issues.   I have an open case with IBM TSM

Support now and working on it with support now on it.   I will let you

know if I hear anything different once they review my output I sent them
over the weekend.

 

 

 

Nancy Leugemors

Enterprise Systems

HealthNow, NY

716-887-7979

 

 

 

From:

Wolfgang J Moeller moel...@gwdg.de

To:

ADSM-L@VM.MARIST.EDU

Date:

04/18/2011 09:46 AM

Subject:

Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code
Sent by:

ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU

 

 

 

On Apr 17, 2011, Nick Laflamme wrote:

[...]

 And 5.5.5.2 was just released; this has, among other things, a fix for

a

 bug that hangs TSM when you use automatic relabeling of scratch

volumes

 (ie, VTL users, usually) and MOVE DRM. I've had 5.5.5.2 on about eight


 servers that had been running 5.5.2.1 or 5.5.3.0, with no observed ill


 effects from

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Loon, EJ van - SPLXO]

Hi TSM-ers!
I too upgraded my servers to 5.5.5.2 and the log is still filling up to
fast. During last nights backup window, three incrementals were
triggered by the db backup trigger. My log is set to 12Gb with the
trigger at 75%.
I will open a PMR too for this and again, I urge everybody else too do
this too. I'm 100% convinced this excessive log utilization is a bug
introduced in the 5.5 code. I have never seen any db backup triggering
when we were using the 5.3 code.
Nancy, could you please post your PMR number to the list, so I can refer
to it in my PMR?
Thanks!!!
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Nancy L Leugemors
Sent: maandag 18 april 2011 16:34
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Thanks to everyone who replied with the not so wonderful news on the
performance/recovery log issues.   I have an open case with IBM TSM
Support now and working on it with support now on it.   I will let you
know if I hear anything different once they review my output I sent them
over the weekend.



Nancy Leugemors
Enterprise Systems
HealthNow, NY
716-887-7979



From:
Wolfgang J Moeller moel...@gwdg.de
To:
ADSM-L@VM.MARIST.EDU
Date:
04/18/2011 09:46 AM
Subject:
Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code
Sent by:
ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



On Apr 17, 2011, Nick Laflamme wrote:
[...]
 And 5.5.5.2 was just released; this has, among other things, a fix for
a
 bug that hangs TSM when you use automatic relabeling of scratch
volumes
 (ie, VTL users, usually) and MOVE DRM. I've had 5.5.5.2 on about eight
 servers that had been running 5.5.2.1 or 5.5.3.0, with no observed ill
 effects from the new code after more than a week.

Same here - running 5.5.5.2 since beginning of March (just for LTO-5
support).

Log max. util shows values slightly higher than 80% on 3 of 8 relevant
servers,
without any problems that we had noticed (with Logmode = Normal) -
that's
about
the same as under previous versions (that's 5.5.3.0 and 5.5.2.1).

So far (and that's since V3.7), the habit of installing server versions
from the patches hierarchy when possible, didn't hurt us here ;-).

Best regards,

 Wolfgang J. Moeller moel...@gwdg.de

Tel. +49 551 201-1516 ... not representing ... GWDG, Goettingen, Germany





CONFIDENTIALITY NOTICE: This email message and any attachments are for
the sole use of the intended recipient(s) and may contain proprietary,
confidential, trade secret or privileged information.  Any unauthorized
review, use, disclosure or distribution is prohibited and may be a
violation of law.  If you are not the intended recipient or a person
responsible for delivering this message to an intended recipient, please
contact the sender by reply email and destroy all copies of the original
message.

For information, services and offers, please visit our web site: 
http://www.klm.com. This e-mail and any attachment may contain confidential and 
privileged material intended for the addressee only. If you are not the 
addressee, you are notified that no part of the e-mail or any attachment may be 
disclosed, copied or distributed, and that any other action related to this 
e-mail or attachment is strictly prohibited, and may be unlawful. If you have 
received this e-mail by error, please notify the sender immediately by return 
e-mail, and delete this message. 

Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its 
employees shall not be liable for the incorrect or incomplete transmission of 
this e-mail or any attachments, nor responsible for any delay in receipt. 
Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch 
Airlines) is registered in Amstelveen, The Netherlands, with registered number 
33014286

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Nancy L Leugemors]

Eric,

My PMR is 32989,180.I too was told that my TSM server is very busy and
maybe it's time to split it off into another instance of TSM.My log
pinned again yesterday on a Windows server this time where the other times
it was a database backups for different servers.

Please send out your  PMRs to the group so we can reference them in our
tickets.

Thanks everyone who replied, I will let you know the end result if there
is any :-)

Thank You,



Nancy Leugemors
Enterprise Systems
HealthNow, NY
716-887-7979



From:
Loon, EJ van - SPLXO eric-van.l...@klm.com
To:
ADSM-L@VM.MARIST.EDU
Date:
04/20/2011 04:50 AM
Subject:
Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code
Sent by:
ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



Hi TSM-ers!
I too upgraded my servers to 5.5.5.2 and the log is still filling up to
fast. During last nights backup window, three incrementals were
triggered by the db backup trigger. My log is set to 12Gb with the
trigger at 75%.
I will open a PMR too for this and again, I urge everybody else too do
this too. I'm 100% convinced this excessive log utilization is a bug
introduced in the 5.5 code. I have never seen any db backup triggering
when we were using the 5.3 code.
Nancy, could you please post your PMR number to the list, so I can refer
to it in my PMR?
Thanks!!!
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Nancy L Leugemors
Sent: maandag 18 april 2011 16:34
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Thanks to everyone who replied with the not so wonderful news on the
performance/recovery log issues.   I have an open case with IBM TSM
Support now and working on it with support now on it.   I will let you
know if I hear anything different once they review my output I sent them
over the weekend.



Nancy Leugemors
Enterprise Systems
HealthNow, NY
716-887-7979



From:
Wolfgang J Moeller moel...@gwdg.de
To:
ADSM-L@VM.MARIST.EDU
Date:
04/18/2011 09:46 AM
Subject:
Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code
Sent by:
ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



On Apr 17, 2011, Nick Laflamme wrote:
[...]
 And 5.5.5.2 was just released; this has, among other things, a fix for
a
 bug that hangs TSM when you use automatic relabeling of scratch
volumes
 (ie, VTL users, usually) and MOVE DRM. I've had 5.5.5.2 on about eight
 servers that had been running 5.5.2.1 or 5.5.3.0, with no observed ill
 effects from the new code after more than a week.

Same here - running 5.5.5.2 since beginning of March (just for LTO-5
support).

Log max. util shows values slightly higher than 80% on 3 of 8 relevant
servers,
without any problems that we had noticed (with Logmode = Normal) -
that's
about
the same as under previous versions (that's 5.5.3.0 and 5.5.2.1).

So far (and that's since V3.7), the habit of installing server versions
from the patches hierarchy when possible, didn't hurt us here ;-).

Best regards,

 Wolfgang J. Moeller moel...@gwdg.de

Tel. +49 551 201-1516 ... not representing ... GWDG, Goettingen, Germany





CONFIDENTIALITY NOTICE: This email message and any attachments are for
the sole use of the intended recipient(s) and may contain proprietary,
confidential, trade secret or privileged information.  Any unauthorized
review, use, disclosure or distribution is prohibited and may be a
violation of law.  If you are not the intended recipient or a person
responsible for delivering this message to an intended recipient, please
contact the sender by reply email and destroy all copies of the original
message.

For information, services and offers, please visit our web site:
http://www.klm.com. This e-mail and any attachment may contain
confidential and privileged material intended for the addressee only. If
you are not the addressee, you are notified that no part of the e-mail or
any attachment may be disclosed, copied or distributed, and that any other
action related to this e-mail or attachment is strictly prohibited, and
may be unlawful. If you have received this e-mail by error, please notify
the sender immediately by return e-mail, and delete this message.

Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its
employees shall not be liable for the incorrect or incomplete transmission
of this e-mail or any attachments, nor responsible for any delay in
receipt.
Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch
Airlines) is registered in Amstelveen, The Netherlands, with registered
number 33014286

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Loon, EJ van - SPLXO]

Hi Nancy!
Thanks for you PMR number, I will open one too tomorrow.
That's what support told Zoltan too, it's bull!!! We never had any
backup triggering when we were running 5.3, the problems started after
upgrading to 5.5, on all 3 servers. One of them cannot have a higher
load since we do not add additional clients on this server for more than
two years. The library is nearly full on this server.
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Nancy L Leugemors
Sent: woensdag 20 april 2011 16:20
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Eric,

My PMR is 32989,180.I too was told that my TSM server is very busy
and
maybe it's time to split it off into another instance of TSM.My log
pinned again yesterday on a Windows server this time where the other
times
it was a database backups for different servers.

Please send out your  PMRs to the group so we can reference them in our
tickets.

Thanks everyone who replied, I will let you know the end result if there
is any :-)

Thank You,



Nancy Leugemors
Enterprise Systems
HealthNow, NY
716-887-7979



From:
Loon, EJ van - SPLXO eric-van.l...@klm.com
To:
ADSM-L@VM.MARIST.EDU
Date:
04/20/2011 04:50 AM
Subject:
Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code
Sent by:
ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



Hi TSM-ers!
I too upgraded my servers to 5.5.5.2 and the log is still filling up to
fast. During last nights backup window, three incrementals were
triggered by the db backup trigger. My log is set to 12Gb with the
trigger at 75%.
I will open a PMR too for this and again, I urge everybody else too do
this too. I'm 100% convinced this excessive log utilization is a bug
introduced in the 5.5 code. I have never seen any db backup triggering
when we were using the 5.3 code.
Nancy, could you please post your PMR number to the list, so I can refer
to it in my PMR?
Thanks!!!
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Nancy L Leugemors
Sent: maandag 18 april 2011 16:34
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Thanks to everyone who replied with the not so wonderful news on the
performance/recovery log issues.   I have an open case with IBM TSM
Support now and working on it with support now on it.   I will let you
know if I hear anything different once they review my output I sent them
over the weekend.



Nancy Leugemors
Enterprise Systems
HealthNow, NY
716-887-7979



From:
Wolfgang J Moeller moel...@gwdg.de
To:
ADSM-L@VM.MARIST.EDU
Date:
04/18/2011 09:46 AM
Subject:
Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code
Sent by:
ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



On Apr 17, 2011, Nick Laflamme wrote:
[...]
 And 5.5.5.2 was just released; this has, among other things, a fix for
a
 bug that hangs TSM when you use automatic relabeling of scratch
volumes
 (ie, VTL users, usually) and MOVE DRM. I've had 5.5.5.2 on about eight
 servers that had been running 5.5.2.1 or 5.5.3.0, with no observed ill
 effects from the new code after more than a week.

Same here - running 5.5.5.2 since beginning of March (just for LTO-5
support).

Log max. util shows values slightly higher than 80% on 3 of 8 relevant
servers,
without any problems that we had noticed (with Logmode = Normal) -
that's
about
the same as under previous versions (that's 5.5.3.0 and 5.5.2.1).

So far (and that's since V3.7), the habit of installing server versions
from the patches hierarchy when possible, didn't hurt us here ;-).

Best regards,

 Wolfgang J. Moeller moel...@gwdg.de

Tel. +49 551 201-1516 ... not representing ... GWDG, Goettingen, Germany





CONFIDENTIALITY NOTICE: This email message and any attachments are for
the sole use of the intended recipient(s) and may contain proprietary,
confidential, trade secret or privileged information.  Any unauthorized
review, use, disclosure or distribution is prohibited and may be a
violation of law.  If you are not the intended recipient or a person
responsible for delivering this message to an intended recipient, please
contact the sender by reply email and destroy all copies of the original
message.

For information, services and offers, please visit our web site:
http://www.klm.com. This e-mail and any attachment may contain
confidential and privileged material intended for the addressee only. If
you are not the addressee, you are notified that no part of the e-mail
or
any attachment may be disclosed, copied or distributed, and that any
other
action related to this e-mail or attachment is strictly prohibited, and
may be unlawful. If you have received this e-mail by error, please
notify
the sender

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Zoltan Forray/AC/VCU]

I went back in old emails and found mine from June 2010

PMR 93470,487,000
Zoltan Forray
TSM Software  Hardware Administrator
Virginia Commonwealth University
UCC/Office of Technology Services
zfor...@vcu.edu - 804-828-4807
Don't be a phishing victim - VCU and other reputable organizations will
never use email to request that you reply with your password, social
security number or confidential personal information. For more details
visit http://infosecurity.vcu.edu/phishing.html



From:
Loon, EJ van - SPLXO eric-van.l...@klm.com
To:
ADSM-L@VM.MARIST.EDU
Date:
04/20/2011 11:39 AM
Subject:
Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code
Sent by:
ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



Hi Nancy!
Thanks for you PMR number, I will open one too tomorrow.
That's what support told Zoltan too, it's bull!!! We never had any
backup triggering when we were running 5.3, the problems started after
upgrading to 5.5, on all 3 servers. One of them cannot have a higher
load since we do not add additional clients on this server for more than
two years. The library is nearly full on this server.
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Nancy L Leugemors
Sent: woensdag 20 april 2011 16:20
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Eric,

My PMR is 32989,180.I too was told that my TSM server is very busy
and
maybe it's time to split it off into another instance of TSM.My log
pinned again yesterday on a Windows server this time where the other
times
it was a database backups for different servers.

Please send out your  PMRs to the group so we can reference them in our
tickets.

Thanks everyone who replied, I will let you know the end result if there
is any :-)

Thank You,



Nancy Leugemors
Enterprise Systems
HealthNow, NY
716-887-7979



From:
Loon, EJ van - SPLXO eric-van.l...@klm.com
To:
ADSM-L@VM.MARIST.EDU
Date:
04/20/2011 04:50 AM
Subject:
Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code
Sent by:
ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



Hi TSM-ers!
I too upgraded my servers to 5.5.5.2 and the log is still filling up to
fast. During last nights backup window, three incrementals were
triggered by the db backup trigger. My log is set to 12Gb with the
trigger at 75%.
I will open a PMR too for this and again, I urge everybody else too do
this too. I'm 100% convinced this excessive log utilization is a bug
introduced in the 5.5 code. I have never seen any db backup triggering
when we were using the 5.3 code.
Nancy, could you please post your PMR number to the list, so I can refer
to it in my PMR?
Thanks!!!
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Nancy L Leugemors
Sent: maandag 18 april 2011 16:34
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Thanks to everyone who replied with the not so wonderful news on the
performance/recovery log issues.   I have an open case with IBM TSM
Support now and working on it with support now on it.   I will let you
know if I hear anything different once they review my output I sent them
over the weekend.



Nancy Leugemors
Enterprise Systems
HealthNow, NY
716-887-7979



From:
Wolfgang J Moeller moel...@gwdg.de
To:
ADSM-L@VM.MARIST.EDU
Date:
04/18/2011 09:46 AM
Subject:
Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code
Sent by:
ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



On Apr 17, 2011, Nick Laflamme wrote:
[...]
 And 5.5.5.2 was just released; this has, among other things, a fix for
a
 bug that hangs TSM when you use automatic relabeling of scratch
volumes
 (ie, VTL users, usually) and MOVE DRM. I've had 5.5.5.2 on about eight
 servers that had been running 5.5.2.1 or 5.5.3.0, with no observed ill
 effects from the new code after more than a week.

Same here - running 5.5.5.2 since beginning of March (just for LTO-5
support).

Log max. util shows values slightly higher than 80% on 3 of 8 relevant
servers,
without any problems that we had noticed (with Logmode = Normal) -
that's
about
the same as under previous versions (that's 5.5.3.0 and 5.5.2.1).

So far (and that's since V3.7), the habit of installing server versions
from the patches hierarchy when possible, didn't hurt us here ;-).

Best regards,

 Wolfgang J. Moeller moel...@gwdg.de

Tel. +49 551 201-1516 ... not representing ... GWDG, Goettingen, Germany





CONFIDENTIALITY NOTICE: This email message and any attachments are for
the sole use of the intended recipient(s) and may contain proprietary,
confidential, trade secret or privileged information.  Any unauthorized
review, use, disclosure or distribution is prohibited and may be a
violation of law.  If you are not the intended recipient or a person

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Paul van Dongen]

Hello, 

I don't know the behavior on 5.5.5 (we went straight to 5.5.5.2 because of the 
SQL SELECT bug), but on 5.5.4 we had a lot of issues with logs being pinned due 
to IC65781.

Problems have disappeared since the upgrade.


Paul

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Bob 
Booth
Sent: zondag 17 april 2011 16:17
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code

On Sat, Apr 16, 2011 at 11:41:13PM -0400, Nancy L Leugemors wrote:
 Hello,

 We just upgraded our TSM Server this  from 5.5.4.0 to 5.5.5.0  to fix APAR
  IC66116.   Since the server upgrade we have experienced  several recovery
 log pinning incidents from a few different database backup clients.   I
 haven't found any hits on new bugs related to our issue and I just opened
 up a case with support but was wondering if anyone had a similar issue
 with this level of server and client code or know of any APARS this might
 fit?   Our recovery log has always been set to 13GB so I can't expand it
 anymore. Running a database backup doesn't free any space while the
 client database backup is running either.   We run 1 full TSM database
 backup and multiple incrementals throughout the day.   The only difference
 is the upgrade to the latest code.   I'm not seeing any other errors, just
 showing different database clients pinning the log after issuing show log
 pinned command.   Sometimes the client backup finishes before hitting 80%
 and sometimes TSM server cancels the longest running backup that is
 pinning the log.

We are still at 5.5.4, and have seen more instances of log pins and database
lock conflicts, however, look to see if you have any of these factors:

Increased the size of your database, possibly to sub-optimal disks/controllers,
or RAID devices.

Longer running expirations, and or additions of systems that may have an
increased number of files (say in the millions).

Change in client levels on some nodes, due to compatibility problems (or some
other issue).  Mac's and TDP's come to mind. --

Our database is in an almost constant state of backup, as we use rollforward
recovery.  I would be most interested in anything that IBM support says, so
pass on the good word if you hear anything.  We are not quite ready for V6
yet, since we have to stand up new infrastructure, and get prepared for that
whole other set of problems.

The suggestion that you back your log off to 12GB is a good one, since you will 
very screwed if you fill up the log and fall over.

My questions for the list are this,

Would it help to get out of rollforward recovery mode and just to periodic
database backups at some standard hour?  I know the reason for rollforward, but
I'm willing to live with the losses if it fixes this problem.

Is there any way to stop the server from killing the oldest transaction?  It
never seems to be the one that is pinning the log, and all it does is force
the long running jobs to start over, usually making the issue worse.  We do
a lot of image backups which take several hours, and it would be nice to just
allow them to finish and get it over with.

Is there a fix for the error message about the log transaction delay?  I saw
an APAR that showed the 3ms text is bogus, and is actually 1s.   This should
have been fixed before my current level.

TSM: 5.5.4.0
AIX 5.3 - 11
TSM clients 5.5.X - 6.X
DB size 243GB
Log size 12GB

Good luck.

 example of error:
   29857)
 04/16/11   04:03:26  ANR0524W Transaction failed for session 24418 for
 node
   DEVNODE_API (SQL-BACKTRACK) -  data transfer
 interrupted.
   (SESSION: 24418)
 04/16/11   04:03:26  ANR2997W The server log is 81 percent full. The
 server
   will delay transactions by 3 milliseconds.
 (SESSION:
   29846)


 TSM Background:

 TSM Server:  5.5.5.0
 TSM OS:  AIX 5300 -12
 TSM Clients:  5.5.2.0
 TSM Client OS:  AIX 5.3 (UDB-DB2  SQL-Backtrack Sybase seen so far)

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Zoltan Forray/AC/VCU]

Been there - read the book - saw the movieetc.

We went through the same problem plus other issues with the log
stair-stepping  (more transactions that it could handle - things slowing
down until it could catch up - flooded with transactions again -
rinse-lather-repeat)  generating false log-full conditions all night long.

Spent numerous hours/days working with IBM who swore that there weren't
any changes that would effect this process and that my server was simply
overloaded.  I argued that I hadn't added any new nodes or processes to
the server having the problem.  They said we simply hit the saturation
wall and the only answer was to redistribute/reduce the workload.

I too setup cron jobs to monitor the transaction log % full values and
issues show logpinned commands and send me emails (wore out the
batteries in my BB).  Yes there were some Oracle backups causing some of
the problems (eventhough they hadn't changed in years) and I did get the
Oracle folks to split the backup transactions into chunks (this is in the
book) but the problems still persisted until I forced folks off this
server to another TSM server.  We also worked hard at redistributing the
scheduled backups to smooth out the peaks.  Eventually the problems
stopped.

Yes, what IBM told be to do fixed the problem but considering the
workload mix had not changed, I still argued that since the only change
was the TSM server level, there must be a problem in the server update.

I am somewhat heartened to see other folks experiencing the same issues I
went through and confirming that I was not going crazy.

Zoltan Forray
TSM Software  Hardware Administrator
Virginia Commonwealth University
UCC/Office of Technology Services
zfor...@vcu.edu - 804-828-4807
Don't be a phishing victim - VCU and other reputable organizations will
never use email to request that you reply with your password, social
security number or confidential personal information. For more details
visit http://infosecurity.vcu.edu/phishing.html



From:
Nancy L Leugemors leugemors.na...@healthnow.org
To:
ADSM-L@VM.MARIST.EDU
Date:
04/16/2011 11:42 PM
Subject:
[ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code
Sent by:
ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



Hello,

We just upgraded our TSM Server this  from 5.5.4.0 to 5.5.5.0  to fix APAR
 IC66116.   Since the server upgrade we have experienced  several recovery
log pinning incidents from a few different database backup clients.   I
haven't found any hits on new bugs related to our issue and I just opened
up a case with support but was wondering if anyone had a similar issue
with this level of server and client code or know of any APARS this might
fit?   Our recovery log has always been set to 13GB so I can't expand it
anymore. Running a database backup doesn't free any space while the
client database backup is running either.   We run 1 full TSM database
backup and multiple incrementals throughout the day.   The only difference
is the upgrade to the latest code.   I'm not seeing any other errors, just
showing different database clients pinning the log after issuing show log
pinned command.   Sometimes the client backup finishes before hitting 80%
and sometimes TSM server cancels the longest running backup that is
pinning the log.

example of error:
  29857)
04/16/11   04:03:26  ANR0524W Transaction failed for session 24418 for
node
  DEVNODE_API (SQL-BACKTRACK) -  data transfer
interrupted.
  (SESSION: 24418)
04/16/11   04:03:26  ANR2997W The server log is 81 percent full. The
server
  will delay transactions by 3 milliseconds.
(SESSION:
  29846)


TSM Background:

TSM Server:  5.5.5.0
TSM OS:  AIX 5300 -12
TSM Clients:  5.5.2.0
TSM Client OS:  AIX 5.3 (UDB-DB2  SQL-Backtrack Sybase seen so far)

Thank You,


Nancy Leugemors
Enterprise Systems
HealthNow, NY
716-887-7979

CONFIDENTIALITY NOTICE: This email message and any attachments are for the
sole use of the intended recipient(s) and may contain proprietary,
confidential, trade secret or privileged information.  Any unauthorized
review, use, disclosure or distribution is prohibited and may be a
violation of law.  If you are not the intended recipient or a person
responsible for delivering this message to an intended recipient, please
contact the sender by reply email and destroy all copies of the original
message.

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Wolfgang J Moeller]

On Apr 17, 2011, Nick Laflamme wrote:
[...]
 And 5.5.5.2 was just released; this has, among other things, a fix for a
 bug that hangs TSM when you use automatic relabeling of scratch volumes
 (ie, VTL users, usually) and MOVE DRM. I've had 5.5.5.2 on about eight
 servers that had been running 5.5.2.1 or 5.5.3.0, with no observed ill
 effects from the new code after more than a week.

Same here - running 5.5.5.2 since beginning of March (just for LTO-5 support).

Log max. util shows values slightly higher than 80% on 3 of 8 relevant 
servers,
without any problems that we had noticed (with Logmode = Normal) - that's about
the same as under previous versions (that's 5.5.3.0 and 5.5.2.1).

So far (and that's since V3.7), the habit of installing server versions
from the patches hierarchy when possible, didn't hurt us here ;-).

Best regards,

Wolfgang J. Moeller moel...@gwdg.de

Tel. +49 551 201-1516 ... not representing ... GWDG, Goettingen, Germany

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Nancy L Leugemors]

Thanks to everyone who replied with the not so wonderful news on the
performance/recovery log issues.   I have an open case with IBM TSM
Support now and working on it with support now on it.   I will let you
know if I hear anything different once they review my output I sent them
over the weekend.



Nancy Leugemors
Enterprise Systems
HealthNow, NY
716-887-7979



From:
Wolfgang J Moeller moel...@gwdg.de
To:
ADSM-L@VM.MARIST.EDU
Date:
04/18/2011 09:46 AM
Subject:
Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code
Sent by:
ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



On Apr 17, 2011, Nick Laflamme wrote:
[...]
 And 5.5.5.2 was just released; this has, among other things, a fix for a
 bug that hangs TSM when you use automatic relabeling of scratch volumes
 (ie, VTL users, usually) and MOVE DRM. I've had 5.5.5.2 on about eight
 servers that had been running 5.5.2.1 or 5.5.3.0, with no observed ill
 effects from the new code after more than a week.

Same here - running 5.5.5.2 since beginning of March (just for LTO-5
support).

Log max. util shows values slightly higher than 80% on 3 of 8 relevant
servers,
without any problems that we had noticed (with Logmode = Normal) - that's
about
the same as under previous versions (that's 5.5.3.0 and 5.5.2.1).

So far (and that's since V3.7), the habit of installing server versions
from the patches hierarchy when possible, didn't hurt us here ;-).

Best regards,

 Wolfgang J. Moeller moel...@gwdg.de

Tel. +49 551 201-1516 ... not representing ... GWDG, Goettingen, Germany





CONFIDENTIALITY NOTICE: This email message and any attachments are for the sole 
use of the intended recipient(s) and may contain proprietary, confidential, 
trade secret or privileged information.  Any unauthorized review, use, 
disclosure or distribution is prohibited and may be a violation of law.  If you 
are not the intended recipient or a person responsible for delivering this 
message to an intended recipient, please contact the sender by reply email and 
destroy all copies of the original
message.

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Loon, EJ van - SPLXO]

Oh my god!
I really thought I was the only one fighting the logpinning issue! My
database is filled up to 80% or more at least 20 times a day and all the
triggered incrementals have made me running out of scratches multiple
times. This very weekend our standby was called again because the
recovery log was full on one of our servers. A del volh todate=today-1
type=dbb gave me back 80 tapes, while we only keep 3 days of db backup!
About half a year ago I spoke with Zoltan on this very same issue, he
already had a case open with IBM and they kept on telling him this was a
load related issue. This was keeping me from opening a case too, but
after reading this thread, the first thing I'm going to do is upgrade
the servers from 5.5.4.0 to 5.5.5.2. If the logpinning is still there
(and I think it will be) I will open a case too. I urge everybody else
to do the same, because there really is something broken here. Please
post your case numer to the list, so we can all refer to each others
case.
Paul mentioned APAR IC65781, but this APAR mentions an issue when the
recovery log hits the 80% full condition. Before we upgraded our 5.3
servers to 5.5.4.0, we never even hit the 80%, not even once a day!
That's why I think 5.5.5 will probably keep me from filling up
completely, but it will not solve the excessive log usage and
incremental backup triggering.
Kind regards,
Eric van Loon

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Nancy L Leugemors
Sent: 18 April 2011 16:34
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

Thanks to everyone who replied with the not so wonderful news on the
performance/recovery log issues.   I have an open case with IBM TSM
Support now and working on it with support now on it.   I will let you
know if I hear anything different once they review my output I sent them
over the weekend.



Nancy Leugemors
Enterprise Systems
HealthNow, NY
716-887-7979



From:
Wolfgang J Moeller moel...@gwdg.de
To:
ADSM-L@VM.MARIST.EDU
Date:
04/18/2011 09:46 AM
Subject:
Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code
Sent by:
ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



On Apr 17, 2011, Nick Laflamme wrote:
[...]
 And 5.5.5.2 was just released; this has, among other things, a fix for
a
 bug that hangs TSM when you use automatic relabeling of scratch
volumes
 (ie, VTL users, usually) and MOVE DRM. I've had 5.5.5.2 on about eight
 servers that had been running 5.5.2.1 or 5.5.3.0, with no observed ill
 effects from the new code after more than a week.

Same here - running 5.5.5.2 since beginning of March (just for LTO-5
support).

Log max. util shows values slightly higher than 80% on 3 of 8 relevant
servers,
without any problems that we had noticed (with Logmode = Normal) -
that's
about
the same as under previous versions (that's 5.5.3.0 and 5.5.2.1).

So far (and that's since V3.7), the habit of installing server versions
from the patches hierarchy when possible, didn't hurt us here ;-).

Best regards,

 Wolfgang J. Moeller moel...@gwdg.de

Tel. +49 551 201-1516 ... not representing ... GWDG, Goettingen, Germany





CONFIDENTIALITY NOTICE: This email message and any attachments are for
the sole use of the intended recipient(s) and may contain proprietary,
confidential, trade secret or privileged information.  Any unauthorized
review, use, disclosure or distribution is prohibited and may be a
violation of law.  If you are not the intended recipient or a person
responsible for delivering this message to an intended recipient, please
contact the sender by reply email and destroy all copies of the original
message.

For information, services and offers, please visit our web site: 
http://www.klm.com. This e-mail and any attachment may contain confidential and 
privileged material intended for the addressee only. If you are not the 
addressee, you are notified that no part of the e-mail or any attachment may be 
disclosed, copied or distributed, and that any other action related to this 
e-mail or attachment is strictly prohibited, and may be unlawful. If you have 
received this e-mail by error, please notify the sender immediately by return 
e-mail, and delete this message. 

Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its 
employees shall not be liable for the incorrect or incomplete transmission of 
this e-mail or any attachments, nor responsible for any delay in receipt. 
Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch 
Airlines) is registered in Amstelveen, The Netherlands, with registered number 
33014286

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Nancy L Leugemors]

Sorry Paul, Just seeing your reply in the thread..

We had no issues with recovery log pinning at 5.5.4.0 and now are
experiencing them at 5.5.5.0,but will put in to upgrade the TSM Server to
go to .5.5.5.2 for the IC58852 next week.   I want to let this code run
this week and I need to finish our DD Clean Process to regain Data Domain
Space back.



Did fix for us in the 5.5.5.0 Code:

Going from 5.5.4.0 to 5.5.5.0 did fix the deadlock issues for us when you
ran a delete volhist=dbb and had the relabelscratch=yes turned on.


Tivoli Storage Manager server may encounter a deadlock situation
when delete volhistory and label libvol activity are
running at the same time.
Show output at the time of this problem reveals:
From show resqueue - waiters for type=8
From show locks - waiting for lock type=8



http://www-01.ibm.com/support/docview.wss?uid=swg1IC67947







Nancy Leugemors
Enterprise Systems
HealthNow, NY
716-887-7979



From:
Paul van Dongen paul.vandon...@vancis.nl
To:
ADSM-L@VM.MARIST.EDU
Date:
04/18/2011 03:33 AM
Subject:
Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0 code
Sent by:
ADSM: Dist Stor Manager ADSM-L@VM.MARIST.EDU



Hello,

I don't know the behavior on 5.5.5 (we went straight to 5.5.5.2 because of
the SQL SELECT bug), but on 5.5.4 we had a lot of issues with logs being
pinned due to IC65781.

Problems have disappeared since the upgrade.


Paul

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
Bob Booth
Sent: zondag 17 april 2011 16:17
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0
code

On Sat, Apr 16, 2011 at 11:41:13PM -0400, Nancy L Leugemors wrote:
 Hello,

 We just upgraded our TSM Server this  from 5.5.4.0 to 5.5.5.0  to fix
APAR
  IC66116.   Since the server upgrade we have experienced  several
recovery
 log pinning incidents from a few different database backup clients.   I
 haven't found any hits on new bugs related to our issue and I just
opened
 up a case with support but was wondering if anyone had a similar issue
 with this level of server and client code or know of any APARS this
might
 fit?   Our recovery log has always been set to 13GB so I can't expand it
 anymore. Running a database backup doesn't free any space while the
 client database backup is running either.   We run 1 full TSM database
 backup and multiple incrementals throughout the day.   The only
difference
 is the upgrade to the latest code.   I'm not seeing any other errors,
just
 showing different database clients pinning the log after issuing show
log
 pinned command.   Sometimes the client backup finishes before hitting
80%
 and sometimes TSM server cancels the longest running backup that is
 pinning the log.

We are still at 5.5.4, and have seen more instances of log pins and
database
lock conflicts, however, look to see if you have any of these factors:

Increased the size of your database, possibly to sub-optimal
disks/controllers,
or RAID devices.

Longer running expirations, and or additions of systems that may have an
increased number of files (say in the millions).

Change in client levels on some nodes, due to compatibility problems (or
some
other issue).  Mac's and TDP's come to mind. --

Our database is in an almost constant state of backup, as we use
rollforward
recovery.  I would be most interested in anything that IBM support says,
so
pass on the good word if you hear anything.  We are not quite ready for V6
yet, since we have to stand up new infrastructure, and get prepared for
that
whole other set of problems.

The suggestion that you back your log off to 12GB is a good one, since you
will very screwed if you fill up the log and fall over.

My questions for the list are this,

Would it help to get out of rollforward recovery mode and just to periodic
database backups at some standard hour?  I know the reason for
rollforward, but
I'm willing to live with the losses if it fixes this problem.

Is there any way to stop the server from killing the oldest transaction?
It
never seems to be the one that is pinning the log, and all it does is
force
the long running jobs to start over, usually making the issue worse.  We
do
a lot of image backups which take several hours, and it would be nice to
just
allow them to finish and get it over with.

Is there a fix for the error message about the log transaction delay?  I
saw
an APAR that showed the 3ms text is bogus, and is actually 1s.   This
should
have been fixed before my current level.

TSM: 5.5.4.0
AIX 5.3 - 11
TSM clients 5.5.X - 6.X
DB size 243GB
Log size 12GB

Good luck.

 example of error:
   29857)
 04/16/11   04:03:26  ANR0524W Transaction failed for session 24418
for
 node
   DEVNODE_API (SQL-BACKTRACK) -  data transfer
 interrupted.
   (SESSION: 24418)
 04/16/11   04:03:26  ANR2997W The server log is 81 percent full

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Steven Langdale]

Nancy

Not a fix for your problem, but I'd strongly urge you to reduce your log
from the 13GB max to maybe 12GB - reason being is that if you do hit the
situation where the log gets to 100% and the instance stops, the only way
to get it back is to do an emegency log growth. If your already at the
maximum, you will be stuck.

Steven

On Apr 17, 2011 4:41am, Nancy L Leugemors leugemors.na...@healthnow.org
wrote:


fit? Our recovery log has always been set to 13GB so I can't expand it
anymore. Running a database backup doesn't free any space while the

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Bob Booth]

On Sat, Apr 16, 2011 at 11:41:13PM -0400, Nancy L Leugemors wrote:
 Hello,

 We just upgraded our TSM Server this  from 5.5.4.0 to 5.5.5.0  to fix APAR
  IC66116.   Since the server upgrade we have experienced  several recovery
 log pinning incidents from a few different database backup clients.   I
 haven't found any hits on new bugs related to our issue and I just opened
 up a case with support but was wondering if anyone had a similar issue
 with this level of server and client code or know of any APARS this might
 fit?   Our recovery log has always been set to 13GB so I can't expand it
 anymore. Running a database backup doesn't free any space while the
 client database backup is running either.   We run 1 full TSM database
 backup and multiple incrementals throughout the day.   The only difference
 is the upgrade to the latest code.   I'm not seeing any other errors, just
 showing different database clients pinning the log after issuing show log
 pinned command.   Sometimes the client backup finishes before hitting 80%
 and sometimes TSM server cancels the longest running backup that is
 pinning the log.

We are still at 5.5.4, and have seen more instances of log pins and database
lock conflicts, however, look to see if you have any of these factors:

Increased the size of your database, possibly to sub-optimal disks/controllers,
or RAID devices.

Longer running expirations, and or additions of systems that may have an
increased number of files (say in the millions).

Change in client levels on some nodes, due to compatibility problems (or some
other issue).  Mac's and TDP's come to mind. --

Our database is in an almost constant state of backup, as we use rollforward
recovery.  I would be most interested in anything that IBM support says, so
pass on the good word if you hear anything.  We are not quite ready for V6
yet, since we have to stand up new infrastructure, and get prepared for that
whole other set of problems.

The suggestion that you back your log off to 12GB is a good one, since you will 
very screwed if you fill up the log and fall over.

My questions for the list are this,

Would it help to get out of rollforward recovery mode and just to periodic
database backups at some standard hour?  I know the reason for rollforward, but
I'm willing to live with the losses if it fixes this problem.

Is there any way to stop the server from killing the oldest transaction?  It
never seems to be the one that is pinning the log, and all it does is force
the long running jobs to start over, usually making the issue worse.  We do
a lot of image backups which take several hours, and it would be nice to just
allow them to finish and get it over with.

Is there a fix for the error message about the log transaction delay?  I saw
an APAR that showed the 3ms text is bogus, and is actually 1s.   This should
have been fixed before my current level.

TSM: 5.5.4.0
AIX 5.3 - 11
TSM clients 5.5.X - 6.X
DB size 243GB
Log size 12GB

Good luck.

 example of error:
   29857)
 04/16/11   04:03:26  ANR0524W Transaction failed for session 24418 for
 node
   DEVNODE_API (SQL-BACKTRACK) -  data transfer
 interrupted.
   (SESSION: 24418)
 04/16/11   04:03:26  ANR2997W The server log is 81 percent full. The
 server
   will delay transactions by 3 milliseconds.
 (SESSION:
   29846)


 TSM Background:

 TSM Server:  5.5.5.0
 TSM OS:  AIX 5300 -12
 TSM Clients:  5.5.2.0
 TSM Client OS:  AIX 5.3 (UDB-DB2  SQL-Backtrack Sybase seen so far)

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Thomas J]

We have seen the same log pinning issue once we upgraded from TSM 5.4.3 to
5.5.4.x. We have had a couple of TSM crashed due to log fill. The workaround
was to:
-set logmode normal
-scripts to monitor logs80% every 15 mins. and kill process like NDMP ba
stgp, to get log usage down
-get woken up in the middle of the night, thanks to BMC Patrol, if scripts
still don't do the job.

This is an irritating bug. Worked with TSM support and was told 5.5.5.0 code
would resolve this issue. We have just upgrade two of environments to
5.5.5.0 last week. I am not happy to see that 5.5.5.0 still has the log
pinning issue. I guess IBM is showing us the road to V6.

Thomas


-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Bob
Booth
Sent: Sunday, April 17, 2011 7:17 AM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to 5.5.5.0
code

On Sat, Apr 16, 2011 at 11:41:13PM -0400, Nancy L Leugemors wrote:
 Hello,

 We just upgraded our TSM Server this  from 5.5.4.0 to 5.5.5.0  to fix APAR
  IC66116.   Since the server upgrade we have experienced  several recovery
 log pinning incidents from a few different database backup clients.   I
 haven't found any hits on new bugs related to our issue and I just opened
 up a case with support but was wondering if anyone had a similar issue
 with this level of server and client code or know of any APARS this might
 fit?   Our recovery log has always been set to 13GB so I can't expand it
 anymore. Running a database backup doesn't free any space while the
 client database backup is running either.   We run 1 full TSM database
 backup and multiple incrementals throughout the day.   The only difference
 is the upgrade to the latest code.   I'm not seeing any other errors, just
 showing different database clients pinning the log after issuing show log
 pinned command.   Sometimes the client backup finishes before hitting 80%
 and sometimes TSM server cancels the longest running backup that is
 pinning the log.

We are still at 5.5.4, and have seen more instances of log pins and database
lock conflicts, however, look to see if you have any of these factors:

Increased the size of your database, possibly to sub-optimal
disks/controllers,
or RAID devices.

Longer running expirations, and or additions of systems that may have an
increased number of files (say in the millions).

Change in client levels on some nodes, due to compatibility problems (or
some
other issue).  Mac's and TDP's come to mind. --

Our database is in an almost constant state of backup, as we use rollforward
recovery.  I would be most interested in anything that IBM support says, so
pass on the good word if you hear anything.  We are not quite ready for V6
yet, since we have to stand up new infrastructure, and get prepared for that
whole other set of problems.

The suggestion that you back your log off to 12GB is a good one, since you
will very screwed if you fill up the log and fall over.

My questions for the list are this,

Would it help to get out of rollforward recovery mode and just to periodic
database backups at some standard hour?  I know the reason for rollforward,
but
I'm willing to live with the losses if it fixes this problem.

Is there any way to stop the server from killing the oldest transaction?  It
never seems to be the one that is pinning the log, and all it does is force
the long running jobs to start over, usually making the issue worse.  We do
a lot of image backups which take several hours, and it would be nice to
just
allow them to finish and get it over with.

Is there a fix for the error message about the log transaction delay?  I saw
an APAR that showed the 3ms text is bogus, and is actually 1s.   This should
have been fixed before my current level.

TSM: 5.5.4.0
AIX 5.3 - 11
TSM clients 5.5.X - 6.X
DB size 243GB
Log size 12GB

Good luck.

 example of error:
   29857)
 04/16/11   04:03:26  ANR0524W Transaction failed for session 24418 for
 node
   DEVNODE_API (SQL-BACKTRACK) -  data transfer
 interrupted.
   (SESSION: 24418)
 04/16/11   04:03:26  ANR2997W The server log is 81 percent full. The
 server
   will delay transactions by 3 milliseconds.
 (SESSION:
   29846)


 TSM Background:

 TSM Server:  5.5.5.0
 TSM OS:  AIX 5300 -12
 TSM Clients:  5.5.2.0
 TSM Client OS:  AIX 5.3 (UDB-DB2  SQL-Backtrack Sybase seen so far)

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Steve Harris]

For anyone looking to go to TSM server 5.5.5.0, be aware of a nasty bug
that requires SQL identifiers to be 18 characters or less in length and
may break your scripts.  IC71586 was fixed at 5.5.5.1

Regards

Steve

Steven Harris

TSM Admin,
Canberra Australia


On Sun, 17 Apr 2011 10:18:25 -0700, Thomas J tjaco...@gmail.com
wrote:

We have seen the same log pinning issue once we upgraded from TSM
5.4.3 to
5.5.4.x. We have had a couple of TSM crashed due to log fill. The
workaround
was to:
-set logmode normal
-scripts to monitor logs80% every 15 mins. and kill process like
NDMP ba
stgp, to get log usage down
-get woken up in the middle of the night, thanks to BMC Patrol, if
scripts
still don't do the job.

This is an irritating bug. Worked with TSM support and was told
5.5.5.0 code
would resolve this issue. We have just upgrade two of environments to
5.5.5.0 last week. I am not happy to see that 5.5.5.0 still has the
log
pinning issue. I guess IBM is showing us the road to V6.

Thomas


-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf
Of Bob
Booth
Sent: Sunday, April 17, 2011 7:17 AM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Recovery log is pinning since upgrade to
5.5.5.0
code

On Sat, Apr 16, 2011 at 11:41:13PM -0400, Nancy L Leugemors wrote:

Hello,

We just upgraded our TSM Server this  from 5.5.4.0 to 5.5.5.0  to
fix APAR
 IC66116.   Since the server upgrade we have experienced  several
recovery
log pinning incidents from a few different database backup clients.
I
haven't found any hits on new bugs related to our issue and I just
opened
up a case with support but was wondering if anyone had a similar
issue
with this level of server and client code or know of any APARS this
might
fit?   Our recovery log has always been set to 13GB so I can't
expand it
anymore. Running a database backup doesn't free any space while
the
client database backup is running either.   We run 1 full TSM
database
backup and multiple incrementals throughout the day.   The only
difference
is the upgrade to the latest code.   I'm not seeing any other
errors, just
showing different database clients pinning the log after issuing
show log
pinned command.   Sometimes the client backup finishes before
hitting 80%
and sometimes TSM server cancels the longest running backup that is
pinning the log.


We are still at 5.5.4, and have seen more instances of log pins and
database
lock conflicts, however, look to see if you have any of these
factors:

Increased the size of your database, possibly to sub-optimal
disks/controllers,
or RAID devices.

Longer running expirations, and or additions of systems that may have
an
increased number of files (say in the millions).

Change in client levels on some nodes, due to compatibility problems
(or
some
other issue).  Mac's and TDP's come to mind. --

Our database is in an almost constant state of backup, as we use
rollforward
recovery.  I would be most interested in anything that IBM support
says, so
pass on the good word if you hear anything.  We are not quite ready
for V6
yet, since we have to stand up new infrastructure, and get prepared
for that
whole other set of problems.

The suggestion that you back your log off to 12GB is a good one,
since you
will very screwed if you fill up the log and fall over.

My questions for the list are this,

Would it help to get out of rollforward recovery mode and just to
periodic
database backups at some standard hour?  I know the reason for
rollforward,
but
I'm willing to live with the losses if it fixes this problem.

Is there any way to stop the server from killing the oldest
transaction?  It
never seems to be the one that is pinning the log, and all it does is
force
the long running jobs to start over, usually making the issue worse.
We do
a lot of image backups which take several hours, and it would be nice
to
just
allow them to finish and get it over with.

Is there a fix for the error message about the log transaction delay?
I saw
an APAR that showed the 3ms text is bogus, and is actually 1s.   This
should
have been fixed before my current level.

TSM: 5.5.4.0
AIX 5.3 - 11
TSM clients 5.5.X - 6.X
DB size 243GB
Log size 12GB

Good luck.


example of error:
  29857)
04/16/11   04:03:26  ANR0524W Transaction failed for session
24418 for
node
  DEVNODE_API (SQL-BACKTRACK) -  data
transfer
interrupted.
  (SESSION: 24418)
04/16/11   04:03:26  ANR2997W The server log is 81 percent full.
The
server
  will delay transactions by 3 milliseconds.
(SESSION:
  29846)


TSM Background:

TSM Server:  5.5.5.0
TSM OS:  AIX 5300 -12
TSM Clients:  5.5.2.0
TSM Client OS:  AIX 5.3 (UDB-DB2  SQL-Backtrack Sybase seen so far)

Re: TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Nick Laflamme]

On Apr 17, 2011, at 6:08 PM, Steve Harris wrote:

 For anyone looking to go to TSM server 5.5.5.0, be aware of a nasty bug
 that requires SQL identifiers to be 18 characters or less in length and
 may break your scripts.  IC71586 was fixed at 5.5.5.1

And 5.5.5.2 was just released; this has, among other things, a fix for a bug 
that hangs TSM when you use automatic relabeling of scratch volumes (ie, VTL 
users, usually) and MOVE DRM. I've had 5.5.5.2 on about eight servers that had 
been running 5.5.2.1 or 5.5.3.0, with no observed ill effects from the new code 
after more than a week. 

 Regards
 
 Steve

Nick

TSM Recovery log is pinning since upgrade to 5.5.5.0 code

[Nancy L Leugemors]

Hello,

We just upgraded our TSM Server this  from 5.5.4.0 to 5.5.5.0  to fix APAR
 IC66116.   Since the server upgrade we have experienced  several recovery
log pinning incidents from a few different database backup clients.   I
haven't found any hits on new bugs related to our issue and I just opened
up a case with support but was wondering if anyone had a similar issue
with this level of server and client code or know of any APARS this might
fit?   Our recovery log has always been set to 13GB so I can't expand it
anymore. Running a database backup doesn't free any space while the
client database backup is running either.   We run 1 full TSM database
backup and multiple incrementals throughout the day.   The only difference
is the upgrade to the latest code.   I'm not seeing any other errors, just
showing different database clients pinning the log after issuing show log
pinned command.   Sometimes the client backup finishes before hitting 80%
and sometimes TSM server cancels the longest running backup that is
pinning the log.

example of error:
  29857)
04/16/11   04:03:26  ANR0524W Transaction failed for session 24418 for
node
  DEVNODE_API (SQL-BACKTRACK) -  data transfer
interrupted.
  (SESSION: 24418)
04/16/11   04:03:26  ANR2997W The server log is 81 percent full. The
server
  will delay transactions by 3 milliseconds.
(SESSION:
  29846)


TSM Background:

TSM Server:  5.5.5.0
TSM OS:  AIX 5300 -12
TSM Clients:  5.5.2.0
TSM Client OS:  AIX 5.3 (UDB-DB2  SQL-Backtrack Sybase seen so far)

Thank You,


Nancy Leugemors
Enterprise Systems
HealthNow, NY
716-887-7979

CONFIDENTIALITY NOTICE: This email message and any attachments are for the sole 
use of the intended recipient(s) and may contain proprietary, confidential, 
trade secret or privileged information.  Any unauthorized review, use, 
disclosure or distribution is prohibited and may be a violation of law.  If you 
are not the intended recipient or a person responsible for delivering this 
message to an intended recipient, please contact the sender by reply email and 
destroy all copies of the original
message.