Re: Recovery log filling up
Hi Zoltan! I too do suspect 5.5, I have seen this behavior gradually appear since the upgrade. When we were using 5.3, the only situation causing a full recovery log was a logpinning client session, in almost all cases caused by a broken network adapter of corrupted drivers on the client side. I see this behavior on all three servers. Since one of them certainly did not grow (the attached libraries are low in scratches, so no new clients have been added for a long time) so the load on this server hasn't increased for quite a while. So the argument of Support about stair-stepping due to too much activity cannot be valid for this server. I too will open a case about this problem next Monday, so please send me (offline) the case number you have opened, so I can refer to it. Together we might be able to get this past Level 2 support. Kind regards, Eric van Loon KLM Royal Dutch Airlines -Original Message- From: ADSM: Dist Stor Manager [mailto:ads...@vm.marist.edu] On Behalf Of Zoltan Forray/AC/VCU Sent: donderdag 12 augustus 2010 16:50 To: ADSM-L@VM.MARIST.EDU Subject: Re: Recovery log filling up AHA..Finally someone else with a similar experience! I have been experiencing the same problem ever since I upgrade a server to 5.5.4 from 5.5.3. I have an open/active ticket with IBM. I have log spiking to 96% and later dropping and yo-yo'ing all night long. IBM says it is the log stair-stepping due to too much activity. Most of the time, "show logpinned" doesn't show anything. A few times I had issues with Oracle backups being the pinner (yes, I saw the article about splitting the Oracle backups to smaller, pieces). Either way, my point is that I didn't have any of these problems until 5.5.4.x. IBM insists they haven't had any documentable/reproducible performance issues with 5.5.4 - mostly anecdotal like with me. I have worked to reduce backups (moved nodes to another server) and redistributed the schedules (at peaks, 150-sessions were active). Yet I still continue to have the log-spikes (I too have a cron running that emails me if the log >55%), albeit less frequently. Nothing else I can offer. Maybe if enough folks start reporting this to IBM, they can dig deeper to figure out what is going on. Zoltan Forray TSM Software & Hardware Administrator Virginia Commonwealth University UCC/Office of Technology Services zfor...@vcu.edu - 804-828-4807 Don't be a phishing victim - VCU and other reputable organizations will never use email to request that you reply with your password, social security number or confidential personal information. For more details visit http://infosecurity.vcu.edu/phishing.html From: "Loon, EJ van - SPLXO" To: ADSM-L@VM.MARIST.EDU Date: 08/12/2010 04:14 AM Subject: [ADSM-L] Recovery log filling up Sent by: "ADSM: Dist Stor Manager" Hi TSM-ers! This is my TSM shop: 3 TSM 5.5.4 servers, running on a AIX 5.3 cluster. Server 1 is receiving about 3 Tb of backup data daily, contains 458 client nodes, DB size is 58 Gb, logsize is 12 Gb. Server 2 is receiving about 3.5 Tb of backup data daily, contains 781 client nodes, DB size is 96 Gb, logsize is 12 Gb. Server 3 is receiving about 2.5 Tb of backup data daily, contains 392 client nodes, DB size is 58 Gb, logsize is 12 Gb. On all servers, the database a full database backup is scheduled in the morning, an incremental is scheduled at 18:00, before the majority of the clients start backing up. The strange thing is that on all three servers I see the recovery log filling up to more than 80 percent at least one time during the night. The backup trigger is set to 75%, but I see ANR2997W multiple times. Sometimes I even notice the following message: ANR4556W Attention: the database backup operation did not free sufficient recovery log space to lower utilization below the database backup trigger. The recovery log size may need to be increased or the database backup trigger parameters may need to be adjusted. I have an admin schedule running every hour issuing a show logpinned, but most of the time there are no logpinning client sessions. I even encountered multiple situations where the log filled up completely, forcing me to kill the dsmserv process and performing me a manual log extension. A 12 Gb logsize should be more then enough to survive one day, right? My servers are not that big, I have seen other configurations on this list much bigger. Does anybody have an idea why my recovery logs are filling up? Thank you very much for your help in advance! Kind regards, Eric van Loon For information, services and offers, please visit our web site: http://www.klm.com. This e-mail and any attachment may contain confidential and privileged material intended for the addressee only. If you are not the addressee, you are notified that no part of the e-mail or any attachment may be disclosed, copied
Re: Recovery log filling up
AHA..Finally someone else with a similar experience! I have been experiencing the same problem ever since I upgrade a server to 5.5.4 from 5.5.3. I have an open/active ticket with IBM. I have log spiking to 96% and later dropping and yo-yo'ing all night long. IBM says it is the log stair-stepping due to too much activity. Most of the time, "show logpinned" doesn't show anything. A few times I had issues with Oracle backups being the pinner (yes, I saw the article about splitting the Oracle backups to smaller, pieces). Either way, my point is that I didn't have any of these problems until 5.5.4.x. IBM insists they haven't had any documentable/reproducible performance issues with 5.5.4 - mostly anecdotal like with me. I have worked to reduce backups (moved nodes to another server) and redistributed the schedules (at peaks, 150-sessions were active). Yet I still continue to have the log-spikes (I too have a cron running that emails me if the log >55%), albeit less frequently. Nothing else I can offer. Maybe if enough folks start reporting this to IBM, they can dig deeper to figure out what is going on. Zoltan Forray TSM Software & Hardware Administrator Virginia Commonwealth University UCC/Office of Technology Services zfor...@vcu.edu - 804-828-4807 Don't be a phishing victim - VCU and other reputable organizations will never use email to request that you reply with your password, social security number or confidential personal information. For more details visit http://infosecurity.vcu.edu/phishing.html From: "Loon, EJ van - SPLXO" To: ADSM-L@VM.MARIST.EDU Date: 08/12/2010 04:14 AM Subject: [ADSM-L] Recovery log filling up Sent by: "ADSM: Dist Stor Manager" Hi TSM-ers! This is my TSM shop: 3 TSM 5.5.4 servers, running on a AIX 5.3 cluster. Server 1 is receiving about 3 Tb of backup data daily, contains 458 client nodes, DB size is 58 Gb, logsize is 12 Gb. Server 2 is receiving about 3.5 Tb of backup data daily, contains 781 client nodes, DB size is 96 Gb, logsize is 12 Gb. Server 3 is receiving about 2.5 Tb of backup data daily, contains 392 client nodes, DB size is 58 Gb, logsize is 12 Gb. On all servers, the database a full database backup is scheduled in the morning, an incremental is scheduled at 18:00, before the majority of the clients start backing up. The strange thing is that on all three servers I see the recovery log filling up to more than 80 percent at least one time during the night. The backup trigger is set to 75%, but I see ANR2997W multiple times. Sometimes I even notice the following message: ANR4556W Attention: the database backup operation did not free sufficient recovery log space to lower utilization below the database backup trigger. The recovery log size may need to be increased or the database backup trigger parameters may need to be adjusted. I have an admin schedule running every hour issuing a show logpinned, but most of the time there are no logpinning client sessions. I even encountered multiple situations where the log filled up completely, forcing me to kill the dsmserv process and performing me a manual log extension. A 12 Gb logsize should be more then enough to survive one day, right? My servers are not that big, I have seen other configurations on this list much bigger. Does anybody have an idea why my recovery logs are filling up? Thank you very much for your help in advance! Kind regards, Eric van Loon For information, services and offers, please visit our web site: http://www.klm.com. This e-mail and any attachment may contain confidential and privileged material intended for the addressee only. If you are not the addressee, you are notified that no part of the e-mail or any attachment may be disclosed, copied or distributed, and that any other action related to this e-mail or attachment is strictly prohibited, and may be unlawful. If you have received this e-mail by error, please notify the sender immediately by return e-mail, and delete this message.Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its employees shall not be liable for the incorrect or incomplete transmission of this e-mail or any attachments, nor responsible for any delay in receipt.Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch Airlines) is registered in Amstelveen, The Netherlands, with registered number 3014286
Re: Recovery log limit in TSM 6
Howdy, The maximum log size in TSM V6.1 is actually 128GB. This limitation is imposed by the log file size we use which is 512MB. In our testing and evaluation for TSM V6, we achieved better performance using 512MB log files which ends up limiting the maximum size at 128GB. This limitation could be changed such that the log file size used is 1GB which would result in a 256GB maximum log size but that would be for a future release or delivery of the product. Thanks, Colin - Colin Dawson TSM Server Development col...@us.ibm.com From: Remco Post To: ADSM-L@VM.MARIST.EDU Date: 06/03/2009 05:23 AM Subject:Re: [ADSM-L] Recovery log limit in TSM 6 On 3 jun 2009, at 14:14, Richard Mochnaczewski wrote: > Hi *, > > Does anyone know what the recovery log limit in TSM 6.0 is ? > for TSM 6.0 that is zero, since there is no such thing. For TSM 6.1 the active log can be 256 GB, and the log archive can not be larger than largest filesystem you can create on your system... > Rich -- Met vriendelijke groeten/Kind regards, Remco Post
Re: Recovery log limit in TSM 6
Actually, the 6.1 reference says the maximum size for the active log is 128GB, which matches up with what I remembered. The limit is based on the maximum number of active log volumes permitted by DB2 (256), and the fact that TSM specifies a log volume size of 512MB (0.5GB). Ken Hannigan Tivoli Storage Management Development hanni...@us.ibm.com "ADSM: Dist Stor Manager" wrote on 06/03/2009 05:21:32 AM: > On 3 jun 2009, at 14:14, Richard Mochnaczewski wrote: > > > Hi *, > > > > Does anyone know what the recovery log limit in TSM 6.0 is ? > > > > for TSM 6.0 that is zero, since there is no such thing. For TSM 6.1 > the active log can be 256 GB, and the log archive can not be larger > than largest filesystem you can create on your system... > > > Rich > > -- > > Met vriendelijke groeten/Kind regards, > > Remco Post
Re: Recovery log limit in TSM 6
On 3 jun 2009, at 14:14, Richard Mochnaczewski wrote: Hi *, Does anyone know what the recovery log limit in TSM 6.0 is ? for TSM 6.0 that is zero, since there is no such thing. For TSM 6.1 the active log can be 256 GB, and the log archive can not be larger than largest filesystem you can create on your system... Rich -- Met vriendelijke groeten/Kind regards, Remco Post
Re: Recovery Log Query
I have had this before run a *sh logpinned* this should show the process or session which pins the log. I would cancel that process or session as its slowing down the whole TSM server, once stopped check that *sh logpinned* has now gone, what can happen is another session or process can take its place, so once again cancel them until *sh logpinned* shows 'nothing is pinning the log' this can free the log up, if not run DB Backup and you should be back to normal. You are in tricky situation DONT let your log blow up its not fun. Good luck Mark Stapleton wrote: You likely have a backup session that is "pinning" the log in place until the backup completes. Create a TSM script that runs QUERY LOG >> every minute or so. Examine the file every day to see if the same sort of thing happens regularly. Note the timestamps, and discover what clients are backing up when the log starts to grow. Oh, and your company's email server spews an "address change" email to everyone who responds to ADSM-L emails. You might want to have your mail admin look into that. -- Mark Stapleton ([EMAIL PROTECTED]) CDW Berbee System engineer 7145 Boone Avenue North, Suite 140 Brooklyn Park MN 55428-1511 763-592-5963 www.berbee.com From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Jeff White I have a query regarding some unusual activity a couple of nights ago. Our recovery log is 12gb and is in NORNAL mode I ran a full database backup at 16:00 Saturday - that should clear the log down. During the night, the log appears to go to 80%. Or so the following messages indicated: 09-11-2008 05:18:59 ANR2997W The server log is 80 percent full. The server will delay transactions by 3 milliseconds. (SESSION: 1734316) 09-11-2008 05:26:51 ANR2997W The server log is 80 percent full. The server will delay transactions by 3 milliseconds. 09-11-2008 06:19:46 ANR2997W The server log is 80 percent full. The server will delay transactions by 3 milliseconds. (SESSION: 1744757) Then 09-11-2008 06:19:49 ANR2996I The server log is 5 percent full. The server is no longer delaying transactions.(SESSION: 1734370) The next full database did not run until approx. 16:00 on Sunday afternoon. So if the database backup did not clear the log to 5% used, what could have. There is no activity other than a few backups between the last ANR2997W message and the ANR2996I message. Thoughts? Jeff White
Re: Recovery Log Query
You likely have a backup session that is "pinning" the log in place until the backup completes. Create a TSM script that runs QUERY LOG >> every minute or so. Examine the file every day to see if the same sort of thing happens regularly. Note the timestamps, and discover what clients are backing up when the log starts to grow. Oh, and your company's email server spews an "address change" email to everyone who responds to ADSM-L emails. You might want to have your mail admin look into that. -- Mark Stapleton ([EMAIL PROTECTED]) CDW Berbee System engineer 7145 Boone Avenue North, Suite 140 Brooklyn Park MN 55428-1511 763-592-5963 www.berbee.com From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Jeff White > I have a query regarding some unusual activity a couple of nights ago. > > Our recovery log is 12gb and is in NORNAL mode > > I ran a full database backup at 16:00 Saturday - that should clear the > log down. During the night, the log appears to go to 80%. Or so the following > messages indicated: > > 09-11-2008 05:18:59 ANR2997W The server log is 80 percent full. > The > server will delay transactions by 3 milliseconds. (SESSION: 1734316) > 09-11-2008 05:26:51 ANR2997W The server log is 80 percent full. > The > server will delay transactions by 3 milliseconds. > 09-11-2008 06:19:46 ANR2997W The server log is 80 percent full. > The > server will delay transactions by 3 milliseconds. (SESSION: 1744757) > > Then > > 09-11-2008 06:19:49 ANR2996I The server log is 5 percent full. The > server is no longer delaying transactions.(SESSION: 1734370) > > The next full database did not run until approx. 16:00 on Sunday > afternoon. > So if the database backup did not clear the log to 5% used, what could > have. > There is no activity other than a few backups between the last ANR2997W > message and the ANR2996I message. > > Thoughts? > > Jeff White
Re: Recovery log removed
Your COMMmethod should be SHAREDMEM instead of SHAREDMEN. Jim -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Remco Post Sent: Sunday, October 05, 2008 8:04 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Recovery log removed dsmserv loadformat dsmserv restore db I'm sorry, there is no other way. On Oct 5, 2008, at 14:55 , Mario Behring wrote: > Hi all, > > I have this situation: the recovery log and disk storage pools > volumes were on a external storage. > > We had to remove the storage so we relocated all disk storage pools > volumes to other area, no problem here. The problem is, we forgot > about the recovery log...now, when I try to start TSM, by running > dsmserv at the cli, I get the following message: > > /opt/tivoli/tsm/server/bin # ./dsmserv > > Tivoli Storage Manager for Linux/i386 > Version 5, Release 2, Level 2.0 > > Licensed Materials - Property of IBM > > (C) Copyright IBM Corporation 1990, 2003. All rights > reserved. > U.S. Government Users Restricted Rights - Use, duplication > or disclosure > restricted by GSA ADP Schedule Contract with IBM > Corporation. > > ANR7800I DSMSERV generated at 08:54:48 on Dec 16 2003. > ANR7801I Subsystem process ID is 22058. > ANR0900I Processing options file dsmserv.opt. > ANR0902W Unsupported keyword found in file dsmserv.opt. > ANR0906I Line No. : 71 > ANR0907I Statement :COMMmethod SHAREDMEN > ANR0908W Error : ...|... > ANR0990I Server restart-recovery in progress. > ANR7806W Unable to open file /tsm_log4/log9. > Explanation (open error): Success > ANR0259E Unable to read complete restart/checkpoint > information from any > database or recovery log volume.How can I fix this? > > Mario -- Met vriendelijke groeten, Remco Post [EMAIL PROTECTED] +31 6 248 21 622
Re: Recovery log removed
These steps are the two major steps in restoring the tsm server database from the latest backup. You'll have to lookup both in the TSM administrators reference (section utilities) to find the syntax that is applicable to your environment. If you feel at any time unsure about restoring the TSM server db, please consult with a tsm specialist, the fact that you're asking what these commands do gives me the feeling that you've never been down this road before, putting you in an awkward spot of having to restore your TSM db in a production environment without ever having the opportunity to practice. You will loose any client data backed up after the last db backup. On Oct 5, 2008, at 15:11 , mbehring wrote: Hi, Please, what should happen when I issue those two commands? Will I loose any client data? + -- |This was sent by [EMAIL PROTECTED] via Backup Central. |Forward SPAM to [EMAIL PROTECTED] + -- -- Met vriendelijke groeten, Remco Post [EMAIL PROTECTED] +31 6 248 21 622
Re: Recovery log removed
dsmserv loadformat dsmserv restore db I'm sorry, there is no other way. On Oct 5, 2008, at 14:55 , Mario Behring wrote: Hi all, I have this situation: the recovery log and disk storage pools volumes were on a external storage. We had to remove the storage so we relocated all disk storage pools volumes to other area, no problem here. The problem is, we forgot about the recovery log...now, when I try to start TSM, by running dsmserv at the cli, I get the following message: /opt/tivoli/tsm/server/bin # ./dsmserv Tivoli Storage Manager for Linux/i386 Version 5, Release 2, Level 2.0 Licensed Materials - Property of IBM (C) Copyright IBM Corporation 1990, 2003. All rights reserved. U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corporation. ANR7800I DSMSERV generated at 08:54:48 on Dec 16 2003. ANR7801I Subsystem process ID is 22058. ANR0900I Processing options file dsmserv.opt. ANR0902W Unsupported keyword found in file dsmserv.opt. ANR0906I Line No. : 71 ANR0907I Statement :COMMmethod SHAREDMEN ANR0908W Error : ...|... ANR0990I Server restart-recovery in progress. ANR7806W Unable to open file /tsm_log4/log9. Explanation (open error): Success ANR0259E Unable to read complete restart/checkpoint information from any database or recovery log volume.How can I fix this? Mario -- Met vriendelijke groeten, Remco Post [EMAIL PROTECTED] +31 6 248 21 622
Re: Recovery Log Pinned
Same here, but our experience was a couple months ago.
Rick
"Park, Rod"
<[EMAIL PROTECTED]
OM>To
Sent by: "ADSM: ADSM-L@VM.MARIST.EDU
Dist Stor cc
Manager"
<[EMAIL PROTECTED] Subject
.EDU> Re: Recovery Log Pinned
02/07/2007 02:08
PM
Please respond to
"ADSM: Dist Stor
Manager"
<[EMAIL PROTECTED]
.EDU>
Kinda weird...i had the exact same thing happen yesterday and I've never
seen it before either. We had to halt/restart to fix the problem..
-Original Message-
From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of
Mark Stapleton
Sent: Wednesday, February 07, 2007 11:35 AM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] Recovery Log Pinned
From: ADSM: Dist Stor Manager on behalf of Choudarapu, Ramakrishna (GTI)
Looks like this pin is 'stubborn', as the successful completion of the
database backup for last three days, did not clear the log. AFAIK, this
has not happened before.
What would happen to the uncommitted transactions if the TSM service is
recycled?
[SNIP]
Can't say, although I've never lost any TSM db data when forced to run
HALT on the server; I suspect that forces a log commit as part of an
orderly shutdown.
If you've got time, I'd log a call with Tivoli support and set a ticket
for this. They may want to gather some data on the situation, time and
your situation permitting. This is the first "stubborn" log pin with no
obvious cause I've heard of since 5.3 came into being.
What size is your recovery log?
--
Mark Stapleton ([EMAIL PROTECTED])
Senior consultant
This email and any files transmitted with it are confidential and intended
solely for the use of the addressee. If you are not the intended addressee,
then you have received this email in error and any use, dissemination,
forwarding, printing, or copying of this email is strictly prohibited.
Please notify us immediately of your unintended receipt by reply and then
delete this email and your reply. Tyson Foods, Inc. and its subsidiaries
and affiliates will not be held liable to any person resulting from the
unintended or unauthorized use of any information contained in this email
or as a result of any additions or deletions of information originally
contained in this email.
-
The information contained in this message is intended only for the
personal and confidential use of the recipient(s) named above. If
the reader of this message is not the intended recipient or an
agent responsible for delivering it to the intended recipient, you
are hereby notified that you have received this document in error
and that any review, dissemination, distribution, or copying of
this message is strictly prohibited. If you have received this
communication in error, please notify us immediately, and delete
the original message.
Re: Recovery Log Pinned
Kinda weird...i had the exact same thing happen yesterday and I've never seen it before either. We had to halt/restart to fix the problem.. -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Mark Stapleton Sent: Wednesday, February 07, 2007 11:35 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Recovery Log Pinned From: ADSM: Dist Stor Manager on behalf of Choudarapu, Ramakrishna (GTI) Looks like this pin is 'stubborn', as the successful completion of the database backup for last three days, did not clear the log. AFAIK, this has not happened before. What would happen to the uncommitted transactions if the TSM service is recycled? [SNIP] Can't say, although I've never lost any TSM db data when forced to run HALT on the server; I suspect that forces a log commit as part of an orderly shutdown. If you've got time, I'd log a call with Tivoli support and set a ticket for this. They may want to gather some data on the situation, time and your situation permitting. This is the first "stubborn" log pin with no obvious cause I've heard of since 5.3 came into being. What size is your recovery log? -- Mark Stapleton ([EMAIL PROTECTED]) Senior consultant This email and any files transmitted with it are confidential and intended solely for the use of the addressee. If you are not the intended addressee, then you have received this email in error and any use, dissemination, forwarding, printing, or copying of this email is strictly prohibited. Please notify us immediately of your unintended receipt by reply and then delete this email and your reply. Tyson Foods, Inc. and its subsidiaries and affiliates will not be held liable to any person resulting from the unintended or unauthorized use of any information contained in this email or as a result of any additions or deletions of information originally contained in this email.
Re: Recovery Log Pinned
I might have missed it, but is there a long-running client operation in progress, or some other long-running process? That could be pinning the log. For example, a client backup of a very large file over a modem. The SHOW LOGPINNED command might shed some light (support would probably want that info anyway). Regards, Andy Andy Raibeck IBM Software Group Tivoli Storage Manager Client Development Internal Notes e-mail: Andrew Raibeck/Tucson/[EMAIL PROTECTED] Internet e-mail: [EMAIL PROTECTED] IBM Tivoli Storage Manager support web page: http://www.ibm.com/software/sysmgmt/products/support/IBMTivoliStorageManager.html The only dumb question is the one that goes unasked. The command line is your friend. "Good enough" is the enemy of excellence. "ADSM: Dist Stor Manager" wrote on 02/07/2007 10:34:50 AM: > From: ADSM: Dist Stor Manager on behalf of Choudarapu, Ramakrishna (GTI) > Looks like this pin is 'stubborn', as the successful completion of the > database backup for last three days, did not clear the log. AFAIK, this > has not happened before. > > What would happen to the uncommitted transactions if the TSM service is > recycled? > > [SNIP] > > Can't say, although I've never lost any TSM db data when forced to > run HALT on the server; I suspect that forces a log commit as part > of an orderly shutdown. > > If you've got time, I'd log a call with Tivoli support and set a > ticket for this. They may want to gather some data on the situation, > time and your situation permitting. This is the first "stubborn" log > pin with no obvious cause I've heard of since 5.3 came into being. > > What size is your recovery log? > > -- > Mark Stapleton ([EMAIL PROTECTED]) > Senior consultant
Re: Recovery Log Pinned
From: ADSM: Dist Stor Manager on behalf of Choudarapu, Ramakrishna (GTI) Looks like this pin is 'stubborn', as the successful completion of the database backup for last three days, did not clear the log. AFAIK, this has not happened before. What would happen to the uncommitted transactions if the TSM service is recycled? [SNIP] Can't say, although I've never lost any TSM db data when forced to run HALT on the server; I suspect that forces a log commit as part of an orderly shutdown. If you've got time, I'd log a call with Tivoli support and set a ticket for this. They may want to gather some data on the situation, time and your situation permitting. This is the first "stubborn" log pin with no obvious cause I've heard of since 5.3 came into being. What size is your recovery log? -- Mark Stapleton ([EMAIL PROTECTED]) Senior consultant
Re: Recovery Log Pinned
Mark, Looks like this pin is 'stubborn', as the successful completion of the database backup for last three days, did not clear the log. AFAIK, this has not happened before. What would happen to the uncommitted transactions if the TSM service is recycled? Regards, Rama -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Mark Stapleton Sent: Wednesday, February 07, 2007 12:12 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Recovery Log Pinned From: ADSM: Dist Stor Manager on behalf of Choudarapu, Ramakrishna (GTI) We had a recovery log pinning problem and the log was growing since last 4 days and currently is at 72% util... TSM Server 5.3.3.0 running on z/OS. Below is the SHOW LOGPINNED output: (snip) It looks like you're currently running a database backup; that should clear the log unless the pin is stubborn. If the db backup doesn't clear up your problem, you may want to consider taking the TSM service/process down and then restarting it. Has this happened before? -- Mark Stapleton ([EMAIL PROTECTED]) Senior consultant If you are not an intended recipient of this e-mail, please notify the sender, delete it and do not read, act upon, print, disclose, copy, retain or redistribute it. Click here for important additional terms relating to this e-mail. http://www.ml.com/email_terms/
Re: Recovery Log Pinned
From: ADSM: Dist Stor Manager on behalf of Choudarapu, Ramakrishna (GTI) We had a recovery log pinning problem and the log was growing since last 4 days and currently is at 72% util... TSM Server 5.3.3.0 running on z/OS. Below is the SHOW LOGPINNED output: (snip) It looks like you're currently running a database backup; that should clear the log unless the pin is stubborn. If the db backup doesn't clear up your problem, you may want to consider taking the TSM service/process down and then restarting it. Has this happened before? -- Mark Stapleton ([EMAIL PROTECTED]) Senior consultant
Re: Recovery Log volumes . . . how many?
I sugest you to read Performance tunning guide. There are some suggestions about number of DB and LOG files. I think that most important is to place DB, LOG and DISKPOOL files ot different HDDs (if it's possible). This will increase preformance. I think that TSM is using round robin to improve prerformance (if you have more than 1 file of some kind - on one disk - this will decrease performance). Richard Rhodes <[EMAIL PROTECTED]> Sent by: "ADSM: Dist Stor Manager" 17.01.2007 18:53 Please respond to "ADSM: Dist Stor Manager" To ADSM-L@VM.MARIST.EDU cc Subject Recovery Log volumes . . . how many? I was just reading a document from IBM's web site about TSM administration tips. One of the tips is to have 4-6 recovery log volumes for a large tsm server. What is the reasoning for this? I guess I've always pictured log processing as mostly a sequential write process, but then that might be Oracle knowledge getting in the way. Will having 4-6 log volumes help speed up log roll back on TSM startup after a crash with a fairly full log? One time we almost ran out of log space due to something pinning the log, but we couldn't figure out what was pinning it. The log had grown to 10gb by the time we finally decided to cycle TSM. On the way back up it took over an hour to roll back the log entries. Thanks! Rick - The information contained in this message is intended only for the personal and confidential use of the recipient(s) named above. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately, and delete the original message.
Re: Recovery log size
you can look back in the recovery log and see that the log filled at the point it abandoned the task. Nothing to be done except to increase the log size. >>> [EMAIL PROTECTED] 12/15/2005 1:30:03 PM >>> Hi, I´ve 42 clients configured and in one of them I´ve a missed status when i try to backup because of the following error: 12/09/2005 08:22:22 ANS1228E Sending of object '\\server01\c$\WINDOWS\msagent\agentctl.dll' failed 12/09/2005 08:22:22 ANS1316E The server does not have enough recovery log space to continue the current operation if I execute the commando q log f=d I get: Available Space (MB): 600 Assigned Capacity (MB): 600 Maximum Extension (MB): 0 Maximum Reduction (MB): 588 Page Size (bytes): 4,096 Total Usable Pages: 153,088 Used Pages: 1,900 Pct Util: 1.2 Max. Pct Util: 99.9 Physical Volumes: 4 Log Pool Pages: 128 Log Pool Pct. Util: 5.49 Log Pool Pct. Wait: 0.00 Cumulative Consumption (MB): 218,945.84 Consumption Reset Date/Time: 02/27/04 12:00:06 PM pct util is very low, how could I resolve the problem? thanks in advance. Marcelo
Re: Recovery Log Pinned
Vince, I forgot something else, if it is 2 of your servers illustrating the same behaviour, the offending operation could be a server to server function that you may be running. Leigh -A TSM cowboy with too much time on my hands- -Although if I want to keep getting work, I shouldn't refer to myself as a cowboy- -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Vincent RATAJSZCZAK Sent: 14 November 2005 10:27 To: ADSM-L@VM.MARIST.EDU Subject: [ADSM-L] Recovery Log Pinned Hi TSM Cow Boys/Girls I would like to abuse of your busy time. I discover that 2 of my several servers have a recovery Log pinned. I use the TSM/ADSM QuickFacts which point me to "IBM site article swg21054574" I'm unable to find this article on the net, neither the IBM site. Someone would be kind enough to send me this article or shortly explain me how to resolve a log pinned, please ? Regards Vince
Re: Recovery Log Pinned
Vince I'm not sure where you got that reference number from, but my bookmarked page of Richard's quick facts gives the following references. IBM Technotes: 1084167; 1105651; 1105830 I have just done a quick check on the IBM website and all 3 of the above are still available. I don't know the full specifics of your problem, but generally a recovery log gets pinned by a long running backup or long running process. If there are other high db transaction processes occurring at the same time, this is where your long starts to fill up and the 'concern' starts to set in. If you have a backup session running over a low bandwidth comms link and it is currently backing up a large file (ie Exchange database), this could pin the log. It could also be a session backing up over a 100Mbps link, but the link is not performing as it should (ie duplex mismatch) and therefore a large file backup is taking considerably longer and thus pinning the log. It could also be a process (migration, reclamation) that is transferring a single large file and it is taking longer than it should. (ie tape problem causing poor tape throughput). The command 'show logpinned' should give you the session id or process id of the offending operation. Cancel the session or process and the log will drop back down to a manageable amount. If you are unable to cancel the session or process, cancel any high db transaction processes that are adding to the log (ie expiration), this could give you breathing space. Lastly, if your log is not at the maximum size, you could consider extending it temporarily to allow the processes/sessions to complete, without the log filling. Hope that this is of some use. Leigh -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Vincent RATAJSZCZAK Sent: 14 November 2005 10:27 To: ADSM-L@VM.MARIST.EDU Subject: [ADSM-L] Recovery Log Pinned Hi TSM Cow Boys/Girls I would like to abuse of your busy time. I discover that 2 of my several servers have a recovery Log pinned. I use the TSM/ADSM QuickFacts which point me to "IBM site article swg21054574" I'm unable to find this article on the net, neither the IBM site. Someone would be kind enough to send me this article or shortly explain me how to resolve a log pinned, please ? Regards Vince
Re: Recovery log filling up quickly, too quickly. Ideas?
Unfortunately this keeps happening with different clients and the sessions can't be cancelled without taking down the entire server. Is there any way to force a session to end? They won't cancel and they never time out so the recovery log just keeps filling up as one of the sessions has the log pinned. This sounds like a bug to me but who knows at this point. I'm running AIX server v5311 and different levels of clients but most are 5300. John From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On=20 Behalf Of John C Dury >As several people suggested, what is happening is that I have=20 >at least one >client session that is seemingly hung and causing the recovery=20 >log to be >pinned. I've tried cancelling the session but it doesn't work and just >hangs. It never seems to timeout either. Any other way to cancel the >session without having to take the whole server down? My=20 >recovery log is >filling up again and this hung session is causing the recovery=20 >log to be >pinned. Any clues why the session wouldn't ever timeout or=20 >also fail to be cancelled? Not really. It could be a corrupt file (or filesystem) on the client. I would: 1. Take the offending client off of its schedule. 2. Stop and restart the TSM server service after all other activity is quiesced. 3. Carefully examine the dsmsched.log and dsmerror.log files to try to determine the offending bit. 4. Tell the offender's owner the problem and work together to come to a resolution. -- Mark Stapleton (stapleton AT berbee DOT com) IBM Certified Advanced Deployment Professional Tivoli Storage Management Solutions 2005 Office 262.521.5627 =20
Re: Recovery log filling up quickly, too quickly. Ideas?
From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of John C Dury >As several people suggested, what is happening is that I have >at least one >client session that is seemingly hung and causing the recovery >log to be >pinned. I've tried cancelling the session but it doesn't work and just >hangs. It never seems to timeout either. Any other way to cancel the >session without having to take the whole server down? My >recovery log is >filling up again and this hung session is causing the recovery >log to be >pinned. Any clues why the session wouldn't ever timeout or >also fail to be cancelled? Not really. It could be a corrupt file (or filesystem) on the client. I would: 1. Take the offending client off of its schedule. 2. Stop and restart the TSM server service after all other activity is quiesced. 3. Carefully examine the dsmsched.log and dsmerror.log files to try to determine the offending bit. 4. Tell the offender's owner the problem and work together to come to a resolution. -- Mark Stapleton ([EMAIL PROTECTED]) IBM Certified Advanced Deployment Professional Tivoli Storage Management Solutions 2005 Office 262.521.5627
Re: Recovery log filling up quickly, too quickly. Ideas?
As several people suggested, what is happening is that I have at least one client session that is seemingly hung and causing the recovery log to be pinned. I've tried cancelling the session but it doesn't work and just hangs. It never seems to timeout either. Any other way to cancel the session without having to take the whole server down? My recovery log is filling up again and this hung session is causing the recovery log to be pinned. Any clues why the session wouldn't ever timeout or also fail to be cancelled? J. - Forwarded by John C Dury/DLC on 05/09/2005 12:01 PM - John C Dury/DLC 05/05/2005 10:58 To AMADSM-L cc Subject Recovery log filling up quickly, too quickly. Ideas? I have v5311 AIX server installed. The DB is not in roll forward mode. There isn't anything that has changed recently that would be causing this. It seems it might be a bug in v5311 that doesn't let the recovery log empty as it happens sporadically. Expiration which runs daily is suddenly taking a much longer time to run also but also sporadically. I checked the number of objects per node in the DB to see if any have increased a significant amount which would cause expiration to run longer, but everything looks normal. My recovery log is 12296 MB(90% full) and my DB is 56480 MB (38.3% full). Any clues on what is going on here? How about a way to empty the recovery log? J.
Re: Recovery log filling up quickly, too quickly. Ideas?
>>You can empty your log with a db backup (incremental or full). Unless, as others have noted, you have pinned data.
Re: Recovery log filling up quickly, too quickly. Ideas?
From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of John C Dury >I have v5311 AIX server installed. The DB is not in roll forward mode. >There isn't anything that has changed recently that would be >causing this. >It seems it might be a bug in v5311 that doesn't let the recovery log >empty as it happens sporadically. Expiration which runs daily >is suddenly >taking a much longer time to run also but also sporadically. >I checked >the number of objects per node in the DB to see if any have increased a >significant amount which would cause expiration to run longer, but >everything looks normal. My recovery log is 12296 MB(90% >full) and my DB >is 56480 MB (38.3% full). Any clues on what is going on here? >How about a >way to empty the recovery log? You can empty your log with a db backup (incremental or full). -- Mark Stapleton ([EMAIL PROTECTED]) IBM Certified Advanced Deployment Professional Tivoli Storage Management Solutions 2005 Office 262.521.5627
Re: Recovery log filling up quickly, too quickly. Ideas?
John - You need to check recent ANE summary stats or dsmaccnt records for the clients and compare against past records, to see if there is actually an increase in client activity. Examining numbers for the server store inventory is rather pointless, as churn is what affects the Recovery Log. That is, a client may, per retention policies, always have 8 versions of a file in server storage, but if they're now re-sending it in every backup, there you have transaction activity and protracted Expirations. This is a classic administration challenge, wherein you may need to redistribute client backups over the day, or schedule incremental backups of your db. You can also perform periodic session queries from the server to see how backups are performing. And make sure your DBBackuptrigger is nicely configured, to avoid an ugly server problem. Richard Sims On May 5, 2005, at 10:58 AM, John C Dury wrote: I have v5311 AIX server installed. The DB is not in roll forward mode. There isn't anything that has changed recently that would be causing this. It seems it might be a bug in v5311 that doesn't let the recovery log empty as it happens sporadically. Expiration which runs daily is suddenly taking a much longer time to run also but also sporadically. I checked the number of objects per node in the DB to see if any have increased a significant amount which would cause expiration to run longer, but everything looks normal. My recovery log is 12296 MB(90% full) and my DB is 56480 MB (38.3% full). Any clues on what is going on here? How about a way to empty the recovery log? J.
Re: Recovery log filling up quickly, too quickly. Ideas?
You could try running the "show logpinned" command, you should be able to identify what is pinning the log. From: ADSM: Dist Stor Manager on behalf of John C Dury Sent: Thu 5/5/2005 10:58 AM To: ADSM-L@VM.MARIST.EDU Subject: [ADSM-L] Recovery log filling up quickly, too quickly. Ideas? I have v5311 AIX server installed. The DB is not in roll forward mode. There isn't anything that has changed recently that would be causing this. It seems it might be a bug in v5311 that doesn't let the recovery log empty as it happens sporadically. Expiration which runs daily is suddenly taking a much longer time to run also but also sporadically. I checked the number of objects per node in the DB to see if any have increased a significant amount which would cause expiration to run longer, but everything looks normal. My recovery log is 12296 MB(90% full) and my DB is 56480 MB (38.3% full). Any clues on what is going on here? How about a way to empty the recovery log? J. __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Re: recovery log utilization is too high
hi, i think you have to add another rec log volume or issue q log to see how large it is , and that issue extend log with smaller size goran - Original Message - From: "Roland Scharlau" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Friday, November 19, 2004 11:39 AM Subject: recovery log utilization is too high Hi all, since three days i get following message via : Server name: DESAETSM1, platform: Windows2000, version: 5.2.3.4, date/time: 11/19/2004 11:30:54 Issues and Recommendations -- The max recovery log utilization is too high. Condition (100.0 > 90) Recommendation: Ensure that TSM database backups are working properly. With "extend log 1000" on commandline i get : ANR2447E EXTEND LOG: Insufficient space to extend recovery log by requested amount. Many thanks for any help. Roland Scharlau Mail is virus-checked by Symantec Mail Security <<<
Re: recovery log filled up
Thank You I just tried it and we are up -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Justin Bleistein Sent: Wednesday, April 21, 2004 3:52 PM To: [EMAIL PROTECTED] Subject: Re: recovery log filled up # dsmfmt -log /path/newlogname 1000 (for 1 gig, make sure your in a fs which has enough space to accomadate 1 gig). # dsmserv extend log /path/newlogname 1000 after it's done formatting restart the tsm server: # nohup /usr/tivoli/tsm/server/bin/rc.adsmserv > /dev/null 2>1& note = you must run all commands from the tsm server install dir, or whereever your dsmserv.dsk file is located on the system. also note, if your recovery log is 13 gig you won't be able to extend it you'll have to restore and recover the tsm server database to recover and restart. thanks!. if you are using rlv's let me know and I'll type that up for you as well. --Justin Richard Bleistein |-+-> | | Ray Louvier | | | <[EMAIL PROTECTED]| | | BURTON.COM> | | | Sent by: "ADSM: | | | Dist Stor Manager"| | | <[EMAIL PROTECTED]| | | EDU> | | | | | | | | | 04/21/2004 04:49 | | | PM| | | Please respond to | | | "ADSM: Dist Stor | | | Manager" | | | | |-+-> >--- ---| | | | To: [EMAIL PROTECTED] | | cc: | | Subject: recovery log filled up | >--- ---| I have a recovery log that has filled and I cannot remember the command to extend it from command line start can someone help out. I am running 5.2.2.3 on AIX 5.2. Any help would be appreciated.
Re: recovery log filled up
# dsmfmt -log /path/newlogname 1000 (for 1 gig, make sure your in a fs which has enough space to accomadate 1 gig). # dsmserv extend log /path/newlogname 1000 after it's done formatting restart the tsm server: # nohup /usr/tivoli/tsm/server/bin/rc.adsmserv > /dev/null 2>1& note = you must run all commands from the tsm server install dir, or whereever your dsmserv.dsk file is located on the system. also note, if your recovery log is 13 gig you won't be able to extend it you'll have to restore and recover the tsm server database to recover and restart. thanks!. if you are using rlv's let me know and I'll type that up for you as well. --Justin Richard Bleistein |-+-> | | Ray Louvier | | | <[EMAIL PROTECTED]| | | BURTON.COM> | | | Sent by: "ADSM: | | | Dist Stor Manager"| | | <[EMAIL PROTECTED]| | | EDU> | | | | | | | | | 04/21/2004 04:49 | | | PM| | | Please respond to | | | "ADSM: Dist Stor | | | Manager" | | | | |-+-> >--| | | | To: [EMAIL PROTECTED] | | cc: | | Subject: recovery log filled up | >--| I have a recovery log that has filled and I cannot remember the command to extend it from command line start can someone help out. I am running 5.2.2.3 on AIX 5.2. Any help would be appreciated.
Re: recovery log question
The concept is the same for Recovery Log volumes as for DB volumes. David B. Longo System Administrator Health First, Inc. 3300 Fiske Blvd. Rockledge, FL 32955-4305 PH 321.434.5536 Pager 321.634.8230 Fax:321.434.5509 [EMAIL PROTECTED] >>> [EMAIL PROTECTED] 07/31/03 11:19PM >>> I want to migrate 4 smaller recovery log volumes to one big volume. If I create the new bigger volume and delete the smaller volumes will the data from the smaller volumes get copied into the bigger volume before deleting themselves from the TSM . I knew for sure this happens with the database volumes just need to confirm if the concept is same for recovery log volumes too ? appreciate your response. with regards, CAUTION - This message may contain privileged and confidential information intended only for the use of the addressee named above. If you are not the intended recipient of this message you are hereby notified that any use, dissemination, distribution or reproduction of this message is prohibited. If you have received this message in error please notify AMCOR immediately. Any views expressed in this message are those of the individual sender and may not necessarily reflect the views of AMCOR. ## This message is for the named person's use only. It may contain confidential, proprietary, or legally privileged information. No confidentiality or privilege is waived or lost by any mistransmission. If you receive this message in error, please immediately delete it and all copies of it from your system, destroy any hard copies of it, and notify the sender. You must not, directly or indirectly, use, disclose, distribute, print, or copy any part of this message if you are not the intended recipient. Health First reserves the right to monitor all e-mail communications through its networks. Any views or opinions expressed in this message are solely those of the individual sender, except (1) where the message states such views or opinions are on behalf of a particular entity; and (2) the sender is authorized by the entity to give such views or opinions. ##
Re: recovery log question
Yes, the data will be safely moved and TSM won't do the wrong thing during the delete of the log volumes. Bill Smoldt STORServer, Inc. -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] Behalf Of Ruksana Siddiqui Sent: Thursday, July 31, 2003 9:19 PM To: [EMAIL PROTECTED] Subject: recovery log question I want to migrate 4 smaller recovery log volumes to one big volume. If I create the new bigger volume and delete the smaller volumes will the data from the smaller volumes get copied into the bigger volume before deleting themselves from the TSM . I knew for sure this happens with the database volumes just need to confirm if the concept is same for recovery log volumes too ? appreciate your response. with regards, CAUTION - This message may contain privileged and confidential information intended only for the use of the addressee named above. If you are not the intended recipient of this message you are hereby notified that any use, dissemination, distribution or reproduction of this message is prohibited. If you have received this message in error please notify AMCOR immediately. Any views expressed in this message are those of the individual sender and may not necessarily reflect the views of AMCOR.
Re: Recovery log
Hi Mark, You are probably suffering from so called "log pinning" symptom. To make it short : log is working on a circular basis, and can only be freed if all transactions that where written into it are commited to TSM DB. If not done, you reach a point where the log bites it's own tail : older record are not freed, therefore making it impossible to create new ones, and the server dies. This is generally due to very long transactions, as for example huge or slow archives (probably your case). The antidote : extend your log, or modify Trougputdatathreshold and trougputtimethreshold options value in dsmserv.opt, to kill automatically the sessions having too low troughput during too much time. Another one would be switchin to "normal" logmode, if currently using rollforward, but this has it's own bad sides ... Anyway, do a search on topic "pinned log" in this list, and you'll find lots of responses to your question ! Hope it helped ... Arnaud =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= | Arnaud Brion, Panalpina Management Ltd., IT Group | | Viaduktstrasse 42, P.O. Box, 4002 Basel - Switzerland | | Phone: +41 61 226 19 78 / Fax: +41 61 226 17 01 | =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= -Original Message- From: Mark Hayden [mailto:[EMAIL PROTECTED]] Sent: Tuesday, 18 February, 2003 16:06 To: [EMAIL PROTECTED] Subject: Recovery log HI all, We have had some issues with our recovery log filling up during archives. Our Db is growing at an a alarming speed , about 82 % of a 41 Gb db. Our log is only a gig in size..Is this to small This seams to happen during remote archives across the Wan...Could this play a role? Thanks, Mark Hayden Informations Systems Analyst E-Mail: [EMAIL PROTECTED]
Re: Recovery log
Hello One gig is too small. I've had the same problem, when slow clients backed up alot of data it took several hours and the log began filling up. Regards Niklas -Original Message- From: Mark Hayden [mailto:[EMAIL PROTECTED]] Sent: den 18 februari 2003 16:06 To: [EMAIL PROTECTED] Subject: Recovery log HI all, We have had some issues with our recovery log filling up during archives. Our Db is growing at an a alarming speed , about 82 % of a 41 Gb db. Our log is only a gig in size..Is this to small This seams to happen during remote archives across the Wan...Could this play a role? Thanks, Mark Hayden Informations Systems Analyst E-Mail: [EMAIL PROTECTED]
Re: recovery log filling
Top of message >>--> 01-16-03 09:11 S.SHEPPARD (SHS)recovery log filling up r The Database Backup Trigger IS available on the OS/390 server. We've been using it for years. Command is: DEFine DBBackuptrigger DEVclass=device-class LOGFullpct=nn NUMincr=n Unfortunately, it is only used if the log is in ROLLFORWARD mode. Sam Sheppard San Diego Data Processing Corp. (858)-581-9668 ---` Top of message >>--> 01-15-03 04:03 ..NETMAIL () recovery log filling up r Date: Wed, 15 Jan 2003 07:01:33 -0500 From: "Joni Moyer" <[EMAIL PROTECTED]> Subject: recovery log filling up rapidly: Please help: EMERGENCY! To: [EMAIL PROTECTED] _Top_of_Message_ My recovery log is filling up at a rapid rate. I am running TSM at 4.1.3 on the mainframe. It is 4.6 GB and my DB is 48 GB with 46 GB in use. I am not running expiration of the inventory yet due to a previous deletion of a nodes data, could that be the cause? I noticed that the log is reaching about 95% full before I run another full backup. I usually run a full DB backup every day at 4:30. What can I do? I can't increase the recovery log because I guess on this version it can only go to 5 GB? What version is this no longer true for? I really need help and I can't find anyone from IBM to return my calls. My previous problem with the recovery log is no longer true. It does now reset back to 0% after a full backup, but I am running out of space. I do have the log fully extended and with the 5 GB limit I am stuck. Also, I can't set up a DB backup trigger because I am on the mainframe and that feature of TSM is not available. Thank you in advance for any help you can give me Joni Moyer Systems Programmer [EMAIL PROTECTED] (717)975-8338 ---`
Re: recovery log filling up rapidly: Please help: EMERGENCY!!!! !
One possibility, which I have seen at clients, is that the database is on
very slow disk.
Orville L. Lantto
Datatrend Technologies, Inc. (http://www.datatrend.com)
IBM Premier Business Partner
121 Cheshire Lane, Suite 700
Minnetonka, MN 55305
Email: [EMAIL PROTECTED]
CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, is
for the sole use of the intended recipient(s) and may contain confidential
and privileged information. Any unauthorized review, use, disclosure or
distribution is prohibited. If you are not the intended recipient, please
contact the sender by reply e-mail and destroy all copies of the original
message.
Raghu S <[EMAIL PROTECTED]>
Sent by: "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]>
01/16/2003 06:52 AM
Please respond to "ADSM: Dist Stor Manager"
To: [EMAIL PROTECTED]
cc:
Subject:Re: recovery log filling up rapidly: Please help:
EMERGENCY !
Hi Joni
Its strange to see recovery log filling up to 5GB level in NORMAL mode (
you mentioned that in ur previous mail).
In NORMALMODE recovery log doesn't keep transactions, it deletes
transaction as soon as it committed to database.Thats why only
Point-in-Retsore ( till the last consistent database backup) possible
with
recovery log in NORMAL mode.
You can't use database backup trigger with recovery log in NORMAL mode.
with TSM 5.x Recovery Log max size is 13GB.
I can't figure out the reason for this behaviour.Let me know if you could
find the reason.
Thanks
regds
Raghu
-Original Message-
From: Joni Moyer [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, January 15, 2003 7:02 AM
To: [EMAIL PROTECTED]
Subject: recovery log filling up rapidly: Please help: EMERGENCY!
My recovery log is filling up at a rapid rate. I am running TSM at 4.1.3
on
the mainframe. It is 4.6 GB and my DB is 48 GB with 46 GB in use. I am
not
running expiration of the inventory yet due to a previous deletion of a
nodes data, could that be the cause? I noticed that the log is reaching
about 95% full before I run another full backup. I usually run a full DB
backup every day at 4:30. What can I do? I can't increase the recovery
log
because I guess on this version it can only go to 5 GB? What version is
this no longer true for? I really need help and I can't find anyone from
IBM to return my calls. My previous problem with the recovery log is no
longer true. It does now reset back to 0% after a full backup, but I am
running out of space. I do have the log fully extended and with the 5 GB
limit I am stuck. Also, I can't set up a DB backup trigger because I am
on
the mainframe and that feature of TSM is not available. Thank you in
advance for any help you can give me
Joni Moyer
Systems Programmer
[EMAIL PROTECTED]
(717)975-8338
Re: recovery log filling up rapidly: Please help: EMERGENCY!!!! !
I've seen the log fill in Normal if you have a client running that's not sending commits often enough. Clients with NICs set to Autonegotiate, then ending up at 100/Half Duplex (or 10/Half duplex, a couple of times) will "pin" the log due to not getting to a commit point. Add in expiration on an active TSM server, and your log can fill up fast. Also, I believe the log doesn't commit during a database backup operation, so if that takes an extended period of time (wait for mount point, then large database to send), you can get the log pretty full. Nick Cassimatis Today is the tomorrow of yesterday.
Re: recovery log filling up rapidly: Please help: EMERGENCY!!!! !
Hi Joni Its strange to see recovery log filling up to 5GB level in NORMAL mode ( you mentioned that in ur previous mail). In NORMALMODE recovery log doesn't keep transactions, it deletes transaction as soon as it committed to database.Thats why only Point-in-Retsore ( till the last consistent database backup) possible with recovery log in NORMAL mode. You can't use database backup trigger with recovery log in NORMAL mode. with TSM 5.x Recovery Log max size is 13GB. I can't figure out the reason for this behaviour.Let me know if you could find the reason. Thanks regds Raghu -Original Message- From: Joni Moyer [mailto:[EMAIL PROTECTED]] Sent: Wednesday, January 15, 2003 7:02 AM To: [EMAIL PROTECTED] Subject: recovery log filling up rapidly: Please help: EMERGENCY! My recovery log is filling up at a rapid rate. I am running TSM at 4.1.3 on the mainframe. It is 4.6 GB and my DB is 48 GB with 46 GB in use. I am not running expiration of the inventory yet due to a previous deletion of a nodes data, could that be the cause? I noticed that the log is reaching about 95% full before I run another full backup. I usually run a full DB backup every day at 4:30. What can I do? I can't increase the recovery log because I guess on this version it can only go to 5 GB? What version is this no longer true for? I really need help and I can't find anyone from IBM to return my calls. My previous problem with the recovery log is no longer true. It does now reset back to 0% after a full backup, but I am running out of space. I do have the log fully extended and with the 5 GB limit I am stuck. Also, I can't set up a DB backup trigger because I am on the mainframe and that feature of TSM is not available. Thank you in advance for any help you can give me Joni Moyer Systems Programmer [EMAIL PROTECTED] (717)975-8338
Re: recovery log filling up rapidly: Please help: EMERGENCY!!!! !
Run an incremental to dump the log. Paul D. Seay, Jr. Technical Specialist Naptheon Inc. 757-688-8180 -Original Message- From: Joni Moyer [mailto:[EMAIL PROTECTED]] Sent: Wednesday, January 15, 2003 7:02 AM To: [EMAIL PROTECTED] Subject: recovery log filling up rapidly: Please help: EMERGENCY! My recovery log is filling up at a rapid rate. I am running TSM at 4.1.3 on the mainframe. It is 4.6 GB and my DB is 48 GB with 46 GB in use. I am not running expiration of the inventory yet due to a previous deletion of a nodes data, could that be the cause? I noticed that the log is reaching about 95% full before I run another full backup. I usually run a full DB backup every day at 4:30. What can I do? I can't increase the recovery log because I guess on this version it can only go to 5 GB? What version is this no longer true for? I really need help and I can't find anyone from IBM to return my calls. My previous problem with the recovery log is no longer true. It does now reset back to 0% after a full backup, but I am running out of space. I do have the log fully extended and with the 5 GB limit I am stuck. Also, I can't set up a DB backup trigger because I am on the mainframe and that feature of TSM is not available. Thank you in advance for any help you can give me Joni Moyer Systems Programmer [EMAIL PROTECTED] (717)975-8338
Re: recovery log filling up rapidly: Please help: EMERGENCY!!!!!
>...I usually run a full DB backup every day at 4:30. What can I do? ... First, determine what's eating your Recovery Log by doing periodic queries through the day of the Recovery Log, Processes, and Sessions. Judicious scheduling of the whoppers will help. Do Incremental backups of your TSM DB, which can be quick relievers of the Recovery Log. Look into using DBBackuptrigger.
Re: recovery log filling up rapidly: Please help: EMERGENCY!!!! !
Hi Joni, I have been there. I am on the Mainframe also. The DB backup trigger does work. I started to do an incremental DB backup just before the backup would start in the evening. It helped me out a lot. The version you are on 4.1.x has the log limit of 5GB. Version 4.2 and above it is 11GB. IF YOUR LOG FILLs YOU LOCK UP. Then you will have to extend your log, make sure NOTHING ELSE BY THE DB BACKUP RUNS, then let everything start up. I personally would stop absolutely everything but the DB backup right now! Do a CANCEL PROC on that deletion you have running. Stop the backups if you can, stop the migrations or any other services and pray that you have enough log space left to do the DB backup. MAtt -Original Message- From: Joni Moyer [mailto:[EMAIL PROTECTED]] Sent: Wednesday, January 15, 2003 7:02 AM To: [EMAIL PROTECTED] Subject:recovery log filling up rapidly: Please help: EMERGENCY! My recovery log is filling up at a rapid rate. I am running TSM at 4.1.3 on the mainframe. It is 4.6 GB and my DB is 48 GB with 46 GB in use. I am not running expiration of the inventory yet due to a previous deletion of a nodes data, could that be the cause? I noticed that the log is reaching about 95% full before I run another full backup. I usually run a full DB backup every day at 4:30. What can I do? I can't increase the recovery log because I guess on this version it can only go to 5 GB? What version is this no longer true for? I really need help and I can't find anyone from IBM to return my calls. My previous problem with the recovery log is no longer true. It does now reset back to 0% after a full backup, but I am running out of space. I do have the log fully extended and with the 5 GB limit I am stuck. Also, I can't set up a DB backup trigger because I am on the mainframe and that feature of TSM is not available. Thank you in advance for any help you can give me Joni Moyer Systems Programmer [EMAIL PROTECTED] (717)975-8338
Re: recovery log filling up rapidly: Please help: EMERGENCY!!!!!
no when you do not expiration you db grows but the log may grow with un-backed up databasr. try to backup your database more frequently or increase your db. or you may change your log type. Burak Demircan CEO / ITT MERCEDES-BENZ TURK A.S. [EMAIL PROTECTED] tel:+90 212 482 35 00 (4676) fax :+90 212 481 11 54 [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 15.01.2003 14:07 Please respond to ADSM-L To: [EMAIL PROTECTED] cc: Subject: recovery log filling up rapidly: Please help: EMERGENCY! My recovery log is filling up at a rapid rate. I am running TSM at 4.1.3 on the mainframe. It is 4.6 GB and my DB is 48 GB with 46 GB in use. I am not running expiration of the inventory yet due to a previous deletion of a nodes data, could that be the cause? I noticed that the log is reaching about 95% full before I run another full backup. I usually run a full DB backup every day at 4:30. What can I do? I can't increase the recovery log because I guess on this version it can only go to 5 GB? What version is this no longer true for? I really need help and I can't find anyone from IBM to return my calls. My previous problem with the recovery log is no longer true. It does now reset back to 0% after a full backup, but I am running out of space. I do have the log fully extended and with the 5 GB limit I am stuck. Also, I can't set up a DB backup trigger because I am on the mainframe and that feature of TSM is not available. Thank you in advance for any help you can give me Joni Moyer Systems Programmer [EMAIL PROTECTED] (717)975-8338
Re: recovery log mode
type q sys from the gui command line... "Mire, Nona" cc: Sent by: "ADSM: Subject: recovery log mode Dist Stor Manager" <[EMAIL PROTECTED] T.EDU> 11/12/2002 02:30 PM Please respond to "ADSM: Dist Stor Manager" We did an upgrade last week and I want to make sure my recovery log is set to rollforward and I can't find it to verify. The only option I see is to set the recovery log mode and when I bring this option up in gui it brings it up with the normal button marked - I've changed it to rollforward but it still brings it up with normal - so I think it's just a command with a default of normal set. Is there a command that I can run that will show me the current mode of the recovery log. AIX 5.1 TSM 4.2.1.9 Thanks
Re: recovery log mode
Q status will show the logmode. -Original Message- From: Mire, Nona [mailto:MireN@;LOURDESRMC.COM] Sent: November 12, 2002 3:30 PM To: [EMAIL PROTECTED] Subject: recovery log mode We did an upgrade last week and I want to make sure my recovery log is set to rollforward and I can't find it to verify. The only option I see is to set the recovery log mode and when I bring this option up in gui it brings it up with the normal button marked - I've changed it to rollforward but it still brings it up with normal - so I think it's just a command with a default of normal set. Is there a command that I can run that will show me the current mode of the recovery log. AIX 5.1 TSM 4.2.1.9 Thanks
Re: recovery log mode
Do a 'q stat' from command line. The web gui doesnt always update all shown information even if your update is accepted by the system/server. //Henrik "Mire, Nona" cc: (bcc: Henrik Wahlstedt) Sent by: Subject: recovery log mode "ADSM: Dist Stor Manager" <[EMAIL PROTECTED] RIST.EDU> 2002-11-12 22:30 Please respond to "ADSM: Dist Stor Manager" We did an upgrade last week and I want to make sure my recovery log is set to rollforward and I can't find it to verify. The only option I see is to set the recovery log mode and when I bring this option up in gui it brings it up with the normal button marked - I've changed it to rollforward but it still brings it up with normal - so I think it's just a command with a default of normal set. Is there a command that I can run that will show me the current mode of the recovery log. AIX 5.1 TSM 4.2.1.9 Thanks --- The information contained in this message may be CONFIDENTIAL and is intended for the addressee only. Any unauthorised use, dissemination of the information or copying of this message is prohibited. If you are not the addressee, please notify the sender immediately by return e-mail and delete this message. Thank you.
Re: recovery log filling
Joe, I've noticed that no one responded to your post and since I have had some experience with recovery log filling with TSM 4.1 and normal mode, I thought I would respond. My experience is that the TSM log can fill with long running clients. In our shop, particularly OS2 and VMS clients will cause the recovery log to fill. It seems that all clients, upon beginning a backup session, place a 'peg' in the recovery log. I am not sure what the purpose of this is, but I assume it is for 'rollback' purposes. Anyhow, any and all database update transactions are then trapped in the recovery log until the oldest 'peg' is cleared by successful completion or failure. So, if you have lnng running (poor network connection?) clients, they can cause you problems. We use an AIX server and with TSM 4.1.x.x our recover log of 5GB would fill at least 5 times a week and crash the server. Upgrade to TSM 4.2 (it is supported and 4.1 is not!) and your recovery log can be expanded to 13GB. This might get you over the hump. Because of our diversity and the fact that we have looonnng running backup clients, I have never been able to enjoy the success of roll-forward recovery at this installation. Hope this helps, John G. Talafous IS Technical Principal The Timken CompanyGlobal Software Support P.O. Box 6927 Data Management 1835 Dueber Ave. S.W. Phone: (330)-471-3390 Canton, Ohio USA 44706-0927 Fax : (330)-471-4034 [EMAIL PROTECTED] http://www.timken.com -Original Message- From: Wholey, Joseph (TGA\MLOL) [mailto:[EMAIL PROTECTED]] Sent: Thursday, July 25, 2002 11:01 AM To: [EMAIL PROTECTED] Subject: recovery log filling Environment: TSM SERVER: 4.1.3.0 S/390 We're currently in the process of consolidating 4 TSM servers into 1. Over the course of about 3 weeks we've been redirecting clients to the new server. Originally this new TSM server was in logmode roll forward. As the volume to this server increased (incrementals and base incrementals running nightly), our recovery log could not handle the volume. We changed logmode to normal. Last night this server processed 1.4 million files. Backups started to fail due to a recovery log full condition. On prior nights this server's processed well over 2 million files and the recovery log did not fill. What's the determining factor? Also, in normal mode, what determines when the recovery log clears? Regards, Joe ** This message and any attachments are intended for the individual or entity named above. If you are not the intended recipient, please do not forward, copy, print, use or disclose this communication to others; also please notify the sender by replying to this message, and then delete it from your system. The Timken Company **
Re: Recovery log space problem
Thanks for the reply. Mine is NORMAL mode. When I received space problem message, my operation did not abort, neither backup failed. It was Level 0 backup of Informix using onbar was running and of course continuous logical logs are backed up as and when they are full. This error has been noticed for the first time since last two years. Do you see still increasing size is recommended Thanks & regards Sachin -Original Message- From: Bill Boyer [mailto:[EMAIL PROTECTED]] Sent: Thursday, June 27, 2002 11:54 PM To: [EMAIL PROTECTED] Subject: Re: Recovery log space problem Only in ROLLFORWARD mode. In NORMAL mode the log is used to hold the current transaction(s) and when they are committed to the database are removed from the log. He doesn't indicate what mode he's using. If this is the only thing the TSM server is doing at the time, a single TDP Informix backup, then I would look to see what the backup is doing. Maybe expiring a lot of backup objects? He also doesn't say whether this has worked and then after a while received this error, or that it just never worked without getting the log full messages. Depending on the size of his DB, I would concider increasing the LOG space anyway. You don't want your important backups to fail because of a log full problem. Bill Boyer DSS, Inc. -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of Ran Harel Sent: Thursday, June 27, 2002 8:41 AM To: [EMAIL PROTECTED] Subject: Re: Recovery log space problem Hi. The log gets filled up all the time. When you backup the TSM database, it gets empty. So, you should define a backup trigger for the database ( Admin Guide ) and consider increasing the size of the log in case it gets filled up too soon. Ran. -Original Message- From: Ganu Sachin, IBM [mailto:[EMAIL PROTECTED]] Sent: Monday, June 24, 2002 8:23 AM To: [EMAIL PROTECTED] Subject: Recovery log space problem Hi We have TSM server 3.7.0 with TDP for Informix 3.7. Yesterday we got following warning messages (blue font). 06/23/02 04:06:43 ANR0314W Recovery log usage exceeds 90 % of its assigned capacity. 06/23/02 04:06:51 ANR0314W Recovery log usage exceeds 92 % of its assigned capacity. 06/23/02 04:07:01 ANR0314W Recovery log usage exceeds 94 % of its assigned capacity. 06/23/02 04:07:09 ANR0314W Recovery log usage exceeds 96 % of its assigned capacity. 06/23/02 04:07:16 ANR0314W Recovery log usage exceeds 98 % of its assigned capacity. 06/23/02 04:07:18 ANR0356I Recovery log compression started. 06/23/02 04:07:18 ANR0357I Recovery log compression ended. 06/23/02 04:07:42 ANR0400I Session 12186 started for node BRAMHA (TDP Infmx AIX42) (ShMem). 06/23/02 04:07:44 ANR0403I Session 12186 ended for node BRAMHA (TDP Infmx AIX42). 06/23/02 04:09:03 ANR0400I Session 12187 started for node BRAMHA (TDP Infmx AIX42) (ShMem). 06/23/02 04:09:06 ANR0403I Session 12187 ended for node BRAMHA (TDP Infmx AIX42). 06/23/02 04:12:40 ANR0314W Recovery log usage exceeds 90 % of its assigned capacity. 06/23/02 04:12:46 ANR0314W Recovery log usage exceeds 92 % of its assigned capacity. Output of TSM > query log is as follows Available Assigned Maximum MaximumPage Total Used Pct Max. Space Capacity Extension ReductionSizeUsable Pages Util Pct (MB) (MB) (MB) (MB) (bytes) Pages Util - - - --- - - - - 200 200 0 196 4,09650,688 254 0.5 99.6 Is it really a space problem or what could be reason ? Thanks & regards Sachin Ganu IBM Global Services (I) Pvt. Ltd.
Re: Recovery log space problem
Only in ROLLFORWARD mode. In NORMAL mode the log is used to hold the current transaction(s) and when they are committed to the database are removed from the log. He doesn't indicate what mode he's using. If this is the only thing the TSM server is doing at the time, a single TDP Informix backup, then I would look to see what the backup is doing. Maybe expiring a lot of backup objects? He also doesn't say whether this has worked and then after a while received this error, or that it just never worked without getting the log full messages. Depending on the size of his DB, I would concider increasing the LOG space anyway. You don't want your important backups to fail because of a log full problem. Bill Boyer DSS, Inc. -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of Ran Harel Sent: Thursday, June 27, 2002 8:41 AM To: [EMAIL PROTECTED] Subject: Re: Recovery log space problem Hi. The log gets filled up all the time. When you backup the TSM database, it gets empty. So, you should define a backup trigger for the database ( Admin Guide ) and consider increasing the size of the log in case it gets filled up too soon. Ran. -Original Message- From: Ganu Sachin, IBM [mailto:[EMAIL PROTECTED]] Sent: Monday, June 24, 2002 8:23 AM To: [EMAIL PROTECTED] Subject: Recovery log space problem Hi We have TSM server 3.7.0 with TDP for Informix 3.7. Yesterday we got following warning messages (blue font). 06/23/02 04:06:43 ANR0314W Recovery log usage exceeds 90 % of its assigned capacity. 06/23/02 04:06:51 ANR0314W Recovery log usage exceeds 92 % of its assigned capacity. 06/23/02 04:07:01 ANR0314W Recovery log usage exceeds 94 % of its assigned capacity. 06/23/02 04:07:09 ANR0314W Recovery log usage exceeds 96 % of its assigned capacity. 06/23/02 04:07:16 ANR0314W Recovery log usage exceeds 98 % of its assigned capacity. 06/23/02 04:07:18 ANR0356I Recovery log compression started. 06/23/02 04:07:18 ANR0357I Recovery log compression ended. 06/23/02 04:07:42 ANR0400I Session 12186 started for node BRAMHA (TDP Infmx AIX42) (ShMem). 06/23/02 04:07:44 ANR0403I Session 12186 ended for node BRAMHA (TDP Infmx AIX42). 06/23/02 04:09:03 ANR0400I Session 12187 started for node BRAMHA (TDP Infmx AIX42) (ShMem). 06/23/02 04:09:06 ANR0403I Session 12187 ended for node BRAMHA (TDP Infmx AIX42). 06/23/02 04:12:40 ANR0314W Recovery log usage exceeds 90 % of its assigned capacity. 06/23/02 04:12:46 ANR0314W Recovery log usage exceeds 92 % of its assigned capacity. Output of TSM > query log is as follows Available Assigned Maximum MaximumPage Total Used Pct Max. Space Capacity Extension ReductionSizeUsable Pages Util Pct (MB) (MB) (MB) (MB) (bytes) Pages Util - - - --- - - - - 200 200 0 196 4,09650,688 254 0.5 99.6 Is it really a space problem or what could be reason ? Thanks & regards Sachin Ganu IBM Global Services (I) Pvt. Ltd.
Re: Recovery log space problem
Hi. The log gets filled up all the time. When you backup the TSM database, it gets empty. So, you should define a backup trigger for the database ( Admin Guide ) and consider increasing the size of the log in case it gets filled up too soon. Ran. -Original Message- From: Ganu Sachin, IBM [mailto:[EMAIL PROTECTED]] Sent: Monday, June 24, 2002 8:23 AM To: [EMAIL PROTECTED] Subject: Recovery log space problem Hi We have TSM server 3.7.0 with TDP for Informix 3.7. Yesterday we got following warning messages (blue font). 06/23/02 04:06:43 ANR0314W Recovery log usage exceeds 90 % of its assigned capacity. 06/23/02 04:06:51 ANR0314W Recovery log usage exceeds 92 % of its assigned capacity. 06/23/02 04:07:01 ANR0314W Recovery log usage exceeds 94 % of its assigned capacity. 06/23/02 04:07:09 ANR0314W Recovery log usage exceeds 96 % of its assigned capacity. 06/23/02 04:07:16 ANR0314W Recovery log usage exceeds 98 % of its assigned capacity. 06/23/02 04:07:18 ANR0356I Recovery log compression started. 06/23/02 04:07:18 ANR0357I Recovery log compression ended. 06/23/02 04:07:42 ANR0400I Session 12186 started for node BRAMHA (TDP Infmx AIX42) (ShMem). 06/23/02 04:07:44 ANR0403I Session 12186 ended for node BRAMHA (TDP Infmx AIX42). 06/23/02 04:09:03 ANR0400I Session 12187 started for node BRAMHA (TDP Infmx AIX42) (ShMem). 06/23/02 04:09:06 ANR0403I Session 12187 ended for node BRAMHA (TDP Infmx AIX42). 06/23/02 04:12:40 ANR0314W Recovery log usage exceeds 90 % of its assigned capacity. 06/23/02 04:12:46 ANR0314W Recovery log usage exceeds 92 % of its assigned capacity. Output of TSM > query log is as follows Available Assigned Maximum MaximumPage Total Used Pct Max. Space Capacity Extension ReductionSizeUsable Pages Util Pct (MB) (MB) (MB) (MB) (bytes) Pages Util - - - --- - - - - 200 200 0 196 4,09650,688 254 0.5 99.6 Is it really a space problem or what could be reason ? Thanks & regards Sachin Ganu IBM Global Services (I) Pvt. Ltd.
Re: recovery log and ckpt
There are many methods to deal with your recovery log being filled up. I would suggested that first make sure no other processes are running (expiration, reclaimation, and etc). I don't know if you are already using dbbackuptrigger. If you are not using then setup dbbackuptrigger so that when the log reaches a certain point, TSM kicks off a backup of database(which should clear the log). What happens after a database backup and/or an incremental database backup and the log utilization does not drop? This is when CKPT command can come in handy. Undocumented command CKPT can be used to reset the log utilization, but may not work for all situations. The cause according to TSM is the log tail does not get reset by the checkpoint operation that is performed as part of the database backup process when running in roll-forward mode. Supposely this is supposed to be fix in TSM version 4.1.5 which is what you have. Personally I have seen recovery log utilization not dropping after an incremental backup and issuing of CKPT in version 4.1.5 So, it is possible that something else is pinning the recovery log .. which I don't believe TSM has any solution for and appears to be on going issue. If all fails consider going to Normal mode. Sung Y. Lee E-mail [EMAIL PROTECTED] chris rees cc: Sent by: "ADSM: Subject: recovery log and ckpt Dist Stor Manager" <[EMAIL PROTECTED] .EDU> 05/17/2002 08:36 AM Please respond to "ADSM: Dist Stor Manager" Hi all Firstly let me thank those who helped me out last night getting my server back up after the recovery log filled.much better than IBM support...! I have found the culprit that caused the log to fill. It seems that when the incremental backups are running for a win2k cluster the log gets written to at approx 2.2Gb an hour. The incremental didn't even backup that much data, in terms of size but I guess there must have been loads of small files in there. The problem I have now is, how can I back up this cluster if it is going to fill the recovery log everytime. The log is 5196 Mb so cannot grow much bigger, alas we're on 4.1.5 Server and yes I've made management aware that we need to upgrade to take advantage of the new recovery log size features. Looking back through adsm.org people have mentioned the ckpt command. Does anyone know how well this works in rollforward mode?? Many Thanks again Chris _ MSN Photos is the easiest way to share and print your photos: http://photos.msn.com/support/worldwide.aspx
Re: recovery log and ckpt
Well, the good news is, you're backing up really fast! Do you have a dbbackup trigger defined? You can set that up so that when your log hits a threshold (since you know it's filling fast, say 50% full), TSM automatically fires a DB incremental backup. You could have the incremental go to tape if your tape is fast, or to disk if you want it faster. When the incremental finishes, TSM SHOULD clear the log. That might take care of your problem, if that client is backing up many small files and doesn't keep the log pinned. If the client keeps the log pinned, it won't help. But it's harmless to try. (BTW, if you increased your log yesterday, don't forget to DECREASE it today back to where it was - DON"T let it get to that 5.4 GB limit. If the log DOES fill again, you need to be able to add a bit of space to get the server back up.) -Original Message- From: chris rees [mailto:[EMAIL PROTECTED]] Sent: Friday, May 17, 2002 9:37 AM To: [EMAIL PROTECTED] Subject: recovery log and ckpt Hi all Firstly let me thank those who helped me out last night getting my server back up after the recovery log filled.much better than IBM support...! I have found the culprit that caused the log to fill. It seems that when the incremental backups are running for a win2k cluster the log gets written to at approx 2.2Gb an hour. The incremental didn't even backup that much data, in terms of size but I guess there must have been loads of small files in there. The problem I have now is, how can I back up this cluster if it is going to fill the recovery log everytime. The log is 5196 Mb so cannot grow much bigger, alas we're on 4.1.5 Server and yes I've made management aware that we need to upgrade to take advantage of the new recovery log size features. Looking back through adsm.org people have mentioned the ckpt command. Does anyone know how well this works in rollforward mode?? Many Thanks again Chris _ MSN Photos is the easiest way to share and print your photos: http://photos.msn.com/support/worldwide.aspx
Re: Recovery Log almost 100%
Paul - I have run *SM on a mainframe since 1992 - my company participated in both the alpha and beta test. DB2 was never used for the database. What I have heard is that the database engine in tsm is an early model of DB2, so we are all using DB2 in a sense. No doubt you are right that things are optimized, but the question is if the internal database is a sow's ear, how much can it be optimized; it will never be a silk purse. I don't know if tsm would perform better with today's version of db2 or oracle. People with large tsm db's aren't looking for better performance, we are looking for better manageability. With commercial dbms's you don't have the logging problem, you can do multithreaded db backups, you can reorg it without an extended outage. In any case I think this is just an academic debate; I am 99% certain that it will never happen if only for the reason that vendors like blackboxes to protect their software assets. - Bill At 07:13 PM 5/2/2002 -0400, you wrote: >ADSM on the mainframe use to use DB2 and they killed it because of the cost >to the customer, both maintaining another product and the actual license >cost. The TSM relational engine is optimized for TSM processing. A general >relational database cannot hold a candle to the performance differences on >equal hardware. The other issue is Tivoli does not want you mucking around >in the tables updating them with update commands. The referential integrity >is paramount to TSM stability. The real missing piece in TSM's DB engine is >the ability to partition the database and parallel backup/restore. More >important is the 4K block size that kills IO performance on sequential >operations such as a backup/restore. I think Tivoli will see the need and >do something about it. These have been discussed at Share. > >You will find that "black box" is what most customers require to protect >themselves. I realize if they used a general RDBMS that we could extend the >code of TSM significantly further than with the current command >capabilities. But, that is exactly what they want to prevent. You end up >with large customers developing extensions that are impacted by Tivoli >architectural changes that carry a loud voice about making changes that >affect them and thus prevent progress. I am one of them, trust me it >happens. > >This all said. If you can define the requirements that a RDBMS would solve >for you that I have not mentioned, we will carry those requirements forward. > >Paul D. Seay, Jr. >Technical Specialist >Naptheon, INC >757-688-8180 > > >-Original Message- >From: Prather, Wanda [mailto:[EMAIL PROTECTED]] >Sent: Thursday, May 02, 2002 2:05 PM >To: [EMAIL PROTECTED] >Subject: Re: Recovery Log almost 100% > > > and that is JUST the problem. > >I used to (try to) run an IBM lan management product that used DB/2 as its >database underneath. IT WAS IMPOSSIBLE. > >Every problem we ran into, we got finger pointing - the product people said >they were waiting for DB/2 to to fix the problem, the DB/2 people said they >couldn't fix it because it was a product problem. > >YOU DON"T WANT TO GO THERE! > >CRINGE AND BE AFRAID > >My opinions and nobody else's... >Wanda Prather > > >-Original Message- >From: William F. Colwell [mailto:[EMAIL PROTECTED]] >Sent: Thursday, May 02, 2002 12:28 PM >To: [EMAIL PROTECTED] >Subject: Re: Recovery Log almost 100% > > >Tom, I like the taste of this food for thought! > >I have raised this issue with TSM developers at SHARE and the short summary >of their response is "Cringe". So I don't think it will happen anytime soon >if and most likely it will never happen. I agree with you completely that >it would be a great option for site with large databases. Plus TSM would >have 2 development teams working on the product - the current one plus the >database developers who are always trying to make Oracle, DB2 etc. faster. > >- Bill > > >At 06:20 AM 5/2/2002 -0700, you wrote: >>I wonder, also, if there is still any discussion about supporting the >>use of an alternate RDBMS underneat TSM. It is quite clear that there >>are many more sites with database sizes in the >>25-50GB+ range. Five years ago I felt very lonely with a database >>of this size, but given the discussions on the listserv over the past >>year I feel more comfortable that we are no longer one of the only >>sites supporting TSM instances that large. It has always seemed to me >>that the database functions of TSM have been the most problematic >>(deadlock issues, log full issues, SQL query performance problems, >>complicated and unclear recommendations
Re: Recovery Log almost 100%
At 08:40 AM 5/2/2002 -0400, Paul Zarnowski wrote: >In the short term, look for tools to show up that will help TSM >administrators to identify which session has the log tail pinned ... Gretchen Theile of Princeton just told me last night that the new "show logpinned" command is now in 4.2.2.1.
Re: Recovery Log almost 100%
ADSM on the mainframe use to use DB2 and they killed it because of the cost to the customer, both maintaining another product and the actual license cost. The TSM relational engine is optimized for TSM processing. A general relational database cannot hold a candle to the performance differences on equal hardware. The other issue is Tivoli does not want you mucking around in the tables updating them with update commands. The referential integrity is paramount to TSM stability. The real missing piece in TSM's DB engine is the ability to partition the database and parallel backup/restore. More important is the 4K block size that kills IO performance on sequential operations such as a backup/restore. I think Tivoli will see the need and do something about it. These have been discussed at Share. You will find that "black box" is what most customers require to protect themselves. I realize if they used a general RDBMS that we could extend the code of TSM significantly further than with the current command capabilities. But, that is exactly what they want to prevent. You end up with large customers developing extensions that are impacted by Tivoli architectural changes that carry a loud voice about making changes that affect them and thus prevent progress. I am one of them, trust me it happens. This all said. If you can define the requirements that a RDBMS would solve for you that I have not mentioned, we will carry those requirements forward. Paul D. Seay, Jr. Technical Specialist Naptheon, INC 757-688-8180 -Original Message- From: Prather, Wanda [mailto:[EMAIL PROTECTED]] Sent: Thursday, May 02, 2002 2:05 PM To: [EMAIL PROTECTED] Subject: Re: Recovery Log almost 100% and that is JUST the problem. I used to (try to) run an IBM lan management product that used DB/2 as its database underneath. IT WAS IMPOSSIBLE. Every problem we ran into, we got finger pointing - the product people said they were waiting for DB/2 to to fix the problem, the DB/2 people said they couldn't fix it because it was a product problem. YOU DON"T WANT TO GO THERE! CRINGE AND BE AFRAID My opinions and nobody else's... Wanda Prather -Original Message- From: William F. Colwell [mailto:[EMAIL PROTECTED]] Sent: Thursday, May 02, 2002 12:28 PM To: [EMAIL PROTECTED] Subject: Re: Recovery Log almost 100% Tom, I like the taste of this food for thought! I have raised this issue with TSM developers at SHARE and the short summary of their response is "Cringe". So I don't think it will happen anytime soon if and most likely it will never happen. I agree with you completely that it would be a great option for site with large databases. Plus TSM would have 2 development teams working on the product - the current one plus the database developers who are always trying to make Oracle, DB2 etc. faster. - Bill At 06:20 AM 5/2/2002 -0700, you wrote: >I wonder, also, if there is still any discussion about supporting the >use of an alternate RDBMS underneat TSM. It is quite clear that there >are many more sites with database sizes in the >25-50GB+ range. Five years ago I felt very lonely with a database >of this size, but given the discussions on the listserv over the past >year I feel more comfortable that we are no longer one of the only >sites supporting TSM instances that large. It has always seemed to me >that the database functions of TSM have been the most problematic >(deadlock issues, log full issues, SQL query performance problems, >complicated and unclear recommendations for physical database layout, >etc.). All of these problems have been solved by Oracle, DB2, and >Sybase. Granted there is the issue that plugging in an external >database adds greatly to the complexity of TSM, and reduces it's "black >box-ness", but I think the resources are available to administer such a >beast at the large sites that require very large databases. > >More food for thought *early* on a Thursday morning. > > -- Tom > >Thomas A. La Porte >DreamWorks SKG >[EMAIL PROTECTED] -- Bill Colwell C. S. Draper Lab Cambridge Ma.
Re: Recovery Log almost 100%
> This a very good requirement. Also given that TSM DB is very close to DB2 > in design we can hope this TSM-DB2 integration to be possible. Maybe with > some restrictions - locally on the TSM server machine, for sure not EEE > version, etc. Probably IBM will not discuss Oracle or Sybase usage. This kind of proposal has always struck me as confusing the symptom with the disease. The fundamental problem is the attitude that addressing the current performance and usability problems is a luxury rather than a necessity. If that attitude is fixed, we can safely let the developers decide for themselves whether they need a different infrastructure to meet their performance and usability objectives. If that attitude isn't fixed, pressuring the developers into adopting a different infrastructure will get us slow, cumbersome facilities built on a different infrastructure.
Re: Recovery Log almost 100%
and that is JUST the problem. I used to (try to) run an IBM lan management product that used DB/2 as its database underneath. IT WAS IMPOSSIBLE. Every problem we ran into, we got finger pointing - the product people said they were waiting for DB/2 to to fix the problem, the DB/2 people said they couldn't fix it because it was a product problem. YOU DON"T WANT TO GO THERE! CRINGE AND BE AFRAID My opinions and nobody else's... Wanda Prather -Original Message- From: William F. Colwell [mailto:[EMAIL PROTECTED]] Sent: Thursday, May 02, 2002 12:28 PM To: [EMAIL PROTECTED] Subject: Re: Recovery Log almost 100% Tom, I like the taste of this food for thought! I have raised this issue with TSM developers at SHARE and the short summary of their response is "Cringe". So I don't think it will happen anytime soon if and most likely it will never happen. I agree with you completely that it would be a great option for site with large databases. Plus TSM would have 2 development teams working on the product - the current one plus the database developers who are always trying to make Oracle, DB2 etc. faster. - Bill At 06:20 AM 5/2/2002 -0700, you wrote: >I wonder, also, if there is still any discussion about supporting >the use of an alternate RDBMS underneat TSM. It is quite clear >that there are many more sites with database sizes in the >25-50GB+ range. Five years ago I felt very lonely with a database >of this size, but given the discussions on the listserv over the >past year I feel more comfortable that we are no longer one of >the only sites supporting TSM instances that large. It has always >seemed to me that the database functions of TSM have been the >most problematic (deadlock issues, log full issues, SQL query >performance problems, complicated and unclear recommendations for >physical database layout, etc.). All of these problems have been >solved by Oracle, DB2, and Sybase. Granted there is the issue >that plugging in an external database adds greatly to the >complexity of TSM, and reduces it's "black box-ness", but I think >the resources are available to administer such a beast at >the large sites that require very large databases. > >More food for thought *early* on a Thursday morning. > > -- Tom > >Thomas A. La Porte >DreamWorks SKG >[EMAIL PROTECTED] -- Bill Colwell C. S. Draper Lab Cambridge Ma.
Re: Recovery Log almost 100%
This a very good requirement. Also given that TSM DB is very close to DB2 in design we can hope this TSM-DB2 integration to be possible. Maybe with some restrictions - locally on the TSM server machine, for sure not EEE version, etc. Probably IBM will not discuss Oracle or Sybase usage. The main issue would be with support - which version to be used and how often to move to newer version. At the moment DB2 v7.2 is the current version but when new DB2 version comes available should TSM team migrate or not? If yes, we will go in compatibility difficulaties. If no, TSM server will have to rely on something out of support. And what to do with small sites - DB2 would give them additional complexity. If current engine is to be used for them server code will become more complex and error-prone. At the end - how many sites would really buy it if this feature is separately priced (like ISM for Hardware). I guess not too many and this would be not attractive to IBM/Tivoli. Just some thoughts. Zlatko Krastev IT Consultant Please respond to "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]> Sent by:"ADSM: Dist Stor Manager" <[EMAIL PROTECTED]> To: [EMAIL PROTECTED] cc: Subject:Re: Recovery Log almost 100% I wonder, also, if there is still any discussion about supporting the use of an alternate RDBMS underneat TSM. It is quite clear that there are many more sites with database sizes in the 25-50GB+ range. Five years ago I felt very lonely with a database of this size, but given the discussions on the listserv over the past year I feel more comfortable that we are no longer one of the only sites supporting TSM instances that large. It has always seemed to me that the database functions of TSM have been the most problematic (deadlock issues, log full issues, SQL query performance problems, complicated and unclear recommendations for physical database layout, etc.). All of these problems have been solved by Oracle, DB2, and Sybase. Granted there is the issue that plugging in an external database adds greatly to the complexity of TSM, and reduces it's "black box-ness", but I think the resources are available to administer such a beast at the large sites that require very large databases. More food for thought *early* on a Thursday morning. -- Tom Thomas A. La Porte DreamWorks SKG [EMAIL PROTECTED] On Thu, 2 May 2002, Paul Zarnowski wrote: >TSM Development is fully aware of the log issue and based on some >conversations at SHARE, I am comfortable that they are taking steps to >address it (with or without a requirement). I don't think this issue will >be completely solved quickly, as it is a rather complex set of >problems. In the short term, look for tools to show up that will help TSM >administrators to identify which session has the log tail pinned, and also >address one of the issues that Paul refers to below, which causes the log >head to advance quickly (and shows up as a high dirty page count). > >When the log fills, two things happen: The log tail must be pinned by a >long-running in-flight transaction, and the log head must advance around to >catch up to the tail. To keep the log from filling, you can either release >the tail or slow down the head. It is not easy to identify the session or >thread that has the log tail pinned. I don't know if the tools I refer to >above have shown up in 4.2.2 or 5.1 (we're still running 4.2.1). There are >a couple of things that can advance the head quickly. Inventory expiration >and filespace deletion. If you find yourself in a situation where you see >the log filling quickly and don't know what has the tail pinned, check for >these two processes and kill them if you see them. This will significantly >slow down the growth rate of the log, and give the oldest in-flight >transaction more of a chance to complete. We have written a monitor to do >this automatically, and it has really helped us. If neither of these >processes are running, then you can start guessing about which session >might have the tail pinned. In this situation, we look for an old session >that has been running for a long time. This might be a session backing up >over a slow speed line. If the log nears 100%, we try to avoid it filling >completely by cancelling all sessions (if we have time) or simply HALTing >the server and restarting it. This generally clears the log when the >server comes back up, and avoids having to do an offline extend of the log >(which has already been discussed). If you are running >logmode=rollforward, be aware that when you later reduce the log size to >delete the temporary extension, you will (I think) trigger a full database >backup. > >If you are at v4.2, you can have a larger log, up to 13GB. This can also >provide some relief. > >..Paul > >At 12:13 AM 5
Re: Recovery Log almost 100%
Tom, I like the taste of this food for thought! I have raised this issue with TSM developers at SHARE and the short summary of their response is "Cringe". So I don't think it will happen anytime soon if and most likely it will never happen. I agree with you completely that it would be a great option for site with large databases. Plus TSM would have 2 development teams working on the product - the current one plus the database developers who are always trying to make Oracle, DB2 etc. faster. - Bill At 06:20 AM 5/2/2002 -0700, you wrote: >I wonder, also, if there is still any discussion about supporting >the use of an alternate RDBMS underneat TSM. It is quite clear >that there are many more sites with database sizes in the >25-50GB+ range. Five years ago I felt very lonely with a database >of this size, but given the discussions on the listserv over the >past year I feel more comfortable that we are no longer one of >the only sites supporting TSM instances that large. It has always >seemed to me that the database functions of TSM have been the >most problematic (deadlock issues, log full issues, SQL query >performance problems, complicated and unclear recommendations for >physical database layout, etc.). All of these problems have been >solved by Oracle, DB2, and Sybase. Granted there is the issue >that plugging in an external database adds greatly to the >complexity of TSM, and reduces it's "black box-ness", but I think >the resources are available to administer such a beast at >the large sites that require very large databases. > >More food for thought *early* on a Thursday morning. > > -- Tom > >Thomas A. La Porte >DreamWorks SKG >[EMAIL PROTECTED] -- Bill Colwell C. S. Draper Lab Cambridge Ma.
Re: Recovery Log almost 100%
I wonder, also, if there is still any discussion about supporting the use of an alternate RDBMS underneat TSM. It is quite clear that there are many more sites with database sizes in the 25-50GB+ range. Five years ago I felt very lonely with a database of this size, but given the discussions on the listserv over the past year I feel more comfortable that we are no longer one of the only sites supporting TSM instances that large. It has always seemed to me that the database functions of TSM have been the most problematic (deadlock issues, log full issues, SQL query performance problems, complicated and unclear recommendations for physical database layout, etc.). All of these problems have been solved by Oracle, DB2, and Sybase. Granted there is the issue that plugging in an external database adds greatly to the complexity of TSM, and reduces it's "black box-ness", but I think the resources are available to administer such a beast at the large sites that require very large databases. More food for thought *early* on a Thursday morning. -- Tom Thomas A. La Porte DreamWorks SKG [EMAIL PROTECTED] On Thu, 2 May 2002, Paul Zarnowski wrote: >TSM Development is fully aware of the log issue and based on some >conversations at SHARE, I am comfortable that they are taking steps to >address it (with or without a requirement). I don't think this issue will >be completely solved quickly, as it is a rather complex set of >problems. In the short term, look for tools to show up that will help TSM >administrators to identify which session has the log tail pinned, and also >address one of the issues that Paul refers to below, which causes the log >head to advance quickly (and shows up as a high dirty page count). > >When the log fills, two things happen: The log tail must be pinned by a >long-running in-flight transaction, and the log head must advance around to >catch up to the tail. To keep the log from filling, you can either release >the tail or slow down the head. It is not easy to identify the session or >thread that has the log tail pinned. I don't know if the tools I refer to >above have shown up in 4.2.2 or 5.1 (we're still running 4.2.1). There are >a couple of things that can advance the head quickly. Inventory expiration >and filespace deletion. If you find yourself in a situation where you see >the log filling quickly and don't know what has the tail pinned, check for >these two processes and kill them if you see them. This will significantly >slow down the growth rate of the log, and give the oldest in-flight >transaction more of a chance to complete. We have written a monitor to do >this automatically, and it has really helped us. If neither of these >processes are running, then you can start guessing about which session >might have the tail pinned. In this situation, we look for an old session >that has been running for a long time. This might be a session backing up >over a slow speed line. If the log nears 100%, we try to avoid it filling >completely by cancelling all sessions (if we have time) or simply HALTing >the server and restarting it. This generally clears the log when the >server comes back up, and avoids having to do an offline extend of the log >(which has already been discussed). If you are running >logmode=rollforward, be aware that when you later reduce the log size to >delete the temporary extension, you will (I think) trigger a full database >backup. > >If you are at v4.2, you can have a larger log, up to 13GB. This can also >provide some relief. > >..Paul > >At 12:13 AM 5/2/2002 -0400, Seay, Paul wrote: >>Actually, this was significantly discussed at Share and the basic >>requirement is TSM, take action whatever necessary to keep the server up. >>Start by cancelling expiration. Then nail the client that has the log >>pinned. There were also a number of issues discussed. Apparently, there >>are a lot of dirty blocks being recorded in the log that do not have to be. >>I am working to get these requirements voted on. >> >>Paul D. Seay, Jr. >>Technical Specialist >>Naptheon, INC >>757-688-8180 >> >> >>-Original Message- >>From: Thomas A. La Porte [mailto:[EMAIL PROTECTED]] >>Sent: Wednesday, May 01, 2002 4:36 PM >>To: [EMAIL PROTECTED] >>Subject: Re: Recovery Log almost 100% >> >> >>Given that this is one of the more comman FAQ style questions on this >>listserv, I wonder if it's not time for someone to submit a TSM requirement >>that the server behave better in a recovery log full situation. This happens >>in other databases w/o causing a SIGSEGV. Oracle, for example, simply >>prevents any database changes, and only allows new administrative >>c
Re: Recovery Log almost 100%
TSM Development is fully aware of the log issue and based on some conversations at SHARE, I am comfortable that they are taking steps to address it (with or without a requirement). I don't think this issue will be completely solved quickly, as it is a rather complex set of problems. In the short term, look for tools to show up that will help TSM administrators to identify which session has the log tail pinned, and also address one of the issues that Paul refers to below, which causes the log head to advance quickly (and shows up as a high dirty page count). When the log fills, two things happen: The log tail must be pinned by a long-running in-flight transaction, and the log head must advance around to catch up to the tail. To keep the log from filling, you can either release the tail or slow down the head. It is not easy to identify the session or thread that has the log tail pinned. I don't know if the tools I refer to above have shown up in 4.2.2 or 5.1 (we're still running 4.2.1). There are a couple of things that can advance the head quickly. Inventory expiration and filespace deletion. If you find yourself in a situation where you see the log filling quickly and don't know what has the tail pinned, check for these two processes and kill them if you see them. This will significantly slow down the growth rate of the log, and give the oldest in-flight transaction more of a chance to complete. We have written a monitor to do this automatically, and it has really helped us. If neither of these processes are running, then you can start guessing about which session might have the tail pinned. In this situation, we look for an old session that has been running for a long time. This might be a session backing up over a slow speed line. If the log nears 100%, we try to avoid it filling completely by cancelling all sessions (if we have time) or simply HALTing the server and restarting it. This generally clears the log when the server comes back up, and avoids having to do an offline extend of the log (which has already been discussed). If you are running logmode=rollforward, be aware that when you later reduce the log size to delete the temporary extension, you will (I think) trigger a full database backup. If you are at v4.2, you can have a larger log, up to 13GB. This can also provide some relief. ..Paul At 12:13 AM 5/2/2002 -0400, Seay, Paul wrote: >Actually, this was significantly discussed at Share and the basic >requirement is TSM, take action whatever necessary to keep the server up. >Start by cancelling expiration. Then nail the client that has the log >pinned. There were also a number of issues discussed. Apparently, there >are a lot of dirty blocks being recorded in the log that do not have to be. >I am working to get these requirements voted on. > >Paul D. Seay, Jr. >Technical Specialist >Naptheon, INC >757-688-8180 > > >-Original Message- >From: Thomas A. La Porte [mailto:[EMAIL PROTECTED]] >Sent: Wednesday, May 01, 2002 4:36 PM >To: [EMAIL PROTECTED] >Subject: Re: Recovery Log almost 100% > > >Given that this is one of the more comman FAQ style questions on this >listserv, I wonder if it's not time for someone to submit a TSM requirement >that the server behave better in a recovery log full situation. This happens >in other databases w/o causing a SIGSEGV. Oracle, for example, simply >prevents any database changes, and only allows new administrative >connections to the database until the log full situation is cleared (by >archiving the online redo logs). It seems that TSM could behave similarly. > >Certainly the server is not in a great state when the log segments are full, >but it would seem easier to recover, and somewhat less confusing to >administrators, if it could be done online, rather than in the manner in >which it is handled now. We've all probably experienced a situation where we >are close to the limit on the log size, so we only extend the log a little >bit, and then there is a rush to see if our database backup is going to >finish and clear the log full condition before we use up the additional log >space--lest we find ourselves in the same perilous condition, only *closer* >to the seemingly arbitrary maximum log size. > > -- Tom > >Thomas A. La Porte >DreamWorks SKG >[EMAIL PROTECTED] > >On Wed, 1 May 2002, Sung Y Lee wrote: > > >When log reaches 100%, just pray that TSM server process will not > >crash. > > > > > >I say the key is prevention. Whatever you can do to prevent that from > >happening is the best answer. > > > >There are many things you can do to prevent from growing to 100%. One > >that works for me is I have LogMode set to Roll Forward mode with dbb > >trigger at 38% with incremental between at 3(q dbb) Log
Re: Recovery Log almost 100%
Actually, this was significantly discussed at Share and the basic requirement is TSM, take action whatever necessary to keep the server up. Start by cancelling expiration. Then nail the client that has the log pinned. There were also a number of issues discussed. Apparently, there are a lot of dirty blocks being recorded in the log that do not have to be. I am working to get these requirements voted on. Paul D. Seay, Jr. Technical Specialist Naptheon, INC 757-688-8180 -Original Message- From: Thomas A. La Porte [mailto:[EMAIL PROTECTED]] Sent: Wednesday, May 01, 2002 4:36 PM To: [EMAIL PROTECTED] Subject: Re: Recovery Log almost 100% Given that this is one of the more comman FAQ style questions on this listserv, I wonder if it's not time for someone to submit a TSM requirement that the server behave better in a recovery log full situation. This happens in other databases w/o causing a SIGSEGV. Oracle, for example, simply prevents any database changes, and only allows new administrative connections to the database until the log full situation is cleared (by archiving the online redo logs). It seems that TSM could behave similarly. Certainly the server is not in a great state when the log segments are full, but it would seem easier to recover, and somewhat less confusing to administrators, if it could be done online, rather than in the manner in which it is handled now. We've all probably experienced a situation where we are close to the limit on the log size, so we only extend the log a little bit, and then there is a rush to see if our database backup is going to finish and clear the log full condition before we use up the additional log space--lest we find ourselves in the same perilous condition, only *closer* to the seemingly arbitrary maximum log size. -- Tom Thomas A. La Porte DreamWorks SKG [EMAIL PROTECTED] On Wed, 1 May 2002, Sung Y Lee wrote: >When log reaches 100%, just pray that TSM server process will not >crash. > > >I say the key is prevention. Whatever you can do to prevent that from >happening is the best answer. > >There are many things you can do to prevent from growing to 100%. One >that works for me is I have LogMode set to Roll Forward mode with dbb >trigger at 38% with incremental between at 3(q dbb) Log is also set to >maximum allowed without going over limit plus room for extension should >it ever reaches 100% and TSM crashes. Have it set at 4.5 GB(To be >safe). Max allowed recovery log for TSM 4.1 is 5.3 GB?? I can't recall >exact value. > > >If the TSM server is in Log mode than more than likely it will have dbb >trigger set at certain point. For example, >adsm> q dbb > >Full Incremental Log Full Incrementals >Device Device Percentage Between >Class ClassFulls >-- --- -- >IBM3590IBM3590 38 3 > >When the recovery log reaches 38%, an incremental database backup will kick >off up to 3 threes before a full database backup performed. The most of >time TSM server will prevent other sessions from establishing when the >recovery log reaches 100% but will allow the trigger database backup to >complete and it will bring the recovery log down to 0. Sometimes TSM >will simply crash. If it crashes then you will need to do an emergency >recovery log extend and bring TSM backup. > >Sung Y. Lee >E-mail [EMAIL PROTECTED] > > > > brian welsh >AIL.COM> cc: > Sent by: "ADSM: Subject: Recovery Log almost 100% > Dist Stor > Manager" > <[EMAIL PROTECTED] > .EDU> > > > 05/01/2002 01:23 > PM > Please respond to > "ADSM: Dist Stor > Manager" > > > > > >Hello, > >AIX 4.3.3 and server 4.1.1.0. > >Last night two archive-schedules had a problem. On the clients there >were files in a kind of loop and TSM tried to archive them. Result, >recovery log almost 100%. This was the first time our log is that high. >Problem on the client solved, but now I have the following question. > >I was wondering how other people prevent the log from growing to 100%, >and how to handle after the log have reached 100%. > >Any tip is welcome. > >Brian. > > >_ >MSN Foto's is de makkelijkste manier om je foto's te delen met anderen >en af te drukken: http://photos.msn.nl/Support/WorldWide.aspx > >
Re: Recovery Log almost 100%
Given that this is one of the more comman FAQ style questions on this listserv, I wonder if it's not time for someone to submit a TSM requirement that the server behave better in a recovery log full situation. This happens in other databases w/o causing a SIGSEGV. Oracle, for example, simply prevents any database changes, and only allows new administrative connections to the database until the log full situation is cleared (by archiving the online redo logs). It seems that TSM could behave similarly. Certainly the server is not in a great state when the log segments are full, but it would seem easier to recover, and somewhat less confusing to administrators, if it could be done online, rather than in the manner in which it is handled now. We've all probably experienced a situation where we are close to the limit on the log size, so we only extend the log a little bit, and then there is a rush to see if our database backup is going to finish and clear the log full condition before we use up the additional log space--lest we find ourselves in the same perilous condition, only *closer* to the seemingly arbitrary maximum log size. -- Tom Thomas A. La Porte DreamWorks SKG [EMAIL PROTECTED] On Wed, 1 May 2002, Sung Y Lee wrote: >When log reaches 100%, just pray that TSM server process will not crash. > > >I say the key is prevention. Whatever you can do to prevent that from >happening is the best answer. > >There are many things you can do to prevent from growing to 100%. >One that works for me is I have LogMode set to Roll Forward mode with dbb >trigger at 38% with incremental between at 3(q dbb) >Log is also set to maximum allowed without going over limit plus room for >extension should it ever reaches 100% and TSM crashes. Have it set at 4.5 >GB(To be safe). Max allowed recovery log for TSM 4.1 is 5.3 GB?? I can't >recall exact value. > > >If the TSM server is in Log mode than more than likely it will have dbb >trigger set at certain point. For example, >adsm> q dbb > >Full Incremental Log Full Incrementals >Device Device Percentage Between >Class ClassFulls >-- --- -- >IBM3590IBM3590 38 3 > >When the recovery log reaches 38%, an incremental database backup will kick >off up to 3 threes before a full database backup performed. The most of >time TSM server will prevent other sessions from establishing when the >recovery log reaches 100% but will allow the trigger database backup to >complete and it will bring the recovery log down to 0. Sometimes TSM will >simply crash. If it crashes then you will need to do an emergency recovery >log extend and bring TSM backup. > >Sung Y. Lee >E-mail [EMAIL PROTECTED] > > > > brian welsh >AIL.COM> cc: > Sent by: "ADSM: Subject: Recovery Log almost 100% > Dist Stor > Manager" > <[EMAIL PROTECTED] > .EDU> > > > 05/01/2002 01:23 > PM > Please respond to > "ADSM: Dist Stor > Manager" > > > > > >Hello, > >AIX 4.3.3 and server 4.1.1.0. > >Last night two archive-schedules had a problem. On the clients there were >files in a kind of loop and TSM tried to archive them. Result, recovery log >almost 100%. This was the first time our log is that high. Problem on the >client solved, but now I have the following question. > >I was wondering how other people prevent the log from growing to 100%, and >how to handle after the log have reached 100%. > >Any tip is welcome. > >Brian. > > >_ >MSN Foto's is de makkelijkste manier om je foto's te delen met anderen en >af >te drukken: http://photos.msn.nl/Support/WorldWide.aspx > >
Re: Recovery Log almost 100%
For starters most people will define spacetriggers to automatically expand their DB and recovery log before it reaches 100% (do a help define spacetrigger on dsmadmc cmdline). What this does is let you set a threshold (say 80%) at which a process gets kicked off to create a new volume for your DB or recovery log. Then TSM automatically does an extend on your DB or recovery log and voila, TSM has more space to work with. I've always thought this was kind of a patch personally because it insinuates you have unused filesystem space sitting someplace for TSM to create volumes in. Why not just make your recovery log that large in the first place if you have the space? Secondly, if your database is in roll forward mode, performing a DB backup will reset the recovery log to 0 utilization. There is a mechanism you can implement to cause a DB backup to kick off anytime your recovery log reaches a certain threshold. This is done by setting a database backup trigger and is for when your database is in roll-forward mode (you didn't specify whether it was or wasn't. See 'help dbbackuptrigger' or look up "automating database backups" in the admin guide). You need to make sure your threshold isn't so high that the recovery log fills up before the DB backup completes (i.e. 98% threshold is unlikely to work). Now despite all this the recovery log still has a size limit.. 13GB.. and TSM only automatically expands it to 12GB.. furthermore you may have your recovery log on a filesystem that's even smaller in size and TSM can't automatically expand a filesystem, it can only create volumes on it. so it's still possible to hit 100% one way or another. The key here is identifying why it's occurring despite having the above in place and addressing it. It shouldn't really happen if you have the above in place and implemented properly if you think about it. Gerald Wichmann Sr. Systems Development Engineer Zantaz, Inc. 925.598.3099 w 408.836.9062 c -Original Message- From: brian welsh [mailto:[EMAIL PROTECTED]] Sent: Wednesday, May 01, 2002 11:24 AM To: [EMAIL PROTECTED] Subject: Recovery Log almost 100% Hello, AIX 4.3.3 and server 4.1.1.0. Last night two archive-schedules had a problem. On the clients there were files in a kind of loop and TSM tried to archive them. Result, recovery log almost 100%. This was the first time our log is that high. Problem on the client solved, but now I have the following question. I was wondering how other people prevent the log from growing to 100%, and how to handle after the log have reached 100%. Any tip is welcome. Brian. _ MSN Foto's is de makkelijkste manier om je foto's te delen met anderen en af te drukken: http://photos.msn.nl/Support/WorldWide.aspx
Re: Recovery Log almost 100%
When log reaches 100%, just pray that TSM server process will not crash. I say the key is prevention. Whatever you can do to prevent that from happening is the best answer. There are many things you can do to prevent from growing to 100%. One that works for me is I have LogMode set to Roll Forward mode with dbb trigger at 38% with incremental between at 3(q dbb) Log is also set to maximum allowed without going over limit plus room for extension should it ever reaches 100% and TSM crashes. Have it set at 4.5 GB(To be safe). Max allowed recovery log for TSM 4.1 is 5.3 GB?? I can't recall exact value. If the TSM server is in Log mode than more than likely it will have dbb trigger set at certain point. For example, adsm> q dbb Full Incremental Log Full Incrementals Device Device Percentage Between Class ClassFulls -- --- -- IBM3590IBM3590 38 3 When the recovery log reaches 38%, an incremental database backup will kick off up to 3 threes before a full database backup performed. The most of time TSM server will prevent other sessions from establishing when the recovery log reaches 100% but will allow the trigger database backup to complete and it will bring the recovery log down to 0. Sometimes TSM will simply crash. If it crashes then you will need to do an emergency recovery log extend and bring TSM backup. Sung Y. Lee E-mail [EMAIL PROTECTED] brian welsh cc: Sent by: "ADSM: Subject: Recovery Log almost 100% Dist Stor Manager" <[EMAIL PROTECTED] .EDU> 05/01/2002 01:23 PM Please respond to "ADSM: Dist Stor Manager" Hello, AIX 4.3.3 and server 4.1.1.0. Last night two archive-schedules had a problem. On the clients there were files in a kind of loop and TSM tried to archive them. Result, recovery log almost 100%. This was the first time our log is that high. Problem on the client solved, but now I have the following question. I was wondering how other people prevent the log from growing to 100%, and how to handle after the log have reached 100%. Any tip is welcome. Brian. _ MSN Foto's is de makkelijkste manier om je foto's te delen met anderen en af te drukken: http://photos.msn.nl/Support/WorldWide.aspx
Re: Recovery Log almost 100%
Brian, When this happens to us even with a Space Trigger in place it is necessary to: 1. define a new log volume with dsmfmt 2. extend the log using dsmserv 3. then restart your TSM server Once it is running you can remove and delete the log vol you added. Tom Fought Sr System Engineer/TSM Administrator -Original Message- From: brian welsh [mailto:[EMAIL PROTECTED]] Sent: Wednesday, May 01, 2002 2:24 PM To: [EMAIL PROTECTED] Subject: Recovery Log almost 100% Hello, AIX 4.3.3 and server 4.1.1.0. Last night two archive-schedules had a problem. On the clients there were files in a kind of loop and TSM tried to archive them. Result, recovery log almost 100%. This was the first time our log is that high. Problem on the client solved, but now I have the following question. I was wondering how other people prevent the log from growing to 100%, and how to handle after the log have reached 100%. Any tip is welcome. Brian. _ MSN Foto's is de makkelijkste manier om je foto's te delen met anderen en af te drukken: http://photos.msn.nl/Support/WorldWide.aspx
Re: Recovery Log almost 100%
Brian, For starters to prevent this from happening in the future you can define a space trigger for your recovery log to automatically extend it when it reaches a certain percentage! See the admin reference guide under DEFINE SPACETRIGGER! Regards, Demetrius Malbrough UNIX/TSM Administrator -Original Message- From: brian welsh [mailto:[EMAIL PROTECTED]] Sent: Wednesday, May 01, 2002 1:24 PM To: [EMAIL PROTECTED] Subject: Recovery Log almost 100% Hello, AIX 4.3.3 and server 4.1.1.0. Last night two archive-schedules had a problem. On the clients there were files in a kind of loop and TSM tried to archive them. Result, recovery log almost 100%. This was the first time our log is that high. Problem on the client solved, but now I have the following question. I was wondering how other people prevent the log from growing to 100%, and how to handle after the log have reached 100%. Any tip is welcome. Brian. _ MSN Foto's is de makkelijkste manier om je foto's te delen met anderen en af te drukken: http://photos.msn.nl/Support/WorldWide.aspx
Re: Recovery log problem
This is not unusual at all. The log size is related to the amount of activity, not to the DB size. Some people have a log size bigger than their DB, some do not. It is not uncommon to have TSM logs > 5 GB! The simple thing, just make the log bigger. Here's another idea you could try: Create a devclass with TYPE=FILE (goes to disk). Schedule an admin command on the weekends that does a DB backup with type=incr, using the TYPE=FILE devclass. That can run with no operator intervention, and will clear your log. Then on Monday you can go back to your DB backups on DLT. So : on Friday DB BACKUP TYPE=FULL to DLT devclass on Sat DB BACKUP TYPE=INCR to file devclass on Sun DB BACKUP TYPE=INCR to file devclass on Mon DB BACKUP TYPE=FULL to DLT devclass -Original Message- From: Andrea GRIGOLO [mailto:[EMAIL PROTECTED]] Sent: Tuesday, March 12, 2002 11:29 AM To: [EMAIL PROTECTED] Subject: Recovery log problem Hello ! I have a little problem with the recovery log of a TSM server 4.2.1.7 (with a Compaq TL891DLX). There are about 50 client workstations, 5 client servers (mostly file servers) and 5 TDP for MSSQL. This grows very fast and I have no idea if it is normal or not. I have 500 megs for dbvolumes and 900 for the log, but I still get the "insufficent recovery log" message after the weekend (when the dbbackup is not done, as we do it on an external stand-alone DLT and no operator is there during the weekend to swap the tape). So, is it normal to have a recovery log twice bigger than the dB ? How could I reduce dB size (I already force expiration and I delete old volhistory entries)? Thank you Andrea ** This message may contain confidential information and is intended solely for the use of the addressee. If you are not the intended recipient of this message, please notify the sender immediately and do not disclose, use, disseminate or copy this message. The information contained in this message is subject to Systemat's General Terms and Conditions. Any personal opinion expressed in this message reflects the opinion of the sender and not Systemat's opinion. **
Re: Recovery Log behavior after 100% util reached
You already have the answer - log mode is normal. In this mode log is used only to rollback a transaction. On commit the log is cleaned. So when there is no activity the log must be 0% (or at least very close). The drawback is that any transaction after last DBB is lost in case of major failure. The reason for no crash on 100% utilisation might be one transaction committing while second (and others) is held waiting for free log space. After commit held transaction(s) continues. And possibly the crash happens when there is very-very long transaction filling the log alone or deadlock. I do not know is there long transaction limit (and rollback) or deadlock resolving methods in TSM DB as we have in RDBMSes. The only people can answer to this are server developers. Zlatko Krastev IT Consultant Denis L'Huillier <[EMAIL PROTECTED]>@VM.MARIST.EDU> on 10.01.2002 19:45:54 Please respond to "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]> Sent by:"ADSM: Dist Stor Manager" <[EMAIL PROTECTED]> To: [EMAIL PROTECTED] cc: Subject:Recovery Log behavior after 100% util reached Hello, I have recently inherited an over worked, maxed out TSM server where the recovery log maxes out on just about a nightly basis. I have 2 questions maybe someone can help me with. 1. Why does the server only crash sometimes when the log fills up. For example, yesterday it reached 100% but never went down. 2 days ago it reached 100% and TSM crashed. 2. When the server didn't crash the log kind of automatically zero'd itself after reaching 100%. What happened to all those uncommitted changes to the database? Did I lose data? Shouldn't the log remain at 100% actual used until a db backup takes place? I have a 'q log' script in cron that runs every 10 minutes and this is the output during that time frame: Server date/time: 01/10/02 02:10:10 Last access: 01/10/02 02ANS8000I Server command: 'q log' 2,5642,564 0 564 4,096 655,872 510,699 77.9 77.9 Server date/time: 01/10/02 03:00:04 Last access: 01/10/02 02ANS8000I Server command: 'q log' 2,5642,564 0 8 4,096 655,872 653,290 99.6 99.6 Server date/time: 01/10/02 03:20:00 Last access: 01/10/02 03ANS8000I Server command: 'q log' 2,5642,564 0 2,560 4,096 655,872 189 0.0 100.0 Server date/time: 01/10/02 03:10:01 Last access: 01/10/02 03ANS8000I Server command: 'q log' 2,5642,564 0 2,560 4,096 655,872 189 0.0 100.0 As you can see the log reached 100% (99.6) at 3:00am and at 3:10 zero'd out. But no backups took place until about 4:50am and my database backup doesn't kick in until 9:00 am.. Also we are in normal log mode, no triggers. Here's the act log for this time frame: tsm: ADSMFP>q act begint=03:10 endt=04:54 Date/TimeMessage -- 01/10/02 03:10:01 ANR0402I Session 2282 started for administrator SCRIPT_ID (AIX) (ShMem). 01/10/02 03:10:01 ANR2017I Administrator SCRIPT_ID issued command: QUERY LOG 01/10/02 03:10:01 ANR0405I Session 2282 ended for administrator SCRIPT_ID (AIX). 01/10/02 03:20:00 ANR0402I Session 2283 started for administrator SCRIPT_ID (AIX) (ShMem). 01/10/02 03:20:00 ANR2017I Administrator SCRIPT_ID issued command: QUERY LOG 01/10/02 03:20:00 ANR0405I Session 2283 ended for administrator SCRIPT_ID (AIX). 01/10/02 03:21:40 ANR0406I Session 2284 started for node INAFPUXPAS02 (SUN SOLARIS) (Tcp/Ip 172.22.70.4(57097)). 01/10/02 03:21:40 ANR0403I Session 2284 ended for node INAFPUXPAS02 (SUN SOLARIS). 01/10/02 03:30:00 ANR0402I Session 2285 started for administrator SCRIPT_ID (AIX) (ShMem). 01/10/02 03:30:00 ANR2017I Administrator SCRIPT_ID issued command: QUERY LOG 01/10/02 03:30:00 ANR0405I Session 2285 ended for administrator SCRIPT_ID (AIX). 01/10/02 03:40:01 ANR0402I Session 2286 started for administrator SCRIPT_ID (AIX) (ShMem). 01/10/02 03:40:01 ANR2017I Administrator SCRIPT_ID issued command: QUERY LOG 01/10/02 03:40:01 ANR0405I Session 2286 ended for administrator SCRIPT_ID (AIX). 01/10/02 03:50:00 ANR0402I Session 2287 started for administrator SCRIPT_ID (AIX) (ShMem). 01/10/02 03:50:00 ANR2017I Administrator SCRIPT_ID issued command: QUERY LOG 01/10/02 03:50:00 ANR0405I Session 2287 ended for administrator SCRIPT_ID (AIX). 01/10/02 04:00:00 ANR0402I Session 2288 started for a
Re: Recovery Log behavior after 100% util reached
Dwight,
That's why I check the box that says allow the db to grow automatically. I
don't like what that does, but at least it won't let the server die. I can
clean up what the automatic dude does later. Cleaning up after the db fills
is really not fun.
Kelly J. Lipp
Storage Solutions Specialists, Inc.
PO Box 51313
Colorado Springs, CO 80949
[EMAIL PROTECTED] or [EMAIL PROTECTED]
www.storsol.com or www.storserver.com
(719)531-5926
Fax: (240)539-7175
-Original Message-
From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of
Denis L'Huillier
Sent: Thursday, January 10, 2002 11:26 AM
To: [EMAIL PROTECTED]
Subject: Re: Recovery Log behavior after 100% util reached
My database is at approx 54% utilized..
tsm: ADSMFP>q db
Available Assigned Maximum MaximumPage
Total Used Pct Max.
Space Capacity Extension ReductionSize
Usable Pages Util Pct
(MB) (MB) (MB) (MB) (bytes)
Pages Util
- - - ---
- - - -
48,092 48,092 0 2,476 4,096
12,311,55 6,732,305 54.7 55.0
And there are no warning messages in the actlog about the db size, only
warning messages about the recovery log size.
We are running NORMAL recovery log mode, not roll forward. And max pct
util on the db has not been reset in months.
"Cook, Dwight
E (SAIC)"To: "'ADSM: Dist Stor Manager'"
<[EMAIL PROTECTED]>,
<[EMAIL PROTECTED]"'[EMAIL PROTECTED]'"
<[EMAIL PROTECTED]>
m> cc:
Subject: RE: Recovery Log
behavior after 100% util reached
01/10/2002
01:07 PM
Double check the status of your DB !
I glitched the other day in that my log was doing odd things like going to
100.1% utilized, then would zero back out
(oh, check to see if you have logmode normal or rollforward ! )
with logmode normal you should rarely see it get much over a few %
utilized...
ANYWAY I didn't catch that the DB was FULL ! YIKES !
I missed that during the day and that night at 03:24 (that hurt enough that
I remember the time down to the minute)
all client activity came to a HALT !
just something I ran into...
Dwight
-Original Message-
From: Denis L'Huillier [mailto:[EMAIL PROTECTED]]
Sent: Thursday, January 10, 2002 11:46 AM
To: [EMAIL PROTECTED]
Subject: Recovery Log behavior after 100% util reached
Hello,
I have recently inherited an over worked, maxed out TSM server where the
recovery log maxes out
on just about a nightly basis. I have 2 questions maybe someone can help
me with.
1. Why does the server only crash sometimes when the log fills up. For
example, yesterday it reached
100% but never went down. 2 days ago it reached 100% and TSM crashed.
2. When the server didn't crash the log kind of automatically zero'd
itself after reaching 100%.
What happened to all those uncommitted changes to the database? Did I lose
data? Shouldn't the
log remain at 100% actual used until a db backup takes place?
I have a 'q log' script in cron that runs every 10 minutes and this is the
output during that time frame:
Server date/time: 01/10/02 02:10:10 Last access: 01/10/02 02ANS8000I
Server command: 'q log'
2,5642,564 0 564 4,096
655,872 510,699 77.9 77.9
Server date/time: 01/10/02 03:00:04 Last access: 01/10/02 02ANS8000I
Server command: 'q log'
2,5642,564 0 8 4,096
655,872 653,290 99.6 99.6
Server date/time: 01/10/02 03:20:00 Last access: 01/10/02 03ANS8000I
Server command: 'q log'
2,5642,564 0 2,560 4,096
655,872 189 0.0 100.0
Server date/time: 01/10/02 03:10:01 Last access: 01/10/02 03ANS8000I
Server command: 'q log'
2,5642,564 0 2,560 4,096
655,872 189 0.0 100.0
As you can see the log reached 100% (99.6) at 3:00am and at 3:10 zero'd
out. But no backups took place until about 4:50am
and my database backup doesn't kick in until 9:00 am.. Also we are in
normal log mode, no triggers.
Here's the act log for this time frame:
tsm: ADSMFP>q act begint=03:10 endt=04:54
Date/TimeMessage
--
01/10/02 03:10:01 ANR0402I Session 2282 started for administrator
SCRIPT_ID
(AIX) (ShMem).
01/10/02 03:10:01
Re: Recovery Log behavior after 100% util reached
My database is at approx 54% utilized.. tsm: ADSMFP>q db Available Assigned Maximum MaximumPage Total Used Pct Max. Space Capacity Extension ReductionSize Usable Pages Util Pct (MB) (MB) (MB) (MB) (bytes) Pages Util - - - --- - - - - 48,092 48,092 0 2,476 4,096 12,311,55 6,732,305 54.7 55.0 And there are no warning messages in the actlog about the db size, only warning messages about the recovery log size. We are running NORMAL recovery log mode, not roll forward. And max pct util on the db has not been reset in months. "Cook, Dwight E (SAIC)"To: "'ADSM: Dist Stor Manager'" <[EMAIL PROTECTED]>, <[EMAIL PROTECTED]"'[EMAIL PROTECTED]'" <[EMAIL PROTECTED]> m> cc: Subject: RE: Recovery Log behavior after 100% util reached 01/10/2002 01:07 PM Double check the status of your DB ! I glitched the other day in that my log was doing odd things like going to 100.1% utilized, then would zero back out (oh, check to see if you have logmode normal or rollforward ! ) with logmode normal you should rarely see it get much over a few % utilized... ANYWAY I didn't catch that the DB was FULL ! YIKES ! I missed that during the day and that night at 03:24 (that hurt enough that I remember the time down to the minute) all client activity came to a HALT ! just something I ran into... Dwight -Original Message- From: Denis L'Huillier [mailto:[EMAIL PROTECTED]] Sent: Thursday, January 10, 2002 11:46 AM To: [EMAIL PROTECTED] Subject: Recovery Log behavior after 100% util reached Hello, I have recently inherited an over worked, maxed out TSM server where the recovery log maxes out on just about a nightly basis. I have 2 questions maybe someone can help me with. 1. Why does the server only crash sometimes when the log fills up. For example, yesterday it reached 100% but never went down. 2 days ago it reached 100% and TSM crashed. 2. When the server didn't crash the log kind of automatically zero'd itself after reaching 100%. What happened to all those uncommitted changes to the database? Did I lose data? Shouldn't the log remain at 100% actual used until a db backup takes place? I have a 'q log' script in cron that runs every 10 minutes and this is the output during that time frame: Server date/time: 01/10/02 02:10:10 Last access: 01/10/02 02ANS8000I Server command: 'q log' 2,5642,564 0 564 4,096 655,872 510,699 77.9 77.9 Server date/time: 01/10/02 03:00:04 Last access: 01/10/02 02ANS8000I Server command: 'q log' 2,5642,564 0 8 4,096 655,872 653,290 99.6 99.6 Server date/time: 01/10/02 03:20:00 Last access: 01/10/02 03ANS8000I Server command: 'q log' 2,5642,564 0 2,560 4,096 655,872 189 0.0 100.0 Server date/time: 01/10/02 03:10:01 Last access: 01/10/02 03ANS8000I Server command: 'q log' 2,5642,564 0 2,560 4,096 655,872 189 0.0 100.0 As you can see the log reached 100% (99.6) at 3:00am and at 3:10 zero'd out. But no backups took place until about 4:50am and my database backup doesn't kick in until 9:00 am.. Also we are in normal log mode, no triggers. Here's the act log for this time frame: tsm: ADSMFP>q act begint=03:10 endt=04:54 Date/TimeMessage -- 01/10/02 03:10:01 ANR0402I Session 2282 started for administrator SCRIPT_ID (AIX) (ShMem). 01/10/02 03:10:01 ANR2017I Administrator SCRIPT_ID issued command: QUERY LOG 01/10/02 03:10:01 ANR0405I Session 2282 ended for administrator SCRIPT_ID (AIX). 01/10/02 03:20:00 ANR0402I Session 2283 started for administrator SCRIPT_ID (AIX) (ShMem). 01/10/02 03:20:00 ANR2017I Administrator SCRIPT_ID issued command: QUERY LOG 01/10/02 03:20:00 ANR0405I Session 2283 ended for administrator SCRIPT_ID (AIX). 01/10/02 03:21:40 ANR0406I Session 2284 started for node INAFPUXPAS02 (SUN SOLARIS) (Tcp/Ip 172.22.70.4(57097)). 01/10/02 03:21:40 ANR0403I Session 2284 ended for node INA
Re: Recovery Log behavior after 100% util reached
Double check the status of your DB ! I glitched the other day in that my log was doing odd things like going to 100.1% utilized, then would zero back out (oh, check to see if you have logmode normal or rollforward ! ) with logmode normal you should rarely see it get much over a few % utilized... ANYWAY I didn't catch that the DB was FULL ! YIKES ! I missed that during the day and that night at 03:24 (that hurt enough that I remember the time down to the minute) all client activity came to a HALT ! just something I ran into... Dwight -Original Message- From: Denis L'Huillier [mailto:[EMAIL PROTECTED]] Sent: Thursday, January 10, 2002 11:46 AM To: [EMAIL PROTECTED] Subject: Recovery Log behavior after 100% util reached Hello, I have recently inherited an over worked, maxed out TSM server where the recovery log maxes out on just about a nightly basis. I have 2 questions maybe someone can help me with. 1. Why does the server only crash sometimes when the log fills up. For example, yesterday it reached 100% but never went down. 2 days ago it reached 100% and TSM crashed. 2. When the server didn't crash the log kind of automatically zero'd itself after reaching 100%. What happened to all those uncommitted changes to the database? Did I lose data? Shouldn't the log remain at 100% actual used until a db backup takes place? I have a 'q log' script in cron that runs every 10 minutes and this is the output during that time frame: Server date/time: 01/10/02 02:10:10 Last access: 01/10/02 02ANS8000I Server command: 'q log' 2,5642,564 0 564 4,096 655,872 510,699 77.9 77.9 Server date/time: 01/10/02 03:00:04 Last access: 01/10/02 02ANS8000I Server command: 'q log' 2,5642,564 0 8 4,096 655,872 653,290 99.6 99.6 Server date/time: 01/10/02 03:20:00 Last access: 01/10/02 03ANS8000I Server command: 'q log' 2,5642,564 0 2,560 4,096 655,872 189 0.0 100.0 Server date/time: 01/10/02 03:10:01 Last access: 01/10/02 03ANS8000I Server command: 'q log' 2,5642,564 0 2,560 4,096 655,872 189 0.0 100.0 As you can see the log reached 100% (99.6) at 3:00am and at 3:10 zero'd out. But no backups took place until about 4:50am and my database backup doesn't kick in until 9:00 am.. Also we are in normal log mode, no triggers. Here's the act log for this time frame: tsm: ADSMFP>q act begint=03:10 endt=04:54 Date/TimeMessage -- 01/10/02 03:10:01 ANR0402I Session 2282 started for administrator SCRIPT_ID (AIX) (ShMem). 01/10/02 03:10:01 ANR2017I Administrator SCRIPT_ID issued command: QUERY LOG 01/10/02 03:10:01 ANR0405I Session 2282 ended for administrator SCRIPT_ID (AIX). 01/10/02 03:20:00 ANR0402I Session 2283 started for administrator SCRIPT_ID (AIX) (ShMem). 01/10/02 03:20:00 ANR2017I Administrator SCRIPT_ID issued command: QUERY LOG 01/10/02 03:20:00 ANR0405I Session 2283 ended for administrator SCRIPT_ID (AIX). 01/10/02 03:21:40 ANR0406I Session 2284 started for node INAFPUXPAS02 (SUN SOLARIS) (Tcp/Ip 172.22.70.4(57097)). 01/10/02 03:21:40 ANR0403I Session 2284 ended for node INAFPUXPAS02 (SUN SOLARIS). 01/10/02 03:30:00 ANR0402I Session 2285 started for administrator SCRIPT_ID (AIX) (ShMem). 01/10/02 03:30:00 ANR2017I Administrator SCRIPT_ID issued command: QUERY LOG 01/10/02 03:30:00 ANR0405I Session 2285 ended for administrator SCRIPT_ID (AIX). 01/10/02 03:40:01 ANR0402I Session 2286 started for administrator SCRIPT_ID (AIX) (ShMem). 01/10/02 03:40:01 ANR2017I Administrator SCRIPT_ID issued command: QUERY LOG 01/10/02 03:40:01 ANR0405I Session 2286 ended for administrator SCRIPT_ID (AIX). 01/10/02 03:50:00 ANR0402I Session 2287 started for administrator SCRIPT_ID (AIX) (ShMem). 01/10/02 03:50:00 ANR2017I Administrator SCRIPT_ID issued command: QUERY LOG 01/10/02 03:50:00 ANR0405I Session 2287 ended for administrator SCRIPT_ID (AIX). 01/10/02 04:00:00 ANR0402I Session 2288 started for administrator SCRIPT_ID 01/10/02 04:00:00 ANR2017I Administrator SCRIPT_ID issued command: QUERY SESSION 01/10/02 04:00:00 ANR0405I Session 2288 ended for administrator SCRIPT_ID (AIX). 01/10/02 04:00:00 ANR2017I Administrator SCRIPT_ID issued command: QUERY LOG 01/10/02 04:00
Re: recovery log is filling up so quickly
The new limit for LOG on V4.2 is 13GB. Rgds, Mauricio "Seay, Paul" cc: Sent by: Subject: Re: recovery log is filling up so quickly "ADSM: Dist Stor Manager" <[EMAIL PROTECTED] IST.EDU> 10/18/01 02:03 PM Please respond to "ADSM: Dist Stor Manager" Tivoli recognizes this as one of their most pressing scalability issues. In 4.2 they dramatically increased the allowable size of the recovery log. Do not remember the details. -Original Message- From: Prather, Wanda [mailto:[EMAIL PROTECTED]] Sent: Thursday, October 18, 2001 11:48 AM To: [EMAIL PROTECTED] Subject: Re: recovery log is filling up so quickly This problem has been reported on the list many times; it tends to be worse in some versions of TSM than others. What platform/version are you running? And how big is your log?. And as Josh said, it is more likely to happen when there are multiple DB-intensive tasks running together. Also more likely to happen when in roll-forward mode. One of the TSM servers here finally grew so busy I could not run expiration without filling the log, which was already max size (5 GB), had to go back to NORMAL mode instead of roll-forward. -Original Message- From: Joshua S. Bassi [mailto:[EMAIL PROTECTED]] Sent: Thursday, October 18, 2001 12:11 PM To: [EMAIL PROTECTED] Subject: Re: recovery log is filling up so quickly I typically *try* and backed up the TSM DB while no other processes are running. Of course this doesn't always occur, especially when I am designing TSM solutions around products like TDP for SAP which dump transaction logs randomly throughout the day. -- Joshua S. Bassi Independent IT Consultant IBM Certified - AIX/HACMP, SAN, Shark Tivoli Certified Consultant- ADSM/TSM Cell (408)&(831) 332-4006 [EMAIL PROTECTED] -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]] On Behalf Of Pothula S Paparao Sent: Thursday, October 18, 2001 1:31 AM To: [EMAIL PROTECTED] Subject: recovery log is filling up so quickly Importance: High Hi floks, Strange problem. Recovery log fills up rapidly and causes server to crash. This only seems to occur when DB2 backup happening. I tried backing up Db2 database parellel with 4 session 8 buffers, but of no use. I also tried increasing the log , didnt click. any suggestions.
Re: recovery log is filling up so quickly
Tivoli recognizes this as one of their most pressing scalability issues. In 4.2 they dramatically increased the allowable size of the recovery log. Do not remember the details. -Original Message- From: Prather, Wanda [mailto:[EMAIL PROTECTED]] Sent: Thursday, October 18, 2001 11:48 AM To: [EMAIL PROTECTED] Subject: Re: recovery log is filling up so quickly This problem has been reported on the list many times; it tends to be worse in some versions of TSM than others. What platform/version are you running? And how big is your log?. And as Josh said, it is more likely to happen when there are multiple DB-intensive tasks running together. Also more likely to happen when in roll-forward mode. One of the TSM servers here finally grew so busy I could not run expiration without filling the log, which was already max size (5 GB), had to go back to NORMAL mode instead of roll-forward. -Original Message- From: Joshua S. Bassi [mailto:[EMAIL PROTECTED]] Sent: Thursday, October 18, 2001 12:11 PM To: [EMAIL PROTECTED] Subject: Re: recovery log is filling up so quickly I typically *try* and backed up the TSM DB while no other processes are running. Of course this doesn't always occur, especially when I am designing TSM solutions around products like TDP for SAP which dump transaction logs randomly throughout the day. -- Joshua S. Bassi Independent IT Consultant IBM Certified - AIX/HACMP, SAN, Shark Tivoli Certified Consultant- ADSM/TSM Cell (408)&(831) 332-4006 [EMAIL PROTECTED] -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]] On Behalf Of Pothula S Paparao Sent: Thursday, October 18, 2001 1:31 AM To: [EMAIL PROTECTED] Subject: recovery log is filling up so quickly Importance: High Hi floks, Strange problem. Recovery log fills up rapidly and causes server to crash. This only seems to occur when DB2 backup happening. I tried backing up Db2 database parellel with 4 session 8 buffers, but of no use. I also tried increasing the log , didnt click. any suggestions.
Re: recovery log is filling up so quickly
This problem has been reported on the list many times; it tends to be worse in some versions of TSM than others. What platform/version are you running? And how big is your log?. And as Josh said, it is more likely to happen when there are multiple DB-intensive tasks running together. Also more likely to happen when in roll-forward mode. One of the TSM servers here finally grew so busy I could not run expiration without filling the log, which was already max size (5 GB), had to go back to NORMAL mode instead of roll-forward. -Original Message- From: Joshua S. Bassi [mailto:[EMAIL PROTECTED]] Sent: Thursday, October 18, 2001 12:11 PM To: [EMAIL PROTECTED] Subject: Re: recovery log is filling up so quickly I typically *try* and backed up the TSM DB while no other processes are running. Of course this doesn't always occur, especially when I am designing TSM solutions around products like TDP for SAP which dump transaction logs randomly throughout the day. -- Joshua S. Bassi Independent IT Consultant IBM Certified - AIX/HACMP, SAN, Shark Tivoli Certified Consultant- ADSM/TSM Cell (408)&(831) 332-4006 [EMAIL PROTECTED] -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]] On Behalf Of Pothula S Paparao Sent: Thursday, October 18, 2001 1:31 AM To: [EMAIL PROTECTED] Subject: recovery log is filling up so quickly Importance: High Hi floks, Strange problem. Recovery log fills up rapidly and causes server to crash. This only seems to occur when DB2 backup happening. I tried backing up Db2 database parellel with 4 session 8 buffers, but of no use. I also tried increasing the log , didnt click. any suggestions.
Re: recovery log is filling up so quickly
I typically *try* and backed up the TSM DB while no other processes are running. Of course this doesn't always occur, especially when I am designing TSM solutions around products like TDP for SAP which dump transaction logs randomly throughout the day. -- Joshua S. Bassi Independent IT Consultant IBM Certified - AIX/HACMP, SAN, Shark Tivoli Certified Consultant- ADSM/TSM Cell (408)&(831) 332-4006 [EMAIL PROTECTED] -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]] On Behalf Of Pothula S Paparao Sent: Thursday, October 18, 2001 1:31 AM To: [EMAIL PROTECTED] Subject: recovery log is filling up so quickly Importance: High Hi floks, Strange problem. Recovery log fills up rapidly and causes server to crash. This only seems to occur when DB2 backup happening. I tried backing up Db2 database parellel with 4 session 8 buffers, but of no use. I also tried increasing the log , didnt click. any suggestions.
Re: recovery log is filling up so quickly
Separate the long transactions in the ADSM database (DB2 backup, HUGE files) from the database activity. Especially expires. A single object is still a single transaction of ADSM and pins the log. Jeff Bach Home Office Open Systems Engineering Wal-Mart Stores, Inc. WAL-MART CONFIDENTIAL -Original Message- From: Pothula S Paparao [SMTP:[EMAIL PROTECTED]] Sent: Thursday, October 18, 2001 4:31 AM To: [EMAIL PROTECTED] Subject:recovery log is filling up so quickly Importance: High Hi floks, Strange problem. Recovery log fills up rapidly and causes server to crash. This only seems to occur when DB2 backup happening. I tried backing up Db2 database parellel with 4 session 8 buffers, but of no use. I also tried increasing the log , didnt click. any suggestions. ** This email and any files transmitted with it are confidential and intended solely for the individual or entity to whom they are addressed. If you have received this email in error destroy it immediately. **
Re: Recovery log space spikes
If you run a database backup, does the log go down? If it does not, you probably have a long transaction in the database. A certain client is "pinning" the log. Jeff Bach Home Office Open Systems Engineering Wal-Mart Stores, Inc. WAL-MART CONFIDENTIAL -Original Message- From: Joshua S. Bassi [SMTP:[EMAIL PROTECTED]] Sent: Thursday, October 04, 2001 9:48 AM To: [EMAIL PROTECTED] Subject: Re: Recovery log space spikes Are you using "roll forward recovery"? From what you are saying it appears so. I would recommend turning it off and using a logmode=normal which will commit recovery log transactions to the DB as they complete instead of every time the DB fills up. -- Joshua S. Bassi Independent IT Consultant IBM Certified - AIX/HACMP, SAN, Shark Tivoli Certified Consultant- ADSM/TSM Cell (408)&(831) 332-4006 [EMAIL PROTECTED] -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]] On Behalf Of David Browne. Sent: Thursday, October 04, 2001 6:46 AM To: [EMAIL PROTECTED] Subject: Recovery log space spikes The maximum size of the recovery log is 5.5GB, my recovery log is 4.5 GB now and I have just recently increased it 1GB from 3.5GB. It filled up again last night. My database is 59.5 GB. Could I have a problem that is causing my recovery log to fill up or is this normal for this size of database? I am running TSM 3.7.4 on OS390 R 2.10. I have been testing TSM 4.1.4 and am planning to go to the new release in the next few weeks, could this help the situation? Below shows the past three days monitoring. * October 4, 2001 * tsm: ADSM3>q log Available Assigned MaximumMaximum Page Total Used Pct Max. Space Capacity ExtensionReduction Size Usable PagesUtil Pct (MB) (MB) (MB) (MB) (bytes) Pages Util - - - --- -- - - 4,620 4,620 0 4,264 4,096 1,182,208 89,762 7.6100.0 Available Assigned Maximum Maximum Page TotalUsed Pct Max. Space CapacityExtension Reduction Size UsablePagesUtil Pct (MB) (MB)(MB)(MB) (bytes) Pages Util - - ------- ---- 60,908 60,908 0 6,852 4,096 15,592,448 13,839,06688.888.8 tsm: ADSM3> October 3, 2001 ** tsm: ADSM3>q log AvailableAssignedMaximum Maximum Page TotalUsed PctMax. Space Capacity Extension Reduction Size Usable PagesUtilPct (MB)(MB) (MB)(MB) (bytes) Pages Util - - - --- - - - 4,6204,620 0 2,480 4,096 1,182,208 546,31346.2 88.6 tsm: ADSM3>q db Available AssignedMaximum Maximum Page Total Used Pct Max. SpaceCapacity Extension Reduction Size Usable Pages Util Pct (MB)(MB)(MB) (MB) (bytes)Pages Util - -- - - ----- --- - - 60,908
Re: Recovery log space spikes
Are you using "roll forward recovery"? From what you are saying it appears so. I would recommend turning it off and using a logmode=normal which will commit recovery log transactions to the DB as they complete instead of every time the DB fills up. -- Joshua S. Bassi Independent IT Consultant IBM Certified - AIX/HACMP, SAN, Shark Tivoli Certified Consultant- ADSM/TSM Cell (408)&(831) 332-4006 [EMAIL PROTECTED] -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]] On Behalf Of David Browne. Sent: Thursday, October 04, 2001 6:46 AM To: [EMAIL PROTECTED] Subject: Recovery log space spikes The maximum size of the recovery log is 5.5GB, my recovery log is 4.5 GB now and I have just recently increased it 1GB from 3.5GB. It filled up again last night. My database is 59.5 GB. Could I have a problem that is causing my recovery log to fill up or is this normal for this size of database? I am running TSM 3.7.4 on OS390 R 2.10. I have been testing TSM 4.1.4 and am planning to go to the new release in the next few weeks, could this help the situation? Below shows the past three days monitoring. * October 4, 2001 * tsm: ADSM3>q log Available Assigned MaximumMaximum Page Total Used Pct Max. Space Capacity ExtensionReduction Size Usable PagesUtil Pct (MB) (MB) (MB) (MB) (bytes) PagesUtil - - - --- -- - - 4,620 4,620 0 4,264 4,096 1,182,208 89,762 7.6100.0 Available Assigned Maximum Maximum Page TotalUsed Pct Max. Space CapacityExtension ReductionSize UsablePagesUtil Pct (MB) (MB)(MB)(MB) (bytes) Pages Util - - ------- ---- 60,908 60,908 06,852 4,096 15,592,448 13,839,06688.888.8 tsm: ADSM3> October 3, 2001 ** tsm: ADSM3>q log AvailableAssignedMaximum Maximum Page TotalUsed PctMax. Space Capacity Extension Reduction Size Usable PagesUtilPct (MB)(MB) (MB)(MB) (bytes) Pages Util - - - --- - - - 4,6204,620 0 2,480 4,096 1,182,208 546,31346.2 88.6 tsm: ADSM3>q db Available AssignedMaximum Maximum Page Total Used Pct Max. SpaceCapacity Extension Reduction Size Usable Pages Util Pct (MB)(MB)(MB) (MB) (bytes)Pages Util - -- - - ----- --- - - 60,908 60,908 0 7,208 4,096 15,592,44813,748,751 88.2 88.2 tsm: ADSM3> * October 2, 2001 tsm: ADSM3>q log AvailableAssigned Maximum Maximum Page Total Used Pct Max. SpaceCapacity Extension Reduction Size Usable Pages Util Pct (MB) (MB)(MB)(MB) (bytes) Pages Util - - - --- -- --- - - 4,620 4,6200 9884,096 1,182,208 928,83078.6 78.6 tsm: ADSM3>q db Available Assigned Maximum Maximum Page Total UsedPct Max. SpaceCapacity Extension Reduction Size Usable Pages Util
Re: Recovery log space spikes
Hi, I have an DB of 20Gb and an recovery log of 4Gb. During backup at night the maximum consumption is round about 50%. This consumption depends on how much data has been backed up. The database depends on how much data you backup and how many versions you save. But it seems normal to me. Ronald Bazuin -Original Message- From: David Browne. [mailto:[EMAIL PROTECTED]] Sent: donderdag 4 oktober 2001 13:46 To: [EMAIL PROTECTED] Subject: Recovery log space spikes The maximum size of the recovery log is 5.5GB, my recovery log is 4.5 GB now and I have just recently increased it 1GB from 3.5GB. It filled up again last night. My database is 59.5 GB. Could I have a problem that is causing my recovery log to fill up or is this normal for this size of database? I am running TSM 3.7.4 on OS390 R 2.10. I have been testing TSM 4.1.4 and am planning to go to the new release in the next few weeks, could this help the situation? Below shows the past three days monitoring. * October 4, 2001 * tsm: ADSM3>q log Available Assigned MaximumMaximum Page Total Used Pct Max. Space Capacity ExtensionReduction Size Usable PagesUtil Pct (MB) (MB) (MB) (MB) (bytes) PagesUtil - - - --- -- - - 4,620 4,620 0 4,264 4,096 1,182,208 89,762 7.6100.0 Available Assigned Maximum Maximum Page TotalUsed Pct Max. Space CapacityExtension ReductionSize UsablePagesUtil Pct (MB) (MB)(MB)(MB) (bytes) Pages Util - - ------- ---- 60,908 60,908 06,852 4,096 15,592,448 13,839,06688.888.8 tsm: ADSM3> October 3, 2001 ** tsm: ADSM3>q log AvailableAssignedMaximum Maximum Page TotalUsed PctMax. Space Capacity Extension ReductionSize Usable PagesUtilPct (MB)(MB) (MB)(MB) (bytes) Pages Util - - - --- - - - 4,6204,620 02,480 4,096 1,182,208 546,31346.2 88.6 tsm: ADSM3>q db Available AssignedMaximum Maximum Page Total Used Pct Max. SpaceCapacity Extension Reduction Size Usable Pages Util Pct (MB)(MB)(MB) (MB) (bytes)Pages Util - -- - - ----- --- - - 60,908 60,908 0 7,208 4,096 15,592,44813,748,751 88.2 88.2 tsm: ADSM3> * October 2, 2001 tsm: ADSM3>q log AvailableAssigned Maximum Maximum Page Total Used Pct Max. SpaceCapacity Extension Reduction Size Usable Pages Util Pct (MB) (MB)(MB)(MB) (bytes) Pages Util - - - --- ----- - - 4,620 4,6200 9884,096 1,182,208 928,83078.6 78.6 tsm: ADSM3>q db Available Assigned Maximum Maximum Page Total UsedPct Max. SpaceCapacity Extension Reduction Size Usable Pages Util Pct (MB) (MB) (MB)
Database "trashing" (was Re: Recovery Log size / roll forward benefits)
On Thu, 2 Aug 2001 17:28:54 -0400, you wrote: >If I ask the list how rare it is, I'm sure I'll get flames from everybody >who HAS seen software corruption, and probably NO response from the many >people who are running fine. Guess again. >Maybe somebody in Tivoli support could venture a guess as to how likely it >is to have TSM trash its own database - and what steps might be taken to >reduce that risk. I have *never* seen TSM (or ADSM) "trash its own database". I've never seen any bug in any version of *SM lead directly to database loss. What I *have* seen are hardware errors that trash a database volume or (if not configured smartly) the entire database's worth of volumes. I've seen human error (read: manual deletion of database volumes at the OS level) whack a TSM database. (I've done that myself once.) The opportunity that Tivoli offers to (1) mirror database volumes, and (2) create multiple backups of the internal database is all the reduction of risk that, I think, one should feel entitled to ask for. This all leads to a semi-rant that I've wanted to make in this mailing list for a while. With the exception of a moderate number of known, documented, or generally acknowledged situations, I've *never* seen a properly constructed network of TSM server(s) and clients *not* work as advertised. The large majority of problems that I've experienced, and (I daresay) brought up in this mailing list, stem from one of five causes: --an improperly designed and/or constructed network --an improperly installed and/or configured TSM server --improperly installed and/or configured TSM clients --hardware failure or bugs --failed communication between people I'm not saying that TSM doesn't have faults; any software package of its complexity will have its bugs and its shortcomings. I'm not saying that Tivoli support (particularly level 1) doesn't have its problems.[1] What I *am* saying is that a stiff dose of RTFM (most emphatically a good read of the QuickStart manual) would probably pare down this mailing to about 50% of its present traffic level.[2] And would free up Tivoli Support Desk's workload to the point where those of us who have a legitimate need for support wouldn't have to wait 30 minutes to get to a warm body. [1] People who pitch and moan about Tivoli support have obviously *never* dealt with Veritas support. Or Legato support. Or Domino support. Or Microsoft "support". Tivoli support actually stacks up fairly well in the support arena. [2] And a good read of the administrative guide would cut *that* down by 50% again. -- Mark Stapleton ([EMAIL PROTECTED])
Re: Recovery Log size / roll forward benefits
I agree with Paul, it is pretty rare - but it can happen. You can't guarantee to prevent software or hardware corruption : you just have to plan the recovery path ... The last corruption I had was on a (pretty large) test DB when trying to see the impact of a problem I had at 3.7 (live) on the 4.1 (test) DB. Well, the 4.1 DB got corrupted, the server crashed and it wouldn't restart - I won't bore you with the details. The point is, I was able to recover the DB using the last FULL, INCRs and recovery log. This level of protection is essential for us. Like Paul, we have also had problems with the recovery log filling up too quickly (as well as the occasional 'pinned' logtail). Recently, we have escaped by the skin of our teeth when the recovery log hit 99% and the triggered INCR could not complete fast enough ( - trigger now lowered and we split expire inventory into a twice weekly run). I too have reservations about the 13GB log (although we certainly intend to use it) and I have been looking at timings for triggered INCRs; 13GB may take rather a long time ... Regards, Sheelagh -- Sheelagh Treweek Oxford University Computing Services Email: [EMAIL PROTECTED] Phone: +44 (0)1865 273205 Fax:-273275 +-+ | http://tsm-symposium.oucs.ox.ac.uk/ OXFORD 20/21 September 2001 | | REQUIREMENTS http://tsm-symposium.oucs.ox.ac.uk/requirements.html | +-+
Re: Recovery Log size / roll forward benefits
I think it is pretty rare, Linday. I count myself as being very unlucky in that regard. But as unlikely as it can get, I'd think you would still want to protect yourself against the possibility. Perhaps I am overly sensitive to it since I was bitten already. At 05:28 PM 8/2/2001 -0400, you wrote: >Thanks for the insight, Paul. In your long experience, you've probably seen >everything bad that can happen. I was assuming that software corruption was >rare. > >If I ask the list how rare it is, I'm sure I'll get flames from everybody >who HAS seen software corruption, and probably NO response from the many >people who are running fine. > >Maybe somebody in Tivoli support could venture a guess as to how likely it >is to have TSM trash its own database - and what steps might be taken to >reduce that risk. > > > -Original Message- > > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of > > Paul Zarnowski > > Sent: Thursday, August 02, 2001 4:16 PM > > To: [EMAIL PROTECTED] > > Subject: Re: Recovery Log size / roll forward benefits > > > > > > Lindsay, et al, > > > > It is very possible to lose your database but not the log (we have done > > it). The database can become corrupted through means other than hardware > > failure, though even a hardware-initiated corruption can occur even when > > your disk is mirrored. It is a GOOD IDEA to put the log on different > > disks, and if possible, a different disk subsystem than your database, if > > you can afford to. I don't view using logmode rollforward as an > > alternative to mirroring your database volumes. > > > > Since our full database backups take so long, we cannot afford to run full > > backups regularly (we do them twice a week). If you can afford to run a > > full backup regularly, then by all means do it. I think rollforward has > > value in environments such as ours, because it allows us to run more > > frequent incremental database backups. > > > > I will also dispute the opinion that a 13GB log will solve everyone's > > problems. It will help, for sure. But, there are situations where a > > thread has the log tail pinned and the log head is advancing quickly. The > > larger log will give the pinning thread longer to complete, but IMHO there > > will still be situations where the log can still wrap. Examples I have > > seen of a quickly advancing head include inventory expiration and delete > > filespace. We have recently automated a process to cancel inventory > > expiration when the log starts filling, and restart it later after the log > > utilization drops. This has helped us tremendously. Since implementing > > this, we have only gotten bitten by a large delete filespace. > > > > BTW, we have an 83GB database and a 5GB log. > > > > ..Paul > > > > At 10:54 AM 6/22/2001 -0400, Lindsay Morris wrote: > > >I'm with Kelly in thinking that roll-forward is unnecessary in > > most cases. > > > > > >The TSM admin guide seems to recommend roll-forward: > > > > > > "To get the best protection for your TSM data, you should use ... > > >mirrored copies of your database and recovery log, with the recovery log > > >mode set to roll-forward" > > > "Roll-forward mode offers the greatest protection for your data." > > > "(Roll-forward) With an intact recovery log, recover to the > > most current > > >state with no loss of client data." > > > > > >But for roll-forward mode to be useful, you have to lose your > > TSM database > > >volumes, yet somehow NOT lose the TSM log volumes. > > >How is that ever going to happen? > > > > > >Most people mirror the TSM database on a separate disk (and, > > ideally, on a > > >separate disk controller). So they have to lose TWO disk drives > > before they > > >lose the database. > > > > > >If TWO disk drives die at once, the whole machine is probably > > dead in a fire > > >or something, right? So the recovery log volumes are gone too, and > > >roll-forward can't be used. You have to restore the DB from tape, and > > >that's as current as you can get - just as if you had NEVER used > > >roll-forward. > > > > > >Does anybody have a situation that they feel really merits > > >logmode=rollforward? If so, would you mind discussing it briefly? > > > > > > > > > > > > > > > > > > > > > > -Orig
Re: Recovery Log size / roll forward benefits
Thanks for your input, Tony. If I understand you, you're concerned that your log might fill up, so you set logmode roll-forward just because dbbackuptrigger requires it. Then dbbackuptrigger kicks off incremental db dbackups which clean out the log periodically. Is that what you mean? > -Original Message- > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of > Garrison, Tony > Sent: Thursday, August 02, 2001 2:21 PM > To: [EMAIL PROTECTED] > Subject: Re: Recovery Log size / roll forward benefits > > > We use Roll-forward here. This is primarily due to the fact that > we can set > the db trigger to backup the db if the log reaches a predefined > percentage. > This is extremely important due to the size of our environment. We would > rather run in normal, but until we can use the db trigger we are > stuck with > what we have. > > -Original Message- > From: Lindsay Morris [mailto:[EMAIL PROTECTED]] > Sent: Friday, June 22, 2001 9:55 AM > To: [EMAIL PROTECTED] > Subject:Re: Recovery Log size / roll forward benefits > > I'm with Kelly in thinking that roll-forward is unnecessary in most cases. > > The TSM admin guide seems to recommend roll-forward: > > "To get the best protection for your TSM data, you should use ... > mirrored copies of your database and recovery log, with the recovery log > mode set to roll-forward" > "Roll-forward mode offers the greatest protection for your data." > "(Roll-forward) With an intact recovery log, recover to the > most current > state with no loss of client data." > > But for roll-forward mode to be useful, you have to lose your TSM database > volumes, yet somehow NOT lose the TSM log volumes. > How is that ever going to happen? > > Most people mirror the TSM database on a separate disk (and, ideally, on a > separate disk controller). So they have to lose TWO disk drives > before they > lose the database. > > If TWO disk drives die at once, the whole machine is probably > dead in a fire > or something, right? So the recovery log volumes are gone too, and > roll-forward can't be used. You have to restore the DB from tape, and > that's as current as you can get - just as if you had NEVER used > roll-forward. > > Does anybody have a situation that they feel really merits > logmode=rollforward? If so, would you mind discussing it briefly? > > > > > > > > -Original Message- > > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of > > Kelly J. Lipp > > Sent: Friday, June 22, 2001 10:36 AM > > To: [EMAIL PROTECTED] > > Subject: Re: Recovery Log size > > > > > > Doesn't this bring back the issue of Roll-Forward vs. not? I guess I'm > > still in the not camp with no compelling reason to join the > other. Also, > > for those in the other camp, I'm thinking if your environment is > > so large as > > to fill a 13 GB log in a 24 hour period perhaps the environment > is busting > > at the seems in other areas and should perhaps be split anyway. > > > > Even if larger logs were supported, how long would a db restore > take with > > roll-forward enabled? I'm thinking way too long. > > > > Perhaps someone can share their longest db restore story with us. > > > > Kelly J. Lipp > > Storage Solutions Specialists, Inc. > > PO Box 51313 > > Colorado Springs CO 80949-1313 > > (719) 531-5926 > > Fax: (240) 539-7175 > > Email: [EMAIL PROTECTED] or [EMAIL PROTECTED] > > www.storsol.com > > www.storserver.com > > > > > > -Original Message- > > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of > > Nicholas Cassimatis > > Sent: Friday, June 22, 2001 8:22 AM > > To: [EMAIL PROTECTED] > > Subject: Re: Recovery Log size > > > > > > With the current limit of 5.3GB, and the steps we've all taken to work > > within that limit, I don't think many people will be hitting the > > 13GB limit > > all that quick. An incremental database backup will still > flush the log, > > and, with 13GB, I think we'll be OK if we don't change the way we > > are doing > > our business. > > > > Nick Cassimatis > > [EMAIL PROTECTED] > > > > > > > > > > >An FYI - TSM 4.2 will increase the limit to 13GB. > > > > I think we're all wondering when the first voice will be heard asking > > when the 13 GB limit will be increased. And I think we're all wondering > > why the increase was little more than a doubling of the current limit - > > which customers are already straining to go beyond. As an Enterprise > > level product, I would expect TSM to be a lot more open-ended. Unless > > this boost was merely a stop-gap in advance of major architectural > > relief, it's not going to be enough to keep up with the demand. > > > >Richard Sims, BU > > >
Re: Recovery Log size / roll forward benefits
Thanks for the insight, Paul. In your long experience, you've probably seen everything bad that can happen. I was assuming that software corruption was rare. If I ask the list how rare it is, I'm sure I'll get flames from everybody who HAS seen software corruption, and probably NO response from the many people who are running fine. Maybe somebody in Tivoli support could venture a guess as to how likely it is to have TSM trash its own database - and what steps might be taken to reduce that risk. > -Original Message- > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of > Paul Zarnowski > Sent: Thursday, August 02, 2001 4:16 PM > To: [EMAIL PROTECTED] > Subject: Re: Recovery Log size / roll forward benefits > > > Lindsay, et al, > > It is very possible to lose your database but not the log (we have done > it). The database can become corrupted through means other than hardware > failure, though even a hardware-initiated corruption can occur even when > your disk is mirrored. It is a GOOD IDEA to put the log on different > disks, and if possible, a different disk subsystem than your database, if > you can afford to. I don't view using logmode rollforward as an > alternative to mirroring your database volumes. > > Since our full database backups take so long, we cannot afford to run full > backups regularly (we do them twice a week). If you can afford to run a > full backup regularly, then by all means do it. I think rollforward has > value in environments such as ours, because it allows us to run more > frequent incremental database backups. > > I will also dispute the opinion that a 13GB log will solve everyone's > problems. It will help, for sure. But, there are situations where a > thread has the log tail pinned and the log head is advancing quickly. The > larger log will give the pinning thread longer to complete, but IMHO there > will still be situations where the log can still wrap. Examples I have > seen of a quickly advancing head include inventory expiration and delete > filespace. We have recently automated a process to cancel inventory > expiration when the log starts filling, and restart it later after the log > utilization drops. This has helped us tremendously. Since implementing > this, we have only gotten bitten by a large delete filespace. > > BTW, we have an 83GB database and a 5GB log. > > ..Paul > > At 10:54 AM 6/22/2001 -0400, Lindsay Morris wrote: > >I'm with Kelly in thinking that roll-forward is unnecessary in > most cases. > > > >The TSM admin guide seems to recommend roll-forward: > > > > "To get the best protection for your TSM data, you should use ... > >mirrored copies of your database and recovery log, with the recovery log > >mode set to roll-forward" > > "Roll-forward mode offers the greatest protection for your data." > > "(Roll-forward) With an intact recovery log, recover to the > most current > >state with no loss of client data." > > > >But for roll-forward mode to be useful, you have to lose your > TSM database > >volumes, yet somehow NOT lose the TSM log volumes. > >How is that ever going to happen? > > > >Most people mirror the TSM database on a separate disk (and, > ideally, on a > >separate disk controller). So they have to lose TWO disk drives > before they > >lose the database. > > > >If TWO disk drives die at once, the whole machine is probably > dead in a fire > >or something, right? So the recovery log volumes are gone too, and > >roll-forward can't be used. You have to restore the DB from tape, and > >that's as current as you can get - just as if you had NEVER used > >roll-forward. > > > >Does anybody have a situation that they feel really merits > >logmode=rollforward? If so, would you mind discussing it briefly? > > > > > > > > > > > > > > > -Original Message- > > > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On > Behalf Of > > > Kelly J. Lipp > > > Sent: Friday, June 22, 2001 10:36 AM > > > To: [EMAIL PROTECTED] > > > Subject: Re: Recovery Log size > > > > > > > > > Doesn't this bring back the issue of Roll-Forward vs. not? I > guess I'm > > > still in the not camp with no compelling reason to join the > other. Also, > > > for those in the other camp, I'm thinking if your environment is > > > so large as > > > to fill a 13 GB log in a 24 hour period perhaps the > environment is busting > > > at the seems in other areas and shou
Re: Recovery Log size / roll forward benefits
We use Roll-forward here. This is primarily due to the fact that we can set the db trigger to backup the db if the log reaches a predefined percentage. This is extremely important due to the size of our environment. We would rather run in normal, but until we can use the db trigger we are stuck with what we have. -Original Message- From: Lindsay Morris [mailto:[EMAIL PROTECTED]] Sent: Friday, June 22, 2001 9:55 AM To: [EMAIL PROTECTED] Subject:Re: Recovery Log size / roll forward benefits I'm with Kelly in thinking that roll-forward is unnecessary in most cases. The TSM admin guide seems to recommend roll-forward: "To get the best protection for your TSM data, you should use ... mirrored copies of your database and recovery log, with the recovery log mode set to roll-forward" "Roll-forward mode offers the greatest protection for your data." "(Roll-forward) With an intact recovery log, recover to the most current state with no loss of client data." But for roll-forward mode to be useful, you have to lose your TSM database volumes, yet somehow NOT lose the TSM log volumes. How is that ever going to happen? Most people mirror the TSM database on a separate disk (and, ideally, on a separate disk controller). So they have to lose TWO disk drives before they lose the database. If TWO disk drives die at once, the whole machine is probably dead in a fire or something, right? So the recovery log volumes are gone too, and roll-forward can't be used. You have to restore the DB from tape, and that's as current as you can get - just as if you had NEVER used roll-forward. Does anybody have a situation that they feel really merits logmode=rollforward? If so, would you mind discussing it briefly? > -Original Message- > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of > Kelly J. Lipp > Sent: Friday, June 22, 2001 10:36 AM > To: [EMAIL PROTECTED] > Subject: Re: Recovery Log size > > > Doesn't this bring back the issue of Roll-Forward vs. not? I guess I'm > still in the not camp with no compelling reason to join the other. Also, > for those in the other camp, I'm thinking if your environment is > so large as > to fill a 13 GB log in a 24 hour period perhaps the environment is busting > at the seems in other areas and should perhaps be split anyway. > > Even if larger logs were supported, how long would a db restore take with > roll-forward enabled? I'm thinking way too long. > > Perhaps someone can share their longest db restore story with us. > > Kelly J. Lipp > Storage Solutions Specialists, Inc. > PO Box 51313 > Colorado Springs CO 80949-1313 > (719) 531-5926 > Fax: (240) 539-7175 > Email: [EMAIL PROTECTED] or [EMAIL PROTECTED] > www.storsol.com > www.storserver.com > > > -Original Message- > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of > Nicholas Cassimatis > Sent: Friday, June 22, 2001 8:22 AM > To: [EMAIL PROTECTED] > Subject: Re: Recovery Log size > > > With the current limit of 5.3GB, and the steps we've all taken to work > within that limit, I don't think many people will be hitting the > 13GB limit > all that quick. An incremental database backup will still flush the log, > and, with 13GB, I think we'll be OK if we don't change the way we > are doing > our business. > > Nick Cassimatis > [EMAIL PROTECTED] > > > > > >An FYI - TSM 4.2 will increase the limit to 13GB. > > I think we're all wondering when the first voice will be heard asking > when the 13 GB limit will be increased. And I think we're all wondering > why the increase was little more than a doubling of the current limit - > which customers are already straining to go beyond. As an Enterprise > level product, I would expect TSM to be a lot more open-ended. Unless > this boost was merely a stop-gap in advance of major architectural > relief, it's not going to be enough to keep up with the demand. > >Richard Sims, BU >
Re: Recovery Log size / roll forward benefits
Lindsay, et al, It is very possible to lose your database but not the log (we have done it). The database can become corrupted through means other than hardware failure, though even a hardware-initiated corruption can occur even when your disk is mirrored. It is a GOOD IDEA to put the log on different disks, and if possible, a different disk subsystem than your database, if you can afford to. I don't view using logmode rollforward as an alternative to mirroring your database volumes. Since our full database backups take so long, we cannot afford to run full backups regularly (we do them twice a week). If you can afford to run a full backup regularly, then by all means do it. I think rollforward has value in environments such as ours, because it allows us to run more frequent incremental database backups. I will also dispute the opinion that a 13GB log will solve everyone's problems. It will help, for sure. But, there are situations where a thread has the log tail pinned and the log head is advancing quickly. The larger log will give the pinning thread longer to complete, but IMHO there will still be situations where the log can still wrap. Examples I have seen of a quickly advancing head include inventory expiration and delete filespace. We have recently automated a process to cancel inventory expiration when the log starts filling, and restart it later after the log utilization drops. This has helped us tremendously. Since implementing this, we have only gotten bitten by a large delete filespace. BTW, we have an 83GB database and a 5GB log. ..Paul At 10:54 AM 6/22/2001 -0400, Lindsay Morris wrote: >I'm with Kelly in thinking that roll-forward is unnecessary in most cases. > >The TSM admin guide seems to recommend roll-forward: > > "To get the best protection for your TSM data, you should use ... >mirrored copies of your database and recovery log, with the recovery log >mode set to roll-forward" > "Roll-forward mode offers the greatest protection for your data." > "(Roll-forward) With an intact recovery log, recover to the most current >state with no loss of client data." > >But for roll-forward mode to be useful, you have to lose your TSM database >volumes, yet somehow NOT lose the TSM log volumes. >How is that ever going to happen? > >Most people mirror the TSM database on a separate disk (and, ideally, on a >separate disk controller). So they have to lose TWO disk drives before they >lose the database. > >If TWO disk drives die at once, the whole machine is probably dead in a fire >or something, right? So the recovery log volumes are gone too, and >roll-forward can't be used. You have to restore the DB from tape, and >that's as current as you can get - just as if you had NEVER used >roll-forward. > >Does anybody have a situation that they feel really merits >logmode=rollforward? If so, would you mind discussing it briefly? > > > > > > > > -Original Message- > > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of > > Kelly J. Lipp > > Sent: Friday, June 22, 2001 10:36 AM > > To: [EMAIL PROTECTED] > > Subject: Re: Recovery Log size > > > > > > Doesn't this bring back the issue of Roll-Forward vs. not? I guess I'm > > still in the not camp with no compelling reason to join the other. Also, > > for those in the other camp, I'm thinking if your environment is > > so large as > > to fill a 13 GB log in a 24 hour period perhaps the environment is busting > > at the seems in other areas and should perhaps be split anyway. > > > > Even if larger logs were supported, how long would a db restore take with > > roll-forward enabled? I'm thinking way too long. > > > > Perhaps someone can share their longest db restore story with us. > > > > Kelly J. Lipp > > Storage Solutions Specialists, Inc. > > PO Box 51313 > > Colorado Springs CO 80949-1313 > > (719) 531-5926 > > Fax: (240) 539-7175 > > Email: [EMAIL PROTECTED] or [EMAIL PROTECTED] > > www.storsol.com > > www.storserver.com > > > > > > -Original Message- > > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of > > Nicholas Cassimatis > > Sent: Friday, June 22, 2001 8:22 AM > > To: [EMAIL PROTECTED] > > Subject: Re: Recovery Log size > > > > > > With the current limit of 5.3GB, and the steps we've all taken to work > > within that limit, I don't think many people will be hitting the > > 13GB limit > > all that quick. An incremental database backup will still flush the log, > > and, with 13GB, I think we'll be OK if we don't ch
Re: Recovery log Pct. Utilized
Gill, That's unfortunatelly a temporary bug, you can use the "ckpt" command to reset the pct_util. Regards Georges
Re: Recovery log Pct. Utilized
>Are there any processes or sessions running? They could be holding the log >utilization up. This is a tough one to answer Nick. The server kicked me out the first time I tried to query anything. I was lucky enough to have a console window open and could see the system wasn't allowing anyone to get in, backups were either missed or failed. Unfortunately I don't know the exact message that was scrolling the screen but it was related to the log being full, and obviously since the log was full it didn't write to it. The last log entry was about 10:45 PM then it picks up again after I extended the log and restarted at 5 AM. I could see during that evening the server was getting lots of log entries for this "Session open with XXX.XXX.XXX.XXX failed due to connection refusal". There are quite a few other servers which had the same thing going on so there are hundreds of these. Unfortunately I don't have acces to these nodes to stop and start the Scheduler service to make the backup kick in. I obviously never expected this to happen to me. With as few backups I have on Saturday night, doing one db backup and one incremental daily I thought was enough to keep a 4GB log from filling up. I can see that I was mistaken and have now taken the necessary steps to, hopefully, not see this again. The burning question still is this: If I have scratch tapes available, a schedule with 1 full and 1 incremental db backup daily about 12 hours apart, and the log fills, will the full or incremental db backup be able to run and clear the log or do I still have to shutdown and make space? Geoff Gill TSM Administrator NT Systems Support Engineer SAIC E-Mail: [EMAIL PROTECTED] Phone: (858) 826-4062 Pager: (888) 997-9614
Re: Recovery log Pct. Utilized
If nothing is running (q ses, q pro) and the log is not clearing, the only way I have been able to clear it is to bounce the ADSM instance. Alternatively, kill -11 will create a core dump for them to analyze. Jeff Bach Home Office Open Systems Engineering Wal-Mart Stores, Inc. WAL-MART CONFIDENTIAL -Original Message- From: Gill, Geoffrey L. [SMTP:[EMAIL PROTECTED]] Sent: Sunday, June 24, 2001 8:13 AM To: [EMAIL PROTECTED] Subject:Recovery log Pct. Utilized Can anyone here or from Tivoli explain why the recovery log percent utilized doesn't reset to 0 as soon as the full database backup is complete? This is very annoying, especially since I was called at 3 AM. I'm sitting here with sessions disabled to minimize activity, waiting for this damn thing to reset. I hate to enable sessions now had have to go through this whole thing again Available Space (MB): 4,196 Assigned Capacity (MB): 4,196 Maximum Extension (MB): 0 Maximum Reduction (MB): 88 Page Size (bytes): 4,096 Total Usable Pages: 1,073,664 Used Pages: 1,050,659 Pct Util: 97.9 Max. Pct Util: 97.9 Physical Volumes: 4 Log Pool Pages: 2,048 Log Pool Pct. Util: 0.05 Log Pool Pct. Wait: 0.00 Cumulative Consumption (MB): 0.06 Consumption Reset Date/Time: 06/24/01 05:48:00 Geoff Gill TSM Administrator NT Systems Support Engineer SAIC E-Mail: [EMAIL PROTECTED] Phone: (858) 826-4062 Pager: (888) 997-9614 ** This email and any files transmitted with it are confidential and intended solely for the individual or entity to whom they are addressed. If you have received this email in error destroy it immediately. **
Re: Recovery log Pct. Utilized
Geoff, Are there any processes or sessions running? They could be holding the log utilization up. I've also seen it take a while to commit the log to the database after the backup ends - you have 4GB of transactions in the log to commit to the database there, and, depending on your system, that could take a while (especially if the log and DB volumes are on the same spindle). When this happens to someone on my team, I always caution patience - "the watched log never clears." Nick Cassimatis [EMAIL PROTECTED] "Gill, Geoffrey L."To: [EMAIL PROTECTED] Subject: Recovery log Pct. Utilized Sent by: "ADSM: Dist Stor Manager" <[EMAIL PROTECTED] T.EDU> 06/24/01 09:12 AM Please respond to "ADSM: Dist Stor Manager" Can anyone here or from Tivoli explain why the recovery log percent utilized doesn't reset to 0 as soon as the full database backup is complete? This is very annoying, especially since I was called at 3 AM. I'm sitting here with sessions disabled to minimize activity, waiting for this damn thing to reset. I hate to enable sessions now had have to go through this whole thing again Available Space (MB): 4,196 Assigned Capacity (MB): 4,196 Maximum Extension (MB): 0 Maximum Reduction (MB): 88 Page Size (bytes): 4,096 Total Usable Pages: 1,073,664 Used Pages: 1,050,659 Pct Util: 97.9 Max. Pct Util: 97.9 Physical Volumes: 4 Log Pool Pages: 2,048 Log Pool Pct. Util: 0.05 Log Pool Pct. Wait: 0.00 Cumulative Consumption (MB): 0.06 Consumption Reset Date/Time: 06/24/01 05:48:00 Geoff Gill TSM Administrator NT Systems Support Engineer SAIC E-Mail: [EMAIL PROTECTED] Phone: (858) 826-4062 Pager: (888) 997-9614
Re: recovery log full
Geoff, I "THINK" your db backup will still run and clear your log as long as you have scratch tapes available, but I'm not 100% sure on that. I do know that it will NOT run if you are out of scratch tapes AND your recovery log is full. Why not go ahead and extend your recovery log anyway just to be sure...? Thank you, Jeff Caffey Enterprise Systems Programmer (AIX & Storage Administrator) Pier 1 imports, Inc. - Information Services [EMAIL PROTECTED] Voice: (817) 252-6222 Fax: (817) 252-7299 -Original Message- From: Gill, Geoffrey L. [mailto:[EMAIL PROTECTED]] Sent: Sunday, June 24, 2001 06:36 To: [EMAIL PROTECTED] Subject:recovery log full Question: If the recovery log is full and a database backup is sheduled to run, will the db backup run since the server is up and clear the log? Geoff Gill TSM Administrator NT Systems Support Engineer SAIC E-Mail: [EMAIL PROTECTED] Phone: (858) 826-4062 Pager: (888) 997-9614
Re: Recovery Log size
At 08:35 AM 6/22/2001 -0600, Kelly J. Lipp wrote: >Doesn't this bring back the issue of Roll-Forward vs. not? I guess I'm >still in the not camp with no compelling reason to join the other. Also, >for those in the other camp, I'm thinking if your environment is so large as >to fill a 13 GB log in a 24 hour period perhaps the environment is busting >at the seems in other areas and should perhaps be split anyway. > >Even if larger logs were supported, how long would a db restore take with >roll-forward enabled? I'm thinking way too long. I don't believe this is a pure issue of database size. Rather, I think it is the combination of performance characteristics of the server, the network, and the client - the entire environment. roll-forward makes sense when the time it takes to run an incremental backup is much less than what it takes to run a full backup. If you can afford to run a full backup every day, then go for it. Your recovery time will be short. The time it takes depends not just on DB size, but also tape speed, processor speed, processor memory, etc. When we upgraded our server, we found that database backups ran faster - no surprise there. We run roll-forward because we don't have enough wall clock time to perform a full backup every day, but we want the protection. While recovery time is an issue, backup time can be too and it is in our environment. In the configuration we have now, the log is a key limitation that might have forced us to split our database. And this is not just because the log size controls the time interval between incremental backups. In logmode=normal, the log size also affects how long a single transaction can pin the log tail without causing an operational problem. The larger the log, the longer the transaction can be in flight. Even with logmode=rollforward this can be an issue, and has been for us. As servers and networks get faster, this becomes a bigger issue, IMHO. Especially in complex environments that have a variety of clients. ..Paul
Re: Recovery Log size
With database sizes on client systems increasing each day to 100s Gigs from tens of Gigs, if the log size only goes up by 2.5 times the current size limit, some piece has to get 4 times faster to maintain the current way things are working. If Tivoli is not going forward faster than its customers, it is not going forward. TSM either needs to be ported to use a 21st century database or at least copy what they are doing today. Multiple recovery log volumes, replication between servers for recundancy, the ability to actually use 2+ to 8 Gigs of database buffer memory. Jeff Bach > -Original Message- > From: Nicholas Cassimatis [SMTP:[EMAIL PROTECTED]] > Sent: Friday, June 22, 2001 9:22 AM > To: [EMAIL PROTECTED] > Subject: Re: Recovery Log size > > With the current limit of 5.3GB, and the steps we've all taken to work > within that limit, I don't think many people will be hitting the 13GB > limit > all that quick. An incremental database backup will still flush the log, > and, with 13GB, I think we'll be OK if we don't change the way we are > doing > our business. > > Nick Cassimatis > [EMAIL PROTECTED] > > > > > >An FYI - TSM 4.2 will increase the limit to 13GB. > > I think we're all wondering when the first voice will be heard asking > when the 13 GB limit will be increased. And I think we're all wondering > why the increase was little more than a doubling of the current limit - > which customers are already straining to go beyond. As an Enterprise > level product, I would expect TSM to be a lot more open-ended. Unless > this boost was merely a stop-gap in advance of major architectural > relief, it's not going to be enough to keep up with the demand. > >Richard Sims, BU ** This email and any files transmitted with it are confidential and intended solely for the individual or entity to whom they are addressed. If you have received this email in error destroy it immediately. **
Re: Recovery Log size
>With the current limit of 5.3GB, and the steps we've all taken to work >within that limit, I don't think many people will be hitting the 13GB limit >all that quick. An incremental database backup will still flush the log, >and, with 13GB, I think we'll be OK if we don't change the way we are doing >our business. Hi, Nick - I agree that a lot of us will be fine with a 13 GB limit: neither you nor I anticipate getting near it in our processing. In the abstract, though, I take the long-term view, over the whole base of the product... One can sum it up by saying that a gigabyte isn't what it used to be. It wasn't that long ago when a mainframe with 64 MB of memory was awesome. A few years later and here we are accustomed to servers with a couple of gigabytes of memory. I remember IBM getting a reality check when I worked in a large corporation: VSAM was architected without a realistic view of the future, and referenced its data via a Relative Byte Address. Well, one day we added a disk to our growing, VSAM-based IMS database and VSAM could not address it. Very big ouch. 13 GB may seem like a lot for the Recovery Log; but, then, the customers who today are fighting the 5.3 GB limit thought that was plenty - until their corporate data grew. Consider also that 13 GB is less than half the size of today's hard drives, and that lends further perspective. And that we were limited to 5.3 GB for years, instead of seeing it grow over the evolution of the product. >From the perspective of DP history, I see this boost as a modest improvement, but yet another limit that larger customers will soon approach. Again, TSM is supposed to be an Enterprise-level product, and as such should have far more expansive architecture than it's currently exhibiting. As many customers have been complaining, the TSM database has been the dominant constraining element in the product, and sorely needs to be addressed by Tivoli. Richard Sims, BU
Re: Recovery Log size / roll forward benefits
I'm with Kelly in thinking that roll-forward is unnecessary in most cases. The TSM admin guide seems to recommend roll-forward: "To get the best protection for your TSM data, you should use ... mirrored copies of your database and recovery log, with the recovery log mode set to roll-forward" "Roll-forward mode offers the greatest protection for your data." "(Roll-forward) With an intact recovery log, recover to the most current state with no loss of client data." But for roll-forward mode to be useful, you have to lose your TSM database volumes, yet somehow NOT lose the TSM log volumes. How is that ever going to happen? Most people mirror the TSM database on a separate disk (and, ideally, on a separate disk controller). So they have to lose TWO disk drives before they lose the database. If TWO disk drives die at once, the whole machine is probably dead in a fire or something, right? So the recovery log volumes are gone too, and roll-forward can't be used. You have to restore the DB from tape, and that's as current as you can get - just as if you had NEVER used roll-forward. Does anybody have a situation that they feel really merits logmode=rollforward? If so, would you mind discussing it briefly? > -Original Message- > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of > Kelly J. Lipp > Sent: Friday, June 22, 2001 10:36 AM > To: [EMAIL PROTECTED] > Subject: Re: Recovery Log size > > > Doesn't this bring back the issue of Roll-Forward vs. not? I guess I'm > still in the not camp with no compelling reason to join the other. Also, > for those in the other camp, I'm thinking if your environment is > so large as > to fill a 13 GB log in a 24 hour period perhaps the environment is busting > at the seems in other areas and should perhaps be split anyway. > > Even if larger logs were supported, how long would a db restore take with > roll-forward enabled? I'm thinking way too long. > > Perhaps someone can share their longest db restore story with us. > > Kelly J. Lipp > Storage Solutions Specialists, Inc. > PO Box 51313 > Colorado Springs CO 80949-1313 > (719) 531-5926 > Fax: (240) 539-7175 > Email: [EMAIL PROTECTED] or [EMAIL PROTECTED] > www.storsol.com > www.storserver.com > > > -Original Message- > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of > Nicholas Cassimatis > Sent: Friday, June 22, 2001 8:22 AM > To: [EMAIL PROTECTED] > Subject: Re: Recovery Log size > > > With the current limit of 5.3GB, and the steps we've all taken to work > within that limit, I don't think many people will be hitting the > 13GB limit > all that quick. An incremental database backup will still flush the log, > and, with 13GB, I think we'll be OK if we don't change the way we > are doing > our business. > > Nick Cassimatis > [EMAIL PROTECTED] > > > > > >An FYI - TSM 4.2 will increase the limit to 13GB. > > I think we're all wondering when the first voice will be heard asking > when the 13 GB limit will be increased. And I think we're all wondering > why the increase was little more than a doubling of the current limit - > which customers are already straining to go beyond. As an Enterprise > level product, I would expect TSM to be a lot more open-ended. Unless > this boost was merely a stop-gap in advance of major architectural > relief, it's not going to be enough to keep up with the demand. > >Richard Sims, BU >
Re: Recovery Log size
Doesn't this bring back the issue of Roll-Forward vs. not? I guess I'm still in the not camp with no compelling reason to join the other. Also, for those in the other camp, I'm thinking if your environment is so large as to fill a 13 GB log in a 24 hour period perhaps the environment is busting at the seems in other areas and should perhaps be split anyway. Even if larger logs were supported, how long would a db restore take with roll-forward enabled? I'm thinking way too long. Perhaps someone can share their longest db restore story with us. Kelly J. Lipp Storage Solutions Specialists, Inc. PO Box 51313 Colorado Springs CO 80949-1313 (719) 531-5926 Fax: (240) 539-7175 Email: [EMAIL PROTECTED] or [EMAIL PROTECTED] www.storsol.com www.storserver.com -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of Nicholas Cassimatis Sent: Friday, June 22, 2001 8:22 AM To: [EMAIL PROTECTED] Subject: Re: Recovery Log size With the current limit of 5.3GB, and the steps we've all taken to work within that limit, I don't think many people will be hitting the 13GB limit all that quick. An incremental database backup will still flush the log, and, with 13GB, I think we'll be OK if we don't change the way we are doing our business. Nick Cassimatis [EMAIL PROTECTED] >An FYI - TSM 4.2 will increase the limit to 13GB. I think we're all wondering when the first voice will be heard asking when the 13 GB limit will be increased. And I think we're all wondering why the increase was little more than a doubling of the current limit - which customers are already straining to go beyond. As an Enterprise level product, I would expect TSM to be a lot more open-ended. Unless this boost was merely a stop-gap in advance of major architectural relief, it's not going to be enough to keep up with the demand. Richard Sims, BU
Re: Recovery Log size
With the current limit of 5.3GB, and the steps we've all taken to work within that limit, I don't think many people will be hitting the 13GB limit all that quick. An incremental database backup will still flush the log, and, with 13GB, I think we'll be OK if we don't change the way we are doing our business. Nick Cassimatis [EMAIL PROTECTED] >An FYI - TSM 4.2 will increase the limit to 13GB. I think we're all wondering when the first voice will be heard asking when the 13 GB limit will be increased. And I think we're all wondering why the increase was little more than a doubling of the current limit - which customers are already straining to go beyond. As an Enterprise level product, I would expect TSM to be a lot more open-ended. Unless this boost was merely a stop-gap in advance of major architectural relief, it's not going to be enough to keep up with the demand. Richard Sims, BU
Re: Recovery Log size
>An FYI - TSM 4.2 will increase the limit to 13GB. I think we're all wondering when the first voice will be heard asking when the 13 GB limit will be increased. And I think we're all wondering why the increase was little more than a doubling of the current limit - which customers are already straining to go beyond. As an Enterprise level product, I would expect TSM to be a lot more open-ended. Unless this boost was merely a stop-gap in advance of major architectural relief, it's not going to be enough to keep up with the demand. Richard Sims, BU
Re: Recovery Log size
An FYI - TSM 4.2 will increase the limit to 13GB. Available son from a Tivoli rep near you. At 10:47 AM 6/21/2001 -0500, you wrote: >Maybe the instructor just wanted a good eval, so he told you all something >you wanted to hear ;-) > >I have heard that development was working on getting around the >limitation, but not that any special "fix" was available. I asked an >IBM'er if he had heard anything new on this, he said no and he chats with >internal TSM folks on a daily basis. > >I expect that when it is fixed, it will be broadcast-- on this list, and >at any SHARE, Storage symposium, Oxford- type meeting. > >lisa > > > > >"Gill, Geoffrey L." <[EMAIL PROTECTED]> >06/21/2001 08:31 AM >Please respond to "ADSM: Dist Stor Manager" > > > To: [EMAIL PROTECTED] > cc: (bcc: Lisa Cabanas/SC/MODOT) > Subject:Recovery Log size > > > >I posted this 2 days ago but saw no responses. Did it make it to the list? > >Hello all, > >Last week I attended the 2 day DRM class in L.A.. In one the discussions >we >had it was mentioned by the instructor that the size of the recovery log >could be over the 5.5gb current limit. I questioned him on this since >there >has been much discussion on it here, quite recently in fact. According to >him anyone with this size problem could get a special fix from Tivoli. > >Has anyone else heard of this? Has anyone had this problem and got this >special fix? > >Geoff Gill >TSM Administrator >NT Systems Support Engineer >SAIC >E-Mail: [EMAIL PROTECTED] >Phone: (858) 826-4062 >Pager: (888) 997-9614 Money is not the root of all evil - full backups are.
Re: Recovery Log size
Per the "Summary of Changes for Tivoli Storage Manager Version 4" section of the "Quick Start" manual, the "Changes for Version 4 Release 2 - June 2001" topic lists: Maximum Size of the Recovery Log Increased The maximum size of the recovery log is increased to 13GB. Significantly increasing the size of your recovery log could also significantly increase the time required to restart the server, to back up the database, and to restore the database. JRC > -- > From: Paul Zarnowski[SMTP:[EMAIL PROTECTED]] > Reply To: ADSM: Dist Stor Manager > Sent: Thursday, June 21, 2001 9:55 AM > To: [EMAIL PROTECTED] > Subject: Re: Recovery Log size > > Yes, it made it to the list. I've been anxiously awaiting a response from > someone confirming this. I am unaware of this "special fix", but we could > certainly benefit from it. > > I have heard that some relief *may* be in a new release of TSM, perhaps > 4.2, which I just received in the mail yesterday. I haven't had a chance > to check this. Anyone else know if this is in 4.2? > > ..Paul > > At 06:31 AM 6/21/2001 -0700, you wrote: > >I posted this 2 days ago but saw no responses. Did it make it to the > list? > > > >Hello all, > > > >Last week I attended the 2 day DRM class in L.A.. In one the discussions > we > >had it was mentioned by the instructor that the size of the recovery log > >could be over the 5.5gb current limit. I questioned him on this since > there > >has been much discussion on it here, quite recently in fact. According to > >him anyone with this size problem could get a special fix from Tivoli. > > > >Has anyone else heard of this? Has anyone had this problem and got this > >special fix? > > > >Geoff Gill > >TSM Administrator > >NT Systems Support Engineer > >SAIC > >E-Mail: [EMAIL PROTECTED] > >Phone: (858) 826-4062 > >Pager: (888) 997-9614 >
