Orphaned archive logs (was: tsm archive log full, causes repeated backups)

2014-08-30 Thread Prather, Wanda
Well.  
We just got hit with the same problem - 10 triggered DB backups in a row 
without clearing the archivelog.
TSM on Windows 6.3.4.300.

We did figure out what - just not why.

After adding an archlogoverflow directory and watching MORE db backups, what we 
found was that every time the DB backup ran, it did clear the recent stuff out 
of archlogoverflow and about 55 G out of the 1.2 TB archivelog directory.  So 
we could keep running the most critical part of our load and stay up.

Further investigation showed that we had a big groups of what I will call 
orphaned log files in the archivelog directory.  There was a chunk of log 
files from 7/11, a chunk from 8/20, 8/22, and 8/27.  Then a gap in time, then 
the log files since the most recent DB backup.  

So whatever hit us, hit once on 7/11, then started happening big time on 8/20.  
Left more than 700G of orphans. 
It looks like TSM DB backup is never going to go back and look for those.
Anybody know what causes this?

Remco, thanks VERY much for your post, that's the most useful info I've seen so 
far.

I don't see anything in db2diagx.log that I can relate to failing to locate or 
delete archive logs - OTOH I really don't know how to interpret what I see in 
there.  We opened a PMR and sent the db2diag.log to IBM support and AFAIK they 
didn't read it at all.
 
But what we've done for now is move the old chunks out, so that the archivelog 
directory is clean up until the time of the last DB backup.  Will let it run a 
few days and see if it happens again.  Then at least Ill have an accurate 
point of reference to try and match up the db2diag and TSM actlog.

 Wanda 



-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Remco 
Post
Sent: Tuesday, August 26, 2014 6:19 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] tsm archive log full, causes repeated backups

check the following:

- db2 recovery history retention. IIRC it should be set to 0 for TSM. In one 
environment I used to manage a db2 DBA once decided to sanitize all db2 
installations to a recovery history retention of 30 days... major pains. You'll 
have to do the google foo yourself, I don't recall if it's part of db2 database 
or db2 database manager parameters.
- if at all possible grow J:
- if growing J is hard: try harder
- if growing J: is absolutely impossible it is not very dangerous to move the 
oldest db2 log files to some other place, they are indeed archived log files 
and are no longer needed for your operation. They might be needed for recovery 
until you've made sufficient full database backups. Check the db2 event log for 
any messages regarding required log files not being found. In principle a full 
backup should contain a full copy of the database and all active log files that 
have been touched during the making of the backup.
- if in doubt: call IBM

Op 26 aug. 2014, om 18:37 heeft Tim Brown tbr...@cenhud.com het volgende 
geschreven:

 Archive logs are in j:\tsm\archlog
 
 Drive J is 250Gb total, free space just 386mb
 
 Tim
 
 -Original Message-
 From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of 
 Rhodes, Richard L.
 Sent: Tuesday, 26 August, 2014 12:26 PM
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: [ADSM-L] tsm archive log full, causes repeated backups
 
 Cound something wrong with your archive log filesystem?  Might it be 
 unmounted or something?
 
 Rick
 
 
 
 -Original Message-
 From: ADSM: Dist Stor Manager 
 [https://urldefense.proofpoint.com/v1/url?u=http://mailto:ADSM-L%40VM.MARIST.EDUk=pCpgOv%2FKLW5dYRss05kLEw%3D%3D%0Ar=tQq0J85k4w4CeO1cI0sWanGhPyb3Fq7EC6ctgNHLxNs%3D%0Am=wLQwo6jEM4gBohY6PaUggUKrw4TZKeuDMgOs7FtRQdQ%3D%0As=37a9c6f9d2e7c472c894e68ee88ab04eb74a4c87016a83326c058eb0e36cd30d]
  On Behalf Of Tim Brown
 Sent: Tuesday, August 26, 2014 12:13 PM
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: tsm archive log full, causes repeated backups
 
 There have actually been 5 backups completed  in a row since server was 
 started all with success! ?
 
 All started due to ANR0299I A full database backup will be started.  The 
 archive log space used is 99 % and the archive log space used threshold is  
 80 percent.
 
 
 Tim
 
 -Original Message-
 From: ADSM: Dist Stor Manager 
 [https://urldefense.proofpoint.com/v1/url?u=http://mailto:ADSM-L%40VM.MARIST.EDUk=pCpgOv%2FKLW5dYRss05kLEw%3D%3D%0Ar=tQq0J85k4w4CeO1cI0sWanGhPyb3Fq7EC6ctgNHLxNs%3D%0Am=wLQwo6jEM4gBohY6PaUggUKrw4TZKeuDMgOs7FtRQdQ%3D%0As=37a9c6f9d2e7c472c894e68ee88ab04eb74a4c87016a83326c058eb0e36cd30d]
  On Behalf Of Prather, Wanda
 Sent: Tuesday, 26 August, 2014 12:05 PM
 To: ADSM-L@VM.MARIST.EDU
 Subject: Re: [ADSM-L] tsm archive log full, causes repeated backups
 
 That usually means your activity log is enormous!  :) But restarting the 
 server would have killed any reorg or pinning transaction.
 
 Try 2 db backups in a row now, your archive log should clear.
 
 W
 
 -Original Message-
 From: ADSM: Dist 

Re: Orphaned archive logs (was: tsm archive log full, causes repeated backups)

2014-08-30 Thread Remco Post
Op 30 aug. 2014, om 18:35 heeft Prather, Wanda wanda.prat...@icfi.com het 
volgende geschreven:

 
 I don't see anything in db2diagx.log that I can relate to failing to locate 
 or delete archive logs - OTOH I really don't know how to interpret what I see 
 in there.  We opened a PMR and sent the db2diag.log to IBM support and AFAIK 
 they didn't read it at all.


I would not expect any either… the logs are completely irrelevant to the 
operation, and from what I read they may at best be related to an ancient 
database backup. If they were in any way relevant, then, and only then, db2 
would complain about their absence.


-- 

 Met vriendelijke groeten/Kind Regards,

Remco Post
r.p...@plcs.nl
+31 6 248 21 622