We're big. (2tb/day backup data Mon-Fri nights) Expiration is the 20-ton
elephant in the room. If it doesn't run to completion every once in a
while, we're in trouble and we have the dreaded database bloat. It has
happened, and it wasn't pretty.
You can call this schedule crazy, but it's what works for a big TSM system:
CLIENT BACKUP + EXPIRATION + RECLAMATION (5PM)
BACKUP STGPOOLS + EXPIRATION
MIGRATION + EXPIRATION
BACKUP DEVCONFIG + BACKUP VOLHIST ***
BACKUP DB (incr on weekdays, full on Sun) + RECLAMATION
DELETE VOLHIST (triggered by end of BACKUP DB) + RECLAMATION
RECLAMATION + EXPIRATION (1-5PM - a relatively slow time. This is my
maintenance window.)
back to top
*** These are fast. Runs while the tape lib is dismounting the migration
tapes, and mounting the DB BACKUP tape.
Saturday morning only: DELETE FILESPACE (we queue them up all week)
You've got to work DELETE FILESPACE into your schedule, because it
involves very heavy database I/O. You can't EXPIRE INVENTORY or BACKUP
DB during DELETE FILESPACE. MIGRATION won't even start if DELETE
FILESPACE is running. We do allow users to do it themselves, but in
actual practice they never do. We've got to nag them about ancient,
abandoned filespaces, and then we do it for them Sat AM. This is also
when we remove old nodes.
We use a tape reuse delay of 2 days, as disaster protection for doing
things in the "wrong" order.
The idea here is to keep the CPU and the 15,000rpm database disks busy
24/7, and to use as many tape drives as possible for as much of the time
as possible. I am constantly tuning this schedule. The basic problem is
that there are only 24 hours in a day.
Roger Deschner University of Illinois at Chicago rog...@uic.edu
Academic Computing & Communications Center
======I have not lost my mind -- it is backed up on tape somewhere.=====
On Fri, 28 Aug 2009, Howard Coles wrote:
>Correction, it's "marked for expiration" and it is still recoverable,
>until the "Expire" process runs and removes it from the Database. I
>know this from experience, as we disable our expiration process for a
>few days due to a server failure, and once due to a legal request. The
>Expire Process actually removes the DB entry for that version of the
>file.
>
>See Ya'
>Howard
>
>
>> -----Original Message-----
>> From: ADSM: Dist Stor Manager [mailto:ads...@vm.marist.edu] On Behalf
>> Of Wanda Prather
>> Sent: Friday, August 28, 2009 11:21 AM
>> To: ADSM-L@VM.MARIST.EDU
>> Subject: Re: [ADSM-L] Daily TSM maintenance schedules
>>
>> Agreed.
>> "expire inventory" is actually something of a misnomer.
>>
>> If you have your retention set to 14 versions, and someone takes the
>> 15th
>> backup, the oldest version expires right then, you can't get it back.
>> That
>> type of "expiration" takes place, no matter whether EXPIRE INVENTORY
>> runs or
>> not.
>>
>> The EXPIRE INVENTORY is what updates the %utilization on your tapes,
>> based
>> on the files that have expired. I think it also does some cleanup to
>> make
>> space in the DB for the expired files reusable.
>>
>> So you want to run EXPIRE INVENTORY before reclaim.
>> But I don't think it affects your migration in any way.
>>
>> W
>>
>> On Fri, Aug 28, 2009 at 11:23 AM, Thomas Denier <
>> thomas.den...@jeffersonhospital.org> wrote:
>>
>> > -----Sergio O. Fuentes wrote: -----
>> >
>> > >I'm revising my TSM administrative schedules and wanted to take an
>> > >informal poll on how many of you lay out your daily TSM maintenance
>> > >routines. Functions I'm talking about here include:
>> > >
>> > >BACKUP DISK STGPOOLS
>> > >BACKUP TAPE STGPOOLS
>> > >BACKUP DEVCONFIG
>> > >BACKUP VOLHIST
>> > >BACKUP DB TYPE=FULL
>> > >PREPARE
>> > >DELETE VOLHIST
>> > >MIGRATE STG
>> > >EXPIRE INV
>> > >RECLAIM TAPE
>> > >RECLAIM OFFSITES
>> > >CLIENT BACKUP WINDOW STARTS (back to top)
>> > >
>> > >The above sequence is roughly how I handle our maintenance and is
>> > >based off of the IBM Redbook (sg247379) TSM Deployment Guide for
>5.5
>> > >(page 300). I'm seriously considering altering it in this manner:
>> > >
>> > >BACKUP STGPOOLS
>> > >BACKUP DEVCONFIG
>> > >BACKUP VOLHIST
>> > >BACKUP DB
>> > >PREPARE
>> > >DELETE VOLHIST
>> > >EXPIRE INV
>> > >RECLAIM
>> > >MIGRATE STG
>> > >CLIENT BACKUP WINDOW STARTS (back to top)
>> > >
>> > >The key difference here, is that I'd be expiring right after the DB
>> > >Backups, and reclaiming space before migration. I feel that this
>> > >would be more efficient in terms of processing actual unexpired
>data
>> > >and data storage (since reclamation would have freed up storage
>> > >space). I would be concerned that migration would run in
>perpetuity
>> > >in cases where the migration window runs into the client backup
>> > >window. Therefore, I might have migrations run before
>reclamations.
>> > >Does anyone else expire data right after your DB backups on a daily
>> > >basis? Suggestions from anyone? Thank you kindly.
>> >
>> > I don't understand what benefit to hope for in terms of 'processing
>> > actual unexpired data'. You seem to be described a system in which
>> > disk storage pools are short-term buffers for incoming backups.
>> > With this usage pattern, disk storage pools will contain few
>> > inactive backup files of any sort, and no backup files that have
>> > been inactive long enough to be candidates for removal by an
>> > expiration process.
>> >
>> > The 'expire-reclaim-migrate' version of the proposed change will
>> > cause reclaimed volumes to revert to scratch or 'Empty' status a
>> > few hours earlier than they would under your current policy. The
>> > 'expire-migrate-reclaim' version of the proposed change would
>> > eliminate even this marginal advantage.
>> >
>
