John,
I heartily agree that everybody wants the fastest restores
possible. But a TSM (or any backup design, for that matter), is always
a series of compromises between minimizing daily processing time and
media, while still achieving "adequate" restore times.
If money were no object, and management would let us purchase
anything we wanted to maximize restore time, then we would purchase
enough disk space to keep everything on disk uncompressed, and never be
forced to tape. Or, we would do full backups of everything every day,
so our restores would always be from the minimum number of tapes. All
the clients should be Lan-free where possible, with their own dedicated
tape drives. You get the idea.
The harsh reality is that at our site we have to backup 6 TB per
day with the disk and tape we have. We only do one or fewer restores a
week, and then it is usually between 10 and 200 GB when we do. Using
disk for the last 24-48 hours worth of data, and tape with collocation
groups, and client compression, we can successfully do it all. Do the
restores take longer than without client compression? Possibly. But
the performance of the network and disk/tape has some bearing on restore
speed, too, so it may not be enough to outweigh the benefits.
Best Regards,
John D. Schneider
Sr. System Administrator - Storage
Sisters of Mercy Health System
3637 South Geyer Road
St. Louis, MO. 63127
Email: [EMAIL PROTECTED]
Office: 314-364-3150, Cell: 314-486-2359
-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of
John Monahan
Sent: Tuesday, May 01, 2007 7:51 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] Ramifications of turning client compression on?
My first thought after reading this goes back to what the main goal
should be in any TSM environment: How are your restores affected? Will
you still be able to meet your SLAs for restore times (especially a full
restore)? That is an area that is often overlooked until it is too
late. IMO, too much engineering time in many TSM environments is spent
around backups, when a bulk of the time should be spent around
engineering the fastest and most reliable restores possible. Just
something to remember when considering the benefits/drawbacks of client
compression.
______________________________
John Monahan
Consultant
Logicalis, Inc.
5500 Wayzata Blvd Suite 315
Golden Valley, MN 55416
Office: 763-417-0552 x109
Mobile: 952-221-6938
Fax: 763-417-0554
[EMAIL PROTECTED]
http://www.us.logicalis.com
"Schneider, John" <[EMAIL PROTECTED]>
Sent by: "ADSM: Dist Stor Manager" <ADSM-L@VM.MARIST.EDU> 05/01/2007
05:18 PM Please respond to
"ADSM: Dist Stor Manager" <ADSM-L@VM.MARIST.EDU>
To
ADSM-L@VM.MARIST.EDU
cc
Subject
Re: Ramifications of turning client compression on?
I am going to weigh in on this one. Client compression can work great,
depending on the kind of problems you are trying to solve, and if you do
it right. Here was our situation:
I recently took on responsibility for a TSM environment with 1 shared
library with 14 tape drives, 6 TSM AIX servers, and about 760 clients
backed up per night.
When I started working here, we were at the very edge of keeping up.
Disk storage pools were filling up in the middle of the night so much
that migration was running for hours in the middle of backups, and
backup storage pools weren't completing in the timeframe we had during
the day. So we were forced to cancel backup storage pools on some days
just to get the offsite tapes off before 5PM. Then migration would run
most of the time until the night's backups began. We only had a couple
hours per day to run reclamation, because migration would need to kick
off and use the tape drives. Things were a mess.
I gradually rolled in client compression. Today we are backing up over
100 clients more than before, and about 1.5 TB per day more data.
Migration never happens in the middle of the night (except for an
extraordinary circumstance). Backup stgpools complete every day, and
are finished by about noon. Migration runs until early evening, and we
have 6-7 hours to run reclamation.
While part of the improvement came from scheduling changes to keep all
the tape drives busier, I credit client compression with a big part of
it. The TSM servers are only having to absorb and write to disk about
half as much data. So the network load is cut in half, the data to
migrate is cut in half, backup stgpools are cut in half, and so on.
As you know, a TSM server has to read and write the same data lots of
times during the life of the data. Migration, backup stgpool, and who
knows how many iterations of reclamation before the data finally expires
will mean that the data has to be copied from disk or tape multiple
times. When you are using compression on the tape drive, each time you
have to copy the data, it is uncompressed, copied in it's original form
into TSM's memory, and back out trough the FC switches, where it is
recompressed back onto tape. But during the time it is going through
the SAN, and TSM's memory and disk, it is in uncompressed form, and that
means more load and slower performance.
With client compression, the data is compressed once, and stays in it's
most efficient size for the entire time TSM has to handle it, no matter
how many times TSM has to copy it. I think that constitutes a
significant improvement in efficiency.
But everybody asks, "Don't the clients take a lot longer to back up?"
Each individual client takes longer to back up, but you can compensate
for that by starting more backups earlier in the evening. Since the
clients are only going to send about half as much data, you can have
twice as many simultaneous clients running without hurting anything. We
found that most incremental backups ran about 50% longer. That sounds
bad, but most of our client backups ran in 1-2 hours. So instead of
starting them at 2am for instance, we start them at midnight instead,
and they finish at the same time as before. What is the harm in that?
Even the clients that take 4 hours might take only 5-6, but as long as
we can start them early enough to all get done within the window, that
is no problem.
The only exceptions to this rule are very large clients that might take
10-12 hours already, or clients that are significantly CPU constrained,
and don't have any CPU to throw at doing the compression. But those
turned out to be a tiny fraction of the clients in our environment. Even
our older and slower Windows servers were able to use client compression
with no impact to the production application.
Best Regards,
John D. Schneider
Sr. System Administrator - Storage
Sisters of Mercy Health System
3637 South Geyer Road
St. Louis, MO. 63127
Email: [EMAIL PROTECTED]
Office: 314-364-3150, Cell: 314-486-2359
-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of
Kelly Lipp
Sent: Tuesday, May 01, 2007 2:10 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] Ramifications of turning client compression on?
The first question I would ask is why are you doing this? If you hope
to reduce the amount of TIME it takes to do a backup, you won't. If you
hope to reduce the amount of DISK SPACE in your disk pools, you will,
but at what cost.
In almost all cases (I don't say all because words like all or never
annoy me) compression is best done in hardware rather than software. So
let your tape technology do the compression.
That said, in an all disk based TSM environment it may well make sense
to let clients do compression to reduce the amount of disk required.
Personally, I'm hoping for some clever SAN vendor to put compression
into their controllers so the hardware will do it for me.
Thanks,
Kelly J. Lipp
VP Manufacturing & CTO
STORServer, Inc.
485-B Elkton Drive
Colorado Springs, CO 80907
719-266-8777
[EMAIL PROTECTED]
-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of
Joni Moyer
Sent: Tuesday, May 01, 2007 12:44 PM
To: ADSM-L@VM.MARIST.EDU
Subject: [ADSM-L] Ramifications of turning client compression on?
Hello All!
I am considering turning client compression on for our Lotus Notes
Domino and Oracle servers. What should be looked at/considered before
client compression is turned on? How do you monitor the impact of
turning client compression on? Any ideas/suggestions are appreciated!
Thanks!
********************************
Joni Moyer
Highmark
Storage Systems, Storage Mngt Analyst III Phone Number: (717)302-9966
Fax: (717) 302-9826
[EMAIL PROTECTED]
********************************
