Re: RES: Very slow DB backups

2005-07-05 Thread Troy Frank
I would be interested in that, if you'd care to keep posting updates.


Troy Frank
Network Services
University of Wisconsin Medical Foundation
608.829.5384

>>> [EMAIL PROTECTED] 7/4/2005 4:03:41 PM >>>
Hello All, 
 
   Just to keep you updated on my quest :)
 
   I began to modify some settings, and have already reduced the number of log 
volumes from 10 to 2, and stgpool volumes from 108 to 6. Backups are a little 
bit faster now, but I expect the gain to be bigger when I re-create the DB 
volumes.
   I am very concerned about the expiration process. Analyzing the actlog, I 
found out that the expiration process (two daily runs with DURATION=240 each) 
is taking nearly a MONTH to process the same filespace again. I am not joking :(
   I also found out what might be the cause of the database size and expiration 
performance (in addition to the volume allocation). Just look at the output of 
a "select type,sum(num_files) as Files from occupancy group by type" command:
 
TYPE FILES
-- ---
Arch 384354353
Bkup  35759844

   It seems that someone likes archiving. I am now in the process of replacing 
the majority of these with backup operations.
 
   If there is interest, I can (maybe in private emails) post the results in 
terms of changed made vs. performance improvement.
 
Best regards, 
 
Paul

-Mensagem original- 
De: Richard Sims [mailto:[EMAIL PROTECTED] 
Enviada: sex 7/1/05 8:37 
Para: ADSM-L@VM.MARIST.EDU 
    Cc: 
    Assunto: Re: Very slow DB backups



Hi, Paul -

You have your work cut out for you with that train-wreck of a system.
I'd begin by questioning the need for all that DB content... There
may be abandoned filespaces which should go, obsolete retention
periods which no one has looked at in years, and perhaps even
Expirations which never run to completion.
As to performance, I'd begin by inspecting SCSI topology, to assure
no dumb stuff, like tape and disk on the same cable. Next I'd review
the LTO microcode levels, as older levels had serious problems (see
"LTO performance" in ADSM QuickFacts). The "sleep" may be the drive
struggling to load and position the tape. You might consider a tape
case study using the tapeutil command, or the like, to get some
baseline numbers on the performance of those drives and tapes for
comparison as improvements are pursued.

The disk layout certainly needs a lot of review, as it doesn't seem
to make sense, even if one is not familiar with EMC boxes.

Only as very last resort would I even consider db unload/reload:
that's ill-advised even in the best of circumstances, as proven
dangerous and often unproductive.

Richard Sims



Confidentiality Notice follows:

The information in this message (and the documents attached to it, if any)
is confidential and may be legally privileged. It is intended solely for
the addressee. Access to this message by anyone else is unauthorized. If
you are not the intended recipient, any disclosure, copying, distribution
or any action taken, or omitted to be taken in reliance on it is
prohibited and may be unlawful. If you have received this message in
error, please delete all electronic copies of this message (and the
documents attached to it, if any), destroy any hard copies you may have
created and notify me immediately by replying to this email. Thank you.


RES: Very slow DB backups

2005-07-04 Thread Paul van Dongen
Hello All, 
 
   Just to keep you updated on my quest :)
 
   I began to modify some settings, and have already reduced the number of log 
volumes from 10 to 2, and stgpool volumes from 108 to 6. Backups are a little 
bit faster now, but I expect the gain to be bigger when I re-create the DB 
volumes.
   I am very concerned about the expiration process. Analyzing the actlog, I 
found out that the expiration process (two daily runs with DURATION=240 each) 
is taking nearly a MONTH to process the same filespace again. I am not joking :(
   I also found out what might be the cause of the database size and expiration 
performance (in addition to the volume allocation). Just look at the output of 
a "select type,sum(num_files) as Files from occupancy group by type" command:
 
TYPE FILES
-- ---
Arch 384354353
Bkup  35759844

   It seems that someone likes archiving. I am now in the process of replacing 
the majority of these with backup operations.
 
   If there is interest, I can (maybe in private emails) post the results in 
terms of changed made vs. performance improvement.
 
Best regards, 
 
Paul

-Mensagem original- 
De: Richard Sims [mailto:[EMAIL PROTECTED] 
Enviada: sex 7/1/05 8:37 
Para: ADSM-L@VM.MARIST.EDU 
Cc: 
Assunto: Re: Very slow DB backups



Hi, Paul -

You have your work cut out for you with that train-wreck of a system.
I'd begin by questioning the need for all that DB content... There
may be abandoned filespaces which should go, obsolete retention
periods which no one has looked at in years, and perhaps even
Expirations which never run to completion.
As to performance, I'd begin by inspecting SCSI topology, to assure
no dumb stuff, like tape and disk on the same cable. Next I'd review
the LTO microcode levels, as older levels had serious problems (see
"LTO performance" in ADSM QuickFacts). The "sleep" may be the drive
struggling to load and position the tape. You might consider a tape
case study using the tapeutil command, or the like, to get some
baseline numbers on the performance of those drives and tapes for
comparison as improvements are pursued.

The disk layout certainly needs a lot of review, as it doesn't seem
to make sense, even if one is not familiar with EMC boxes.

Only as very last resort would I even consider db unload/reload:
that's ill-advised even in the best of circumstances, as proven
dangerous and often unproductive.

Richard Sims




Re: Very slow DB backups

2005-07-01 Thread Allen S. Rout
==> On Fri, 1 Jul 2005 07:37:44 -0300, Paul van Dongen <[EMAIL PROTECTED]> said:

> I was called to examine a TSM server in order to make some suggestions to 
> improve performance. Upon arrival, I found out a not-so-standard 
> configuration:

> TSM 5.1.6.4 on HP-UX 11
> DB: 208 GB split on 208(!) volumes of 1GB each spread on 4 LUNs on an EMC box 
> (95% in use)
> Log: 10 volumes of 1GB each, all on the same EMC LUN
> Diskpool: 101 volumes of 1GB each, all on the same EMC LUN

[...]

> I am trying to talk to the admin of this server to upgrade to 5.2 or 5.3 and
> to review their disk config, but would like to hear from you if someone has
> been trough something alike, and of course the line of action that was taken.


I'd bet that your client admin went all the way through the "multiple TSM
volumes for IO parallelism" and out the other side.

My suggestion would be to make the DB volumes something like 10 times as big.

The log is written serially, don't worry so much about lots of cylynders for
it.

But I don't know that 4 hours is all that evil a time for a 200-some GB DB
backup.  200 GB in 4 hours is something like 14MB/s;  That might be "perfectly
tuned LTO 1"... Or it might be "slightly damaged 3590" or it might be "3592
with a millstone around its' neck"...


One way I moved my DB backup bottleneck was to start backing up databases to a
remote TSM server.  This let me toss data straight to disk, and also add
under-the-covers copies of the DB backups.   If you were to do something like
that, you might find that the bottleneck is really in your tape architecture.

The nice thing is, you can set up some other TSM server, define the devclass
on your ailing box, and run a snapshot without even fiddling with your current
production DB backup streams.


- Allen S. Rout


Re: Very slow DB backups

2005-07-01 Thread Ben Bullock
Good gosh, what is it with the "unload/reload" craze on here lately.

It's like...
Patient: "ya doctor, I have had a shortness of breath lately."
Doctor: "We better do open heart surgery!!!"

I have a large TSM environment, with 8 TSM servers, most of them
in the 75-80GB database range. Most of them have been in production for
5 years or more, some of them are going on 8 years and I have NEVER had
to do a unload/reload on a production TSM server. 

I did do one in our test environment and was impressed with the
improvements in speed and size of DB, but found that it was short lived
and within 60 days we were back to the same backup-time ranges as
before. 

With the 9 hours of downtime that was associated with an
unload/reload, it's not an option in our 24X7 production environment for
such a short-lived benefit.

Paul,
Sounds like perhaps a bottleneck at the log device. You might
have them check the EMC SAN to make sure that the write cache is turned
on. I would also check out a 'q logvol f=d" and make sure that the "Log
Pool Pct. Wait" value is 0 (or almost 0). If it is higher, it points to
log volume performance issue.

Ben
 

-Original Message-
From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of
Rees, Chris (Corp)
Sent: Friday, July 01, 2005 4:44 AM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: Very slow DB backups

Hi Paul

A strange volume config indeed

One thing I'd ask is when was the last time this server was
unloaded/reloaded ?

Chris

-Original Message-
From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of
Paul van Dongen
Sent: Friday, July 01, 2005 11:38 AM
To: ADSM-L@VM.MARIST.EDU
Subject: [ADSM-L] Very slow DB backups

Hello All,
 
I was called to examine a TSM server in order to make some
suggestions to improve performance. Upon arrival, I found out a
not-so-standard configuration:
 
TSM 5.1.6.4 on HP-UX 11
DB: 208 GB split on 208(!) volumes of 1GB each spread on 4 LUNs on an
EMC box (95% in use)
Log: 10 volumes of 1GB each, all on the same EMC LUN
Diskpool: 101 volumes of 1GB each, all on the same EMC LUN
 
I know that splitting this server should be the best solution, and there
are other factors contributing to this big DB size, but I will stay away
from this for  while.
 
The first I noticed is that there are two full DB backups being executed
each day. They go to a LTO device class, and are taking some 3 to 4
hours to complete, including a stange 5 to 10 minute "sleep" between the
backup command being issued and the actual start of the process. I tried
to minimize this by setting the log to rollforward and trying to take a
incremental backup. To my surprise, the triggered incremental (log
became 70% full) started, with the same delay described above, but began
to crawl, at 8000 pages per MINUTE. It would give me almost 4 hours to
copy the 7 or so GB!! Naturally, I went back to the fulls until I found
out what could be the problem.
 
I am trying to talk to the admin of this server to upgrade to 5.2 or 5.3
and to review their disk config, but would like to hear from you if
someone has been trough something alike, and of course the line of
action that was taken.
 
Thanks to all,
 
Paul



___ Disclaimer Notice  This
message and any attachments are confidential and should only be read by
those to whom they are addressed. If you are not the intended recipient,
please contact us, delete the message from your computer and destroy any
copies. Any distribution or copying without our prior permission is
prohibited.

Internet communications are not always secure and therefore the E.ON
Group does not accept legal responsibility for this message. The
recipient is responsible for verifying its authenticity before acting on
the contents. Any views or opinions presented are solely those of the
author and do not necessarily represent those of the E.ON Group. 
 
E.ON UK plc, Westwood Way, Westwood Business Park, Coventry, CV4 8LG.
Registered in England & Wales No. 2366970

E.ON UK Trading Ltd, Westwood Way, Westwood Business Park, Coventry, CV4
8LG Registered in England & Wales No. 4178314

E.ON UK Trading Ltd is regulated by the Financial Services Authority to
carry out investment activities. 

Telephone +44 (0) 2476 42 4000
Fax +44 (0) 2476 42 5432


Re: Very slow DB backups

2005-07-01 Thread John E. Vincent

I could see a situation where the disk array is legacy and no money has
been spent to upgrade it. It sound curiously like someone making due
with what they had and trying to distribute the I/O in any way possible.

Richard Sims wrote:

Hi, Paul -

You have your work cut out for you with that train-wreck of a system.
I'd begin by questioning the need for all that DB content... There
may be abandoned filespaces which should go, obsolete retention
periods which no one has looked at in years, and perhaps even
Expirations which never run to completion.
As to performance, I'd begin by inspecting SCSI topology, to assure
no dumb stuff, like tape and disk on the same cable. Next I'd review
the LTO microcode levels, as older levels had serious problems (see
"LTO performance" in ADSM QuickFacts). The "sleep" may be the drive
struggling to load and position the tape. You might consider a tape
case study using the tapeutil command, or the like, to get some
baseline numbers on the performance of those drives and tapes for
comparison as improvements are pursued.

The disk layout certainly needs a lot of review, as it doesn't seem
to make sense, even if one is not familiar with EMC boxes.

Only as very last resort would I even consider db unload/reload:
that's ill-advised even in the best of circumstances, as proven
dangerous and often unproductive.

   Richard Sims


Re: Very slow DB backups

2005-07-01 Thread Richard Rhodes
We are currently on tsm v5.1 also, and I have seen the "sleep" that you
describe, although not nearly as bad.  Up to a couple months ago, the db
backup for one of our TSM server was taking 4-5 hours (140gb).  From when
the start db backup command was issues until the process showed up took
about 5 min.  Part of this was waiting for a virtual volume mount, but not
all.  I never figured out want TSM was doing, but It looked like it was
performing highly random access around the db preparing for the backup.
Since we moved the db and log to a new disk system and heavily striped it,
we've cut our backup time to 1hr.  There is still a wait for the virtual
volume mount, but nothing like it was before.

Along with what others have recommended, I'd suggest talking with the
storage admins about the layout on the backend of the Symm.  They need to
look at the layout of the 4 TSM luns on the backend, what else is on those
spindles, load, striping, striped or concatinated meta's.  To me it sounds
like a disk system severely short on spindles and striping.

Rick





  Paul van Dongen
  <[EMAIL PROTECTED]To:   ADSM-L@VM.MARIST.EDU
  .COM.BR> cc:
  Sent by: "ADSM:  Subject:  Very slow DB backups
  Dist Stor
  Manager"
  <[EMAIL PROTECTED]
  .EDU>


  07/01/2005 07:37
  AM
  Please respond to
  "ADSM: Dist Stor
  Manager"






Hello All,

I was called to examine a TSM server in order to make some suggestions
to improve performance. Upon arrival, I found out a not-so-standard
configuration:

TSM 5.1.6.4 on HP-UX 11
DB: 208 GB split on 208(!) volumes of 1GB each spread on 4 LUNs on an EMC
box (95% in use)
Log: 10 volumes of 1GB each, all on the same EMC LUN
Diskpool: 101 volumes of 1GB each, all on the same EMC LUN

I know that splitting this server should be the best solution, and there
are other factors contributing to this big DB size, but I will stay away
from this for  while.

The first I noticed is that there are two full DB backups being executed
each day. They go to a LTO device class, and are taking some 3 to 4 hours
to complete, including a stange 5 to 10 minute "sleep" between the backup
command being issued and the actual start of the process. I tried to
minimize this by setting the log to rollforward and trying to take a
incremental backup. To my surprise, the triggered incremental (log became
70% full) started, with the same delay described above, but began to crawl,
at 8000 pages per MINUTE. It would give me almost 4 hours to copy the 7 or
so GB!! Naturally, I went back to the fulls until I found out what could be
the problem.

I am trying to talk to the admin of this server to upgrade to 5.2 or 5.3
and to review their disk config, but would like to hear from you if someone
has been trough something alike, and of course the line of action that was
taken.

Thanks to all,

Paul




-
The information contained in this message is intended only for the personal
and confidential use of the recipient(s) named above. If the reader of this
message is not the intended recipient or an agent responsible for
delivering it to the intended recipient, you are hereby notified that you
have received this document in error and that any review, dissemination,
distribution, or copying of this message is strictly prohibited. If you
have received this communication in error, please notify us immediately,
and delete the original message.


Re: Very slow DB backups

2005-07-01 Thread Richard Sims

Hi, Paul -

You have your work cut out for you with that train-wreck of a system.
I'd begin by questioning the need for all that DB content... There
may be abandoned filespaces which should go, obsolete retention
periods which no one has looked at in years, and perhaps even
Expirations which never run to completion.
As to performance, I'd begin by inspecting SCSI topology, to assure
no dumb stuff, like tape and disk on the same cable. Next I'd review
the LTO microcode levels, as older levels had serious problems (see
"LTO performance" in ADSM QuickFacts). The "sleep" may be the drive
struggling to load and position the tape. You might consider a tape
case study using the tapeutil command, or the like, to get some
baseline numbers on the performance of those drives and tapes for
comparison as improvements are pursued.

The disk layout certainly needs a lot of review, as it doesn't seem
to make sense, even if one is not familiar with EMC boxes.

Only as very last resort would I even consider db unload/reload:
that's ill-advised even in the best of circumstances, as proven
dangerous and often unproductive.

   Richard Sims


Re: Very slow DB backups

2005-07-01 Thread Rees, Chris (Corp)
Hi Paul

A strange volume config indeed

One thing I'd ask is when was the last time this server was
unloaded/reloaded ?

Chris

-Original Message-
From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of
Paul van Dongen
Sent: Friday, July 01, 2005 11:38 AM
To: ADSM-L@VM.MARIST.EDU
Subject: [ADSM-L] Very slow DB backups

Hello All,
 
I was called to examine a TSM server in order to make some
suggestions to improve performance. Upon arrival, I found out a
not-so-standard configuration:
 
TSM 5.1.6.4 on HP-UX 11
DB: 208 GB split on 208(!) volumes of 1GB each spread on 4 LUNs on an
EMC box (95% in use)
Log: 10 volumes of 1GB each, all on the same EMC LUN
Diskpool: 101 volumes of 1GB each, all on the same EMC LUN
 
I know that splitting this server should be the best solution, and there
are other factors contributing to this big DB size, but I will stay away
from this for  while.
 
The first I noticed is that there are two full DB backups being executed
each day. They go to a LTO device class, and are taking some 3 to 4
hours to complete, including a stange 5 to 10 minute "sleep" between the
backup command being issued and the actual start of the process. I tried
to minimize this by setting the log to rollforward and trying to take a
incremental backup. To my surprise, the triggered incremental (log
became 70% full) started, with the same delay described above, but began
to crawl, at 8000 pages per MINUTE. It would give me almost 4 hours to
copy the 7 or so GB!! Naturally, I went back to the fulls until I found
out what could be the problem.
 
I am trying to talk to the admin of this server to upgrade to 5.2 or 5.3
and to review their disk config, but would like to hear from you if
someone has been trough something alike, and of course the line of
action that was taken.
 
Thanks to all,
 
Paul



___ Disclaimer Notice 
This message and any attachments are confidential and should only be read by 
those to whom they are addressed. If you are not the intended recipient, please 
contact us, delete the message from your computer and destroy any copies. Any 
distribution or copying without our prior permission is prohibited.

Internet communications are not always secure and therefore the E.ON Group does 
not accept legal responsibility for this message. The recipient is responsible 
for verifying its authenticity before acting on the contents. Any views or 
opinions presented are solely those of the author and do not necessarily 
represent those of the E.ON Group. 
 
E.ON UK plc, Westwood Way, Westwood Business Park, Coventry, CV4 8LG.
Registered in England & Wales No. 2366970

E.ON UK Trading Ltd, Westwood Way, Westwood Business Park, Coventry, CV4 8LG
Registered in England & Wales No. 4178314

E.ON UK Trading Ltd is regulated by the Financial Services Authority to carry 
out investment activities. 

Telephone +44 (0) 2476 42 4000
Fax +44 (0) 2476 42 5432


Very slow DB backups

2005-07-01 Thread Paul van Dongen
Hello All,
 
I was called to examine a TSM server in order to make some suggestions to 
improve performance. Upon arrival, I found out a not-so-standard configuration:
 
TSM 5.1.6.4 on HP-UX 11
DB: 208 GB split on 208(!) volumes of 1GB each spread on 4 LUNs on an EMC box 
(95% in use)
Log: 10 volumes of 1GB each, all on the same EMC LUN
Diskpool: 101 volumes of 1GB each, all on the same EMC LUN
 
I know that splitting this server should be the best solution, and there are 
other factors contributing to this big DB size, but I will stay away from this 
for  while.
 
The first I noticed is that there are two full DB backups being executed each 
day. They go to a LTO device class, and are taking some 3 to 4 hours to 
complete, including a stange 5 to 10 minute "sleep" between the backup command 
being issued and the actual start of the process. I tried to minimize this by 
setting the log to rollforward and trying to take a incremental backup. To my 
surprise, the triggered incremental (log became 70% full) started, with the 
same delay described above, but began to crawl, at 8000 pages per MINUTE. It 
would give me almost 4 hours to copy the 7 or so GB!! Naturally, I went back to 
the fulls until I found out what could be the problem.
 
I am trying to talk to the admin of this server to upgrade to 5.2 or 5.3 and to 
review their disk config, but would like to hear from you if someone has been 
trough something alike, and of course the line of action that was taken.
 
Thanks to all,
 
Paul


Re: Slow DB backups.

2002-10-21 Thread Bill Boyer
I thought I saw a thread not too long ago about a problem with DBBackups
running longer if you had DBCOPY volumes...? I was trying to search for the
thread, but didn't see anything... Should looked at Tivoli knowledge base
first...here's the APAR IC34146 (amongst others):
APAR= IC34146  SER=PF PERF
TSM SERVER DATABASE BACKUP MAY PERFORM SLOW IF DATABASE MIRRORS
ARE DEFINED

Status: CLOSED  Closed: 07/15/02

Apar Information:

RCOMP= 5698ISMSVTSM SERVER 510  RREL= R51A
FCOMP= 5698ISMSVTSM SERVER 510  PFREL= F999  TREL= T
SRLS:  NONE

Return Codes:

Applicable Component Level/SU:
R51A PSY UP
R51H PSY UP
R51S PSY UP
R51W PSY UP

Error Description:
While in the process of working through some database recovery
procedures the customer noticed that their TSM database backups
ran faster than they had seen them run in the past. At the end
of their recovery procedure they added their db mirrors back to
their server and noticed the TSM database backups slow to the
run times they were seeing before. Analysis of this scenario

by development found that the alternating reading of db volumes
during a TSM server database backup when mirrored volumes were
in place was keeping read ahead function from being exploited
as it was without mirrored volumes in place. An alternate db
read design needs to be developed to avoid this processing delay
in applicable environments and ensure TSM server db backup
processing performs the same whether mirrored volumes exist or
not.

Local Fix:


Problem Summary:

* USERS AFFECTED: All users with mirrored log or data base *
* volumes. *

* PROBLEM DESCRIPTION: Backup data base performance is poor.   *

* RECOMMENDATION:  *

The algorithm TSM Server used to alternate between log and data

base mirrors was not optimized for sequential access.

Temporary Fix:


Comments:
MODULES/MACROS:   LVMREAD  ICBACK

Problem Conclusion:
During backup of the database, the algorithm TSM server uses
to alternate between log and database mirrors is optimized for
sequential access.

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@;VM.MARIST.EDU]On Behalf Of
Alex Paschal
Sent: Thursday, October 17, 2002 5:14 PM
To: [EMAIL PROTECTED]
Subject: Re: Slow DB backups.


In addition to your dbbackup rate, have you taken a look at your vmstat for
your memory and CPU params?  Initially, you can look for paging, CPU%, and
CPU WaitIO%.  Are any of those pegged?  That could also get you started
zeroing in on the problem.

Alex Paschal
Storage Administrator
Freightliner, LLC
(503) 745-6850 phone/vmail

-Original Message-
From: Seay, Paul [mailto:seay_pd@;NAPTHEON.COM]
Sent: Wednesday, October 16, 2002 11:08 PM
To: [EMAIL PROTECTED]
Subject: Re: Slow DB backups.


Did you recently have a database expansion?
Are you using RAW volumes or a JFS for the database?
 The layout of the above could really make a difference.  You can all of a
sudden get horrible performance.

What you need to do is look at all of the database backup messages and see
if the rate of a full or dbs type dump is consistent throughout, pages per
30 seconds.  If you notice a steep drop off, that is your problem.  Part of
the database is on a configuration that is bad.  We had this situation.

There is also another little scenario that can really bust you if you do not
realize what you are doing.  If you increase the DB buffer pool and it
causes the computational working set on an AIX TSM server to get larger than
the default of 20 percent of memory you will page your butt off.  This is
easy to fix.  On AIX, just use VMTUNE to set the maxperm (-P) and minperm
(-p) parameters like Mark says.  40 and 10 are a good start.  The other
thing you can do that we found really speed up the backups was raising the
maxfree and max page read ahead.  Remember though, all of these parameters
apply to JFS buffers.  If you are using any JFS at all the maxperm and
minperm could be factors.

I saw the same things you did once my TSM server grew up.  With Mark's
suggestions mine purrs like a kitten now.

What is your hardware?

Paul D. Seay, Jr.
Technical Specialist
Naptheon Inc.
757-688-8180


-Original Message-
From: Suad Musovich [mailto:s.musovich@;AUCKLAND.AC.NZ]
Sent: Wednesday, October 16, 2002 2:52 PM
To: [EMAIL PROTECTED]
Subject: Re: Slow DB backups.


Depends if you are running other processes (especially expiration). Also you
havent mentioned your HW configuration.

On Thu, 2002-10-17 at 04:54, Dearman, Richard wrote:
> I haven't done aTSM DB backup in t

Re: Slow DB backups.

2002-10-17 Thread Alex Paschal
In addition to your dbbackup rate, have you taken a look at your vmstat for
your memory and CPU params?  Initially, you can look for paging, CPU%, and
CPU WaitIO%.  Are any of those pegged?  That could also get you started
zeroing in on the problem.

Alex Paschal
Storage Administrator
Freightliner, LLC
(503) 745-6850 phone/vmail

-Original Message-
From: Seay, Paul [mailto:seay_pd@;NAPTHEON.COM]
Sent: Wednesday, October 16, 2002 11:08 PM
To: [EMAIL PROTECTED]
Subject: Re: Slow DB backups.


Did you recently have a database expansion?
Are you using RAW volumes or a JFS for the database?
 The layout of the above could really make a difference.  You can all of a
sudden get horrible performance.

What you need to do is look at all of the database backup messages and see
if the rate of a full or dbs type dump is consistent throughout, pages per
30 seconds.  If you notice a steep drop off, that is your problem.  Part of
the database is on a configuration that is bad.  We had this situation.

There is also another little scenario that can really bust you if you do not
realize what you are doing.  If you increase the DB buffer pool and it
causes the computational working set on an AIX TSM server to get larger than
the default of 20 percent of memory you will page your butt off.  This is
easy to fix.  On AIX, just use VMTUNE to set the maxperm (-P) and minperm
(-p) parameters like Mark says.  40 and 10 are a good start.  The other
thing you can do that we found really speed up the backups was raising the
maxfree and max page read ahead.  Remember though, all of these parameters
apply to JFS buffers.  If you are using any JFS at all the maxperm and
minperm could be factors.

I saw the same things you did once my TSM server grew up.  With Mark's
suggestions mine purrs like a kitten now.

What is your hardware?

Paul D. Seay, Jr.
Technical Specialist
Naptheon Inc.
757-688-8180


-Original Message-
From: Suad Musovich [mailto:s.musovich@;AUCKLAND.AC.NZ]
Sent: Wednesday, October 16, 2002 2:52 PM
To: [EMAIL PROTECTED]
Subject: Re: Slow DB backups.


Depends if you are running other processes (especially expiration). Also you
havent mentioned your HW configuration.

On Thu, 2002-10-17 at 04:54, Dearman, Richard wrote:
> I haven't done aTSM DB backup in the last few days.  So I started one
> today and it is moving very slow I have a 30GB database and it is only
> reading at about 500KB/s.  At this rate it will take hours to backup.
> Before it would only take 45minutes.
>
> Is this normal?
>
> Thanks
> ***EMAIL DISCLAIMER***
> This email and any files transmitted with it may be confidential and
> are intended solely for the use of th individual or entity to whom
> they are addressed. If you are not the intended recipient or the
> individual responsible for delivering the e-mail to the intended
> recipient, any disclosure, copying, distribution or any action taken
> or omitted to be taken in reliance on it, is strictly prohibited.  If
> you have received this e-mail in error, please delete it and notify
> the sender or contact Health Information Management 312.996.3941.



Re: Slow DB backups.

2002-10-16 Thread Seay, Paul

Did you recently have a database expansion?
Are you using RAW volumes or a JFS for the database?
 The layout of the above could really make a difference.  You can all of a
sudden get horrible performance.

What you need to do is look at all of the database backup messages and see
if the rate of a full or dbs type dump is consistent throughout, pages per
30 seconds.  If you notice a steep drop off, that is your problem.  Part of
the database is on a configuration that is bad.  We had this situation.

There is also another little scenario that can really bust you if you do not
realize what you are doing.  If you increase the DB buffer pool and it
causes the computational working set on an AIX TSM server to get larger than
the default of 20 percent of memory you will page your butt off.  This is
easy to fix.  On AIX, just use VMTUNE to set the maxperm (-P) and minperm
(-p) parameters like Mark says.  40 and 10 are a good start.  The other
thing you can do that we found really speed up the backups was raising the
maxfree and max page read ahead.  Remember though, all of these parameters
apply to JFS buffers.  If you are using any JFS at all the maxperm and
minperm could be factors.

I saw the same things you did once my TSM server grew up.  With Mark's
suggestions mine purrs like a kitten now.

What is your hardware?

Paul D. Seay, Jr.
Technical Specialist
Naptheon Inc.
757-688-8180


-Original Message-
From: Suad Musovich [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, October 16, 2002 2:52 PM
To: [EMAIL PROTECTED]
Subject: Re: Slow DB backups.


Depends if you are running other processes (especially expiration). Also you
havent mentioned your HW configuration.

On Thu, 2002-10-17 at 04:54, Dearman, Richard wrote:
> I haven't done aTSM DB backup in the last few days.  So I started one
> today and it is moving very slow I have a 30GB database and it is only
> reading at about 500KB/s.  At this rate it will take hours to backup.
> Before it would only take 45minutes.
>
> Is this normal?
>
> Thanks
> ***EMAIL DISCLAIMER***
> This email and any files transmitted with it may be confidential and
> are intended solely for the use of th individual or entity to whom
> they are addressed. If you are not the intended recipient or the
> individual responsible for delivering the e-mail to the intended
> recipient, any disclosure, copying, distribution or any action taken
> or omitted to be taken in reliance on it, is strictly prohibited.  If
> you have received this e-mail in error, please delete it and notify
> the sender or contact Health Information Management 312.996.3941.



Re: Slow DB backups.

2002-10-16 Thread Suad Musovich

Depends if you are running other processes (especially expiration).
Also you havent mentioned your HW configuration.

On Thu, 2002-10-17 at 04:54, Dearman, Richard wrote:
> I haven't done aTSM DB backup in the last few days.  So I started one today
> and it is moving very slow I have a 30GB database and it is only reading at
> about 500KB/s.  At this rate it will take hours to backup.  Before it would
> only take 45minutes.
>
> Is this normal?
>
> Thanks
> ***EMAIL DISCLAIMER*** This
> email and any files transmitted with it may be confidential and are intended
> solely for the use of th individual or entity to whom they are addressed.
> If you are not the intended recipient or the individual responsible for
> delivering the e-mail to the intended recipient, any disclosure, copying,
> distribution or any action taken or omitted to be taken in reliance on it,
> is strictly prohibited.  If you have received this e-mail in error, please
> delete it and notify the sender or contact Health Information Management
> 312.996.3941.



Slow DB backups.

2002-10-16 Thread Dearman, Richard

I haven't done aTSM DB backup in the last few days.  So I started one today
and it is moving very slow I have a 30GB database and it is only reading at
about 500KB/s.  At this rate it will take hours to backup.  Before it would
only take 45minutes.

Is this normal?

Thanks
***EMAIL DISCLAIMER*** This
email and any files transmitted with it may be confidential and are intended
solely for the use of th individual or entity to whom they are addressed.
If you are not the intended recipient or the individual responsible for
delivering the e-mail to the intended recipient, any disclosure, copying,
distribution or any action taken or omitted to be taken in reliance on it,
is strictly prohibited.  If you have received this e-mail in error, please
delete it and notify the sender or contact Health Information Management
312.996.3941.