Thanks to all who responded. The problem turned out to be a DB Buffer Pool that was just too small, despite the fact that the old system gave "acceptable" performance with half as much.
Since with TSM 4+ the DB Buffer Pool can be updated dynamically, I increased it 128 MB at a step and watched to see what happened. This was with Primary tape reclamation, copypool reclamation, and storage pool backups to copypool underway simultaneously. At 384 MB there was a little relief. At 512 MB the doors blew open. At 640 MB the doors blew off. Once I got to 640 MB I was getting sustained throughput of 40 MB/s with peaks of 48 MB/s. Before making any changes the sustained throughput was often less than 1 MB/s. Also, wear and tear on the disks was reduced significantly. Yesterday, the disk lights almost never went out. Now they blink. My theory, based on about three HOURS experience with this new system, is that the number of disks in the old system masked the effect of the small (128 MB) buffer pool. The new server, with fewer DB disks, needed more buffer to compensate for less load sharing across disks. Doubling it wasn't enough. It needed a five-fold enlargement. Right now, with reclamation, storage pool backups, client sessions, and a DB backup running, I'm getting 99.87% cache hit on the database - and that's with a 27 GB database. Oh. All disks in the system are ordinary SCSI disks with JFS file systems. Only the OS is mirrored; all other mirroring is done in TSM. Thanks again. Tab Trepagnier TSM Administrator Laitram LLC William SO Ng <[EMAIL PROTECTED]> Sent by: "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]> 04/01/2003 12:12 PM Please respond to "ADSM: Dist Stor Manager" To: [EMAIL PROTECTED] cc: Subject: Re: Poor database performance Spreading the I/O among several disks is preferred as it increase the throughput. As RAID 0,1 is prefered to RAID 5 as each write is faster. This of course is more expensive. Thanks & Regards William |---------+------------------------------> | | David Longo | | | <[EMAIL PROTECTED]| | | -FIRST.ORG> | | | Sent by: "ADSM: | | | Dist Stor Manager" | | | <[EMAIL PROTECTED]| | | DU> | | | | | | | | | 01/04/2003 23:25 | | | Please respond to | | | "ADSM: Dist Stor | | | Manager" | | | | |---------+------------------------------> >--------------------------------------------------------------------------------------------------------------------------------------------| | | | To: [EMAIL PROTECTED] | | cc: | | Subject: Re: Poor database performance | | | | | >--------------------------------------------------------------------------------------------------------------------------------------------| You didn't mention what kind of disk (or array) you have? David Longo David B. Longo System Administrator Health First, Inc. 3300 Fiske Blvd. Rockledge, FL 32955-4305 PH 321.434.5536 Pager 321.634.8230 Fax: 321.434.5509 [EMAIL PROTECTED] >>> [EMAIL PROTECTED] 04/01/03 09:51 AM >>> I just migrated our TSM system to a new pSeries 6H1 from our old F50. While I/O throughput is much higher, the database performance is surprisingly poor. I am asking for help in trying to figure out why and what I can do about it. TSM 4.1.5.0 running on AIX 4.3.3 ML 10 6H1 2-way 600 MHz 64-bit 4 GB RAM 27 GB TSM database with 256 MB DB buffer pool Four 4-drive libraries, each double-connected via SCSI (two drives per adapter) This system has been in service for one day. The same backup system had been running on the old F50 since ADSM 2. On our old F50, I had split the database into many 1 GB volumes as an experiment when we had only 512 MB in our TSM server. It didn't seem to hurt performance so I'd left it that way. When I built the new server, I took advantage of info on this forum and created a single large DB volume on each DB disk. The DB is currently laid out this way: SCSI Adapter #1 --> DB disk 1 (one vol) --> DB disk 3 (one vol) --> Log disk 1 (five vols) --> Pool disk 1 SCSI Adapter #2 --> DB disk 2 (one vol) --> DB disk 4 (one vol) --> Log disk 2 (five vols) --> Pool disk 2 DB and log volumes on odd-number disks are TSM mirrored to volumes on the even-number disks (the volume on DB disk 1 is TSM mirrored to the volume on DB disk 2 for example). Yesterday we saw decent performance during expiration and reclamation of two primary tape pools. When a third pool's reclamation started, and especially when the system had to handle reclamation of all three primary pools and the copy pool and client sessions, the overall performance was very poor. Watching system activity with topas showed overall disk I/O on the DB disks to be very low - much lower than we saw on the old F50. Oh... I know TSM 4.1 is not supported any more. This server upgrade was a prerequisite to the software upgrade which will occur in about a month or so. Also, I know I can set a larger DB buffer pool than 256 MB and will do so over the next couple/few days. But we had better database performance on the F50 with a DB buffer pool of only 128 MB. ==> So what should I do? Split the database into "a few" volumes? Or should I look elsewhere? Thanks in advance. Tab Trepagnier TSM Administrator Laitram LLC "MMS <health-first.org>" made the following annotations on 04/01/2003 10:27:16 AM ------------------------------------------------------------------------------ This message is for the named person's use only. It may contain confidential, proprietary, or legally privileged information. No confidentiality or privilege is waived or lost by any mistransmission. If you receive this message in error, please immediately delete it and all copies of it from your system, destroy any hard copies of it, and notify the sender. You must not, directly or indirectly, use, disclose, distribute, print, or copy any part of this message if you are not the intended recipient. Health First reserves the right to monitor all e-mail communications through its networks. Any views or opinions expressed in this message are solely those of the individual sender, except (1) where the message states such views or opinions are on behalf of a particular entity; and (2) the sender is authorized by the entity to give such views or opinions. ==============================================================================