Re: TSM database maximum recommended size
Based on some comments I heard recently, nothing has been done to DB backup performance since ADSM V2. -Original Message- From: Darrel Gleddie [mailto:[EMAIL PROTECTED]] Sent: Friday, April 12, 2002 9:02 PM To: [EMAIL PROTECTED] Subject: Re: TSM database maximum recommended size Comments from [EMAIL PROTECTED] and [EMAIL PROTECTED] The times listed are HH:MM as you suspected. And why does it take so long? Well, the dbbackup was a normal-while-everything-else-is-running backup, maybe this is a partial excuse. The restore, however, was the only thing running on the system. The restore is essentially rebuilding the database, so its not just a matter of streaming the bytes onto the disk. I watched the "topas" monitor from time to time while this was going on and saw the db hdisk at 90-100% busy. (My guess is that this process has not been souped up in a long time, if ever.) Darrel Gleddie IBM Almaden Research Center
Re: TSM database maximum recommended size
Comments from [EMAIL PROTECTED] and [EMAIL PROTECTED] The times listed are HH:MM as you suspected. And why does it take so long? Well, the dbbackup was a normal-while-everything-else-is-running backup, maybe this is a partial excuse. The restore, however, was the only thing running on the system. The restore is essentially rebuilding the database, so its not just a matter of streaming the bytes onto the disk. I watched the "topas" monitor from time to time while this was going on and saw the db hdisk at 90-100% busy. (My guess is that this process has not been souped up in a long time, if ever.) Darrel Gleddie IBM Almaden Research Center
Re: TSM database maximum recommended size
A further comment on this subject: I will be transplanting a moderately sized TSM server from an H50 to a 7026-6H1. I have just completed a trial run, to determine how long the process is going to take. I used the latest full dbbackup from the H50, and did a dbrstore on the new hardware. And, just for fun, I ran an audit db on the transplant. Both the old and new hardware use raid-5 ssa for the database, the new hardware has 4 processors (600 MHZ) with 8 GB memory. For your planning purposes, here's the stats from this environment: dbsize 73,656 MB @ 84% dbbackup03:32 dbrestore 04:03 auditdb 47:28 Both dbbackup and restore were done from a 3590 "K". And, thankfully, the audit ran cleanly. Darrel Gleddie IBM Almaden Research Center
Re: TSM database maximum recommended size
On Fri, 12 Apr 2002, Darrel Gleddie wrote: > A further comment on this subject: > > I will be transplanting a moderately sized TSM server from an H50 to a 7026-6H1. I > have just completed a trial run, to determine how long the process is going to > take. I used the latest full dbbackup from the H50, and did a dbrstore on the new > hardware. And, just for fun, I ran an audit db on the transplant. > > Both the old and new hardware use raid-5 ssa for the database, the new hardware > has 4 processors (600 MHZ) with 8 GB memory. For your planning purposes, here's the > stats from this environment: > > dbsize 73,656 MB @ 84% > dbbackup03:32 > dbrestore 04:03 > auditdb 47:28 > I have been curious about these numbers. Could you tell me why a 74gig db takes 4 hours to restore off of 3590K tape using a 3590E1A drive and also why it takes 3 1/2 hours to write? These numbers suggest a 3-4MB Read and a 4-5 MB write. Why not 11-12MB for both? > Both dbbackup and restore were done from a 3590 "K". And, thankfully, the audit ran > cleanly. > > Darrel Gleddie > IBM Almaden Research Center > Jonathan Siegle Center for Academic Computing [EMAIL PROTECTED] Penn State University
Re: TSM database maximum recommended size
> At 12:08 PM 4/3/2002 -0500, Scott McCambly wrote: > >How many sites are running with databases this large or larger? On what > >hardware? > > Our DB is now 116GB assigned (92% full). > Hardware is RS6000/M80, 2GB RAM, SSA disks (db is mirrored by TSM) > As others have said, "it depends". If your requirement is to be able to > restore a database quickly, then your max size will be smaller. Besides DB backup/restore time, I also put a lot of stock in how long it takes to run thru expiration to completion. If you cannot do it within your window, and on a daily basis, then your DB is tracking objects that should be deleted, wasting both DB and tape space. Steve Roder, University at Buffalo HOD Service Coordinator VM Systems Programmer UNIX Systems Administrator (Solaris and AIX) TSM/ADSM Administrator ([EMAIL PROTECTED] | (716)645-3564 | http://ubvm.cc.buffalo.edu/~tkssteve)
Re: TSM database maximum recommended size
Actually, I think the key factor is expiration. And one reason it is an issue is that expiration is also single-threaded. And, expiration can take much longer than full DB backup on an active system, making it a more critical factor in determining ultimate capacity of a hardware configuration. When you can no longer expire database entries as quickly as they are being added, you are over the database size limit. If expiration takes longer than the time window allowed for it in your daily schedule, you have exceeded your hardware configuration's absolute maximum database size. Or, put another way, the average time it takes to do expiration, divided by the size of the time window you allow in your daily schedule for it, is the percentage of your maximum database size that you are presently at. One way of determining whether you are expiring adequately or not, is to compare the numbers returned by QUERY OCCUPANCY to the size of the dataspace being backed up. This multiple should not be too much larger than the number of inactive copies of a file you have decided to keep, in your policies. It could be less, if it is relatively static. But if this multiple is a lot higher than the number of inactive copies in your policies, it should be a red flag of expiration trouble. Faster hardware and/or splitting the server are the only answers. If your computer has multiple processors, and they are not all fully utilized, you could split the server into multiple images on the same machine and gain an advantage, in this way multi-threading expiration, as well as database backup. An SLA governing your Disaster Recovery Plan is secondary to the basic principle that you've got to get rid of stuff at least as quickly as it arrives. IT WOULD BE NICE to simply allow multiple simultaneous expiration processes. Roger Deschner University of Illinois at Chicago [EMAIL PROTECTED] == If we do not change our direction === = we are likely to end up where we are headed. = On Fri, 5 Apr 2002, Scott McCambly wrote: >Thanks to all who responded to my inquiry. > >All your comments were valid, and I especially liked Nick Cassimatis point >about TSM DB backups spanning multiple volumes. > >The main point of my question was to gather a survey of database sizes out >there, so I'm happy to keep seeing responses come in on that. > >Obviously as people go beyond a defined SLA for maximum time to restore the >TSM environment then a second server is justified. Those of us who have no >formal SLA's have to make a judgement call. > >My only hope is that Tivoli is working towards enhancing the performance >and capabilities for backing up TSM's database (multi-threaded perhaps?), >just as the TDP agents do for our other business systems. Might we assume >they are lacking incentive due to the possibility that such an enhancement >could result in fewer server license sales? ;-) > >Are any TSM developers listening aware of something to look forward to in >this area? > >Thanks, and have a good weekend! > >Scott. > >At 10:56 AM 4/3/02 -0600, you wrote: >>80 GB is in the range of what I think of as "barely manageable" on your >>average midrange Unix computer (IBM H80, Sun E450). I know folks run >>bigger DBs (somebody's got one in excess of 160GB) but you need >>substantial server hardware and very fast disks. >> >>_ >>William Mansfield >>Senior Consultant >>Solution Technology, Inc >> >> >> >> >> >>Scott McCambly <[EMAIL PROTECTED]> >>Sent by: "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]> >>04/03/2002 11:08 AM >>Please respond to "ADSM: Dist Stor Manager" >> >> >>To: [EMAIL PROTECTED] >>cc: >>Subject:TSM database maximum recommended size >> >> >>Hello all, >> >>Our data storage requirements have been driving our TSM database usage to >>new heights, and we are looking for some references to "just how big is >>too >>big" in order to help build a business case for deploying additional >>servers. >> >>Our current TSM database is 86.5GB and although backup and restore >>operations are running fine, database management tasks are becoming >>painfully long. I personally think this database size is "too big". What >>are your thoughts? >>How many sites are running with databases this large or larger? On what >>hardware? >> >>IBM/Tivoli's maxiumum size recommendations seem to grow each year with the >>introduction of more powerful hardware platforms, but issues such as >>single >>threaded db backup and restore continue to pose problems to unlimited >>growth. >> >>Any input would be greatly appreciated. >> >>Thanks, >> >>Scott. >>Scott McCambly >>AIX/NetView/ADSM Specialist - Unopsys Inc. Ottawa, Ontario, Canada >>(613)799-9269 >> >> >Scott McCambly >AIX/NetView/ADSM Specialist - Unopsys Inc. Ottawa, Ontario, Canada >(613)799-9269 >
Re: TSM database maximum recommended size
Thanks to all who responded to my inquiry. All your comments were valid, and I especially liked Nick Cassimatis point about TSM DB backups spanning multiple volumes. The main point of my question was to gather a survey of database sizes out there, so I'm happy to keep seeing responses come in on that. Obviously as people go beyond a defined SLA for maximum time to restore the TSM environment then a second server is justified. Those of us who have no formal SLA's have to make a judgement call. My only hope is that Tivoli is working towards enhancing the performance and capabilities for backing up TSM's database (multi-threaded perhaps?), just as the TDP agents do for our other business systems. Might we assume they are lacking incentive due to the possibility that such an enhancement could result in fewer server license sales? ;-) Are any TSM developers listening aware of something to look forward to in this area? Thanks, and have a good weekend! Scott. At 10:56 AM 4/3/02 -0600, you wrote: >80 GB is in the range of what I think of as "barely manageable" on your >average midrange Unix computer (IBM H80, Sun E450). I know folks run >bigger DBs (somebody's got one in excess of 160GB) but you need >substantial server hardware and very fast disks. > >_ >William Mansfield >Senior Consultant >Solution Technology, Inc > > > > > >Scott McCambly <[EMAIL PROTECTED]> >Sent by: "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]> >04/03/2002 11:08 AM >Please respond to "ADSM: Dist Stor Manager" > > >To: [EMAIL PROTECTED] >cc: >Subject:TSM database maximum recommended size > > >Hello all, > >Our data storage requirements have been driving our TSM database usage to >new heights, and we are looking for some references to "just how big is >too >big" in order to help build a business case for deploying additional >servers. > >Our current TSM database is 86.5GB and although backup and restore >operations are running fine, database management tasks are becoming >painfully long. I personally think this database size is "too big". What >are your thoughts? >How many sites are running with databases this large or larger? On what >hardware? > >IBM/Tivoli's maxiumum size recommendations seem to grow each year with the >introduction of more powerful hardware platforms, but issues such as >single >threaded db backup and restore continue to pose problems to unlimited >growth. > >Any input would be greatly appreciated. > >Thanks, > >Scott. >Scott McCambly >AIX/NetView/ADSM Specialist - Unopsys Inc. Ottawa, Ontario, Canada >(613)799-9269 > > Scott McCambly AIX/NetView/ADSM Specialist - Unopsys Inc. Ottawa, Ontario, Canada (613)799-9269
Re: TSM database maximum recommended size
I was told by a Tivoli Engineer at one time that you shouldn't let them grow much over 50 gb. Primarily, because of the reason you state here about having the capability to restore. In a disaster, you need to take into consideration how much time it is going to take you to restore that database just to get the TSM server up and running. That time would have to be added to the SLA requirements you might have in place to get any application servers back up and running. Paul Zarnowski <[EMAIL PROTECTED] To: [EMAIL PROTECTED] RNELL.EDU> cc: Sent by: "ADSM: Dist Subject: Re: TSM database maximum recommended Stor Manager" size <[EMAIL PROTECTED] U> 04/03/2002 06:41 PM Please respond to "ADSM: Dist Stor Manager" At 12:08 PM 4/3/2002 -0500, Scott McCambly wrote: >How many sites are running with databases this large or larger? On what >hardware? Our DB is now 116GB assigned (92% full). Hardware is RS6000/M80, 2GB RAM, SSA disks (db is mirrored by TSM) As others have said, "it depends". If your requirement is to be able to restore a database quickly, then your max size will be smaller.
Re: TSM database maximum recommended size
We have 80gb databases on 4 x IBM NSM's (COO, H50's, SSA disk 1gb Memory etc) They take over 4 hours to back up. We recently migrated a TSM database from one NSM to another and it took 18 hours to restore...far far too long if faced with disaster recovery. Its not an easy probelm to fix. > -Original Message- > From: Paul Zarnowski > Sent: Thursday, April 04, 2002 1:41 AM > To: [EMAIL PROTECTED] > Subject: Re: TSM database maximum recommended size > > At 12:08 PM 4/3/2002 -0500, Scott McCambly wrote: > >How many sites are running with databases this large or larger? On what > >hardware? > > Our DB is now 116GB assigned (92% full). > Hardware is RS6000/M80, 2GB RAM, SSA disks (db is mirrored by TSM) > As others have said, "it depends". If your requirement is to be able to > restore a database quickly, then your max size will be smaller.
Re: TSM database maximum recommended size
At 12:08 PM 4/3/2002 -0500, Scott McCambly wrote: >How many sites are running with databases this large or larger? On what >hardware? Our DB is now 116GB assigned (92% full). Hardware is RS6000/M80, 2GB RAM, SSA disks (db is mirrored by TSM) As others have said, "it depends". If your requirement is to be able to restore a database quickly, then your max size will be smaller.
Re: TSM database maximum recommended size
I have 2 "rules" for DB size. 1. Restore time - if it will take longer than your recovery window on your TSM server to restore the database, your DB is too big. A good estimate of restore time is 50% longer than the backup - 2 hours to backup the database, 3 hours to restore. If your maximum outage on your TSM server is 6 hours, and the database backup takes 4 hours, you're cutting it very close. 2. Backup media - I don't like spanning my TSM DB backup across 2 pieces of media. If you have something small (3570, 3995, 8mm, etc) and it takes more than 1 volume to backup the database, it's too big. If you do have to restore the database, having to find two (or more) pieces of media to do it can make things more complex that I'd want to deal with. This is a personal opinion, based on personal preferences, but it can be a good lever toward getting better/bigger/more advanced media technology, too. Nick Cassimatis [EMAIL PROTECTED] Today is the tomorrow of yesterday.
Re: TSM database maximum recommended size
Joe - my db is pretty big on s/390 -- tsm: NEW_ADSM_SERVER_ON_PORT_1600>q db f=d Available Space (MB): 185,400 Assigned Capacity (MB): 185,400 Maximum Extension (MB): 0 Maximum Reduction (MB): 28,396 Page Size (bytes): 4,096 Total Usable Pages: 47,462,400 Used Pages: 40,125,241 Pct Util: 84.5 Max. Pct Util: 84.5 Physical Volumes: 50 Buffer Pool Pages: 65,536 Total Buffer Requests: 100,564,344 Cache Hit Pct.: 96.91 Cache Wait Pct.: 0.00 Backup in Progress?: No Type of Backup In Progress: Incrementals Since Last Full: 3 Changed Since Last Backup (MB): 1,641.23 Percentage Changed: 1.05 Last Complete Backup Date/Time: 04/02/2002 15:46:09 The performance is ok I think on hardware the isn't all that fast; s390 = 9672-ra6 (80 mips, 1 processor) disk = stk sva 9500 tape = stk 9840 in a powderhrn silo. I do a full dbb on the weekend, it takes 6 hrs. I do daily incr dbb's to sms manged disk. Expiration is the slowest thing, it runs only on Saturday from 5 am to 11 pm; it takes 3 Saturdays to make a complete pass. We backup 50- 100 gb a day, mostly small files from enduser desktop machines. While I am satisfied with all this, the new management has decreed the death of ibm mainframe computing so I will be moving off to ??? starting in the next 6 months. I am interested in the sizing issue like alot of others, but I don't have any good answers. -- Bill Colwell C. S. Draper Lab Cambridge Ma. At 12:17 PM 4/3/2002 -0500, you wrote: >Can anyone comment on the DB size on an S390 TSM deployment? > >-Original Message- >From: Bill Mansfield [mailto:[EMAIL PROTECTED]] >Sent: Wednesday, April 03, 2002 11:56 AM >To: [EMAIL PROTECTED] >Subject: Re: TSM database maximum recommended size > > >80 GB is in the range of what I think of as "barely manageable" on your >average midrange Unix computer (IBM H80, Sun E450). I know folks run >bigger DBs (somebody's got one in excess of 160GB) but you need >substantial server hardware and very fast disks. > >_ >William Mansfield >Senior Consultant >Solution Technology, Inc > > > > > >Scott McCambly <[EMAIL PROTECTED]> >Sent by: "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]> >04/03/2002 11:08 AM >Please respond to "ADSM: Dist Stor Manager" > > >To: [EMAIL PROTECTED] >cc: >Subject:TSM database maximum recommended size > > >Hello all, > >Our data storage requirements have been driving our TSM database usage to >new heights, and we are looking for some references to "just how big is >too >big" in order to help build a business case for deploying additional >servers. > >Our current TSM database is 86.5GB and although backup and restore >operations are running fine, database management tasks are becoming >painfully long. I personally think this database size is "too big". What >are your thoughts? >How many sites are running with databases this large or larger? On what >hardware? > >IBM/Tivoli's maxiumum size recommendations seem to grow each year with the >introduction of more powerful hardware platforms, but issues such as >single >threaded db backup and restore continue to pose problems to unlimited >growth. > >Any input would be greatly appreciated. > >Thanks, > >Scott. >Scott McCambly >AIX/NetView/ADSM Specialist - Unopsys Inc. Ottawa, Ontario, Canada >(613)799-9269
Re: TSM database maximum recommended size
Can anyone comment on the DB size on an S390 TSM deployment?
-Original Message-
From: Bill Mansfield [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, April 03, 2002 11:56 AM
To: [EMAIL PROTECTED]
Subject: Re: TSM database maximum recommended size
80 GB is in the range of what I think of as "barely manageable" on your
average midrange Unix computer (IBM H80, Sun E450). I know folks run
bigger DBs (somebody's got one in excess of 160GB) but you need
substantial server hardware and very fast disks.
_
William Mansfield
Senior Consultant
Solution Technology, Inc
Scott McCambly <[EMAIL PROTECTED]>
Sent by: "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]>
04/03/2002 11:08 AM
Please respond to "ADSM: Dist Stor Manager"
To: [EMAIL PROTECTED]
cc:
Subject:TSM database maximum recommended size
Hello all,
Our data storage requirements have been driving our TSM database usage to
new heights, and we are looking for some references to "just how big is
too
big" in order to help build a business case for deploying additional
servers.
Our current TSM database is 86.5GB and although backup and restore
operations are running fine, database management tasks are becoming
painfully long. I personally think this database size is "too big". What
are your thoughts?
How many sites are running with databases this large or larger? On what
hardware?
IBM/Tivoli's maxiumum size recommendations seem to grow each year with the
introduction of more powerful hardware platforms, but issues such as
single
threaded db backup and restore continue to pose problems to unlimited
growth.
Any input would be greatly appreciated.
Thanks,
Scott.
Scott McCambly
AIX/NetView/ADSM Specialist - Unopsys Inc. Ottawa, Ontario, Canada
(613)799-9269
Re: TSM database maximum recommended size
80 GB is in the range of what I think of as "barely manageable" on your
average midrange Unix computer (IBM H80, Sun E450). I know folks run
bigger DBs (somebody's got one in excess of 160GB) but you need
substantial server hardware and very fast disks.
_
William Mansfield
Senior Consultant
Solution Technology, Inc
Scott McCambly <[EMAIL PROTECTED]>
Sent by: "ADSM: Dist Stor Manager" <[EMAIL PROTECTED]>
04/03/2002 11:08 AM
Please respond to "ADSM: Dist Stor Manager"
To: [EMAIL PROTECTED]
cc:
Subject:TSM database maximum recommended size
Hello all,
Our data storage requirements have been driving our TSM database usage to
new heights, and we are looking for some references to "just how big is
too
big" in order to help build a business case for deploying additional
servers.
Our current TSM database is 86.5GB and although backup and restore
operations are running fine, database management tasks are becoming
painfully long. I personally think this database size is "too big". What
are your thoughts?
How many sites are running with databases this large or larger? On what
hardware?
IBM/Tivoli's maxiumum size recommendations seem to grow each year with the
introduction of more powerful hardware platforms, but issues such as
single
threaded db backup and restore continue to pose problems to unlimited
growth.
Any input would be greatly appreciated.
Thanks,
Scott.
Scott McCambly
AIX/NetView/ADSM Specialist - Unopsys Inc. Ottawa, Ontario, Canada
(613)799-9269
TSM database maximum recommended size
Hello all, Our data storage requirements have been driving our TSM database usage to new heights, and we are looking for some references to "just how big is too big" in order to help build a business case for deploying additional servers. Our current TSM database is 86.5GB and although backup and restore operations are running fine, database management tasks are becoming painfully long. I personally think this database size is "too big". What are your thoughts? How many sites are running with databases this large or larger? On what hardware? IBM/Tivoli's maxiumum size recommendations seem to grow each year with the introduction of more powerful hardware platforms, but issues such as single threaded db backup and restore continue to pose problems to unlimited growth. Any input would be greatly appreciated. Thanks, Scott. Scott McCambly AIX/NetView/ADSM Specialist - Unopsys Inc. Ottawa, Ontario, Canada (613)799-9269
