Linux - number of drives/daily backup size
Hi, Based on your experience. What is maximum number of drives, LTO4 or J3/T10KB, and still having an excellent performance when everything is used at the same time in a Linux server? Size of daily backups? Server model? Drive type? Assumptions are larger models of HP or IBM servers on Intel/AMD, with SAN storage and 10 GigE. Thanks Henrik --- The information contained in this message may be CONFIDENTIAL and is intended for the addressee only. Any unauthorised use, dissemination of the information or copying of this message is prohibited. If you are not the addressee, please notify the sender immediately by return e-mail and delete this message. Thank you.
Re: Linux - number of drives/daily backup size
On 28 jan 2010, at 21:47, Henrik Vahlstedt wrote: Hi, Based on your experience. What is maximum number of drives, LTO4 or J3/T10KB, and still having an excellent performance when everything is used at the same time in a Linux server? calculate the bandwidth of your PCI bus/slot in Gb/s - you can only have one 4 Gb/s HBA working at 100% capacity on a PCI-X 133 bus - 10 Gb/s ethernet requires PCI-X 266 or a suitable PCIe slot calculate the bandwidth requirements of your drive - you can have maybe 2 or 3 high-speed drives per HBA Size of daily backups? Server model? Drive type? Assumptions are larger models of HP or IBM servers on Intel/AMD, with SAN storage and 10 GigE. Thanks Henrik --- The information contained in this message may be CONFIDENTIAL and is intended for the addressee only. Any unauthorised use, dissemination of the information or copying of this message is prohibited. If you are not the addressee, please notify the sender immediately by return e-mail and delete this message. Thank you. -- Met vriendelijke groeten/Kind regards, Remco Post
Re: Linux - number of drives/daily backup size
Hi, You are correct and I am aware of I can calculate a theoretical maximum. But in my world there is no definitely equal sign between theory and practice. Hence the question about your experience. Thanks Henrik -Original Message- From: ADSM: Dist Stor Manager [mailto:ads...@vm.marist.edu] On Behalf Of Remco Post Sent: den 29 januari 2010 07:41 To: ADSM-L@vm.marist.edu Subject: Re: [ADSM-L] Linux - number of drives/daily backup size On 28 jan 2010, at 21:47, Henrik Vahlstedt wrote: Hi, Based on your experience. What is maximum number of drives, LTO4 or J3/T10KB, and still having an excellent performance when everything is used at the same time in a Linux server? calculate the bandwidth of your PCI bus/slot in Gb/s - you can only have one 4 Gb/s HBA working at 100% capacity on a PCI-X 133 bus - 10 Gb/s ethernet requires PCI-X 266 or a suitable PCIe slot calculate the bandwidth requirements of your drive - you can have maybe 2 or 3 high-speed drives per HBA Size of daily backups? Server model? Drive type? Assumptions are larger models of HP or IBM servers on Intel/AMD, with SAN storage and 10 GigE. Thanks Henrik --- The information contained in this message may be CONFIDENTIAL and is intended for the addressee only. Any unauthorised use, dissemination of the information or copying of this message is prohibited. If you are not the addressee, please notify the sender immediately by return e-mail and delete this message. Thank you. -- Met vriendelijke groeten/Kind regards, Remco Post --- The information contained in this message may be CONFIDENTIAL and is intended for the addressee only. Any unauthorised use, dissemination of the information or copying of this message is prohibited. If you are not the addressee, please notify the sender immediately by return e-mail and delete this message. Thank you.
Select command for TOTAL DAILY BACKUP
I have a blank I want to know how many GB I'm taking in backup everyday ... I would like the TOTAL not by nodes ... any ideas thanks Luc Beaudoin Administrateur Réseau / Network Administrator Hopital General Juif S.M.B.D. Tel: (514) 340-8222 ext:8254
Re: Select command for TOTAL DAILY BACKUP
On Mar 22, 2007, at 2:13 PM, Luc Beaudoin wrote: I have a blank I want to know how many GB I'm taking in backup everyday ... I would like the TOTAL not by nodes ... any ideas Something like the following, for backups ended thus far today: SELECT Dec((SUM(BYTES) / (1024 * 1024 *1024)),15) as BACKUP GB TODAY - from SUMMARY where (DATE(END_TIME) = CURRENT DATE) and ACTIVITY = 'BACKUP' But the TSM accounting log is a better data source. Richard Sims
Re: Select command for TOTAL DAILY BACKUP
If you trust the summary table (which you probably shouldn't) this is interesting for all activity in last 25hr by activity( I overlap 1 hr for my reports). It's a start to add a where clase for just backup activity. select count(*) as count, - sum(bytes)/1024/1024/1024 as GigaBytes, - activity - from summary - where cast((current_timestamp-start_time)hours as decimal(8,0)) 25 - or cast((current_timestamp-end_time)hours as decimal(8,0)) 25 - group by activity Rick btw . . . I have a ksh script that generates a large report each morning of the following reports. We run this against all our tsm servers each morning. It's a fairly ugly AIX ksh script that's highly customized for our site, but it might be useful for someone out there. If interested . . . please email directly. == r000 === == morning_report for tsm3 on rsfebkup7p == Sun Mar 4 06:00:02 EST 2007 === = = REPORT INDEX = = r000 report index and beginning time stamp = r005 lib info = r010 activity summary = r015 scratch count = r019 scratch tape usage = r021 reclaimable tapes by pct-reclaim = r022 volume info = r023 volumes per stgpool status and maxscratch = r024 volume average utilization by stgpool = r025 q dr = r030 q path = r035 q mount = r036 drive activity = r040 q db = r045 q log = r050 log consumption and utilization = r055 log pin info (not emailed) = r060 q pro = r065 q sess = r070 q stgpool = r075 q copygroup (not emailed) = r076 q events for exceptions - missed backups (not emailed) = r077 slow backups = r080 db backups = r085 expiration - completions = r090 expiration - detail (not emailed) = r095 drive and media errors = r100 recplan dir listings = r105 q volhost type=dbb = r110 q volhost type=dbs (not emailed) = r120 stgpool volumes: 7 day trend (not emailed) = r125 aix errpt = r129 tdp notes - summary = r130 tdp notes - full (not emailed) = r131 tdp notes - incremental (not emailed) = r132 tdp notes - logs (not emailed) = r140 session per node where count 1 (not emailed) = r141 q option (not emailed) = r145 occupancy by server = r150 occupancy by domain = r152 occupancy by stgpool = r155 occupancy by node (not emailed) = r157 q audotocc (not emailed) = r160 nodes with no associations = r161 nodes with no associations EXCLUSION LIST = r165 nodes never backed up (not emailed) = r166 zzrt nodes with associations (not emailed) = r170 filespaces not backed up in 7 days (not emailed) = r175 filespaces never backed up (not emailed) = r180 server critical errors = r184 backup objects and bytes per domain (not emailed) = r185 backup objects and bytes per node (not emailed) = r190 q vol (not emailed) = r195 q libvol (not emailed) = r999 report end timestamp Luc Beaudoin [EMAIL PROTECTED] MCGILL.CA To Sent by: ADSM: ADSM-L@VM.MARIST.EDU Dist Stor cc Manager [EMAIL PROTECTED] Subject .EDU Select command for TOTAL DAILY BACKUP 03/22/2007 02:13 PM Please respond to ADSM: Dist Stor Manager [EMAIL PROTECTED] .EDU I have a blank I want to know how many GB I'm taking in backup everyday ... I would like the TOTAL not by nodes ... any ideas thanks Luc Beaudoin Administrateur Réseau / Network Administrator Hopital General Juif S.M.B.D. Tel: (514) 340-8222 ext:8254 - The information contained in this message is intended only for the personal and confidential use of the recipient(s) named above. If the reader
Re: Re: Select command for TOTAL DAILY BACKUP
Richard Rhodes wrote: If you trust the summary table (which you probably shouldn't) this is interesting for all activity in last 25hr by activity( I overlap 1 hr for my reports). It's a start to add a where clase for just backup activity. I use some similar queries for a daily feel of the TSM workload, wrapped in a perl script and run from cron. Broader scope than just daily backup volume, but I figured I'd share. I don't actually do anything formal with this output, it's just taking the pulse of TSM. Here are the raw select statements: Activity Summary: select activity as Operation , - cast(count(activity) as decimal(5,0)) as Times, - cast(sum(affected) as integer) as Objects, - cast(sum(examined) as integer) as Examined, - cast(sum(bytes) / 1024 / 1024 as decimal(9,0)) as Megabytes - from summary where end_timecurrent_timestamp-(1)day - group by activity Storage Pool Backup Summary: select entity as Storage Pool, - cast(sum(affected) as integer) as Objects, - cast(sum(bytes) / 1024 / 1024 as integer) as Megabytes - from summary where activity='STGPOOL BACKUP' and - end_timecurrent_timestamp-(1)day group by entity Database Performance Summary: select activity,cast((end_time) as date) as Date, - (examined/cast((end_time-start_time) seconds as - decimal(18,13))*3600) Pages/Objects per Hour from summary - where (activity='FULL_DBBACKUP' or activity='EXPIRATION') - and days(end_time)-days(start_time)=0 - and end_timecurrent_timestamp-(1)day -- Hello World.David Bronder - Systems Admin Segmentation Fault ITS-SPA, Univ. of Iowa Core dumped, disk trashed, quota filled, soda warm. [EMAIL PROTECTED]
Re: Select command for TOTAL DAILY BACKUP
Luc If you have a straightforward backup of every storage pool every day, then amount backed up = amount copied to storagepools. This number can be gotten from the actlog for each process or from the summary table. Hope this helps. Steve Steven Harris AIX and TSM Admin Brisbane Australia Luc Beaudoin wrote: I have a blank I want to know how many GB I'm taking in backup everyday ... I would like the TOTAL not by nodes ... any ideas thanks Luc Beaudoin Administrateur Réseau / Network Administrator Hopital General Juif S.M.B.D. Tel: (514) 340-8222 ext:8254
Re: Daily Backup Report: More Issues
Hi Orin,
Just to add to what Paul and Andy has already said:
TDP for Exchange - definately a problem - even a q files nodexyz f=d reports
a successfull backup - although the Rc425 in the exchange log says that
the backup of storage group xyz failed the TDP for exchange does NOT
comunicate
this fact to the TSM server - what we do is run SQL queries against the
TSM actvity log to see the success or failure of the backup.
Andy's statement that the q event is good for scheduled operations is not
true - I have an NT server where I have an ADMIN command to backup the
primary tape pool to a secondary tape pool with a wait=yes added.
When the server starts the command it ends without mounting the tapes,
because there is no free tapes available to make the copy.
In the TSM activity log you can see the process failed - but the q event
says it was fine.
According to me this should report a FAILED EVENT - but is says the event
was
OK return code 0.
Might be a bug - but I don't make use of the events table for reporting
AT ALL.
Cheers
Christo
Mark is absolutely correct. This is an issue we have as well. To solve the
problem we are running post processing of the output looking for a bogus
situation and setting return codes. We use an external scheduler, so we can
do this. With the TSM Scheduler you are just screwed if you are not
satisfied that a successful session does not necessarily equal a successful
backup.
The worst problem we have is the TDP for Exchange can get a 425 return code
because Norton Anti-virus has the Exchange store tied up and you still get a
successful backup. This takes a bounce of the Exchange Server. The issue
is you can go weeks without realizing you have not gotten an Information
Store backup.
Typically, I use SQL to looke for the message id and failed and that is how
this one is found.
I hate to correct IBM again... My statement was correct
query event will NOT tell you if your backup was successful.
As Andy so carefully stated in his last comment:
you should not have any problems determining success or failure of the
operation.
The operation and the actual successful backup are two different things. In
referring to a TSM operations (i.e. ACTION=INCREMENTAL), not a script, can
show proof of missed files and errors from reports from my activity log that
a success of the operation does not mean that you had a successful backup.
I
We currently have a Critsit open with Tivoli and IBM hardware support and
this is one of the major issues.
Just trying to help, I guarantee that if you rely only on a q event to let
your customers know if you have all their files backed up you will get
burned.
Original Message-
From: Andrew Raibeck [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, April 16, 2002 10:07 AM
To: [EMAIL PROTECTED]
Subject: Re: Daily Backup Report
query event will NOT tell you if your backup was successful.
This is true only if the schedule definition launches a script, i.e. DEF SCH
mydomain myschedule ACTION=COMMAND OBJECTS=myscript, and the script
contains commands that run asynchronously; in that case, TSM has no way to
track the actions taken within the script. It can only say, the script was
launched successfully. In short, the success or failure of the command
depends on the return code issued from the script.
For scheduled TSM operations (i.e. ACTION=INCREMENTAL), you should not have
any problems determining success or failure of the operation. For
ACTION=COMMAND operations where the command
Regards,
Andy
Andy Raibeck
IBM Software Group
Tivoli Storage Manager Client Development
Internal Notes e-mail: Andrew Raibeck/Tucson/IBM@IBMUS
Internet e-mail: [EMAIL PROTECTED]
The only dumb question is the one that goes unasked.
The command line is your friend.
Good enough is the enemy of excellence.
Mark Bertrand [EMAIL PROTECTED]
Sent by: ADSM: Dist Stor Manager [EMAIL PROTECTED] 04/16/2002 07:34
Please respond to ADSM: Dist Stor Manager
To: [EMAIL PROTECTED]
cc:
Subject:Re: Daily Backup Report
We also used query event * * begindate=today-1 enddate=today ex=yes until
I learned that this does not report on clients that were or were not backed
up successfully, this ONLY reports on if the script or schedule was
successful. To quote straight from the h q event page Use this command to
check whether schedules were processed successfully.
I will not go into a rant about this, just learn from my mistake, query
event will NOT tell you if your backup was successful.
-Original Message-
From: Williams, Tim P {PBSG} [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, April 16, 2002 9:01 AM
To: [EMAIL PROTECTED]
Subject: Re: Daily Backup Report
we generally run a q event command ex=yes
you can use begindate begintime parms, etc
help q event
fYI
-Original Message-
From: Orin Rehorst [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, April 16
Re: Daily Backup Report
Here's how we do it: At the dsmadmc prompt: def scr daily_report desc=Create a daily backup report line=1 upd scr daily_report cmd=q sch * * begind=-$1 begint=now endd=-$2 endt=now line=2 DON'T omit the negative sign preceding the $1 and $2! (Sorry if I blew the syntax a little bit on line 2, I don't have documentation at home. If there's any trouble, just issue the command help upd scr--you can sort it out for yourself.) The two asterisks in line 2 indicate, first, all policy domains, second, all node names. If you want to separate policy domains, by all means just define more than one script. Then, when you want the daily, at the dsmadmc prompt: run daily_report 1 0 outfilemmdd.txt Where mmdd indicate the month and day, or establish your own convention. So, on Monday, when we want a report for each day over the weekend, we type, again at the dsmadmc prompt (example is for next Monday): run daily_report 3 2 tsma0420.txt run daily_report 2 1 tsma0421.txt run daily_report 1 0 tsma0422.txt This yields a nice little report which prints nicely to a landscape 8.5x11 page, and includes, among other items, the scheduled start time, the actual start time, the policy domain name, the node name, the schedule name, and the completion status. The report lends itself to post-processing if you want to clean it up for import into Word or another text processor, and, on 8.5x11 paper, you even have room to annotate a cause for non-completion status, etc., in case your boss is a little bit interested! If you add f=d at the end of the cmd= in line 2, you will get a little more detailed report which also includes the completion time, but might not fit on a landscape oriented 8.5x11 page. Also, if you want to post-process with Excel, you might consider using the command dsmadmc -comma. Your redirected output files will be in comma-separated value format which is nice for importing to Excel. Hope this helps, Dennis Anyone looking for a pretty fair TSM Administrator? Glover Yes, Andy, the command line is your friend. And mine.
Re: Daily Backup Report
Hi guys
This is the TSM backup log I have set up for my customers.
This mail is sent every day to the administrator.
This is a script I created for AIX and W2K. If anyone of you like to have it
please contact me.
I am willing to give it away free.
some of the information here is in my native langue, but you guys wont be in
any trouble converting it.
here you go.
Kvedja/Regards
Petur Eythorsson
Taeknimadur/Technician
IBM Certified Specialist - AIX
Tivoli Storage Manager Certified Professional
Microsoft Certified System Engineer
[EMAIL PROTECTED]
Nyherji Hf Simi TEL: +354-569-7700
Borgartun 37105 Iceland
URL:http://www.nyherji.is
-Original Message-
From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of
Mark Bertrand
Sent: 16. apríl 2002 19:57
To: [EMAIL PROTECTED]
Subject: Re: Daily Backup Report
So, if I follow your message correctly and the point I have been trying to
make... There is no easy (one button) method of reporting, which should be
the basis of the software, the very basic core of any backup software, that
I don't have.
To answer the other question posted by Mr. Joseph Dawes, How do I track
failures?, I do it the long way. I start with the q events ex=yes as a first
level then we start going through the activity logs starting with a search
for ANE4959i reporting missed files. Next we go through individual error
logs on the clients if necessary.
-Original Message-
From: Rushforth, Tim [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, April 16, 2002 1:05 PM
To: [EMAIL PROTECTED]
Subject: Re: Daily Backup Report
And it depends on what you mean by a successful backup...
We use Bill's method below to report on all events taht do not have a status
of completed. (We only used to report on exceptions also ...)
If you are worried about some files that were not processed then completed
does not tell you what you want (for incrementals on 4.x clients and below).
Here you will have to do additional queries to report on these failed
objects. With archives and selective backups however, files in use will
report as failed (can't confirm if other failed files will) (again 4.x and
below) so that is covered here.
Now, with 5.1, incrementals, archives, and selectives all report completed
if files are not processed because of some error.
But with incrementals you can now check the return code: 4 = files skipped
(doesn't count files excluded by exclude statements). But with selectives
and archives rc=4 could simply mean that your exclude statements are working
so now you will have to do additional queries to report on failed objects
here.
We also do queries on file spaces as was mentioned that reports when the
last backup completed. We've had at least one occasion where security was
changed on an NT drive so that the System Account no longer had access
(that is what the TSM scheduler was using). Here the backup status showes
completed but the filespace in question did not get backed up and the file
space query picked it up.
Tim Rushforth
City of Winnipeg
-Original Message-
From: Bill Boyer [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, April 16, 2002 9:40 AM
To: [EMAIL PROTECTED]
Subject: Re: Daily Backup Report
Be very carefull with the EX=YES. There is a status of an event of '(?)'
that doesn't show as an exception. This condition can occur when a backup
schedule starts for a node, and then due to either the client crashing or a
networking issue the connection is dropped and never re-established within
the DURATION of the event. The status is '(?)'. There is an APAR open to
have meaningful status's instead of the '(?)', but for now this doesn't show
as an EXCEPTION.
We got bit big-time at a client of our where they out-sourced the entire TSM
operation to us. They were having failures, but they were not showing on our
EX=YES report. They were not happy campers! I'm still having a hard time
sitting! :-)
We ended up writing some PERL code to parse the Q EV client events and
reporting on anything that isn't COMPLETED.
This behavior started somewhere in the 4.1.x patch levels. So, just be
careful relying on the EX=YES to give you everything that failed.
Bill Boyer
DSS, Inc.
-Original Message-
From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of
Williams, Tim P {PBSG}
Sent: Tuesday, April 16, 2002 10:01 AM
To: [EMAIL PROTECTED]
Subject: Re: Daily Backup Report
we generally run a q event command ex=yes
you can use begindate begintime parms, etc
help q event
fYI
-Original Message-
From: Orin Rehorst [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, April 16, 2002 8:48 AM
To: [EMAIL PROTECTED]
Subject: Daily Backup Report
What are some good ways to get an automated daily report on which clients
were or were no backed up successfully?
TIA,
Regards,
Orin
Orin Rehorst
Port of Houston Authority
(Largest U.S. port in foreign tonnage)
e-mail: [EMAIL PROTECTED] mailto:[EMAIL
Re: Daily Backup Report
Thank You, I have asked the Critsit team working with me to view this thread. IMHO I really don't think that it's asking too much. Mark B. -Original Message- From: Mr. Lindsay Morris [mailto:[EMAIL PROTECTED]] Sent: Wednesday, April 17, 2002 9:04 AM To: [EMAIL PROTECTED] Subject: Re: Daily Backup Report Well, Tim's right, IMHO: there ought to be an easy (automatic, in fact) way to: find missed schedules find missed files report them to the right people Some techniques we used in building Servergraph/TSM are: 1. Prevention: catch TSM's I can't connect to client X messages in the activity log at 8:00 PM or whenever a schedule starts, and immediately forward them to an operator. If the operator will then restart the scheduler, or fix the network problem, or whatever, then you will have a completed backup in the morning instead of a missed backup. 2. Enlist the users: for large sites, we tie an email address to each node, and send a canned message to that address that explains how to restart the scheduler. So the users can do some of the obvious things for you. Of course this won't work for all sites - some users prefer to be oblivious - but you can always put in the help-desk address. 3. Use filespace dates: We agree that q event is not totally reliable, so we suggest query filespace f=d, and look at the last backed-up date. This also shows you filespace that you should delete - i.e., the one named client_X/mnt, not backed up in 2 years. Somebody left a CD-ROM mounted 2 years ago; TSM backed it up and is religiously saving that useless space. 4. Sort nodes by how many missed files they have. So over time, you can either fix your excludes or learn to shut down databases before backing them up, and the number of missed files goes toward zero. The fact that TSM will report a backup complete even though it misses a few files is a problem, I think. My desktop PC typically misses only one file: outlook.pst, 120 MB of email that *I* consider *VERY* important - but that backup is marked completed! So if you REALLY want to be sure, you HAVE to look at every missed file. - Mr. Lindsay Morris CEO, Servergraph www.servergraph.com 859-253-8000 ofc 425-988-8478 fax -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of Jim Healy Sent: Wednesday, April 17, 2002 8:43 AM To: [EMAIL PROTECTED] Subject: Re: Daily Backup Report The cheapest way to do this is to get yourself familiar with SQL queries, all the information you need is in the tables. Mark Bertrand [EMAIL PROTECTED]@VM.MARIST.EDU on 04/16/2002 03:56:30 PM Please respond to ADSM: Dist Stor Manager [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager [EMAIL PROTECTED] To: [EMAIL PROTECTED] cc: Subject: Re: Daily Backup Report So, if I follow your message correctly and the point I have been trying to make... There is no easy (one button) method of reporting, which should be the basis of the software, the very basic core of any backup software, that I don't have. To answer the other question posted by Mr. Joseph Dawes, How do I track failures?, I do it the long way. I start with the q events ex=yes as a first level then we start going through the activity logs starting with a search for ANE4959i reporting missed files. Next we go through individual error logs on the clients if necessary. -Original Message- From: Rushforth, Tim [mailto:[EMAIL PROTECTED]] Sent: Tuesday, April 16, 2002 1:05 PM To: [EMAIL PROTECTED] Subject: Re: Daily Backup Report And it depends on what you mean by a successful backup... We use Bill's method below to report on all events taht do not have a status of completed. (We only used to report on exceptions also ...) If you are worried about some files that were not processed then completed does not tell you what you want (for incrementals on 4.x clients and below). Here you will have to do additional queries to report on these failed objects. With archives and selective backups however, files in use will report as failed (can't confirm if other failed files will) (again 4.x and below) so that is covered here. Now, with 5.1, incrementals, archives, and selectives all report completed if files are not processed because of some error. But with incrementals you can now check the return code: 4 = files skipped (doesn't count files excluded by exclude statements). But with selectives and archives rc=4 could simply mean that your exclude statements are working so now you will have to do additional queries to report on failed objects here. We also do queries on file spaces as was mentioned that reports when the last backup completed. We've had at least one occasion where security was changed on an NT drive so that the System Account no longer had access (that is what the TSM scheduler was using). Here
Re: Daily Backup Report: More Issues
Mark is absolutely correct. This is an issue we have as well. To solve the
problem we are running post processing of the output looking for a bogus
situation and setting return codes. We use an external scheduler, so we can
do this. With the TSM Scheduler you are just screwed if you are not
satisfied that a successful session does not necessarily equal a successful
backup.
The worst problem we have is the TDP for Exchange can get a 425 return code
because Norton Anti-virus has the Exchange store tied up and you still get a
successful backup. This takes a bounce of the Exchange Server. The issue
is you can go weeks without realizing you have not gotten an Information
Store backup.
Typically, I use SQL to looke for the message id and failed and that is how
this one is found.
-Original Message-
From: Mark Bertrand [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, April 16, 2002 3:29 PM
To: [EMAIL PROTECTED]
Subject: Re: Daily Backup Report
I hate to correct IBM again... My statement was correct
query event will NOT tell you if your backup was successful.
As Andy so carefully stated in his last comment:
you should not have any problems determining success or failure of the
operation.
The operation and the actual successful backup are two different things. In
referring to a TSM operations (i.e. ACTION=INCREMENTAL), not a script, can
show proof of missed files and errors from reports from my activity log that
a success of the operation does not mean that you had a successful backup.
I
We currently have a Critsit open with Tivoli and IBM hardware support and
this is one of the major issues.
Just trying to help, I guarantee that if you rely only on a q event to let
your customers know if you have all their files backed up you will get
burned.
Original Message-
From: Andrew Raibeck [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, April 16, 2002 10:07 AM
To: [EMAIL PROTECTED]
Subject: Re: Daily Backup Report
query event will NOT tell you if your backup was successful.
This is true only if the schedule definition launches a script, i.e. DEF SCH
mydomain myschedule ACTION=COMMAND OBJECTS=myscript, and the script
contains commands that run asynchronously; in that case, TSM has no way to
track the actions taken within the script. It can only say, the script was
launched successfully. In short, the success or failure of the command
depends on the return code issued from the script.
For scheduled TSM operations (i.e. ACTION=INCREMENTAL), you should not have
any problems determining success or failure of the operation. For
ACTION=COMMAND operations where the command
Regards,
Andy
Andy Raibeck
IBM Software Group
Tivoli Storage Manager Client Development
Internal Notes e-mail: Andrew Raibeck/Tucson/IBM@IBMUS
Internet e-mail: [EMAIL PROTECTED]
The only dumb question is the one that goes unasked.
The command line is your friend.
Good enough is the enemy of excellence.
Mark Bertrand [EMAIL PROTECTED]
Sent by: ADSM: Dist Stor Manager [EMAIL PROTECTED] 04/16/2002 07:34
Please respond to ADSM: Dist Stor Manager
To: [EMAIL PROTECTED]
cc:
Subject:Re: Daily Backup Report
We also used query event * * begindate=today-1 enddate=today ex=yes until
I learned that this does not report on clients that were or were not backed
up successfully, this ONLY reports on if the script or schedule was
successful. To quote straight from the h q event page Use this command to
check whether schedules were processed successfully.
I will not go into a rant about this, just learn from my mistake, query
event will NOT tell you if your backup was successful.
-Original Message-
From: Williams, Tim P {PBSG} [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, April 16, 2002 9:01 AM
To: [EMAIL PROTECTED]
Subject: Re: Daily Backup Report
we generally run a q event command ex=yes
you can use begindate begintime parms, etc
help q event
fYI
-Original Message-
From: Orin Rehorst [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, April 16, 2002 8:48 AM
To: [EMAIL PROTECTED]
Subject: Daily Backup Report
What are some good ways to get an automated daily report on which clients
were or were no backed up successfully?
TIA,
Regards,
Orin
Orin Rehorst
Port of Houston Authority
(Largest U.S. port in foreign tonnage)
e-mail: [EMAIL PROTECTED] mailto:[EMAIL PROTECTED]
Phone: (713)670-2443
Fax: (713)670-2457
TOPAS web site: www.homestead.com/topas/topas.html
http://www.homestead.com/topas/topas.html
Daily Backup Report
What are some good ways to get an automated daily report on which clients were or were no backed up successfully? TIA, Regards, Orin Orin Rehorst Port of Houston Authority (Largest U.S. port in foreign tonnage) e-mail: [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] Phone: (713)670-2443 Fax: (713)670-2457 TOPAS web site: www.homestead.com/topas/topas.html http://www.homestead.com/topas/topas.html
Re: Daily Backup Report
we generally run a q event command ex=yes you can use begindate begintime parms, etc help q event fYI -Original Message- From: Orin Rehorst [mailto:[EMAIL PROTECTED]] Sent: Tuesday, April 16, 2002 8:48 AM To: [EMAIL PROTECTED] Subject: Daily Backup Report What are some good ways to get an automated daily report on which clients were or were no backed up successfully? TIA, Regards, Orin Orin Rehorst Port of Houston Authority (Largest U.S. port in foreign tonnage) e-mail: [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] Phone: (713)670-2443 Fax: (713)670-2457 TOPAS web site: www.homestead.com/topas/topas.html http://www.homestead.com/topas/topas.html
Re: Daily Backup Report
We also used query event * * begindate=today-1 enddate=today ex=yes until
I learned that this does not report on clients that were or were not backed
up successfully, this ONLY reports on if the script or schedule was
successful. To quote straight from the h q event page Use this command to
check whether schedules were processed successfully.
I will not go into a rant about this, just learn from my mistake, query
event will NOT tell you if your backup was successful.
-Original Message-
From: Williams, Tim P {PBSG} [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, April 16, 2002 9:01 AM
To: [EMAIL PROTECTED]
Subject: Re: Daily Backup Report
we generally run a q event command ex=yes
you can use begindate begintime parms, etc
help q event
fYI
-Original Message-
From: Orin Rehorst [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, April 16, 2002 8:48 AM
To: [EMAIL PROTECTED]
Subject: Daily Backup Report
What are some good ways to get an automated daily report on which clients
were or were no backed up successfully?
TIA,
Regards,
Orin
Orin Rehorst
Port of Houston Authority
(Largest U.S. port in foreign tonnage)
e-mail: [EMAIL PROTECTED] mailto:[EMAIL PROTECTED]
Phone: (713)670-2443
Fax: (713)670-2457
TOPAS web site: www.homestead.com/topas/topas.html
http://www.homestead.com/topas/topas.html
Re: Daily Backup Report
I have the following script as a POSTSCHEDCMD option in the dsm.sys. This
works real well. I receive a mail every night immediatly after the backup
ran.
in the dsm.sys:
POSTSCHEDULECMD /audit_logs/scripts/adsm.mail
#!/usr/bin/ksh
STAMPIT=`date +%y%m%d`
MAILCNTRL=/audit_logs/scripts/mail.central
MAILFLE=/var/adm/tsm/mail.file
CUT_FLE=/var/adm/tsm/cut_adsmlog
$MAILFLE
HOST=`hostname -s`
OWNER=`grep ^${HOST}_adsm_bklog= $MAILCNTRL | cut -f2 -d'=' | tail -1`
grep ^`date +%m/%d/%y` /var/adm/tsm/sched.log $MAILFLE
if [[ $? = 0 ]]
then
SUBJ=ADSM Backups - $HOST $STAMPIT
tail -20 $MAILFLE $CUT_FLE
else
SUBJ=No Backups Were Taken
fi
mail -s $SUBJ $OWNER $CUT_FLE
rm $CUT_FLE
exit 0
Orin Rehorst wrote:
What are some good ways to get an automated daily report on which clients
were or were no backed up successfully?
TIA,
Regards,
Orin
Orin Rehorst
Port of Houston Authority
(Largest U.S. port in foreign tonnage)
e-mail: [EMAIL PROTECTED] mailto:[EMAIL PROTECTED]
Phone: (713)670-2443
Fax: (713)670-2457
TOPAS web site: www.homestead.com/topas/topas.html
http://www.homestead.com/topas/topas.html
--
**
* Frank Kruse [EMAIL PROTECTED]*
* IBM Austin 512-823-7475 *
**
Re: Daily Backup Report
Does anybody have a bash or ksh script for it?
Regards,
Burak
[EMAIL PROTECTED]
Sent by: [EMAIL PROTECTED]
16.04.2002 17:45
Please respond to ADSM-L
To: [EMAIL PROTECTED]
cc:
Subject: Re: Daily Backup Report
Be very carefull with the EX=YES. There is a status of an event of '(?)'
that doesn't show as an exception. This condition can occur when a backup
schedule starts for a node, and then due to either the client crashing or a
networking issue the connection is dropped and never re-established within
the DURATION of the event. The status is '(?)'. There is an APAR open to
have meaningful status's instead of the '(?)', but for now this doesn't show
as an EXCEPTION.
We got bit big-time at a client of our where they out-sourced the entire TSM
operation to us. They were having failures, but they were not showing on our
EX=YES report. They were not happy campers! I'm still having a hard time
sitting! :-)
We ended up writing some PERL code to parse the Q EV client events and
reporting on anything that isn't COMPLETED.
This behavior started somewhere in the 4.1.x patch levels. So, just be
careful relying on the EX=YES to give you everything that failed.
Bill Boyer
DSS, Inc.
-Original Message-
From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of
Williams, Tim P {PBSG}
Sent: Tuesday, April 16, 2002 10:01 AM
To: [EMAIL PROTECTED]
Subject: Re: Daily Backup Report
we generally run a q event command ex=yes
you can use begindate begintime parms, etc
help q event
fYI
-Original Message-
From: Orin Rehorst [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, April 16, 2002 8:48 AM
To: [EMAIL PROTECTED]
Subject: Daily Backup Report
What are some good ways to get an automated daily report on which clients
were or were no backed up successfully?
TIA,
Regards,
Orin
Orin Rehorst
Port of Houston Authority
(Largest U.S. port in foreign tonnage)
e-mail: [EMAIL PROTECTED] mailto:[EMAIL PROTECTED]
Phone: (713)670-2443
Fax: (713)670-2457
TOPAS web site: www.homestead.com/topas/topas.html
http://www.homestead.com/topas/topas.html
Re: Daily Backup Report
Couldn't you just pipe the output of your Q EV EX=NO command into GREP to
exclude any lines that have 'Completed' as the status? This should give you
everything else included 'Started', 'Pending', 'Missed','Failed','(?)',...
Bill Boyer
DSS, INc.
-Original Message-
From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of
Burak Demircan
Sent: Tuesday, April 16, 2002 10:51 AM
To: [EMAIL PROTECTED]
Subject: Re: Daily Backup Report
Does anybody have a bash or ksh script for it?
Regards,
Burak
[EMAIL PROTECTED]
Sent by: [EMAIL PROTECTED]
16.04.2002 17:45
Please respond to ADSM-L
To:[EMAIL PROTECTED]
cc:
Subject:Re: Daily Backup Report
Be very carefull with the EX=YES. There is a status of an event of '(?)'
that doesn't show as an exception. This condition can occur when a backup
schedule starts for a node, and then due to either the client crashing or a
networking issue the connection is dropped and never re-established within
the DURATION of the event. The status is '(?)'. There is an APAR open to
have meaningful status's instead of the '(?)', but for now this doesn't show
as an EXCEPTION.
We got bit big-time at a client of our where they out-sourced the entire TSM
operation to us. They were having failures, but they were not showing on our
EX=YES report. They were not happy campers! I'm still having a hard time
sitting! :-)
We ended up writing some PERL code to parse the Q EV client events and
reporting on anything that isn't COMPLETED.
This behavior started somewhere in the 4.1.x patch levels. So, just be
careful relying on the EX=YES to give you everything that failed.
Bill Boyer
DSS, Inc.
-Original Message-
From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED]]On Behalf Of
Williams, Tim P {PBSG}
Sent: Tuesday, April 16, 2002 10:01 AM
To: [EMAIL PROTECTED]
Subject: Re: Daily Backup Report
we generally run a q event command ex=yes
you can use begindate begintime parms, etc
help q event
fYI
-Original Message-
From: Orin Rehorst [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, April 16, 2002 8:48 AM
To: [EMAIL PROTECTED]
Subject: Daily Backup Report
What are some good ways to get an automated daily report on which clients
were or were no backed up successfully?
TIA,
Regards,
Orin
Orin Rehorst
Port of Houston Authority
(Largest U.S. port in foreign tonnage)
e-mail: [EMAIL PROTECTED] mailto:[EMAIL PROTECTED]
Phone: (713)670-2443
Fax: (713)670-2457
TOPAS web site: www.homestead.com/topas/topas.html
http://www.homestead.com/topas/topas.html
=
Re: Daily Backup Report
Sorry folks, for the last sentence below that sort of trails off at the
end. I meant to delete it before hitting send.
Regards,
Andy
Andy Raibeck
IBM Software Group
Tivoli Storage Manager Client Development
Internal Notes e-mail: Andrew Raibeck/Tucson/IBM@IBMUS
Internet e-mail: [EMAIL PROTECTED]
The only dumb question is the one that goes unasked.
The command line is your friend.
Good enough is the enemy of excellence.
Andrew Raibeck/Tucson/IBM@IBMUS
Sent by: ADSM: Dist Stor Manager [EMAIL PROTECTED]
04/16/2002 08:07
Please respond to ADSM: Dist Stor Manager
To: [EMAIL PROTECTED]
cc:
Subject:Re: Daily Backup Report
query event will NOT tell you if your backup was successful.
This is true only if the schedule definition launches a script, i.e. DEF
SCH mydomain myschedule ACTION=COMMAND OBJECTS=myscript, and the script
contains commands that run asynchronously; in that case, TSM has no way to
track the actions taken within the script. It can only say, the script
was launched successfully. In short, the success or failure of the
command depends on the return code issued from the script.
For scheduled TSM operations (i.e. ACTION=INCREMENTAL), you should not
have any problems determining success or failure of the operation. For
ACTION=COMMAND operations where the command
Regards,
Andy
Andy Raibeck
IBM Software Group
Tivoli Storage Manager Client Development
Internal Notes e-mail: Andrew Raibeck/Tucson/IBM@IBMUS
Internet e-mail: [EMAIL PROTECTED]
The only dumb question is the one that goes unasked.
The command line is your friend.
Good enough is the enemy of excellence.
Mark Bertrand [EMAIL PROTECTED]
Sent by: ADSM: Dist Stor Manager [EMAIL PROTECTED]
04/16/2002 07:34
Please respond to ADSM: Dist Stor Manager
To: [EMAIL PROTECTED]
cc:
Subject:Re: Daily Backup Report
We also used query event * * begindate=today-1 enddate=today ex=yes
until
I learned that this does not report on clients that were or were not
backed
up successfully, this ONLY reports on if the script or schedule was
successful. To quote straight from the h q event page Use this command to
check whether schedules were processed successfully.
I will not go into a rant about this, just learn from my mistake, query
event will NOT tell you if your backup was successful.
-Original Message-
From: Williams, Tim P {PBSG} [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, April 16, 2002 9:01 AM
To: [EMAIL PROTECTED]
Subject: Re: Daily Backup Report
we generally run a q event command ex=yes
you can use begindate begintime parms, etc
help q event
fYI
-Original Message-
From: Orin Rehorst [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, April 16, 2002 8:48 AM
To: [EMAIL PROTECTED]
Subject: Daily Backup Report
What are some good ways to get an automated daily report on which clients
were or were no backed up successfully?
TIA,
Regards,
Orin
Orin Rehorst
Port of Houston Authority
(Largest U.S. port in foreign tonnage)
e-mail: [EMAIL PROTECTED] mailto:[EMAIL PROTECTED]
Phone: (713)670-2443
Fax: (713)670-2457
TOPAS web site: www.homestead.com/topas/topas.html
http://www.homestead.com/topas/topas.html
Re: Daily Backup Report
I hate to correct IBM again... My statement was correct
query event will NOT tell you if your backup was successful.
As Andy so carefully stated in his last comment:
you should not have any problems determining success or failure of the
operation.
The operation and the actual successful backup are two different things. In
referring to a TSM operations (i.e. ACTION=INCREMENTAL), not a script, can
show proof of missed files and errors from reports from my activity log that
a success of the operation does not mean that you had a successful backup.
I
We currently have a Critsit open with Tivoli and IBM hardware support and
this is one of the major issues.
Just trying to help, I guarantee that if you rely only on a q event to let
your customers know if you have all their files backed up you will get
burned.
Original Message-
From: Andrew Raibeck [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, April 16, 2002 10:07 AM
To: [EMAIL PROTECTED]
Subject: Re: Daily Backup Report
query event will NOT tell you if your backup was successful.
This is true only if the schedule definition launches a script, i.e. DEF
SCH mydomain myschedule ACTION=COMMAND OBJECTS=myscript, and the script
contains commands that run asynchronously; in that case, TSM has no way to
track the actions taken within the script. It can only say, the script
was launched successfully. In short, the success or failure of the
command depends on the return code issued from the script.
For scheduled TSM operations (i.e. ACTION=INCREMENTAL), you should not
have any problems determining success or failure of the operation. For
ACTION=COMMAND operations where the command
Regards,
Andy
Andy Raibeck
IBM Software Group
Tivoli Storage Manager Client Development
Internal Notes e-mail: Andrew Raibeck/Tucson/IBM@IBMUS
Internet e-mail: [EMAIL PROTECTED]
The only dumb question is the one that goes unasked.
The command line is your friend.
Good enough is the enemy of excellence.
Mark Bertrand [EMAIL PROTECTED]
Sent by: ADSM: Dist Stor Manager [EMAIL PROTECTED]
04/16/2002 07:34
Please respond to ADSM: Dist Stor Manager
To: [EMAIL PROTECTED]
cc:
Subject:Re: Daily Backup Report
We also used query event * * begindate=today-1 enddate=today ex=yes
until
I learned that this does not report on clients that were or were not
backed
up successfully, this ONLY reports on if the script or schedule was
successful. To quote straight from the h q event page Use this command to
check whether schedules were processed successfully.
I will not go into a rant about this, just learn from my mistake, query
event will NOT tell you if your backup was successful.
-Original Message-
From: Williams, Tim P {PBSG} [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, April 16, 2002 9:01 AM
To: [EMAIL PROTECTED]
Subject: Re: Daily Backup Report
we generally run a q event command ex=yes
you can use begindate begintime parms, etc
help q event
fYI
-Original Message-
From: Orin Rehorst [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, April 16, 2002 8:48 AM
To: [EMAIL PROTECTED]
Subject: Daily Backup Report
What are some good ways to get an automated daily report on which clients
were or were no backed up successfully?
TIA,
Regards,
Orin
Orin Rehorst
Port of Houston Authority
(Largest U.S. port in foreign tonnage)
e-mail: [EMAIL PROTECTED] mailto:[EMAIL PROTECTED]
Phone: (713)670-2443
Fax: (713)670-2457
TOPAS web site: www.homestead.com/topas/topas.html
http://www.homestead.com/topas/topas.html
Re: Daily Backup Report
so how do you track failures
Mark Bertrand
Mark.Bertrand@USUN To: [EMAIL PROTECTED]
WIRED.COMcc:
Sent by: ADSM: Subject: Re: Daily Backup Report
Dist Stor Manager
[EMAIL PROTECTED]
DU
04/16/02 03:29 PM
Please respond to
ADSM: Dist Stor
Manager
I hate to correct IBM again... My statement was correct
query event will NOT tell you if your backup was successful.
As Andy so carefully stated in his last comment:
you should not have any problems determining success or failure of the
operation.
The operation and the actual successful backup are two different things. In
referring to a TSM operations (i.e. ACTION=INCREMENTAL), not a script, can
show proof of missed files and errors from reports from my activity log
that
a success of the operation does not mean that you had a successful
backup.
I
We currently have a Critsit open with Tivoli and IBM hardware support and
this is one of the major issues.
Just trying to help, I guarantee that if you rely only on a q event to let
your customers know if you have all their files backed up you will get
burned.
Original Message-
From: Andrew Raibeck [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, April 16, 2002 10:07 AM
To: [EMAIL PROTECTED]
Subject: Re: Daily Backup Report
query event will NOT tell you if your backup was successful.
This is true only if the schedule definition launches a script, i.e. DEF
SCH mydomain myschedule ACTION=COMMAND OBJECTS=myscript, and the script
contains commands that run asynchronously; in that case, TSM has no way to
track the actions taken within the script. It can only say, the script
was launched successfully. In short, the success or failure of the
command depends on the return code issued from the script.
For scheduled TSM operations (i.e. ACTION=INCREMENTAL), you should not
have any problems determining success or failure of the operation. For
ACTION=COMMAND operations where the command
Regards,
Andy
Andy Raibeck
IBM Software Group
Tivoli Storage Manager Client Development
Internal Notes e-mail: Andrew Raibeck/Tucson/IBM@IBMUS
Internet e-mail: [EMAIL PROTECTED]
The only dumb question is the one that goes unasked.
The command line is your friend.
Good enough is the enemy of excellence.
Mark Bertrand [EMAIL PROTECTED]
Sent by: ADSM: Dist Stor Manager [EMAIL PROTECTED]
04/16/2002 07:34
Please respond to ADSM: Dist Stor Manager
To: [EMAIL PROTECTED]
cc:
Subject:Re: Daily Backup Report
We also used query event * * begindate=today-1 enddate=today ex=yes
until
I learned that this does not report on clients that were or were not
backed
up successfully, this ONLY reports on if the script or schedule was
successful. To quote straight from the h q event page Use this command to
check whether schedules were processed successfully.
I will not go into a rant about this, just learn from my mistake, query
event will NOT tell you if your backup was successful.
-Original Message-
From: Williams, Tim P {PBSG} [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, April 16, 2002 9:01 AM
To: [EMAIL PROTECTED]
Subject: Re: Daily Backup Report
we generally run a q event command ex=yes
you can use begindate begintime parms, etc
help q event
fYI
-Original Message-
From: Orin Rehorst [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, April 16, 2002 8:48 AM
To: [EMAIL PROTECTED]
Subject: Daily Backup Report
What are some good ways to get an automated daily report on which clients
were or were no backed up successfully?
TIA,
Regards,
Orin
Orin Rehorst
Port of Houston Authority
(Largest U.S. port in foreign tonnage)
e-mail: [EMAIL PROTECTED] mailto:[EMAIL PROTECTED]
Phone: (713)670-2443
Fax: (713)670-2457
TOPAS web site: www.homestead.com/topas/topas.html
http://www.homestead.com/topas/topas.html
Re: Daily Backup Report
Historically (since ADSM V1), successful incremental backups had not taken skipped files into account. The rationale was that when processing entire file systems, you are very likely to run into one or more files that could not be processed. That being the case, you still need to check the client dsmerror.log and/or dsmsched.log file for skipped files. I understand that this may not fit your definition of successful, and it doesn't take a very large TSM installation to make checking individual logs a tedious task (to say the least). But likewise, if we flagged every event as failed where a file was skipped, many users would not be happy with that definition, either, i.e. I don't care if TSM couldn't back up junk.txt, I don't want the backup to be treated as failed. While it has been a long time in coming, version 5.1 addresses this with the consistent return codes feature. If files are skipped, then while the event is still flagged as completed, the result code field in the QUERY EVENT F=D output will show you the return code for the event. If the RC is 0, then all files that were eligible for backup were backed up. If the RC is 4, then you know that one or more files were skipped. If the RC is 8, then you know at least one warning-level message was issued (and it is possible that one or more files were skipped). The purpose of the RC 8 is to let you know that while the file systems were processed to completion, you may want to check the client for more severe problems than skipped files (but also check for skipped files). If the RC is 12, then one or more error messages were issued during the backup. In this case, the problems encountered were probably severe enough to prevent processing of one or more file systems, and the event is flagged as failed. Any behavior that deviates from this would most likely be a bug (i.e. if you saw a skipped file, but the RC still showed as 0). I am not trying to be argumentative, but I do not think it is entirely correct to say that QUERY EVENT is not useful to determine success or failure of the backup operation... as long as you understand the meaning. Agreed, is was not so clear prior to 5.1 if you required a successful backup to mean that no files were skipped (and I know this was a hot button for some customers). Hopefully the 5.1 stuff I mentioned above will address this for you. I apologize in advance if I am missing the boat here, but this discussion so far has been pretty general, and I am not aware of your specific issues since you are addressing this through IBM's formal channels, there most likely there is more to your issue(s) than can be resolved on this forum. Regards, Andy Andy Raibeck IBM Software Group Tivoli Storage Manager Client Development Internal Notes e-mail: Andrew Raibeck/Tucson/IBM@IBMUS Internet e-mail: [EMAIL PROTECTED] The only dumb question is the one that goes unasked. The command line is your friend. Good enough is the enemy of excellence. Mark Bertrand [EMAIL PROTECTED] Sent by: ADSM: Dist Stor Manager [EMAIL PROTECTED] 04/16/2002 12:29 Please respond to ADSM: Dist Stor Manager To: [EMAIL PROTECTED] cc: Subject:Re: Daily Backup Report I hate to correct IBM again... My statement was correct query event will NOT tell you if your backup was successful. As Andy so carefully stated in his last comment: you should not have any problems determining success or failure of the operation. The operation and the actual successful backup are two different things. In referring to a TSM operations (i.e. ACTION=INCREMENTAL), not a script, can show proof of missed files and errors from reports from my activity log that a success of the operation does not mean that you had a successful backup. I We currently have a Critsit open with Tivoli and IBM hardware support and this is one of the major issues. Just trying to help, I guarantee that if you rely only on a q event to let your customers know if you have all their files backed up you will get burned. Original Message- From: Andrew Raibeck [mailto:[EMAIL PROTECTED]] Sent: Tuesday, April 16, 2002 10:07 AM To: [EMAIL PROTECTED] Subject: Re: Daily Backup Report query event will NOT tell you if your backup was successful. This is true only if the schedule definition launches a script, i.e. DEF SCH mydomain myschedule ACTION=COMMAND OBJECTS=myscript, and the script contains commands that run asynchronously; in that case, TSM has no way to track the actions taken within the script. It can only say, the script was launched successfully. In short, the success or failure of the command depends on the return code issued from the script. For scheduled TSM operations (i.e. ACTION=INCREMENTAL), you should not have any problems determining success or failure of the operation. For ACTION=COMMAND operations where the command Regards, Andy Andy Raibeck IBM Software Group Tivoli Storage Manager Client Development Internal Notes e-mail
Re: Daily Backup Report
We use something similar, but only report on client errors (for this
particular report).
select MESSAGE,DOMAINNAME,NODENAME,DATE_TIME from actlog where -
date_timecurrent_timestamp - 1 days and originator='CLIENT' AND
SEVERITY='E' -
ORDER BY DOMAINNAME,NODENAME
Also something to check is for message ANE4959I when the number 0:
04/16/2002 04:53:49 ANE4959I (Session: 3380, Node: X) Total
number of objects failed: 2
I've seen a NetWare client have failed objects (NDS objects failed) with no
other errors logged to server, only way to catch this is via this message or
scan local client error\schedule logs.
Tim Rushforth
City of Winnipeg
-Original Message-
From: Jim Healy [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, April 16, 2002 4:06 PM
To: [EMAIL PROTECTED]
Subject: Re: Daily Backup Report
try this , I use this as a startingpoint to search for failures
SELECT ACTLOG.DATE_TIME, ACTLOG.MESSAGE, ACTLOG.SEVERITY FROM ACTLOG
ACTLOG WHERE (ACTLOG.SEVERITY='E') and (ACTLOG.DATE_TIME{ts (current
date
-1 day, '21:00:00')})
Joseph Dawes [EMAIL PROTECTED]@VM.MARIST.EDU on 04/16/2002 03:34:23 PM
Please respond to ADSM: Dist Stor Manager [EMAIL PROTECTED]
Sent by: ADSM: Dist Stor Manager [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
cc:
Subject: Re: Daily Backup Report
so how do you track failures
Re: Daily Backup Report
thanks alot :)
Jim Healy
James.Healy@AXA To: [EMAIL PROTECTED]
-TECH.COM cc:
Sent by: ADSM:Subject: Re: Daily Backup Report
Dist Stor
Manager
[EMAIL PROTECTED]
T.EDU
04/16/02 05:06
PM
Please respond
to ADSM: Dist
Stor Manager
try this , I use this as a startingpoint to search for failures
SELECT ACTLOG.DATE_TIME, ACTLOG.MESSAGE, ACTLOG.SEVERITY FROM ACTLOG
ACTLOG WHERE (ACTLOG.SEVERITY='E') and (ACTLOG.DATE_TIME{ts (current
date
-1 day, '21:00:00')})
Joseph Dawes [EMAIL PROTECTED]@VM.MARIST.EDU on 04/16/2002 03:34:23 PM
Please respond to ADSM: Dist Stor Manager [EMAIL PROTECTED]
Sent by: ADSM: Dist Stor Manager [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
cc:
Subject: Re: Daily Backup Report
so how do you track failures
Mark Bertrand
Mark.Bertrand@USUN To: [EMAIL PROTECTED]
WIRED.COMcc:
Sent by: ADSM: Subject: Re: Daily Backup
Report
Dist Stor Manager
[EMAIL PROTECTED]
DU
04/16/02 03:29 PM
Please respond to
ADSM: Dist Stor
Manager
I hate to correct IBM again... My statement was correct
query event will NOT tell you if your backup was successful.
As Andy so carefully stated in his last comment:
you should not have any problems determining success or failure of the
operation.
The operation and the actual successful backup are two different things. In
referring to a TSM operations (i.e. ACTION=INCREMENTAL), not a script, can
show proof of missed files and errors from reports from my activity log
that
a success of the operation does not mean that you had a successful
backup.
I
We currently have a Critsit open with Tivoli and IBM hardware support and
this is one of the major issues.
Just trying to help, I guarantee that if you rely only on a q event to let
your customers know if you have all their files backed up you will get
burned.
Original Message-
From: Andrew Raibeck [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, April 16, 2002 10:07 AM
To: [EMAIL PROTECTED]
Subject: Re: Daily Backup Report
query event will NOT tell you if your backup was successful.
This is true only if the schedule definition launches a script, i.e. DEF
SCH mydomain myschedule ACTION=COMMAND OBJECTS=myscript, and the script
contains commands that run asynchronously; in that case, TSM has no way to
track the actions taken within the script. It can only say, the script
was launched successfully. In short, the success or failure of the
command depends on the return code issued from the script.
For scheduled TSM operations (i.e. ACTION=INCREMENTAL), you should not
have any problems determining success or failure of the operation. For
ACTION=COMMAND operations where the command
Regards,
Andy
Andy Raibeck
IBM Software Group
Tivoli Storage Manager Client Development
Internal Notes e-mail: Andrew Raibeck/Tucson/IBM@IBMUS
Internet e-mail: [EMAIL PROTECTED]
The only dumb question is the one that goes unasked.
The command line is your friend.
Good enough is the enemy of excellence.
Mark Bertrand [EMAIL PROTECTED]
Sent by: ADSM: Dist Stor Manager [EMAIL PROTECTED]
04/16/2002 07:34
Please respond to ADSM: Dist Stor Manager
To: [EMAIL PROTECTED]
cc:
Subject:Re: Daily Backup Report
We also used query event * * begindate=today-1 enddate=today ex=yes
until
I learned that this does not report on clients that were or were not
backed
up successfully, this ONLY reports on if the script or schedule was
successful. To quote straight from the h q event page Use this command to
check whether schedules were processed successfully.
I will not go into a rant about this, just learn from my mistake, query
event will NOT tell you if your backup was successful.
-Original Message-
From: Williams, Tim P {PBSG} [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, April 16, 2002 9:01 AM
To: [EMAIL PROTECTED]
Subject: Re: Daily Backup Report
we generally run a q event command ex=yes
you can use begindate begintime parms, etc
help q event
fYI
-Original Message-
From: Orin Rehorst [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, April 16, 2002 8:48 AM
To: [EMAIL PROTECTED]
Subject: Daily Backup Report
What are some good ways to get an automated daily report on which clients
were or were no backed up successfully?
TIA,
Regards,
Orin
Orin Rehorst
Port of Houston Authority
(Largest U.S. port in foreign tonnage)
e-mail: [EMAIL PROTECTED] mailto:[EMAIL PROTECTED]
Phone: (713)670-2443
Fax: (713)670-2457
TOPAS
Re: Daily Backup Report
I had thought I had put my two cents worth in on this subject but when searching the archives I failed to see my comments so here goes I agree with Lindsay Morris that simply monitoring the most recent incremental only gives you half the information you require. Even if the incremental reports no errors by whatever method you choose to monitor the success, it does not indicate events such as a entire filespace has not been backed up or only 1024 bytes was transferred when normally you expect a much larger amount. Based on this theory a combination of the success/failure tests need to be executed to get the whole picture this is not available via an 'out of the box' command. Itherefore found it necessary to write a script to obtain the info via SQL. The script (Pearl on Solaris) issues multiple SQL statements because without the OUTER join capability I was unable to combine all the info into a single statement. The SQL also assumes standardisation on the client schedule names, in this case all the incremental are INCR_nodename. The scripts has three major steps; Step 1 - a list of all clients is generated excluding the nodes with 'NOCHECK' in the CONTACTS fields. The NOCHECK allows me to exclude clients I know will report Incr backup failures, ie clients that no longer exists but there still is a need to keep the backups select substr(domain_name,1,3) as dummy, nodes.node_name, nodes.contact from nodes where upper(nodes.contact) is NULL or upper(nodes.contact) not like 'NOCHECK%' and NODETYPE= 'CLIENT' Step 2 reading each node perform the following for each.. - issue an SQL statement against the SUMMARY table extracting the sum of the failures, the bytes transferred, TSM idea of success the schedule run time based upon the schedule name. (select summary.schedule_name, sum(summary.failed) as failures, sum(summary.bytes) amount, summary.successful, sum(cast((summary.end_time-summary.start_time)minutes as decimal(18,0))) from summary where summary.schedule_name like 'INCR%' and summary.entity='$node_name' and cast((current_timestamp-summary.end_time)days as decimal(18,0)) 1 group by summary.successful, summary.schedule_name) - issue an SQL statement against the Filespace table to identify the greatest number of days that any filespace has not been backed up where the CONTACT field in the NODES table does not contact XFSn (where n is an FSID for the node). The XFS allows me to exclude filespace from generating errors when I know the Filespace will not longer be backed up, ie someone removed a drive from a NT client. select distinct substr(nodes.domain_name,1,3), max(cast((current_timestamp-filespaces.backup_end)days as decimal(18,0))) from filespaces, nodes where filespaces.node_name='$node_name' and filespace_id not in ($x_fsid)) group by nodes.domain_name Step 3 - generate HTML to display the results highlighting unacceptable results in an alternate colour (RED) and possible problems in yellow, ie less than 1K or greater than 5 Gb was transferred. obviously the could be convert to page, email whatever An example HOST NODE SUCCESSFUL BYTES FAILURES ELAPSEDFILESPACE (Minutes)DAYS -- HO_TSMNT128ARASHYDYES 1.74 M 13 0 HO_TSMNT128DFMS YES 7.02 M 3 6 0 HO_TSMNT128PDB3 YES 882.26 M 0 21 0 HO_TSMNT128PDCADS01 YES 4.36 G 0 1556 Peter Griffin Sydney Water --- This e-mail is solely for the use of the intended recipient and may contain information which is confidential or privileged. Unauthorised use of its contents is prohibited. If you have received this e-mail in error, please notify the sender immediately via e-mail and then delete the original e-mail. ---
Re: daily backup
Toni, You could try SELECT COUNT(LL_NAME) FROM BACKUPS - WHERE BACKUP_DATE'2001-12-08 00:01' This will give you a figure which may be reasonably accurate, but what about expiration? Another approach might be to pull out into a spreadsheet the number of files backed up from daily query actives to build up running totals Toni Banire [EMAIL PROTECTED] on 01/07/2002 03:39:03 PM Please respond to ADSM: Dist Stor Manager [EMAIL PROTECTED] To: [EMAIL PROTECTED] cc:(bcc: John Naylor/HAV/SSE) Subject: daily backup Hello I am trying to generate a list of monthly files backed up. I come up with the select statement below which seems to be taking it's time (not sure if it's legal). Does anyone know of a better way generate a summary of the total no. of files backed up TIA Toni select count(contents.file_name), count(volumes.volume_name) from contents,volumes where volumes.volume_name=contents.volume_name and volumes.last_write_date = '2002-01-04 00:00' ** The information in this E-Mail is confidential and may be legally privileged. It may not represent the views of Scottish and Southern Energy plc. It is intended solely for the addressees. Access to this E-Mail by anyone else is unauthorised. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. Any unauthorised recipient should advise the sender immediately of the error in transmission. Scottish Hydro-Electric, Southern Electric, SWALEC and S+S are trading names of the Scottish and Southern Energy Group. **
daily backup
Hello I am trying to generate a list of monthly files backed up. I come up with the select statement below which seems to be taking it's time (not sure if it's legal). Does anyone know of a better way generate a summary of the total no. of files backed up TIA Toni select count(contents.file_name), count(volumes.volume_name) from contents,volumes where volumes.volume_name=contents.volume_name and volumes.last_write_date = '2002-01-04 00:00'
Daily backup
Does someone know of a quick query to find out how much data was backed up last nite? as i am trying to forcast how much tapes are needed on a daily basis Thanks
Re: Daily backup
Does someone know of a quick query to find out how much data was backed up last nite? as i am trying to forcast how much tapes are needed on a daily basis Dwight's response about getting info from accounting records is the best, detailed info source for backup, archive, and HSM usage. You should also consider gathering such information over time. Simple ways to do that are to do Query STGpool each day to watch growth; and just count the number of scratch tapes you have left after each day, which is a good empirical way of gauging how many tapes are needed per day. Richard
Re: Daily backup
If you have a 3.7 server you can query the summary table. Assuming your overnight time is from 8 pm to 6 am, and that you don't have any long running backups from the previous night, this select would sum all the bytes sent to the server by backups -- /* */ /* macro file to select data from the summary table */ /* with tsm 3.7 */ /* */ select sum(bytes) as sum_bytes - from adsm.summary - where (date(end_time) = current date - 1 days - and time(end_time) = '20.00.00') - or (date(end_time) = current date) - and activity = 'BACKUP' (cut and paste to a file then enter macro filename in dsmadmc (the commandline admin).) -- -- Bill Colwell C. S. Draper Lab Cambridge, Ma. [EMAIL PROTECTED] -- In AB7914985D03D211B4BD00805FA7DE7C04E856A0@TOTOMB01, on 09/25/00 at 04:25 PM, "Selva, Perpetua" [EMAIL PROTECTED] said: Does someone know of a quick query to find out how much data was backed up last nite? as i am trying to forcast how much tapes are needed on a daily basis Thanks
