Re: Occupancy comparison script
Thanks to everyone for the help. I haven't seen exactly what I'm looking for yet, so I'm inventing this wheel. I'll post back what I come up with. -Original Message- From: Ilja G. Coolen [mailto:[EMAIL PROTECTED]] Sent: Wednesday, July 24, 2002 2:28 AM To: [EMAIL PROTECTED] Subject: Re: Occupancy comparison script We run this server-script occasionally. Just copy and paste these lines from the mail into the command line admin screen. Hit the enter key after the last line to get it into the script too. You can then run the server-script using tsm:run totocd. No parameters required. Perform audit lic first to get an accurate measurement. The script shows the total number of files occupying TSM. It also shows the amount of GB's grouped by backup, backup-copy, archive and archive-copy type of data. On the end it also shows the total amount. start clipping def script totocd desc='run totocd' upd script totocd '/* ---*/' upd script totocd '/* Script Name: totocd*/' upd script totocd '/* Parameter: none */' upd script totocd '/* Author: [EMAIL PROTECTED]*/' upd script totocd '/* Example: run totocd*/' upd script totocd '/* RUN AUDIT LIC FIRST TO GET AN ACC. RESULT */' upd script totocd '/* ---*/' upd script totocd 'select sum(num_files) as File Count from occupancy' upd script totocd 'select -' upd script totocd 'cast(sum(backup_mb/1024) as decimal(5,1)) as Backup (GB), -' upd script totocd 'cast(sum(backup_copy_mb/1024) as decimal(5,1)) as BackupCp (GB), -' upd script totocd 'cast(sum(archive_mb/1024) as decimal(5,1)) as Archive (GB), -' upd script totocd 'cast(sum(archive_copy_mb/1024) as decimal(5,1)) as ArchCp (GB), -' upd script totocd 'cast(sum(total_mb/1024) as decimal(5,1)) as Total (GB) -' upd script totocd 'from auditocc' end clipping This is an example of the results you should get. It shows our current status from one of our TSM 4.2 servers. tsm: ABPTSM1run totocd File Count --- 46316390 Backup (GB) BackupCp (GB) Archive (GB) ArchCp (GB) Total (GB) --- - --- -- 2477.0 182.0 3852.0 140.0 6697.0 ANR1462I RUN: Command script TOTOCD completed successfully. Use it if you like. We run it once in a while to see how much data we have in TSM. Management loves figures.. Ilja G. Coolen ABP / USZO CIS / BS / TB / Storage Management Telefoon: +31(0)45 579 7938 Fax: +31(0)45 579 3990 Email: [EMAIL PROTECTED] Centrale Mailbox: Centrale Mailbox - BS Storage (eumbx05) - Everybody has a photographic memory, some just don't have film. -
Re: Occupancy comparison script
We run this server-script occasionally. Just copy and paste these lines from the mail into the command line admin screen. Hit the enter key after the last line to get it into the script too. You can then run the server-script using tsm:run totocd. No parameters required. Perform audit lic first to get an accurate measurement. The script shows the total number of files occupying TSM. It also shows the amount of GB's grouped by backup, backup-copy, archive and archive-copy type of data. On the end it also shows the total amount. start clipping def script totocd desc='run totocd' upd script totocd '/* ---*/' upd script totocd '/* Script Name: totocd*/' upd script totocd '/* Parameter: none */' upd script totocd '/* Author: [EMAIL PROTECTED]*/' upd script totocd '/* Example: run totocd*/' upd script totocd '/* RUN AUDIT LIC FIRST TO GET AN ACC. RESULT */' upd script totocd '/* ---*/' upd script totocd 'select sum(num_files) as File Count from occupancy' upd script totocd 'select -' upd script totocd 'cast(sum(backup_mb/1024) as decimal(5,1)) as Backup (GB), -' upd script totocd 'cast(sum(backup_copy_mb/1024) as decimal(5,1)) as BackupCp (GB), -' upd script totocd 'cast(sum(archive_mb/1024) as decimal(5,1)) as Archive (GB), -' upd script totocd 'cast(sum(archive_copy_mb/1024) as decimal(5,1)) as ArchCp (GB), -' upd script totocd 'cast(sum(total_mb/1024) as decimal(5,1)) as Total (GB) -' upd script totocd 'from auditocc' end clipping This is an example of the results you should get. It shows our current status from one of our TSM 4.2 servers. tsm: ABPTSM1run totocd File Count --- 46316390 Backup (GB) BackupCp (GB) Archive (GB) ArchCp (GB) Total (GB) --- - --- -- 2477.0 182.0 3852.0 140.0 6697.0 ANR1462I RUN: Command script TOTOCD completed successfully. Use it if you like. We run it once in a while to see how much data we have in TSM. Management loves figures.. Ilja G. Coolen ABP / USZO CIS / BS / TB / Storage Management Telefoon: +31(0)45 579 7938 Fax: +31(0)45 579 3990 Email: [EMAIL PROTECTED] Centrale Mailbox: Centrale Mailbox - BS Storage (eumbx05) - Everybody has a photographic memory, some just don't have film. -
Re: Occupancy comparison script
I need to restate that - I'm looking for a way to produce output that lines up primary and copy storage pools occupancy side by side -- I just need a sanity check on the numbers before pulling tapes from the vault for a DR exercise. -Original Message- From: Mike Crawford [mailto:[EMAIL PROTECTED]] Sent: Monday, July 22, 2002 11:05 PM To: [EMAIL PROTECTED] Subject: Re: Occupancy comparison script Simple: backup stgpool primarypool copypool preview=yes Check the activity log, it will show which volumes would need to be mounted for a backup, if any. Mike. Jolliff, Dale wrote: I know someone has already invented this wheel... I need to create a script to compare occupancy of primary sequential pools to copypools to verify a complete stgpool backup. Anyone got one handy that won't bring a server to it's knees?
Re: Occupancy comparison script
This goes top down instead of side by side ... but it's quick and easy! select stgpool_name, sum(physical_mb)/1024 as physical_GB from occupancy- group by stgpool_name order by stgpool_name -Original Message- From: Jolliff, Dale [mailto:[EMAIL PROTECTED]] Sent: Tuesday, July 23, 2002 7:37 AM To: [EMAIL PROTECTED] Subject: Re: Occupancy comparison script I need to restate that - I'm looking for a way to produce output that lines up primary and copy storage pools occupancy side by side -- I just need a sanity check on the numbers before pulling tapes from the vault for a DR exercise. -Original Message- From: Mike Crawford [mailto:[EMAIL PROTECTED]] Sent: Monday, July 22, 2002 11:05 PM To: [EMAIL PROTECTED] Subject: Re: Occupancy comparison script Simple: backup stgpool primarypool copypool preview=yes Check the activity log, it will show which volumes would need to be mounted for a backup, if any. Mike. Jolliff, Dale wrote: I know someone has already invented this wheel... I need to create a script to compare occupancy of primary sequential pools to copypools to verify a complete stgpool backup. Anyone got one handy that won't bring a server to it's knees?
Re: Occupancy comparison script
On 23-Jul-02 Prather, Wanda wrote: This goes top down instead of side by side ... but it's quick and easy! select stgpool_name, sum(physical_mb)/1024 as physical_GB from occupancy- group by stgpool_name order by stgpool_name HmmI think it's not quick and dirty...;-) : ANR2963W This SQL query may produce a very large result table, or may require a significant amount of time to compute. And yes, it take some time, but the result is ok. Cheers, Henk ten Have.
Re: Occupancy comparison script
Copy pools do not belong to a primary pool. The destination for the primary pool data is determined by the command that backs it up: BACKUP STGPOOL primary copypool And in theory, you could send it to one copy pool on Monday, and a different one on Tuesday... -Original Message- From: Jolliff, Dale [mailto:[EMAIL PROTECTED]] Sent: Tuesday, July 23, 2002 9:27 AM To: [EMAIL PROTECTED] Subject: Re: Occupancy comparison script I don't think there is a quick answer; that's why I was hoping someone had already invented this wheel. Unless I'm missing some pretty big clues (which is very eminently possible) the report would require determining storage pool type to create a MB/File count Primary Storage Pool column and a Copy Storage Pool column, then some way to determine which Copypool belonged to which primary pool. -Original Message- From: Henk ten Have [mailto:[EMAIL PROTECTED]] Sent: Tuesday, July 23, 2002 8:15 AM To: [EMAIL PROTECTED] Subject: Re: Occupancy comparison script On 23-Jul-02 Prather, Wanda wrote: This goes top down instead of side by side ... but it's quick and easy! select stgpool_name, sum(physical_mb)/1024 as physical_GB from occupancy- group by stgpool_name order by stgpool_name HmmI think it's not quick and dirty...;-) : ANR2963W This SQL query may produce a very large result table, or may require a significant amount of time to compute. And yes, it take some time, but the result is ok. Cheers, Henk ten Have.
Re: Occupancy comparison script
I'm new here, and planned to lurk for a while, but I can perhaps help a
bit. I am NOT a TSM guru, but I'm a serious student of the game.
I just did a DR exercise, and found that my DB backup didn't seem to
exactly match my COPYPOOL tapes. When I restored a server, I found that a
very few small and frequently changed files were not restored, apparently
because they were not on the COPYPOOL tapes, to judge from the TSM logs.
After the test, I checked my admin schedules to make sure all the backups
were getting done before the copy to the COPYPOOL tapes, and such, and
was unable to find anything wrong that needed fixing.
Yesterday, I wrote a Python script on my AIX TSM server that does a q
occupancy and does some simple analysis on the results. I have the
(probably) normal pools: BACKUPPOOL, TAPEPOOL, and COPYPOOL. My script
simply verifies that for every filesystem, the number of files in
COPYPOOL equals the sum of the number of files in BACKUPPOOL and TAPEPOOL.
It's a simple script that has had about 10 minutes of testing, so it's
not ready for prime time, but it does seem to do a quick sanity check,
like you were asking about. You may not know Python and/or have it
available, but you can probably divine what this is doing, and write a
Perl (or other) script to do the same thing. Here is it, for better or
worse. No, I'm not a Python guru either.
-- start of script
#!/usr/local/bin/python
import os
import string
# Global data
# One entry for each filesystem. Key is node concatenated with
# filesystem. Value is an array of three integers, the number of
# files in backuppool, tapepool, and copypool, respectively.
info = {}
def do_line(f):
global info
# print f[0], f[2], f[4], f[5]
key = f[0] + ' ' + f[2]
if not info.has_key(key):
info[key] = [0, 0, 0]
a = info[key]
count = string.atoi(string.replace(f[5], ',', ''))
if f[4] == 'BACKUPPOOL':
a[0] = count
elif f[4] == 'TAPEPOOL':
a[1] = count
elif f[4] == 'COPYPOOL':
a[2] = count
# For each filespace, it should be true that a[0] + a[1] = a[2]
# where a is the value of info[filespace]
def check_data():
global info
for key in info.keys():
a = info[key]
# print key, a
if a[2] a[0] + a[1]:
print 'ERROR:', key, a
# begin execution here
in_header = 1
#WARNING. My email client is probably wrapping the following line:
for line in os.popen('dsmadmc -id=admin -pa=admin -tabdelimited q occ
2/dev/null').readlines():
# print '/', line,
if in_header:
if line.find('ANS8000I') -1:
in_header = 0
continue
line = string.replace(line, 'SYSTEM OBJECT', 'SYSTEM_OBJECT')
f = line.split()
if len(f) == 0 or f[0] == 'ANS8002I':
continue
do_line(f)
check_data()
-- end of script
-Original Message-
From: Jolliff, Dale [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Date: Tue, 23 Jul 2002 06:32:41 -0500
Subject: Re: Occupancy comparison script
We are just trying to do sanity checks prior to shipping offsite
copypools
off for a DR exercise.
-Original Message-
From: Seay, Paul [mailto:[EMAIL PROTECTED]]
Sent: Monday, July 22, 2002 11:19 PM
To: [EMAIL PROTECTED]
Subject: Re: Occupancy comparison script
They often do not exactly match because of the agregates and other
factors
related to expiration, but they are usually close.
May I ask what the problem is you are trying to solve that you think
the
successful completion of a backup stgpool command does not?
Paul D. Seay, Jr.
Technical Specialist
Naptheon, INC
757-688-8180
-Original Message-
From: Jolliff, Dale [mailto:[EMAIL PROTECTED]]
Sent: Monday, July 22, 2002 3:02 PM
To: [EMAIL PROTECTED]
Subject: Occupancy comparison script
I know someone has already invented this wheel...
I need to create a script to compare occupancy of primary sequential
pools
to copypools to verify a complete stgpool backup.
Anyone got one handy that won't bring a server to it's knees?
Re: Occupancy comparison script
They often do not exactly match because of the agregates and other factors related to expiration, but they are usually close. May I ask what the problem is you are trying to solve that you think the successful completion of a backup stgpool command does not? Paul D. Seay, Jr. Technical Specialist Naptheon, INC 757-688-8180 -Original Message- From: Jolliff, Dale [mailto:[EMAIL PROTECTED]] Sent: Monday, July 22, 2002 3:02 PM To: [EMAIL PROTECTED] Subject: Occupancy comparison script I know someone has already invented this wheel... I need to create a script to compare occupancy of primary sequential pools to copypools to verify a complete stgpool backup. Anyone got one handy that won't bring a server to it's knees?
Re: Occupancy comparison script
Simple: backup stgpool primarypool copypool preview=yes Check the activity log, it will show which volumes would need to be mounted for a backup, if any. Mike. Jolliff, Dale wrote: I know someone has already invented this wheel... I need to create a script to compare occupancy of primary sequential pools to copypools to verify a complete stgpool backup. Anyone got one handy that won't bring a server to it's knees?
Re: Occupancy comparison script
We are just trying to do sanity checks prior to shipping offsite copypools off for a DR exercise. -Original Message- From: Seay, Paul [mailto:[EMAIL PROTECTED]] Sent: Monday, July 22, 2002 11:19 PM To: [EMAIL PROTECTED] Subject: Re: Occupancy comparison script They often do not exactly match because of the agregates and other factors related to expiration, but they are usually close. May I ask what the problem is you are trying to solve that you think the successful completion of a backup stgpool command does not? Paul D. Seay, Jr. Technical Specialist Naptheon, INC 757-688-8180 -Original Message- From: Jolliff, Dale [mailto:[EMAIL PROTECTED]] Sent: Monday, July 22, 2002 3:02 PM To: [EMAIL PROTECTED] Subject: Occupancy comparison script I know someone has already invented this wheel... I need to create a script to compare occupancy of primary sequential pools to copypools to verify a complete stgpool backup. Anyone got one handy that won't bring a server to it's knees?
Re: Occupancy comparison script
These may also be useful /* SQL Script: */ /* */ /* backup_volume_last_24hours.sql*/ /* Date Description*/ /* 2002-06-10 PDS Created*/ /* Create Report of total MBs per each session */ select entity as Node Name , cast(bytes/1024/1024 as decimal(10,3)) as Total MB , substr(cast(start_time as char(26)),1,19) as Date/Time from summary where start_time=current_timestamp - 1 day and activity='BACKUP' /* Create Report of total MBs and length of backup for each node */ select entity as Node Name , cast(sum(bytes/1024/1024) as decimal(10,3)) as Total MB, substr(cast(min(start_time) as char(26)),1,19) as Date/Time , cast(substr(cast(max(end_time)-min(start_time) as char(20)),3,8) as char(8)) as Lengthfrom summary where start_time=current_timestamp - 22 hours and activity='BACKUP' group by entity /* Create Report of total backed up*/ select sum(cast(bytes/1024/1024/1024 as decimal(6,3))) Total GB Backup from summary where start_time=current_timestamp - 1 day and activity='BACKUP' /* Create Report of Storage Pool Copies */ select entity as Storage Pool , cast(bytes/1024/1024 as decimal(10,3)) asTotal MB , ' ' as , substr(cast(start_time as char(26)),1,19) as Date/Time , cast(substr(cast(end_time-start_time as char(20)),3,8) as char(8)) as Lengthfrom summary where start_time=current_timestamp - 22 hours and activity='STGPOOL BACKUP' order by 3, entity /* Create Summary Report for each Storage Pool */ select entity as Storage Pool , cast(sum(bytes/1024/1024) as decimal(10,3)) asTotal MB , ' ' as , substr(cast(min(start_time) as char(26)),1,19) as Start Date/Time , substr(cast(max(end_time) as char(26)),1,19) as End Date/Time from summary where start_time=current_timestamp - 22 hours and activity='STGPOOL BACKUP' group by entity /* Create Report of Total Storage Pool GB copied */ select sum(cast(bytes/1024/1024/1024 as decimal(6,3))) Total STG Pool GB Backup from summary where start_time=current_timestamp - 1 day and activity='STGPOOL BACKUP' Paul D. Seay, Jr. Technical Specialist Naptheon, INC 757-688-8180 -Original Message- From: Jolliff, Dale [mailto:[EMAIL PROTECTED]] Sent: Tuesday, July 23, 2002 7:37 AM To: [EMAIL PROTECTED] Subject: Re: Occupancy comparison script I need to restate that - I'm looking for a way to produce output that lines up primary and copy storage pools occupancy side by side -- I just need a sanity check on the numbers before pulling tapes from the vault for a DR exercise. -Original Message- From: Mike Crawford [mailto:[EMAIL PROTECTED]] Sent: Monday, July 22, 2002 11:05 PM To: [EMAIL PROTECTED] Subject: Re: Occupancy comparison script Simple: backup stgpool primarypool copypool preview=yes Check the activity log, it will show which volumes would need to be mounted for a backup, if any. Mike. Jolliff, Dale wrote: I know someone has already invented this wheel... I need to create a script to compare occupancy of primary sequential pools to copypools to verify a complete stgpool backup. Anyone got one handy that won't bring a server to it's knees?
Re: Occupancy comparison script
Dale, The best way to assure a copy pool matches its primary is perform a backup stg pool command and see if it completes successfully. I run a search of the activity log daily that picks out the message of primary pool volumes skipped because they are not available (marked unavailable). You can see the message in a backup stg command by marking a tape that has some data on it not in the copy pool as unavailable and running the backup stg command. I just do not remember the message id. We also use the TEC interface to send the message to us when many conditions occur. Here is the select I run daily. select substr(cast(date_time as char(26)),1,16) as date time, message as Message from actlog where date_time current_timestamp - 24 hours - 5 minutes and (msgno = 8359 or msgno = 8302 or msgno = 1412 or msgno = 1229 or msgno = 1402 or msgno = 1440 or msgno = 8873) Paul D. Seay, Jr. Technical Specialist Naptheon, INC 757-688-8180 -Original Message- From: Jolliff, Dale [mailto:[EMAIL PROTECTED]] Sent: Tuesday, July 23, 2002 7:33 AM To: [EMAIL PROTECTED] Subject: Re: Occupancy comparison script We are just trying to do sanity checks prior to shipping offsite copypools off for a DR exercise. -Original Message- From: Seay, Paul [mailto:[EMAIL PROTECTED]] Sent: Monday, July 22, 2002 11:19 PM To: [EMAIL PROTECTED] Subject: Re: Occupancy comparison script They often do not exactly match because of the agregates and other factors related to expiration, but they are usually close. May I ask what the problem is you are trying to solve that you think the successful completion of a backup stgpool command does not? Paul D. Seay, Jr. Technical Specialist Naptheon, INC 757-688-8180 -Original Message- From: Jolliff, Dale [mailto:[EMAIL PROTECTED]] Sent: Monday, July 22, 2002 3:02 PM To: [EMAIL PROTECTED] Subject: Occupancy comparison script I know someone has already invented this wheel... I need to create a script to compare occupancy of primary sequential pools to copypools to verify a complete stgpool backup. Anyone got one handy that won't bring a server to it's knees?
