As a colleague of mine once put it: 'BANANA - Backups Are Not Archives, NOT ARCHIVES'. Practise this mantra. You should really think about rearchitecting.
That said, if your data isn't important to your firm and you're confident that you can restore from backups (you're making multiple copies, probably not using removable media but if you are you're doing full media verifies and sending them out the building to different offsite facilities in different directions using different carriers right?), and you're looking at this as an interim poor man's one-way HSM (you know your data access patterns and want to keep things around on disk as long as possible but manage your primary storage), then you could 'trust' your backup app, find the last successful backup date at the appropriate retlevel contains the file you want, and the script that you trigger when your filesystem utilisation gets to a certain point is free to remove anything before that date. No, I wouldn't do that either. The point is that you're either comfortable in your backup app or you're not, and if you're not you can't get out of doing anything other than reading the tape. If you trust your backup app but not bparchive you can get your backup app to dump a filelist. If that's millions of files and impacts on other uses of your backup plant then that's something you need to deal with. Alternatively, if you're not prepared to trust bparchive, or it's not timely enough for you (I've never had occasion to use it so can't comment on either point), you could always generate your own filelist that you use to take the regular backup, check the success of the backup and if it worked 'bless' the filelist. Then at some point in the future when you get into space pressure you can purge files in the blessed filelists. Be careful though to only bless fully successful backups (not partials - think locked files etc), but this really falls into the category of trusting the backup application, and hence becomes a circular argument. I'd be more inclined to test bparchive rather than engineer my own, but remember, BANANA. Separate solutions for separate problems: Stretched HA for hardware failure/Disaster Recovery CDP/Snapshots/Backups for User Error Archives for Data Retention -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Justin Piszcz Sent: 27 March 2007 06:17 To: Whelan, Patrick Cc: Veritas-bu@mailman.eng.auburn.edu Subject: Re: [Veritas-bu] Checking to see if millions of files are backed up? A nice idea; however, this is actually part of a much larger and complicated system in which certain files have to kept for certain retentions, both on disk and backed up to tape, think tape archival with different retention rates. If I were to change the entire architecture behind it, this may be a good solution and is something on my plate for the future. However, in the interim, I just need a solution to verify files have been backed up to tape and remove them if they are older than N days if we are running low on space on any particular server. On 3/26/07, Whelan, Patrick <[EMAIL PROTECTED]> wrote: > Why not have a script that runs a backup followed by an archive. Check > the error code of the backup, if is not 0 then don't run the archive. If > it is 0 then run the archive which will automatically delete the files > when it completes successfully. It will not delete antything if it fails > even with a 1. > > Regards, > > Patrick Whelan > > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Justin > Piszcz > Sent: 26 March 2007 22:36 > To: [EMAIL PROTECTED] > Cc: Veritas-bu@mailman.eng.auburn.edu > Subject: Re: [Veritas-bu] Checking to see if millions of files are > backed up? > > > The problem with that is two-fold: > > 1. We backup multiple copies of the data, therefore, the archive option > will not work. 2. What if a tape has an I/O error half way through the > archive process? Yikes. > > Justin. > > On 3/26/07, Bobby Williams <[EMAIL PROTECTED]> wrote: > > Why not set up an archive schedule? That way, the files can be > > archived and NetBackup will ensure that they are on tape before > > removing. > > > > > > > > > > Bobby Williams > > 2205 Peterson Drive > > Chattanooga, Tennessee 37421 > > 423-296-8200 > > > > -----Original Message----- > > From: [EMAIL PROTECTED] > > [mailto:[EMAIL PROTECTED] On Behalf Of Justin > > > Piszcz > > Sent: Monday, March 26, 2007 4:27 PM > > To: Veritas-bu@mailman.eng.auburn.edu > > Subject: [Veritas-bu] Checking to see if millions of files are backed > > up? > > > > If one is to create a script to ensure that the files on the > > filesystem are backed upon before removing them, what is the best > > data-store model for doing so? > > > > Obviously, if you have > 1,000,000 files in the catalog and you need > > to check each of those, you do not want to bplist -B -C -R 999999 > > /path/to/file/1.txt for each file. However, you do not want to grep > > "1" one_gigabyte_catalog.txt either as there is really too much > > overhead in either case. > > > > I have a few ideas that involves neither of these, but I was wondering > > > if anyone out there had already done something similar to this that > > was high performance? > > > > Justin. > > _______________________________________________ > > Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu > > http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu > > > > > _______________________________________________ > Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu > http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu > > > ************************************************************************ ************* > The message is intended for the named addressee only and may not be disclosed to or used by anyone else, nor may it be copied in any way. > > The contents of this message and its attachments are confidential and may also be subject to legal privilege. If you are not the named addressee and/or have received this message in error, please advise us by e-mailing [EMAIL PROTECTED] and delete the message and any attachments without retaining any copies. > > Internet communications are not secure and COLT does not accept responsibility for this message, its contents nor responsibility for any viruses. > > No contracts can be created or varied on behalf of COLT Telecommunications, its subsidiaries or affiliates ("COLT") and any other party by email Communications unless expressly agreed in writing with such other party. > > Please note that incoming emails will be automatically scanned to eliminate potential viruses and unsolicited promotional emails. For more information refer to www.colt.net or contact us on +44(0)20 7390 3900. > > _______________________________________________ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu -------------------------------------------------------- NOTICE: If received in error, please destroy and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error. _______________________________________________ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu