Re: Splitting files across tapes
Wow, that's interesting! I'll set up some tests myself. Thanks! -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of William Colwell Sent: Wednesday, June 08, 2005 5:59 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: Splitting files across tapes Hi Wanda, I have some real data which might mean something for your question. I am in the process of moving all the files on one server into collocation groups. I am copying tapes to sequential disk to do this. The sizes of full disk volumes vary a lot which might mean that an aggregate won't go across disk base sequential volumes. The data, see the Est. cap for the full volumes -- tsm: XXX>q vol stg=seqdisk2 Volume Name Storage Device Estimated Pct Volume Pool Name Class Name Capacity Util Status --- -- - - /seqdisk2/cgseq2_000 SEQDISK2SEQDISK2 2,016.0 100.0 Full /seqdisk2/cgseq2_001 SEQDISK2SEQDISK2 2,037.0 100.0 Full /seqdisk2/cgseq2_002 SEQDISK2SEQDISK2 2,016.9 100.0 Full /seqdisk2/cgseq2_003 SEQDISK2SEQDISK2 2,036.7 100.0 Full /seqdisk2/cgseq2_004 SEQDISK2SEQDISK2 2,043.4 100.0 Full /seqdisk2/cgseq2_005 SEQDISK2SEQDISK2 2,045.1 100.0 Full /seqdisk2/cgseq2_006 SEQDISK2SEQDISK2 2,035.0 100.0 Full /seqdisk2/cgseq2_007 SEQDISK2SEQDISK2 2,042.4 100.0 Full /seqdisk2/cgseq2_008 SEQDISK2SEQDISK2 2,008.0 100.0 Full /seqdisk2/cgseq2_009 SEQDISK2SEQDISK2 2,041.1 100.0 Full /seqdisk2/cgseq2_00A SEQDISK2SEQDISK2 2,034.3 100.0 Full /seqdisk2/cgseq2_00B SEQDISK2SEQDISK2 2,048.0 6.8 Filling /seqdisk2/cgseq2_00C SEQDISK2SEQDISK2 2,048.0 0.0 Empty /seqdisk2/cgseq2_00D SEQDISK2SEQDISK2 2,048.0 0.0 Empty /seqdisk2/cgseq2_01F SEQDISK2SEQDISK2 2,048.0 0.0 Empty /seqdisk2/cgseq2_020 SEQDISK2SEQDISK2 2,024.7 100.0 Full Hope this helps, Bill Colwell > -Original Message- > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] > On Behalf Of Prather, Wanda > Sent: Tuesday, June 07, 2005 10:07 AM > To: ADSM-L@VM.MARIST.EDU > Subject: Re: Splitting files across tapes > > Hi Richard, > > Thanks for responding; maybe this will give you something to > amuse your > brain over morning coffee. > > The reason for the question, mgmt here is considering going > to all-disk > backup (for onsite). > So our sequential "volumes" will be disk instead of tape. > > We occasionally have issues with mis-classified data ending up on a > tape, and the tape has to be pulled and destroyed. > No big deal with a tape. Big deal when the "volume" is a 1 TB raid > array! > > So the question comes, what is the likelihood that we would > contaminate > TWO 1 TB raid arrays with a split file? > > I think for sequential volumes, TSM doesn't know that the volume is > full, until it tries to write to it. > If there isn't space for the next "block", then it mounts a > scratch and > rewrites the block to a new tape, yes? > > So can I assume that the file would have to be larger than an > aggregate > (what is that, MOVESIZETHRESH?) in order to end up split > across 2 tapes? > > Thanks for lending brain power! > > W > > > > > > -Original Message- > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] > On Behalf Of > Richard Sims > Sent: Tuesday, June 07, 2005 9:55 AM > To: ADSM-L@VM.MARIST.EDU > Subject: Re: Splitting files across tapes > > > Hi, Wanda - > > I don't believe there is any rule, per se: it is just the case that > the drive finally reaches end-of-volume (EOV - TSM msg ANR8341I). > This results in the subsequent data being written in a spanned > Segment on a new volume. > > Richard Sims > > On Jun 7, 2005, at 9:36 AM, Prather, Wanda wrote: > > > Does anyone happen to know what rules TSM uses to decide when to > > split a > > backup file/aggregate across 2 tapes? > > Or can you point me to a document? > > > > (Management wants to know.) > >
Re: Splitting files across tapes
Hi Wanda, I have some real data which might mean something for your question. I am in the process of moving all the files on one server into collocation groups. I am copying tapes to sequential disk to do this. The sizes of full disk volumes vary a lot which might mean that an aggregate won't go across disk base sequential volumes. The data, see the Est. cap for the full volumes -- tsm: XXX>q vol stg=seqdisk2 Volume Name Storage Device Estimated Pct Volume Pool Name Class Name Capacity Util Status --- -- - - /seqdisk2/cgseq2_000 SEQDISK2SEQDISK2 2,016.0 100.0 Full /seqdisk2/cgseq2_001 SEQDISK2SEQDISK2 2,037.0 100.0 Full /seqdisk2/cgseq2_002 SEQDISK2SEQDISK2 2,016.9 100.0 Full /seqdisk2/cgseq2_003 SEQDISK2SEQDISK2 2,036.7 100.0 Full /seqdisk2/cgseq2_004 SEQDISK2SEQDISK2 2,043.4 100.0 Full /seqdisk2/cgseq2_005 SEQDISK2SEQDISK2 2,045.1 100.0 Full /seqdisk2/cgseq2_006 SEQDISK2SEQDISK2 2,035.0 100.0 Full /seqdisk2/cgseq2_007 SEQDISK2SEQDISK2 2,042.4 100.0 Full /seqdisk2/cgseq2_008 SEQDISK2SEQDISK2 2,008.0 100.0 Full /seqdisk2/cgseq2_009 SEQDISK2SEQDISK2 2,041.1 100.0 Full /seqdisk2/cgseq2_00A SEQDISK2SEQDISK2 2,034.3 100.0 Full /seqdisk2/cgseq2_00B SEQDISK2SEQDISK2 2,048.0 6.8 Filling /seqdisk2/cgseq2_00C SEQDISK2SEQDISK2 2,048.0 0.0 Empty /seqdisk2/cgseq2_00D SEQDISK2SEQDISK2 2,048.0 0.0 Empty /seqdisk2/cgseq2_01F SEQDISK2SEQDISK2 2,048.0 0.0 Empty /seqdisk2/cgseq2_020 SEQDISK2SEQDISK2 2,024.7 100.0 Full Hope this helps, Bill Colwell > -Original Message- > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] > On Behalf Of Prather, Wanda > Sent: Tuesday, June 07, 2005 10:07 AM > To: ADSM-L@VM.MARIST.EDU > Subject: Re: Splitting files across tapes > > Hi Richard, > > Thanks for responding; maybe this will give you something to > amuse your > brain over morning coffee. > > The reason for the question, mgmt here is considering going > to all-disk > backup (for onsite). > So our sequential "volumes" will be disk instead of tape. > > We occasionally have issues with mis-classified data ending up on a > tape, and the tape has to be pulled and destroyed. > No big deal with a tape. Big deal when the "volume" is a 1 TB raid > array! > > So the question comes, what is the likelihood that we would > contaminate > TWO 1 TB raid arrays with a split file? > > I think for sequential volumes, TSM doesn't know that the volume is > full, until it tries to write to it. > If there isn't space for the next "block", then it mounts a > scratch and > rewrites the block to a new tape, yes? > > So can I assume that the file would have to be larger than an > aggregate > (what is that, MOVESIZETHRESH?) in order to end up split > across 2 tapes? > > Thanks for lending brain power! > > W > > > > > > -----Original Message----- > From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] > On Behalf Of > Richard Sims > Sent: Tuesday, June 07, 2005 9:55 AM > To: ADSM-L@VM.MARIST.EDU > Subject: Re: Splitting files across tapes > > > Hi, Wanda - > > I don't believe there is any rule, per se: it is just the case that > the drive finally reaches end-of-volume (EOV - TSM msg ANR8341I). > This results in the subsequent data being written in a spanned > Segment on a new volume. > > Richard Sims > > On Jun 7, 2005, at 9:36 AM, Prather, Wanda wrote: > > > Does anyone happen to know what rules TSM uses to decide when to > > split a > > backup file/aggregate across 2 tapes? > > Or can you point me to a document? > > > > (Management wants to know.) > >
Re: Splitting files across tapes
Are you talking about a 1TB VOLUME, or several smaller volumes (say 10GB) on a 1TB array? H. Milton Johnson -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Prather, Wanda Sent: Tuesday, June 07, 2005 9:07 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: Splitting files across tapes Hi Richard, Thanks for responding; maybe this will give you something to amuse your brain over morning coffee. The reason for the question, mgmt here is considering going to all-disk backup (for onsite). So our sequential "volumes" will be disk instead of tape. We occasionally have issues with mis-classified data ending up on a tape, and the tape has to be pulled and destroyed. No big deal with a tape. Big deal when the "volume" is a 1 TB raid array! So the question comes, what is the likelihood that we would contaminate TWO 1 TB raid arrays with a split file? I think for sequential volumes, TSM doesn't know that the volume is full, until it tries to write to it. If there isn't space for the next "block", then it mounts a scratch and rewrites the block to a new tape, yes? So can I assume that the file would have to be larger than an aggregate (what is that, MOVESIZETHRESH?) in order to end up split across 2 tapes? Thanks for lending brain power! W -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Richard Sims Sent: Tuesday, June 07, 2005 9:55 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: Splitting files across tapes Hi, Wanda - I don't believe there is any rule, per se: it is just the case that the drive finally reaches end-of-volume (EOV - TSM msg ANR8341I). This results in the subsequent data being written in a spanned Segment on a new volume. Richard Sims On Jun 7, 2005, at 9:36 AM, Prather, Wanda wrote: > Does anyone happen to know what rules TSM uses to decide when to split > a backup file/aggregate across 2 tapes? > Or can you point me to a document? > > (Management wants to know.) >
Re: Splitting files across tapes
On Jun 7, 2005, at 10:06 AM, Prather, Wanda wrote: Hi Richard, Thanks for responding; maybe this will give you something to amuse your brain over morning coffee. The reason for the question, mgmt here is considering going to all- disk backup (for onsite). So our sequential "volumes" will be disk instead of tape. We occasionally have issues with mis-classified data ending up on a tape, and the tape has to be pulled and destroyed. No big deal with a tape. Big deal when the "volume" is a 1 TB raid array! I would recommend against destroying such RAID arrays - sounds expensive. :-)) So the question comes, what is the likelihood that we would contaminate TWO 1 TB raid arrays with a split file? I think for sequential volumes, TSM doesn't know that the volume is full, until it tries to write to it. If there isn't space for the next "block", then it mounts a scratch and rewrites the block to a new tape, yes? So can I assume that the file would have to be larger than an aggregate (what is that, MOVESIZETHRESH?) in order to end up split across 2 tapes? A magic word in your scenario is "sequential". This means you will be using File type devclass. TSM 5.3 allows you to make the volume size whatever you want, up to the limits of the file system. A judicious choice will make for a size which affords nice capacity while not being ponderous when you have to deal with a stray. And I should think that you could deal with the stray by doing an Expire or Delete Backup command from the client, perhaps followed by a MOVe Data ... RECONStruct=Yes on the containing server volume if there needs to be assured erasure of the goner. With this, may it be the case that concerns over volumes and spanning are unwarranted? File type devclass volumes may incite more predictive processing in TSM, given that there is the advantage of known size, whereas the actual length of a tape is not known. That is, TSM will probably know there is insufficient space for the next Aggregate, or clump of a file whose size is larger than an Aggregate, so as to go to a new volume. (Now, I don't know for certain, but have to suspect that the File volume's size is uniquely tracked in the db rather than simply taken from the devclass spec, given that the devclass MAXCAPacity can be updated at will. Thus, the size of volumes over time should be deterministic.) If there's definitive doc about all this, I have not yet encountered it. Someone out there may have more info. Thanks for lending brain power! If I were really smart I'd be charging for exposition of what I know, rather than working for a living. :-) Richard Sims
Re: Splitting files across tapes
Hi Richard, Thanks for responding; maybe this will give you something to amuse your brain over morning coffee. The reason for the question, mgmt here is considering going to all-disk backup (for onsite). So our sequential "volumes" will be disk instead of tape. We occasionally have issues with mis-classified data ending up on a tape, and the tape has to be pulled and destroyed. No big deal with a tape. Big deal when the "volume" is a 1 TB raid array! So the question comes, what is the likelihood that we would contaminate TWO 1 TB raid arrays with a split file? I think for sequential volumes, TSM doesn't know that the volume is full, until it tries to write to it. If there isn't space for the next "block", then it mounts a scratch and rewrites the block to a new tape, yes? So can I assume that the file would have to be larger than an aggregate (what is that, MOVESIZETHRESH?) in order to end up split across 2 tapes? Thanks for lending brain power! W -Original Message- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Richard Sims Sent: Tuesday, June 07, 2005 9:55 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: Splitting files across tapes Hi, Wanda - I don't believe there is any rule, per se: it is just the case that the drive finally reaches end-of-volume (EOV - TSM msg ANR8341I). This results in the subsequent data being written in a spanned Segment on a new volume. Richard Sims On Jun 7, 2005, at 9:36 AM, Prather, Wanda wrote: > Does anyone happen to know what rules TSM uses to decide when to > split a > backup file/aggregate across 2 tapes? > Or can you point me to a document? > > (Management wants to know.) >
Re: Splitting files across tapes
Hi, Wanda - I don't believe there is any rule, per se: it is just the case that the drive finally reaches end-of-volume (EOV - TSM msg ANR8341I). This results in the subsequent data being written in a spanned Segment on a new volume. Richard Sims On Jun 7, 2005, at 9:36 AM, Prather, Wanda wrote: Does anyone happen to know what rules TSM uses to decide when to split a backup file/aggregate across 2 tapes? Or can you point me to a document? (Management wants to know.)