Re: amanda tape algorithm
On Thu, Jan 04, 2018 at 08:59:32PM -0500, Ned Danieley wrote: > On Thu, Jan 04, 2018 at 05:16:59PM -0700, Steven Backus wrote: > > Ned Danieleywrote: > > > > > ah, that makes sense. no, I haven't run 'amtapetype'; I just assumed that > > > the rated capacity would be accurate. I'll give it a try; in the meantime, > > > has anyone run 'amtapetype' on an LTO6 tape? I have an HP Ultrium 6 drive. > > > > I did and got: > > > > define tapetype LTO6 { > > comment "Created by amtapetype; compression disabled" > > length 2442954880 kbytes > > filemark 7456397 kbytes > > speed 154519 kps > > blocksize 32 kbytes > > } > > thanks. that fairly well matches what I was using > > define tapetype LTO6comp { > length 244352 kbytes > filemark 868 kbytes > speed 157129 kps > blocksize 2048 kbytes > } > > except for filemark; can anyone comment on that? looking at the tapetype > definitions on the zmanda wiki, it seems that most of the LTO entries have > zero kbytes for filemark... > It is the space (if any) left between files written to the tape. Suppose you write a continuous stream of data to a tape and you can write exactly 100GB. If no space was left between files you should be able to write 100 x 1GB files. When you try it (as amtapetype does) and you find you can only write 98 x 1GB files, 2GB was used by "filemarks". Divide 2GB/98 files and you have your filemark. I believe Amanda does consider filemarks in its determination of what will fit on a tape. jl -- Jon H. LaBadie j...@jgcomp.com 11226 South Shore Rd. (703) 787-0688 (H) Reston, VA 20190 (703) 935-6720 (C)
Re: amanda tape algorithm
On Thu, Jan 04, 2018 at 05:16:59PM -0700, Steven Backus wrote: > Ned Danieleywrote: > > > ah, that makes sense. no, I haven't run 'amtapetype'; I just assumed that > > the rated capacity would be accurate. I'll give it a try; in the meantime, > > has anyone run 'amtapetype' on an LTO6 tape? I have an HP Ultrium 6 drive. > > I did and got: > > define tapetype LTO6 { > comment "Created by amtapetype; compression disabled" > length 2442954880 kbytes > filemark 7456397 kbytes > speed 154519 kps > blocksize 32 kbytes > } thanks. that fairly well matches what I was using define tapetype LTO6comp { length 244352 kbytes filemark 868 kbytes speed 157129 kps blocksize 2048 kbytes } except for filemark; can anyone comment on that? looking at the tapetype definitions on the zmanda wiki, it seems that most of the LTO entries have zero kbytes for filemark... -- Ned Danieley (ned.danie...@duke.edu) Department of Biomedical Engineering Box 90281, Duke University Durham, NC 27708 (919) 660-5111 http://dilbert.com/strips/comic/2012-02-11/
Re: amanda tape algorithm
Ned Danieleywrote: > ah, that makes sense. no, I haven't run 'amtapetype'; I just assumed that > the rated capacity would be accurate. I'll give it a try; in the meantime, > has anyone run 'amtapetype' on an LTO6 tape? I have an HP Ultrium 6 drive. I did and got: define tapetype LTO6 { comment "Created by amtapetype; compression disabled" length 2442954880 kbytes filemark 7456397 kbytes speed 154519 kps blocksize 32 kbytes } Steve -- Steven J. BackusComputer Systems Manager University of Utah E-Mail: steven.bac...@utah.edu Genetic EpidemiologyAlternate: bac...@math.utah.edu 391 Chipeta Way -- Suite D Office: 801.587.9308 Salt Lake City, UT 84108-1266 http://www.math.utah.edu/~backus
Re: Serious tape error
On Thu, Jan 04, 2018 at 05:12:27PM -0500, Jon LaBadie wrote: > I use vtapes of 100GB and runtapes == 3. Most days only > one tape is used. About once every 2 or 3 weeks a second > tape is needed. Yesterday was one of those days. But > amanda chose the same vtape for the second tape and over > wrote the data it had just written to the "first" tape. > > >From the report: > > Tape Size (meg) 179621.1 171549.4 8071.8 > > Need multiple tapes. > > USAGE BY TAPE: > Label Time Size % DLEs Parts > DS1-215 1:45 179621M 175.42558 > > Only one tape listed, filled to 175%. > > Notes: > taper: tape DS1-215 kb 104856364 fm 34 [OK] > taper: tape DS1-215 kb 79075697 fm 24 [OK] > > Tape DS1-215 selected twice. Only the 24 files from the > later run are present on the vtape, not the 58 total files. > The first run was overwritten. > > > JLM I've saved all the log files I could find from this run. > What might you need to analyze this further? > > I looked at /var/log/amanda/server/DS1/taper. and it > appears to only contain information about the 2nd usage of > the tape. > I was wrong, there is info from both uses of the same tape in the taper logfile. Jon -- Jon H. LaBadie j...@jgcomp.com 11226 South Shore Rd. (703) 787-0688 (H) Reston, VA 20190 (703) 935-6720 (C)
Re: amanda tape algorithm
On Thu, Jan 04, 2018 at 04:58:13PM -0500, Ned Danieley wrote: > On Thu, Jan 04, 2018 at 09:43:35PM +, Debra S Baddorf wrote: > > > > > On Jan 4, 2018, at 11:59 AM, Ned Danieleywrote: > > > > > > > > > I'm using > > > > > > taperalgo largestfit > > > > > > which I assume means that amanda will write to tape the largest DLE > > > available. I have 'runtapes' set to 2, and occasionally I'll see amanda > > > move on to the second tape when the first tape is reporting around 80% > > > full. > > > I'm dumping almost 200 DLE, so it seems like there ought to be some DLEs > > > that would fit in the remaining 20%. is there any way to find out (after > > > the fact) what DLEs were in the queue? I've looked at the taper debug file > > > but it just seems to have a list of each DLE as it is written. > > > > > > I'm using LTO6 tapes, so I should have about 2.5 TB to work with. I know > > > about the ability to split DLEs, but I'm not ready to take that step. > > > > > > > Did you actually test the size of your own tapes? > > > > ah, that makes sense. no, I haven't run 'amtapetype'; I just assumed that > the rated capacity would be accurate. I'll give it a try; in the meantime, > has anyone run 'amtapetype' on an LTO6 tape? I have an HP Ultrium 6 drive. That 2.5TB is likely marketing-speak using decimal number. Switching to binary TB would immediately reduce the size by nearly 250GB (about 9% of the listed capacity). jl -- Jon H. LaBadie j...@jgcomp.com 11226 South Shore Rd. (703) 787-0688 (H) Reston, VA 20190 (703) 935-6720 (C)
Serious tape error
I use vtapes of 100GB and runtapes == 3. Most days only one tape is used. About once every 2 or 3 weeks a second tape is needed. Yesterday was one of those days. But amanda chose the same vtape for the second tape and over wrote the data it had just written to the "first" tape. >From the report: Tape Size (meg) 179621.1 171549.4 8071.8 Need multiple tapes. USAGE BY TAPE: Label Time Size % DLEs Parts DS1-215 1:45 179621M 175.42558 Only one tape listed, filled to 175%. Notes: taper: tape DS1-215 kb 104856364 fm 34 [OK] taper: tape DS1-215 kb 79075697 fm 24 [OK] Tape DS1-215 selected twice. Only the 24 files from the later run are present on the vtape, not the 58 total files. The first run was overwritten. JLM I've saved all the log files I could find from this run. What might you need to analyze this further? I looked at /var/log/amanda/server/DS1/taper. and it appears to only contain information about the 2nd usage of the tape. Jon -- Jon H. LaBadie j...@jgcomp.com 11226 South Shore Rd. (703) 787-0688 (H) Reston, VA 20190 (703) 935-6720 (C)
Re: amanda tape algorithm
On Thu, Jan 04, 2018 at 09:43:35PM +, Debra S Baddorf wrote: > > > On Jan 4, 2018, at 11:59 AM, Ned Danieleywrote: > > > > > > I'm using > > > > taperalgo largestfit > > > > which I assume means that amanda will write to tape the largest DLE > > available. I have 'runtapes' set to 2, and occasionally I'll see amanda > > move on to the second tape when the first tape is reporting around 80% full. > > I'm dumping almost 200 DLE, so it seems like there ought to be some DLEs > > that would fit in the remaining 20%. is there any way to find out (after > > the fact) what DLEs were in the queue? I've looked at the taper debug file > > but it just seems to have a list of each DLE as it is written. > > > > I'm using LTO6 tapes, so I should have about 2.5 TB to work with. I know > > about the ability to split DLEs, but I'm not ready to take that step. > > > > -- > > Ned Danieley (ned.danie...@duke.edu) > > Department of Biomedical Engineering > > Box 90281, Duke University > > Durham, NC 27708 (919) 660-5111 > > > > https://urldefense.proofpoint.com/v2/url?u=http-3A__dilbert.com_strips_comic_2012-2D02-2D11_=DwIGaQ=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc=seIgoLdvEn-8badm2JmCmh78EJ9F9-Hw-LulwGbFPZI=DGzKNKCqh_8C-Y1Y3Smdu29gOMu_5Y82AnrH-oAP-vg=dB2_g2FDMHahFgcGA1CJLdbMwaXZJjrvHGFYP_pmwGM= > > > > > Did you actually test the size of your own tapes? (There’s a proceedure > that does this …. help me > out with the name, guys?) > > Amanda, I believe, uses your number to decide how much space is left on the > tape. It’ll > pick the biggest DLE that ought to fit in that spot (“largestfit”). If > you’ve over-estimated > the tape size, then this DLE will NOT fit. At that point, amanda can’t > back up and try > a smaller DLE. It merely stops trying with that DLE (saves it for later) > and moves on to > the next tape. > At this point, the “largestfit” is probably a bigger DLE, since the > whole tape is available > for the second tape. > > Rinse and repeat. > > So — maybe tell amanda a smaller size for the tape? Then it’ll try a > smaller “last” DLE, > and it will actually fit. > > At least, this is my understanding of how she works. ah, that makes sense. no, I haven't run 'amtapetype'; I just assumed that the rated capacity would be accurate. I'll give it a try; in the meantime, has anyone run 'amtapetype' on an LTO6 tape? I have an HP Ultrium 6 drive. -- Ned Danieley (ned.danie...@duke.edu) Department of Biomedical Engineering Box 90281, Duke University Durham, NC 27708 (919) 660-5111 http://dilbert.com/strips/comic/2012-02-11/
Re: amanda tape algorithm
> On Jan 4, 2018, at 11:59 AM, Ned Danieleywrote: > > > I'm using > > taperalgo largestfit > > which I assume means that amanda will write to tape the largest DLE > available. I have 'runtapes' set to 2, and occasionally I'll see amanda > move on to the second tape when the first tape is reporting around 80% full. > I'm dumping almost 200 DLE, so it seems like there ought to be some DLEs > that would fit in the remaining 20%. is there any way to find out (after > the fact) what DLEs were in the queue? I've looked at the taper debug file > but it just seems to have a list of each DLE as it is written. > > I'm using LTO6 tapes, so I should have about 2.5 TB to work with. I know > about the ability to split DLEs, but I'm not ready to take that step. > > -- > Ned Danieley (ned.danie...@duke.edu) > Department of Biomedical Engineering > Box 90281, Duke University > Durham, NC 27708 (919) 660-5111 > > http://dilbert.com/strips/comic/2012-02-11/ Did you actually test the size of your own tapes? (There’s a proceedure that does this …. help me out with the name, guys?) Amanda, I believe, uses your number to decide how much space is left on the tape. It’ll pick the biggest DLE that ought to fit in that spot (“largestfit”). If you’ve over-estimated the tape size, then this DLE will NOT fit. At that point, amanda can’t back up and try a smaller DLE. It merely stops trying with that DLE (saves it for later) and moves on to the next tape. At this point, the “largestfit” is probably a bigger DLE, since the whole tape is available for the second tape. Rinse and repeat. So — maybe tell amanda a smaller size for the tape? Then it’ll try a smaller “last” DLE, and it will actually fit. At least, this is my understanding of how she works. Deb Baddorf Fermilab
amanda tape algorithm
I'm using taperalgo largestfit which I assume means that amanda will write to tape the largest DLE available. I have 'runtapes' set to 2, and occasionally I'll see amanda move on to the second tape when the first tape is reporting around 80% full. I'm dumping almost 200 DLE, so it seems like there ought to be some DLEs that would fit in the remaining 20%. is there any way to find out (after the fact) what DLEs were in the queue? I've looked at the taper debug file but it just seems to have a list of each DLE as it is written. I'm using LTO6 tapes, so I should have about 2.5 TB to work with. I know about the ability to split DLEs, but I'm not ready to take that step. -- Ned Danieley (ned.danie...@duke.edu) Department of Biomedical Engineering Box 90281, Duke University Durham, NC 27708 (919) 660-5111 http://dilbert.com/strips/comic/2012-02-11/