RE: followup on recommendations for tape libraries
Thanks for the links, Chris. Very interesting info indeed! I was basing my longevity concerns from the spec: http://ocz.com/consumer/vector-150-ssd/specifications of a very high performance consumer SSD. But these are consumer SSD's for sure (with MLC memory). Not enterprise grade or SLC. Regardless, at the link above, the spec says "Endurance: Rated for 50GB/day of host writes for 5 years under typical client workloads". To me, the implication is that writing more than 50 GB per day could shorten the life below 5 years. In an application where it is used as a staging disk for Amanda, _might_ your systems write much _more_ than 50GB per day, if your site were big enough to warrant use of an LTO tape in the first place? Anyway, we do not have any experience with SSD's in this kind of scenario. Yes, we do have some large EMC storage systems for our Oracle databases connected to our Sun servers via optical fiber, where there are indeed some SSD's used for intermediate storage before the read/write to the main hard disk drives. But, the systems are too new (less than a year old) for any kind of assessment of SSD life at this point. Z -Original Message- From: owner-amanda-us...@amanda.org [mailto:owner-amanda-us...@amanda.org] On Behalf Of Chris Hoogendyk Sent: Tuesday, March 11, 2014 2:04 PM To: AMANDA users Subject: Re: followup on recommendations for tape libraries I've had a couple of responses talking about the potentially short lifetime of SSDs based on how much is written to them. I have some comments on that, some links, and am still interested if anyone has any first hand experience with the scenario we are considering. First off, we expect to be looking at server class SSD devices, probably SLC based, but we haven't gotten specs on what options and prices we have with Supermicro yet. I would note that Sun/Oracle have for several years been configuring data storage systems with SSDs for the write-intent-logs for ZFS filesystems. It is fairly routine to spec a Sun/Oracle storage system with a fair number of SSDs alongside a large number of HDDs, all of them server class devices. The write-intent-log is the highest usage component of the storage system, and they wouldn't be using SSDs there if they were routinely failing in a year or two. This link is one of the better ones explaining calculations, usage scenarios, and life spans for SSDs: http://hblok.net/blog/posts/2013/03/03/concerns-about-ssd-reliability-debunked-again/ The above was based on work in the link in this one: http://beta.slashdot.org/story/182227 Some interesting comments, one from someone who cycled SLC NAND well over a million cycles without failure. This link from Toshiba is with regard to their consumer brand, but their comments on SSD Myth 5 also mention enterprise SSDs: http://www.toshiba.com/taec/news/media_resources/docs/SSDmyths.pdf Finally, for actual experimental data, these guys took a bunch of consumer grade SSDs and hammered them: http://techreport.com/review/24841/introducing-the-ssd-endurance-experiment They stood up pretty well, and that's consumer grade devices. So, I'm still interested in whether anyone has any real life experiences with SSDs as holding disks for Amanda. Would a pair work well and allow Amanda to drive an LTO6 (up to 160MB/s) in streaming mode somewhere near its rated speed? Also still open as a question is whether Amanda can stage backups to larger holding disk drives, and, when complete, move them to the SSD holding drives before writing them from there to the LTO6 tape. TIA Chris Hoogendyk On 3/10/14 6:20 PM, Syed Zaeem Hosain (syed.hos...@aeris.net) wrote: > 1. Continuous writing-reading-deleting to an SDD will wear them out _way_ > faster than you might like - these drives tend to be rated in > "so-many-GBytes-per-day" to get their typical rated life of 5 years or so. > That could be an expensive "solution" too quickly. Although, I do see > performance that is generally two to three times faster than SATA III disk > drives write speeds (non-RAID) for the tests I have done. > > 2. I think that using four large (2 to 4 TB) drives in RAID 0 (or RAID 10 if > you want drive reliability ... don't use RAID 5 since that will have parity > calculation performance hits) will get the performance you need. If the > drives are in a server (rather than an external USB3 or Thunderbolt box like > a Drobo or Synology). You should be able to sustain 200 to 400 MBytes/sec > such a to the RAID pretty readily, I'd think! > > 3. If you still plan on using SSD, OCZ Technology makes PCI Express SSD's > that run much faster than the typical SATA III interfaces (see > http://ocz.com/enterprise for info). But, the cost is high. > > 4. Finally, don't RAID your SSD drives - this usually disables TRIM support > in the drive, as I recall. > > Z > > P.S.: What is the streaming rate of the LTO you a
Re: can amanda auto-size DLE's?
Am 12.03.2014 17:52, schrieb Michael Stauffer: > Thanks Stefan! I'll take a look. > > How did this work for you in terms of daily, or almost daily, creating new > DLE's? I imagine it made for near-constant level 0 dumps? Maybe that was > what you needed anyway with lots of new data? I have to admit that I didn't use this for very long ... as you see from the date the script is from ~2010 ... back then I used it for weekly dumps of my video data but I was far from consequent ;-) Additionally my amanda tape server is now another physical machine so the created DLEs and includelists would have to be transferred somehow ... another todo left. I'd be happy to hear some feedback and maybe some scripting-improvement ... Stefan
Re: Why have my tapes 'shrunk' in size?
Don't know if its relevant, but I've got an LTO5/juke and it
dropped in both speed and capacity. I'm now trying to remember
if I had the host ESAS card replaced, or the I/O module in the
juke...
On Wed, Mar 12, 2014 at 12:28:33PM -0400, Jon LaBadie wrote:
> On Wed, Mar 12, 2014 at 02:38:48PM +, Dave Ewart wrote:
> > Hello,
> >
> > Four years ago I deployed a pair of Tandberg LTO-5 Ultrium (SAS) tape,
> > connected a Dell PowerEdge server via a Dell H200 SAS controller.
> >
> > At that time ran the amtapetype utility which produced this output:
> >
> > define tapetype Tandberg-LTO5 {
> > comment "Tandberg LTO5 1500/3000, produced by tapetype prog (hardware
> > compression off)"
> > length 1410 gbytes
> > filemark 0 kbytes
> > speed 125762 kps
> > }
> >
> > (That was created by an older version of AMANDA, which we were using at
> > the time: probably from Debian/Lenny, which was version 2.5.2p1, I
> > believe)
> >
> > These tapes are native 1.5TB and so that looks pretty reasonable. We've
> > never used these tapes to their fullest capacity and all was fun and
> > shiny until recently when the tapes reported "No space left on device".
> > However, the concerning thing is that the tapes reported 'full' at less
> > than what I was expecting as full capacity, just above 1.1TB in fact.
> > This means that our backup space 'growth', which I had been assuming was
> > only 75%/80% full is in fact at 100%!
> >
> > I re-ran the tapetype utility from our current AMANDA (version 2.6.1p2-3
> > from Debian/Squeeze) and it showed this:
> >
> > define tapetype unknown-tapetype {
> > comment "Created by amtapetype; compression disabled"
> > length 1148746080 kbytes
> > filemark 0 kbytes
> > speed 69815 kps
> > blocksize 32 kbytes
> > }
> >
> >
> > The length reported here is ~1.1TB which ties up with the "no space left
> > on device" message, but ...
> >
> > ... these are genuine LTO-5 (Tandberg brand) tapes - just like
> > http://img.misco.eu/Resources/images/Modules/InformationBlocks/1210/TAN/TAN-2/202175-tandberg-LTO-5-tape-cartridge-small.jpg
> > - and the second tapetype above was created using a previously-unused
> > tape and they really are 1.5TB native!
> >
> > What's going on? Why am I not getting to use the full capacity??
> >
>
> A guess only.
>
> I note the measured speed has dropped by 45%. Due to what I haven't a clue,
> but maybe some hardware change or cables or ???
>
> Perhaps your system's ability to feed the drive has dropped below the
> minimum needed to keep the drive streaming. In that case, the drive
> must "shoe-shine" and each restart costs a bit of tape.
>
> Jon
> --
> Jon H. LaBadie j...@jgcomp.com
> 11226 South Shore Rd. (703) 787-0688 (H)
> Reston, VA 20190 (609) 477-8330 (C)
---
Brian R Cuttler brian.cutt...@wadsworth.org
Computer Systems Support(v) 518 486-1697
Wadsworth Center(f) 518 473-6384
NYS Department of HealthHelp Desk 518 473-0773
Re: Why have my tapes 'shrunk' in size?
On Wed, Mar 12, 2014 at 01:11:03PM -0400, Brian Cuttler wrote:
>
> Don't know if its relevant, but I've got an LTO5/juke and it
> dropped in both speed and capacity. I'm now trying to remember
> if I had the host ESAS card replaced, or the I/O module in the
> juke...
I'd found that reseating the esas cable helped temporarily, I
guess resetting the driver config. In testing I was able to somehow
rule out the tape drive itself.
YRWBCD.
Brian
> On Wed, Mar 12, 2014 at 12:28:33PM -0400, Jon LaBadie wrote:
> > On Wed, Mar 12, 2014 at 02:38:48PM +, Dave Ewart wrote:
> > > Hello,
> > >
> > > Four years ago I deployed a pair of Tandberg LTO-5 Ultrium (SAS) tape,
> > > connected a Dell PowerEdge server via a Dell H200 SAS controller.
> > >
> > > At that time ran the amtapetype utility which produced this output:
> > >
> > > define tapetype Tandberg-LTO5 {
> > > comment "Tandberg LTO5 1500/3000, produced by tapetype prog (hardware
> > > compression off)"
> > > length 1410 gbytes
> > > filemark 0 kbytes
> > > speed 125762 kps
> > > }
> > >
> > > (That was created by an older version of AMANDA, which we were using at
> > > the time: probably from Debian/Lenny, which was version 2.5.2p1, I
> > > believe)
> > >
> > > These tapes are native 1.5TB and so that looks pretty reasonable. We've
> > > never used these tapes to their fullest capacity and all was fun and
> > > shiny until recently when the tapes reported "No space left on device".
> > > However, the concerning thing is that the tapes reported 'full' at less
> > > than what I was expecting as full capacity, just above 1.1TB in fact.
> > > This means that our backup space 'growth', which I had been assuming was
> > > only 75%/80% full is in fact at 100%!
> > >
> > > I re-ran the tapetype utility from our current AMANDA (version 2.6.1p2-3
> > > from Debian/Squeeze) and it showed this:
> > >
> > > define tapetype unknown-tapetype {
> > > comment "Created by amtapetype; compression disabled"
> > > length 1148746080 kbytes
> > > filemark 0 kbytes
> > > speed 69815 kps
> > > blocksize 32 kbytes
> > > }
> > >
> > >
> > > The length reported here is ~1.1TB which ties up with the "no space left
> > > on device" message, but ...
> > >
> > > ... these are genuine LTO-5 (Tandberg brand) tapes - just like
> > > http://img.misco.eu/Resources/images/Modules/InformationBlocks/1210/TAN/TAN-2/202175-tandberg-LTO-5-tape-cartridge-small.jpg
> > > - and the second tapetype above was created using a previously-unused
> > > tape and they really are 1.5TB native!
> > >
> > > What's going on? Why am I not getting to use the full capacity??
> > >
> >
> > A guess only.
> >
> > I note the measured speed has dropped by 45%. Due to what I haven't a clue,
> > but maybe some hardware change or cables or ???
> >
> > Perhaps your system's ability to feed the drive has dropped below the
> > minimum needed to keep the drive streaming. In that case, the drive
> > must "shoe-shine" and each restart costs a bit of tape.
> >
> > Jon
> > --
> > Jon H. LaBadie j...@jgcomp.com
> > 11226 South Shore Rd. (703) 787-0688 (H)
> > Reston, VA 20190 (609) 477-8330 (C)
> ---
>Brian R Cuttler brian.cutt...@wadsworth.org
>Computer Systems Support(v) 518 486-1697
>Wadsworth Center(f) 518 473-6384
>NYS Department of HealthHelp Desk 518 473-0773
>
---
Brian R Cuttler brian.cutt...@wadsworth.org
Computer Systems Support(v) 518 486-1697
Wadsworth Center(f) 518 473-6384
NYS Department of HealthHelp Desk 518 473-0773
Re: Why have my tapes 'shrunk' in size?
On Wed, Mar 12, 2014 at 04:55:30PM +, Dave Ewart wrote: > On Wednesday, 12.03.2014 at 12:28 -0400, Jon LaBadie wrote: > > > A guess only. > > > > I note the measured speed has dropped by 45%. Due to what I haven't a > > clue, but maybe some hardware change or cables or ??? > > Hmmm, yeah: I noticed that too, just after I'd posted. It's the same > controller card as always and the same cables, so it's either a driver > issue with the controller (OS has been upgraded once or twice) or a > fault with the controller or damage to the cable, maybe. All those things, plus, is the controller dedicated to the tape drive? If not, maybe other devices on the same controller are competing for bandwidth. > > > Perhaps your system's ability to feed the drive has dropped below the > > minimum needed to keep the drive streaming. In that case, the drive > > must "shoe-shine" and each restart costs a bit of tape. > > Would this behaviour be noticeable just by looking at the drive in > operation, do you think, seeing some kind of stop and start? Backups > normally run during 'out of hours', but I can run a job while physically > sat in the room with the drive... You could run amtapetype :) If you can hear the drive when it starts and stops (and possibly a different sound when it rewinds) you might be able to hear it regularly changing when it should be constant except when it reaches the end of a track and must reverse to tape the next track. -- Jon H. LaBadie j...@jgcomp.com 11226 South Shore Rd. (703) 787-0688 (H) Reston, VA 20190 (609) 477-8330 (C)
Re: Why have my tapes 'shrunk' in size?
On Wednesday, 12.03.2014 at 12:28 -0400, Jon LaBadie wrote: > A guess only. > > I note the measured speed has dropped by 45%. Due to what I haven't a > clue, but maybe some hardware change or cables or ??? Hmmm, yeah: I noticed that too, just after I'd posted. It's the same controller card as always and the same cables, so it's either a driver issue with the controller (OS has been upgraded once or twice) or a fault with the controller or damage to the cable, maybe. > Perhaps your system's ability to feed the drive has dropped below the > minimum needed to keep the drive streaming. In that case, the drive > must "shoe-shine" and each restart costs a bit of tape. Would this behaviour be noticeable just by looking at the drive in operation, do you think, seeing some kind of stop and start? Backups normally run during 'out of hours', but I can run a job while physically sat in the room with the drive... Dave. -- Dave Ewart da...@ceu.ox.ac.uk Computing Manager, Cancer Epidemiology Unit University of Oxford N 51.7516, W 1.2152 signature.asc Description: Digital signature
Re: can amanda auto-size DLE's?
Thanks Stefan! I'll take a look. How did this work for you in terms of daily, or almost daily, creating new DLE's? I imagine it made for near-constant level 0 dumps? Maybe that was what you needed anyway with lots of new data? -M On Wed, Mar 12, 2014 at 5:48 AM, Stefan G. Weichinger wrote: > Am 05.03.2014 14:10, schrieb Stefan G. Weichinger: > > > Aside from this I back then had some other scripts that generated > > include-lists resulting in chunks of <= X GB (smaller than one tape) ... > > I wanted to dump the videos in my mythtv-config and had the problem of > > very dynamic data in there ;-) > > > > So the goal was to re-create dynamic include-lists for DLEs everyday (or > > even at the actual time of amdump). It worked mostly. I would have to > > dig that up again. > > Digged that up and put it on github: > > https://github.com/stefangweichinger/am_dyn_dles > > feel free to use or improve. > > Stefan > > >
Re: Why have my tapes 'shrunk' in size?
On Wed, Mar 12, 2014 at 02:38:48PM +, Dave Ewart wrote:
> Hello,
>
> Four years ago I deployed a pair of Tandberg LTO-5 Ultrium (SAS) tape,
> connected a Dell PowerEdge server via a Dell H200 SAS controller.
>
> At that time ran the amtapetype utility which produced this output:
>
> define tapetype Tandberg-LTO5 {
> comment "Tandberg LTO5 1500/3000, produced by tapetype prog (hardware
> compression off)"
> length 1410 gbytes
> filemark 0 kbytes
> speed 125762 kps
> }
>
> (That was created by an older version of AMANDA, which we were using at
> the time: probably from Debian/Lenny, which was version 2.5.2p1, I
> believe)
>
> These tapes are native 1.5TB and so that looks pretty reasonable. We've
> never used these tapes to their fullest capacity and all was fun and
> shiny until recently when the tapes reported "No space left on device".
> However, the concerning thing is that the tapes reported 'full' at less
> than what I was expecting as full capacity, just above 1.1TB in fact.
> This means that our backup space 'growth', which I had been assuming was
> only 75%/80% full is in fact at 100%!
>
> I re-ran the tapetype utility from our current AMANDA (version 2.6.1p2-3
> from Debian/Squeeze) and it showed this:
>
> define tapetype unknown-tapetype {
> comment "Created by amtapetype; compression disabled"
> length 1148746080 kbytes
> filemark 0 kbytes
> speed 69815 kps
> blocksize 32 kbytes
> }
>
>
> The length reported here is ~1.1TB which ties up with the "no space left
> on device" message, but ...
>
> ... these are genuine LTO-5 (Tandberg brand) tapes - just like
> http://img.misco.eu/Resources/images/Modules/InformationBlocks/1210/TAN/TAN-2/202175-tandberg-LTO-5-tape-cartridge-small.jpg
> - and the second tapetype above was created using a previously-unused
> tape and they really are 1.5TB native!
>
> What's going on? Why am I not getting to use the full capacity??
>
A guess only.
I note the measured speed has dropped by 45%. Due to what I haven't a clue,
but maybe some hardware change or cables or ???
Perhaps your system's ability to feed the drive has dropped below the
minimum needed to keep the drive streaming. In that case, the drive
must "shoe-shine" and each restart costs a bit of tape.
Jon
--
Jon H. LaBadie j...@jgcomp.com
11226 South Shore Rd. (703) 787-0688 (H)
Reston, VA 20190 (609) 477-8330 (C)
Why have my tapes 'shrunk' in size?
Hello,
Four years ago I deployed a pair of Tandberg LTO-5 Ultrium (SAS) tape,
connected a Dell PowerEdge server via a Dell H200 SAS controller.
At that time ran the amtapetype utility which produced this output:
define tapetype Tandberg-LTO5 {
comment "Tandberg LTO5 1500/3000, produced by tapetype prog (hardware
compression off)"
length 1410 gbytes
filemark 0 kbytes
speed 125762 kps
}
(That was created by an older version of AMANDA, which we were using at
the time: probably from Debian/Lenny, which was version 2.5.2p1, I
believe)
These tapes are native 1.5TB and so that looks pretty reasonable. We've
never used these tapes to their fullest capacity and all was fun and
shiny until recently when the tapes reported "No space left on device".
However, the concerning thing is that the tapes reported 'full' at less
than what I was expecting as full capacity, just above 1.1TB in fact.
This means that our backup space 'growth', which I had been assuming was
only 75%/80% full is in fact at 100%!
I re-ran the tapetype utility from our current AMANDA (version 2.6.1p2-3
from Debian/Squeeze) and it showed this:
define tapetype unknown-tapetype {
comment "Created by amtapetype; compression disabled"
length 1148746080 kbytes
filemark 0 kbytes
speed 69815 kps
blocksize 32 kbytes
}
The length reported here is ~1.1TB which ties up with the "no space left
on device" message, but ...
... these are genuine LTO-5 (Tandberg brand) tapes - just like
http://img.misco.eu/Resources/images/Modules/InformationBlocks/1210/TAN/TAN-2/202175-tandberg-LTO-5-tape-cartridge-small.jpg
- and the second tapetype above was created using a previously-unused
tape and they really are 1.5TB native!
What's going on? Why am I not getting to use the full capacity??
Cheers,
Dave.
--
Dave Ewart
da...@ceu.ox.ac.uk
Computing Manager, Cancer Epidemiology Unit
University of Oxford
N 51.7516, W 1.2152
signature.asc
Description: Digital signature
Re: can amanda auto-size DLE's?
Am 05.03.2014 14:10, schrieb Stefan G. Weichinger: > Aside from this I back then had some other scripts that generated > include-lists resulting in chunks of <= X GB (smaller than one tape) ... > I wanted to dump the videos in my mythtv-config and had the problem of > very dynamic data in there ;-) > > So the goal was to re-create dynamic include-lists for DLEs everyday (or > even at the actual time of amdump). It worked mostly. I would have to > dig that up again. Digged that up and put it on github: https://github.com/stefangweichinger/am_dyn_dles feel free to use or improve. Stefan
