Re: hacker's digest 1104
Paul, On Wed, Dec 13, 2006 at 10:37:16PM +0100, Paul Bijnens wrote: > Brian Cuttler schreef: > >An interesting but confusing (for me) read, but the wiki page > >was very helpful. > > > >http://wiki.zmanda.com/index.php/Splitting_dumps_across_tapes > > > >We are will likely be implementing this shortly, I have a number > >of jukeboxes with large DLE or many small DLE where I'm only > >partially filling multiple tapes, I have a related question. > > > >Is there a value in trying to match the tape_slitsize to the chunksize ? > > They are completely unrelated, and can be used orthogonal. > > "chunksize" is a holdingdisk parameter mainly to overcome filesize > limitations on certain filesystems. One additional benefit is that > when using many holdingdisk areas, one dump one holdingdisk can be > spread over all these disks, even if each of them would be too small > to hold one complete image. Understood, had just been wondering if there was any benefit to having them match, ie, is there any efficiency to having each be 1 Gig for instance, does it save anything when part of the process is to assemble chunks and the other part of the process is to take the data streem and chunk it for output. I would suppose that even if the chunksizes where the same the boundries would be different. Making the point moot, but I thought I'd ask. > "tapesplit_size" is the size of chunk on tape. When writing such > a chunk to tape, and the tape bumps into EOT, the next tape is loaded, > and the complete chunk is rewritten again. Compare this to the > alternative to restart the complete dumpimage on that tape: it > avoids wasting too much tape (and time writing it). Yes, we have at this time several jukebox enabled systems with many DLE and some of the output tapes end up with perhaps 60 % utilization. I would actually end up with a 4 tape run rather than a 5, useful not only for elapse time but cost savings for media. > Note also that tapesplit_size is a dumptype parameter: you may > specify which DLE's should be split, and which not (e.g. not > splitting the DLE's that contain the amanda software makes > restores easier, when that disk itself was lost). While reading the wiki (again, very nicely written) I took note of that I realized the implications immediately. Allow chunking of the large user drives, don't allow it for the root partition, that sort of thing. Prior to reading the wiki I hadn't realized that and had (incorrectly) assumed it was a (binary) taper switch. Thank you for helping to clear this up. Brian > -- > Paul Bijnens, XplanationTel +32 16 397.511 > Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 > http://www.xplanation.com/ email: [EMAIL PROTECTED] > *** > * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, F6, * > * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * > * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * > * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * > * kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ...* > * ... "Are you sure?" ... YES ... Phew ... I'm out * > *** --- Brian R Cuttler [EMAIL PROTECTED] Computer Systems Support(v) 518 486-1697 Wadsworth Center(f) 518 473-6384 NYS Department of HealthHelp Desk 518 473-0773
RE: Upgrade failure?
I wanted to give an update on this problem I have with 2.5.1p2. To try to help fix the problems with 2.5.1, I went back to version 2.5.0b1, updated dump on the client machine (which included some other rpms), and I was able to get a full dump from the client with no problem. (That's with client and server at the same version). I then moved both client and server to 2.5.1p2, and when running amdump again, it failed after only getting 390m from the client. (It's about 90G of data). So, I'm back to where I was before: 2.5.1p2 doesn't work correctly, and 2.5.0b1 times out when doing gtar from certain directories from a different client. I'm going to stick with 2.5.0b1 and just exclude the other client for now.
RE: Using no-reuse with vtapes
Jean-Louis, thanks for the patch! I applied it when you sent it but had to wait to see if the system would cycle through past the "no-reuse" tape. It did so successfully and all is working fine now! Thanks again! Regards, Gordon Mills -Original Message- From: Jean-Louis Martineau [mailto:[EMAIL PROTECTED] Sent: Monday, November 20, 2006 2:20 PM To: [EMAIL PROTECTED] Cc: amanda-users@amanda.org Subject: Re: Using no-reuse with vtapes I attached a bad version of the patch, Try this one instead. Jean-Louis Jean-Louis Martineau wrote: > > Changing tapecycle will not fix your problem because the chg-disk > changer script use it to set it's LASTSLOT. > > The use of chg-disk with 'no-reuse' is incompatible, but it could be > easily fix (attached patch): > - chg-disk must set its LASTSLOT from a changer configuration file. > > After that, you will have 2 possibilities: > set LASTSLOT to 22, create the slot directory and label it. > or > set LASTSLOT to 21 and tapecycle to 20. > > > Try the attached patch, the changer configuration filename is your > setting of changerfile with a ".conf" appended. > Add a "LASTSLOT=22" to the changer configuration file. > > Jean-Louis > > > Gordon J. Mills III wrote: >> I am having a problem using the no-reuse feature with vtapes. I have >> a tape that contains files from a machine that is going to be >> decommissioned. I want to save the last full backup of that machine. >> So I put no-reuse next to that tape in the tapelist file. The problem >> is that the next time the tape cycle comes around to that tape it >> acts like there is no available tape. It cycles through all the tapes >> and says "(expecting a new tape)" at the end. >> So the question is: how can I use the "no-reuse" feature? It seems >> that amanda has to have tapecycle number of tapes even if dumpcycle >> is set to a number lower than that. The relative section of my >> amanda.conf is below. >> This is on 2.5.0b1. >> >> dumpcycle 10 days # the number of days in the normal dump cycle >> runspercycle 10 >> tapecycle 21 tapes # the number of tapes in rotation >> >> bumpsize 10 MB # minimum savings (threshold) to bump level 1 >> -> 2 >> bumpdays 2 # minimum days at each level >> bumpmult 1.5# threshold = bumpsize * (level-1)**bumpmult >> >> runtapes 1 # explained in WHATS.NEW >> tpchanger "chg-disk" # the tape-changer glue script, see >> TAPE.CHANGERS changerfile "/etc/amanda/DailySet1/changer" >> tapedev "file:/amandatapes/DailySet1/" # Linux @ tuck, important: >> norewinding >> # tapedev "/dev/nrst8" # or use the (no-rewind!) tape device >> directly rawtapedev "file:/amandatapes/DailySet1/" >> >> tapetype HARD-DISK # what kind of tape it is (see tapetypes >> below) >> labelstr "^TAPE[0-9][0-9]*$"# label constraint regex: all tapes must >> match >> >> Regards, >> Gordon >> >> >> > > -- > -- > >
new backup server
After a long time, I've finally gotten approval to proceed with building a centralized backup server. I've had the configuration and pricing set for a while, but need to check and confirm everything before proceeding. I'm interested in whether anyone on the list has any experience or comments on my choice of tape changer, or comments on issues related to how it is configured and potential modes of upgrading (adding another tape drive, adding another changer, etc.) I have a number of hand-me-down Sun Enterprise 250 servers. So, that is my base. Dual UltraSPARC processor, 2G memory. I'm going to add a dual channel Ultra-SCSI PCI card, and configure it with two 73G internal drives for system and mirror/live upgrade and two 300G internal drives for Amanda to use for scratch/disk backup space (all Seagate Cheetah). The installation is Solaris 9. I'm already playing with it on a single "old" 36G drive. The tape changer I'm looking at is the Sony StorStation AIT Library LIB-162/A4. It is a carousel rather than a robot. It holds 16 tapes (3.2TB native, anybody's guess compressed) and can have a second tape drive added. It is significantly less expensive than the expandable robot systems I was looking at. Also, in the "expandable" systems, adding the expansions was very expensive. So, I'm not sure what advantage I would have been buying in the original box. With the Sony system, I would just buy another and stack them (even stick it on a separate SCSI bus). Amanda seems to be flexible enough that I can just configure it to work with both. Seems to me like a no-brainer, as though it's really the marketing folks who are pushing the more expensive robotic systems. Price has been a significant issue in moving this project forward. I will get an APC Smart UPS 1500 to connect the whole setup. I have a handful of other E250 servers that will be backed up by this system over a 100MB network. We will initially not build any sort of backside network for backing up. If it is important, we may later. The servers have been added to over the years by various research labs in the department and have external disk cabinets with various disks from 18G, 73G, 146G to 300G. I may end up with a basic Amanda configuration and some additional archival configurations for different labs. The total disk space used appears to be in the range of a few hundred gigabytes at present, although it fluctuates a lot and is constantly growing, usually gradually but with periodic surges. At present we have DAT3 drives on almost all our servers and run tapes almost every night using a tapewriter script that I wrote to handle snapshots, tunnel through ssh across the network if necessary, and use a configuration file. We'll phase that out when the amanda system gets running. I've been looking at all options for backup software, including bacula and amanda. At the moment, I just feel more comfortable with amanda for a variety of reasons. I would appreciate any feedback and comments on my configuration and choice of tape changer. TIA --- Chris Hoogendyk - O__ Systems Administrator c/ /'_ --- Biology & Geology Departments (*) \(*) -- 140 Morrill Science Center ~~ - University of Massachusetts, Amherst <[EMAIL PROTECTED]> --- Erdös 4
Re: new backup server
On Thu, 14 Dec 2006, Chris Hoogendyk wrote: I'm interested in whether anyone on the list has any experience or comments on my choice of tape changer, or comments on issues related to how it is configured and potential modes of upgrading (adding another tape drive, adding another changer, etc.) You didn't say which AIT drive is going in your AIT changer. Here we have gone from AIT1 to AIT2 to AIT3. Just yesterday I ordered a new library with LTO3. What soured us on the AIT line is that AIT4 is not backwards read compatible with any earlier AIT drives. In other words if I went to AIT4 I would not be able to use it to even read any of our large existing collection of AIT1, 2, and 3 tapes. So at this point it no longer matters to us whether we stay with the AIT line or not. Depending on which AIT drive you're choosing, this may or may not be a concern for you. Given that it is for us, we took this as our opportunity to move to LTO, which is at least an industry standard with multiple vendors supplying drives. (Sony can take as long as they want to come out with the next generation of AIT, since they're the only supplier. We waited what seemed like forever for AIT3 to finally come out. Way past its expected release date. And AIT4 was promised all along to be backwards read compatible, but that was dropped at the very last minute.) The tape changer I'm looking at is the Sony StorStation AIT Library LIB-162/A4. It is a carousel rather than a robot. It holds 16 tapes (3.2TB native, anybody's guess compressed) and can have a second tape drive added. It is significantly less expensive than the expandable robot systems I was looking at. Also, in the "expandable" systems, adding the expansions was very expensive. Not sure what systems you looked at, but I was surprised to find that in the Qualstar RLS series of expandable libraries, adding more tape slots is not a big money proposition. The LTO library I ordered starts with 12 slots and is expandable up to 44 slots in increments of 8, for $1000 (list) per increment. As an .edu you may do better than that on price. Also with AIT slots being smaller, they might come cheaper, too. I don't know. Hope this helps in some way. -Mitch
Re: new backup server
Mitch Collinsworth wrote: > On Thu, 14 Dec 2006, Chris Hoogendyk wrote: > >> I'm interested in whether anyone on the list has any experience or >> comments on my choice of tape changer, or comments on issues related to >> how it is configured and potential modes of upgrading (adding another >> tape drive, adding another changer, etc.) > > You didn't say which AIT drive is going in your AIT changer. Here we > have gone from AIT1 to AIT2 to AIT3. Just yesterday I ordered a new > library with LTO3. What soured us on the AIT line is that AIT4 is not > backwards read compatible with any earlier AIT drives. In other words > if I went to AIT4 I would not be able to use it to even read any of our > large existing collection of AIT1, 2, and 3 tapes. So at this point it > no longer matters to us whether we stay with the AIT line or not. > Depending on which AIT drive you're choosing, this may or may not be a > concern for you. AIT5 recently came out, and it can read AIT3 and AIT4 tapes, and has 400GB native capacity. In addition, it supports WORM tapes, for the folks that have requirements for unmodifiable backups (you can write a tape and append to a tape, but not overwrite or erase). Unless you frequently have a need to read old tapes, keeping a an old drive or two around just to read old tapes isn't a big deal. The advantage of not switching formats is that you can just replace the drives and the tapes to upgrade a library to higher capacity. > > Given that it is for us, we took this as our opportunity to move to LTO, > which is at least an industry standard with multiple vendors supplying > drives. (Sony can take as long as they want to come out with the next > generation of AIT, since they're the only supplier. We waited what > seemed like forever for AIT3 to finally come out. Way past its expected > release date. And AIT4 was promised all along to be backwards read > compatible, but that was dropped at the very last minute.) > >> The tape changer I'm looking at is the Sony StorStation AIT Library >> LIB-162/A4. It is a carousel rather than a robot. It holds 16 tapes >> (3.2TB native, anybody's guess compressed) and can have a second tape >> drive added. It is significantly less expensive than the expandable >> robot systems I was looking at. Also, in the "expandable" systems, >> adding the expansions was very expensive. > > Not sure what systems you looked at, but I was surprised to find that > in the Qualstar RLS series of expandable libraries, adding more tape > slots is not a big money proposition. The LTO library I ordered starts > with 12 slots and is expandable up to 44 slots in increments of 8, for > $1000 (list) per increment. As an .edu you may do better than that on > price. Also with AIT slots being smaller, they might come cheaper, too. > I don't know. I'll second the recommendation for Qualstar libraries. Frank > > Hope this helps in some way. > > -Mitch -- Frank Smith [EMAIL PROTECTED] Sr. Systems Administrator Voice: 512-374-4673 Hoover's Online Fax: 512-374-4501
Re: Problem after years of backing up successful: "data timeout"
All, I am replying to my own email cause I fixed it ... however, I am not sure whether this is a bug! I tried a lot of things to find the problem ... until I did a "mount" to display the mounted devices. I just wanted to see whether a USB HD (which I use for the ODD backup of the system and this thing is BIG) is mounted. I saw that a WINDOWS workstation which I had mounted (to check the size of a partition and the used filespace) was still mounted ... (the amanda host is a samba host too). Further I saw lots of thesew messages in the logs ... smb_add_request: request [3d6a9e00, mid=1646] timed out ... I umounted the windows workstation partition and everything is fine now ... What I find strange: * If I have a drive mounted from a UNIX box and the the Unix workstation is now turned off it doesnt matter, this does NOT affect amanda. * The windows workstation in question is NOT part of the disklist (I do not backup any windows boxen) and I do not have any windows based servers. * all machines backed up by Amanda are UNIX based. So how can a windows workstation that has one of its partition mounted and is now turned of affect Amanda although the workstation is not part of the daily backup routine jobst On Wed, Dec 13, 2006 at 11:48:19AM +1100, Jobst Schmalenbach ([EMAIL PROTECTED]) wrote: > > Hi. > > All over sudden I get a "data timeout", e.g.: > > ... > ... > adminserver /var lev 0 FAILED [data timeout] > adminserver /home lev 3 FAILED [data timeout] > ... > ... > > I havent changed a thing on the machines involved for > a while (I work on the basic principle "if aint broken, dont fix") > > > Now I have searched the net and found a few pointers, > but all those I found do not help: > > * I do not use compression (I use hardware compression) > > * its NOT backing up another host, its the LOCAL machine >backing up itself, and I have never had any problems with this > > * the dtimeout already is 1800 > > * since the last reboot of the machine I have done >lots of successfull backups. > > * I havent installed anything new for a while on that machine. > > * I havent hanged anything (xinet, hosts.allow, .amandahost, directories, >amanda, whatever). > > > Looking at the "sendbackup..debug I can see the following message: > > ... > index tee cannot write [Broken pipe] > ... > > The index file for the latest ones are missing, that is correct. > > > I checked all the permissions on the "amanda" based directories, and they > look fine to me. > > > What can be the cause for the errors popping up? > > > > jobst > > > > > > -- > Learn the rules so you know how to break them properly. -F1 > > | |0| | Jobst Schmalenbach, [EMAIL PROTECTED], Technical Director > | | |0| Barrett Consulting Group P/L & The Meditation Room P/L > |0|0|0| +61 3 9532 7677, POBox 277, Caulfield South, 3162, Australia -- My Carpenter has a 1956 VW Beetle. He still can go to any place in Australia, use any Oil, spark plugs, pertol, tires, wiper blades, etc available today with a car that old. If only software would be like that. | |0| | Jobst Schmalenbach, [EMAIL PROTECTED], Technical Director | | |0| Barrett Consulting Group P/L & The Meditation Room P/L |0|0|0| +61 3 9532 7677, POBox 277, Caulfield South, 3162, Australia
