Eric,
Stefan,
Eric Siegerman wrote:
# On Wed, Dec 01, 2004 at 03:37:21PM -0500, Brian Cuttler wrote:
# > I unfortunately don't have the resources to create new holding
# > areas out of old disks [...]
# My "old disks" comment wasn't meant as a specific suggestion, but
# only as an example that it's ok to use disks that would be
# considered unacceptable for other purposes.
No, no, I understood that. What I meant was that I don't have any
old disks, nothing about 2 Gbyte anyway, no raid controllers sitting
around nor the resources to put any of this together. While I'm sure
there is a good how-to I simply don't have the resources to be adding
disks to servers I manage but don't own.
I'm not complaining, you deal with what you can within the framework
and it is a good idea, I just can't.
# > PCI to dual SCSI interface cards are available [...] but I've
# > been unable to rouse interest in its purchase.
# > [...]
# > I just know the
# > first time a user tries to save a large file and can't allocate
# > the space there is going to be a problem.
# And the first time someone needs a file restored and it wasn't
# backed up because the backups are too slow, it's going to be a
# problem.
# I think a lot of us have been there: management decrees (or in
# this case, at least understands) that some service is essential,
# but won't fund your providing of it. This is not a technical
# issue :-/
You are right on both counts. We will see what happens we we start
to use the new system with the 4TByte array for an amanda server with
virtual tapes (I wasn't joking yesterday, that is the project). How
long a tape cycle I can have backing up a .8TByte and a 2TByte array
to a 4 TByte disk I don't know but we will see.
# > I also need to remind myself that with
# > the data being stored in amanda work in chunksized pieces that the new
# > spindle need not be all that large in order to be useful.
# Well, *each* spindle need not be all that large, but remember
# that an entire DLE must be written to holding disk before taper
# even looks at it. Thus, the total amount of holding disk needs
# to be at least as large as the largest DLE (plus slop, expansion
# room, etc., of course), if you're to avoid direct-to-tape dumps.
# Even then that DLE will block any others from being dumped at the
# same time, since Amanda tries to avoid overcommitting the holding
# disk. So if you can't afford that big chunk of non-parallel
# operation, you need even more holding-disk space.
Yup, well aware. At this point, if in fact and EOT signal reschedules
a DLE with the dumper then having enough holding disk for the largest
DLE after compression will still be a huge time saver. Elapse time as
well as CPU time since its both the dump and the gzip.
# > Because dump does not cross partition boundries (at least no dump
# > that I've worked with) I feel comfortable using the xfsdump utility
# > to dump root without worring about encoding exceptions in the disklist
# > file.
# Agreed.
# > Is "dumped" really the right term there ? "Dumped with Tar"
# Strictly speaking, I guess not, but it's a common usage on the
# Amanda lists. Around here, "to dump" can mean any of:
# a. to back up a DLE
# b. to do the (mostly) client-side phase of (a) (as opposed to
# "to tape", which is the server-side phase), regardless of the
# actual backup program used
# c. to do (b) using the dump tool specifically (as opposed to
# gtar) (though in practice, I don't see this meaning much on
# these lists, if at all)
# This is historical; originally, Amanda only supported "dump", so
# (b) and (c) were equivalent. They stopped being equivalent ages
# ago, but the usage has stuck.
# > /dev/dsk/dks1d2s0 xfs 884945404 803357524 81587880 91 /usr5
# Oh, .88 *tera*bytes! That's a bit less drool-worthy :-) I didn't
# think too many people were up to petabytes yet...
Totally my fault, I wasn't sure if it was Terabytes or PTerabytes
(pteradactyl) and have since looked up Tera 10**12 and Peta 10**15.
Apparently there is no "ptera". Just waiting to hear that one of the
managers has order a raid in trilobyte capacity.
# For a dump (sense (b) :-) to holding disk, no. For a dump
# directly to tape, yes; the DLE is indeed redumped.
Ah, excellent. Expensive to perform (reperform dump/compress) but
this is exactly what one would hope amanda would do.
I send log.YYYYMMMDD and amdump.N files if I still have them, I'm
pretty sure that I can reproduce, even if its for the largest DLE,
simply to setting a lower size limit on the current work area.
# > taper: tape SAMAR02 kb 161967808 fm 9 writing file: No space left on device
# > 161161372 /usr5/joy
# How compressible is your data? This DLE is pretty darned close
# to filling a tape all on its own, before compression is taken
# into account. As if you didn't have enough problems...
Compression, even on the user data which I'd have thought to be of similar
content is suprisingly variable.
samar / 0 19728208 16073739 81.5 114:55 2331.1 20:49 12868.0
samar /usr1 1 197 26 13.2 1:12 0.3 0:01 25.7
samar /usr5/allengr 1 5870 929 15.8 1:19 11.7 0:01 898.6
samar /usr5/amanda 0 10550 3042 28.8 0:03 1041.9 0:01 2824.2
samar /usr5/bimal 0 20062320 3588605 17.9 43:10 1385.4 4:49 12430.0
samar /usr5/dtaylor 1 5360 876 16.3 1:50 7.9 0:01 862.8
samar /usr5/hxgao 1 8030 1187 14.8 2:51 7.0 0:01 1152.0
samar /usr5/joy 1 9260 1599 17.3 3:22 7.9 0:01 1526.3
samar /usr5/lalor 1 3460 539 15.6 0:46 11.7 0:01 532.4
samar /usr5/leith 0 21322290 14734699 69.1 120:10 2043.5 17:41 13885.9
samar /usr5/liw 1 186520 83830 44.9 6:14 224.3 0:05 16547.7
samar /usr5/ninggao 0 53788180 37111076 69.0 334:13 1850.6 41:18 14978.2
samar /usr5/skaur 0 40117288 29953080 74.7 265:28 1880.6 41:24 12058.5
samar /usr5/tapu 0 2745180 1725077 62.8 13:20 2156.1 2:28 11689.3
Stefan G. Weichinger wrote:
# Hm ... I assume you have set your holdingdisk-definition with using
# something like "use -1Gb" to let AMANDA use all the available space
# except that one Gb (in my example).
# It would be better to know how much holdingdisk you may use, but I
# understand the situation.
# I would try to use a pretty small chunksize (compared to the
# DLE-sizes), "taperalgo largestfit", and as much of the free space as
# you can get for the holdingdisk.
In Amanda.conf chunksize is set to 1 GByte which is small, relatively.
I will try to pick a good "use -NGb" value, would help to know how large
typical user files where. Will talk to someone in the department.
# BC> We always like to use dump rather than tar for root partitions. It
# BC> has been my understanding that tar is just a character archiver where
# BC> dump will understand the vendor specific special devices. Dump/restore
# BC> are the recommended way to preserve and restore a bootable partition.
# Ok, good point, no question.
# >> I know that you as the responsible admin tend to see things
# >> pessimistic. It is your job to do so, and I understand this perfectly
# >> as I am "the responsible admin" for several sites, too.
# >>
# >> We can't afford to be too OPTIMISTIC, do we? ;-)
# >>
# >> I also know from my view as an active member of the list that your
# >> installation is still in the process of development.
# BC> On that note I've got to tell you that one fo the other guys here
# BC> insists (rightly I should add) that the purpose of data backup is not
# BC> getting the data to tape, its getting the data back off of the tape.
# Errm, it seems as if I was mis-understood: What I tried to say was
# that your installation is in the stage of developing the proper
# schedule (AMANDA needs to run through the tapecycle at least once to
# get things right ...). I did not want to say that YOU fail to get
# your configuration working or something similar.
I have trouble in email. I was trying to agree with you. Sys admins keep
there eye on different things than end users, sometimes even to an
ultimate good the user wasn't able to focus on. We are paid to know (or
figure out) which things to be pesimistic about.
# I think that AMANDA will find out about the fact that there is no data
# for that DLE in the holdingdisk and THEN go back to the dumpers again.
I'll check the logs, see if I can't understand them well enough.
# >> I tend to use one "program" for the whole config as it is easier to
# >> configure (and wrap your head around).
# BC> It would be easier, but I question reliablity to restore root and can't
# BC> use dump on the raid, too big.
# As said before, OK. As I wrote this I did not know your partitioning
# so I thought that some overlapping DLEs could bite you with GNUTAR ..
You asked the right questions. Given the right warnings.
thank you,
Brian
---
Brian R Cuttler [EMAIL PROTECTED]
Computer Systems Support (v) 518 486-1697
Wadsworth Center (f) 518 473-6384
NYS Department of Health Help Desk 518 473-0773
