Re: Order of dumps? (Want large ones first, I think)
On 13/04 2002 01:58 John R. Jackson wrote: Wouldn't it be simpler to just set a 'starttime' value for all the 'unimportant' disks so that all the dumpers will do the 'important' ones first? But, but, but .. if it doesn't involve a pipe, how can it be Unix??? :-) Yes, starttime would be another way to encourage Amanda to do things in a different order. Yes, I never really though of that. I guess I might try that, although I'm not sure I need it now; I'm just about to set up a larger holding this. I'd prefer to specify starttime as an offset from the time when amdump actually started, though, but I maybe that isn't possible? Apart from that, thanks for the original response, it was very helpful indeed. FRank John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED] - Toralf
Order of dumps? (Want large ones first, I think)
What exactly controls the order of which file systems are dumped by Amanda? Based on my brief experience, it looks large filesystems tend to be taped after the smaller ones included in the run. I guess this is good in the sense that it ensures *some* data is always backed up, but it's really bad if the data turns out to be only just too large to fit on the tape. I just had a 17Gb (compressed size) directory reaching EOT, and thus postponed to the next day, which leaves approximately 5Gb on a 25Gb tape. This is NOT what I call good utilisation... And the 5Gb is split across 4 filesystems. I think 3 of these and the 17Gb one would have been dumped successfully with a different order. Comments, anyone? -- - Toralf
Re: Order of dumps? (Want large ones first, I think)
On Fri, Apr 12, 2002 at 01:59:12PM +0200, Toralf Lund wrote: What exactly controls the order of which file systems are dumped by Amanda? Based on my brief experience, it looks large filesystems tend to be taped after the smaller ones included in the run. I've been wondering about how this is prioritized as well. In my case, I've got a flaky tape drive that could decide to reset the bus or just stop working at any time[1], so it would be very useful if I could tell it to write the latest backups of, say, the NFS server's /home and the partition that holds the company's primary database first before it starts doing the various / and /usr partitions that probably haven't changed since the last set of security updates. [1] I have a replacement, but the backup server does other things, too, so I haven't had an opportunity to take the box down and install the new drive yet.
Re: Order of dumps? (Want large ones first, I think)
Toralf wrote: What exactly controls the order of which file systems are dumped by Amanda? ... Atmospheric conditions over my house. :-) Seriously, this is a fairly complicated area. The schedule planner comes up with is sorted by the estimated time it will take to do the disk. That, in turn, is figured out based on the estimated size of the image and the average of the last three full or incremental backups (depending on what type of backup is done this time). The number of dumpers working for driver is based on your inparallel amanda.conf value. The default is 10, so we'll work with that for the moment. The first thing driver works on are images that will fit in the holding disk. The first three (LITTLE_DUMPERS in the driver source) dumpers work from the head of the sorted list, i.e. from the short end. The last seven (inparallel - LITTLE_DUMPERS, e.g. 10 - 3) dumpers work from the big end. After everything has been processed that can go through the holding disk, driver goes into direct to tape mode where a single dumper is involved writing the image directly to taper. These entries are taken from what remains of the list and come from the small end. Now, while multiple dumpers are working through the holding disk, any number of things may cause a candidate dump to be deferred. For instance, there might be five dumps from a single client at the head of the list. But if maxdumps is set to the default of 1, only the first entry will be started. The other four will have to wait until it finishes. However, driver keeps going through the list and may start a backup from some other client that is not strictly next in the list, but is able to be started. There are about a dozen restrictions that can cause a dump to be skipped and looked at later, such as maxdumps, a spindle contention, not enough holding disk space, no free dumpers, not enough bandwidth and so on. Note that if inparallel is set very small, things act much differently. If it is three or less, all of the dumpers will be working from the small end of the queue. If set to four, three will work from the small end and only one from the large end. And so on. The above all applies to 2.4.2p2 (and before). At 2.4.3, you have fine grain control over which dumpers are assigned to what via the new dumporder amanda.conf keyword. Dumpers may be assigned based on estimated size, time or speed and from either end of the queue. - Toralf Dave wrote: ... it would be very useful if I could tell it to write the latest backups of, say, the NFS server's /home and the partition that holds the company's primary database first ... The schedule is passed from planner to driver via a normal Unix shell pipeline in the amdump script. You could insert a filter of your own to fake the estimated time field down to zero to trick driver into doing important things first. Or, at 2.4.3, we could probably add a new dumporder flag to do things by priority, or something like that. John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED]
Re: Order of dumps? (Want large ones first, I think)
Dave wrote: ... it would be very useful if I could tell it to write the latest backups of, say, the NFS server's /home and the partition that holds the company's primary database first ... The schedule is passed from planner to driver via a normal Unix shell pipeline in the amdump script. You could insert a filter of your own to fake the estimated time field down to zero to trick driver into doing important things first. Or, at 2.4.3, we could probably add a new dumporder flag to do things by priority, or something like that. John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED] Wouldn't it be simpler to just set a 'starttime' value for all the 'unimportant' disks so that all the dumpers will do the 'important' ones first? FRank -- Frank Smith[EMAIL PROTECTED] Systems Administrator Voice: 512-374-4673 Hoover's Online Fax: 512-374-4501
Re: Order of dumps? (Want large ones first, I think)
Wouldn't it be simpler to just set a 'starttime' value for all the 'unimportant' disks so that all the dumpers will do the 'important' ones first? But, but, but .. if it doesn't involve a pipe, how can it be Unix??? :-) Yes, starttime would be another way to encourage Amanda to do things in a different order. FRank John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED]
