Problem backing up: "no backup size line"
Hi I never had problems with amanada, been using it for more than 15 years (i think) Had to delete one machine that had only two purposes: behind DMZ firewall and amanda host. Due to changes of my setup I did not need the firewall anymore so I moved the amanda tape host to another existing machine. Had trouble ever since. Now I get the following error: "no backup size line", in the log it looks like this FAIL dumper 192.168.0.1 /boot 20171024230002 1 [no backup size line] sendbackup: start [192.168.0.1:/boot level 1] sendbackup: info BACKUP=/bin/tar sendbackup: info RECOVER_CMD=/bin/tar -xpGf - ... sendbackup: info end sendbackup: error [no backup size line] PARTIAL chunker 192.168.0.1 /boot 20171024230002 1 [sec 0.174 kb 11 kps 246.4] Looking at this https://wiki.zmanda.com/index.php/Sendbackup_error:_no_backup_size_line I have a few options: /var too small - cannot be, on all machines I have not even reached 50%. gnutar is 2:1.23-15.el6_8 The directory definitely exists ... I would not be able to boot. But I also do not know how to check this: GNU tar state file /var/lib/amanda/gnutar-lists/XYZSOMETHING.new is corrupted. It's getting frustrating now. So anyone have other ideas why I get this error? Jobst -- There are 10 kinds of people in the world, those who understand binary and those who don't. | |0| | Jobst Schmalenbach, General Manager | | |0| Barrett & Sales Essentials |0|0|0| +61 3 9533 , POBox 277, Caulfield South, 3162, Australia
List Contents of gnutar-list files
Hi How can I see the contents of gnutar-list files? I tried tar/gunzip, ckecked amadmin etc but it seems I cannot read the content. Need to know whats in those files to see whether my inclusion/exclusion list is correct and why it takes too much time to backup one particular host. Jobst -- Computers run on smoke, once the smoke gets out they don't work anymore! | |0| | Jobst Schmalenbach, General Manager | | |0| Barrett & Sales Essentials |0|0|0| +61 3 9533 , POBox 277, Caulfield South, 3162, Australia
Re: approaches to Amanda vaulting?
Stefan,
can you try the attached patch?
Jean-Louis
On 24/10/17 03:00 PM, Jean-Louis Martineau wrote:
On 24/10/17 02:45 PM, Jean-Louis Martineau wrote:
On 24/10/17 02:29 PM, Stefan G. Weichinger wrote:
Am 2017-10-24 um 20:25 schrieb Jean-Louis Martineau:
This is not what you want.
You want amlabel to do its operation on the vault storage.
amlabel operate on the first configured storage, so this is what you
must change.
$ amlabel -ostorage=vault daily vault01 slot 1
thanks, but now:
$ amlabel -ostorage=vault daily vault01 slot 1
argument 'vault': string expected
argument 'vault': string expected
I don't get it, can you try the following:
amlabel -ostorage='vault' daily vault01 slot 1
amlabel -ostorage="vault" daily vault01 slot 1
amlabel -ostorage='"vault"' daily vault01 slot 1
I found why it doesn't work with a 'vault' storage.
The same parser is used for the configuration files and the -o
argument, but 'vault' is a reserved keyword in amanda, so you must
pass a string to the parser
amlabel -ostorage='"vault"' daily vault01 slot 1
Jean-Louis
This message is the property of CARBONITE, INC. and may contain confidential or
privileged information.
If this message has been delivered to you by mistake, then do not copy or
deliver this message to anyone. Instead, destroy it and notify me by reply
e-mail
diff --git a/common-src/conffile.c b/common-src/conffile.c
index f8df190..bb670ce 100644
--- a/common-src/conffile.c
+++ b/common-src/conffile.c
@@ -4280,7 +4280,7 @@ read_storage_identlist(
free_val_t(val);
ckseen(>seen);
val->v.identlist = NULL;
-get_conftoken(CONF_ANY);
+get_conftoken(CONF_IDENT);
while (tok == CONF_STRING || tok == CONF_IDENT) {
val->v.identlist = g_slist_append(val->v.identlist,
g_strdup(tokenval.v.s));
Re: approaches to Amanda vaulting?
On 24/10/17 02:45 PM, Jean-Louis Martineau wrote: On 24/10/17 02:29 PM, Stefan G. Weichinger wrote: Am 2017-10-24 um 20:25 schrieb Jean-Louis Martineau: This is not what you want. You want amlabel to do its operation on the vault storage. amlabel operate on the first configured storage, so this is what you must change. $ amlabel -ostorage=vault daily vault01 slot 1 thanks, but now: $ amlabel -ostorage=vault daily vault01 slot 1 argument 'vault': string expected argument 'vault': string expected I don't get it, can you try the following: amlabel -ostorage='vault' daily vault01 slot 1 amlabel -ostorage="vault" daily vault01 slot 1 amlabel -ostorage='"vault"' daily vault01 slot 1 I found why it doesn't work with a 'vault' storage. The same parser is used for the configuration files and the -o argument, but 'vault' is a reserved keyword in amanda, so you must pass a string to the parser amlabel -ostorage='"vault"' daily vault01 slot 1 Jean-Louis This message is the property of CARBONITE, INC. and may contain confidential or privileged information. If this message has been delivered to you by mistake, then do not copy or deliver this message to anyone. Instead, destroy it and notify me by reply e-mail
Re: approaches to Amanda vaulting?
On 24/10/17 02:29 PM, Stefan G. Weichinger wrote: Am 2017-10-24 um 20:25 schrieb Jean-Louis Martineau: This is not what you want. You want amlabel to do its operation on the vault storage. amlabel operate on the first configured storage, so this is what you must change. $ amlabel -ostorage=vault daily vault01 slot 1 thanks, but now: $ amlabel -ostorage=vault daily vault01 slot 1 argument 'vault': string expected argument 'vault': string expected I don't get it, can you try the following: amlabel -ostorage='vault' daily vault01 slot 1 amlabel -ostorage="vault" daily vault01 slot 1 amlabel -ostorage='"vault"' daily vault01 slot 1 This message is the property of CARBONITE, INC. and may contain confidential or privileged information. If this message has been delivered to you by mistake, then do not copy or deliver this message to anyone. Instead, destroy it and notify me by reply e-mail
Re: approaches to Amanda vaulting?
On 24/10/17 02:07 PM, Stefan G. Weichinger wrote: I was able to label the vtapes in storage "daily" but can't figure out why this command accesses changer "disk1" instead of "disk2" -> $ /usr/sbin/amlabel --storage vault --pool vault daily vault01 slot 1 Reading label... Found label 'daily01' but it doesn't match the labelstr '^daily[0-9][0-9]*$'. Not writing label. Not writing label. tried several combinations of parameters etc ... $ man amlabel --storage storage Assign the storage to the label, it can only be used with --assign, you can't assign a storage to a new labelled volume. This is not what you want. You want amlabel to do its operation on the vault storage. amlabel operate on the first configured storage, so this is what you must change. $ amlabel -ostorage=vault daily vault01 slot 1 Jean-Louis This message is the property of CARBONITE, INC. and may contain confidential or privileged information. If this message has been delivered to you by mistake, then do not copy or deliver this message to anyone. Instead, destroy it and notify me by reply e-mail
Re: approaches to Amanda vaulting?
Am 2017-10-24 um 19:07 schrieb Austin S. Hemmelgarn: > Based on what you're saying you want, I think you want the following in > your config: > > flush-threshold-dumped 400 > flush-threshold-scheduled 400 > taperflush 400 > autoflush yes > > The first two control flushing during a run, while taperflush controls > flushing at the end of a run. To get the flushing to actually happen, > you then need autoflush set to yes (and amanda will complain if it's not > set to yes and taperflush is more than zero). > > Now, I'm not 100% certain that will work, as I've not done this type of > thing myself (at work, we just use the holding disk as a cache so that > we can finish dumps as quickly as possible without our (slow, > parity-raid backed) persistent storage being the bottleneck, and at home > I don't use it since I don't need parallelization and I don't have any > disks that are faster than any others), but based on what I understand > from the documentation, I'm pretty sure this should do it. thanks for the explanation, I have that setup there and just let the normal amdump-cronjob run tonight. And see what happens. The admin forgot to swap tapes, so degraded mode only anyway for now ;-)
Re: approaches to Amanda vaulting?
Am 2017-10-24 um 18:28 schrieb Stefan G. Weichinger:
> Am 2017-10-24 um 13:38 schrieb Austin S. Hemmelgarn:
>> On 2017-10-22 13:38, Stefan G. Weichinger wrote:
>>> After or before I additionally can do something like:
>>>
>>> amvault myconf --dest-storage --latest-fulls archive
>>>
>>> correct?
>> I think so, but I'm not 100% certain.
>
> oh ;-)
>
>>> An additional hurdle is that the customer wants to use WORM tapes for
>>> archive, so I should get that right at the first run to not waste any
>>> tapes
>
>> Perhaps create a temporary virtual tape library for testing that the
>> archiving schedule works as expected? This is what I generally do
>> when testing changes at work (although I usually do it using a copy of
>> the main configuration so that I don't confuse the planner for the
>> production backups with half a dozen runs in one day).
>
> Sure, that would be good, but I don't have that much disk space available.
>
> I am currently trying to wrap my head around the tuning of these
> parameters (and understand the exact meaning by reading the man page):
>
> flush-threshold-dumped
> flush-threshold-scheduled
> taperflush
>
> I had lev0 of all DLEs in the holding disk and both flush-threshold
> values on 400 -> I thought this would keep data for 4 tapes inside the
> disk, but no, some lev0 backups were flushed to primary storage already.
>
> Maybe I set up a VM with 2 vtape changers and play around there to learn
> and understand.
did that and created a basic config with 2 chg-disk changers:
DEFINE CHANGER disk1 {
COMMENT ""
TAPEDEV ""
TPCHANGER "chg-disk:/mnt/disk1"
CHANGERDEV ""
CHANGERFILE ""
PROPERTY"removable" "no"
PROPERTY"umount" "no"
PROPERTY"num-slot" "5"
PROPERTY"auto-create-slot" "yes"
PROPERTY"mount" "no"
}
DEFINE CHANGER disk2 {
COMMENT ""
TAPEDEV ""
TPCHANGER "chg-disk:/mnt/disk2"
CHANGERDEV ""
CHANGERFILE ""
PROPERTY"removable" "no"
PROPERTY"umount" "no"
PROPERTY"num-slot" "5"
PROPERTY"auto-create-slot" "yes"
PROPERTY"mount" "no"
}
DEFINE STORAGE daily {
COMMENT ""
POLICY "daily"
TAPEDEV ""
TPCHANGER "disk1"
LABELSTR"^daily[0-9][0-9]*$"
AUTOLABEL ""
META-AUTOLABEL ""
TAPEPOOL"daily"
RUNTAPES1
TAPERSCAN "lexi"
TAPETYPE"HARD-DISK"
MAX-DLE-BY-VOLUME 10
TAPERALGO SMALLEST
TAPER-PARALLEL-WRITE 1
EJECT-VOLUMEno
ERASE-VOLUMEno
DEVICE-OUTPUT-BUFFER-SIZE 1310720
AUTOFLUSH yes
FLUSH-THRESHOLD-DUMPED 0
FLUSH-THRESHOLD-SCHEDULED 0
TAPERFLUSH 0
REPORT-USE-MEDIAyes
REPORT-NEXT-MEDIA yes
INTERACTIVITY ""
SET-NO-REUSEno
ERASE-ON-FAILUREno
ERASE-ON-FULL no
}
DEFINE STORAGE vault {
COMMENT ""
POLICY "daily"
TAPEDEV ""
TPCHANGER "disk2"
LABELSTR"^vault[0-9][0-9]*$"
AUTOLABEL ""
META-AUTOLABEL ""
TAPEPOOL"vault"
RUNTAPES1
TAPERSCAN "lexi"
TAPETYPE"HARD-DISK"
MAX-DLE-BY-VOLUME 10
TAPERALGO SMALLEST
TAPER-PARALLEL-WRITE 1
EJECT-VOLUMEno
ERASE-VOLUMEno
DEVICE-OUTPUT-BUFFER-SIZE 1310720
AUTOFLUSH yes
FLUSH-THRESHOLD-DUMPED 0
FLUSH-THRESHOLD-SCHEDULED 0
TAPERFLUSH 0
REPORT-USE-MEDIAyes
REPORT-NEXT-MEDIA yes
INTERACTIVITY ""
SET-NO-REUSEno
ERASE-ON-FAILUREno
ERASE-ON-FULL no
}
-
I was able to label the vtapes in storage "daily" but can't figure out
why this command accesses changer "disk1" instead of "disk2" ->
$ /usr/sbin/amlabel --storage vault --pool vault daily vault01 slot 1
Reading label...
Found label 'daily01' but it doesn't match the labelstr
'^daily[0-9][0-9]*$'.
Not writing label.
Not writing label.
tried several combinations of parameters etc ...
Re: approaches to Amanda vaulting?
On 2017-10-24 12:28, Stefan G. Weichinger wrote: Am 2017-10-24 um 13:38 schrieb Austin S. Hemmelgarn: On 2017-10-22 13:38, Stefan G. Weichinger wrote: After or before I additionally can do something like: amvault myconf --dest-storage --latest-fulls archive correct? I think so, but I'm not 100% certain. oh ;-) An additional hurdle is that the customer wants to use WORM tapes for archive, so I should get that right at the first run to not waste any tapes Perhaps create a temporary virtual tape library for testing that the archiving schedule works as expected? This is what I generally do when testing changes at work (although I usually do it using a copy of the main configuration so that I don't confuse the planner for the production backups with half a dozen runs in one day). Sure, that would be good, but I don't have that much disk space available. I am currently trying to wrap my head around the tuning of these parameters (and understand the exact meaning by reading the man page): flush-threshold-dumped flush-threshold-scheduled taperflush I had lev0 of all DLEs in the holding disk and both flush-threshold values on 400 -> I thought this would keep data for 4 tapes inside the disk, but no, some lev0 backups were flushed to primary storage already. Maybe I set up a VM with 2 vtape changers and play around there to learn and understand. Based on what you're saying you want, I think you want the following in your config: flush-threshold-dumped 400 flush-threshold-scheduled 400 taperflush 400 autoflush yes The first two control flushing during a run, while taperflush controls flushing at the end of a run. To get the flushing to actually happen, you then need autoflush set to yes (and amanda will complain if it's not set to yes and taperflush is more than zero). Now, I'm not 100% certain that will work, as I've not done this type of thing myself (at work, we just use the holding disk as a cache so that we can finish dumps as quickly as possible without our (slow, parity-raid backed) persistent storage being the bottleneck, and at home I don't use it since I don't need parallelization and I don't have any disks that are faster than any others), but based on what I understand from the documentation, I'm pretty sure this should do it.
Re: approaches to Amanda vaulting?
Am 2017-10-24 um 13:38 schrieb Austin S. Hemmelgarn: On 2017-10-22 13:38, Stefan G. Weichinger wrote: After or before I additionally can do something like: amvault myconf --dest-storage --latest-fulls archive correct? I think so, but I'm not 100% certain. oh ;-) An additional hurdle is that the customer wants to use WORM tapes for archive, so I should get that right at the first run to not waste any tapes Perhaps create a temporary virtual tape library for testing that the archiving schedule works as expected? This is what I generally do when testing changes at work (although I usually do it using a copy of the main configuration so that I don't confuse the planner for the production backups with half a dozen runs in one day). Sure, that would be good, but I don't have that much disk space available. I am currently trying to wrap my head around the tuning of these parameters (and understand the exact meaning by reading the man page): flush-threshold-dumped flush-threshold-scheduled taperflush I had lev0 of all DLEs in the holding disk and both flush-threshold values on 400 -> I thought this would keep data for 4 tapes inside the disk, but no, some lev0 backups were flushed to primary storage already. Maybe I set up a VM with 2 vtape changers and play around there to learn and understand. Stefan
Re: approaches to Amanda vaulting?
On 2017-10-22 13:38, Stefan G. Weichinger wrote:
Am 2017-10-16 um 19:22 schrieb Stefan G. Weichinger:
Am 2017-10-16 um 15:20 schrieb Jean-Louis Martineau:
Amanda 3.5 can do everything you want only by running the amdump command.
Using an holding disk:
* You configure two storages
* All dumps go to the holding disk
* All dumps are copied to each storages, not necessarily at the same
time or in the same run.
* The dumps stay in holding until they are copied to both storages
* You can tell amanda that everything must go to both storage or only
some dle full/incr
I now have set up a config like this:
define changer robot {
tpchanger "chg-robot:/dev/sg3"
property "tape-device" "0=tape:/dev/nst0"
property "eject-before-unload" "no"
property "use-slots" "1-8"
}
define tapetype LTO6 {
#comment "Created by amtapetype; compression enabled"
length 2442818848 kbytes # etwa 2.4 TB (sgw)
filemark 1806 kbytes
speed 74006 kps
blocksize 32 kbytes
part_size 200G
}
define storage myconf {
tapepool "myconf"
tapetype "LTO6"
tpchanger "robot"
labelstr "^CMR[0-9][0-9]*$"
autoflush yes
# flush-threshold-dumped 100
# flush-threshold-scheduled 100
#
# lass alles in holding disk
flush-threshold-dumped 400 # (or more)
flush-threshold-scheduled 400 # (or more)
taperflush 400
runtapes 4
}
define storage archive {
tapepool "archive"
tapetype "LTO6"
tpchanger "robot"
labelstr "^ARC[0-9][0-9]*$"
autoflush yes
flush-threshold-dumped 100
flush-threshold-scheduled 100
runtapes 4
dump-selection ALL FULL
}
storage "myconf"
maxdumpsize -1
amrecover_changer "robot"
> 8<
my goal:
I have to create a set of archive tapes for that customer, every 3
months or so.
With the above setup I now ran "amdump --no-taper myconf" which
collected all dumps on holdingdisk (did an "amadmin myconf force *"
before to force FULLs now).
As I understand that I could now do a plain amflush which should
(a) write to the tapes of tapepool "myconf" and
(b) leave the holdingdisk tarballs where they are, right?
(I am not yet sure about that "400" above, I want to keep data for all 4
tapes in the holdingdisk now and may reduce that to 100 for normal daily
runs without "--no-taper" or so)
After or before I additionally can do something like:
amvault myconf --dest-storage --latest-fulls archive
correct?
I think so, but I'm not 100% certain.
An additional hurdle is that the customer wants to use WORM tapes for
archive, so I should get that right at the first run to not waste any tapesPerhaps create a temporary virtual tape library for testing that the
archiving schedule works as expected? This is what I generally do when
testing changes at work (although I usually do it using a copy of the
main configuration so that I don't confuse the planner for the
production backups with half a dozen runs in one day).
Re: Dump Summary cuts off hostnames
On Tue, Oct 24, 2017 at 04:38:43PM +1100, Jobst Schmalenbach (jo...@barrett.com.au) wrote: > Hi > I searched for colum ... bugger. Thanks, guys. I have added this to my amanda.conf file. Jobst
