Re: Amanda 3.5.1 - "ERROR: amcheck-device terminated with signal 11"
On Sat, Oct 10, 2020 at 00:29:04 -0400, Nathan Stratton Treadway wrote: > Can you post the output of > > $ sed -n '99,$p;105q' > /usr/local/share/perl/5.22.1/Amanda/Taper/Scan/oldest.pm > ? > > (In other words, what's line 102 in that file on your system, with a > few lines of context?) (Hmm, perhaps $ cat -n /usr/local/share/perl/5.22.1/Amanda/Taper/Scan/oldest.pm | grep "^ *102" -C3 would be better -- that way, the file line numbers are included in output...) Nathan Nathan Stratton Treadway - natha...@ontko.com - Mid-Atlantic region Ray Ontko & Co. - Software consulting services - http://www.ontko.com/ GPG Key: http://www.ontko.com/~nathanst/gpg_key.txt ID: 1023D/ECFB6239 Key fingerprint = 6AD8 485E 20B9 5C71 231C 0C32 15F3 ADCD ECFB 6239
Re: Amanda 3.5.1 - "ERROR: amcheck-device terminated with signal 11"
On Fri, Oct 09, 2020 at 22:22:47 -0400, Chris Hoogendyk wrote: > both servers, I'm getting this error (ERROR: amcheck-device > terminated with signal 11). When I ran the amcheck before swapping amcheck-device is a Perl program, so it's a little bit impressive to be triggering a SEGV of the process :( . Do you get any coredump-related kernel messages in your syslog file when the process crashes? > Fri Oct 09 21:57:09.795469021 2020: pid 24239: thd-0xc2e600: amcheck-device: > NEO200x48: updating state > Fri Oct 09 21:57:09.802340532 2020: pid 24239: thd-0xc2e600: > amcheck-device: warning: Use of uninitialized value in numeric eq > (==) at /usr/local/share/perl/5.22.1/Amanda/Taper/Scan/oldest.pm > line 102. > Fri Oct 09 21:57:09.802537511 2020: pid 24239: thd-0xc2e600: > amcheck-device: warning: Use of uninitialized value in numeric eq > (==) at /usr/local/share/perl/5.22.1/Amanda/Taper/Scan/oldest.pm > line 102. > Fri Oct 09 21:57:09.802622523 2020: pid 24239: thd-0xc2e600: > amcheck-device: warning: Use of uninitialized value in numeric eq > (==) at /usr/local/share/perl/5.22.1/Amanda/Taper/Scan/oldest.pm > line 102. I am guessing this "uninitialized value" warning is not directly causing the crash, but those log message might possibly hint as to where in the program execution had reached just prior to the crash. Can you post the output of $ sed -n '99,$p;105q' /usr/local/share/perl/5.22.1/Amanda/Taper/Scan/oldest.pm ? (In other words, what's line 102 in that file on your system, with a few lines of context?) Do you see those warning lines in the log files from a successfully amcheck run (i.e. from a few days ago)? What do those logs show after the warning lines (or the "updating state" line)? Nathan Nathan Stratton Treadway - natha...@ontko.com - Mid-Atlantic region Ray Ontko & Co. - Software consulting services - http://www.ontko.com/ GPG Key: http://www.ontko.com/~nathanst/gpg_key.txt ID: 1023D/ECFB6239 Key fingerprint = 6AD8 485E 20B9 5C71 231C 0C32 15F3 ADCD ECFB 6239
Amanda 3.5.1 - "ERROR: amcheck-device terminated with signal 11"
I have two servers doing Amanda backups for two different departments. They are both Supermicro servers running Ubuntu 16.04 with fairly recent patches and Amanda 3.5.1. This evening I came in to swap magazines in the Overland NeoSeries tape libraries. We have just begun a program to send a set of magazines out to Iron Mountain. The libraries had new magazines with new LTO7 tapes that I had labeled with amlabel and then run through enough days of backups to write them all. Now I'm taking those out, putting back in the original magazines, and sending the new ones out to Iron Mountain. So, part of the point of saying all that is that I have done this magazine swap previously. Once. Now, this evening, on both servers, I'm getting this error (ERROR: amcheck-device terminated with signal 11). When I ran the amcheck before swapping magazines, it worked. I then did an `amtape daily slot 2` to get it pointed at the beginning, went to the web interface and unloaded that tape back into the magazine, swapped the magazines, and went back to amanda. `amtape daily slot whatever` works. It returns telling me the label on the tape, and the web interface shows it loaded. `amlabel daily somelabel` works. It looks at the tape, reads the label, tells me that it is already labeled and in use. `amcheck daily` fails. Any clues? I need to resolve this fairly quickly. The University is doing mandatory furloughs to deal with their budget shortfalls, and I'm scheduled for a one week furlough starting Sunday. I will update this later with information on whether the backups that are scheduled to start shortly end up working. I presume they will at least go to holding disk, but whether they will write to the tape is the question. Following is the contents of the /tmp/amanda/server/daily/amcheck-device.20201009215709.debug: Fri Oct 09 21:57:09.046919855 2020: pid 24239: thd-0xc2e600: amcheck-device: pid 24239 ruid 555 euid 555 version 3.5.1: start at Fri Oct 9 21:57:09 2020 Fri Oct 09 21:57:09.047288267 2020: pid 24239: thd-0xc2e600: amcheck-device: reading config file /usr/local/etc/amanda/daily/amanda.conf Fri Oct 09 21:57:09.076905220 2020: pid 24239: thd-0xc2e600: amcheck-device: pid 24239 ruid 555 euid 555 version 3.5.1: rename at Fri Oct 9 21:57:09 2020 Fri Oct 09 21:57:09.105544741 2020: pid 24239: thd-0xc2e600: amcheck-device: chg-robot: using statefile '/usr/local/var/amanda/chg-robot-dev-tape-by-id-scsi-1BDT-FlexStor-II-00MX64200626-LL0' Fri Oct 09 21:57:09.124493185 2020: pid 24239: thd-0xc2e600: amcheck-device: invoking /usr/sbin/mtx -f /dev/tape/by-id/scsi-1BDT_FlexStor_II_00MX64200626_LL0 status Fri Oct 09 21:57:09.793516825 2020: pid 24239: thd-0xc2e600: amcheck-device: mtx: Storage Changer /dev/tape/by-id/scsi-1BDT_FlexStor_II_00MX64200626_LL0:2 Drives, 48 Slots ( 0 Import/Export ) Fri Oct 09 21:57:09.793592258 2020: pid 24239: thd-0xc2e600: amcheck-device: mtx: Data Transfer Element 0:Empty Fri Oct 09 21:57:09.793645704 2020: pid 24239: thd-0xc2e600: amcheck-device: mtx: Data Transfer Element 1:Full (Storage Element 13 Loaded):VolumeTag = 01L7 Fri Oct 09 21:57:09.793696913 2020: pid 24239: thd-0xc2e600: amcheck-device: mtx: Storage Element 1:Empty Fri Oct 09 21:57:09.793740270 2020: pid 24239: thd-0xc2e600: amcheck-device: mtx: Storage Element 2:Empty Fri Oct 09 21:57:09.793779177 2020: pid 24239: thd-0xc2e600: amcheck-device: mtx: Storage Element 3:Full :VolumeTag=15L7 Fri Oct 09 21:57:09.793822925 2020: pid 24239: thd-0xc2e600: amcheck-device: mtx: Storage Element 4:Full :VolumeTag=16L7 Fri Oct 09 21:57:09.793860922 2020: pid 24239: thd-0xc2e600: amcheck-device: mtx: Storage Element 5:Full :VolumeTag=17L7 Fri Oct 09 21:57:09.793898857 2020: pid 24239: thd-0xc2e600: amcheck-device: mtx: Storage Element 6:Full :VolumeTag=14L7 Fri Oct 09 21:57:09.793932089 2020: pid 24239: thd-0xc2e600: amcheck-device: mtx: Storage Element 7:Empty Fri Oct 09 21:57:09.793961884 2020: pid 24239: thd-0xc2e600: amcheck-device: mtx: Storage Element 8:Empty Fri Oct 09 21:57:09.794026494 2020: pid 24239: thd-0xc2e600: amcheck-device: mtx: Storage Element 9:Full :VolumeTag=21L7 Fri Oct 09 21:57:09.794067771 2020: pid 24239: thd-0xc2e600: amcheck-device: mtx: Storage Element 10:Full :VolumeTag=18L7 Fri Oct 09 21:57:09.794102988 2020: pid 24239: thd-0xc2e600: amcheck-device: mtx: Storage Element 11:Empty Fri Oct 09 21:57:09.794132586 2020: pid 24239: thd-0xc2e600: amcheck-device: mtx: Storage Element 12:Empty Fri Oct 09 21:57:09.794164280 2020: pid 24239: thd-0xc2e600: amcheck-device: mtx: Storage Element 13:Empty Fri Oct 09 21:57:09.794196459 2020: pid 24239: thd-0xc2e600: amcheck-device: mtx: Storage Element 14:Full :VolumeTag=13L7 Fri Oct 09 21:57:09.794233274 2020: pid 24239: thd-0xc2e600: amcheck-device: mtx: Storage Element 15:Full :VolumeTag=03L7 Fri Oct 09
amanda-excludes file - includes?
Silly question, can an include be added to an amanda-exclude file. I.e., is there a syntax to say "exclude this everything under this directory except for FuBar". The backstory: My mailserver has a quarantine directory (QD) for spam and virus containing mail. This QD directory is listed in the amanda-excudes file at the root of the DLE. Who wants to backup spam and viruses. I have developed several scripts for dealing with the collected spam and as they were of no use except in the QD, I made a QD/.bin directory to hold them. Stupidly I never considered the QD/.bin directory would not be backed up (due to the amanda-excludes entry) nor that I might moronically delete the QD/.bin contents. As I redevelop the scripts, I'll put them in a relocated bin directory. But I just wonder if there was some way to address this in an amanda-excludes file. I doubt it. Jon -- Jon H. LaBadie j...@jgcomp.com 11226 South Shore Rd. (703) 787-0688 (H) Reston, VA 20190 (703) 935-6720 (C)
Re: restore from vtapes written by amvault
Hello, Am 09.10.20 um 01:31 schrieb Nathan Stratton Treadway: > It looks like amfetchdump should be creating a > "$logdir/fetchdump.$timestamp" log file. If so, does that include any > mention of opening the vtape changer and/or detecting storage names? A logfile like log.20201008195900.0 ist created. No fetchdump.$timestamp file. $ cat log.20201008195900.0 INFO amfetchdump fetchdump pid 4503 There is another logfile: /tmp/amanda/server/be-full/amfetchdump.20201009083605.debug $ cat /tmp/amanda/server/be-full/amfetchdump.20201009083605.debug Fr Okt 09 08:36:05.594766041 2020: pid 2215: thd-0x1bab4f0: amfetchdump: pid 2215 ruid 33 euid 33 version 3.5.1: start at Fri Oct 9 08:36:05 2020 Fr Okt 09 08:36:05.594871851 2020: pid 2215: thd-0x1bab4f0: amfetchdump: Arguments: -d vtape be-full svr ^/$ 2119 Fr Okt 09 08:36:05.595963551 2020: pid 2215: thd-0x1bab4f0: amfetchdump: reading config file /var/lib/amanda/be-full/amanda.conf Fr Okt 09 08:36:05.596291148 2020: pid 2215: thd-0x1bab4f0: amfetchdump: reading config file /var/lib/amanda/amanda.conf.main Fr Okt 09 08:36:05.605085245 2020: pid 2215: thd-0x1bab4f0: amfetchdump: pid 2215 ruid 33 euid 33 version 3.5.1: rename at Fri Oct 9 08:36:05 2020 Fr Okt 09 08:36:05.608700988 2020: pid 2215: thd-0x1bab4f0: amfetchdump: beginning trace log: /var/lib/amanda/be-full/log/log.20201009083605.0 Fr Okt 09 08:36:05.624978043 2020: pid 2215: thd-0x1bab4f0: amfetchdump: chg-disk: Dir /var/lib/vtape/be-full Fr Okt 09 08:36:05.625018252 2020: pid 2215: thd-0x1bab4f0: amfetchdump: chg-disk: Using statefile '/var/lib/vtape/be-full/state' Fr Okt 09 08:36:05.839215832 2020: pid 2215: thd-0x1bab4f0: amfetchdump: /opt/amanda/lib/amanda/perl/Amanda/Restore.pm:1303:success:492 1 volume(s) needed for restoration The following volumes are needed: BE-full-00 Fr Okt 09 08:36:08.174733052 2020: pid 2215: thd-0x1bab4f0: amfetchdump: Amanda::Recovery::Clerk: loading volume 'BE-full-00' Fr Okt 09 08:36:08.174965611 2020: pid 2215: thd-0x1bab4f0: amfetchdump: find_volume labeled 'BE-full-00' Fr Okt 09 08:36:08.195919769 2020: pid 2215: thd-0x1bab4f0: amfetchdump: parse_inventory: load slot 6 Fr Okt 09 08:36:08.196089927 2020: pid 2215: thd-0x1bab4f0: amfetchdump: /opt/amanda/lib/amanda/perl/Amanda/Recovery/Scan.pm:420:info:120 slot 6 Fr Okt 09 08:36:08.23707 2020: pid 2215: thd-0x1bab4f0: amfetchdump: dir_name: /var/lib/vtape/be-full/slot6/ Fr Okt 09 08:36:08.201052481 2020: pid 2215: thd-0x1bab4f0: amfetchdump: Device file:/var/lib/vtape/be-full/slot6 error = 'File 0 not found' Fr Okt 09 08:36:08.201109545 2020: pid 2215: thd-0x1bab4f0: amfetchdump: Device file:/var/lib/vtape/be-full/slot6 setting status flag(s): DEVICE_STATUS_VOLUME_UNLABELED Fr Okt 09 08:36:08.203808430 2020: pid 2215: thd-0x1bab4f0: amfetchdump: /opt/amanda/lib/amanda/perl/Amanda/Recovery/Scan.pm:471:error:122 File 0 not found Fr Okt 09 08:36:08.204280888 2020: pid 2215: thd-0x1bab4f0: amfetchdump: new Amanda::Changer::Error: type='fatal', message='File 0 not found' Fr Okt 09 08:36:08.221645777 2020: pid 2215: thd-0x1bab4f0: amfetchdump: parse_inventory: load slot 7 Fr Okt 09 08:36:08.221763588 2020: pid 2215: thd-0x1bab4f0: amfetchdump: /opt/amanda/lib/amanda/perl/Amanda/Recovery/Scan.pm:420:info:120 slot 7 Fr Okt 09 08:36:08.225548944 2020: pid 2215: thd-0x1bab4f0: amfetchdump: dir_name: /var/lib/vtape/be-full/slot7/ Fr Okt 09 08:36:08.226464125 2020: pid 2215: thd-0x1bab4f0: amfetchdump: Device file:/var/lib/vtape/be-full/slot7 error = 'File 0 not found' Fr Okt 09 08:36:08.226503803 2020: pid 2215: thd-0x1bab4f0: amfetchdump: Device file:/var/lib/vtape/be-full/slot7 setting status flag(s): DEVICE_STATUS_VOLUME_UNLABELED Fr Okt 09 08:36:08.229226167 2020: pid 2215: thd-0x1bab4f0: amfetchdump: /opt/amanda/lib/amanda/perl/Amanda/Recovery/Scan.pm:471:error:122 File 0 not found Fr Okt 09 08:36:08.229572359 2020: pid 2215: thd-0x1bab4f0: amfetchdump: new Amanda::Changer::Error: type='fatal', message='File 0 not found' Fr Okt 09 08:36:14.765428839 2020: pid 2215: thd-0x1bab4f0: amfetchdump: new Amanda::Changer::Error: type='fatal', message='Aborted by user' Fr Okt 09 08:36:14.765978065 2020: pid 2215: thd-0x1bab4f0: amfetchdump: /opt/amanda/lib/amanda/perl/Amanda/Restore.pm:1398:error:4900045 Aborted by user Fr Okt 09 08:36:14.766222642 2020: pid 2215: thd-0x1bab4f0: amfetchdump: /opt/amanda/lib/amanda/perl/Amanda/Restore.pm:2174:error:4900068 Aborted by user Fr Okt 09 08:36:14.767321030 2020: pid 2215: thd-0x1bab4f0: amfetchdump: ru_utime : 0 Fr Okt 09 08:36:14.767354700 2020: pid 2215: thd-0x1bab4f0: amfetchdump: ru_stime : 0 Fr Okt 09 08:36:14.767361174 2020: pid 2215: thd-0x1bab4f0: amfetchdump: ru_maxrss : 29200 Fr Okt 09 08:36:14.767382936 2020: pid 2215: thd-0x1bab4f0: amfetchdump: ru_ixrss : 0 Fr Okt 09 08:36:14.767388566 2020: pid 2215: thd-0x1bab4f0: amfetchdump: ru_idrss : 0 Fr Okt 09 08:36:14.767393604 2020: pid 2215: thd-0x1bab4f0: amfetchdump: ru_isrss : 0 Fr Okt
Re: restore from vtapes written by amvault
Hello,
Am 08.10.20 um 20:41 schrieb Nathan Stratton Treadway:
> (What does
> $ amadmin be-full config | grep -i storage
> show right now?)
$ amadmin be-full config |grep -i storage
ACTIVE-STORAGEbe-full
STORAGE be-full
VAULT-STORAGE ""
DEFINE STORAGE vtape {
DEFINE STORAGE be-full {
Am 09.10.20 um 01:31 schrieb Nathan Stratton Treadway:
> It looks like amfetchdump should be creating a
> "$logdir/fetchdump.$timestamp" log file. If so, does that include any
> mention of opening the vtape changer and/or detecting storage names?
A logfile like log.20201008195900.0 ist created. No fetchdump.$timestamp
file.
$ cat log.20201008195900.0
INFO amfetchdump fetchdump pid 4503
