Re: amdump (3.5.1) run from script does not send email report

2019-08-27 Thread Chris Hoogendyk
Thanks, Nathan, that's a good guide for troubleshooting this issue. However, it leaves out one 
crucial item: sysadmin blind spots. ;-)  Looking through my emails and the dates, I realized Amanda 
had hung for several days and I hadn't noticed it because of other problems I've been focused on . . .


(We did release upgrades from Ubuntu 14.04 to Ubuntu 16.04 on two departmental servers that were 
running drupal web sites with MySQL backends as well as sendmail, squirrelmail, request-tracker4, 
proftpd, samba, and all the other things an academic department depends on. I've spent the last week 
patching up services and dealing with the switch from upstart to systemd among other things, trying 
to get things running smoothly before the semester begins.)


Anyway, I had added a line in my script that did `echo "Amanda already running"` and it got lost in 
all the output from the rest of the script documenting the nightly backup of the Amanda 
configuration and indexes and the trimming of the SSDs. Looking back at cron emails, I see the 
message in the output, but I had overlooked it in my quick scanning. Otherwise, I would have looked 
into the status of Amanda more closely and sooner.



On 8/27/19 3:59 PM, Nathan Stratton Treadway wrote:

On Tue, Aug 27, 2019 at 15:15:57 -0400, Chris Hoogendyk wrote:

When I upgraded one of my servers to Amanda 3.5.1, I had to run
amdump from a script (out of cron), because . . .

Now it doesn't send out an email report when it is finished.

Did you switch to using the script at the same time you upgraded to
3.5.1?  That is, did you ever have working reports on this server, either
pre-3.5.1 called from your script, or 3.5.1 called directly from cron?

(Obviously calling it from a script should work; I have a 3.5.1 system
running on Ubuntu which sends email reports from a script-invoked amdump
just fine.)

  

What is my workaround? I've looked through the amdump and amanda man

(What OS is running on this system, and what was the origin of your
Amanda binaries?)

What do you get when you run the following commands?

   $ amgetconf  mailto
   $ amgetconf  send-amreport-on
   $ amgetconf  mailer

Does the executable shown in the "mailer" parameter actually exist?



pages. I don't see an option to tell it, "please DO send an email
report." I also did check all the various places that might hint at
an email getting hung up, but didn't find any evidence in any of the
logs or spool directories that any had ever been sent.

Do you have any "amreport.*.debug" files in your
/server// directory?

If so, look in those log files for a line containing the string "invoking mail
app:", and see if anything interestinng shows up between that line and
onne saying "waiting for child process to finish.." (which normally is
the next line in the log, I think)

If those amreport debug files don't exist, look in the amdump.*.debug
files for lines with the string "amreport" to see if it's trying
and failing to run amreport, or completely skipping that step.

Nathan

Nathan Stratton Treadway  -  natha...@ontko.com  -  Mid-Atlantic region
Ray Ontko & Co.  -  Software consulting services  -   http://www.ontko.com/
  GPG Key: http://www.ontko.com/~nathanst/gpg_key.txt   ID: 1023D/ECFB6239
  Key fingerprint = 6AD8 485E 20B9 5C71 231C  0C32 15F3 ADCD ECFB 6239


--
---

Chris Hoogendyk

-
   O__   Systems Administrator
  c/ /'_ --- Biology & Geosciences Departments
 (*) \(*) -- 315 Morrill Science Center
~~ - University of Massachusetts, Amherst



---

Erdös 4



Re: amdump (3.5.1) run from script does not send email report

2019-08-27 Thread Nathan Stratton Treadway
On Tue, Aug 27, 2019 at 15:15:57 -0400, Chris Hoogendyk wrote:
> When I upgraded one of my servers to Amanda 3.5.1, I had to run
> amdump from a script (out of cron), because . . .
> 
> Now it doesn't send out an email report when it is finished.

Did you switch to using the script at the same time you upgraded to
3.5.1?  That is, did you ever have working reports on this server, either
pre-3.5.1 called from your script, or 3.5.1 called directly from cron?

(Obviously calling it from a script should work; I have a 3.5.1 system
running on Ubuntu which sends email reports from a script-invoked amdump
just fine.)

 
> What is my workaround? I've looked through the amdump and amanda man

(What OS is running on this system, and what was the origin of your
Amanda binaries?)

What do you get when you run the following commands?

  $ amgetconf  mailto 
  $ amgetconf  send-amreport-on
  $ amgetconf  mailer

Does the executable shown in the "mailer" parameter actually exist?


> pages. I don't see an option to tell it, "please DO send an email
> report." I also did check all the various places that might hint at
> an email getting hung up, but didn't find any evidence in any of the
> logs or spool directories that any had ever been sent.

Do you have any "amreport.*.debug" files in your
/server// directory? 

If so, look in those log files for a line containing the string "invoking mail
app:", and see if anything interestinng shows up between that line and
onne saying "waiting for child process to finish.." (which normally is
the next line in the log, I think)

If those amreport debug files don't exist, look in the amdump.*.debug
files for lines with the string "amreport" to see if it's trying
and failing to run amreport, or completely skipping that step.

Nathan

Nathan Stratton Treadway  -  natha...@ontko.com  -  Mid-Atlantic region
Ray Ontko & Co.  -  Software consulting services  -   http://www.ontko.com/
 GPG Key: http://www.ontko.com/~nathanst/gpg_key.txt   ID: 1023D/ECFB6239
 Key fingerprint = 6AD8 485E 20B9 5C71 231C  0C32 15F3 ADCD ECFB 6239


amdump (3.5.1) run from script does not send email report

2019-08-27 Thread Chris Hoogendyk
When I upgraded one of my servers to Amanda 3.5.1, I had to run amdump from a script (out of cron), 
because . . .


Now it doesn't send out an email report when it is finished.

What is my workaround? I've looked through the amdump and amanda man pages. I don't see an option to 
tell it, "please DO send an email report." I also did check all the various places that might hint 
at an email getting hung up, but didn't find any evidence in any of the logs or spool directories 
that any had ever been sent.


The reason I'm running it out of a script instead just a straight call from the crontab line is that 
3.5.1 will now run more than one instance of amanda in parallel, and this server can't handle that. 
So, I have a script now that checks to see if amanda is already running before launching a new run. 
That script is called from cron. I have another server that manages just fine with Amanda 3.5.1's 
behavior, because it has a tape library with two LTO7 drives and also with a pair of 4TB server 
grade SSDs for holding disk space.



--
---

Chris Hoogendyk

-
   O__   Systems Administrator
  c/ /'_ --- Biology & Geosciences Departments
 (*) \(*) -- 315 Morrill Science Center
~~ - University of Massachusetts, Amherst



---

Erdös 4