Re: Send notification at the end of a deployment

2022-03-24 Diskussionsfäden Martin Krämer
I am sending a compressed file with errors via mail in case some appear
(but nothing that is sent in case of successful deployment) with a modified
version of "savelog.LAST.sh" hook.
Find attached.

As gmail allows to sent via smtps to other gmail addresses I use the
following variables for configuration:

FAI_INFOMAIL_SENDERR='true'
FAI_INFOMAIL_SMTPURL='smtps://:@smtp.gmail.com'
FAI_INFOMAIL_TOADDRESS="@gmail.com"

Obviously you need to have "mutt" application installed and you might need
to adjust the following variables to match your needed patterns..
[ -z "${FAI_ERR_PATTERNS}" ] && FAI_ERR_PATTERNS="X_X-X_XX"
[ -z "${FAI_ERR_WARN}" ] && FAI_ERR_WARN="X_X-X_XX"
[ -z "${FAI_ERR_IGNORE}" ] && FAI_ERR_IGNORE="X_X-X_XX"

Am Do., 24. März 2022 um 14:46 Uhr schrieb Romain Pelissier <
romain.peliss...@gmail.com>:

> Hello everybody,
> Do you know a way to execute a script on the server (FAI) side after a
> successful or unsuccessful client's deployment?
> In case of an unsuccessful attempt, is there a way to get some logs and
> attach them to an email for example?
> Thanks for any useful info you can provide about this :)
> Romain
>
#! /bin/bash
# version: 2019.11.25

# parse all log files for error messages
# print errors and warnings found to error.log
# WARNING: This will only work with english error messages!

errfile=$LOGDIR/error.log

# Define grep patterns. Do not start or end with an empty line!
globalerrorpatterns="error
fail
warn
 bad
bad 
no space
syntax
Couldn't stat
Cannot access
 conflict
is bigger than the limit
did not exist
non existent
not found
couldn't
can't
E: Sorry, broken packages
^E: 
operator expected
ambiguous redirect
No previous regular expression
No such
Device or resource busy
unknown option
[a-z]\+\.log:E:
No candidate version found
segfault
Couldn't find any package whose name or description matched
cannot create
The following packages have unmet dependencies"

globalignorepatterns="[a-z]\+\.log:#
Error: Driver 'pcspkr' is already registered, aborting
: bytes  packets  errors  dropped
:+ error=0
:+ trap error=
task_error_func=
STOP_ON_ERROR=
courier-webadmin
plugins-bad
Enabling conf localized-error-pages
ibwebadmin
kernel-patch-badram
kolab-webadmin
kolabadmin
gstreamer.\+-plugins-really-bad
liberrors.so
liberrors-samba
libsamba-errors
gsambad
libad
libtest-nowarnings-perl
libtest-warn-perl
libclass-errorhandler-perl
zope-ploneerrorreporting
libroxen-errormessage
liberror-perl
libgpg-error-dev
libgpg-error0
Opts:.\+errors=remount
[RT]X packets:
WARNING: unexpected IO-APIC
warned about = ( )
daemon.warn
kern.warn
rw,errors=
Expect some cache
no error
failmsg
RPC call returned error 101
deverror.out
(floppy), sector 0
mount version older than kernel
Can't locate module
Warning only .\+MB will be used.
hostname: Host name lookup failure
I can't tell the difference.
warning, not much extra random data, consider using the -rand option
confC._FILE
Warning: 3 database(s) sources
were not found, (but were created)
removing exim
The home dir you specified already exists.
No Rule for /usr/lib/ispell/default.hash.
/usr/sbin/update-fonts-.\+: warning: absolute path
hostname: Unknown server error
EXT2-fs warning: checktime reached
RPC: sendmsg returned error 101
can't print them to stdout. Define these classes
warning: downgrading
suppress emacs errors
echo Error:
Can't open dependencies file
documents in /usr/doc are no longer supported
if you have both a SCSI and an IDE CD-ROM
Warning: /proc/ide/hd?/settings interface is obsolete, and will be removed soon
Monitoring disabled
Error: only one processor found.
Error Recovery Strategy:
sector 0 does not have an
syslogin_perform_logout: logout() returned an error
grub is not in an XFS filesystem.
grub-install: line 374:
grub-probe: error: Cannot open \`/boot/grub/device.map'
is harmless
not updating .\+ font directory data.
register_serial(): autoconfig failed
Fontconfig error: Cannot load default config file
asking for cache data failed
However, I can not read the target:
Warning: The partition table looks like it was made
task_error=0
task_local_error=0
^info: Trying to set
warning: /usr/lib/X11/fonts
can't read /etc/udev/rules.d/z25_persistent-net.rules
/cow': No such file or directory
Dummy start-stop-daemon called
X: bytes  packets  errors
ACPI Error
ACPI Warning
AE_NOT_FOUND
conflicts with ACPI region
cannot stat \`/etc/modprobe.d/\*.conf'
cdrom: open failed.
libgpg-error
process \`kudzu' used the deprecated sysctl system call
PM: Resume from disk failed
JBD: barrier-based sync failed
aufs: module is from the staging directory, the quality is unknown
warning: linuxlogo stop runlevel arguments (none) do not match
insserv: warning: script .\+ missing LSB tags and overrides
live-premount.\+ If this fails
cannot read table of mounted file systems
error: no alternatives for
ERST: Error Record Serialization Table (ERST) support is initialized
ERST: Table is not found
HEST: Table not found
failed to stat /dev/pts
Failed to connect to socket /v

Re: Send notification at the end of a deployment

2022-03-24 Diskussionsfäden Robert Markula

Hi Romain,

take a look at the chapter "Saving log files" in the FAI guide [1]. If 
configured, FAI can save all logs to a remote log server via SSH.


That being said, I find it difficult to define when a host install 
process has actually failed. There are often situations where an install 
completes successfully, but the following bootup fails because some 
minor thing broke during install, e.g. because GRUB hasn't been set up 
correctly. And even if the system boots successfully, maybe a critical 
component during install failed which prevents you from logging in remotely.


So I always check at least the FAI error.log on the log server to get a 
quick glimpse if anything went wrong during install. On automated 
systems I have mechanisms in place that essentially perform integrity 
checks on the newly installed system and that are capable of fixing 
select issues like a broken SSH access.


Cheers,


Robert

[1] 
https://fai-project.org/fai-guide/#_a_id_isavelog_a_saving_log_files_task_savelog



Am 24.03.22 um 14:45 schrieb Romain Pelissier:

Hello everybody,
Do you know a way to execute a script on the server (FAI) side after a 
successful or unsuccessful client's deployment?
In case of an unsuccessful attempt, is there a way to get some logs 
and attach them to an email for example?

Thanks for any useful info you can provide about this :)
Romain