Hi Jacques, On Thu, 6 May 2021, Jacques Le Roux wrote:
That sounds good indeed. There is also the daily applications restart at 3AM UTC. But I don't see it there, did you made a trick about that?
Perhaps the restarts don't usually cause what Icinga thinks of as a 'hard' failure. Icinga attempts to make a connection only every few minutes, and it counts the failed attempts. More than three failed attempts consecutively for this service is configured to be a 'hard' failure and triggers a warning email to me. All failures are logged, but fewer than four consecutively will not cause an email to be sent. There are a few short clusters of failures around 03:40 UTC, are you sure that the restarts will be at 03:00 UTC and not at around 03:40? Anyway they're more or less indistinguishable from the occasional and usually very short network outages that we see all the time. The original logs don't show the timestamps in easily-readable form. In my earlier message I used 'localtime' in the Perl one-liner which formats the output, so the timezone was BST (+0100); below I've used 'gmtime' so it's UTC. 73, Ged. 8<---------------------------------------------------------------------- $ grep Ofbiz /var/log/icinga/archives/* | grep -v OK | perl -pe 's/.*\[(\d+)\].*?;/ /;print scalar gmtime($1);' | grep ' 03:' | sed -e 's/;/\t/;' Fri Mar 26 03:25:25 2021 Ofbiz - stable CRITICAL;SOFT;1;CRITICAL - Socket timeout after 10 seconds Fri Mar 26 03:26:25 2021 Ofbiz - stable CRITICAL;SOFT;2;CRITICAL - Socket timeout after 10 seconds Sun Apr 4 03:39:26 2021 Ofbiz - trunk CRITICAL;SOFT;1;CRITICAL - Socket timeout after 10 seconds Sun Apr 4 03:39:26 2021 Ofbiz - stable CRITICAL;SOFT;1;CRITICAL - Socket timeout after 10 seconds Sun Apr 11 03:37:44 2021 Ofbiz - old CRITICAL;SOFT;1;CRITICAL - Socket timeout after 10 seconds Sun Apr 11 03:37:44 2021 Ofbiz - stable CRITICAL;SOFT;1;CRITICAL - Socket timeout after 10 seconds Sun Apr 11 03:38:43 2021 Ofbiz - old CRITICAL;SOFT;2;CRITICAL - Socket timeout after 10 seconds Sun Apr 11 03:39:44 2021 Ofbiz - old CRITICAL;SOFT;3;CRITICAL - Socket timeout after 10 seconds Sun Apr 11 03:39:53 2021 Ofbiz - trunk CRITICAL;SOFT;1;CRITICAL - Socket timeout after 10 seconds Sun Apr 11 03:40:45 2021 Ofbiz - old CRITICAL;HARD;4;CRITICAL - Socket timeout after 10 seconds Sun Apr 11 03:40:45 2021 Ofbiz - old;CRITICAL;notify-service-by-email;CRITICAL - Socket timeout after 10 seconds Sun Apr 11 03:40:53 2021 Ofbiz - trunk CRITICAL;SOFT;2;CRITICAL - Socket timeout after 10 seconds Sun Apr 11 03:41:53 2021 Ofbiz - trunk CRITICAL;SOFT;3;CRITICAL - Socket timeout after 10 seconds Fri Apr 16 03:22:16 2021 Ofbiz - old CRITICAL;SOFT;1;CRITICAL - Socket timeout after 10 seconds Fri Apr 16 03:38:17 2021 Ofbiz - old CRITICAL;SOFT;1;CRITICAL - Socket timeout after 10 seconds Sun Apr 25 03:38:17 2021 Ofbiz - old CRITICAL;SOFT;1;CRITICAL - Socket timeout after 10 seconds Sun Apr 25 03:39:16 2021 Ofbiz - old CRITICAL;SOFT;2;CRITICAL - Socket timeout after 10 seconds Sun Apr 25 03:39:57 2021 Ofbiz - trunk CRITICAL;SOFT;1;CRITICAL - Socket timeout after 10 seconds Sun Apr 25 03:40:56 2021 Ofbiz - trunk CRITICAL;SOFT;2;CRITICAL - Socket timeout after 10 seconds Sun Apr 25 03:41:56 2021 Ofbiz - trunk CRITICAL;SOFT;3;CRITICAL - Socket timeout after 10 seconds Sun Apr 25 03:42:57 2021 Ofbiz - trunk CRITICAL;HARD;4;CRITICAL - Socket timeout after 10 seconds Sun Apr 25 03:42:57 2021 Ofbiz - trunk;CRITICAL;notify-service-by-email;CRITICAL - Socket timeout after 10 seconds Sun May 2 03:39:07 2021 Ofbiz - stable CRITICAL;SOFT;1;CRITICAL - Socket timeout after 10 seconds Sun May 2 03:41:27 2021 Ofbiz - old CRITICAL;SOFT;1;CRITICAL - Socket timeout after 10 seconds Sun May 2 03:42:29 2021 Ofbiz - old CRITICAL;SOFT;2;CRITICAL - Socket timeout after 10 seconds 8<----------------------------------------------------------------------
