100%, because at some point in the last year or two, some code changed
that would cause the (some) alert code to trigger when the sensor
value was NULL, we added this logic and the issues went away.
Someone else posted the solution to this list on 7/9/21 and it worked
for us.
------------------------------------------------------------------------
Spencer J. Ryan| Manager, Technology and Infrastructure
*Miller Canfield***
*T* +1.313.496.7979 | *F* +1.313.496.7500
------------------------------------------------------------------------
*From:*Adam Armstrong via observium <observium@lists.observium.org>
*Sent:* Wednesday, November 23, 2022 10:21 AM
*To:* Ahmed Rahal via observium <observium@lists.observium.org>
*Cc:* Adam Armstrong <ad...@observium.org>
*Subject:* [Observium] Re: {EXT}Re: Group Email Alerts
*Caution:*This is an external email. Do not open attachments or click
links from unknown or unexpected emails.
Hmm. Are you sure this has ever worked?
The thresholds shouldn't have ever been present in the metric array.
The code the does the check doesn't pass these as metrics, and the
code that rewrites the @thresholds on the test side doesn't work on
the "metric" side of the comparison.
adam.
Ahmed Rahal via observium wrote on 2022-11-23 14:59:
That's a clever hack ;)
Cheers.
Le mer. 23 nov. 2022, à 09 h 58, Ryan, Spencer J. via observium
<observium@lists.observium.org
<mailto:observium@lists.observium.org>> a écrit :
Because on random discovery failures the sensor threshold
would go to NULL and then start alarming.
------------------------------------------------------------------------
Spencer J. Ryan| Manager, Technology and Infrastructure
*Miller Canfield***
*T* +1.313.496.7979 | *F* +1.313.496.7500
------------------------------------------------------------------------
*From:*Adam Armstrong via observium
<observium@lists.observium.org
<mailto:observium@lists.observium.org>>
*Sent:* Wednesday, November 23, 2022 9:53 AM
*To:* Ryan, Spencer J. via observium
<observium@lists.observium.org
<mailto:observium@lists.observium.org>>
*Cc:* Adam Armstrong <ad...@observium.org
<mailto:ad...@observium.org>>
*Subject:* [Observium] Re: {EXT}Re: Group Email Alerts
*Caution:*This is an external email. Do not open attachments
or click links from unknown or unexpected emails.
Why are you checking if the threshold is null?
You want to check if the value is null :)
adam.
Ryan, Spencer J. via observium wrote on 2022-11-23 14:20:
Funny enough, yes, although I’ve never seen that before
today (when I loaded the check to grab you the
conditions). I’m guessing they added some checks at some
point for input sanity, but it seems to still work normally.
Adam might know (or be able to fix it!)
------------------------------------------------------------------------
Spencer J. Ryan| Manager, Technology and Infrastructure
*Miller Canfield***
*T* +1.313.496.7979 | *F* +1.313.496.7500
------------------------------------------------------------------------
*From:*Luis Londoño <llond...@disti.com>
<mailto:llond...@disti.com>
*Sent:* Wednesday, November 23, 2022 9:18 AM
*To:* Ryan, Spencer J. <r...@millercanfield.com>
<mailto:r...@millercanfield.com>; Observium
<observium@lists.observium.org>
<mailto:observium@lists.observium.org>
*Cc:* Adam Armstrong <ad...@observium.org>
<mailto:ad...@observium.org>
*Subject:* RE: [Observium] Re: {EXT}Re: Group Email Alerts
*Caution:*This is an external email. Do not open
attachments or click links from unknown or unexpected emails.
Thank you for the suggestion. Do you get this error by any
chance?
*From:*Ryan, Spencer J. <r...@millercanfield.com
<mailto:r...@millercanfield.com>>
*Sent:* Wednesday, November 23, 2022 9:13 AM
*To:* Observium <observium@lists.observium.org
<mailto:observium@lists.observium.org>>
*Cc:* Adam Armstrong <ad...@observium.org
<mailto:ad...@observium.org>>; Luis Londoño
<llond...@disti.com <mailto:llond...@disti.com>>
*Subject:* RE: [Observium] Re: {EXT}Re: Group Email Alerts
You don't often get email from r...@millercanfield.com
<mailto:r...@millercanfield.com>. Learn why this is
important <https://aka.ms/LearnAboutSenderIdentification>
0, or null?
I have two similar checks (one for sensor below and one
for above thresholds) but a while ago I had to add null
checks to avoid this.
My Sensor High/Low alarms have a delay of 1 and set it so
all conditions have to be true:
|sensor_value less @sensor_limit_low|
|@sensor_limit_low ne NULL|
||
|(Obviously replace low for high on the other check)|
||
||
|That seems to work for us. |
------------------------------------------------------------------------
Spencer J. Ryan| Manager, Technology and Infrastructure
*Miller Canfield***
*T* +1.313.496.7979 | *F* +1.313.496.7500
------------------------------------------------------------------------
*From:*Luis Londoño via observium
<observium@lists.observium.org
<mailto:observium@lists.observium.org>>
*Sent:* Wednesday, November 23, 2022 9:02 AM
*To:* Observium <observium@lists.observium.org
<mailto:observium@lists.observium.org>>
*Cc:* Adam Armstrong <ad...@observium.org
<mailto:ad...@observium.org>>; Luis Londoño
<llond...@disti.com <mailto:llond...@disti.com>>
*Subject:* [Observium] Re: {EXT}Re: Group Email Alerts
*Caution:*This is an external email. Do not open
attachments or click links from unknown or unexpected emails.
To give an example of something that happened today and
why I ask; One of the servers we are monitoring we have an
alert check for the fans being below the sensor threshold
something timed out on the connection and caused the
sensor values to all drop to 0 and then showed the device
timed out as well. We got about 20 emails about the
sensors both about the alert and the recovery. I did go
ahead and add a alert delay to the checker so that would
at least fix that for now but maybe grouping the alerts
per host if its happening all at the same time would
reduce the noise of the alerts and provide a better way to
know that something on that host itself might be an issue.
*From:*Adam Armstrong via observium
<observium@lists.observium.org
<mailto:observium@lists.observium.org>>
*Sent:* Wednesday, November 23, 2022 8:55 AM
*To:* Ryan, Spencer J. via observium
<observium@lists.observium.org
<mailto:observium@lists.observium.org>>
*Cc:* Adam Armstrong <ad...@observium.org
<mailto:ad...@observium.org>>
*Subject:* {EXT}[Observium] Re: Group Email Alerts
There's not really a way to "group" alerts. It could
potentially be done, but not with our current alert
sending code.
If you frequently get situations where you get multiple
alerts all at once, being annoyed by those alerts should
serve as motivation to fix the problem, no? :D
adam.
Ryan, Spencer J. via observium wrote on 2022-11-23 13:42:
How are your alert checkers built?
While we have asked for alert dependencies forever
(and may never get it) I don't get any individual
alarms for sensors when a device is offline.
Sent from my Verizon, Samsung Galaxy smartphone
Get Outlook for Android
<https://usg02.safelinks.protection.office365.us/?url=https%3A%2F%2Faka.ms%2FAAb9ysg&data=05%7C01%7Cllondono%40disti.com%7C7d01993ce32f4d93afc908dacd5cd194%7Cd4b51790318d4698a7956c33a8d4b716%7C1%7C0%7C638048095794850730%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Y3noLRhM4Vie%2FESKB7IeCpME7owq8ZOWMcbbcrgRXss%3D&reserved=0>
------------------------------------------------------------------------
*From:* Luis Londoño via observium
<observium@lists.observium.org>
<mailto:observium@lists.observium.org>
*Sent:* Wednesday, November 23, 2022 8:31:57 AM
*To:* observium@lists.observium.org
<mailto:observium@lists.observium.org>
<observium@lists.observium.org>
<mailto:observium@lists.observium.org>
*Cc:* Luis Londoño <llond...@disti.com>
<mailto:llond...@disti.com>
*Subject:* [Observium] Group Email Alerts
*Caution:*This is an external email. Do not open
attachments or click links from unknown or unexpected
emails.
Is there a way to group email alerts so there is not
an email alert for every sensor that might go down at
the same time? For example multiple fans on a server
might show as an alert if the device itself is not
responding
You have received a message from the law firm Miller
Canfield. The information contained in or attached to
this electronic mail may be privileged and/or
confidential. If you received this transmission and
are not the intended recipient, you should not read
this message and are hereby notified that any
dissemination, distribution or copying of this
communication and/or its attachments is strictly
prohibited. If you have received this communication in
error or are not sure whether it is privileged, please
immediately notify us by return e-mail and delete or
destroy the original and any copies, electronic, paper
or otherwise, that you may have of this communication
and any attachments.
_______________________________________________
observium mailing list --observium@lists.observium.org
<mailto:observium@lists.observium.org>
To unsubscribe send an email
toobservium-le...@lists.observium.org
<mailto:observium-le...@lists.observium.org>
--
Sent from Postbox
<https://usg02.safelinks.protection.office365.us/?url=https%3A%2F%2Fwww.postbox-inc.com%2F%3Futm_source%3Demail%26utm_medium%3Dsiglink%26utm_campaign%3Dreach&data=05%7C01%7Cllondono%40disti.com%7C7d01993ce32f4d93afc908dacd5cd194%7Cd4b51790318d4698a7956c33a8d4b716%7C1%7C0%7C638048095794850730%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=givrKgpt7EmK32gCgqqobO4%2FktvYCEM%2FI%2FcqMQDOcpc%3D&reserved=0>
_______________________________________________
observium mailing list --observium@lists.observium.org
<mailto:observium@lists.observium.org>
To unsubscribe send an email toobservium-le...@lists.observium.org
<mailto:observium-le...@lists.observium.org>
--
Sent from Postbox
<https://www.postbox-inc.com/?utm_source=email&utm_medium=siglink&utm_campaign=reach>
_______________________________________________
observium mailing list -- observium@lists.observium.org
<mailto:observium@lists.observium.org>
To unsubscribe send an email to
observium-le...@lists.observium.org
<mailto:observium-le...@lists.observium.org>
--
Ahmed Rahal
Administrateur de Systèmes / Systems Administrator
*Videotron**/Fibre**noire* - www.fibrenoire.ca
<http://www.fibrenoire.ca/>
A: 612 Rue Saint-Jacques, Montréal, QC H3C 1E2
ahmed.ra...@videotron.com <mailto:ahmed.ra...@videotron.com>
Twitter: @fibrenoire
_______________________________________________
observium mailing list --observium@lists.observium.org
<mailto:observium@lists.observium.org>
To unsubscribe send an email toobservium-le...@lists.observium.org
<mailto:observium-le...@lists.observium.org>
--
Sent from Postbox
<https://www.postbox-inc.com/?utm_source=email&utm_medium=siglink&utm_campaign=reach>
_______________________________________________
observium mailing list -- observium@lists.observium.org
To unsubscribe send an email to observium-le...@lists.observium.org
