Interestingly enough, when I add the .noRecoveries() property to the alert
node I only get one DOWN alert even though there are 7 servers that are
within the alert range
On Wednesday, February 22, 2017 at 11:10:09 AM UTC-8, nath...@influxdb.com
wrote:
>
> If you want to ignore the OK alerts use the `.noRecoveries` property of
> the alert node. This will suppress the OK alerts.
>
> On Friday, February 17, 2017 at 3:33:16 PM UTC-7, Archie Archbold wrote:
>>
>> Hey all. Pretty new to TICK but I have a problem that I can't wrap my
>> head around.
>>
>> I am monitoring multiple servers all sending data to one influxdb
>> database and using the 'host' tag to separate the servers in the DB
>>
>> My 'disk' measurement is taking in mulitiple disk paths from the servers
>> (HOSTS) which each have a respective 'PATH' tag.
>>
>> So basically each server is assigned a HOST tag and each HOST has
>> multiple PATH tags.
>>
>> EXPECTED FUNCTIONALITY: kapacitor should alert upon state change of a
>> HOST's PATH if that path is within the alerting Lambda.
>> PROBLEM: When I start the kapacitor service, it looks like it's sensing a
>> state change any time it sees another host/path with a opposite status.
>>
>> This is a simplified example of the alerts I am getting:
>>
>> Host: host1 Path: /path1 Status: UP
>> Host: host1 Path: /path2 Status: DOWN
>> Host: host1 Path: /path3 Status: UP
>> Host: host2 Path: /path1 Status: DOWN
>> Host: host2 Path: /path2 Status: UP
>>
>> These alerts happen once for each host/path combination and then the
>> service performs as expected, alerting properly when lambda is achieved.
>>
>> The result of this is that I receive a slew of up/down alerts every time
>> I restart the kapacitor service
>>
>> Here is my current tick:
>> var data = stream
>> |from()
>> .measurement('disk')
>> .groupBy('host','path')
>> |alert()
>> .message('{{ .ID }} Server:{{ index .Tags "host" }} Path: {{
>> index .Tags "path" }} USED PERCENT: {{ index .Fields "used_percent" }}')
>> .warn(lambda: "used_percent" >= 80)
>> .id('DISK SPACE WARNING')
>> .email($DISK_WARN_GRP)
>>
>> And the corresponding DOT
>>
>> ID: disk_alert_warn
>>
>> Error:
>>
>> Template:
>>
>> Type: stream
>>
>> Status: enabled
>>
>> Executing: true
>>
>> Created: 17 Feb 17 22:27 UTC
>>
>> Modified: 17 Feb 17 22:27 UTC
>>
>> LastEnabled: 17 Feb 17 22:27 UTC
>>
>> Databases Retention Policies: ["main"."autogen"]
>>
>> TICKscript:
>>
>> var data = stream
>>
>> |from()
>>
>> .measurement('disk')
>>
>> .groupBy('host', 'path')
>>
>> |alert()
>>
>> .message('{{ .ID }} Server:{{ index .Tags "host" }} Path: {{
>> index .Tags "path" }} USED PERCENT: {{ index .Fields "used_percent" }}')
>>
>> .warn(lambda: "used_percent" >= 80)
>>
>> .id('DISK SPACE WARNING')
>>
>> .email()
>>
>>
>> DOT:
>>
>> digraph disk_alert_warn {
>>
>> graph [throughput="38.00 points/s"];
>>
>>
>> stream0 [avg_exec_time_ns="0s" ];
>>
>> stream0 -> from1 [processed="284"];
>>
>>
>> from1 [avg_exec_time_ns="3.9µs" ];
>>
>> from1 -> alert2 [processed="284"];
>>
>>
>> alert2 [alerts_triggered="14" avg_exec_time_ns="72.33µs"
>> crits_triggered="0" infos_triggered="0" oks_triggered="7"
>> warns_triggered="7" ];
>>
>> }
>>
>> As you can see, I get 7 oks triggered (for host/path groups that are not
>> in alert range) and 7 warns triggered (for the 7 host/path groups that are
>> within the alert range) upon start up.
>> Then it behaves as normal.
>>
>> I understand that it should be alerting for the 7 host/path groups that
>> are over 80 but why follow it with an alert about the ok groups?
>>
>> MORE INFO: When I raise the lambda to 90% (out of range for all
>> host/paths) I get no alerts at all (which is expected)
>>
>> Thanks to anyone who can help me understand this
>>
>
--
Remember to include the version number!
---
You received this message because you are subscribed to the Google Groups
"InfluxData" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to influxdb+unsubscr...@googlegroups.com.
To post to this group, send email to influxdb@googlegroups.com.
Visit this group at https://groups.google.com/group/influxdb.
To view this discussion on the web visit
https://groups.google.com/d/msgid/influxdb/b71d0ed6-b863-411a-85aa-f18cd3076df6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
