Re: [prometheus-users] Alert goes to Firing --> Resolved --> Firing immediately.

On 25/11/2020 16:27, Yagyansh S. Kumar wrote: On Wed, 25 Nov, 2020, 9:34 pm Stuart Clark, > wrote: Is the second instance still running? If you are having some cluster communications issues that could result in what you are seeing. Both instances

Re: [prometheus-users] Alert goes to Firing --> Resolved --> Firing immediately.

On Wed, 25 Nov, 2020, 9:34 pm Stuart Clark, wrote: > Is the second instance still running? > > If you are having some cluster communications issues that could result in > what you are seeing. Both instances learn of an alert but then one instance > missed some of the renewal messages, so resolves

Re: [prometheus-users] Alert goes to Firing --> Resolved --> Firing immediately.

Is the second instance still running? If you are having some cluster communications issues that could result in what you are seeing. Both instances learn of an alert but then one instance missed some of the renewal messages, so resolves it. Then it gets updated and the alert is fired again. If

Re: [prometheus-users] Alert goes to Firing --> Resolved --> Firing immediately.

On Wed, 25 Nov, 2020, 8:26 pm Stuart Clark, wrote: > How many Alertmanager instances are there? Can they talk to each other and > is Prometheus configured and able to push alerts to them all? > >> Single instance as of now. I did setup a Alertmanager Mesh of 2 Alertmanagers but I am facing duplic

Re: [prometheus-users] Alert goes to Firing --> Resolved --> Firing immediately.

How many Alertmanager instances are there? Can they talk to each other and is Prometheus configured and able to push alerts to them all? On 25 November 2020 14:07:41 GMT, "Yagyansh S. Kumar" wrote: >Hi Stuart. > >On Wed, 25 Nov, 2020, 6:56 pm Stuart Clark, >wrote: > >> On 25/11/2020 11:46, ya

Re: [prometheus-users] Alert goes to Firing --> Resolved --> Firing immediately.

Hi Stuart. On Wed, 25 Nov, 2020, 6:56 pm Stuart Clark, wrote: > On 25/11/2020 11:46, yagyans...@gmail.com wrote: > > The alert formation doesn't seem to be a problem here, because it > > happens for different alerts randomly. Below is the alert for Exporter > > being down for which it has happen

Re: [prometheus-users] Alert goes to Firing --> Resolved --> Firing immediately.

On 25/11/2020 11:46, yagyans...@gmail.com wrote: The alert formation doesn't seem to be a problem here, because it happens for different alerts randomly. Below is the alert for Exporter being down for which it has happened thrice today.   - alert: ExporterDown     expr: up == 0     for: 10m   

Re: [prometheus-users] Alert goes to Firing --> Resolved --> Firing immediately.

The alert formation doesn't seem to be a problem here, because it happens for different alerts randomly. Below is the alert for Exporter being down for which it has happened thrice today. - alert: ExporterDown expr: up == 0 for: 10m labels: severity: "CRITICAL" annotation

Re: [prometheus-users] Alert goes to Firing --> Resolved --> Firing immediately.

This could be many things, likely it has to do with the formulation of the alert. What does it look like in Prometheus? Specifically - the ALERTS metric shows what is pending or firing over time - evaluate the alert expression in Prometheus for the given time period. Are there gaps or does e.g. a

[prometheus-users] Alert goes to Firing --> Resolved --> Firing immediately.

Hi. I am using Alertmanager 0.21.0. Occasionally, the active alerts go to resolved state for a second and then come back to firing state immediately. There is no pattern of this happening, it happens randomly. Haven't been able to identify why this is happening. Any thoughts here? Where should