Re: [prometheus-users] is it already a known issue with alertmanager integration to pagerduty

2022-09-26 Thread Stuart Clark 

On 26/09/2022 12:02, Brian Candler wrote:
> When, I have multiple alerts for the same route with same severity, 
all ended up in pagerduty as expected


Yes, because these are separate alerts that are not grouped together.

> But, when I have alerts triggered for the same instance with 
different severity, then I have only one (may be also the first alert) 
sent to pagerduty


But you said you've told alertmanager to group on "instance", so 
that's what it will do: all alerts for the same instance (i.e. with 
the same value of the 'instance' label) will be delivered in a single 
message.  Whether the *body* of the pagerduty message shows those 
individual alerts, depends on how you've configured things.


You might want to show your alertmanager pagerduty_config 
, 
and also your alert grouping config.


In general if you are sending alerts through to an external incident 
management system (PagerDuty, OpsGenie, JIRA, etc.) you are probably 
best not doing any grouping at all:


|group_by: ['...']|

--
Stuart Clark

--
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/f3618026-bfd9-7a42-ac99-7fe07b01e39a%40Jahingo.com.

Re: [prometheus-users] is it already a known issue with alertmanager integration to pagerduty

2022-09-26 Thread Brian Candler 
> When, I have multiple alerts for the same route with same severity, all 
ended up in pagerduty as expected

Yes, because these are separate alerts that are not grouped together.

> But, when I have alerts triggered for the same instance with different 
severity, then I have only one (may be also the first alert) sent to 
pagerduty

But you said you've told alertmanager to group on "instance", so that's 
what it will do: all alerts for the same instance (i.e. with the same value 
of the 'instance' label) will be delivered in a single message.  Whether 
the *body* of the pagerduty message shows those individual alerts, depends 
on how you've configured things.

You might want to show your alertmanager pagerduty_config 
, 
and also your alert grouping config.

On Monday, 26 September 2022 at 09:55:03 UTC+1 ishu...@gmail.com wrote:

> Hi,
>
> Thanks for the reply.
>
> I have only one route per environment and doesn't have separate routes per 
> severity within environment. When, I have multiple alerts for the same 
> route with same severity, all ended up in pagerduty as expected. But, when 
> I have alerts triggered for the same instance with different severity, then 
> I have only one (may be also the first alert) sent to pagerduty and the 
> other alerts are although showing up on alertmanager but not reaching 
> pagerduty.
>
> Thanks
> Eswar
> On Friday, 23 September 2022 at 23:26:43 UTC+2 juliu...@promlabs.com 
> wrote:
>
>> Hi Eswar,
>>
>> If you are grouping only by "instance" in the Alertmanager, then all 
>> alerts with the same instance label in the same matched route will be 
>> grouped into a single notification. Usually you would have separate routes 
>> for different severities though, which would then also generate separate 
>> groups (even if both group only by instance). But I guess that's not the 
>> case in your case?
>>
>> Regards,
>> Julius
>>
>> On Fri, Sep 23, 2022 at 10:45 AM ishu...@gmail.com  
>> wrote:
>>
>>> Hallo,
>>>
>>> I have an integration between alertmanager and pagerduty which has been 
>>> since couple of years without any issue. 
>>> On alertmanager, we are grouping by instance and when for the same 
>>> instance, there are alerts with different severity (like critical and 
>>> warning), the alertmanager is sending only one alert to pagerduty or may be 
>>> pagerduty might also dropping it. 
>>>
>>> Just wanted to know, how this integration works when alerts with the 
>>> same instance name and two severity levels are triggered. 
>>>
>>> Thanks
>>> Eswar
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Prometheus Users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to prometheus-use...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/prometheus-users/788f29e0-0f58-41cc-81c0-5cb80b375949n%40googlegroups.com
>>>  
>>> 
>>> .
>>>
>>
>>
>> -- 
>> Julius Volz
>> PromLabs - promlabs.com
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/17bde63b-e143-4618-b7bb-8b65b4dc14c7n%40googlegroups.com.

Re: [prometheus-users] is it already a known issue with alertmanager integration to pagerduty

2022-09-26 Thread ishu...@gmail.com 
Hi,

Thanks for the reply.

I have only one route per environment and doesn't have separate routes per 
severity within environment. When, I have multiple alerts for the same 
route with same severity, all ended up in pagerduty as expected. But, when 
I have alerts triggered for the same instance with different severity, then 
I have only one (may be also the first alert) sent to pagerduty and the 
other alerts are although showing up on alertmanager but not reaching 
pagerduty.

Thanks
Eswar
On Friday, 23 September 2022 at 23:26:43 UTC+2 juliu...@promlabs.com wrote:

> Hi Eswar,
>
> If you are grouping only by "instance" in the Alertmanager, then all 
> alerts with the same instance label in the same matched route will be 
> grouped into a single notification. Usually you would have separate routes 
> for different severities though, which would then also generate separate 
> groups (even if both group only by instance). But I guess that's not the 
> case in your case?
>
> Regards,
> Julius
>
> On Fri, Sep 23, 2022 at 10:45 AM ishu...@gmail.com  
> wrote:
>
>> Hallo,
>>
>> I have an integration between alertmanager and pagerduty which has been 
>> since couple of years without any issue. 
>> On alertmanager, we are grouping by instance and when for the same 
>> instance, there are alerts with different severity (like critical and 
>> warning), the alertmanager is sending only one alert to pagerduty or may be 
>> pagerduty might also dropping it. 
>>
>> Just wanted to know, how this integration works when alerts with the same 
>> instance name and two severity levels are triggered. 
>>
>> Thanks
>> Eswar
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Prometheus Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to prometheus-use...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/prometheus-users/788f29e0-0f58-41cc-81c0-5cb80b375949n%40googlegroups.com
>>  
>> 
>> .
>>
>
>
> -- 
> Julius Volz
> PromLabs - promlabs.com
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/5cd4aaa4-0b14-48ef-b67f-04abc0fee4f4n%40googlegroups.com.

Re: [prometheus-users] is it already a known issue with alertmanager integration to pagerduty

2022-09-23 Thread Julius Volz 
Hi Eswar,

If you are grouping only by "instance" in the Alertmanager, then all alerts
with the same instance label in the same matched route will be grouped into
a single notification. Usually you would have separate routes for different
severities though, which would then also generate separate groups (even if
both group only by instance). But I guess that's not the case in your case?

Regards,
Julius

On Fri, Sep 23, 2022 at 10:45 AM ishu...@gmail.com 
wrote:

> Hallo,
>
> I have an integration between alertmanager and pagerduty which has been
> since couple of years without any issue.
> On alertmanager, we are grouping by instance and when for the same
> instance, there are alerts with different severity (like critical and
> warning), the alertmanager is sending only one alert to pagerduty or may be
> pagerduty might also dropping it.
>
> Just wanted to know, how this integration works when alerts with the same
> instance name and two severity levels are triggered.
>
> Thanks
> Eswar
>
> --
> You received this message because you are subscribed to the Google Groups
> "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to prometheus-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/prometheus-users/788f29e0-0f58-41cc-81c0-5cb80b375949n%40googlegroups.com
> 
> .
>


-- 
Julius Volz
PromLabs - promlabs.com

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/CAObpH5zAXPwzpD7C%3D1j90463wyDu0X%2BhKny92drAn7Qx0Pc85w%40mail.gmail.com.

[prometheus-users] is it already a known issue with alertmanager integration to pagerduty

2022-09-23 Thread ishu...@gmail.com 
Hallo,

I have an integration between alertmanager and pagerduty which has been 
since couple of years without any issue. 
On alertmanager, we are grouping by instance and when for the same 
instance, there are alerts with different severity (like critical and 
warning), the alertmanager is sending only one alert to pagerduty or may be 
pagerduty might also dropping it. 

Just wanted to know, how this integration works when alerts with the same 
instance name and two severity levels are triggered. 

Thanks
Eswar

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/788f29e0-0f58-41cc-81c0-5cb80b375949n%40googlegroups.com.