On 2020-07-29 09:19, Nabarun Sen wrote:
I have three replica now for alertmanager
NAME READY
STATUS RESTARTS AGE IP NODE NOMINATED NODE
READINESS GATES
alertmanager-prometheus-operator-alertmanager-0 2/2
Running 0 113s 10.233.110.73 worker1 <none>
<none>
alertmanager-prometheus-operator-alertmanager-1 2/2
Running 0 109s 10.233.103.120 worker2 <none>
<none>
alertmanager-prometheus-operator-alertmanager-2 2/2
Running 0 109s 10.233.110.72 worker1 <none>
<none>
Ahh. You are using Kubernetes. Is this via Prometheus Operator or just
running Alertmanager independently?
Can you list the command line being run for each of those pods?
root@ansible-node .kube]# kubectl get svc -n ns-prometheus-lab|grep
alertmanager
alertmanager-operated ClusterIP None
<none> 9093/TCP,9094/TCP,9094/UDP 11m
prometheus-operator-alertmanager NodePort
10.233.6.218 <none> 9093:31570/TCP 11m
[root@ansible-node .kube]#
I can get response from telnet inside one alertmanager pod to
alertmanager service
[root@ansible-node prometheus-operator-qa-lab-final]# kubectl exec -it
alertmanager-prometheus-operator-alertmanager-0 alertmanager -n
ns-prometheus-lab -- sh
Defaulting container name to alertmanager.
Use 'kubectl describe
pod/alertmanager-prometheus-operator-alertmanager-0 -n
ns-prometheus-lab' to see all of the containers in this pod.
/alertmanager $ telnet 10.233.6.218:9093
Connected to 10.233.6.218:9093
On Wednesday, 29 July 2020 15:49:06 UTC+8, Nabarun Sen wrote:
Hi All,
I need a help as my alertmanager sending duplicate alert( for 1
alert 3 notification) to webhook receiver when we set alertmanager
replica to 3 but when I put alertmanager replica 1 it is sending one
notofication.
Please let me know if anyone faced similar issue.
Alertmanager configuration:
global:
resolve_timeout: 5m
route:
receiver: 'webhook_receiver'
group_wait: 30s
group_interval: 10h
repeat_interval: 24h
routes:
- receiver: 'webhook_receiver'
match_re:
alertname: NodedownAlert|ServiceDown
group_wait: 30s
group_interval: 10h
repeat_interval: 24h
group_by: ['description']
receivers:
- name: webhook_receiver
webhook_configs:
- send_resolved: true
url:
http://prometheus-webhook.monitoring:8080/v1/webhook?group=operation
[1]
templates:
- '/etc/alertmanager/config/notification.tmpl'
Thanks
Nabarun Sen
--
You received this message because you are subscribed to the Google
Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/5034fe12-8931-4659-b40d-0d697b644bd2o%40googlegroups.com
[2].
Links:
------
[1]
http://prometheus-webhook.monitoring:8080/v1/webhook?group=operation
[2]
https://groups.google.com/d/msgid/prometheus-users/5034fe12-8931-4659-b40d-0d697b644bd2o%40googlegroups.com?utm_medium=email&utm_source=footer
--
Stuart Clark
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/b2e907fd6d91d2242338d944ed822be0%40Jahingo.com.
