I have few java applications that I'll deploy in my cluster. I need to know how can I detect if a instance is up or down with Prometheus.
*Alerting with AlertManager* I have a alert that check for "instanceDown" and send a alert to AlertManager-webhook. So when one instance is down, i'm receiving alerts in my application. But how can I extract the labels that are in that instance ? ex : I have a special labels in all my application that link the pod to the information that I have in the database releaseUUIDGroup=bf79b8ab-a7c1-4d27-8f3c-6e0f0a089c70 there is a way to add that information in the message that AlertManager send ? right now I configure AlertManager to send the alert to : https://webhook.site/#!/815a0b0b-f40c-4fc2-984d-e29cb9606840/b0dd701d-e972-48d4-9083-385e6a788d55/1 for an example, I kill the pod : prometheus-pushgateway and I received this message : { "receiver": "default-receiver", "status": "resolved", "alerts": [ { "status": "resolved", "labels": { "alertname": "InstanceDown", "instance": "prometheus-pushgateway.default.svc:9091", "job": "prometheus-pushgateway", "severity": "page" }, "annotations": { "description": "prometheus-pushgateway.default.svc:9091 of job prometheus-pushgateway has been down for more than 1 minute.", "summary": "Instance prometheus-pushgateway.default.svc:9091 down" }, "startsAt": "2020-06-19T17:09:53.862877577Z", "endsAt": "2020-06-22T11:23:53.862877577Z", "generatorURL": "http://prometheus-server-57d8dcc67f-qnmkj:9090/graph?g0.expr=up+%3D%3D+0&g0.tab=1";, "fingerprint": "1ed4a1dca68d64fb" } ], "groupLabels": {}, "commonLabels": { "alertname": "InstanceDown", "instance": "prometheus-pushgateway.default.svc:9091", "job": "prometheus-pushgateway", "severity": "page" }, "commonAnnotations": { "description": "prometheus-pushgateway.default.svc:9091 of job prometheus-pushgateway has been down for more than 1 minute.", "summary": "Instance prometheus-pushgateway.default.svc:9091 down" }, "externalURL": "http://localhost:9093";, "version": "4", "groupKey": "{}:{}" } -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/20ec33e0-e9bf-4f2a-b366-092743dad957o%40googlegroups.com.
