Re: [prometheus-users] Alerting in Prometheus federated environment.

2021-01-17 Thread Julius Volz
Another option: if you cannot reach Slack from the individual environments, but you can provision some other global endpoint that would be reachable from them, you could have the local Alertmanagers send all of their alerts to a webhook receiver ( https://prometheus.io/docs/alerting/latest/configur

Re: [prometheus-users] Alerting in Prometheus federated environment.

2021-01-17 Thread Julius Volz
s/each location's Alertmanager/each location's Prometheus/ On Sun, Jan 17, 2021 at 10:40 AM Julius Volz wrote: > Hi, > > Indeed that's not ideal and annotations and such will only be available in > the source Prometheus servers (and then pushed to the Alertmanagers). I'm > not sure what exactly

Re: [prometheus-users] Alerting in Prometheus federated environment.

2021-01-17 Thread Julius Volz
Hi, Indeed that's not ideal and annotations and such will only be available in the source Prometheus servers (and then pushed to the Alertmanagers). I'm not sure what exactly your network constraints are, but they seem to prohibit Alertmanager in each location talking directly to Slack. Would they

[prometheus-users] Alerting in Prometheus federated environment.

2021-01-15 Thread 'Dennis Murtic' via Prometheus Users
Hi all, Currently we have different environments where Prometheus and Alertmanager are used. These environments are also protected by certain network settings, so we need to use PushProx. To collect data from all environments, we use a "global" Prometheus server that federates with the environment