Re: Changing Prometheus rules

2019-11-20 Thread Simon Pasquier
er-monitoring-operator was restarted. >> >> >> >> Looks like I'll have to live with silencing the alert. >> >> >> >> On 19/11/2019 07:56, Vladimir REMENAR wrote: >> >> >> >> Hi Tim, >> >> >> >> You need to

Re: Changing Prometheus rules

2019-11-20 Thread Mateus Caruccio
p. If cluster-monitoring-operator is running while editing > configmap it will always revert it to default. > >> > >> > >> Uz pozdrav, > >> Vladimir Remenar > >> > >> > >> > >> From:Tim Dudgeon > >> To

Re: Changing Prometheus rules

2019-11-20 Thread Simon Pasquier
configmap. >> If cluster-monitoring-operator is running while editing configmap it will >> always revert it to default. >> >> >> Uz pozdrav, >> Vladimir Remenar >> >> >> >> From:Tim

Re: Changing Prometheus rules

2019-11-19 Thread Mateus Caruccio
* > > > > From:Tim Dudgeon > To:Simon Pasquier > Cc: users > > Date:18.11.2019 17:46 > Subject:Re: Changing Prometheus rules > Sent by:users-boun...@lists.openshift.redhat.com > -- >

Re: Changing Prometheus rules

2019-11-19 Thread Tim Dudgeon
than and then edit configmap. If cluster-monitoring-operator is running while editing configmap it will always revert it to default. Uz pozdrav, *Vladimir Remenar* From: Tim Dudgeon To: Simon Pasquier Cc: users Date: 18.11.2019 17:46 Subject: Re: Changing Prometheus rules Sent by: users

Re: Changing Prometheus rules

2019-11-19 Thread Simon Pasquier
There's also a plan to improve that specific alert: https://bugzilla.redhat.com/show_bug.cgi?id=1743911 On Tue, Nov 19, 2019 at 9:15 AM Simon Pasquier wrote: > > If muting alerts in Alertmanager is enough (eg they would still fire > in Prometheus), you could inhibit alerts in the Alertmanager >

Re: Changing Prometheus rules

2019-11-19 Thread Simon Pasquier
If muting alerts in Alertmanager is enough (eg they would still fire in Prometheus), you could inhibit alerts in the Alertmanager configuration. inhibition_rules: - target_match: alertname: KubeAPILatencyHigh source_match: alertname: DeadMansSwitch On Mon, Nov 18, 2019 at 5:46 PM Tim

Re: Changing Prometheus rules

2019-11-18 Thread Vladimir REMENAR
: 18.11.2019 17:46 Subject:Re: Changing Prometheus rules Sent by:users-boun...@lists.openshift.redhat.com The KubeAPILatencyHigh alert fires several times a day for us (on 2 different OKD clusters). On 18/11/2019 15:17, Simon Pasquier wrote: > The Prometheus instances deplo

Re: Changing Prometheus rules

2019-11-18 Thread Tim Dudgeon
The KubeAPILatencyHigh alert fires several times a day for us (on 2 different OKD clusters). On 18/11/2019 15:17, Simon Pasquier wrote: The Prometheus instances deployed by the cluster monitoring operator are read-only and can't be customized.

Re: Changing Prometheus rules

2019-11-18 Thread Simon Pasquier
The Prometheus instances deployed by the cluster monitoring operator are read-only and can't be customized. https://docs.openshift.com/container-platform/3.11/install_config/prometheus_cluster_monitoring.html#alerting-rules_prometheus-cluster-monitoring Can you provide more details about which

Changing Prometheus rules

2019-11-18 Thread Tim Dudgeon
What is the "right" way to edit Prometheus rules that are deployed by default on OKD 3.11? I have alerts that are annoyingly noisy, and want to silence them forever! I tried editing the definition of the PrometheusRule CRD and/or the prometheus-k8s-rulefiles-0 ConfigMap in the