Re: [prometheus-users] Re: Mutiple PromQLs for each expr in alert rules

2021-12-08 Thread Brian Candler
As I said before: if you can show real examples of the complete metrics, with their full sets of labels, then we may be able to help. What do I mean by this? For example, if I go into the PromQL web interface and type "node_filesystem_avail_bytes" as the query, I get the following:

Re: [prometheus-users] Re: Mutiple PromQLs for each expr in alert rules

2021-12-07 Thread nina guo
Could you provide some help on this topic? On Tuesday, December 7, 2021 at 10:17:59 AM UTC+8 nina guo wrote: > in container_cpu_usage_seconds_total, there are following labels: > instance > name > kubernetes_io_arch > hostname > job > cpu (the value of this label is "total") > > > On Tuesday,

Re: [prometheus-users] Re: Mutiple PromQLs for each expr in alert rules

2021-12-06 Thread nina guo
in container_cpu_usage_seconds_total, there are following labels: instance name kubernetes_io_arch hostname job cpu (the value of this label is "total") On Tuesday, December 7, 2021 at 10:15:15 AM UTC+8 nina guo wrote: > Thank you. > > in container_cpu_usage_seconds_total, there are following

Re: [prometheus-users] Re: Mutiple PromQLs for each expr in alert rules

2021-12-06 Thread nina guo
Thank you. in container_cpu_usage_seconds_total, there are following labels: instance name kubernetes_io_arch hostname job On Monday, December 6, 2021 at 5:54:08 PM UTC+8 Stuart Clark wrote: > On 06/12/2021 08:43, nina guo wrote: > > May I ask how to check all the available labels for this?

Re: [prometheus-users] Re: Mutiple PromQLs for each expr in alert rules

2021-12-06 Thread Stuart Clark
On 06/12/2021 08:43, nina guo wrote: May I ask how to check all the available labels for this? Can we define new labels? For instances, we use file discovery to get the monitored target list. Just enter the metric name in the query page of the Prometheus UI and you'll see all the different

Re: [prometheus-users] Re: Mutiple PromQLs for each expr in alert rules

2021-12-06 Thread nina guo
May I ask how to check all the available labels for this? Can we define new labels? For instances, we use file discovery to get the monitored target list. On Monday, December 6, 2021 at 4:29:27 PM UTC+8 Stuart Clark wrote: > On 06/12/2021 08:19, nina guo wrote: > > Thank you for your reply. >

Re: [prometheus-users] Re: Mutiple PromQLs for each expr in alert rules

2021-12-06 Thread Stuart Clark
On 06/12/2021 08:19, nina guo wrote: Thank you for your reply. Let me clarify what we are going to do more. We have the following alert rules for a container. But we still want to include the CPU usage of the corresponding host which host this container. That is:  - when CPU usage for

[prometheus-users] Re: Mutiple PromQLs for each expr in alert rules

2021-12-06 Thread nina guo
Thank you for your reply. Let me clarify what we are going to do more. We have the following alert rules for a container. But we still want to include the CPU usage of the corresponding host which host this container. That is: - when CPU usage for container is larger than 85%, and CPU usage

[prometheus-users] Re: Mutiple PromQLs for each expr in alert rules

2021-12-06 Thread Brian Candler
Yes, but you need to be really clear on what you're doing. What you need to realise is that this is a *vector* expression: expr: containerCPUusage > 80 This may return zero, one, or many results in a vector: that is, a set of all timeseries with metric name "containerCPUusage" that meet