I've got a similar problem to this unanswered SO issue being posed here:
https://stackoverflow.com/questions/43400092/query-for-the-same-time-period-of-every-day-in-prometheus I need to produce a table report in Grafana (not via ad-hoc API queries to Prometheus) that looks something like this: ``` Average CPU Utilization by Namespace Namespace | 12am-6am 6am-12pm 12pm-6pm 6pm-12am --------- | ----------------------------------------- ns1 | 12.4 33.4 43.6 56.7 ns2 | 12.3 56.7 45.6 89.6 ns3 | 15.7 45.6 23.5 23.8 ... ``` The basi query I'm using is: avg_over_time(namespace_name:container_cpu_usage_seconds_total:sum_rate[6h]) Which is an out of the box K8S recording rule: rules: - expr: | sum(rate(container_cpu_usage_seconds_total{job="kubelet", image!=""}[5m])) by (namespace) record: namespace:container_cpu_usage_seconds_total:sum_rate What I think I need is something like this that will give me the average over time for only those elements in the series selected by the user in the Grafana UI that are recorded between 12am and 6am EST: avg_over_time(namespace_name:container_cpu_usage_seconds_total:sum_rate[12am-6am]) That is, if the user has "last 24 hours" selected in the Grafana UI, there will be one contiguous range of data in the series, and if they have "last 48 hours" selected, there will be two 12am-6pm series in the range to give me the avg_over_time on. Of course, "query['time of day start'-'time of day end']" appears not to be valid in PromQL. I thought about using the offset function, but that would require that the user set a specific time in the UI range, which is not what I want. I want something like "the average of all values over the past week (or whatever the user has selected in the UI) but only among those scraped between 12am and 6am no matter what day those were recorded on". Is this doable with Prometheus+Grafana and if so, how? Thanks in advance. -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/fa1233a8-6f7c-4a22-9ee2-764e09d91ccc%40googlegroups.com.
