Ok, thanks for confirming my suspicion 👍 On Wednesday, November 22, 2023 at 1:01:51 PM UTC+1 Bryan Boreham wrote:
> Federation is a bit of a neglected feature. The Thanos project is rather > more popular as a way to aggregate data from multiple Prometheus. > > (Other projects also exist which can let you centralise metrics storage) > > Bryan > > On Thursday, 16 November 2023 at 15:51:12 UTC hannesst...@gmail.com wrote: > >> I take the silence as a "no, it is not possible to connect recording >> rules with federation requests (or vice versa)" ...? >> >> On Tuesday, October 31, 2023 at 5:54:05 PM UTC+1 hannesst...@gmail.com >> wrote: >> >>> *Is it possible to include an aggregation formula in a federation query?* >>> This would avoid creating and storing aggregated metrics on the >>> lower-level Prometheus, and also remove the delay between aggregation and >>> federation. >>> >>> *Or, can a recording rule fetch metrics directly from another Prometheus >>> server?* >>> This would avoid the delay between federation and execution of the >>> recording rule. >>> >>> We've got hundreds of pods, delivering millions of metrics. We plan to >>> partition our pods and deploy one Prometheus per partition. A top-level >>> Prometheus will then offer globally aggregated metrics. >>> >>> How should we set this up? My current assumption is based on aggregation >>> recording rules within each partition, then again at the top-level to get >>> the global aggregation. This seems both complicated and a waste of >>> resources, plus also introduces delays since recording rules and federation >>> cannot be synced to each other. >>> >>> To minimize delay, each partition-level Prometheus needs to aggregate as >>> often as possible to offer "fresh" metrics to federation requests. Then the >>> top level Prometheus also needs to federate these aggregated metrics as >>> often as possible to offer "fresh" values at the global level. Doing things >>> as often as possible "just in case" seems wasteful, which is why I am >>> asking if this is the right approach for us. >>> >> -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/ab32c10a-eb9d-4867-bb6f-de47c342983bn%40googlegroups.com.
