On Wed, 14 Oct 2020 at 02:32, Peter Bourgon <pe...@bourgon.org> wrote:
> The fastly-exporter [0] converts statistics scraped from the Fastly > real-time stats API to metrics that can be scraped by Prometheus. When > configured with many services that receive a lot of global traffic, > the metric cardinality can grow quite large. And I'll soon add a new > data source that will potentially ~double metric cardinality again. > > The exporter can already be configured to export only e.g. specific > metrics. But users are suggesting it would be good to be able to > manipulate the exported metrics in a more powerful way. For example, > they'd like a way to define regions as sets-of-datacenters (a label > applied to all metrics) and then "collapse" (merge) the metrics so > that the datacenter labels are erased and the region label applied. > Or, to define feature groups as sets-of-metric-names, and then turn > entire feature groups on or off. Importantly, this should all be done > within the exporter itself, not via Prometheus relabeling rules, > because we're explicitly trying to reduce load on the Prometheus > servers scraping the exporter. > > I can come up with a little DSL or config format that could get the > job done. But is there some prior art here that's been successful? > I'm not aware of anything generic in this space. Disabling specific expensive collectors via flags isn't unusual, both the mysqld and node exporters have it for example. There's also a handful of things out there where you exclude labels that some metrics would usually have, e.g. removing the datacenter label. Dynamically adding in a region label is a bit odd though, as new instrumentation labels are a breaking change for downstream. If that's a useful label it should be there in the first place, not only something that appears when you're trying to reduce cardinality. I'd suggest keeping with simple flags over creating yet another language that users to have to learn. Brian > > [0] https://github.com/peterbourgon/fastly-exporter > > -- > You received this message because you are subscribed to the Google Groups > "Prometheus Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to prometheus-users+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/CAFm1DRFpP5_5XF7DRXD%3DnjqsHEwE1%3DAU70tccj5UDH2JiSP_hA%40mail.gmail.com > . > -- Brian Brazil www.robustperception.io -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAHJKeLogvnTsjJyxU17UTY2kOMuXFR2DZ6D5zwpY-sLTwXLypA%40mail.gmail.com.
