Hi William, I missed Pierre's email. I'm CCing him.
Le 18/11/2019 à 21:00, William Dauchy a écrit :
Thanks. Having a way to filter metrics in the Prometheus exporter was on my todo-list :) Filtering on scopes is pretty simple and it is a good start to solve performance issues for huge configs. It is a good idea. However, IMHO, it is easier to use the query-string to select exported metrics. I don't know if it is compatible with the way Prometheus is used. For instance, based on your idea, to get only metrics about servers, the URI could be "/metric?scope=server". And to get metrics about frontends and backends, it could be "/metric?scope=frontend&scope=backend". Of course, it is extensible. We can imagine to add filters on frontend/backend/server names. Here is a quick patch based on this. What do you think about it ? If you said me your way to select metrics is better from the Prometheus point of view, I'm ok with that.In fact I even did not thought about it because my state of mind was to filter at startup like we were doing before. I like you propostion as it is way more extensible than mine. I was not even thinking it would be possible to access the URL, I now have a lot of ideas of the future ;) example config would look like: - job_name: 'lb-builtin-metrics' metrics_path: '/metrics?scope=backend' consul_sd_configs: - server: localhost:8500 services: - lb-builtin-exporter relabel_configs: - source_labels: ['__meta_consul_dc'] target_label: 'dc' - source_labels: ['__address__', '__meta_consul_service_id'] target_label: 'instance' If you have other things on your todo list regarding that feel free to share so we might share the work around it.
Well, for now, there is nothing concrete. But I've planned to add a filter on frontend/backend names and, if possible, a way to list the metrics to export. The scope idea may be used too for this last one, by grouping metrics by categories (http, conn, timers, checks, rates...). In the end, it's more a question of time than anything else :)
Also have you seen the message from Pierre? They are a few things we would like to discuss whether it would be possible to aggregate a few things on backend side. https://www.mail-archive.com/haproxy@formilux.org/msg35369.html (we somehow forgot to put you in copy though)
Regarding the problem of servers in maintenance, since we parse the query-string, it is possible to add more filters. I may add a parameter to filter out servers in maintenance. For instance, by passing "no-maint" in the query-string, all servers in maintenance could be ignored. This way, it would still be possible to have servers metrics.
For others points, I will reply to the previous email.
Thank you for this patch. Do you think it could be acceptable to mark it as a possible backport for v2.0.x? It's quite critical on our side as we are dealing with ~130MB on some cases. If not we will do the backport on our wide while waiting for the v2.1.
It can safely be backported to 2.0. It is not a critical component and it is independent of other parts.
I will push a test based on your patch tomorrow on our side.
Ok, let me know if you encounter any issues. Thanks, -- Christopher Faulet
