janiussyafiq commented on issue #13877: URL: https://github.com/apache/apisix/issues/13877#issuecomment-5419024069
Reproduced on `apache/apisix:3.18.0-debian` with your exact `pluginAttrs`: the `node` label is still there because `plugin_attr.prometheus.metrics.<metric>.labels` is not a supported key. The only key read under `metrics.<metric>` is `extra_labels`, and unknown keys in `plugin_attr` are silently ignored. The supported way is Plugin Metadata `disabled_labels` (#13202, included in 3.18.0). It takes effect at runtime, no restart: ```bash curl "http://127.0.0.1:9180/apisix/admin/plugin_metadata/prometheus"; -X PUT \ -H "X-API-KEY: ${ADMIN_API_KEY}" \ -d '{"disabled_labels":{"http_status":["node","matched_uri","matched_host"]}}' ``` After this, new samples look like `apisix_http_status{...,matched_uri="",matched_host="",node="",...}`. The label name stays with an empty value by design (existing dashboards and alerts keep working); the cardinality drops because all `node` values collapse into one series. Series recorded before the change remain until they expire or APISIX restarts. You can refer to this doc for more clarification: https://apisix.apache.org/docs/apisix/plugins/prometheus/#reduce-metric-cardinality-by-disabling-labels Side note: `prefer_name` is a plugin attribute set on the route or global rule (`"plugins":{"prometheus":{"prefer_name":true}}`), not a `plugin_attr` key, so the one in your `pluginAttrs` is ignored as well. You can drop both `metrics.http_status.labels` and `prefer_name` from `pluginAttrs`. Let me know if this resolves your issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
