xuanskyer opened a new issue, #190: URL: https://github.com/apache/rocketmq-exporter/issues/190
**Is your feature request related to a problem? Please describe.** All consumer-side metrics are currently aggregated to broker granularity, so there is no way to tell *which queue* of a topic is lagging. When a single queue is stuck (a slow message, a stuck consumer instance, or an uneven rebalance), `rocketmq_group_diff` only shows that the group as a whole is behind — the skew between queues is invisible in Prometheus, and one has to fall back to the console (`Topic -> Consumer detail`) to see per-queue `brokerOffset` / `consumerOffset` / `diff`. Interestingly, the label list for queue granularity already exists in the code but is never used: https://github.com/apache/rocketmq-exporter/blob/master/src/main/java/org/apache/rocketmq/exporter/collector/RMQMetricsCollector.java#L479-L481 ```java private static final List<String> GROUP_PULL_LATENCY_LABEL_NAMES = Arrays.asList( "cluster", "broker", "topic", "group", "queueid" ); ``` **Describe the solution you'd like** The per-queue data is already fetched and iterated in `MetricsCollectTask#collectConsumerOffset`: `consumeStats.getOffsetTable()` is a `Map<MessageQueue, OffsetWrapper>`, and the per-queue `lagTime` is already computed one queue at a time. Both loops then fold the values into `HashMap<String /* brokerName */, Long>` and the `queueId` is dropped. So exposing queue granularity needs no new admin call at all — `queryMsgByOffset` is already invoked once per queue, and the proposal keeps that unchanged. Suggested metrics: | metric | meaning | | --- | --- | | `rocketmq_queue_group_diff` | per-queue unconsumed messages (`brokerOffset - consumerOffset`) | | `rocketmq_queue_group_get_latency_by_storetime` | per-queue consume latency (ms) | | `rocketmq_queue_consumer_offset` | per-queue consumer offset | **Cardinality concern and how it is addressed** Queue-level series multiply the existing consumer series by the number of queues per broker, which is not acceptable to enable globally on a large cluster. The proposal therefore keeps it **off by default** and gated by an explicit topic whitelist: ```yaml rocketmq: config: queueLevelTopics: "topic-a,topic-b" # empty = disabled (default), "*" = all topics ``` Existing broker-level metrics are untouched, so current dashboards and alerts keep working. **Additional context** I have a working implementation (pure addition, no existing line modified, with unit tests for the whitelist parsing) and will open a PR referencing this issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
