unbridled-41 opened a new issue, #5456:
URL: https://github.com/apache/rocketmq-dashboard/issues/5456

   ### Studio Version
   branch: `rocketmq-studio` @ `6a68042f` (the revision verified; the branch is 
based on it)
   deployed as: built from source (unit test)
   
   ### Runtime Environment
   
   Not environment-specific: the defect is in a bundled Prometheus rule file, 
verified with a unit test.
   
   ### Connected RocketMQ Cluster
   
   Not cluster-specific.
   
   ### Describe the Bug
   
   The bundled `RocketMQDLQResendHigh` rule 
(`server/src/main/resources/alerts/rocketmq-dlq-resend-high.yaml:9`, generated 
by `server/scripts/gen_alert_rule_yaml.py`) annotates itself as "More than 10 
dead-letter queue resends occurred in 5 minutes", but thresholds with a 
per-second rate:
   
   ```
   rate(rocketmq_dlq_resend_count[5m]) > 10
   ```
   
   `rate()` returns the average increase per second, so the rule needs about 10 
resends **per second** — roughly 3000 inside the documented 5-minute window — 
before it fires.
   
   ### Steps to Reproduce
   
   1. Read the asset (or export the default rule set through the alert rules 
API).
   2. Compare the expression with the annotation on the same rule, and with the 
sibling rule for the same "more than N in a window" contract, 
`rocketmq-consumer-rebalance.yaml`, which uses 
`increase(rocketmq_consumer_rebalance_times[5m]) > 10`.
   
   ### What Did You Expect to See?
   
   The rule fires when more than 10 dead-letter resends happen in 5 minutes, as 
its annotation says.
   
   ### What Did You See Instead?
   
   It requires 10 resends per second, so a DLQ replaying tens of messages per 
minute stays silent.
   
   ### Evidence
   
   - Asset `server/src/main/resources/alerts/rocketmq-dlq-resend-high.yaml:9`; 
generator `server/scripts/gen_alert_rule_yaml.py` (entry for 
`rocketmq-dlq-resend-high`).
   - `cd server && mvn -o test -Dtest=AlertRuleAssetServiceTest` — the added 
case `dlqResendAlertShouldCountResendsOverTheDocumentedWindow` fails against 
`rate(...)` and passes against `increase(...)`; regenerating all 23 assets 
reproduces the checked-in directory byte for byte.
   
   ### Impact
   
   An operator who deploys the exported default rule set gets a DLQ alert whose 
sensitivity is ~300× off its documentation: the alert fires far later than 
intended.
   
   ### Acceptance Criteria
   
   - The bundled rule counts resends over the documented window.
   - A regression test pins the expression.
   
   **Corresponding PR:** #5420
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to