yuqi1129 opened a new issue, #12419:
URL: https://github.com/apache/gravitino/issues/12419

   ### Describe the subtask
   
   `EntityChangeLogPoller` currently exposes no metrics — there is no gauge, 
counter, or timer anywhere in the class. Its failure mode is silent: if polling 
stalls, every node keeps serving stale cached entities and nothing surfaces the 
problem. Operators have no signal to alert on and no value to look at when 
diagnosing a stale read.
   
   The multi-node design doc states the poller "must be watched for lag", but 
there is currently nothing to watch.
   
   Metrics to expose (via the existing `MetricsSource` infrastructure):
   
   - **lag** — `max(entity_change_log.id) - entityPollHighWaterId`, the primary 
alerting signal
   - **current cursor** — `entityPollHighWaterId`
   - **time since last successful poll** — catches a stalled or dead scheduler 
thread
   - **listener failures** — counter per listener, plus whether a batch is 
currently paused for retry (`pendingDelivery` non-null) and its attempt count
   - **batch size / poll duration** — useful for tuning `pollIntervalSecs`
   
   This is small and self-contained, and pairs with the two-node integration 
test as the other prerequisite for enabling multi-node with confidence.
   
   ### Parent issue
   
   https://github.com/apache/gravitino/issues/11737
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to