mjsax commented on PR #22614: URL: https://github.com/apache/kafka/pull/22614#issuecomment-5007511775
Thanks. I guess both a very close, but could still differ, with `records-lag` being ahead, ie smaller -- it believe it's computed based on "fetch lag", ie, each time a fetch request returns, but the data won't be returned to application yet nor would it be processed already. The KS metric `restore-remaining-records-total` is updated only after records are applied to the store, so it's more accurate. `records-lag` could become zero before the restore actually finished. (Even if the delay should be small in practice?) Not that we have a record buffer inside KS as well, in which we cache record for restoration, potentially wide the gap between both metric (Cf https://cwiki.apache.org/confluence/spaces/KAFKA/pages/421957079/KIP-1325+Add+restore.buffered.records.per.partition+to+cap+Kafka+Streams+state-restore+memory) I believe the main reason we add the KS one is convenience. We have also seen used who "struggle" to use consumer metrics and/or "connect the dots" that they could monitor the consumer-lag metric to track restoration progress. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
