Question about Lag Calculations in Apache Kafka Source Code

2023-07-25 Thread Henry GALVEZ
Hi everyone, I am interested in understanding how the broker performs lag calculations. Specifically, I would like to explore the possibility of improving the calculation method to use the latest stable offset instead of the latest offset. I noticed that there might be differences between the

RE: Offsets: consumption and production in rollback

2023-06-29 Thread Henry GALVEZ
o gaps when transactions roll back. I design my consumers so they are not surprised by the gaps and they don’t try to calculate the number of records. Instead, they just continually consume. Hope this helps, Andrew > On 28 Jun 2023, at 09:28, Henry GALVEZ wrote: > > Hi Andrew, > &g

RE: Offsets: consumption and production in rollback

2023-06-28 Thread Henry GALVEZ
m not sure that this is really “bogus lag” but, when you’re using transactions, there’s not a one-to-one relationship between offsets and consumable records. Hope this helps, Andrew Begin forwarded message: From: Henry GALVEZ Subject: Offsets: consumption and production in rollback Date: 27 June 2023 at 10

Offsets: consumption and production in rollback

2023-06-27 Thread Henry GALVEZ
I have some doubts regarding message consumption and production, as well as transactional capabilities. I am using a Kafka template to produce a message within a transaction. After that, I execute another transaction that produces a message and intentionally throws a runtime exception to