Heya Gantigmaa, Apologies for the (very) late reply!
Now that 3.6 has been released and reviewers have a bit more time I will be picking up this KIP again. I am more than happy to add useful new metrics to the KIP, I would just ask for a couple of days to review your pull request and I will come back to you. Best, Christo On Mon, 25 Sept 2023 at 10:49, Gantigmaa Selenge <gsele...@redhat.com> wrote: > Hi Christo, > > Thank you for writing the KIP. > > I recently raised a PR to add metrics for tracking remote segment deletions > (https://github.com/apache/kafka/pull/14375) but realised those metrics > were not mentioned in the original KIP-405 or KIP-930. Do you think these > would make sense to be added to this KIP and get included in the > discussion? > > Regards, > Gantigmaa > > On Wed, Aug 9, 2023 at 1:53 PM Christo Lolov <christolo...@gmail.com> > wrote: > > > Heya Kamal, > > > > Thank you for going through the KIP and for the question! > > > > I have been thinking about this and as an operator I might find it the > most > > useful to know all three of them actually. > > > > I would find knowing the size in bytes useful to determine how much disk > I > > might need to add temporarily to compensate for the slowdown. > > I would find knowing the number of records useful, because using the > > MessagesInPerSec metric I would be able to determine how old the records > > which are facing problems are. > > I would find knowing the number of segments useful because I would be > able > > to correlate this with whether I need to change > > *remote.log.manager.task.interval.ms > > <http://remote.log.manager.task.interval.ms> *to a lower or higher > value. > > > > What are your thoughts on the above? Would you find some of them more > > useful than others? > > > > Best, > > Christo > > > > On Tue, 8 Aug 2023 at 16:43, Kamal Chandraprakash < > > kamal.chandraprak...@gmail.com> wrote: > > > > > Hi Christo, > > > > > > Thanks for the KIP! > > > > > > The proposed tiered storage metrics are useful. The unit mentioned in > the > > > KIP is the number of records. > > > Each topic can have varying amounts of records in a segment depending > on > > > the record size. > > > > > > Do you think having the tier-lag by number of segments (or) size of > > > segments in bytes will be useful > > > to the operator? > > > > > > Thanks, > > > Kamal > > > > > > On Tue, Aug 8, 2023 at 8:56 PM Christo Lolov <christolo...@gmail.com> > > > wrote: > > > > > > > Hello all! > > > > > > > > I would like to start a discussion for KIP-963: Upload and delete lag > > > > metrics in Tiered Storage ( > > https://cwiki.apache.org/confluence/x/sZGzDw > > > ). > > > > > > > > The purpose of this KIP is to introduce a couple of metrics to track > > lag > > > > with respect to remote storage from the point of view of Kafka. > > > > > > > > Thanks in advance for leaving a review! > > > > > > > > Best, > > > > Christo > > > > > > > > > >
