Hi Mayuresh, Thanks for the comments.
1) I added the link to the Kafka protocol guide. 2) True that for LogAppendTime the message timestamp in different cluster will not be the same. I updated the wiki. But notice that even the LogAppendTime on different clusters are still comparable given a reasonable delay bound. This use case was discussed in KIP-31/32. 3) TimestampOffset is just a simple container class of Timestamp and Offset. Thanks, Jiangjie (Becket) Qin On Wed, Aug 31, 2016 at 1:38 PM, Mayuresh Gharat <gharatmayures...@gmail.com > wrote: > Hi Becket, > > Thanks for the write up. I had few minor comments : > > 1) The wiki says ListOffsetRequest v0 still behaves in the old way. > ---> It would be good to include what it means although that's not the part > of this KIP. > You can add a note what V0 means like : > (Kafka allows querying offsets of messages by time and it does so at > segment granularity. The timestamp parameter is the unix timestamp and > querying the offset by timestamp returns the latest possible offset of the > message that is appended no later than the given timestamp. There are 2 > special values of the timestamp - latest and earliest. For any other value > of the unix timestamp, Kafka will get the starting offset of the log > segment that is created no later than the given timestamp. Due to this, and > since the offset request is served only at segment granularity, the offset > fetch request returns less accurate results for larger segment sizes.) > > This will be easier for first time reader to just understand the difference > between old and new without having to dig through code or refer online > docs. > > > 2) For the point : > In a multi-datacenter enviroment, users may have different Kafka clusters > in each datacenter for disater recovery. If one of the datacenter failed, > the applications may need to switch the consumption from one data center to > another datacenter. Because the offset between two different Kafka clusters > are independent, users cannot use the offsets from the failed datacenter to > consume from the DR datacenter. In this case, searching by timestamp will > help because the messages should have the same timestamp. > ---> This may not be true always right because it depends upon how the > timestamp is created like if its the create time of the message or the file > append time. So if its file append time, it will be different on two > different broker machines right? In that case the offsets for the same > timestamp will be different on the two broker machines right? > > 3) Can describe how the “TimestampOffset” class looks like? > > > Nice wiki ! > > Thanks, > > Mayuresh > > On Tue, Aug 30, 2016 at 8:38 PM, Becket Qin <becket....@gmail.com> wrote: > > > Hi Kafka devs, > > > > I created KIP-79 to allow consumer to precisely query the offsets based > on > > timestamp. > > > > In short we propose to : > > 1. add a ListOffsetRequest/ListOffsetResponse v1, and > > 2. add an offsetForTime() method in new consumer. > > > > The KIP wiki is the following: > > https://cwiki.apache.org/confluence/pages/viewpage. > action?pageId=65868090 > > > > Comments are welcome. > > > > Thanks, > > > > Jiangjie (Becket) Qin > > > > > > -- > -Regards, > Mayuresh R. Gharat > (862) 250-7125 >
