Hi Becket, Thanks for the write up. I had few minor comments :
1) The wiki says ListOffsetRequest v0 still behaves in the old way. ---> It would be good to include what it means although that's not the part of this KIP. You can add a note what V0 means like : (Kafka allows querying offsets of messages by time and it does so at segment granularity. The timestamp parameter is the unix timestamp and querying the offset by timestamp returns the latest possible offset of the message that is appended no later than the given timestamp. There are 2 special values of the timestamp - latest and earliest. For any other value of the unix timestamp, Kafka will get the starting offset of the log segment that is created no later than the given timestamp. Due to this, and since the offset request is served only at segment granularity, the offset fetch request returns less accurate results for larger segment sizes.) This will be easier for first time reader to just understand the difference between old and new without having to dig through code or refer online docs. 2) For the point : In a multi-datacenter enviroment, users may have different Kafka clusters in each datacenter for disater recovery. If one of the datacenter failed, the applications may need to switch the consumption from one data center to another datacenter. Because the offset between two different Kafka clusters are independent, users cannot use the offsets from the failed datacenter to consume from the DR datacenter. In this case, searching by timestamp will help because the messages should have the same timestamp. ---> This may not be true always right because it depends upon how the timestamp is created like if its the create time of the message or the file append time. So if its file append time, it will be different on two different broker machines right? In that case the offsets for the same timestamp will be different on the two broker machines right? 3) Can describe how the “TimestampOffset” class looks like? Nice wiki ! Thanks, Mayuresh On Tue, Aug 30, 2016 at 8:38 PM, Becket Qin <becket....@gmail.com> wrote: > Hi Kafka devs, > > I created KIP-79 to allow consumer to precisely query the offsets based on > timestamp. > > In short we propose to : > 1. add a ListOffsetRequest/ListOffsetResponse v1, and > 2. add an offsetForTime() method in new consumer. > > The KIP wiki is the following: > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65868090 > > Comments are welcome. > > Thanks, > > Jiangjie (Becket) Qin > -- -Regards, Mayuresh R. Gharat (862) 250-7125
