Hi Dev, It would be great if anybody share your experience on Search Message by Timestamp ..
Cheer's, Senthil On May 28, 2017 2:08 AM, "SenthilKumar K" <senthilec...@gmail.com> wrote: > Hi Team , Any help here Pls ? > > Cheers, > Senthil > > On Sat, May 27, 2017 at 8:25 PM, SenthilKumar K <senthilec...@gmail.com> > wrote: > >> Hello Kafka Developers , Users , >> >> We are exploring the SearchMessageByTimestamp feature in Kafka for >> our use case . >> >> Use Case : Kafka will be realtime message bus , users should be able >> to pull Logs by specifying start_date and end_date or Pull me last five >> minutes data etc ... >> >> I did POC on SearchMessageByTimestamp , here is the code >> https://gist.github.com/senthilec566/16e8e28b32834666fea132afc3a4e2f9 . >> And i observed that Searching Messages is slow .. >> >> Here is small test i did : >> Query :Fetch Logs of Last *5 minutes*: >> Result: >> No of Records fetched : *30* >> Fetch Time *6210* ms >> >> Above test performed in a topic which has 4 partitions. In each partition >> search & query processing happened .. in other words >> consumer.offsetsForTimes() >> consumer.assign(Arrays.asList(partition)) >> consumer.seek(this.partition, offsetTimestamp.offset()) >> consumer.poll(100) >> >> are the API calls of each partition.. I realized that , this was the >> reason for Kafka taking more time.. >> >> What is efficient way of implementing SerachMessageByTimeStamp ? Is >> Kafka right candidate for our Use Case ? >> >> Pls add your thoughts here ... >> >> >> Cheers, >> Senthil >> > >
