[
https://issues.apache.org/jira/browse/KAFKA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389700#comment-14389700
]
Jay Kreps commented on KAFKA-2076:
----------------------------------
I guess the question we need to answer is whether we will want to add other
offset-related APIs, in particular:
- get the map of known time->offset
- get the least offset
I think if we have a picture for where we want to go with this it will be
easier to know what to do now for getting the last offset.
If we want to do both of these things then maybe the api should be
{code}
PartitionOffsets offsetTimes(TopicPartition tp)
{code}
where PartitionOffsets would basically be SortedMap<Long, Long> mapping
timestamp to offset (with some additional helper methods to make usage clearer).
A separate question is whether the api needs to be batched or not--this should
ideally be as consistent as possible across the methods. Many of the other
methods don't need to batch as they refresh all subscribed partitions whenever
you ask for any one, so the actual lookup is just in-memory. But this approach
may not work here as I think we are saying we want you to be able to check
non-subscribed partitions.
Thoughts?
> Add an API to new consumer to allow user get high watermark of partitions.
> --------------------------------------------------------------------------
>
> Key: KAFKA-2076
> URL: https://issues.apache.org/jira/browse/KAFKA-2076
> Project: Kafka
> Issue Type: Improvement
> Reporter: Jiangjie Qin
>
> We have a use case that user wants to know how far it is behind a particular
> partition on startup. Currently in each fetch response, we have high
> watermark for each partition, we only keep a global max-lag metric. It would
> be better that we keep a record of high watermark per partition and update it
> on each fetch response. We can add a new API to let user query the high
> watermark.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
