[
https://issues.apache.org/jira/browse/KAFKA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389483#comment-14389483
]
Jay Kreps commented on KAFKA-2076:
----------------------------------
Actually rather than trying to ensure we have everything the simple consumer
has our criteria was to make sure we added no api we would regret later. It is
okay to do a first version of the consumer that still has a few gaps (we can
add them in the next version), but if we add something that is half thought out
now we can't remove it later so it will be a permanent blight.
Here is a summary of the prior discussion on this:
1. Currently we have OffsetRequest, our most screwy request. It is
per-partition (you have to route it to the exact correct node to get an
answer). We want to retire this request as it is so hacky.
2. There are many other questions you can ask on a per-partition basis. You
might want to know the log end offset, the size on disk, segment timestamps,
compaction point, etc. One possible generalization of the OffsetRequest would
be PartitionMetadataRequest which would give all this stuff back.
3. Seeking to the beginning and end of the stream are things we can't ship even
a first version of the consumer without, but those are so useful we can just
add special purpose shortcuts (seekToEnd, seekToBeginning) and we won't regret
these as they are the most common usage anyway.
So basically I agree it would be good to add these APIs but we just need to
figure out what the protocol end state and how we will expose this
functionality.
> Add an API to new consumer to allow user get high watermark of partitions.
> --------------------------------------------------------------------------
>
> Key: KAFKA-2076
> URL: https://issues.apache.org/jira/browse/KAFKA-2076
> Project: Kafka
> Issue Type: Improvement
> Reporter: Jiangjie Qin
>
> We have a use case that user wants to know how far it is behind a particular
> partition on startup. Currently in each fetch response, we have high
> watermark for each partition, we only keep a global max-lag metric. It would
> be better that we keep a record of high watermark per partition and update it
> on each fetch response. We can add a new API to let user query the high
> watermark.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
