[
https://issues.apache.org/jira/browse/KAFKA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191350#comment-13191350
]
Jun Rao commented on KAFKA-238:
-------------------------------
Some comments:
1. Use the new Logger helper for new log4j logging.
2. PartitionMetaData: The leader indicator just needs 1 byte, instead of 2.
3. AdminUtils.getTopicMetaDataFromZK(): Instead of getting broker info from ZK
each time, could we collect a set of unique broker ids that are needed and call
ZK only once for each unique id to get the broker info?
3. LogMetaData: LogMetaData is actually a replica level info, not a partition
level info. Collecting such data in the same getGetMetaData api is tricky since
log meta data is only available at the broker that stores a replica locally.
So, this info is not available at every broker, you have to talk to the right
broker to get such information. One possibility is to keep this as a separate
api, something like getReplicaMetadata(topic, partition, brokerid). Such
information is only returned if the api is called on the broker that owns the
partition.
4. SyncProducerTest: no need to extend ZooKeeperTestHarness since it's included
in KafkaServerTestHarness now.
5. We probably need to separate ZK from LogManager. This allows us to use and
test LogManager independently. We can probably attach a handler to the
LogManager. This can be a separate jira.
> add a getTopicMetaData method in broker and expose it to producer
> ------------------------------------------------------------------
>
> Key: KAFKA-238
> URL: https://issues.apache.org/jira/browse/KAFKA-238
> Project: Kafka
> Issue Type: Sub-task
> Components: core
> Reporter: Jun Rao
> Assignee: Neha Narkhede
> Attachments: kafka-238-v1.patch
>
>
> We need a way to propagate the leader and the partition information to the
> producer so that it can do load balancing and semantic partitioning. One way
> to do that is to have the producer get the information from ZK directly. This
> means that the producer needs to maintain a ZK session and has to subscribe
> to watchers, which can be complicated. An alternative approach is to have the
> following api on the broker.
> TopicMetaData getTopicMetaData(String: topic)
> TopicMetaData {
> Array[PartitionMetaData]: partitionsMetaData
> }
> PartitionMetaData {
> Int: partitionId
> String: leaderHostname
> Int: leaderPort
> }
> Using this api, the producer can get the metadata about a topic during
> initial startup or leadership change of a partition.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
