[
https://issues.apache.org/jira/browse/KAFKA-3068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15086554#comment-15086554
]
Jason Gustafson commented on KAFKA-3068:
----------------------------------------
[~ijuma],[~enothereska] Introducing a notion of cluster identity seems like a
good idea to solve this problem more generally. I'm not sure I understand the
point about not having enough bootstrap brokers though. The cached metadata
always contains the full list of brokers in the cluster (i.e. those registered
in Zookeeper). The only time we might need to dip into an additional set is
when all of the known brokers simultaneously become unreachable. When that
happens, it seems to make more sense to revert to the list of bootstrap brokers
rather than attempting to reach a broker which we discovered previously but had
itself became unreachable at some point (if it was still reachable on the last
metadata refresh, then it would be contained in the metadata). And yes, the
bootstrap brokers can also move to another cluster, but I would consider this a
user configuration error rather than an application error. You can hardly fault
an application for trying to connect to the endpoints that had been explicitly
configured, but you might if it connects to a broker it previously discovered
which had been shutdown and moved to another cluster. Does that make sense?
> NetworkClient may connect to a different Kafka cluster than originally
> configured
> ---------------------------------------------------------------------------------
>
> Key: KAFKA-3068
> URL: https://issues.apache.org/jira/browse/KAFKA-3068
> Project: Kafka
> Issue Type: Bug
> Components: clients
> Affects Versions: 0.9.0.0
> Reporter: Jun Rao
>
> In https://github.com/apache/kafka/pull/290, we added the logic to cache all
> brokers (id and ip) that the client has ever seen. If we can't find an
> available broker from the current Metadata, we will pick a broker that we
> have ever seen (in NetworkClient.leastLoadedNode()).
> One potential problem this logic can introduce is the following. Suppose that
> we have a broker with id 1 in a Kafka cluster. A producer client remembers
> this broker in nodesEverSeen. At some point, we bring down this broker and
> use the host in a different Kafka cluster. Then, the producer client uses
> this broker from nodesEverSeen to refresh metadata. It will find the metadata
> in a different Kafka cluster and start producing data there.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
