[
https://issues.apache.org/jira/browse/KAFKA-1919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14306357#comment-14306357
]
Jay Kreps commented on KAFKA-1919:
----------------------------------
Here is what I think is happening: in 0.8.1 the metadata request just gives
back nodes for the requested topics. But in this case there are no requested
topics so there are no nodes. When we get a metadata response with no nodes we
actually ignore it, because if we update our view of the cluster with one that
has no machines then we have nowhere to even make metadata requests.
Unfortunately by not updating the cluster we are bypassing the backoff the
prevents another request from being issued.
I suspect this doesn't happen against a 0.8.2 server because in 0.8.2 we are
giving back all the nodes no matter what in the metadata response specifically
to avoid every running into a situation where there is no one to issue metadata
requests against. I think the fix is to "update" the metadata with the existing
metadata not the new (empty) metadata so as to guarantee that the backoff will
always come into effect.
[~stevenz3wu] I have a patch that I think will fix this if I understand the
root cause. Want to give it a shot?
> Metadata request issued with no backoff in new producer if there are no topics
> ------------------------------------------------------------------------------
>
> Key: KAFKA-1919
> URL: https://issues.apache.org/jira/browse/KAFKA-1919
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8.2
> Reporter: Jay Kreps
>
> Original report:
> We have observed high cpu and high network traffic problem when
> 1) cluster (0.8.1.1) has no topic
> 2) KafkaProducer (0.8.2-beta) object is created without sending any traffic
> We have observed such problem twice. In both cases, problem went away
> immediately after one/any topic is created.
> Is this a known issue? Just want to check with the community first before I
> spend much time to reproduce it.
> I couldn't reproduce the issue with similar setup with unit test code in
> IDE. start two brokers with no topic locally on my laptop. create a
> KafkaProducer object without sending any msgs. but I only tested with
> 0.8.2-beta for both broker and producer.
> Issue exists in 0.8.2 as well:
> I have re-run my unit test with 0.8.2.0. same tight-loop problem happened
> after a few mins.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
