[
https://issues.apache.org/jira/browse/KAFKA-369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13431220#comment-13431220
]
Jun Rao commented on KAFKA-369:
-------------------------------
Thanks for patch v5. A few more comments:
50. There are a bunch of places where zk.connect is still used in
ProducerConfig, could you fix them?
ProducerPerformance, ReplayLogProducer, ConsoleProducer, TestEndToEndLatency,
AsyncProducerTest, KafkaOutputFormat.java, Producer.java
51. Single host multi broker system test seems to hang:
2012-08-08 09:08:22 =======================================
2012-08-08 09:08:22 Iteration 1 of 3
2012-08-08 09:08:22 =======================================
2012-08-08 09:08:22 looking up leader
2012-08-08 09:08:22 found the log line: [2012-08-08 09:08:17,434] INFO
Controller 1, at initializing leaders for new partitions, the leaderAndISR
request sent to broker 3 is LeaderAndISRRequest(1,,false,1000,Map((mytest,0) ->
{ "ISR": "1,2,3","leader": "1","leaderEpoch": "0" }))
(kafka.server.KafkaController)
bin/run-test.sh: line 151: return: [2012-08-08: numeric argument required
2012-08-08 09:08:22 current leader's broker id : 255
2012-08-08 09:08:22 stopping server: 255
2012-08-08 09:08:22 sleeping for 10s
52. remove unused imports
53. ProducerPool.updateProducer: use a HashSet instead of ListBuffer for
storing newBrokers.
54. The following unit test seems to fail occasionally for me. Is that related
to this patch?
[error] Test Failed:
testMultiProduceResend(kafka.integration.LazyInitProducerTest)
kafka.common.KafkaException: fetching broker partition metadata for topics
[List(test1)] from broker
[ArrayBuffer(id:0,creatorId:192.168.1.111-1344443455425,host:192.168.1.111,port:61693)]
failed
at
kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:109)
at
kafka.producer.BrokerPartitionInfo.getBrokerPartitionInfo(BrokerPartitionInfo.scala:49)
at
kafka.producer.async.DefaultEventHandler.kafka$producer$async$DefaultEventHandler$$getPartitionListForTopic(DefaultEventHandler.scala:140)
at
kafka.producer.async.DefaultEventHandler$$anonfun$partitionAndCollate$1.apply(DefaultEventHandler.scala:101)
at
kafka.producer.async.DefaultEventHandler$$anonfun$partitionAndCollate$1.apply(DefaultEventHandler.scala:100)
at
scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:61)
at scala.collection.immutable.List.foreach(List.scala:45)
at
kafka.producer.async.DefaultEventHandler.partitionAndCollate(DefaultEventHandler.scala:100)
at
kafka.producer.async.DefaultEventHandler.dispatchSerializedData(DefaultEventHandler.scala:65)
at
kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:48)
at kafka.producer.Producer.send(Producer.scala:65)
at
kafka.integration.LazyInitProducerTest.testMultiProduceResend(LazyInitProducerTest.scala:163)
55. Could you change the comment for broker.list in ProducerConfig to the
following?
/** This is for bootstrapping and the producer will only use it for getting
metadata (topics, partitions and replicas).
* The socket connections for sending the actual data will be established
based on the broker information
* returned in the metadata. The format is host1:por1,host2:port2, and the
list can be a subset of brokers or
* a VIP pointing to a subset of brokers.
*/
> remove ZK dependency on producer
> --------------------------------
>
> Key: KAFKA-369
> URL: https://issues.apache.org/jira/browse/KAFKA-369
> Project: Kafka
> Issue Type: Sub-task
> Components: core
> Affects Versions: 0.8
> Reporter: Jun Rao
> Assignee: Yang Ye
> Attachments: kafka_369_v1.diff, kafka_369_v2.diff, kafka_369_v3.diff,
> kafka_369_v4.diff, kafka_369_v5.diff
>
> Original Estimate: 252h
> Remaining Estimate: 252h
>
> Currently, the only place that ZK is actually used is in BrokerPartitionInfo.
> We use ZK to get a list of brokers for making TopicMetadataRequest requests.
> Instead, we can provide a list of brokers in the producer config directly.
> That way, the producer client is no longer dependant on ZK.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
