[ https://issues.apache.org/jira/browse/SPARK-6594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
q79969786 updated SPARK-6594: ----------------------------- Comment: was deleted (was: 1. Create kafka topic as follow: $kafka-topics.sh --create --zookeeper zkServer1:2181,zkServer2:2181,zkServer3:2181 --replication-factor 1 --partitions 5 --topic ORDER 2. I'm use Java API to process data as follow: SparkConf sparkConf = new SparkConf().setAppName("TestOrder"); sparkConf.set("spark.cleaner.ttl", "600"); JavaStreamingContext jssc = new JavaStreamingContext(sparkConf, new Duration(1000)); Map<String, Integer> topicorder = new HashMap<String, Integer>(); topicorder.put("order", 5); JavaPairReceiverInputDStream<String, String> jPRIDSOrder = KafkaUtils.createStream(jssc, ’zkServer1:2181,zkServer2:2181,zkServer3:2181‘, ’test-consumer-group‘, topicorder); jPRIDSOrder.map(new Function<Tuple2<String, String>, String>() { @Override public String call(Tuple2<String, String> tuple2) { return tuple2._2(); } }).print(); 3. Submit this application as follow: spark-submit --class com.bigdata.TestOrder --master spark://SPKMASTER:19002 /home/bigdata/test-spark.jar TestOrder 4. It will shows five warnings as follows when submit application: 15/03/29 21:23:03 WARN ZookeeperConsumerConnector: [test-consumer-group_work1-1427462582342-5714642d], No broker partitions consumed by consumer thread test-consumer-group_work1-1427462582342-5714642d-0 for topic ORDER ...... ) > Spark Streaming can't receive data from kafka > --------------------------------------------- > > Key: SPARK-6594 > URL: https://issues.apache.org/jira/browse/SPARK-6594 > Project: Spark > Issue Type: Bug > Affects Versions: 1.2.1 > Environment: kafka_2.10-0.8.1.1 + Spark-1.2.1 > Reporter: q79969786 > > I use KafkaUtils to receive data from Kafka In my Spark streaming application > as follows: > Map<String, Integer> topicorder = new HashMap<String, Integer>(); > topicorder.put("order", Integer.valueOf(readThread)); > JavaPairReceiverInputDStream<String, String> jPRIDSOrder = > KafkaUtils.createStream(jssc, zkQuorum, group, topicorder); > It worked well at fist, but after I submit this application several times, > Spark streaming can‘t receive data anymore(Kafka works well). -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org