I am trying to user code similar to following :
public JavaPairDStream<String, String> openStream() {
HashMap<String, String> kafkaParams = Maps.newHashMap();
kafkaParams.put(ZK_CONNECT,kafkaConfig.getString(ZK_CONNECT));
kafkaParams.put(CONSUMER_GRP_ID,kafkaConfig.getString(CONSUMER_GRP_ID));
Map<String,Integer> topicMap = Maps.newHashMap();
topicMap.put(kafkaConfig.getString(ZK_TOPIC),
kafkaConfig.getInteger(CONSUMER_THREAD_COUNT, 1));
JavaPairDStream<String, String> inputStream =
KafkaUtils.createStream(streamingContext, String.class,
String.class,
StringDecoder.class,
StringDecoder.class,
kafkaParams, topicMap,
StorageLevel.MEMORY_AND_DISK_SER_2());
return inputStream;
}
I have spark-streaming_2.10-0.9.0-incubating.jar and
spark-streaming-kafka_2.10-0.9.0-incubating.jar
in the classpath using POM and m2e in Eclipse. JVM version is set to 1.6
I get the following error,
14/03/02 09:29:15 INFO kafka.KafkaReceiver: Connected to localhost:2181
14/03/02 09:29:15 ERROR kafka.KafkaReceiver: Error receiving data
java.lang.NoSuchMethodException:
java.lang.Object.<init>(kafka.utils.VerifiableProperties)
at java.lang.Class.getConstructor0(Class.java:2763)
at java.lang.Class.getConstructor(Class.java:1693)
at
org.apache.spark.streaming.kafka.KafkaReceiver.onStart(KafkaInputDStream.scala:108)
at
org.apache.spark.streaming.dstream.NetworkReceiver.start(NetworkInputDStream.scala:126)
at
org.apache.spark.streaming.scheduler.NetworkInputTracker$ReceiverExecutor$$anonfun$8.apply(NetworkInputTracker.scala:173)
at
org.apache.spark.streaming.scheduler.NetworkInputTracker$ReceiverExecutor$$anonfun$8.apply(NetworkInputTracker.scala:169)
at
org.apache.spark.SparkContext$$anonfun$runJob$4.apply(SparkContext.scala:884)
at
org.apache.spark.SparkContext$$anonfun$runJob$4.apply(SparkContext.scala:884)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:109)
at org.apache.spark.scheduler.Task.run(Task.scala:53)
at .....
This is similar to code in JavaKafkaStreamSuite.testKafkaStream. I find that
the kafka jar - kafka_2.10-0.8.0 does have such a constructor.
What is going wrong? Can someone help solve this mystery and help with my
misery? Basically stuck for last 2 days - as I am a Java Guy and would like
to develop downstream code in Java
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/NoSuchMethodError-in-KafkaReciever-tp2209.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
