Sorry, I had a duplicated kafka dependency with another older version in another pom.xml
2015-05-05 14:46 GMT+02:00 Guillermo Ortiz <konstt2...@gmail.com>: > I'm tryting to execute the "Hello World" example with Spark + Kafka ( > https://github.com/apache/spark/blob/master/examples/scala-2.10/src/main/scala/org/apache/spark/examples/streaming/DirectKafkaWordCount.scala) > with createDirectStream and I get this error. > > > java.lang.NoSuchMethodError: > kafka.message.MessageAndMetadata.<init>(Ljava/lang/String;ILkafka/message/Message;JLkafka/serializer/Decoder;Lkafka/serializer/Decoder;)V > at > org.apache.spark.streaming.kafka.KafkaRDD$KafkaRDDIterator.getNext(KafkaRDD.scala:176) > at org.apache.spark.util.NextIterator.hasNext(NextIterator.scala:71) > at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327) > > I have checked the jars and it's the kafka-2.10-0.8.2.jar in the classpath > with the MessageAndMetadata class. Even I navigated through eclipse and I > came to this class with the right parameters. > > My pom.xml has these dependecies > > <dependency> > <groupId>org.apache.spark</groupId> > <artifactId>spark-streaming_2.10</artifactId> > <version>1.3.1</version> > <!-- <scope>provided</scope> --> > <exclusions> > <exclusion> > <groupId>org.codehaus.jackson</groupId> > <artifactId>jackson-mapper-asl</artifactId> > </exclusion> > <exclusion> > <groupId>org.codehaus.jackson</groupId> > <artifactId>jackson-core-asl</artifactId> > </exclusion> > </exclusions> > </dependency> > > > <dependency> > <groupId>org.apache.kafka</groupId> > <artifactId>kafka_2.10</artifactId> > <version>0.8.2.1</version> > </dependency> > > <dependency> > <groupId>org.apache.spark</groupId> > <artifactId>spark-streaming-kafka_2.10</artifactId> > <version>1.3.1</version> > <!-- <scope>provided</scope> --> > <exclusions> > <exclusion> > <groupId>org.codehaus.jackson</groupId> > <artifactId>jackson-mapper-asl</artifactId> > </exclusion> > <exclusion> > <groupId>org.codehaus.jackson</groupId> > <artifactId>jackson-core-asl</artifactId> > </exclusion> > </exclusions> > </dependency> > </dependencies> > > Does somebody know where it's the error?? Thanks. >
