Re: Kafka + Spark 1.3 Integration

That's what the kafkaParams argument is for. Not all of the kafka configuration parameters will be relevant, though. On Thu, Feb 11, 2016 at 12:07 PM, Nipun Arora wrote: > Hi , > > Thanks for the explanation and the example link. Got it working. > A follow up

Re: Kafka + Spark 1.3 Integration

Hi , Thanks for the explanation and the example link. Got it working. A follow up question. In Kafka one can define properties as follows: Properties props = new Properties(); props.put("zookeeper.connect", zookeeper); props.put("group.id", groupId); props.put("zookeeper.session.timeout.ms",

Kafka + Spark 1.3 Integration

Hi, I am trying some basic integration and was going through the manual. I would like to read from a topic, and get a JavaReceiverInputDStream for messages in that topic. However the example is of JavaPairReceiverInputDStream<>. How do I get a stream for only a single topic in Java? Reference

Re: Kafka + Spark 1.3 Integration

It's a pair because there's a key and value for each message. If you just want a single topic, put a single topic in the map of topic -> number of partitions. See https://github.com/apache/spark/blob/master/examples/src/main/java/org/apache/spark/examples/streaming/JavaKafkaWordCount.java On