What are you trying to achieve? Looks like you want to provide offsets but you're not managing them and I'm assuming you're using the direct stream approach.
In that case, use the simpler constructor that takes the kafka config and the topics. Let it figure it out the offsets (it will contact kafka and request the partitions for the topics provided) KafkaUtils.createDirectStream[...](ssc, kafkaConfig, topics) -kr, Gerard On Mon, Jan 25, 2016 at 5:31 PM, Ashish Soni <asoni.le...@gmail.com> wrote: > Hi All , > > What is the best way to tell spark streaming job for the no of partition > to to a given topic - > > Should that be provided as a parameter or command line argument > or > We should connect to kafka in the driver program and query it > > Map<TopicAndPartition, Long> fromOffsets = new HashMap<TopicAndPartition, > Long>(); > fromOffsets.put(new TopicAndPartition(driverArgs.inputTopic, 0), 0L); > > Thanks, > Ashish >
