Hi Colin, my understanding is that this is currently not possible with KafkaUtils. You would have to write a custom receiver using Kafka's SimpleConsumer API.
https://spark.apache.org/docs/1.2.0/streaming-custom-receivers.html https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Example Regards, Jeff 2015-03-12 17:58 GMT+01:00 ColinMc <colin.mcqu...@shiftenergy.com>: > Hi, > > How do you use KafkaUtils to specify a specific partition? I'm writing > customer Marathon jobs where a customer is given 1 partition in a topic in > Kafka. The job will get the partition from our database for that customer > and use that to get the messages for that customer. > > I misinterpreted KafkaUtils when creating the stream and didn't know that > it > was the number of partitions per topic in the map. > > If KafkaUtils doesn't support this, is there another Spark API call for > Kafka that supports this? > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/KafkaUtils-and-specifying-a-specific-partition-tp22018.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
