Re: FW: Kafka Direct Stream - dynamic topic subscription

2017-10-29 Thread Cody Koeninger
; I forward the email I sent to spark users group that contains a little more > background on my question. > > > > Thank you, > > Regards, > > Buvana > > > > From: Ramanan, Buvana (Nokia - US/Murray Hill) > [mailto:buvana.rama...@nokia-bell-labs.com] > Sent:

Kafka Direct Stream - dynamic topic subscription

2017-10-27 Thread Ramanan, Buvana (Nokia - US/Murray Hill)
Hello, Using Spark 2.2.0. Interested in seeing the action of dynamic topic subscription. Tried this example: streaming.DirectKafkaWordCount (which uses org.apache.spark.streaming.kafka010) I start with 8 Kafka partitions in my topic and found that Spark Streaming executes 8 tasks (one per