Re: [Structured Streaming][Kafka] For a Kafka topic with 3 partitions, how does the parallelism work ?

Yes as long as there are 3 cores available on your local machine. On Fri, Apr 20, 2018 at 10:56 AM karthikjay wrote: > I have the following code to read data from Kafka topic using the > structured > streaming. The topic has 3 partitions: > > val spark = SparkSession >

[Structured Streaming][Kafka] For a Kafka topic with 3 partitions, how does the parallelism work ?

I have the following code to read data from Kafka topic using the structured streaming. The topic has 3 partitions: val spark = SparkSession .builder .appName("TestPartition") .master("local[*]") .getOrCreate() import spark.implicits._ val dataFrame = spark