Re: Distribute data from Kafka evenly on cluster

[Tobias Pfeiffer]

Thank you very much for the link, that was very helpful!

So, apparently the `topics: Map[String, Int]` parameter controls the number
of partitions that the data is initially added to; the number N in

  val kafkaInputs = (1 to N).map { _ =>
    ssc.kafkaStream(zkQuorum, groupId, Map("topic" -> 1))
  }
  val union = ssc.union(kafkaInputs)

controls how many connections are made to Kafka. Note that the number of
Kafka partitions for that topic must be at least N for this to work.

Thanks
Tobias