Hi, All,
I have one issue here about how to process multiple Kafka topics in a
Spark 2.* program. My question is: How to get the topic name from a message
received from Kafka? E.g:
......
val messages = KafkaUtils.createDirectStream[String, String,
StringDecoder, StringDecoder](
ssc, kafkaParams, topicsSet)
// Get the lines, split them into words, count the words and print
val lines = messages.map(_._2)
val words = lines.flatMap(_.split(" "))
val wordCounts = words.map(x => (x, 1L)).reduceByKey(_ + _)
wordCounts.print()
......
Kafka send the messages in multiple topics through console producer for
example. But when Spark receive the message, how it will know which topic
is this piece of message coming from? Thanks a lot for any of your helps!
Cheers,
Dan
