I would like to know how to create stream and sink operations outside
"main" method - just like another class which I can invoke from main. So
that I can have different implementations for each topic which I subscribed
in a specific class file. Is it a good practice or always the whole
You can have below statement for multiple topics
val dfStatus = spark.readStream.
format("kafka").
option("subscribe", "utility-status, utility-critical").
option("kafka.bootstrap.servers", "localhost:9092").
option("startingOffsets", "earliest")
.load()
On Mon,
Hi
I have integrated Spark Streaming with Kafka in which Im listening 2 topics
def main(args: Array[String]): Unit = {
val schema = StructType(
List(
StructField("gatewayId", StringType, true),
StructField("userId", StringType, true)
)
)
val spark =
