Hi! I want to exucute code inside forEachBatch that will trigger regardless of whether there is data in the batch or not.
val kafkaReadStream = spark .readStream .format("kafka") .option("kafka.bootstrap.servers", broker) .option("subscribe", topicName) .option("startingOffsets", startingOffsetsMode) .option("maxOffsetsPerTrigger", maxOffsetsPerTrigger) .load() kafkaReadStream .writeStream .trigger(Trigger.ProcessingTime(s"$triggerProcessingTime seconds")) .foreachBatch { .... } .start() .awaitTermination()