Hi All, I tried to use Datasource Writer to read streaming data from Kafka topic and write to Hudi dataset on HDFS. I used following codes:
val output = data .writeStream .trigger(Trigger.ProcessingTime("300 seconds")) .format("org.apache.hudi") .option("hoodie.table.name", "hudi_ro_table") .outputMode("append") .option("path", fileLocation) .option("checkpointLocation", s"${fileLocation}_chpk") .start() However, when I run this spark job it cannot write anything onto HDFS. Can anyone tell me how to do that? Thanks. Best, Eric