How to print data to console in structured streaming using Spark 2.1.0?

[kant kodali]

Hi All,

I have the following code.

 val ds = sparkSession.readStream()
                .format("kafka")
                .option("kafka.bootstrap.servers",bootstrapServers))
                .option("subscribe", topicName)
                .option("checkpointLocation", hdfsCheckPointDir)
                .load();

 val ds1 = ds.select($"value")
 val query = ds1.writeStream.outputMode("append").format("console").start()
 query.awaitTermination()

There are no errors when I execute this code however I don't see any data
being printed out to console? When I run my standalone test Kafka consumer
jar I can see that it is receiving messages. so I am not sure what is going
on with above code? any ideas?

Thanks!