JDBC sink is not in 2.1. You can see here for an example implementation using the ForEachWriter sink instead: https://databricks.com/blog/2017/04/04/real-time-end-to-end-integration-with-apache-kafka-in-apache-sparks-structured-streaming.html
From: Hemanth Gudela <hemanth.gud...@qvantel.com> Date: Sunday, April 9, 2017 at 4:30 PM To: "user@spark.apache.org" <user@spark.apache.org> Subject: Does spark 2.1.0 structured streaming support jdbc sink? Hello Everyone, I am new to Spark, especially spark streaming. I am trying to read an input stream from Kafka, perform windowed aggregations in spark using structured streaming, and finally write aggregates to a sink. - MySQL as an output sink doesn’t seem to be an option, because this block of code throws an error streamingDF.writeStream.format("jdbc").start("jdbc:mysql…”) ava.lang.UnsupportedOperationException: Data source jdbc does not support streamed writing This is strange because, this<http://rxin.github.io/talks/2016-02-18_spark_summit_streaming.pdf> document shows that jdbc is supported as an output sink! - Parquet doesn’t seem to be an option, because it doesn’t support “complete” output mode, but “append” only. As I’m preforming windows aggregations in spark streaming, the output mode has to be complete, and cannot be “append” - Memory and console sinks are good for debugging, but are not suitable for production jobs. So, please correct me if I’m missing something in my code to enable jdbc output sink. If jdbc output sink is not option, please suggest me an alternative output sink that suits my needs better. Or since structured streaming is still ‘alpha’, should I resort to spark dstreams to achieve my use case described above. Please suggest. Thanks in advance, Hemanth