Hi Swetha,
I also had the same requirement reading from json from kafka and writing
back to parquet format.
I did a work around :
1. Inferred the schema using the batch api by reading first few rows
2. started streaming using the inferred schema in step1
*Limitation*: Will not work if you
Hi,
Is there a way that Automatic Json Schema inference can be done using
Structured Streaming? I do not want to supply a predefined schema and bind
it.
With Spark Kafka Direct I could do spark.read.json(). I see that this is not
supported in Structured Streaming.
Thanks!
--
Sent from:
