Hi Sriram, Thanks. This is what I was looking for.
one question, where do we need to specify the checkpoint directory in case of structured streaming? Thanks, Asmath On Wed, Oct 25, 2017 at 2:52 PM, Subhash Sriram <subhash.sri...@gmail.com> wrote: > Hi Asmath, > > Here is an example of using structured streaming to read from Kafka: > > https://github.com/apache/spark/blob/master/examples/ > src/main/scala/org/apache/spark/examples/sql/streaming/ > StructuredKafkaWordCount.scala > > In terms of parsing the JSON, there is a from_json function that you can > use. The following might help: > > https://databricks.com/blog/2017/02/23/working-complex- > data-formats-structured-streaming-apache-spark-2-1.html > > I hope this helps. > > Thanks, > Subhash > > On Wed, Oct 25, 2017 at 2:59 PM, KhajaAsmath Mohammed < > mdkhajaasm...@gmail.com> wrote: > >> Hi, >> >> Could anyone provide suggestions on how to parse json data from kafka and >> load it back in hive. >> >> I have read about structured streaming but didn't find any examples. is >> there any best practise on how to read it and parse it with structured >> streaming for this use case? >> >> Thanks, >> Asmath >> > >
