you have 2 options 1 )Clean ->Write your own parser to through each property and create a dataset 2) Hacky but simple -> Convert to json string then read in using spark.read.json(jsonString)
Please bear in mind the second option is expensive which is why it is hacky I wrote my own parser here <https://github.com/samelamin/spark-bigquery/blob/master/src/main/scala/com/samelamin/spark/bigquery/converters/SchemaConverters.scala> which you can use to convert between JsonObjects to StructType schemas Regards Sam On Sun, Apr 23, 2017 at 7:50 PM, kant kodali <kanth...@gmail.com> wrote: > Hi All, > > How to convert Dstream of JsonObject to Dataframe in spark 2.1.0? That > JsonObject is from Gson Library. > > Thanks! >
