Why do you want to write out NULL if the column has no data? Just insert the fields that you have.
> On Mar 3, 2016, at 9:10 AM, barisak <baris.akg...@gmail.com> wrote: > > Hi, > > I have a problem with Json Parser. I am using spark streaming with > hiveContext for keeping json format tweets. The flume collects tweets and > sink to hdfs path. My spark streaming job checks the hdfs path and convert > coming json tweets and insert them to hive table. > > My problem is that ; > > Some of tweets have location name information or in reply to information > etc. but some of tweets have not any information for that columns. > > In Spark streaming job, I try to make generic json handler for example if > the tweet has no location information spark can write null value to hive > table for location column. > > Is there any way to make this? > > I am using Spark 1.3.0 version > > Thanks. > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-Json-Parse-tp26391.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
