Hi, I have a problem with Json Parser. I am using spark streaming with hiveContext for keeping json format tweets. The flume collects tweets and sink to hdfs path. My spark streaming job checks the hdfs path and convert coming json tweets and insert them to hive table.
My problem is that ; Some of tweets have location name information or in reply to information etc. but some of tweets have not any information for that columns. In Spark streaming job, I try to make generic json handler for example if the tweet has no location information spark can write null value to hive table for location column. Is there any way to make this? I am using Spark 1.3.0 version Thanks. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-Json-Parse-tp26391.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
