Yin, Yup thanks. I fixed that shortly after I posted and it worked.
Thanks, Kevin On Fri, Mar 13, 2015 at 8:28 PM, Yin Huai <yh...@databricks.com> wrote: > Seems you want to use array for the field of "providers", like > "providers":[{"id": > ...}, {"id":...}] instead of "providers":{{"id": ...}, {"id":...}} > > On Fri, Mar 13, 2015 at 7:45 PM, kpeng1 <kpe...@gmail.com> wrote: > >> Hi All, >> >> I was noodling around with loading in a json file into spark sql's hive >> context and I noticed that I get the following message after loading in >> the >> json file: >> PhysicalRDD [_corrupt_record#0], MappedRDD[5] at map at JsonRDD.scala:47 >> >> I am using the HiveContext to load in the json file using the jsonFile >> command. I also have 1 json object per line on the file. Here is a >> sample >> of the contents in the json file: >> >> {"user_id":"7070","providers":{{"id":"8753","name":"pjfig","behaviors":{"b1":"erwxt","b2":"yjooj"}},{"id":"8329","name":"dfvhh","behaviors":{"b1":"pjjdn","b2":"ooqsh"}}}} >> >> {"user_id":"1615","providers":{{"id":"6105","name":"rsfon","behaviors":{"b1":"whlje","b2":"lpjnq"}},{"id":"6828","name":"pnmrb","behaviors":{"b1":"fjpmz","b2":"dxqxk"}}}} >> >> {"user_id":"5210","providers":{{"id":"9360","name":"xdylm","behaviors":{"b1":"gcdze","b2":"cndcs"}},{"id":"4812","name":"gxboh","behaviors":{"b1":"qsxao","b2":"ixdzq"}}}} >> >> >> >> >> -- >> View this message in context: >> http://apache-spark-user-list.1001560.n3.nabble.com/Loading-in-json-with-spark-sql-tp22044.html >> Sent from the Apache Spark User List mailing list archive at Nabble.com. >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >> For additional commands, e-mail: user-h...@spark.apache.org >> >> >