Re: RDD to DataFrame question with JsValue in the mix

On 7/1/2016 6:42 AM, Akhil Das wrote: case class Holder(str: String, js:JsValue) Hello, Thanks! I tried that before posting the question to the list but I keep getting an error such as this even after the map() operation to convert (String,JsValue) -> Holder and then toDF(). I am simply

Re: RDD to DataFrame question with JsValue in the mix

Something like this? import sqlContext.implicits._ case class Holder(str: String, js:JsValue) yourRDD.map(x => Holder(x._1, x._2)).toDF() On Fri, Jul 1, 2016 at 3:36 AM, Dood@ODDO wrote: > Hello, > > I have an RDD[(String,JsValue)] that I want to convert into a

RDD to DataFrame question with JsValue in the mix

Hello, I have an RDD[(String,JsValue)] that I want to convert into a DataFrame and then run SQL on. What is the easiest way to get the JSON (in form of JsValue) "understood" by the process? Thanks! - To unsubscribe e-mail:

DataFrame question

Hi All, I am working with dataframes and have been struggling with this thing, any pointers would be helpful. I've a Json file with the schema like this, links: array (nullable = true) ||-- element: struct (containsNull = true) |||-- desc: string (nullable = true) |||--

Re: DataFrame question

You probably want to explode the array to produce one row per element: df.select(explode(df(links)).alias(link)) On Tue, Jul 7, 2015 at 10:29 AM, Naveen Madhire vmadh...@umail.iu.edu wrote: Hi All, I am working with dataframes and have been struggling with this thing, any pointers would be