Re: Type Casting Error in Spark Data Frame

formatted = Assuming MessageHelper.sqlMapping schema is correctly mapped with input json (it would help if the schema and sample json is shared) here is explode function with dataframes similar functionality is available with SQL import sparkSession.implicits._ import

Re: Type Casting Error in Spark Data Frame

Assuming MessageHelper.sqlMapping schema is correctly mapped with input json (it would help if the schema and sample json is shared)here is explode function with dataframes similar functionality is available with SQL import sparkSession.implicits._import org.apache.spark.sql.functions._val

Re: Type Casting Error in Spark Data Frame

You can try to create new columns with the nested value, > On Jan 29, 2018, at 15:26, Arnav kumar wrote: > > Hello Experts, > > I would need your advice in resolving the below issue when I am trying to > retrieving the data from a dataframe. > > Can you please let me

Re: Type Casting Error in Spark Data Frame

You can't select from an array like that, try instead using 'lateral view explode' in the query for that element, or before the sql stage (py)spark.sql.functions.explode. On Mon, Jan 29, 2018 at 4:26 PM, Arnav kumar wrote: > Hello Experts, > > I would need your advice in

Type Casting Error in Spark Data Frame

Hello Experts, I would need your advice in resolving the below issue when I am trying to retrieving the data from a dataframe. Can you please let me know where I am going wrong. code : // create the dataframe by parsing the json // Message Helper describes the JSON Struct //data out is the