What's the format you have in json file? On Fri, Apr 10, 2015 at 6:57 PM, Suraj Shetiya <[email protected]> wrote: > Hi, > > In pyspark when if I read a json file using sqlcontext I find that the date > field is not infered as date instead it is converted to string. And when I > try to convert it to date using df.withColumn(df.DateCol.cast("timestamp")) > it does not parse it successfuly and adds a null instead there. Should I > use UDF to convert the date ? Is this expected behaviour (not throwing an > error after failure to cast all fields)? > > -- > Regards, > Suraj
--------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
