Re: [Spark Java] Add new column in DataSet based on existed column

Hi , Here is example snippet in scala // Convert to a Date typeval timestamp2datetype: (Column) => Column = (x) => { to_date(x) }df = df.withColumn("date", timestamp2datetype(col("end_date"))) Hope this helps ! Thanks, Divya On 28 March 2018 at 15:16, Junfeng Chen

[Spark Java] Add new column in DataSet based on existed column

I am working on adding a date transformed field on existed dataset. The current dataset contains a column named timestamp in ISO format. I want to parse this field to joda time type, and then extract the year, month, day, hour info as new column attaching to original dataset. I have tried