I am working on adding a date transformed field on existed dataset. The current dataset contains a column named timestamp in ISO format. I want to parse this field to joda time type, and then extract the year, month, day, hour info as new column attaching to original dataset. I have tried df.withColumn function, but it seems only support simple expression rather than customized function like MapFunction. How to solve it?
Thanks! Regard, Junfeng Chen
