Hi, I am using Spark1.5.2 My requirement is as below df.withColumn("NoOfDays",lit(datediff(df("Start_date"),df("end_date"))))
Now have to add one more columnn where my datediff(Start_date,end_date)) should match with map keys Map looks like MyMap(1->1D,2->2D,3->3M,4->4W........) I want to do something like this > val > condition= MyMap.contains(lit(datediff(df("END_DATE"),df("START_DATE")))) > val geId =MyMap(datediff(df("END_DATE"),df("START_DATE"))) > df.withColumn("AddColumn",when(cond,lit(getId))) Is it possible ? What I am missing here .. I am beginner in scala and Spark. Would really appreciate the help. Thanks, Divya