Fwd: This simple UDF is not working!

Thanks great Dhaval. scala> import java.text.SimpleDateFormat import java.text.SimpleDateFormat scala> scala> import java.sql.Date import java.sql.Date scala> scala> import scala.util.{Try, Success, Failure} import scala.util.{Try, Success, Failure} scala> val toDate = udf{(out:String, form:

Re: This simple UDF is not working!

Hi Mich, You can try this: val toDate = udf{(out:String, form: String) => { val format = new SimpleDateFormat(s"$form"); Try(new Date(format.parse(out.toString()).getTime))match { case Success(t) => Some(t) case Failure(_) => None }}}; Usage: src = src.withColumn(s"$columnName",

Re: This simple UDF is not working!

Hi Ted, I decided to take a short cut here. I created the map leaving date as it is (p(1)) below def CleanupCurrency (word : String) : Double = { return word.toString.substring(1).replace(",", "").toDouble } sqlContext.udf.register("CleanupCurrency", CleanupCurrency(_:String)) val a =

Re: This simple UDF is not working!

This works with sql sqltext = """ INSERT INTO TABLE t14 SELECT INVOICENUMBER , TO_DATE(FROM_UNIXTIME(UNIX_TIMESTAMP(paymentdate,'dd/MM/'),'-MM-dd')) AS paymentdate , NET , VAT , TOTAL FROM tmp """ sql(sqltext) but not in UDF. I want to convert

Re: This simple UDF is not working!

Do you mind showing body of TO_DATE() ? Thanks On Fri, Mar 25, 2016 at 7:38 AM, Ted Yu wrote: > Looks like you forgot an import for Date. > > FYI > > On Fri, Mar 25, 2016 at 7:36 AM, Mich Talebzadeh < > mich.talebza...@gmail.com> wrote: > >> >> >> Hi, >> >> writing a UDF

Re: This simple UDF is not working!

Looks like you forgot an import for Date. FYI On Fri, Mar 25, 2016 at 7:36 AM, Mich Talebzadeh wrote: > > > Hi, > > writing a UDF to convert a string into Date > > def ChangeDate(word : String) : Date = { > | return >

This simple UDF is not working!

Hi, writing a UDF to convert a string into Date def ChangeDate(word : String) : Date = { | return TO_DATE(FROM_UNIXTIME(UNIX_TIMESTAMP(word),"dd/MM/"),"-MM-dd") | } :19: error: not found: type Date That code to_date.. works OK in sql but not here. It is complaining about