Re: Converting String to Datetime using map

Thanks. Tried this scala> val a = df.filter(col("Total") > "").map(p => Invoices(p(0).toString, p(1).toString.TO_DATE(FROM_UNIXTIME(UNIX_TIMESTAMP(p(1),"dd/MM/"),"-MM-dd")), p(2).toString.substring(1).replace(",", "").toDouble, p(3).toString.substring(1).replace(",", "").toDouble,

Re: Converting String to Datetime using map

You can invoke exactly the same functions on scala side as well i.e. http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.functions$ Have you tried them? On Thu, Mar 24, 2016 at 10:29 PM, Mich Talebzadeh wrote: > > Hi, > > Read a CSV in with

Converting String to Datetime using map

Hi, Read a CSV in with the following schema scala> df.printSchema root |-- Invoice Number: string (nullable = true) |-- Payment date: string (nullable = true) |-- Net: string (nullable = true) |-- VAT: string (nullable = true) |-- Total: string (nullable = true) I use mapping as below