Hi Jacek,
I was wondering if I could use this approach itself.
It is basically a CSV read in as follows:
val HiveContext = new org.apache.spark.sql.hive.HiveContext(sc)
val df =
sqlContext.read.format("com.databricks.spark.csv").option("inferSchema",
"true").option("header",
Hi,
Why don't you use Datasets? You'd cut the number of getStrings and
it'd read nicer to your eyes. Also, doing such transformations would
*likely* be easier.
p.s. Please gist your example to fix it.
Pozdrawiam,
Jacek Laskowski
https://medium.com/@jaceklaskowski/
Mastering Apache Spark
How can I convert the following from String to datetime
scala> df.map(x => (x.getString(1), ChangeDate(x.getString(1.take(1)
res60: Array[(String, String)] = Array((10/02/2014,2014-02-10))
Please note that the custom UDF ChangeDate() has revered the string value
from "dd/MM/" to
