HI all
i am trying to convert a string column, in a Dataframe , to a
java.util.Date but i am getting this exception
[dispatcher-event-loop-0] INFO org.apache.spark.storage.BlockManagerInfo -
Removed broadcast_0_piece0 on 169.254.2.140:53468 in memory (size: 14.3 KB,
free: 767.4 MB)
Exception in thread "main" java.lang.UnsupportedOperationException: Schema
for type java.util.Date is not supported
here's my code
val tstampToDateFunc:(java.sql.Timestamp => java.util.Date) = ts => new
java.util.Date(ts.getTime)
val tsampConversionFunc = udf(tstampToDateFunc)
sharesDf.withColumn("price", col("_c2").cast("double"))
.withColumn("creationTime",
tsampConversionFunc(col("_c1")))
Are there any workarounds?
i m trying to import data into mongoDB via Spark. The source is a csv file
where
i have 1 timestamp column and a bunch of strings..... i will need to
convert that
to something compatible with a mongo's ISODate
kr
marco
