I've been using Joda Time in all my spark jobs (by using the nscala-time package) and have not run into any issues until I started trying to use spark sql. When I try to convert a case class that has a com.github.nscala_time.time.Imports.DateTime object in it, an exception is thrown for with a MatchError
My assumption is that this is because the basic types of spark sql are java.sql.Timestamp and java.sql.Date and therefor spark doesn't know what to do about the DateTime value. How can I get around this? I would prefer not to have to change my code to make the values be Timestamps but I'm concerned that might be the only way. Would something like implicit conversions work here? It seems that even if I specify the schema manually then I would still have the issue since you have to specify the column type which has to be of type org.apache.spark.sql.types.DataType -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-use-Joda-Time-with-Spark-SQL-tp22415.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
