Re: Date class not supported by SparkSQL

[Lior Chaga]

Here's a code example:

public class DateSparkSQLExample {

    public static void main(String[] args) {
        SparkConf conf = new SparkConf().setAppName("test").setMaster("local");
        JavaSparkContext sc = new JavaSparkContext(conf);

        List<SomeObject> itemsList = Lists.newArrayListWithCapacity(1);
        itemsList.add(new SomeObject(new Date(), 1L));
        JavaRDD<SomeObject> someObjectJavaRDD = sc.parallelize(itemsList);

        JavaSQLContext sqlContext = new
org.apache.spark.sql.api.java.JavaSQLContext(sc);
        sqlContext.applySchema(someObjectJavaRDD,
SomeObject.class).registerTempTable("temp_table");
    }

    private static class SomeObject implements Serializable{
        private Date timestamp;
        private Long value;

        public SomeObject() {
        }

        public SomeObject(Date timestamp, Long value) {
            this.timestamp = timestamp;
            this.value = value;
        }

        public Date getTimestamp() {
            return timestamp;
        }

        public void setTimestamp(Date timestamp) {
            this.timestamp = timestamp;
        }

        public Long getValue() {
            return value;
        }

        public void setValue(Long value) {
            this.value = value;
        }
    }
}


On Sun, Apr 19, 2015 at 4:27 PM, Lior Chaga <lio...@taboola.com> wrote:

> Using Spark 1.2.0. Tried to apply register an RDD and got:
> scala.MatchError: class java.util.Date (of class java.lang.Class)
>
> I see it was resolved in https://issues.apache.org/jira/browse/SPARK-2562
> (included in 1.2.0)
>
> Anyone encountered this issue?
>
> Thanks,
> Lior
>