RE: scala.MatchError: class java.sql.Timestamp

[Ge, Yao (Y.)]

scala.MatchError: class java.sql.Timestamp (of class java.lang.Class)
                at 
org.apache.spark.sql.api.java.JavaSQLContext$$anonfun$getSchema$1.apply(JavaSQLContext.scala:189)
                at 
org.apache.spark.sql.api.java.JavaSQLContext$$anonfun$getSchema$1.apply(JavaSQLContext.scala:188)
                at 
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
                at 
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
                at 
scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
                at 
scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)
                at 
scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
                at 
scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:108)
                at 
org.apache.spark.sql.api.java.JavaSQLContext.getSchema(JavaSQLContext.scala:188)
                at 
org.apache.spark.sql.api.java.JavaSQLContext.applySchema(JavaSQLContext.scala:90)
                at 
com.ford.dtc.ff.SessionStats.testTimeStamp(SessionStats.java:111)
                at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
                at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown 
Source)
                at java.lang.reflect.Method.invoke(Unknown Source)
                at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
                at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
                at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
                at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
                at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
                at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
                at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
                at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
                at 
org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
                at 
org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
                at 
org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
                at 
org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
                at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
                at 
org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
                at 
org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
                at 
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
                at 
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
                at 
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
                at 
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)


From: Wang, Daoyuan [mailto:daoyuan.w...@intel.com]
Sent: Sunday, October 19, 2014 10:31 AM
To: Ge, Yao (Y.); user@spark.apache.org
Subject: RE: scala.MatchError: class java.sql.Timestamp

Can you provide the exception stack?

Thanks,
Daoyuan

From: Ge, Yao (Y.) [mailto:y...@ford.com]
Sent: Sunday, October 19, 2014 10:17 PM
To: user@spark.apache.org<mailto:user@spark.apache.org>
Subject: scala.MatchError: class java.sql.Timestamp

I am working with Spark 1.1.0 and I believe Timestamp is a supported data type 
for Spark SQL. However I keep getting this MatchError for java.sql.Timestamp 
when I try to use reflection to register a Java Bean with Timestamp field. 
Anything wrong with my code below?

                public static class Event implements Serializable {
                                private String name;
                                private Timestamp time;
                                public String getName() {
                                                return name;
                                }
                                public void setName(String name) {
                                                this.name = name;
                                }
                                public Timestamp getTime() {
                                                return time;
                                }
                                public void setTime(Timestamp time) {
                                                this.time = time;
                                }
                }

                @Test
                public void testTimeStamp() {
                                JavaSparkContext sc = new 
JavaSparkContext("local", "timestamp");
                                String[] data = {"1,2014-01-01", 
"2,2014-02-01"};
                                JavaRDD<String> input = 
sc.parallelize(Arrays.asList(data));
                                JavaRDD<Event> events = input.map(new 
Function<String,Event>() {
                                                public Event call(String arg0) 
throws Exception {
                                                                String[] c = 
arg0.split(",");
                                                                Event e = new 
Event();
                                                                e.setName(c[0]);
                                                                DateFormat fmt 
= new SimpleDateFormat("yyyy-MM-dd");
                                                                e.setTime(new 
Timestamp(fmt.parse(c[1]).getTime()));
                                                                return e;
                                                }
                                });

                                JavaSQLContext sqlCtx = new JavaSQLContext(sc);
                                JavaSchemaRDD schemaEvent = 
sqlCtx.applySchema(events, Event.class);
                                schemaEvent.registerTempTable("event");

                                sc.stop();
                }