spark sql: timestamp in json - fails

[tridib]

Hello Experts,
After repeated attempt I am unable to run query on map json date string. I
tried two approaches:

*** Approach 1 *** created a Bean class with timespamp field. When I try to
run it I get scala.MatchError: class java.sql.Timestamp (of class
java.lang.Class). Here is the code:
import java.sql.Timestamp;

public class ComplexClaim {
    private Timestamp timestamp;

    public Timestamp getTimestamp() {
        return timestamp;
    }

    public void setTimestamp(Timestamp timestamp) {
        this.timestamp = timestamp;
    }
}

        JavaSparkContext ctx = getCtx(sc);
        JavaSQLContext sqlCtx = getSqlCtx(getCtx(sc));
        String path = "/hdd/spark/test.json";
        JavaSchemaRDD test = sqlCtx.applySchema(ctx.textFile(path) ,
ComplexClaim.class);
        sqlCtx.registerRDDAsTable(test, "test");
        execSql(sqlCtx, "select * from test", 1);



*** Approach 2 ***
Created a StructType to map the date field. I got scala.MatchError:
TimestampType (of class org.apache.spark.sql.catalyst.types.TimestampType$).
here is the code:

    public StructType createStructType() {
        List<StructField> fields = new ArrayList<StructField>();
        fields.add(DataType.createStructField("timestamp",
DataType.TimestampType, false));
        return DataType.createStructType(fields);
    }

    public void testJsonStruct(SparkContext sc) {
        JavaSQLContext sqlCtx = getSqlCtx(getCtx(sc));
        String path = "/hdd/spark/test.json";
        JavaSchemaRDD test = sqlCtx.jsonFile(path, createStructType());
        sqlCtx.registerRDDAsTable(test, "test");
        execSql(sqlCtx, "select * from test", 1);
    }

Input file has a single record:
{"timestamp":"2014-10-10T01:01:01"}


Thanks
Tridib





--
View this message in context: 
http://apache-spark-user-list.1001560.n3.nabble.com/spark-sql-timestamp-in-json-fails-tp16864.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org