Re: Spark Code to read RCFiles

2014-09-24 Thread cem
with disabling kryo serializer. Best Regards, Cem Cayiroglu On Tue, Sep 23, 2014 at 7:23 PM, Matei Zaharia matei.zaha...@gmail.com wrote: Is your file managed by Hive (and thus present in a Hive metastore)? In that case, Spark SQL ( https://spark.apache.org/docs/latest/sql-programming-guide.html

Re: Spark Code to read RCFiles

2014-09-24 Thread cem
I used the following code as an example to deserialize BytesRefArrayWritable. http://www.massapi.com/source/hive-0.5.0-dev/src/ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java.html Best Regards, Cem. On Wed, Sep 24, 2014 at 1:34 PM, Pramod Biligiri pramodbilig...@gmail.com wrote

Spark processing small files.

2014-09-16 Thread cem
Regards, Cem Cayiroglu