Ideas?
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-1-0-0-RDD-from-snappy-compress-avro-file-tp19998p20267.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
Hi Vikas and Simone,
thanks for the replies.
Yeah I understand this would be easier with 1.2 but this is completely out
of my control. I really have to work with 1.0.0.
About Simone's approach, during the imports I get:
/scala import org.apache.avro.mapreduce.{ AvroJob, AvroKeyInputFormat,
btw the same error from above also happen on 1.1.0 (just tested)
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-1-0-0-RDD-from-snappy-compress-avro-file-tp19998p20106.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
Just in case it helps: https://github.com/databricks/spark-avro
On Fri, Nov 28, 2014 at 8:48 PM, cjdc cristovao.corde...@cern.ch wrote:
To make it simpler, for now forget the snappy compression. Just assume they
are binary Avro files...
--
View this message in context:
Did you have a look at my reply in this thread?
http://apache-spark-user-list.1001560.n3.nabble.com/How-can-I-read-this-avro-file-using-spark-amp-scala-td19400.html
I am using 1.1.0 though, so not sure if that code would work entirely with
1.0.0, but you can try.
Simone Franzini, PhD
Hi everyone,
I am using Spark 1.0.0 and I am facing some issues with handling binary
snappy compressed avro files which I get form HDFS. I know there are
improved mechanisms to handle these files on more recent version of Spark,
but updating is not an option since I am operating on a Cloudera
To make it simpler, for now forget the snappy compression. Just assume they
are binary Avro files...
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-1-0-0-RDD-from-snappy-compress-avro-file-tp19998p20008.html
Sent from the Apache Spark User List
