Re: sparkavro for PySpark 1.3

Here's what I'm currently trying: -- I'm including --packages com.databricks:spark-avro_2.10:1.0.0 in my pyspark call. This seems to work: Ivy Default Cache set to: /home/jrgregg/.ivy2/cache The jars for the packages stored in: /home/jrgregg/.ivy2/jars ::

Re: sparkavro for PySpark 1.3

Hi You can use newAPIHadoopFile and use AvroInputFormat. On Sun, Dec 6, 2015 at 4:59 AM, YaoPau wrote: > Here's what I'm currently trying: > > -- > I'm including --packages com.databricks:spark-avro_2.10:1.0.0 in my pyspark > call. This

sparkavro for PySpark 1.3

How can I read from and write to Avro using PySpark in 1.3? I can only find the 1.4 documentation , which uses a sqlContext.read method that isn't available to me in 1.3. -- View this message in context: