Re: How to read avro in SparkR

2015-06-14 Thread Shing Hing Man
Burak's suggestion works : read.df(sqlContext, file:///home/matmsh/myfile.avro,com.databricks.spark.avro ) Maybe the above should be added to SparkR (R on Spark) - Spark 1.4.0 Documentation |   | |   |   |   |   |   | | SparkR (R on Spark) - Spark 1.4.0 DocumentationSparkR (R on Spark)

Re: How to read avro in SparkR

2015-06-13 Thread Burak Yavuz
Hi, Not sure if this is it, but could you please try com.databricks.spark.avro instead of just avro. Thanks, Burak On Jun 13, 2015 9:55 AM, Shing Hing Man mat...@yahoo.com.invalid wrote: Hi, I am trying to read a avro file in SparkR (in Spark 1.4.0). I started R using the following.

Re: How to read avro in SparkR

2015-06-13 Thread Shivaram Venkataraman
Yep - Burak's answer should work. FWIW the error message from the stack trace that shows this is the line Failed to load class for data source: avro Thanks Shivaram On Sat, Jun 13, 2015 at 6:13 PM, Burak Yavuz brk...@gmail.com wrote: Hi, Not sure if this is it, but could you please try

How to read avro in SparkR

2015-06-13 Thread Shing Hing Man
Hi,  I am trying to read a avro file in SparkR (in Spark 1.4.0). I started R using the following. matmsh@gauss:~$ sparkR --packages com.databricks:spark-avro_2.10:1.0.0 Inside the R shell, when I issue the following, read.df(sqlContext, file:///home/matmsh/myfile.avro,avro) I get the following