Re: Using third party libraries in pyspark

2015-01-22 Thread Davies Liu
You need to install these libraries on all the slaves, or submit via spark-submit: spark-submit --py-files xxx On Thu, Jan 22, 2015 at 11:23 AM, Mohit Singh mohit1...@gmail.com wrote: Hi, I might be asking something very trivial, but whats the recommend way of using third party libraries.

Using third party libraries in pyspark

2015-01-22 Thread Mohit Singh
Hi, I might be asking something very trivial, but whats the recommend way of using third party libraries. I am using tables to read hdf5 format file.. And here is the error trace: print rdd.take(2) File /tmp/spark/python/pyspark/rdd.py, line , in take res =

Re: Using third party libraries in pyspark

2015-01-22 Thread Felix C
party libraries in pyspark You need to install these libraries on all the slaves, or submit via spark-submit: spark-submit --py-files xxx On Thu, Jan 22, 2015 at 11:23 AM, Mohit Singh mohit1...@gmail.com wrote: Hi, I might be asking something very trivial, but whats the recommend way of using