Re: Reading CSV file using pyspark

Hi Devesh, you have to start your SPARK Shell using the packages. The command is mentioned below (you can use pyspark instead of spark-shell), anyways all the required commands for this is mentioned here https://github.com/databricks/spark-csv and I prefer using the 2.11 version instead of 2.10

Re: Reading CSV file using pyspark

download a right version of this jar http://mvnrepository.com/artifact/com.databricks/spark-csv_2.10 (or 2.11), and append it to SPARK_CLASSPATH 2016-02-18 11:05 GMT+01:00 Devesh Raj Singh : > Hi, > > I want to read CSV file in pyspark > > I am running pyspark on pycharm

Reading CSV file using pyspark

Hi, I want to read CSV file in pyspark I am running pyspark on pycharm I am trying to load a csv using pyspark import os import sys os.environ['SPARK_HOME']="/Users/devesh/Downloads/spark-1.5.1-bin-hadoop2.6" sys.path.append("/Users/devesh/Downloads/spark-1.5.1-bin-hadoop2.6/python/") # Now