I am trying to read csv in spark dataframe . My Os = Ubuntu 18.04, spark-version 2.3.1, python -version 2.7.15 My code : from pyspark import SparkConf from pyspark import SparkContext from pyspark import SQLContext from pyspark.sql import SparkSession conf = SparkConf() sc = SparkContext(conf = conf) spark = SparkSession.builder.config(conf=conf).appName('FinancialRecon').getOrCreate() sqlContext =SQLContext(sc) df=spark.read.csv('/home/iconnect4/finrecon/test2.csv') df.show()
Error : Traceback (most recent call last): File "/home/iconnect4/finrecon/scratch_3.py", line 43, in <module> df=sqlContext.read.csv('/home/iconnect4/finrecon/test2.csv') File "/usr/local/spark/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 441, in csv File "/usr/local/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1257, in __call__ File "/usr/local/spark/python/lib/pyspark.zip/pyspark/sql/utils.py", line 79, in deco pyspark.sql.utils.IllegalArgumentException: 'Illegal pattern component: XXX' Please help -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org