Re: pyspark EOFError after calling map

Oh great, thank you for clearing that up. On Fri, Apr 22, 2016 at 5:15 PM, Davies Liu wrote: > This exception is already handled well, just noisy, should be muted. > > On Wed, Apr 13, 2016 at 4:52 PM, Pete Werner > wrote: > >> Hi >> >> I am new to

Re: pyspark EOFError after calling map

This exception is already handled well, just noisy, should be muted. On Wed, Apr 13, 2016 at 4:52 PM, Pete Werner wrote: > Hi > > I am new to spark & pyspark. > > I am reading a small csv file (~40k rows) into a dataframe. > > from pyspark.sql import functions as F > df

pyspark EOFError after calling map

Hi I am new to spark & pyspark. I am reading a small csv file (~40k rows) into a dataframe. from pyspark.sql import functions as F df = sqlContext.read.format('com.databricks.spark.csv').options(header='true', inferschema='true').load('/tmp/sm.csv') df = df.withColumn('verified',