subject:"Error \- Spark reading from HDFS via dataframes \- Java"

RE: Error - Spark reading from HDFS via dataframes - Java

2017-10-02 Thread JG Perrin

<kpra...@salesforce.com> Cc: user @spark <user@spark.apache.org> Subject: Re: Error - Spark reading from HDFS via dataframes - Java Hi, Set the inferschema option to true in spark-csv. you may also want to set the mode option. See readme below https://github.com/databricks/spark-csv

Re: Error - Spark reading from HDFS via dataframes - Java

2017-10-01 Thread Anastasios Zouzias

Hi, Set the inferschema option to true in spark-csv. you may also want to set the mode option. See readme below https://github.com/databricks/spark-csv/blob/master/README.md Best, Anastasios Am 01.10.2017 07:58 schrieb "Kanagha Kumar" : Hi, I'm trying to read data

Error - Spark reading from HDFS via dataframes - Java

2017-09-30 Thread Kanagha Kumar

Hi, I'm trying to read data from HDFS in spark as dataframes. Printing the schema, I see all columns are being read as strings. I'm converting it to RDDs and creating another dataframe by passing in the correct schema ( how the rows should be interpreted finally). I'm getting the following