Another option is to provide the schema to the load method. One variant of the sqlContext.load takes a schema as a input parameter. You can define the schema programmatically as shown here:
https://spark.apache.org/docs/latest/sql-programming-guide.html#programmatically-specifying-the-schema Mohammed From: Krishna Sankar [mailto:ksanka...@gmail.com] Sent: Wednesday, July 1, 2015 3:09 PM To: Hafiz Mujadid Cc: user@spark.apache.org Subject: Re: making dataframe for different types using spark-csv · use .cast("...").alias('...') after the DataFrame is read. · sql.functions.udf for any domain-specific conversions. Cheers [https://ssl.gstatic.com/ui/v1/icons/mail/images/cleardot.gif]<k/> On Wed, Jul 1, 2015 at 11:03 AM, Hafiz Mujadid <hafizmujadi...@gmail.com<mailto:hafizmujadi...@gmail.com>> wrote: Hi experts! I am using spark-csv to lead csv data into dataframe. By default it makes type of each column as string. Is there some way to get dataframe of actual types like int,double etc.? Thanks -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/making-dataframe-for-different-types-using-spark-csv-tp23570.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org<mailto:user-unsubscr...@spark.apache.org> For additional commands, e-mail: user-h...@spark.apache.org<mailto:user-h...@spark.apache.org>
