: making dataframe for different types using spark-csv
hi Mohammed Guller!
How can I specify schema in load method?
On Thu, Jul 2, 2015 at 6:43 AM, Mohammed Guller
moham...@glassbeam.commailto:moham...@glassbeam.com wrote:
Another option is to provide the schema to the load method. One variant
*To:* Hafiz Mujadid
*Cc:* user@spark.apache.org
*Subject:* Re: making dataframe for different types using spark-csv
· use .cast(...).alias('...') after the DataFrame is read.
· sql.functions.udf for any domain-specific conversions.
Cheers
k/
On Wed, Jul 1, 2015 at 11:03 AM, Hafiz Mujadid
-dataframe-for-different-types-using-spark-csv-tp23570.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h
@spark.apache.org
*Subject:* Re: making dataframe for different types using spark-csv
· use .cast(...).alias('...') after the DataFrame is read.
· sql.functions.udf for any domain-specific conversions.
Cheers
k/
On Wed, Jul 1, 2015 at 11:03 AM, Hafiz Mujadid hafizmujadi...@gmail.com
wrote
Mohammed
From: Krishna Sankar [mailto:ksanka...@gmail.com]
Sent: Wednesday, July 1, 2015 3:09 PM
To: Hafiz Mujadid
Cc: user@spark.apache.org
Subject: Re: making dataframe for different types using spark-csv
· use .cast(...).alias('...') after the DataFrame is read.
· sql.functions.udf for any
it makes
type of each column as string. Is there some way to get dataframe of actual
types like int,double etc.?
Thanks
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/making-dataframe-for-different-types-using-spark-csv-tp23570.html
Sent from the Apache
