f its structure. It
> doesn't
> convert to RDD but uses RDD partitions to produce logical plan.
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/RDD-and-Dataframes-tp27306p27346.html
> Sent from the Apache Spark Us
DataFrames uses RDDs as internal implementation of its structure. It doesn't
convert to RDD but uses RDD partitions to produce logical plan.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/RDD-and-Dataframes-tp27306p27346.html
Sent from the Apache Spark
>> step
>> it will be transformed into a RDD to be executed in Spark?
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-spark-user-list.1001560.n3.nabble.com/RDD-and-Dataframes-tp27306.html
>> Sent from the Apache Spark User List mail
t:
> http://apache-spark-user-list.1001560.n3.nabble.com/RDD-and-Dataframes-tp27306.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> -
> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
>
>
, in the final step
it will be transformed into a RDD to be executed in Spark?
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/RDD-and-Dataframes-tp27306.html
Sent from the Apache Spark User List mailing list archive at Nabble.com
On Mon, Jun 29, 2015 at 1:27 PM, Axel Dahl a...@whisperstream.com wrote:
In pyspark, when I convert from rdds to dataframes it looks like the rdd is
being materialized/collected/repartitioned before it's converted to a
dataframe.
It's not true. When converting a RDD to dataframe, it only take
In pyspark, when I convert from rdds to dataframes it looks like the rdd is
being materialized/collected/repartitioned before it's converted to a
dataframe.
Just wondering if there's any guidelines for doing this conversion and
whether it's best to do it early to get the performance benefits of
