.nabble.com/Re-Use-Case-of-mutable-RDD-any-ideas-around-will-help-tp14095p14112.html
Sent from the Apache Spark User List mailing list archive at
Nabble.com.
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
as it will surpass total available memory in system.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Re-Use-Case-of-mutable-RDD-any-ideas-around-will-help-tp14095p14112.html
Sent from the Apache Spark User List mailing list archive at Nabble.com
[moving to user@]
This would typically be accomplished with a union() operation. You
can't mutate an RDD in-place, but you can create a new RDD with a
union() which is an inexpensive operator.
On Fri, Sep 12, 2014 at 5:28 AM, Archit Thakur
archit279tha...@gmail.com wrote:
Hi,
We have a use
the RDD as schema RDD uses
columnar compression
existingRDD.union(newRDD).registerAsTable(newTable)
sqlContext.cacheTable(newTable) -- duplicated data
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Re-Use-Case-of-mutable-RDD-any-ideas-around-will-help
) -- duplicated data
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Re-Use-Case-of-mutable-RDD-any-ideas-around-will-help-tp14095p14107.html
Sent from the Apache Spark User List mailing list archive at Nabble.com
be fast enough so that user does not get
irritated.
Also i cannot keep two copies of data(till newrdd materialize) into memory
as it will surpass total available memory in system.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Re-Use-Case-of-mutable-RDD
cannot keep two copies of data(till newrdd materialize) into memory
as it will surpass total available memory in system.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Re-Use-Case-of-mutable-RDD-any-ideas-around-will-help-tp14095p14112.html
Sent from
