: dgoldenberg; user
Subject: Re: Objects serialized before foreachRDD/foreachPartition ?
Considering memory footprint of param as mentioned by Dmitry, option b seems
better.
Cheers
On Wed, Jun 3, 2015 at 6:27 AM, Evo Eftimov evo.efti...@isecc.com wrote:
Hmmm a spark streaming app code
[mailto:dgoldenberg...@gmail.com]
Sent: Wednesday, June 3, 2015 1:56 PM
To: user@spark.apache.org
Subject: Objects serialized before foreachRDD/foreachPartition ?
I'm looking at https://spark.apache.org/docs/latest/tuning.html. Basically
the takeaway is that all objects passed into the code processing RDD's
.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Objects-serialized-before-foreachRDD-foreachPartition-tp23134.html
Sent from the Apache Spark User List mailing list archive at Nabble.com
serialized before foreachRDD/foreachPartition ?
I'm looking at https://spark.apache.org/docs/latest/tuning.html.
Basically
the takeaway is that all objects passed into the code processing RDD's must
be serializable. So if I've got a few objects that I'd rather initialize
once and deinitialize once
, 2015 2:44 PM
*To:* Evo Eftimov
*Cc:* dgoldenberg; user
*Subject:* Re: Objects serialized before foreachRDD/foreachPartition ?
Considering memory footprint of param as mentioned by Dmitry, option b
seems better.
Cheers
On Wed, Jun 3, 2015 at 6:27 AM, Evo Eftimov evo.efti...@isecc.com
