: dgoldenberg; user
Subject: Re: Objects serialized before foreachRDD/foreachPartition ?
Considering memory footprint of param as mentioned by Dmitry, option b seems
better.
Cheers
On Wed, Jun 3, 2015 at 6:27 AM, Evo Eftimov evo.efti...@isecc.com wrote:
Hmmm a spark streaming app code
Hmmm a spark streaming app code doesn't execute in the linear fashion
assumed in your previous code snippet - to achieve your objectives you
should do something like the following
in terms of your second objective - saving the initialization and
serialization of the params you can:
a) broadcast
Considering memory footprint of param as mentioned by Dmitry, option b
seems better.
Cheers
On Wed, Jun 3, 2015 at 6:27 AM, Evo Eftimov evo.efti...@isecc.com wrote:
Hmmm a spark streaming app code doesn't execute in the linear fashion
assumed in your previous code snippet - to achieve your
, 2015 2:44 PM
*To:* Evo Eftimov
*Cc:* dgoldenberg; user
*Subject:* Re: Objects serialized before foreachRDD/foreachPartition ?
Considering memory footprint of param as mentioned by Dmitry, option b
seems better.
Cheers
On Wed, Jun 3, 2015 at 6:27 AM, Evo Eftimov evo.efti...@isecc.com
