Hi, On Thu, Dec 4, 2014 at 2:59 AM, Ashic Mahtab <as...@live.com> wrote: > > I've been doing this with foreachPartition (i.e. have the parameters for > creating the singleton outside the loop, do a foreachPartition, create the > instance, loop over entries in the partition, close the partition), but > it's quite cludgy. Is there a pattern by which I can have an instance of > something nonserializable on each worker? >
I think the pattern you describe is the standard way of doing this, several people on this list (including me) have used it for database access etc. Tobias
