Re: Help with object access from mapper (simple question)

2014-06-23 Thread Yana Kadiyska
Thank you so much! I was trying for a singleton and opted against a class but clearly this backfired. Clearly time to revisit Scala lessons. Thanks again On Mon, Jun 23, 2014 at 1:16 PM, Marcelo Vanzin wrote: > "object" in Scala is similar to a class with only static fields / > methods in Java.

Re: Help with object access from mapper (simple question)

2014-06-23 Thread Marcelo Vanzin
"object" in Scala is similar to a class with only static fields / methods in Java. So when you set its fields in the driver, the "object" does not get serialized and sent to the executors; they have their own copy of the class and its static fields, which haven't been initialized. Use a proper cla

Help with object access from mapper (simple question)

2014-06-23 Thread Yana Kadiyska
Hi folks, hoping someone can explain to me what's going on: I have the following code, largely based on RecoverableNetworkWordCount example ( https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/streaming/RecoverableNetworkWordCount.scala ): I am setting f