On Fri, Jun 5, 2015 at 2:05 PM Will Briggs <wrbri...@gmail.com> wrote:
> Your lambda expressions on the RDDs in the SecondRollup class are closing > around the context, and Spark has special logic to ensure that all > variables in a closure used on an RDD are Serializable - I hate linking to > Quora, but there's a good explanation here: > http://www.quora.com/What-does-Closure-cleaner-func-mean-in-Spark > Ah, I see! So if I broke out the lambda expressions into a method on an object it would prevent this issue. Essentially, "don't use lambda expressions when using threads". Thanks again, I appreciate the help.
