Re: BUG: when running as extends App, closures don't capture variables

2014-10-30 Thread Sean Owen
Very coincidentally I ran into something equally puzzling yesterday where something was bizarrely null when it can't have been in a Spark program that extends App. I also changed to use main() and it works fine. So definitely some issue here. If nobody makes a JIRA before I get home I'll do it. On

BUG: when running as extends App, closures don't capture variables

2014-10-29 Thread Michael Albert
Greetings! This might be a documentation issue as opposed to a coding issue, in that perhaps the correct answer is don't do that, but as this is not obvious, I am writing. The following code produces output most would not expect: package misc import org.apache.spark.SparkConfimport

Re: BUG: when running as extends App, closures don't capture variables

2014-10-29 Thread Matei Zaharia
Good catch! If you'd like, you can send a pull request changing the files in docs/ to do this (see https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark), otherwise maybe open an issue on