Re: ClosureCleaner slowing down Spark SQL queries

2015-05-27 Thread Nitin Goyal
Hi Ted, Thanks a lot for replying. First of all, moving to 1.4.0 RC2 is not easy for us as migration cost is big since lot has changed in Spark SQL since 1.2. Regarding SPARK-7233, I had already looked at it few hours back and it solves the problem for concurrent queries but my problem is just

[build system] jenkins downtime tomorrow morning ~730am PDT

2015-05-27 Thread shane knapp
i'm going to be performing system, jenkins, and plugin updates tomorrow morning beginning at 730am PDT. 0700: pause build queue 0800: kill off any errant jobs (retrigger when everything comes back up) 0800-0900: system and plugin updates 0900-1000: final debugging, roll back versions of

RE: Spark 1.4.0 pyspark and pylint breaking

2015-05-27 Thread Michael Nazario
I've done some investigation into what work needed to be done to keep the _types module named types. This isn't a relative / absolute path problem, but actually a problem with the way the tests were run. I've filed a jira ticket on it here: https://issues.apache.org/jira/browse/SPARK-7899) I

Re: SparkR and RDDs

2015-05-27 Thread Andrew Psaltis
Hi Shivaram, Thanks for the details, it is greatly appreciated. Thanks On Wed, May 27, 2015 at 7:25 PM, Shivaram Venkataraman shiva...@eecs.berkeley.edu wrote: Sorry for the delay in getting back on this. So the RDD interface is private in the 1.4 release but as Alek mentioned you can still

Re: [VOTE] Release Apache Spark 1.4.0 (RC2)

2015-05-27 Thread Patrick Wendell
Hi James, As I said before that is not a blocker issue for this release, thanks. Separately, there are some comments in this code review that indicate you may be facing a bug in your own code rather than with Spark: https://github.com/apache/spark/pull/5688#issuecomment-104491410 Please follow

Re: [VOTE] Release Apache Spark 1.4.0 (RC2)

2015-05-27 Thread jameszhouyi
-1 , SPARK-7119 blocker issue -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/VOTE-Release-Apache-Spark-1-4-0-RC2-tp12420p12472.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com.