Re: Mailbomb from amplabs jenkins ?

2014-03-27 Thread Patrick Wendell
Yeah sorry guys - Jenkins is having some issues and there isn't a way to fix this that doesn't spam people following github. Apologies! On Thu, Mar 27, 2014 at 8:16 PM, Nan Zhu wrote: > yes, it sends for every PR you were involved > > I think Patrick is doing something on Jenkins, he just stopp

Re: Mailbomb from amplabs jenkins ?

2014-03-27 Thread Nan Zhu
yes, it sends for every PR you were involved I think Patrick is doing something on Jenkins, he just stopped some testing jobs manually Best, -- Nan Zhu On Thursday, March 27, 2014 at 11:07 PM, Mridul Muralidharan wrote: > Got some 100 odd mails from jenkins (?) with "Can one of the admin

Mailbomb from amplabs jenkins ?

2014-03-27 Thread Mridul Muralidharan
Got some 100 odd mails from jenkins (?) with "Can one of the admins verify this patch?" Part of upgrade or some other issue ? Significantly reduced the snr of my inbox ! Regards, Mridul

Re: Scala 2.10.4

2014-03-27 Thread Matei Zaharia
Sounds good. Feel free to send a PR even though it’s a small change (it leads to better Git history and such). Matei On Mar 27, 2014, at 4:15 PM, Mark Hamstra wrote: > FYI, Spark master does build cleanly and the tests do run successfully with > Scala version set to 2.10.4, so we can probably

Scala 2.10.4

2014-03-27 Thread Mark Hamstra
FYI, Spark master does build cleanly and the tests do run successfully with Scala version set to 2.10.4, so we can probably bump 1.0.0-SNAPSHOT to use the new version anytime we care to.

Re: Any suggestion about JIRA 1006 "MLlib ALS gets stack overflow with too many iterations"?

2014-03-27 Thread Debasish Das
Hi Matei, I am hitting similar problems with 10 ALS iterations...I am running with 24 gb executor memory on 10 nodes for 20M x 3 M matrix with rank =50 The first iteration of flatMaps run fine which means that the memory requirements are good per iteration... If I do check-pointing on RDD, most

Re: Error executing sql using shark 0.9.0 / hadoop 2.2.0 - incompatible protobuf 2.5 and 2.4.1

2014-03-27 Thread yao
So it looks like shark introduces protobuf-2.4, are you sure shark depends on the right spark version ? I am not familiar with shark's dependencies, but in your case, you could delete the protobuf-2.4 and it should not affect anything. On Wed, Mar 26, 2014 at 6:24 PM, qingyang li wrote: > my spa

Re: [DISCUSS] Shepherding PRs

2014-03-27 Thread Mark Hamstra
So what happens when having a shepherd is not helpful, e.g. when the designated shepherd gets encumbered with a pile of higher-priority responsibilities while the PR being shepherded is still in process? On Thu, Mar 27, 2014 at 10:12 AM, Evan Chan wrote: > +1. > > I really like this idea. I kn

Re: [DISCUSS] Shepherding PRs

2014-03-27 Thread Evan Chan
+1. I really like this idea. I know having a shepherd would have been really helpful for a couple of changes. On Thu, Mar 27, 2014 at 8:18 AM, Andy Konwinski wrote: > I thought this email exchange from the Mesos dev list was worth sharing. > The Mesos project is trying out a process wherein th

[DISCUSS] Shepherding PRs

2014-03-27 Thread Andy Konwinski
I thought this email exchange from the Mesos dev list was worth sharing. The Mesos project is trying out a process wherein they assign shepherds (who are committers) to significant issues. I'm not proposing that this necessarily makes sense for us, but I thought it might be worth discussing. Andy

Re: Spark 0.9.1 release

2014-03-27 Thread Tathagata Das
I have cut another release candidate, RC3, with two important bug fixes. See the following JIRAs for more details. 1. Bug with intercepts in MLLib's GLM: https://spark-project.atlassian.net/browse/SPARK-1327 2. Bug in PySpark's RDD.top() ordering: https://spark-project.atlassian.net/browse/SPARK-13

Re: [VOTE] Release Apache Spark 0.9.1 (RC2)

2014-03-27 Thread Tathagata Das
This vote has been cancelled to accommodate two important bug fixes. See JIRA issues for more details. 1. Bug with intercepts in MLLib's GLM: https://spark-project.atlassian.net/browse/SPARK-1327 2. Bug in PySpark's RDD.top() ordering: https://spark-project.atlassian.net/browse/SPARK-1322 TD On

Re: [spark-streaming] Is this LocalInputDStream useful to someone?

2014-03-27 Thread Pascal Voitot Dev
Hi guys, Nobody saw/answered my question so I resend it ;) Thanks Pascal On Thu, Mar 20, 2014 at 9:17 AM, Pascal Voitot Dev < pascal.voitot@gmail.com> wrote: > Hi guys, > > In my recent blog post (http://mandubian.com/2014/03/08/zpark-ml-nio-1/), > I needed to have an InputDStream helper lo