Hi Tim, Thanks for doing the integration tests, that's something that I wanted to do but never got to yet.
I have great interest ensuring spark and mesos work, and I know Brenden as well does. I have been tracking these spark mesos problems with spark jira and labeling them mesos. Can you create these bugs on jira and we can dig more on each one? Also is this integration test automated? Thanks! Tim > On Oct 28, 2014, at 8:03 AM, Tim St Clair <tstcl...@redhat.com> wrote: > > inline > > ----- Original Message ----- > >> From: "Brenden Matthews" <brenden.matth...@airbedandbreakfast.com> >> To: user@mesos.apache.org >> Cc: "mesos-devel" <d...@mesos.apache.org>, "RJ Nowling" >> <rnowl...@redhat.com>, >> "Erik Erlandson" <e...@redhat.com> >> Sent: Tuesday, October 28, 2014 9:51:58 AM >> Subject: Re: 0.21.0-pre & Spark latest > >> Since we've recently adopted Spark, I'll second Tim's comment. We had an >> issue with 0.20.1 that was possibly related to Spark[1], so it's important >> for us to get this stuff fixed in 0.21.0. > >> Tim, can you elaborate on the issues you saw? Have you tested with my recent >> Spark patches[2][3]? > > We are building against Spark 1.1.0 unpatched: > - Fine grained mode appears broken. > > - Course grained mode appears to work via normal runs, but crashes in the > REPL. > http://fpaste.org/145782/14506564/ > >> [1]: https://issues.apache.org/jira/browse/MESOS-1973 >> [2]: https://github.com/apache/spark/pull/2401 >> [3]: https://github.com/apache/spark/pull/2453 > >> On Tue, Oct 28, 2014 at 7:46 AM, Tim St Clair < tstcl...@redhat.com > wrote: > >>> Folks - > >>> We have some automated tests that run the latest Mesos against the latest >>> Spark, and we've run across a series of issues in both fine, and course >>> grained mode that I believe stem from a series of changes in the 0.21 >>> cycle. > >>> I'm not certain if anyone "owns" this integration, but we should probably >>> ensure it's fixed before we push out 0.21. > >>> -- >> >>> Cheers, >> >>> Timothy St. Clair >> >>> Red Hat Inc. > > -- > Cheers, > Timothy St. Clair > Red Hat Inc.
