Hi RJ, I see, are you or the team on working on this problem already? If not I'd like to take a look as well.
Tim On Tue, Oct 28, 2014 at 8:47 AM, RJ Nowling <[email protected]> wrote: > Hi Tim, > > The integration test is simply to open the spark shell (Spark 1.1.0) using > mesos 0.21 in coarse-grained mode. We didn't even have to run any commands. > > RJ > > ----- Original Message ----- >> From: "Timothy Chen" <[email protected]> >> To: [email protected] >> Cc: [email protected], "RJ Nowling" <[email protected]>, "Erik >> Erlandson" <[email protected]> >> Sent: Tuesday, October 28, 2014 11:40:19 AM >> Subject: Re: 0.21.0-pre & Spark latest >> >> Hi Tim, >> >> Thanks for doing the integration tests, that's something that I wanted to do >> but never got to yet. >> >> I have great interest ensuring spark and mesos work, and I know Brenden as >> well does. >> >> I have been tracking these spark mesos problems with spark jira and labeling >> them mesos. Can you create these bugs on jira and we can dig more on each >> one? >> >> Also is this integration test automated? >> >> Thanks! >> >> Tim >> >> > On Oct 28, 2014, at 8:03 AM, Tim St Clair <[email protected]> wrote: >> > >> > inline >> > >> > ----- Original Message ----- >> > >> >> From: "Brenden Matthews" <[email protected]> >> >> To: [email protected] >> >> Cc: "mesos-devel" <[email protected]>, "RJ Nowling" >> >> <[email protected]>, >> >> "Erik Erlandson" <[email protected]> >> >> Sent: Tuesday, October 28, 2014 9:51:58 AM >> >> Subject: Re: 0.21.0-pre & Spark latest >> > >> >> Since we've recently adopted Spark, I'll second Tim's comment. We had an >> >> issue with 0.20.1 that was possibly related to Spark[1], so it's important >> >> for us to get this stuff fixed in 0.21.0. >> > >> >> Tim, can you elaborate on the issues you saw? Have you tested with my >> >> recent >> >> Spark patches[2][3]? >> > >> > We are building against Spark 1.1.0 unpatched: >> > - Fine grained mode appears broken. >> > >> > - Course grained mode appears to work via normal runs, but crashes in the >> > REPL. >> > http://fpaste.org/145782/14506564/ >> > >> >> [1]: https://issues.apache.org/jira/browse/MESOS-1973 >> >> [2]: https://github.com/apache/spark/pull/2401 >> >> [3]: https://github.com/apache/spark/pull/2453 >> > >> >> On Tue, Oct 28, 2014 at 7:46 AM, Tim St Clair < [email protected] > >> >> wrote: >> > >> >>> Folks - >> > >> >>> We have some automated tests that run the latest Mesos against the latest >> >>> Spark, and we've run across a series of issues in both fine, and course >> >>> grained mode that I believe stem from a series of changes in the 0.21 >> >>> cycle. >> > >> >>> I'm not certain if anyone "owns" this integration, but we should probably >> >>> ensure it's fixed before we push out 0.21. >> > >> >>> -- >> >> >> >>> Cheers, >> >> >> >>> Timothy St. Clair >> >> >> >>> Red Hat Inc. >> > >> > -- >> > Cheers, >> > Timothy St. Clair >> > Red Hat Inc. >>
