Hi RJ, I see, are you or the team on working on this problem already? If not I'd like to take a look as well.
Tim On Tue, Oct 28, 2014 at 8:47 AM, RJ Nowling <rnowl...@redhat.com> wrote: > Hi Tim, > > The integration test is simply to open the spark shell (Spark 1.1.0) using > mesos 0.21 in coarse-grained mode. We didn't even have to run any commands. > > RJ > > ----- Original Message ----- >> From: "Timothy Chen" <tnac...@gmail.com> >> To: d...@mesos.apache.org >> Cc: user@mesos.apache.org, "RJ Nowling" <rnowl...@redhat.com>, "Erik >> Erlandson" <e...@redhat.com> >> Sent: Tuesday, October 28, 2014 11:40:19 AM >> Subject: Re: 0.21.0-pre & Spark latest >> >> Hi Tim, >> >> Thanks for doing the integration tests, that's something that I wanted to do >> but never got to yet. >> >> I have great interest ensuring spark and mesos work, and I know Brenden as >> well does. >> >> I have been tracking these spark mesos problems with spark jira and labeling >> them mesos. Can you create these bugs on jira and we can dig more on each >> one? >> >> Also is this integration test automated? >> >> Thanks! >> >> Tim >> >> > On Oct 28, 2014, at 8:03 AM, Tim St Clair <tstcl...@redhat.com> wrote: >> > >> > inline >> > >> > ----- Original Message ----- >> > >> >> From: "Brenden Matthews" <brenden.matth...@airbedandbreakfast.com> >> >> To: user@mesos.apache.org >> >> Cc: "mesos-devel" <d...@mesos.apache.org>, "RJ Nowling" >> >> <rnowl...@redhat.com>, >> >> "Erik Erlandson" <e...@redhat.com> >> >> Sent: Tuesday, October 28, 2014 9:51:58 AM >> >> Subject: Re: 0.21.0-pre & Spark latest >> > >> >> Since we've recently adopted Spark, I'll second Tim's comment. We had an >> >> issue with 0.20.1 that was possibly related to Spark[1], so it's important >> >> for us to get this stuff fixed in 0.21.0. >> > >> >> Tim, can you elaborate on the issues you saw? Have you tested with my >> >> recent >> >> Spark patches[2][3]? >> > >> > We are building against Spark 1.1.0 unpatched: >> > - Fine grained mode appears broken. >> > >> > - Course grained mode appears to work via normal runs, but crashes in the >> > REPL. >> > http://fpaste.org/145782/14506564/ >> > >> >> [1]: https://issues.apache.org/jira/browse/MESOS-1973 >> >> [2]: https://github.com/apache/spark/pull/2401 >> >> [3]: https://github.com/apache/spark/pull/2453 >> > >> >> On Tue, Oct 28, 2014 at 7:46 AM, Tim St Clair < tstcl...@redhat.com > >> >> wrote: >> > >> >>> Folks - >> > >> >>> We have some automated tests that run the latest Mesos against the latest >> >>> Spark, and we've run across a series of issues in both fine, and course >> >>> grained mode that I believe stem from a series of changes in the 0.21 >> >>> cycle. >> > >> >>> I'm not certain if anyone "owns" this integration, but we should probably >> >>> ensure it's fixed before we push out 0.21. >> > >> >>> -- >> >> >> >>> Cheers, >> >> >> >>> Timothy St. Clair >> >> >> >>> Red Hat Inc. >> > >> > -- >> > Cheers, >> > Timothy St. Clair >> > Red Hat Inc. >>
