Re: Mesos is now a maven module

Thank you all for quick fix! :D Dongjoon. On Tuesday, August 30, 2016, Michael Gummelt wrote: > https://github.com/apache/spark/pull/14885 > > Thanks > > On Tue, Aug 30, 2016 at 11:36 AM, Marcelo Vanzin

Re: Mesos is now a maven module

https://github.com/apache/spark/pull/14885 Thanks On Tue, Aug 30, 2016 at 11:36 AM, Marcelo Vanzin wrote: > On Tue, Aug 30, 2016 at 11:32 AM, Sean Owen wrote: > > Ah, I helped miss that. We don't enable -Pyarn for YARN because it's > > already always

Re: Mesos is now a maven module

On Tue, Aug 30, 2016 at 11:32 AM, Sean Owen wrote: > Ah, I helped miss that. We don't enable -Pyarn for YARN because it's > already always set? I wonder if it makes sense to make that optional > in order to speed up builds, or, maybe I'm missing a reason it's > always

Re: Mesos is now a maven module

Ah, I helped miss that. We don't enable -Pyarn for YARN because it's already always set? I wonder if it makes sense to make that optional in order to speed up builds, or, maybe I'm missing a reason it's always essential. I think it's not setting -Pmesos indeed because no Mesos code was changed

Re: Mesos is now a maven module

Thank you for confirming, Sean and Marcelo! Bests, Dongjoon. On Tue, Aug 30, 2016 at 10:05 AM, Marcelo Vanzin wrote: > A quick look shows that maybe dev/sparktestsupport/modules.py needs to > be modified, and a "build_profile_flags" added to the mesos section > (similar to

Re: Mesos is now a maven module

A quick look shows that maybe dev/sparktestsupport/modules.py needs to be modified, and a "build_profile_flags" added to the mesos section (similar to hive / hive-thriftserver). Note not all PR builds will trigger mesos currently, since it's listed as an independent module in the above file. On

Re: Mesos is now a maven module

I have the heady power to modify Jenkins jobs now, so I will carefully take a look at them and see if any of the config needs -Pmesos. But yeah I thought this should be baked into the script. On Tue, Aug 30, 2016 at 5:56 PM, Dongjoon Hyun wrote: > Hi, Michael. > > It's a

Re: Mesos is now a maven module

Michael added the profile to the build scripts, but maybe some script or code path was missed... On Tue, Aug 30, 2016 at 9:56 AM, Dongjoon Hyun wrote: > Hi, Michael. > > It's a great news! > > BTW, I'm wondering if the Jenkins (SparkPullRequestBuilder) knows this new >

Re: Mesos is now a maven module

Hi, Michael. It's a great news! BTW, I'm wondering if the Jenkins (SparkPullRequestBuilder) knows this new profile, -Pmesos. The PR was passed with the following Jenkins build arguments without `-Pmesos` option. (at the last test) ``` [info] Building Spark (w/Hive 1.2.1) using SBT with these

Re: Mesos is now a maven module

If it's separable, then sure. Consistency is nice. On Fri, Aug 26, 2016 at 2:14 PM, Jacek Laskowski wrote: > Hi Michael, > > Congrats! > > BTW What I like about the change the most is that it uses the > pluggable interface for TaskScheduler and SchedulerBackend (as >

Re: Mesos is now a maven module

Hi Michael, Congrats! BTW What I like about the change the most is that it uses the pluggable interface for TaskScheduler and SchedulerBackend (as introduced by YARN). Think Standalone should follow the steps. WDYT? Pozdrawiam, Jacek Laskowski https://medium.com/@jaceklaskowski/ Mastering

Re: Mesos is now a maven module

This is great! On Fri, Aug 26, 2016 at 1:20 PM, Michael Gummelt wrote: > Hello devs, > > Much like YARN, Mesos has been refactored into a Maven module. So when > building, you must add "-Pmesos" to enable Mesos support. > > The pre-built distributions from Apache will

Mesos is now a maven module

Hello devs, Much like YARN, Mesos has been refactored into a Maven module. So when building, you must add "-Pmesos" to enable Mesos support. The pre-built distributions from Apache will continue to enable Mesos. PR: https://github.com/apache/spark/pull/14637 Cheers -- Michael Gummelt