Re: Can Mesos support same multi-frameworks in same cluster?

You can pretty much run as many frameworks as you'd like. One trick you can use to drive up CPU use (if you want) is to overcommit CPUs. For example, if you have a machine with 16 CPUs, set the number of CPUs to 32 on the Mesos agent, and you'll be overcommitted by 100% (i.e., 1 CPU share will pra

Re: Welcome Anand and Joseph as new committers!

Welcome and congrats! On Fri, Jun 10, 2016 at 2:11 PM, Benjamin Mahler wrote: > Welcome Anand and Joseph, thanks for all of your contributions! > > Looking forward to seeing your ongoing positive influences on the > community and the project, let's build great software! > > On Thu, Jun 9, 2016 a

Re: marathon-lb at scale

As a sidenote, I wouldn't recommend running marathon-lb on every node in the cluster. Running it on 3-5 should be sufficient for HA. You can simply round-robin (with DNS) between the marathon-lb instances (using, say, Mesos-DNS). The additional round trip delay you save by running marathon-lb on ea

Re: marathon-lb at scale

I'd suggest you also try posting on the Marathon group: https://groups.google.com/forum/#!forum/marathon-framework On Tue, Mar 1, 2016 at 6:21 PM, Bill Green wrote: > For folks using marathon-lb, how far have you scaled it? I’d be very > interested to hear your experiences with it, especially in

Re: Selectively enable/disable oom killer on tasks

You may also want to consider giving your tasks more memory. On Fri, Dec 11, 2015 at 1:00 PM, Ankan Mukherjee wrote: > Thanks for clearing that up. I realize that going beyond allocated cpu > will get the container throttled (only.) > > We use both cgroups/cpu and cgroups/mem isolators. However,

Re: [VOTE] Release Apache Mesos 0.26.0 (rc5)

+1 (binding) Make check'd on OS X with SSL & libevent. On Fri, Dec 11, 2015 at 11:11 AM, Benjamin Mahler wrote: > +1 (binding) > > Make check on OS X with and without '--enable-ssl --enable-libevent'. > > On Thu, Dec 10, 2015 at 2:55 PM, Till Toenshoff wrote: > >> Hi friends, >> >> we did unfo

Re: Welcome Kapil as Mesos committer and PMC member!

Awesome! Welcome Kapil! On Thu, Nov 5, 2015 at 5:20 PM, Benjamin Mahler wrote: > Congrats and welcome! > > On Thu, Nov 5, 2015 at 4:04 PM, Kapil Arya wrote: > >> Thanks everyone! :-). >> >> >> >> On Thu, Nov 5, 2015 at 3:50 PM, Elizabeth Lingg >> wrote: >> >>> Congrats Professor! You definitel

Re:

raft the configuration (because we can config like we need) instead of use > something that could change soon and we don't have much control. > > @Rad, Thank you, It will help a lot. > > > > > > On Mon, Oct 12, 2015 at 11:45 AM, Brenden Matthews > wrote: >

Re: [VOTE] Release Apache Mesos 0.25.0 (rc3)

+1 (binding) Tested on CI. On Sun, Oct 11, 2015 at 4:12 AM, Michael Park wrote: > +1 (binding) > > Ran *make distcheck* successfully on Ubuntu 14.04 with gcc + clang, CentOS > 7.1 with gcc > Ran *make check* with one non-blocker failure (MESOS-3604 >

Re:

It's worth noting that there's a purpose-built framework for Cassandra: https://github.com/mesosphere/cassandra-mesos You probably want to use this instead of trying to run C* on Marathon. On Sun, Oct 11, 2015 at 6:05 PM, Rad Gruchalski wrote: > Rafael, > > According to the cassandra documentat

Re: [VOTE] Release Apache Mesos 0.22.2 (rc1)

+1 (binding), tested on internal CI On Thu, Sep 24, 2015 at 8:58 AM, haosdent wrote: > Oh, got it. CentOS 6.6 don't have automake 1.4, so have to build it for > source code. I fetch it from > https://git-wip-us.apache.org/repos/asf/mesos.git before. > > On Thu, Sep 24, 2015 at 11:29 PM, Adam Bor

Re: [VOTE] Release Apache Mesos 0.23.1 (rc1)

+1 (binding) On Thu, Sep 24, 2015 at 3:21 PM, Niklas Nielsen wrote: > +1 (binding) > > Tested on centos7 and ubuntu 14.04 > > On 24 September 2015 at 07:53, Alexander Rojas > wrote: > >> +1 (non binding) >> >> Tested Ubuntu 14.04, OSX >> >> > On 22 Sep 2015, at 03:06, Adam Bordelon wrote: >> >

Re: 答复: [DISCUSS] Renaming Mesos Slave

+1 to what Adam wrote. 1. Mesos Worker [Node] 2. Mesos Worker or Agent 3. No 4. Carefully On Fri, Jun 5, 2015 at 6:31 AM, Sam Salisbury wrote: > Master/Minion +1 > > On 5 June 2015 at 15:14, CCAAT wrote: > >> >> "+1 master/slave, no change needed." is the same as >> "master/slave"I.E. kee

Re: Spark on Mesos / Executor Memory

I ran into some issues with it a while ago, and submitted a couple PRs to fix it: https://github.com/apache/spark/pull/2401 https://github.com/apache/spark/pull/3024 Do these look relevant? What version of Spark are you running? On Sat, Apr 11, 2015 at 9:33 AM, Tom Arnfeld wrote: > Hey, > > No

Re: Mesos Hadoop Framework 0.1.0

This is great, thanks Tom. On Sat, Mar 28, 2015 at 6:52 AM, Tom Arnfeld wrote: > Hey everyone, > > > I thought it best to send an email to the list before merging and tagging > a 0.1.0 release for the Hadoop on Mesos framework. This release is for a > new feature we've been working on for quite

Re: [VOTE] Release Apache Mesos 0.22.0 (rc4)

+1 Tested with internal testing cluster. On Wed, Mar 18, 2015 at 1:25 PM, craig w wrote: > +1 > > On Wed, Mar 18, 2015 at 3:52 PM, Niklas Nielsen > wrote: > >> Hi all, >> >> Please vote on releasing the following candidate as Apache Mesos 0.22.0. >> >> >> 0.22.0 includes the following: >> >> -

Re: libsubversion-1 is required for mesos to build.

You likely need to install the `libsvn-dev` package. On Thu, Jan 29, 2015 at 1:35 PM, Dan Dong wrote: > Hi, > When I tried to build mesos-0.21.0 on Ubuntu-14.04, I get this error: > --- > libsubversion-1 is required for mesos to b

Re: Storm on Mesos, Anyone Using?

laborate a bit on those shortcomings :-) > > On Tue, Jan 20, 2015 at 11:06 AM, Brenden Matthews > wrote: > >> Hi Cory, >> >> We were using the project in production at Airbnb. It may have some >> shortcomings, but it does, in fact, work. >> >> >

Re: Storm on Mesos, Anyone Using?

Hi Cory, We were using the project in production at Airbnb. It may have some shortcomings, but it does, in fact, work. > Hello all! > I'm interested in Storm on Mesos, but my coworkers don't wanna be guinea > pigs. Is anyone using mesos/storm in > production? I s

Re: hadoop job stuck.

mesos, but it > stucks when running on mesos. > > Cheers, > Dan > > 2015-01-14 15:02 GMT-06:00 Brenden Matthews : > > At a first glance, it looks like `/etc/hosts` might be set incorrectly and >> it cannot resolve the hostname of the worker. >> >> S

Re: hadoop job stuck.

At a first glance, it looks like `/etc/hosts` might be set incorrectly and it cannot resolve the hostname of the worker. See here for more: https://wiki.apache.org/hadoop/UnknownHost On Wed, Jan 14, 2015 at 12:32 PM, Vinod Kone wrote: > What do the master logs say? > > On Wed, Jan 14, 2015 at 1

Transferring Chronos project to the community

Hello Mesos users, I'm pleased to announce that Airbnb has decided to transfer the Chronos project to community ownership. It will now live under the umbrella of the top-level Apache Mesos project, as a GitHub hosted project at https://github.com/mesos/chronos. Airbnb continues to use and maintai

Re: 0.21.0-pre & Spark latest

Since we've recently adopted Spark, I'll second Tim's comment. We had an issue with 0.20.1 that was possibly related to Spark[1], so it's important for us to get this stuff fixed in 0.21.0. Tim, can you elaborate on the issues you saw? Have you tested with my recent Spark patches[2][3]? [1]: ht

Re: Does Mesos support Hadoop MR V2

Porting YARN to run atop Mesos is quite reasonable. Some folks at eBay have started some work on this (https://github.com/mesos/myriad). If you're interested, you should check it out, and contribute to the project. On Tue, Oct 28, 2014 at 5:21 AM, Yaneeve Shekel wrote: > To quote John below,

Re: Impala on Mesos

We've run Presto on Marathon, but not yet Impala. Seems like it should be pretty straightforward. Let me know how it goes. On Wed, Sep 24, 2014 at 8:18 PM, Night Wolf wrote: > Is it possible to run Impala on Mesos? > > Has anyone tried this before? > > I know there is Llama

Re: spark and mesos issue

I started hitting a similar problem, and it seems to be related to memory overhead and tasks getting OOM killed. I filed a ticket here: https://issues.apache.org/jira/browse/SPARK-3535 On Wed, Jul 16, 2014 at 5:27 AM, Ray Rodriguez wrote: > I'll set some time aside today to gather and post som

Re: [RESULT][VOTE] Release Apache Mesos 0.20.0 (rc2)

> I don't know why maven central does not show the jar. Any one has any clue? > > - Jie > > > On Mon, Aug 25, 2014 at 5:28 PM, Brenden Matthews < > brenden.matth...@airbedandbreakfast.com> wrote: > >> Hey Jie, >> >> I'm not seeing t

Re: [RESULT][VOTE] Release Apache Mesos 0.20.0 (rc2)

Hey Jie, I'm not seeing the jar in the maven repo: http://search.maven.org/#search%7Cgav%7C1%7Cg%3A%22org.apache.mesos%22%20AND%20a%3A%22mesos%22 Did something go awry when publishing the jar? On Sun, Aug 24, 2014 at 4:41 PM, Jie Yu wrote: > Thanks Dave! > > > On Sun, Aug 24, 2014 at 4:39 PM

Re: Struggling with task controller Permissions on Hadoop Mesos

I'm not sure which file you're referring to (since that doesn't exist), but I can assure you the permissions are set correctly. On Tue, Aug 19, 2014 at 9:14 AM, John Omernik wrote: > Can you do me a favor? on one of your running tasks, or recently completed > tasks, in the Mesos task, click on

Re: Mesos + storm on top of Docker

676:26(0x7f3573010700):ZOO_WARN@zookeeper_interest@1557: > Exceeded deadline by 13ms > 2014-08-18 19:20:48,375:26(0x7f3573010700):ZOO_WARN@zookeeper_interest@1557: > Exceeded deadline by 12ms > W0818 19:22:33.935534 244 sched.cpp:901] Attempting to launch task > slave-31001

Re: Struggling with task controller Permissions on Hadoop Mesos

This may not solve your problem, but I will point out that we run the JT (and therefore all the TTs) as the user `mapred`. On Tue, Aug 19, 2014 at 4:14 AM, John Omernik wrote: > Yes > > > On Monday, August 18, 2014, Vinod Kone wrote: > >> >> On Sat, Aug 16, 2014 at 4:26 AM, John Omernik wrote

Re: Mesos + storm on top of Docker

Is the hostname set correctly on the machine running nimbus? It looks like that may not be correct. On Mon, Aug 18, 2014 at 9:39 AM, Yaron Rosenbaum wrote: > @vinodkone > > Finally found some relevant logs.. > Let's start with the slave: > > slave_1 | I0818 16:18:51.700827 9 slave.cpp:

Re: MesosCon attendee introduction thread

I'm Brenden, an engineer at Airbnb. I'm the one responsible for our Mesos clusters. We run Hadoop, Storm, Chronos, and Marathon. I'm a Mesos contributor, and I'm also the maintainer/contributor for a number of related projects. I've been hacking away on computers since I was old enough to reach

Re: [VOTE] Release Apache Mesos 0.18.1 (rc2)

+1 Make check'd on Debian sid with: gcc version 4.8.2 (Debian 4.8.2-21) On Thu, May 1, 2014 at 11:27 PM, Manivannan wrote: > +1 > > > On Fri, May 2, 2014 at 11:38 AM, Vinod Kone wrote: > >> +1 >> >> make check passes on OSX 10.9 w/ gcc-4.8 >> >> >> On Wed, Apr 30, 2014 at 11:18 PM, Niklas Nie

Re: [VOTE] Release Apache Mesos 0.18.0 (rc6)

+1! On Thu, Apr 3, 2014 at 9:00 AM, Tim St Clair wrote: > make check passes, and setup mini cluster so, +1 > > --- > However, fwiw I decided to verify against Spark-0.9 and it appears they > had some implied context, which I think is actually a bu

Re: Multiple role behaviour

Are you specifying the correct role in the TaskInfo when launching the task? You would normally receive an offer for the '*' role as well as your 'role1' role. The offer for '*' would contain whatever resources are available, and the 'role1' offer would contain the remaining unreserved resources