Are the resource options documented?

2015-08-25 Thread craig w
When configuring a mesos-slave with --resources, I know cpu, mem and ports are available. Are there others? Are these documented somewhere? I've found some examples here https://open.mesosphere.com/reference/mesos-slave/ and the configuration page

Mesos/Marathon/HAProxy Logging

2015-08-25 Thread John Omernik
I have been playing with an application that is a very simple app: A webservice running in Python. I've created a docker container, it runs in the container, I setup marathon to run it, I use mesos-dns and ha proxy and I can access the service just fine anywhere in the cluster. First let me say

Re: Not getting resource offers for 20 min

2015-08-25 Thread CCAAT
THANKS, as I have not kept up on the spark lists James On 08/25/2015 04:28 AM, Iulian Dragoș wrote: On Mon, Aug 24, 2015 at 7:16 PM, CCAAT cc...@tampabay.rr.com mailto:cc...@tampabay.rr.com wrote: On 08/24/2015 05:33 AM, Iulian Dragoș wrote: Hello Iulian, Ok, so I

Re: Mesos/Marathon/HAProxy Logging

2015-08-25 Thread Jeff Schroeder
This is the header that should be passed: https://en.m.wikipedia.org/wiki/X-Forwarded-For Most of the modern internet routes through reverse proxies and this is how we log the actual source clients to solve similar auditing and compliance needs. On Tuesday, August 25, 2015, John Omernik

Re: Are the resource options documented?

2015-08-25 Thread Alex Rukletsov
From Mesos point of view, a resource is just a string, your agents may advertise gpu, bananas, pandas and so on. However, some resources are known to Mesos, and for them isolation is possible. A good example is a cgroups isolator for mem resources, which will invoke OOM killer if necessary.

Re: Mesos/Marathon/HAProxy Logging

2015-08-25 Thread Ankur Chauhan
This may help: http://serverfault.com/questions/331079/haproxy-and-forwarding-client-ip-address-to-servers We use similar options to ensure we have the remote ip. On 25 Aug 2015, at 09:30, John Omernik j...@omernik.com wrote: I have been playing with an application that is a very simple

Re: Custom Scheduler: Diagnosing cause of container task failures

2015-08-25 Thread Alex Rukletsov
It looks like we can have a better error message here. @Jay, mind filing a JIRA ticket for with description, status update, and your fix attached? Thanks! On Fri, Aug 21, 2015 at 7:36 PM, Jay Taylor j...@jaytaylor.com wrote: Eventually I was able to isolate what was going on; in this case the

Re: Master UI - Tasks section is empty

2015-08-25 Thread Rogier Dikkes
Thank you Craig, just ran into this myself when updating to 0.23 On 8/23/15 7:59 PM, craig w wrote: See https://issues.apache.org/jira/browse/MESOS-3282 On Aug 23, 2015 12:28 PM, Jeremy Olexa jol...@spscommerce.com mailto:jol...@spscommerce.com wrote: Hi all, On a new cluster, the

Re: SSL in Mesos 0.23

2015-08-25 Thread Joris Van Remoortere
@Dharmit If you want to be really sure that the communication is happening over SSL, you can use a packet sniffing tool like wireshark, or depending on your operating system you can dump the packet streams directly to a file. For example TCP dump. Another thing you can do is to try and hit the

Re: SSL in Mesos 0.23

2015-08-25 Thread Joris Van Remoortere
@carlos Are you building 0.23.0 from source? Just so we don't miss anything: Can you make sure to run ./bootstrap, and build in a clean directory with your configuration similar to this: ../configure --enable-libevent --enable-ssl Here http://mesos.apache.org/documentation/latest/mesos-ssl/ is

Re: SSL in Mesos 0.23

2015-08-25 Thread Carlos Sanchez
Hi Joris, I did build from sources, following instructions in http://mesos.apache.org/gettingstarted/ Is the mesosphere binary compiled with libevent and ssl enabled as mentioned previously? would make debugging easier if I don't have to rebuild On Tue, Aug 25, 2015 at 8:52 PM, Joris Van

Re: Are the resource options documented?

2015-08-25 Thread haosdent
Also have disk resource. It is documented in attributes-resources.md https://github.com/apache/mesos/blob/master/docs/attributes-resources.md On Wed, Aug 26, 2015 at 1:31 AM, craig w codecr...@gmail.com wrote: When configuring a mesos-slave with --resources, I know cpu, mem and ports are

Re: Allocation algorithm

2015-08-25 Thread Vinod Kone
The hierarchical allocator looks at one agent's resource at a time. For each agent, it runs DRF to figure out the candidate framework. More details here: https://github.com/apache/mesos/blob/master/src/master/allocator/mesos/hierarchical.hpp#L935 Regarding starvation you observed, yes that is

Re: Allocation algorithm

2015-08-25 Thread Iulian Dragoș
On Mon, Aug 24, 2015 at 5:42 PM, Hans van den Bogert hansbog...@gmail.com wrote: Can anyone tell how the Mesos allocation algorithm works: Does Mesos offer every free resource it has to one framework at a time? Or does the allocator divide the max offer size by the amount of active/registered

Re: Not getting resource offers for 20 min

2015-08-25 Thread Iulian Dragoș
On Mon, Aug 24, 2015 at 7:16 PM, CCAAT cc...@tampabay.rr.com wrote: On 08/24/2015 05:33 AM, Iulian Dragoș wrote: Hello Iulian, Ok, so I eventually build spark from 100% sources, after some intermediate builds on gentoo. Gentoo is not the best platform for Java development, but those

Re: Mesos/Marathon/HAProxy Logging

2015-08-25 Thread John Omernik
So I agree that is how it should be done, however the current implementation on Mesos, requires me to manually code something like. In addition, this is only for http traffic, not tcp... what happens when the service running on Mesos isn't HTTP? I was hoping for some discussion beyond just

Re: Not getting resource offers for 20 min

2015-08-25 Thread Hans van den Bogert
Wanted to add that, even if there wasn’t a preview package, you can clone from GIT, and checkout a tag, where in this case v1.5.0-rc1 is tagged. Then proceeded normally as you would’ve had a source distro as described in the already mentioned

slave_ping_timeout 1secs

2015-08-25 Thread Nastooh Avessta (navesta)
Hi Running Mesos 0.23.0 and noted that cannot start mesos-master with slave_ping_timeout less than 1 second, tried 0.5secs, 500ms and 50us, etc. Is this by design or am I missing something? Cheers, [http://www.cisco.com/web/europe/images/email/signature/logo05.jpg] Nastooh Avessta

Re: slave_ping_timeout 1secs

2015-08-25 Thread Yan Xu
Yes: https://github.com/apache/mesos/blob/5de7ea455ec577e19c67a75b1cf98493b40c53fb/src/master/flags.cpp#L383 Was the error message not shown in stderr? -- Jiang Yan Xu y...@jxu.me @xujyan http://twitter.com/xujyan On Tue, Aug 25, 2015 at 5:41 PM, Nastooh Avessta (navesta) nave...@cisco.com

RE: slave_ping_timeout 1secs

2015-08-25 Thread Nastooh Avessta (navesta)
I see. Thank you for the clarification. Can I just change the boundaries in the source code, to suit my needs, or there is more to it? Cheers, [http://www.cisco.com/web/europe/images/email/signature/logo05.jpg] Nastooh Avessta ENGINEER.SOFTWARE ENGINEERING nave...@cisco.com Phone: +1 604 647

Re: SSL in Mesos 0.23

2015-08-25 Thread Joris Van Remoortere
@Carlos Mesosphere currently doesn't build packages with ssl enabled. On Tue, Aug 25, 2015 at 3:12 PM, Carlos Sanchez car...@apache.org wrote: Hi Joris, I did build from sources, following instructions in http://mesos.apache.org/gettingstarted/ Is the mesosphere binary compiled with