Re: Looking for volunteers to help improve Mesos documentation

2013-10-31 Thread Sam Taha
What will be the process to submit changes? Will be the same process that is document for submitting code patches along with the same review process? I can help in the area of document get started writing a Framework. Thanks, Sam Taha http://www.grandlogic.com On Thu, Oct 31, 2013 at 9:26 PM

Re: Recent blog post about Slave Recovery

2013-10-23 Thread Sam Taha
good post and nice to hear about the new slave recovery feature and futures as well. Thanks, Sam Taha http://www.grandlogic.com On Wed, Oct 23, 2013 at 11:28 AM, Dave Lester wrote: > I recently pushed changes to the Mesos website that added a blog. The > initial post by Vinod Kone g

Application Dependency Managaement ala YARN

2013-10-22 Thread Sam Taha
eals with these of lifecycle resource manage application dependencies across the cluster? Here is theYARN overview of what I am kind of asking about: http://hortonworks.com/blog/management-of-application-dependencies-in-yarn/ Thanks in advance. Thanks, Sam Taha http://www.grandlogic.com

Powered by Mesos page

2013-10-22 Thread Sam Taha
Would it be possible to add Grand Logic and JobServer to the organizations and products using and built on Mesos? Thanks, Sam Taha http://www.grandlogic.com

JobServer scheduling and processing jobs on Mesos

2013-10-21 Thread Sam Taha
go for the most part and you can then create jobs that run any script on Mesos. Make sure you configure your "Partition" to use Mesos as the cluster type in the Admin UIs. http://www.grandlogic.com/downloads_content/downloads/rawDownload-v3-6.html Thanks for everyone's help on the m

Re: process isolation

2013-10-21 Thread Sam Taha
/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/sec-cpu.html Ben On Mon, Sep 30, 2013 at 1:20 PM, Sam Taha wrote: > What does it mean to request fractiional CPU resources like 0.5 or 1.5. Is > this meaningful when making requests and how does it relate to isolation > lev

Re: Check status of Framework connection

2013-10-18 Thread Sam Taha
s own and I will start receiving offers? Or will I have to reconnect from my end by periodically checking if the master/zookeeper is back up again? Thanks, Sam Taha http://www.grandlogic.com On Fri, Oct 18, 2013 at 5:38 PM, Vinod Kone wrote: > You are right. disconnected() callback is no

Re: Check status of Framework connection

2013-10-18 Thread Sam Taha
think. But I would think either way I should get a callback disconnect if the single master or master/zookeeper cluster are shutdown either manually or crash? Thanks, Sam Taha http://www.grandlogic.com On Fri, Oct 18, 2013 at 4:42 PM, Sam Taha wrote: > When my Framework has connected to

Re: Check status of Framework connection

2013-10-18 Thread Sam Taha
When my Framework has connected to the master, I do not seem to get the Scheduler.disconnected() callback when I kill the Master. Is this expected behavior or should my Framework get a disconnected() callback if I kill the master? Thanks, Sam Taha http://www.grandlogic.com On Wed, Oct 9, 2013

Re: Resource utilization stats

2013-10-18 Thread Sam Taha
hanks, Sam Taha http://www.grandlogic.com On Fri, Oct 18, 2013 at 2:40 PM, Benjamin Mahler wrote: > The stats on that endpoint are instantaneous values, we keep historical > stats but do not yet expose them. > > What kind of "basic usage stats" would you benefit from rec

Re: Resource utilization stats

2013-10-18 Thread Sam Taha
fired. I capture stats in my local app database as I track the events per job. Thanks, Sam Taha http://www.grandlogic.com On Fri, Oct 18, 2013 at 1:34 PM, Benjamin Mahler wrote: > Hey Sam, > > You can use the json monitoring endpoint on the slave to retrieve cpu > usage information

Re: Resource utilization stats

2013-10-17 Thread Sam Taha
Looks like it is not implemented On Thu, Oct 17, 2013 at 7:55 AM, Sam Taha wrote: > Is there currently a way for a Framework to get back CPU/memory usage > statistics as a task is running or after the task has finished? From Java > API specifically? > > Thanks, >

Re: Resource utilization stats

2013-10-17 Thread Sam Taha
https://issues.apache.org/jira/browse/MESOS-581 On Thu, Oct 17, 2013 at 3:05 PM, Sam Taha wrote: > Looks like it is not implemented > > > > On Thu, Oct 17, 2013 at 7:55 AM, Sam Taha wrote: > >> Is there currently a way for a Framework to get back CPU/memory usage &g

Resource utilization stats

2013-10-17 Thread Sam Taha
Is there currently a way for a Framework to get back CPU/memory usage statistics as a task is running or after the task has finished? From Java API specifically? Thanks, Sam Taha http://www.grandlogic.com

Check status of Framework connection

2013-10-09 Thread Sam Taha
urs ago and I may have declined offers, but I still need to check if my Framework is connected. Thanks, Sam Taha http://www.grandlogic.com

Re: Matching a single Offer with multiple Requests

2013-10-01 Thread Sam Taha
ld be cool. Thanks, Sam Taha http://www.grandlogic.com On Tue, Oct 1, 2013 at 1:37 PM, Vinod Kone wrote: > On Tue, Oct 1, 2013 at 9:55 AM, Sam Taha wrote: > >> Sorry, just noticed that SchedulerDriver.launchTasks() takes a List of >> Tasks, so I guess you can launch mul

Re: Matching a single Offer with multiple Requests

2013-10-01 Thread Sam Taha
. Thanks, Sam Taha http://www.grandlogic.com On Tue, Oct 1, 2013 at 12:32 PM, Sam Taha wrote: > Simple example scenario: > > If my Framework/Scheduler gets an Offer for say 2 cpu and 10G (from a > single Slave/OfferID) and let's say I have two job requests that each need > 1

Matching a single Offer with multiple Requests

2013-10-01 Thread Sam Taha
r the next time Mesos presents me with the rest of the Offer resource in order to launch my second request or do I track all this on my Framework side as I am matching Offers to Requests? Thanks, Sam Taha http://www.grandlogic.com

Re: CPU resource requests

2013-09-30 Thread Sam Taha
tps://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/sec-cpu.html > > Ben > > > On Mon, Sep 30, 2013 at 1:20 PM, Sam Taha wrote: > >> What does it mean to request fractiional CPU resources like 0.5 or 1.5. >> Is this me

CPU resource requests

2013-09-30 Thread Sam Taha
What does it mean to request fractiional CPU resources like 0.5 or 1.5. Is this meaningful when making requests and how does it relate to isolation levels. For example, you can't really pin 1.5 cores in a cgroup. Should I really bother letting users/jobs request less than 1 cpu? Thanks, Sam

Re: What is the recommended use of SchedulerDriver.requestResources

2013-09-30 Thread Sam Taha
yea that would be nice to have to improve more efficient allocation especially when multiple frameworks are involved. Thanks, Sam Taha http://www.grandlogic.com On Sun, Sep 29, 2013 at 6:53 PM, Vinod Kone wrote: > Hey Sam. "requestResources" call is currently a no-op. In t

What is the recommended use of SchedulerDriver.requestResources

2013-09-29 Thread Sam Taha
I thought mesos offers up ALL the resources to the framework that uses least amount of resources. Thanks, Sam Taha http://www.grandlogic.com

Re: Aurora, Marathon and long lived job frameworks

2013-09-27 Thread Sam Taha
enterprise type job scheduling/monitoring/tracking...etc features. Thanks, Sam Taha http://www.grandlogic.com On Fri, Sep 27, 2013 at 12:59 PM, Dan Colish wrote: > > On Fri, Sep 27, 2013 at 9:04 AM, Damien Hardy wrote: > >> Hello, >> >> What about chronos http://airbnb.github

Re: Mesos Scheduler/Executor coordination mechanism

2013-09-26 Thread Sam Taha
invokes the script, I assume passes the executor ID and related setup data to the java executor shell script when it is started to properly setup the executor configs. Thanks, Sam Taha http://www.grandlogic.com On Thu, Sep 26, 2013 at 3:26 PM, Vladimir Vivien wrote: > Vinod > Thanks.

Re: Reliable Task Launching

2013-09-18 Thread Sam Taha
her job if resources are a match for the next job. Thanks, Sam Taha http://www.grandlogic.com On Wed, Sep 18, 2013 at 5:16 AM, Bernerd Schaefer wrote: > This is somewhat related to the existing thread about messaging > reliability, though specific to launching tasks. > > I noticed th

Re: Companies providing professional support for Mesos

2013-09-16 Thread Sam Taha
Hi Dave, Grand Logic plans to offer support services along with product integration via our job automation system - JobServer. It would be great if we can get listed. Thanks, Sam Taha http://www.grandlogic.com On Mon, Sep 16, 2013 at 8:24 PM, Dave Lester wrote: > Hi All, > > What

Re: Declining all offers

2013-09-16 Thread Sam Taha
start sending me offers as soon as it can and ignore the previous decline time period, correct? Thanks, Sam Taha http://www.grandlogic.com On Sun, Sep 15, 2013 at 6:04 PM, Vinod Kone wrote: > Hey Sam. You can set a timeout in the "Filters" when you decline an offer. > That will

Declining all offers

2013-09-15 Thread Sam Taha
If I want decline ALL offers for say the next 5 minutes, is there an effective way to do that without completely stopping the framework and disconnecting from the master? Write now I am just shutting down the driver and then starting new driver/scheduler when new jobs are ready. Thanks, Sam Taha

Design advice

2013-09-10 Thread Sam Taha
with mesos) or can I create one per Partition thread. The per partition thread approach can give me finer resource control (request, Filters,etc) but I am not sure if this is practical. Thanks, Sam Taha http://www.grandlogic.com

Re: Design advice

2013-09-10 Thread Sam Taha
executor for this...I think). I will take your advice and build a central framework for now and have it dispatch resources to each Partition thread. Thanks for you help. Sam Taha http://www.grandlogic.com On Tue, Sep 10, 2013 at 1:01 PM, Vinod Kone wrote: > Sam, > > Glad to see you are i

Re: Some more question on resource sharing

2013-09-09 Thread Sam Taha
I am still in learning phase with the project :) but from what I have gathered Mesos makes offers to multiple frameworks and the first framework to accept gets the resources and others get rescinded. Thanks, Sam Taha http://grandlogic.com On Mon, Sep 9, 2013 at 5:47 PM, Li Jin wrote: >

Required JARs for java Framework

2013-09-09 Thread Sam Taha
Greetings, I am building a java Framework Scheduler. Is it correct that there are only three required JARs: mesos jar zookeeper jar protocol buffer jar Is this accurate? Thanks, Sam Taha http://grandlogic.com