Re: Design advice

2013-09-10 Thread Sam Taha
Yes that was my thinking for the slave/execution side of things. My partitions should map pretty well to slave executors/tasks and the cgroup/isolation capabilities of mesos will be great benefit. However, I still have some cases where users want to run each job (even jobs in the same partition gro

Re: Design advice

2013-09-10 Thread Vinod Kone
Sam, Glad to see you are interested in building a framework on top of Mesos! >From your description, it looks like your "partitions" can be directly mapped to Mesos "executors" and "jobs" to Mesos "tasks". In Mesos, each executor is run under a cgroup with a given set of resources. This enables m

Design advice

2013-09-10 Thread Sam Taha
I am hoping to get some advice on how to build out my framework scheduler for a job scheduling/processing platform, so I don't paint myself in a corner with any potential limitations in mesos resource allocation or the java SchedulerDriver capabilities. Some background: - I have a JVM based job sc

HBase

2013-09-10 Thread Kyle Ellrott
Does anybody have any experience with running an HBase cluster under mesos? Has anybody written a framework for it yet? Kyle