Greetings All, Wanted to announce that JobServer, a job scheduling/processing/management application, is now integrated with Mesos.
I put out a brief post on my blog: http://grandlogic.blogspot.com/2013/10/jobserver-with-mesos.html This initial integration with Mesos went a bit better than I expected. JobServer was pretty well suited to have Mesos plugged into it and it fit well with JobServer's architecture. This initial version only has support for the built-in Mesos command executor. I plan to add a custom JobServer executor in the future so jobs can execute with the full plugin API of JobServer (called soafaces). And there is still more work around failover, retrying jobs and resiliency to get Mesos on par with the native JobServer fearures, but this is a pretty good start. This version is pretty capable but still need some work. You can create jobs and schedule them just like any other JobServer job and track them in the monitoring/reporting UIs like any other native JobServer job. You are just limited to executing shell scripts per the Mesos command executor. But overall Mesos jobs have many of the capabilities as native JobServer jobs, they just now run on a Mesos cluster. Also, you can mix and match Mesos clusters with existing statically defined distributed Agent model in JobServer, so users can migrate gradually to Mesos cluster or have a hybrid environment if they choose. JobServer has the concept of Partitions each with its own resources and queues. Users can create many Partitions and jobs run in a Partition with defined resource limits. So Mesos was a perfect fit for this and obviously provided many awesome clustering and resource utilization features that the static distributed Agent model that exists in JobServer did not have. I am still looking at refining how I "fairly" distribute the offered jobs within my Framework so that all the Partitions get a fair crack at the offers. The algo I think is pretty cool but I expect to tweak it a bit. I also added a highlevel Mesos Admin page that lets admins see what is going on, on the Mesos side of things, and how request are being matched up with offers (and if they are not for some reason), to help troubleshoot resource matching issues...etc. I am planning more reporting features as I integrate more with Mesos, so looking forward to all new features like integrating isolation controls as well as gathering/reporting usage stats...etc. Please give JobServer a try if you can and provide any feedback. The current released version is the JobServer Standard edition, which is pretty easy to setup and self contained. Just untar and setup Mesos URL and you should be set to go for the most part and you can then create jobs that run any script on Mesos. Make sure you configure your "Partition" to use Mesos as the cluster type in the Admin UIs. http://www.grandlogic.com/downloads_content/downloads/rawDownload-v3-6.html Thanks for everyone's help on the mailing list. Thanks, Sam Taha http://www.grandlogic.com
