On Apr 30, 2013, at 4:59 PM, Chiradeep Vittal <chiradeep.vit...@citrix.com> wrote:
> > > On 4/30/13 5:01 AM, "Sebastien Goasguen" <run...@gmail.com> wrote: > >> Dharmesh, see in-line >> >> On Apr 30, 2013, at 5:34 AM, Dharmesh Kakadia <dhkaka...@gmail.com> wrote: >> >>> Hi, >>> >>> I am Dharmesh Kakdia and interested in project "Integration project to >>> deploy and use Mesos on a CloudStack based cloud" ( >>> https://issues.apache.org/jira/browse/CLOUDSTACK-1784) >>> >>> I am working on proposal and want to get feedback. Please provide >>> suggestions :) >>> >>> * >>> >>> Abstract: >>> >>> The project aims to bring cloudformation[1] like service to cloudstack. >>> One >>> of the prime use-case is cluster computing frameworks on cloudstack. A >>> cloudformation service will give users and administrators of cloudstack >>> ability to manage and control a set of resources easily. The >>> cloudformation >>> will allow booting and configuring a set of VMs and form a cluster. >>> Simple >>> example would be LAMP stack. More complex clusters such as mesos or >>> hadoop >>> cluster requires a little more advanced configuration. There is already >>> some work done by Chiradeep Vittal at this front [5] using route and >> >> it's using ruote: http://ruote.rubyforge.org >> >>> sinatra. In this project, I will implement cloudformation service and >>> demonstrate how to run mesos cluster using it. >> >> You will create cloud formation templates that describe a mesos cluster >> >>> >>> Mesos: >>> >>> Mesos is a resource management platform for clusters [2]. It aims to >>> increase resource utilization of clusters by sharing cluster resources >>> among multiple processing frameworks(like MapReduce, MPI, Graph >>> Processing) >>> or multiple instances of same framework. It provides efficient resource >>> isolation through use of containers. Uses zookeeper for state >>> maintenance >>> and fault tolerance. >>> >>> What can run on mesos ? >>> >>> Spark: A cluster computing framework based on the Resilient Distributed >>> Datasets (RDDs) abstraction. RDD is more generalized than MapReduce and >>> can >>> support iterative and interactive computation while retaining fault >>> tolerance, scalability, data locality etc. >>> >>> Hadoop: Hadoop is fault tolerant and scalable distributed computing >>> framework based on MapReduce abstraction. >>> >>> Begel: A graph processing framework based on pregel. >>> >>> and other frameworks like MPI, Hypertable. >>> >>> How to deploy mesos >>> >>> Mesos provides cluster installation scripts [7] for cluster deployment. >>> There are also scripts available to deploy a cluster on Amazon EC2 [8]. >> >> It would be nice to see if these scripts can be used as is with the >> CloudStack EC2 service. >> >>> >>> Deliverables: >>> >>> 1. Cloudformation service implementation on cloudstack. >>> >>> 2. Integration of cloudformation with cloudmonkey, CLI tool. >> >> 2. is a little confusing. I believe that what Chiradeep prototype runs on >> the client side. What is needed is a server side implementation. >> That way we could use existing cloudformation cli tools to talk to it. >> I don't understand where cloudmonkey comes into play. CloudMonkey is a >> cli for the CloudStack API. Unless you plan to integrate the >> cloudformation API directly in the cloudstack source code, the >> integration you propose is not clear to me. >> > > Sebastien is correct. I intend to put in the query API server around the > core of stack mate soon (as soon as I'm done helping on the internal > loadbalancer). This will be written in Ruby. > > Dharmesh I suggest you propose the following: 1-Deploy CloudStack and understand instance configuration/contextualization 2-Test and deploy Mesos on a set of CloudStack based VM, manually. Design/propose an automation framework. 3-Test stackmate and engage chiradeep (report bugs, make suggestion, make pull request) 4-Create cloud formation template to provision a Mesos Cluster 5-Compare with Apache Whirr or other cluster provisioning tools. 6-Potentially if you see a link with cloudmonkey, see how you could extend it to talk to stackmate in a similar manner that it talks to CloudStack. You are pretty close and this is a very exciting projects, so go ahead, modify a bit your proposal and submit it. Deadline for applications is this Friday May 3rd. -sebastien
