On May 7, 2013, at 1:12 PM, Dharmesh Kakadia <dhkaka...@gmail.com> wrote:
> I am able to create singel instance LAMP-stack with Chiradeeps's > implementation. I will spend a little time, understanding his code (new to > ruby). Will take it forward from there. > > Thanks, > Dharmesh > That's great news, feel free to write a "how to" and blog it. You can also write it in docbook xml format and send a patch. I have been including third party tools examples in our documentation in the Developer's guide under tools. If you git clone cloudstack, then check under /docs/en-US you will see examples of xml files in docbook. Then check tools.xml ( i think that's the one). -sebastien > On Tue, May 7, 2013 at 1:16 PM, Sebastien Goasguen <run...@gmail.com> wrote: > >> >> On May 1, 2013, at 7:27 AM, Dharmesh Kakadia < >> dharmesh.kaka...@research.iiit.ac.in> wrote: >> >>> Sebastien and Chiradeep, thanks for the comments !! That clarified a lot >> of >>> things. I just read Chiradeep's blog ( >>> >> http://cloudierthanthou.wordpress.com/2013/04/26/stackmate-execute-cloudformation-templates-on-cloudstack/ >> ) >>> which details the service. >>> >>> I am proposing a server side implementation of cloudformation. >>> >>> I misunderstood the ReST and Query API. Thanks for correcting. >> Information >>> here(http://gehrcke.de/2009/06/aws-about-api/) helped me. In case we >> want >>> to use existing AWS tools for cloudformation, we also would be designing >>> Query API, not ReST. >>> >>> Sorry for the confusion regarding cloudmonkey. I was proposing to >> integrate >>> cloudformation API into cloudstack source code, directly and add >>> corresponding support in cloudmonkey. But as you suggested, it might be >>> easy to start with prototype decoupled from cloudstack (Uses cloudstack >> API >>> and does not reside in cloudstack). I assume by existing cloudformation >>> tools you mean AWS tools( >>> http://aws.amazon.com/developertools/AWS-CloudFormation). Reusing them >> will >>> be a really good idea. >> >> Yes first step should be with cloud formation (stackmate) outside >> cloudstack. And I strongly suggest that you make sure it is compatible with >> the AWS tools. >> >>> >>> There are lot of options for configuration mgmt tools. I have used knife >>> previously and good to know that it has cloudstack plugin based on fog ( >>> https://github.com/fifthecho/knife-cloudstack-fog). Reasons rundeck >> looked >>> better was support for rollbacking and is full workflow execution engine. >>> Finally rundeck can use chef/puppet. I have seen provisonr/whirr and they >>> look promising. Definitely a lot to explore here !! >> >> Yes, I have nothing against rundeck. But it would be one more dependency. >> >>> >>> Thanks for suggesting clear proposal. >>> >>> Thanks, >>> Dharmesh >>> >>> >>> >>> >>> >>> On Wed, May 1, 2013 at 3:07 PM, Sebastien Goasguen <run...@gmail.com> >> wrote: >>> >>>> >>>> On Apr 30, 2013, at 4:59 PM, Chiradeep Vittal < >> chiradeep.vit...@citrix.com> >>>> wrote: >>>> >>>>> >>>>> >>>>> On 4/30/13 5:01 AM, "Sebastien Goasguen" <run...@gmail.com> wrote: >>>>> >>>>>> Dharmesh, see in-line >>>>>> >>>>>> On Apr 30, 2013, at 5:34 AM, Dharmesh Kakadia <dhkaka...@gmail.com> >>>> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I am Dharmesh Kakdia and interested in project "Integration project >> to >>>>>>> deploy and use Mesos on a CloudStack based cloud" ( >>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1784) >>>>>>> >>>>>>> I am working on proposal and want to get feedback. Please provide >>>>>>> suggestions :) >>>>>>> >>>>>>> * >>>>>>> >>>>>>> Abstract: >>>>>>> >>>>>>> The project aims to bring cloudformation[1] like service to >> cloudstack. >>>>>>> One >>>>>>> of the prime use-case is cluster computing frameworks on cloudstack. >> A >>>>>>> cloudformation service will give users and administrators of >> cloudstack >>>>>>> ability to manage and control a set of resources easily. The >>>>>>> cloudformation >>>>>>> will allow booting and configuring a set of VMs and form a cluster. >>>>>>> Simple >>>>>>> example would be LAMP stack. More complex clusters such as mesos or >>>>>>> hadoop >>>>>>> cluster requires a little more advanced configuration. There is >> already >>>>>>> some work done by Chiradeep Vittal at this front [5] using route and >>>>>> >>>>>> it's using ruote: http://ruote.rubyforge.org >>>>>> >>>>>>> sinatra. In this project, I will implement cloudformation service and >>>>>>> demonstrate how to run mesos cluster using it. >>>>>> >>>>>> You will create cloud formation templates that describe a mesos >> cluster >>>>>> >>>>>>> >>>>>>> Mesos: >>>>>>> >>>>>>> Mesos is a resource management platform for clusters [2]. It aims to >>>>>>> increase resource utilization of clusters by sharing cluster >> resources >>>>>>> among multiple processing frameworks(like MapReduce, MPI, Graph >>>>>>> Processing) >>>>>>> or multiple instances of same framework. It provides efficient >> resource >>>>>>> isolation through use of containers. Uses zookeeper for state >>>>>>> maintenance >>>>>>> and fault tolerance. >>>>>>> >>>>>>> What can run on mesos ? >>>>>>> >>>>>>> Spark: A cluster computing framework based on the Resilient >> Distributed >>>>>>> Datasets (RDDs) abstraction. RDD is more generalized than MapReduce >> and >>>>>>> can >>>>>>> support iterative and interactive computation while retaining fault >>>>>>> tolerance, scalability, data locality etc. >>>>>>> >>>>>>> Hadoop: Hadoop is fault tolerant and scalable distributed computing >>>>>>> framework based on MapReduce abstraction. >>>>>>> >>>>>>> Begel: A graph processing framework based on pregel. >>>>>>> >>>>>>> and other frameworks like MPI, Hypertable. >>>>>>> >>>>>>> How to deploy mesos >>>>>>> >>>>>>> Mesos provides cluster installation scripts [7] for cluster >> deployment. >>>>>>> There are also scripts available to deploy a cluster on Amazon EC2 >> [8]. >>>>>> >>>>>> It would be nice to see if these scripts can be used as is with the >>>>>> CloudStack EC2 service. >>>>>> >>>>>>> >>>>>>> Deliverables: >>>>>>> >>>>>>> 1. Cloudformation service implementation on cloudstack. >>>>>>> >>>>>>> 2. Integration of cloudformation with cloudmonkey, CLI tool. >>>>>> >>>>>> 2. is a little confusing. I believe that what Chiradeep prototype runs >>>> on >>>>>> the client side. What is needed is a server side implementation. >>>>>> That way we could use existing cloudformation cli tools to talk to it. >>>>>> I don't understand where cloudmonkey comes into play. CloudMonkey is a >>>>>> cli for the CloudStack API. Unless you plan to integrate the >>>>>> cloudformation API directly in the cloudstack source code, the >>>>>> integration you propose is not clear to me. >>>>>> >>>>> >>>>> Sebastien is correct. I intend to put in the query API server around >> the >>>>> core of stack mate soon (as soon as I'm done helping on the internal >>>>> loadbalancer). This will be written in Ruby. >>>>> >>>>> >>>> >>>> Dharmesh I suggest you propose the following: >>>> >>>> 1-Deploy CloudStack and understand instance >> configuration/contextualization >>>> 2-Test and deploy Mesos on a set of CloudStack based VM, manually. >>>> Design/propose an automation framework. >>>> 3-Test stackmate and engage chiradeep (report bugs, make suggestion, >> make >>>> pull request) >>>> 4-Create cloud formation template to provision a Mesos Cluster >>>> 5-Compare with Apache Whirr or other cluster provisioning tools. >>>> 6-Potentially if you see a link with cloudmonkey, see how you could >> extend >>>> it to talk to stackmate in a similar manner that it talks to CloudStack. >>>> >>>> >>>> You are pretty close and this is a very exciting projects, so go ahead, >>>> modify a bit your proposal and submit it. >>>> >>>> Deadline for applications is this Friday May 3rd. >>>> >>>> -sebastien >>>> >>>> >>>> >>>> >> >>
