I'm impressed! On 14 December 2012 15:34, Andrei Savu <[email protected]> wrote:
> Hi guys, > > There is no secret that at Axemblr we are using Apache Whirr for > provisioning and initial basic cluster configuration for Hadoop. As soon as > the machines are running we configure Hadoop by leveraging APIs from > existing tools like Cloudera Manager or Ambari. > > All the orchestration needed to make this happen is not trivial if you want > the final system to be predictable, robust, restartable and easy to inspect > while running. > > A few months ago we've realised that we need to re-work the machine > provisioning layer from Whirr and build a system that has the following > features: > > * should be able to provision 10s or 100s of virtual machines by doing a > good job at handling API throttling and by using batch operations as much > as possible > > * all the internal workflows should be persistent and as granular as > possible and each step should be idempotent > > * it should be possible to restart the application server while starting > virtual machines with no impact > > * it should have a modular architecture and provide enough flexibility to > be able to work with a large number of public and private clouds just by > replacing modules > > * it should hide all this complexity behind a simple REST API and a simple > interactive shell > > * it should be able to automatically build gold base images and use the to > spawn large clusters > > We've spent some time looking for existing products that do all this and in > the end we've decided that it's better to start from scratch and build this > system as a new project based on Activiti, Apache Karaf, jclouds and native > sdks. > > The source code is now publicly available at: > > https://github.com/axemblr/axemblr-provisionr > > I would really like to know what you think about the work we've done so > far. The project will improve a lot over the next couple of weeks / months > so I encourage you to stay tunned. > > We want to bring this project to the Apache Foundation later on. I will > give a talk in february at ApacheCon NA on this. > > Cheers, > > -- Andrei Savu / axemblr.com >
