It looks really cool (+1 non binding ;)). FYI, if it can help you, in
ServiceMix, we already ship Activiti with Karaf.
Regards
JB
On 12/14/2012 04:34 PM, Andrei Savu wrote:
Hi guys,
There is no secret that at Axemblr we are using Apache Whirr for
provisioning and initial basic cluster configuration for Hadoop. As soon as
the machines are running we configure Hadoop by leveraging APIs from
existing tools like Cloudera Manager or Ambari.
All the orchestration needed to make this happen is not trivial if you want
the final system to be predictable, robust, restartable and easy to inspect
while running.
A few months ago we've realised that we need to re-work the machine
provisioning layer from Whirr and build a system that has the following
features:
* should be able to provision 10s or 100s of virtual machines by doing a
good job at handling API throttling and by using batch operations as much
as possible
* all the internal workflows should be persistent and as granular as
possible and each step should be idempotent
* it should be possible to restart the application server while starting
virtual machines with no impact
* it should have a modular architecture and provide enough flexibility to
be able to work with a large number of public and private clouds just by
replacing modules
* it should hide all this complexity behind a simple REST API and a simple
interactive shell
* it should be able to automatically build gold base images and use the to
spawn large clusters
We've spent some time looking for existing products that do all this and in
the end we've decided that it's better to start from scratch and build this
system as a new project based on Activiti, Apache Karaf, jclouds and native
sdks.
The source code is now publicly available at:
https://github.com/axemblr/axemblr-provisionr
I would really like to know what you think about the work we've done so
far. The project will improve a lot over the next couple of weeks / months
so I encourage you to stay tunned.
We want to bring this project to the Apache Foundation later on. I will
give a talk in february at ApacheCon NA on this.
Cheers,
-- Andrei Savu / axemblr.com
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
