Adam,
The things you have brought up here are most timely. As we begin to address
the way that Crowbar2 will be packaged to facilitate the creation and
deployment of admin nodes and workloads we need to adopt a different strategy
from the Crowbar1 ISO build approach. The Crowbar project should not concern
itself with the need to build a re-mastered operating system ISO. I believe
the preferred approach is to build a set of OS platform packages (RPM or Deb -
as needed for the target admin node platform).
My thinking on this is to build platform-independent, LSB compliant, packages
for all core Crowbar2 components. There will be a need for one
platform-dependent package that will contain the Crowbar2 repo info as well as
repos for all its dependencies, plus it will contain the pattern info for the
Crowbar2 infrastructure plus its dependencies, as well as the initialization
scripts that will deliver a working admin node and then launch a browser
attached to http://localhost:3000 so that the administrator is ready to move to
the next guided step.
Packaging the barclamps as gems that are converted into wrappered RPMS makes
sense to me. Please see my in-line comments below.
- John T.
-----Original Message-----
From: crowbar-bounces On Behalf Of Adam Spiers
Sent: Friday, January 03, 2014 12:12 PM
To: crowbar
Subject: [Crowbar] packaging barclamps in gems
As requested by Victor, I'd like to start a conversation about the idea of
packaging barclamps as gems (which are then optionally automatically wrapped in
.rpms / .debs). This was first proposed a
[Uncle] +1
*long* time ago by James Tan, and has occasionally surfaced in conversations
since then, but other than a few concerns being raised which are likely
surmountable, no progress has been made yet.
[Uncle] How can we move this forwards? What is the next step and who should
drive it? How can I help?
As I see it, re-using the gem mechanism and eco-system could be a big win for
the reasons I already stated in the other thread. Here's a recap with
additions (read to the end for more details about how it could work):
- Any barclamp repo could easily be unit-tested standalone in Travis
simply by ensuring its Gemfile expressed the right dependencies
(e.g. on the core framework and barclamp gems), rather than the
awful https://github.com/crowbar/travis-ci-crowbar monolithic tree
flattening hack we were forced to build previously, which was
never able to automatically test pull requests.
[Uncle] Agreed.
- Dependencies could be expressed using version numbers adhering to
semantic versioning, which gives much more flexibility during
testing than a monolithic approach which insists git cloning an
exact set of commits/branches/tags from a set of remotes chosen
in an opinionated fashion using git submodule / subtree or ./dev.
[Uncle] +2
- Doesn't reinvent any wheels.
[Uncle] +2
- Proven technology already adopted by the whole Ruby community.
[Uncle] +2
- Everyone already knows bundler / Gemfile etc. very well => very
shallow learning curve => more welcoming for newbies.
[Uncle] +1
- No need for a git "meta" tool which clones lots of repositories;
this could be handled by a single Gemfile which points at the
relevant (repository, branch) tuples, plus "bundle config" for
any desired local overrides.
[Uncle] Makes sense. +1
- No need for custom code which builds barclamp "packages" as
tar balls.
[Uncle] Agreed - we should package in a format that achieves greater platform
control, consistency, and reliability for updates/upgrades.
IIRC, the main objections are that barclamps contain more meta-data and content
than gems typically package:
- meta-data such as crowbar.yml, which includes:
- dependencies on debs / rpms
- dependencies on binary packages
- Chef cookbooks, roles, data bags etc.
- scripts in bin/, updates/ etc.
[Uncle] We are working on cleaning this up so we can avoid unnecessary baggage.
However I think these can all be handled pretty easily. Firstly there is no
restriction on what files you can put in a gem. Secondly, a gem can be wrapped
inside an rpm or deb which correctly reflects any of the dependencies defined
by the barclamp metadata. Tools such as gem2rpm (and possibly gem2deb) already
do a lot of the heavy lifting for this - it's how we're able to maintain over
1000 gems as rpms for our various SUSE distributions:
https://build.opensuse.org/project/show/devel:languages:ruby:extensions
[Uncle] I am on the same page here. Sounds good.
.travis.yml would also require the correct package dependencies, and if
necessary a simple script could be hacked to auto-generate it.
Ensuring that all files get copied / moved / symlinked to the right places
(e.g. /opt/dell/chef) can still be done automatically at package build-time by
barclamp_install.rb - this is how we already do it for SUSE Cloud. (Build-time
is better than install-time because then you don't get a bunch of files in
/opt/dell untracked by rpm / deb, and checksumming is important for
supportability.)
[Uncle] +1, where can we see examples of this? Any patterns to work from?
Ensuring the cookbooks / roles / data bags get uploaded to Chef can also be
done automatically at package install-time by barclamp_install.rb.
And finally, ensuring that barclamp_install.rb can still be run standalone
preserves the ability to develop directly from git without requiring packages
to be built every time you want to test a change.
_______________________________________________
Crowbar mailing list
[email protected]
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/
_______________________________________________
Crowbar mailing list
[email protected]
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/
