Rob, I read all the postings on that link and it appears to me to actually support RPM/Deb packaging over GEM-only based distribution of Rails code. Have I mis-read this?
- John T. From: Hirschfeld, Rob Sent: Friday, January 03, 2014 10:50 PM To: Terpstra, John; [email protected]; crowbar Subject: RE: [Crowbar] packaging barclamps in gems John, My sense on this is that gems are more for Rails extensions (parts of apps) than a full application. While OpenCrowbar is mostly Rails, it is NOT only Rails. There's quite a bit of extra stuff (chef, erlang, scripts, docs, etc) that we package at the same level as the Rails app. I found this link that supports this: https://groups.google.com/forum/#!topic/puppet-users/aVTPAfcVBcE FWIW - I do not think we can/need to resolve all issues raised. We're discussing options for implementation and we'll have to compromise. Rob From: crowbar-bounces On Behalf Of Terpstra, John Sent: Friday, January 03, 2014 1:29 PM To: [email protected]<mailto:[email protected]>; crowbar Subject: Re: [Crowbar] packaging barclamps in gems Adam, The things you have brought up here are most timely. As we begin to address the way that Crowbar2 will be packaged to facilitate the creation and deployment of admin nodes and workloads we need to adopt a different strategy from the Crowbar1 ISO build approach. The Crowbar project should not concern itself with the need to build a re-mastered operating system ISO. I believe the preferred approach is to build a set of OS platform packages (RPM or Deb - as needed for the target admin node platform). My thinking on this is to build platform-independent, LSB compliant, packages for all core Crowbar2 components. There will be a need for one platform-dependent package that will contain the Crowbar2 repo info as well as repos for all its dependencies, plus it will contain the pattern info for the Crowbar2 infrastructure plus its dependencies, as well as the initialization scripts that will deliver a working admin node and then launch a browser attached to http://localhost:3000 so that the administrator is ready to move to the next guided step. Packaging the barclamps as gems that are converted into wrappered RPMS makes sense to me. Please see my in-line comments below. - John T. -----Original Message----- From: crowbar-bounces On Behalf Of Adam Spiers Sent: Friday, January 03, 2014 12:12 PM To: crowbar Subject: [Crowbar] packaging barclamps in gems As requested by Victor, I'd like to start a conversation about the idea of packaging barclamps as gems (which are then optionally automatically wrapped in .rpms / .debs). This was first proposed a [Uncle] +1 *long* time ago by James Tan, and has occasionally surfaced in conversations since then, but other than a few concerns being raised which are likely surmountable, no progress has been made yet. [Uncle] How can we move this forwards? What is the next step and who should drive it? How can I help? As I see it, re-using the gem mechanism and eco-system could be a big win for the reasons I already stated in the other thread. Here's a recap with additions (read to the end for more details about how it could work): - Any barclamp repo could easily be unit-tested standalone in Travis simply by ensuring its Gemfile expressed the right dependencies (e.g. on the core framework and barclamp gems), rather than the awful https://github.com/crowbar/travis-ci-crowbar monolithic tree flattening hack we were forced to build previously, which was never able to automatically test pull requests. [Uncle] Agreed. - Dependencies could be expressed using version numbers adhering to semantic versioning, which gives much more flexibility during testing than a monolithic approach which insists git cloning an exact set of commits/branches/tags from a set of remotes chosen in an opinionated fashion using git submodule / subtree or ./dev. [Uncle] +2 - Doesn't reinvent any wheels. [Uncle] +2 - Proven technology already adopted by the whole Ruby community. [Uncle] +2 - Everyone already knows bundler / Gemfile etc. very well => very shallow learning curve => more welcoming for newbies. [Uncle] +1 - No need for a git "meta" tool which clones lots of repositories; this could be handled by a single Gemfile which points at the relevant (repository, branch) tuples, plus "bundle config" for any desired local overrides. [Uncle] Makes sense. +1 - No need for custom code which builds barclamp "packages" as tar balls. [Uncle] Agreed - we should package in a format that achieves greater platform control, consistency, and reliability for updates/upgrades. IIRC, the main objections are that barclamps contain more meta-data and content than gems typically package: - meta-data such as crowbar.yml, which includes: - dependencies on debs / rpms - dependencies on binary packages - Chef cookbooks, roles, data bags etc. - scripts in bin/, updates/ etc. [Uncle] We are working on cleaning this up so we can avoid unnecessary baggage. However I think these can all be handled pretty easily. Firstly there is no restriction on what files you can put in a gem. Secondly, a gem can be wrapped inside an rpm or deb which correctly reflects any of the dependencies defined by the barclamp metadata. Tools such as gem2rpm (and possibly gem2deb) already do a lot of the heavy lifting for this - it's how we're able to maintain over 1000 gems as rpms for our various SUSE distributions: https://build.opensuse.org/project/show/devel:languages:ruby:extensions [Uncle] I am on the same page here. Sounds good. .travis.yml would also require the correct package dependencies, and if necessary a simple script could be hacked to auto-generate it. Ensuring that all files get copied / moved / symlinked to the right places (e.g. /opt/dell/chef) can still be done automatically at package build-time by barclamp_install.rb - this is how we already do it for SUSE Cloud. (Build-time is better than install-time because then you don't get a bunch of files in /opt/dell untracked by rpm / deb, and checksumming is important for supportability.) [Uncle] +1, where can we see examples of this? Any patterns to work from? Ensuring the cookbooks / roles / data bags get uploaded to Chef can also be done automatically at package install-time by barclamp_install.rb. And finally, ensuring that barclamp_install.rb can still be run standalone preserves the ability to develop directly from git without requiring packages to be built every time you want to test a change. _______________________________________________ Crowbar mailing list [email protected]<mailto:[email protected]> https://lists.us.dell.com/mailman/listinfo/crowbar For more information: http://crowbar.github.com/
_______________________________________________ Crowbar mailing list [email protected] https://lists.us.dell.com/mailman/listinfo/crowbar For more information: http://crowbar.github.com/
