Hi everyone,
First. Sorry for this long email :-)
I started working on refactoring of how we handle gems loading in
Deltacloud API. This is what I discover:
1. In 'server', we don't use Bundler. If you run Deltacloud using:
'./bin/deltacloudd'
we use the old-school 'require' all over the DC server code (do a "grep
'require ' -r lib/"). Having this, we always use the version of the gem
which is currently available in the system. In most cases is the latest
version available.
2. For 'server' tests, we do use Bundler (we call the Bundler.require in
the Rakefile). So before you run 'rake test' you need to run 'bundle
install'. This is not true, when the user want to execute just one
single test file (ruby tests/.../test.rb) (yeah, he can use 'bundle ruby
...', but srsly, how often you forget to do it ;-). In that case, we
again trust the rubygems and use the system versions.
I think what we currently have is a bit mess :-) I found that up to 50%
of problems, users have with DC installation is handling all various gem
dependencies and versions (tilt, sinatra, rabbit, openstack, etc...).
Also many bugs/jiras are related to breakages caused by incompatible
changes in upstream gems we use.
So. How we can do things better? :-)
*** Idea ***:
Use bundler everywhere. I don't like bundler. But I think currently it
is the only sane way how to manage gems in DC. We should check in the
Gemfile.lock into 'server' dir, create an initialization file with
Bundler.require(:default) and use this to require all gems we do need
for server to operate. Also might we need to remove all "require
'upstream_gem'" in DC, we we load the gems just once (when starting DC).
*** Problems ***:
- Different drivers require different gems. Having them all loaded by
Bundler will increase DC boot time badly and increase the memory DC consume.
Solution:
We can create a 'group' for each driver in Bundler, like:
group :ec2 do
gem 'aws'
end
group :rhevm do
gem 'rbovirt', :version => '...'
end
The the 'driver' method in Deltacloud will need to be a bit smarter and
call the "Bundler.require(driver_symbol)" when the driver is changed.
- We need to make sure that 'gem install deltacloud-core' work properly.
Ruby gems does not play well with Gemfile.lock.
Solution:
We don't need 'gemspec', right? We do need gemspec **only** when we are
building a gem (release). Can we generate 'gemspec' from what we have in
Gemfile.lock before release? In that case we will make sure all gems we
require have the correct versions (the one we tested on).
-------
I think having all gems managed by Bundler and loaded on one place
(initializer) will make managing gems we use easier, where we can track
what exact version we are currently supporting and we can avoid the
issues similar to 'tilt' or 'sinatra'.
There is a good article about clarification how to use gemspec vs Gemfile:
http://yehudakatz.com/2010/12/16/clarifying-the-roles-of-the-gemspec-and-gemfile/
"When developing a gem, do not check your Gemfile.lock into version control"
vs.
"When developing an app, check in your Gemfile.lock"
In our case, we develop an app, right? The 'gem' we produce is just a
distribution of this app.
What you think? Do you have a better idea?
-- Michal
--
Michal Fojtik <[email protected]>
Deltacloud API, CloudForms
