Hi,
I'm maintaining gem packages in Gentoo Linux and having some concerns
about rubygems, including source patching, extensions clean install
and gem cache.
The first issue is source patching. It is needed when I want to fix
some minor issues without waiting for upstream maintainers to release
a new gem package. Currently there is no way to do this, IMO, because
gem installation is a one-step action. If Gem::Installer#install is
splitted into smaller functions, like unpack, compile and install,
then distros like Gentoo could reuse those functions to do patching
when needed.
The second issue is not really important though. I notice that gem
does compile extensions in install_dir. This leaves temporary files in
install_dir such as .o files and others. I'd rather install clean
package in my system. This could be done if extensions are compiled in
a temporary directory and then installed into install_dir.
The last one is a feature request. Correct me if I'm wrong. It seems
*.gem files in cache directory are not essential. Gentoo already cache
downloaded files in /usr/portage/distfiles so keeping another versions
is not neccesary. Therefore I'd request an option in "gem install" to
not cache gem packages.
For the first and second issue, I can solve it if Installer#install is
splitted into several parts. I cooked up a patch for this. The patch
reduces Installer#install and call three functions: unpack, compile
and real_install to do its task. @directory is removed because only
generate_bin_symlink uses it. This patch should not change "gem
install" behaviour. However because I change generate_bin and
build_extensions call order, this might have side effects.
The patch also contains several changes in gem_commands.rb and
cmd_manager.rb to:
- add "gem compile" to export Installer#compile
- add option --all to "gem unpack" to be able to unpack any gem
package, --target-dir to unpack to specified directory
- add option --partial to "gem install" to export Installer#real_install
I'm looking for other solutions as well. Comments are highly appreciated.
--
Duy
diff -ur rubygems-0.9.0/lib/rubygems/cmd_manager.rb rubygems-0.9.0.new/lib/rubygems/cmd_manager.rb
--- rubygems-0.9.0/lib/rubygems/cmd_manager.rb 2006-06-07 10:39:53.000000000 +0700
+++ rubygems-0.9.0.new/lib/rubygems/cmd_manager.rb 2006-09-06 06:32:35.000000000 +0700
@@ -51,6 +51,7 @@
@commands = {}
register_command HelpCommand.new
register_command InstallCommand.new
+ register_command CompileCommand.new
register_command UninstallCommand.new
register_command CheckCommand.new
register_command BuildCommand.new
diff -ur rubygems-0.9.0/lib/rubygems/gem_commands.rb rubygems-0.9.0.new/lib/rubygems/gem_commands.rb
--- rubygems-0.9.0/lib/rubygems/gem_commands.rb 2006-06-22 09:44:51.000000000 +0700
+++ rubygems-0.9.0.new/lib/rubygems/gem_commands.rb 2006-09-06 07:11:40.000000000 +0700
@@ -191,10 +191,16 @@
:version => "> 0",
:install_dir => Gem.dir,
:security_policy => nil,
+ :partial => false,
})
add_version_option('install')
add_local_remote_options
add_install_update_options
+ add_option('--partial',
+ 'Do partial install'
+ ) do |value, options|
+ options[:partial] = value
+ end
end
def usage
@@ -232,9 +238,12 @@
alert_error "Local gem file not found: #{filepattern}"
end
else
- result = Gem::Installer.new(entries.last, options).install(
- options[:force],
- options[:install_dir])
+ installer = Gem::Installer.new(entries.last, options)
+ if options[:partial] == true
+ result = installer.real_install(options[:install_dir])
+ else
+ result = installer.install(options[:force], options[:install_dir])
+ end
installed_gems = [result].flatten
say "Successfully installed #{installed_gems[0].name}, " +
"version #{installed_gems[0].version}" if installed_gems
@@ -305,6 +314,85 @@
end
end
+
+ class CompileCommand < Command
+ include CommandAids
+
+ def initialize
+ require 'fileutils'
+ super(
+ 'compile',
+ 'Compile extensions',
+ {
+ :install_dir => Gem.dir,
+ :compile_dir => FileUtils.pwd,
+ })
+ add_option('--install-dir DIR',
+ 'Gem repository directory to get installed gems.') do
+ |value, options|
+ options[:install_dir] = File.expand_path(value)
+ end
+ add_option('--compile-dir DIR',
+ 'Gem repository directory to get compiled gems.') do
+ |value, options|
+ options[:compile_dir] = File.expand_path(value)
+ end
+ end
+
+ def usage
+ "#{program_name} GEMNAME [options]
+ or: #{program_name} GEMNAME [options] -- --build-flags"
+ end
+
+ def arguments
+ "GEMNAME name of gem to install"
+ end
+
+ def defaults_str
+ "--install-dir #{Gem.dir} --compile-dir <current-directory>"
+ end
+
+ def execute
+ ENV['GEM_PATH'] = options[:install_dir]
+ if(options[:args].empty?)
+ fail Gem::CommandLineError,
+ "Please specify a gem name on the command line (e.g. gem build GEMNAME)"
+ end
+ if(options[:args].length > 1)
+ fail Gem::CommandLineError,
+ "Please specify only one gem name on the command line (e.g. gem build GEMNAME)"
+ end
+ gem_name = options[:args][0]
+ begin
+ entries = []
+ if(File.exist?(gem_name) && !File.directory?(gem_name))
+ entries << gem_name
+ else
+ filepattern = gem_name + "*.gem"
+ entries = Dir[filepattern]
+ end
+ unless entries.size > 0
+ alert_error "Gem file not found: #{filepattern}"
+ else
+ installer = Gem::Installer.new(entries.last, options)
+ results = installer.compile(options[:compile_dir], options[:install_dir])
+ end
+ rescue LocalInstallationError => e
+ say " -> Local installation can't proceed: #{e.message}"
+ rescue Gem::LoadError => e
+ say " -> Local installation can't proceed due to LoadError: #{e.message}"
+ rescue => e
+ # TODO: Fix this handle to allow the error to propagate to
+ # the top level handler. Examine the other errors as
+ # well. This implementation here looks suspicious to me --
+ # JimWeirich (4/Jan/05)
+ alert_error "Error compiling gem #{gem_name}[.gem]: #{e.message}"
+ return
+ end
+ end
+
+ end
+
####################################################################
class UninstallCommand < Command
@@ -1166,9 +1254,23 @@
super(
'unpack',
'Unpack an installed gem to the current directory',
- { :version => '> 0' }
+ {
+ :version => '> 0',
+ :all => false,
+ :target_dir => ''
+ }
)
add_version_option('unpack')
+ add_option('-a', '--all',
+ 'Select gems from local gems instead of installed gems'
+ ) do |value, options|
+ options[:all] = value
+ end
+ add_option('--target-dir DIR',
+ 'Install to specified dir instead of current directory') do
+ |value, options|
+ options[:target_dir] = File.expand_path(value)
+ end
end
def defaults_str
@@ -1187,11 +1289,29 @@
# solution for this, so that it works for uninstall as well. (And
# check other commands at the same time.)
def execute
- gemname = get_one_gem_name
- path = get_path(gemname, options[:version])
+ gem_name = get_one_gem_name
+ if options[:all] == true
+ ENV['GEM_PATH'] = options[:install_dir]
+ entries = []
+ if(File.exist?(gem_name) && !File.directory?(gem_name))
+ entries << gem_name
+ else
+ filepattern = gem_name + "*.gem"
+ entries = Dir[filepattern]
+ end
+ unless entries.size > 0
+ alert_error "Local gem file not found: #{filepattern}"
+ terminate_interaction(1)
+ else
+ path = entries.last
+ end
+ else
+ path = get_path(gem_name, options[:version])
+ end
if path
require 'fileutils'
target_dir = File.basename(path).sub(/\.gem$/, '')
+ target_dir = File.join(options[:target_dir], target_dir) unless options[:target_dir].empty?
FileUtils.mkdir_p target_dir
Installer.new(path).unpack(target_dir)
say "Unpacked gem: '#{target_dir}'"
diff -ur rubygems-0.9.0/lib/rubygems/installer.rb rubygems-0.9.0.new/lib/rubygems/installer.rb
--- rubygems-0.9.0/lib/rubygems/installer.rb 2006-06-07 10:39:54.000000000 +0700
+++ rubygems-0.9.0.new/lib/rubygems/installer.rb 2006-09-06 07:19:43.000000000 +0700
@@ -51,7 +51,6 @@
# return:: [Gem::Specification] The specification for the newly installed Gem.
#
def install(force=false, install_dir=Gem.dir, ignore_this_parameter=false)
- require 'fileutils'
# if we're forcing the install, then disable security, _unless_
# the security policy says that we only install singed gems
@@ -78,26 +77,66 @@
raise Gem::FilePermissionError.new(install_dir) unless File.writable?(install_dir)
# Build spec dir.
- @directory = File.join(install_dir, "gems", format.spec.full_name).untaint
- FileUtils.mkdir_p @directory
+ directory = File.join(install_dir, "gems", format.spec.full_name).untaint
+ FileUtils.mkdir_p directory
+
+ unpack(directory, format)
+ compile(directory,install_dir, format.spec)
+ real_install(install_dir, format.spec)
+ end
+
+ def compile(compile_dir=Gem.dir, install_dir=Gem.dir, spec = nil)
+ require 'fileutils'
+
+ unless spec
+ # if we're forcing the install, then disable security, _unless_
+ # the security policy says that we only install singed gems
+ # (this includes Gem::Security::HighSecurity)
+ security_policy = @options[:security_policy]
+ security_policy = nil if security_policy && security_policy.only_signed != true
+
+ format = Gem::Format.from_file_by_path(@gem, security_policy)
+ spec = format.spec
+ end
+
+ directory = File.join(install_dir, "gems", spec.full_name).untaint
+ FileUtils.mkdir_p directory
+
+ #generate_bin(spec, directory, compile_dir)
+ build_extensions(compile_dir, spec, directory)
+ end
+
+ def real_install(install_dir=Gem.dir, spec = nil)
+ require 'fileutils'
+
+ unless spec
+ # if we're forcing the install, then disable security, _unless_
+ # the security policy says that we only install singed gems
+ # (this includes Gem::Security::HighSecurity)
+ security_policy = @options[:security_policy]
+ security_policy = nil if security_policy && security_policy.only_signed != true
+
+ format = Gem::Format.from_file_by_path(@gem, security_policy)
+ spec = format.spec
+ end
+
+ directory = File.join(install_dir, "gems", spec.full_name).untaint
- extract_files(@directory, format)
- generate_bin(format.spec, install_dir)
- build_extensions(@directory, format.spec)
+ generate_bin(spec, directory, install_dir)
# Build spec/cache/doc dir.
build_support_directories(install_dir)
# Write the spec and cache files.
- write_spec(format.spec, File.join(install_dir, "specifications"))
+ write_spec(spec, File.join(install_dir, "specifications"))
unless File.exist? File.join(install_dir, "cache", @gem.split(/\//).pop)
FileUtils.cp @gem, File.join(install_dir, "cache")
end
- puts format.spec.post_install_message unless format.spec.post_install_message.nil?
+ puts spec.post_install_message unless spec.post_install_message.nil?
- format.spec.loaded_from = File.join(install_dir, 'specifications', format.spec.full_name+".gemspec")
- return format.spec
+ spec.loaded_from = File.join(install_dir, 'specifications', spec.full_name+".gemspec")
+ return spec
end
##
@@ -124,8 +163,8 @@
##
# Unpacks the gem into the given directory.
#
- def unpack(directory)
- format = Gem::Format.from_file_by_path(@gem, @options[:security_policy])
+ def unpack(directory, format = nil)
+ format = Gem::Format.from_file_by_path(@gem, @options[:security_policy]) unless format
extract_files(directory, format)
end
@@ -187,7 +226,7 @@
end
end
- def generate_bin(spec, install_dir=Gem.dir)
+ def generate_bin(spec, source_dir, install_dir=Gem.dir)
return unless spec.executables && ! spec.executables.empty?
# If the user has asked for the gem to be installed in
@@ -203,7 +242,7 @@
if @options[:wrappers] then
generate_bin_script spec, filename, bindir, install_dir
else
- generate_bin_symlink spec, filename, bindir, install_dir
+ generate_bin_symlink spec, filename, bindir, install_dir, source_dir
end
end
end
@@ -222,14 +261,14 @@
# Creates the symlinks to run the applications in the gem. Moves
# the symlink if the gem being installed has a newer version.
#
- def generate_bin_symlink(spec, filename, bindir, install_dir)
+ def generate_bin_symlink(spec, filename, bindir, install_dir, source_dir)
if Config::CONFIG["arch"] =~ /dos|win32/i then
warn "Unable to use symlinks on win32, installing wrapper" unless $TESTING # HACK
generate_bin_script spec, filename, bindir, install_dir
return
end
- src = File.join @directory, 'bin', filename
+ src = File.join source_dir, 'bin', filename
dst = File.join bindir, File.basename(filename)
if File.exist? dst then
@@ -287,11 +326,11 @@
text
end
- def build_extensions(directory, spec)
+ def build_extensions(directory, spec, install_dir)
return unless spec.extensions.size > 0
say "Building native extensions. This could take a while..."
start_dir = Dir.pwd
- dest_path = File.join(directory, spec.require_paths[0])
+ dest_path = File.join(install_dir, spec.require_paths[0])
results = []
spec.extensions.each do |extension|
_______________________________________________
Rubygems-developers mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/rubygems-developers
