Re: [PATCHES] import: pypi: Move generally useful procedures to utils module.
Thompson, David dthomps...@worcester.edu skribis: From d370086d5e1c5878110d0cc4cd1884f601545acb Mon Sep 17 00:00:00 2001 From: David Thompson dthomps...@worcester.edu Date: Sun, 16 Aug 2015 21:15:45 -0400 Subject: [PATCH] import: Add Ruby gem importer. * gnu/scripts/import.scm (importers): Add gem. * gnu/import/gem.scm: New file. * gnu/scripts/import/gem.scm: New file. * Makefile.am (MODULES): Add them. Looks like this fell through the cracks. It looks good to me. Could you document it in guix.texi as well? OK to push with doc! Thanks, Ludo’.
Re: [PATCHES] import: pypi: Move generally useful procedures to utils module.
On Mon, Aug 17, 2015 at 7:23 PM, Thompson, David dthomps...@worcester.edu wrote: On Sun, Aug 16, 2015 at 10:01 PM, David Thompson dthomps...@worcester.edu wrote: Building off of the new and improved Ruby build system, here are two patches that add a handy RubyGems importer. In my usual fashion, I've found issues with a patch after I posted it! And I replied to the *wrong* email. Sorry for the noise everyone... - Dave
Re: [PATCHES] import: pypi: Move generally useful procedures to utils module.
On Sun, Aug 16, 2015 at 10:01 PM, David Thompson dthomps...@worcester.edu wrote: Building off of the new and improved Ruby build system, here are two patches that add a handy RubyGems importer. In my usual fashion, I've found issues with a patch after I posted it! Here's a new patch with a bit of an overhauled Ruby build system. I found that in order to reliably get test suites to run (for gems that include them) I had to replace the upstream gemspec with an auto-generated one that strips out developer-only things like needing git to generate the list of files to include in the gem. Without further ado, the patch. TIA for review. - Dave From 17b7307b11dd5825d7ff881624c4b823056bed50 Mon Sep 17 00:00:00 2001 From: David Thompson dthomps...@worcester.edu Date: Sun, 26 Jul 2015 22:01:54 -0400 Subject: [PATCH] build: ruby: Rewrite build system to use gem archives. Co-Authored-By: Pjotr Prins pjotr.publi...@thebird.nl * guix/build-system/ruby.scm (lower): Remove git dependency. (rubygems-uri): New procedure. * guix/build/ruby-build-system (gitify): Delete. (unpack): Use 'gem unpack' utility. (check): Add docstring. (build): Repack modified gem. (install): Rebuild unpacked gem and install it. (%standard-phases): Remove gitify and build phases. * gnu/packages/ruby.scm (ruby-hoe, ruby-rake-compiler, ruby-i18n, ruby-rspec-support, ruby-rspec-core, ruby-diff-lcs-for-rspec, ruby-rspec-expectations, ruby-rspec-mocks, ruby-rspec, bundler, ruby-useragent, ruby-bacon, ruby-arel, ruby-connection-pool, ruby-net-http-persistent, ruby-minitest, ruby-minitest-sprint, ruby-minitest-bacon, ruby-daemons, ruby-git, ruby-slop, ruby-multipart-post): Convert to new build system. --- gnu/packages/ruby.scm| 222 +++ guix/build-system/ruby.scm | 40 +++ guix/build/ruby-build-system.scm | 72 + 3 files changed, 138 insertions(+), 196 deletions(-) diff --git a/gnu/packages/ruby.scm b/gnu/packages/ruby.scm index a481365..196094c 100644 --- a/gnu/packages/ruby.scm +++ b/gnu/packages/ruby.scm @@ -154,13 +154,11 @@ a focus on simplicity and productivity.) (name ruby-hoe) (version 3.13.1) (source (origin - (method git-fetch) - (uri (git-reference -(url https://github.com/seattlerb/hoe.git;) -(commit 0c11836))) ; no release tags :( + (method url-fetch) + (uri (rubygems-uri hoe version)) (sha256 (base32 -0i8dimf8kxcjgqj9x65bbi3l6hc9p9gbfbb1vmrz42764a4jjbz9))) ) +1mac13krdrasn9819dd65xj27kklfy0xdbj3p6s2ij4vlcb46h8q))) ) (build-system ruby-build-system) (synopsis Ruby project management helper) (description @@ -178,22 +176,13 @@ announcement.) (version 0.9.5) (source (origin (method url-fetch) - (uri (string-append -https://github.com/rake-compiler/rake-compiler/archive/v; -version .tar.gz)) - (file-name (string-append name - version .tar.gz)) + (uri (rubygems-uri rake-compiler version)) (sha256 (base32 -07lk1vl0jqcaqwjjhmg0qshqwcxdyr5kscc9xxm13m03835xgpf3 +1k8im2vzj849xdgjk6wafspkiwwapqwm738majchb4dnhnsk64cx (build-system ruby-build-system) (arguments - '(#:tests? #f ; needs cucumber - #:phases (modify-phases %standard-phases - (add-before 'build 'remove-cucumber-rake-task -(lambda _ - ;; Remove cucumber test file because the - ;; dependencies are not available right now. - (delete-file tasks/cucumber.rake)) + '(#:tests? #f)) ; needs cucumber (synopsis Building and packaging helper for Ruby native extensions) (description Rake-compiler provides a framework for building and packaging native C and Java extensions in Ruby.) @@ -206,12 +195,10 @@ packaging native C and Java extensions in Ruby.) (version 0.6.11) (source (origin (method url-fetch) - (uri (string-append https://github.com/svenfuchs/i18n/archive/v; - version .tar.gz)) - (file-name (string-append name - version .tar.gz)) + (uri (rubygems-uri i18n version)) (sha256 (base32 -1fdhnhh1p5g8vibv44d770z8nq208zrms3m2nswdvr54072y1m6k +0fwjlgmgry2blf8zlxn9c555cf4a16p287l599kz5104ncjxlzdk (build-system ruby-build-system) (arguments '(#:tests? #f)) ; requires bundler @@ -234,13 +221,10 @@ an extensible architecture with a swappable backend.) (version 3.2.2) (source (origin (method url-fetch) - (uri (string-append -https://github.com/rspec/rspec-support/archive/v; -
Re: [PATCHES] import: pypi: Move generally useful procedures to utils module.
On Sun, Aug 16, 2015 at 10:01 PM, David Thompson dthomps...@worcester.edu wrote: Building off of the new and improved Ruby build system, here are two patches that add a handy RubyGems importer. Here's an updated patch that uses the hex encoded SHAs on rubygems.org rather than downloading the source gem and computing the hash locally. - Dave From d370086d5e1c5878110d0cc4cd1884f601545acb Mon Sep 17 00:00:00 2001 From: David Thompson dthomps...@worcester.edu Date: Sun, 16 Aug 2015 21:15:45 -0400 Subject: [PATCH] import: Add Ruby gem importer. * gnu/scripts/import.scm (importers): Add gem. * gnu/import/gem.scm: New file. * gnu/scripts/import/gem.scm: New file. * Makefile.am (MODULES): Add them. --- Makefile.am | 3 +- guix/import/gem.scm | 131 guix/scripts/import.scm | 2 +- guix/scripts/import/gem.scm | 91 ++ 4 files changed, 225 insertions(+), 2 deletions(-) create mode 100644 guix/import/gem.scm create mode 100644 guix/scripts/import/gem.scm diff --git a/Makefile.am b/Makefile.am index ada4cbe..f9203ec 100644 --- a/Makefile.am +++ b/Makefile.am @@ -213,7 +213,8 @@ MODULES += \ guix/import/pypi.scm\ guix/scripts/import/pypi.scm \ guix/import/cpan.scm\ - guix/scripts/import/cpan.scm + guix/scripts/import/gem.scm \ + guix/import/gem.scm SCM_TESTS += \ tests/pypi.scm\ diff --git a/guix/import/gem.scm b/guix/import/gem.scm new file mode 100644 index 000..3c28d1d --- /dev/null +++ b/guix/import/gem.scm @@ -0,0 +1,131 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2015 David Thompson da...@gnu.org +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see http://www.gnu.org/licenses/. + +(define-module (guix import gem) + #:use-module (ice-9 match) + #:use-module (ice-9 pretty-print) + #:use-module (rnrs bytevectors) + #:use-module (json) + #:use-module (web uri) + #:use-module (guix import utils) + #:use-module (guix import json) + #:use-module (guix packages) + #:use-module (guix licenses) + #:use-module (guix base32) + #:export (gem-guix-package)) + +(define (rubygems-fetch name) + Return an alist representation of the RubyGems metadata for the package NAME, +or #f on failure. + (json-fetch + (string-append https://rubygems.org/api/v1/gems/; name .json))) + +(define (ruby-package-name name) + Given the NAME of a package on RubyGems, return a Guix-compliant name for +the package. + (if (string-prefix? ruby- name) + (snake-case name) + (string-append ruby- (snake-case name + +(define (hex-string-bytevector str) + Convert the hexadecimal encoded string STR to a bytevector. + (define hex-char-int +(match-lambda + (#\0 0) + (#\1 1) + (#\2 2) + (#\3 3) + (#\4 4) + (#\5 5) + (#\6 6) + (#\7 7) + (#\8 8) + (#\9 9) + (#\a 10) + (#\b 11) + (#\c 12) + (#\d 13) + (#\e 14) + (#\f 15))) + + (define (read-byte i) +(let ((j (* 2 i))) + (+ (hex-char-int (string-ref str (1+ j))) + (* (hex-char-int (string-ref str j)) 16 + + (let* ((len (/ (string-length str) 2)) + (bv (make-bytevector len))) +(let loop ((i 0)) + (if (= i len) + bv + (begin +(bytevector-u8-set! bv i (read-byte i)) +(loop (1+ i))) + +(define (make-gem-sexp name version hash home-page description + dependencies licenses) + Return the `package' s-expression for a Ruby package with the given NAME, +VERSION, HASH, HOME-PAGE, DESCRIPTION, DEPENDENCIES, and LICENSES. + `(package + (name ,(ruby-package-name name)) + (version ,version) + (source (origin + (method url-fetch) + (uri (rubygems-uri ,name version)) + (sha256 +(base32 + ,(bytevector-nix-base32-string + (hex-string-bytevector hash)) + (build-system ruby-build-system) + ,@(if (null? dependencies) + '() + `((propagated-inputs + (,'quasiquote + ,(map (lambda (name) + `(,name + (,'unquote + ,(string-symbol name + dependencies) + (synopsis ,description);
[PATCHES] import: pypi: Move generally useful procedures to utils module.
Building off of the new and improved Ruby build system, here are two patches that add a handy RubyGems importer. From b60252c0c52f8ab4b096c00238acab9798ee64f4 Mon Sep 17 00:00:00 2001 From: David Thompson dthomps...@worcester.edu Date: Sun, 16 Aug 2015 21:09:19 -0400 Subject: [PATCH 1/2] import: pypi: Move generally useful procedures to utils module. * guix/import/pypi.scm (make-pypi-sexp): Factorize license to symbol conversion code. (string-license, snake-case, guix-hash-url): Move from here... * guix/import/utils.scm: ... to here. (license-symbol): New procedure. --- guix/import/pypi.scm | 29 + guix/import/utils.scm | 43 ++- 2 files changed, 43 insertions(+), 29 deletions(-) diff --git a/guix/import/pypi.scm b/guix/import/pypi.scm index 10d5bad..06d21fe 100644 --- a/guix/import/pypi.scm +++ b/guix/import/pypi.scm @@ -31,8 +31,6 @@ #:use-module (guix utils) #:use-module (guix import utils) #:use-module (guix import json) - #:use-module (guix base32) - #:use-module (guix hash) #:use-module (guix packages) #:use-module (guix licenses) #:use-module (guix build-system python) @@ -49,16 +47,6 @@ DELIMETER. ((elem . rest) (cons* elem delimiter (join rest delimiter) -(define string-license - (match-lambda - (GNU LGPL lgpl2.0) - (GPL gpl3) - ((or BSD BSD License) bsd-3) - ((or MIT MIT license Expat license) expat) - (Public domain public-domain) - (Apache License, Version 2.0 asl2.0) - (_ #f))) - (define (pypi-fetch name) Return an alist representation of the PyPI metadata for the package NAME, or #f on failure. @@ -75,15 +63,6 @@ or #f on failure. (assoc-ref* pypi-package info name) (assoc-ref* pypi-package info version) -(define (snake-case str) - Return a downcased version of the string STR where underscores are replaced -with dashes. - (string-join (string-split (string-downcase str) #\_) -)) - -(define (guix-hash-url filename) - Return the hash of FILENAME in nix-base32 format. - (bytevector-nix-base32-string (file-sha256 filename))) - (define (python-package-name name) Given the NAME of a package on PyPI, return a Guix-compliant name for the package. @@ -205,13 +184,7 @@ VERSION, SOURCE-URL, HOME-PAGE, SYNOPSIS, DESCRIPTION, and LICENSE. (home-page ,home-page) (synopsis ,synopsis) (description ,description) - (license ,(assoc-ref `((,lgpl2.0 . lgpl2.0) -(,gpl3 . gpl3) -(,bsd-3 . bsd-3) -(,expat . expat) -(,public-domain . public-domain) -(,asl2.0 . asl2.0)) - license))) + (license ,(license-symbol license))) (define (pypi-guix-package package-name) Fetch the metadata for PACKAGE-NAME from pypi.python.org, and return the diff --git a/guix/import/utils.scm b/guix/import/utils.scm index 969491d..0734fa1 100644 --- a/guix/import/utils.scm +++ b/guix/import/utils.scm @@ -21,6 +21,8 @@ #:use-module (ice-9 regex) #:use-module (srfi srfi-1) #:use-module (guix hash) + #:use-module (guix base32) + #:use-module (guix licenses) #:use-module (guix utils) #:use-module ((guix build download) #:prefix build:) #:export (factorize-uri @@ -29,7 +31,13 @@ flatten assoc-ref* -url-fetch)) +url-fetch +guix-hash-url + +string-license +license-symbol + +snake-case)) (define (factorize-uri uri version) Factorize URI, a package tarball URI as a string, such that any occurrences @@ -95,3 +103,36 @@ recursively apply the procedure to the sub-list. Save the contents of URL to FILE-NAME. Return #f on failure. (parameterize ((current-output-port (current-error-port))) (build:url-fetch url file-name))) + +(define (guix-hash-url filename) + Return the hash of FILENAME in nix-base32 format. + (bytevector-nix-base32-string (file-sha256 filename))) + +(define (string-license str) + Convert the string STR into a license object. + (match str +(GNU LGPL lgpl2.0) +(GPL gpl3) +((or BSD BSD License) bsd-3) +((or MIT MIT license Expat license) expat) +(Public domain public-domain) +((or Apache License, Version 2.0 Apache 2.0) asl2.0) +(_ #f))) + +(define (license-symbol license) + Convert license to a symbol representing the variable the object is bound +to in the (guix licenses) module, or #f if there is no such known license. + ;; TODO: Traverse list public variables in (guix licenses) instead so we + ;; don't have to maintain a list manualy. + (assoc-ref `((,lgpl2.0 . lgpl2.0) + (,gpl3 . gpl3) + (,bsd-3 . bsd-3) + (,expat . expat) + (,public-domain
