Re: [PATCH] Update Ruby to 2.3.0

[Ben Woodcroft]

On 11/01/16 07:14, Ludovic Courtès wrote:

Ben Woodcroft  skribis:


 From 2f26295b5a163cfc5d37332a501dcba5c40fb956 Mon Sep 17 00:00:00 2001
From: Ben Woodcroft 
Date: Mon, 4 Jan 2016 09:38:42 +1000
Subject: [PATCH 5/5] gnu: ruby: Update to 2.3.0.

* gnu/packages/ruby.scm (ruby): Update to 2.3.0.
[arguments]: Remove bundled libffi.  Use parallel tests.
(ruby-2.2): New variable.

[...]


+ ;; Remove bundled libffi
+ (delete-file-recursively
+  (string-append "ext/fiddle/libffi-3.2.1"))

I should have mentioned it before, but could you move the removal to a
‘snippet’ in the origin?

All pushed to core-updates with a snippet and minor formatting changes.

 From 015a0e17be804dd7f68f21cde8a878ff353a4a97 Mon Sep 17 00:00:00 2001
From: Ben Woodcroft 
Date: Fri, 8 Jan 2016 17:29:39 +1000
Subject: [PATCH 3/5] ruby: Abstract out path to GEM_HOME.

Previously paths to the GEM_HOME of certain Ruby packages were
hard-coded, so packages failed to build when Ruby was updated to 2.3.0.

* guix/build/ruby-build-system.scm (gem-home): New procedure.
* gnu/packages/ruby.scm (ruby-metaclass, ruby-instantiator,
ruby-introspection, ruby-mocha, ruby-minitest-tu-shim): Use it.

LGTM.

However, ultimately, we should pass the Ruby version as a keyword
parameter on the build side or have a build-side procedure akin to
‘get-python-version’ in python-build-system.scm (I’d prefer the former.)

That way, phases could query the version number of the Ruby that’s
actually used instead of using the version number of whatever variant
the global ‘ruby’ variable refers to.  This would allow users to simply
change the “ruby” input of a package and have it automatically work with
the new package.

Does that make sense?

This can always be done after the ‘core-updates’ merge, no rush.
OK then. I reckon we should provide a procedure to get to the lib/ 
directory as well, based on the upstream name property too. Then we can 
use it for the require test too.


Thanks,
ben

Re: [PATCH] Update Ruby to 2.3.0

[Ben Woodcroft]

On 09/01/16 02:52, Ludovic Courtès wrote:

Ben Woodcroft  skribis:


 From 28dee96b0de99adf8657457660b9e3733434d2b4 Mon Sep 17 00:00:00 2001
From: Ben Woodcroft 
Date: Mon, 4 Jan 2016 09:38:42 +1000
Subject: [PATCH 4/4] gnu: ruby: Update to 2.3.0.

* gnu/packages/ruby.scm (ruby): Update to 2.3.0.

Please mention the switch to ‘modify-phases’ here.

I even made it a separate commit.

[..]

I overlooked this part before but: (1) this looks weird (the spaces
around ‘=’ suggest it’s Makefile syntax, not shell syntax), and (2) we
shouldn’t use the bundled libffi.

How much work would it be to fix #2?  It would be awesome if you could
do that.  Sorry for the extra work!
All good, was easy to fix. I also removed it from ruby-2.2 which bundles 
the same libffi version.


However, it seems 11 rubygems are also bundled. To unbundle these and 
deal with the circular dependencies, I reckon we should create a new 
ruby-base package that removes the bundled gems, then we package the 11 
gems and their dependencies using ruby-base, and the 'ruby' package then 
inherits from ruby-base but also propagates the 11 gems. However I don't 
think I have time to implement this right now, especially since most of 
the 11 haven't been packaged by us yet and core-updates is to be merged 
soon.


So do we merge this update as-is or wait for a more perfect solution?

+(define-public ruby-2.2
+  (package (inherit ruby)
+(version "2.2.4")
+(source
+ (origin
+   (method url-fetch)
+   (uri (string-append "http://cache.ruby-lang.org/pub/ruby/;
+   (version-major+minor version)
+   "/ruby-" version ".tar.bz2"))
+   (sha256
+(base32
+ "0g3ps4q3iz7wj9m45n8xyxzw8nh29ljdqb87b0f6i0p3853gz2yj"
+(arguments
+ `(#:test-target "test"
+   #:parallel-tests? #f
+   #:phases
+   (alist-cons-before
+'configure 'replace-bin-sh
+(lambda _
+  (substitute* '("Makefile.in"
+ "ext/pty/pty.c"
+ "io.c"
+ "lib/mkmf.rb"
+ "process.c"
+ "test/rubygems/test_gem_ext_configure_builder.rb"
+ "test/rdoc/test_rdoc_parser.rb"
+ "test/ruby/test_rubyoptions.rb"
+ "test/ruby/test_process.rb"
+ "test/ruby/test_system.rb"
+ "tool/rbinstall.rb")
+(("/bin/sh") (which "sh"
+%standard-phases)

AFAICS all of ‘arguments’ can be omitted once the above libffi
substitution is removed.

(If it there’s still a need for different phases, it should use
‘substitute-keyword-arguments’ to modify only #:phases, and then
‘modify-phases’ rather than ‘alist-cons-before’.)

Thanks for the tip, but ruby-2.2 is a now simple inherit.


Could you send an updated patch, hopefully the last one?  :-)

Please feel free to suggest further changes if you spot anything further.

Thanks for the review,
ben

Re: [PATCH] Update Ruby to 2.3.0

[Ben Woodcroft]

On 10/01/16 22:53, Ben Woodcroft wrote:



On 09/01/16 02:52, Ludovic Courtès wrote:

Ben Woodcroft  skribis:


 From 28dee96b0de99adf8657457660b9e3733434d2b4 Mon Sep 17 00:00:00 2001
From: Ben Woodcroft 
Date: Mon, 4 Jan 2016 09:38:42 +1000
Subject: [PATCH 4/4] gnu: ruby: Update to 2.3.0.

* gnu/packages/ruby.scm (ruby): Update to 2.3.0.

Please mention the switch to ‘modify-phases’ here.

I even made it a separate commit.

Forgot to attach patches.
>From 2f26295b5a163cfc5d37332a501dcba5c40fb956 Mon Sep 17 00:00:00 2001
From: Ben Woodcroft 
Date: Mon, 4 Jan 2016 09:38:42 +1000
Subject: [PATCH 5/5] gnu: ruby: Update to 2.3.0.

* gnu/packages/ruby.scm (ruby): Update to 2.3.0.
[arguments]: Remove bundled libffi.  Use parallel tests.
(ruby-2.2): New variable.
---
 gnu/packages/ruby.scm | 23 +++
 1 file changed, 19 insertions(+), 4 deletions(-)

diff --git a/gnu/packages/ruby.scm b/gnu/packages/ruby.scm
index 2b7496a..d2cdcf0 100644
--- a/gnu/packages/ruby.scm
+++ b/gnu/packages/ruby.scm
@@ -46,7 +46,7 @@
 (define-public ruby
   (package
 (name "ruby")
-(version "2.2.4")
+(version "2.3.0")
 (source
  (origin
(method url-fetch)
@@ -55,14 +55,13 @@
"/ruby-" version ".tar.xz"))
(sha256
 (base32
- "0g3ps4q3iz7wj9m45n8xyxzw8nh29ljdqb87b0f6i0p3853gz2yj"
+ "15s0dsb5ynf3d2w5gzawnszq5594fqvapv2y7a0qw16przq5l4kh"
 (build-system gnu-build-system)
 (arguments
  `(#:test-target "test"
-   #:parallel-tests? #f
#:phases
(modify-phases %standard-phases
- (add-before 'configure 'replace-bin-sh
+ (add-before 'configure 'replace-bin-sh-and-remove-libffi
(lambda _
  (substitute* '("Makefile.in"
 "ext/pty/pty.c"
@@ -76,6 +75,9 @@
 "test/ruby/test_system.rb"
 "tool/rbinstall.rb")
(("/bin/sh") (which "sh")))
+ ;; Remove bundled libffi
+ (delete-file-recursively
+  (string-append "ext/fiddle/libffi-3.2.1"))
  #t)
 (inputs
  `(("readline" ,readline)
@@ -95,6 +97,19 @@ a focus on simplicity and productivity.")
 (home-page "https://ruby-lang.org;)
 (license license:ruby)))
 
+(define-public ruby-2.2
+  (package (inherit ruby)
+(version "2.2.4")
+(source
+ (origin
+   (method url-fetch)
+   (uri (string-append "http://cache.ruby-lang.org/pub/ruby/;
+   (version-major+minor version)
+   "/ruby-" version ".tar.xz"))
+   (sha256
+(base32
+ "0g3ps4q3iz7wj9m45n8xyxzw8nh29ljdqb87b0f6i0p3853gz2yj"))
+
 (define-public ruby-2.1
   (package (inherit ruby)
 (version "2.1.6")
-- 
2.6.3

>From 551ecbd280eb35cb8e67cedf50e4a93f618cab1e Mon Sep 17 00:00:00 2001
From: Ben Woodcroft 
Date: Sun, 10 Jan 2016 22:25:45 +1000
Subject: [PATCH 4/5] gnu: ruby: Use modify-phases.

* gnu/packages/ruby.scm (ruby)[arguments]: Use modify-phases.
---
 gnu/packages/ruby.scm | 32 
 1 file changed, 16 insertions(+), 16 deletions(-)

diff --git a/gnu/packages/ruby.scm b/gnu/packages/ruby.scm
index 1564a54..2b7496a 100644
--- a/gnu/packages/ruby.scm
+++ b/gnu/packages/ruby.scm
@@ -61,22 +61,22 @@
  `(#:test-target "test"
#:parallel-tests? #f
#:phases
-   (alist-cons-before
-'configure 'replace-bin-sh
-(lambda _
-  (substitute* '("Makefile.in"
- "ext/pty/pty.c"
- "io.c"
- "lib/mkmf.rb"
- "process.c"
- "test/rubygems/test_gem_ext_configure_builder.rb"
- "test/rdoc/test_rdoc_parser.rb"
- "test/ruby/test_rubyoptions.rb"
- "test/ruby/test_process.rb"
- "test/ruby/test_system.rb"
- "tool/rbinstall.rb")
-(("/bin/sh") (which "sh"
-%standard-phases)))
+   (modify-phases %standard-phases
+ (add-before 'configure 'replace-bin-sh
+   (lambda _
+ (substitute* '("Makefile.in"
+"ext/pty/pty.c"
+"io.c"
+"lib/mkmf.rb"
+"process.c"
+"test/rubygems/test_gem_ext_configure_builder.rb"
+"test/rdoc/test_rdoc_parser.rb"
+"test/ruby/test_rubyoptions.rb"
+"test/ruby/test_process.rb"
+"test/ruby/test_system.rb"
+"tool/rbinstall.rb")
+   (("/bin/sh") (which "sh")))
+ #t)
 (inputs
  

Re: [PATCH] Update Ruby to 2.3.0

[Ludovic Courtès]

Ben Woodcroft  skribis:

> From 2f26295b5a163cfc5d37332a501dcba5c40fb956 Mon Sep 17 00:00:00 2001
> From: Ben Woodcroft 
> Date: Mon, 4 Jan 2016 09:38:42 +1000
> Subject: [PATCH 5/5] gnu: ruby: Update to 2.3.0.
>
> * gnu/packages/ruby.scm (ruby): Update to 2.3.0.
> [arguments]: Remove bundled libffi.  Use parallel tests.
> (ruby-2.2): New variable.

[...]

> + ;; Remove bundled libffi
> + (delete-file-recursively
> +  (string-append "ext/fiddle/libffi-3.2.1"))

I should have mentioned it before, but could you move the removal to a
‘snippet’ in the origin?

This is not critical, so if you feel like you won’t have time now, we
can leave that for later.

Otherwise the patch is OK for ‘core-updates’.

> From 551ecbd280eb35cb8e67cedf50e4a93f618cab1e Mon Sep 17 00:00:00 2001
> From: Ben Woodcroft 
> Date: Sun, 10 Jan 2016 22:25:45 +1000
> Subject: [PATCH 4/5] gnu: ruby: Use modify-phases.
>
> * gnu/packages/ruby.scm (ruby)[arguments]: Use modify-phases.

OK.

> From 015a0e17be804dd7f68f21cde8a878ff353a4a97 Mon Sep 17 00:00:00 2001
> From: Ben Woodcroft 
> Date: Fri, 8 Jan 2016 17:29:39 +1000
> Subject: [PATCH 3/5] ruby: Abstract out path to GEM_HOME.
>
> Previously paths to the GEM_HOME of certain Ruby packages were
> hard-coded, so packages failed to build when Ruby was updated to 2.3.0.
>
> * guix/build/ruby-build-system.scm (gem-home): New procedure.
> * gnu/packages/ruby.scm (ruby-metaclass, ruby-instantiator,
> ruby-introspection, ruby-mocha, ruby-minitest-tu-shim): Use it.

LGTM.

However, ultimately, we should pass the Ruby version as a keyword
parameter on the build side or have a build-side procedure akin to
‘get-python-version’ in python-build-system.scm (I’d prefer the former.)

That way, phases could query the version number of the Ruby that’s
actually used instead of using the version number of whatever variant
the global ‘ruby’ variable refers to.  This would allow users to simply
change the “ruby” input of a package and have it automatically work with
the new package.

Does that make sense?

This can always be done after the ‘core-updates’ merge, no rush.

> From 048036aee522d6a03436bf530d139ec26d8a438e Mon Sep 17 00:00:00 2001
> From: Ben Woodcroft 
> Date: Wed, 6 Jan 2016 21:57:44 +1000
> Subject: [PATCH 2/5] gnu: ruby-yard: Disable failing test.
>
> * gnu/packages/ruby.scm (ruby-yard)[arguments]: Disable test which fails on
> Ruby 2.3.0.

OK.

> From 3918146b6179f211fb7ef955f74561f9b1460a8b Mon Sep 17 00:00:00 2001
> From: Ben Woodcroft 
> Date: Wed, 6 Jan 2016 21:23:15 +1000
> Subject: [PATCH 1/5] gnu: ruby-power-assert: Update to 0.2.7.
>
> * gnu/packages/ruby.scm (ruby-power-assert): Update to 0.2.7.

OK.

Thank you!

Ludo’.

Re: [PATCH] Update Ruby to 2.3.0 (was Re: Freezing core-updates soon)

[Ben Woodcroft]

On 06/01/16 08:24, Ludovic Courtès wrote:

Ben Woodcroft  skribis:


I had some trouble downloading dependencies because of inability to
download sources e.g. http://graphviz.org/ seems to be down. Ruby
seems to build though, can we just review as normal and then push to
core-updates?

I did rebuild all the packages in ruby.scm, and out of these about 1/3
of packages failed to build, because at least these three packages'
tests fail
* yard
* power_assert
* minitest_tu_shim

The first two presumably will be fixed upstream soon, but the third
hasn't seen a release since Aug 2013, and the homepage listed on
rubygems is dead. I guess we just have to encourage the only package
we have packaged that uses tu_shim (term-ansicolor) to stop relying on
it.

OK, thanks for testing this thoroughly.  It would be good to fix these 3
packages sometime before core-updates is merged.
In the attached patches I've disabled the failing test in both yard and 
power_assert. There's no fix for yard in upstream. For power_assert 
there is a fix, but unfortunately it seems Ruby packages cannot be 
patched in the usual way because .gem files are archives of archives so 
the unpacking/patch/repacking fails. I guess we fix that in future.


Turns out fixing minitest_tu_shim (and others) was easier than I thought 
as the problem was some hardcoded ruby 2.2 paths in package definitions. 
I created a new procedure to remove the hard-coding so the update to 2.4 
will work too, WDYT?


After all that, everything in gnu/packages/ruby.scm builds.

+   (modify-phases %standard-phases
+ (add-before 'configure 'replace-bin-sh
+   (lambda _
+ (substitute* '("Makefile.in"
+"ext/pty/pty.c"
+"io.c"
+"lib/mkmf.rb"
+"process.c"
+"test/rubygems/test_gem_ext_configure_builder.rb"
+"test/rdoc/test_rdoc_parser.rb"
+"test/ruby/test_rubyoptions.rb"
+"test/ruby/test_process.rb"
+"test/ruby/test_system.rb"
+"tool/rbinstall.rb")
+   (("/bin/sh") (which "sh")))
+ (substitute* "ext/fiddle/libffi-3.2.1/configure"
+   (("SHELL = /bin/sh")
+(string-append "SHELL = " (which "sh"
+ #t)
+(inputs
+ `(("readline" ,readline)
+   ("openssl" ,openssl)
+   ("libffi" ,libffi)
+   ("gdbm" ,gdbm)
+   ("zlib" ,zlib)))
+(native-search-paths
+ (list (search-path-specification
+(variable "GEM_PATH")
+(files (list (string-append "lib/ruby/gems/"
+(version-major+minor version)
+".0"))

This seems to be shared with 2.2.  Is it possible to avoid duplication
using something like:

   (define ruby
 (package
   (inherit ruby-2.2)
   (version "2.3.4")
   ;; …
   ))

?

[..]

Yes, I did this but forgot to delete some/most duplication. Now after 
the final patch ruby-2.2 inherits from ruby (not vice-versa as you 
suggest) as this was the previous convention in ruby.scm. I checked that 
the hashes in the store names for ruby 2.2, 2.1 and 1.8 did not change 
after applying the patch.


Better? Thanks for the review.
ben
>From 910c98d233a271bf4ee74ee9edf4c42fb090afd5 Mon Sep 17 00:00:00 2001
From: Ben Woodcroft 
Date: Wed, 6 Jan 2016 21:23:15 +1000
Subject: [PATCH 1/4] gnu: ruby-power-assert: Disable failing test.

* gnu/packages/ruby.scm (ruby-power-assert)[arguments]: Disable test which
fails on Ruby 2.3.0.
---
 gnu/packages/ruby.scm | 12 
 1 file changed, 12 insertions(+)

diff --git a/gnu/packages/ruby.scm b/gnu/packages/ruby.scm
index 47ac392..9706ec7 100644
--- a/gnu/packages/ruby.scm
+++ b/gnu/packages/ruby.scm
@@ -1067,6 +1067,18 @@ using Net::HTTP, supporting reconnection and retry according to RFC 2616.")
(base32
 "0gbj379jhnff8rbb6m3kzdm282szjz1a021xzxa38d1bnswj2jx3"
 (build-system ruby-build-system)
+(arguments
+ '(#:phases
+   (modify-phases %standard-phases
+ (add-before 'check 'patch-test
+   ;; Disable test that fails on Ruby 2.3.  This has been fixed
+   ;; upstream but the fix has not been released.
+   (lambda _
+ (substitute*
+ "test/test_power_assert.rb"
+   (("def test_inspection_failure")
+"def dont_test_inspection_failure"))
+ #t)
 (native-inputs
  `(("bundler" ,bundler)))
 (synopsis "Assert library with descriptive assertion messages")
-- 
2.6.3

>From 207d2fa7769a4fea5a3a32a7227b2c7a2e0293bb Mon Sep 17 00:00:00 2001
From: Ben Woodcroft 
Date: Wed, 6 Jan 2016 21:57:44 +1000
Subject: 

Re: [PATCH] Update Ruby to 2.3.0 (was Re: Freezing core-updates soon)

[Ludovic Courtès]

Ben Woodcroft  skribis:

> I had some trouble downloading dependencies because of inability to
> download sources e.g. http://graphviz.org/ seems to be down. Ruby
> seems to build though, can we just review as normal and then push to
> core-updates?
>
> I did rebuild all the packages in ruby.scm, and out of these about 1/3
> of packages failed to build, because at least these three packages'
> tests fail
> * yard
> * power_assert
> * minitest_tu_shim
>
> The first two presumably will be fixed upstream soon, but the third
> hasn't seen a release since Aug 2013, and the homepage listed on
> rubygems is dead. I guess we just have to encourage the only package
> we have packaged that uses tu_shim (term-ansicolor) to stop relying on
> it.

OK, thanks for testing this thoroughly.  It would be good to fix these 3
packages sometime before core-updates is merged.

> From cd8c4e078cb930d1ce928b3c614b2081ec6719ac Mon Sep 17 00:00:00 2001
> From: Ben Woodcroft 
> Date: Mon, 4 Jan 2016 09:38:42 +1000
> Subject: [PATCH] gnu: ruby: Update to 2.3.0.
>
> * gnu/packages/ruby.scm (ruby): Update to 2.3.0.
> (ruby-2.2): New variable.

[...]

> +   (modify-phases %standard-phases
> + (add-before 'configure 'replace-bin-sh
> +   (lambda _
> + (substitute* '("Makefile.in"
> +"ext/pty/pty.c"
> +"io.c"
> +"lib/mkmf.rb"
> +"process.c"
> +"test/rubygems/test_gem_ext_configure_builder.rb"
> +"test/rdoc/test_rdoc_parser.rb"
> +"test/ruby/test_rubyoptions.rb"
> +"test/ruby/test_process.rb"
> +"test/ruby/test_system.rb"
> +"tool/rbinstall.rb")
> +   (("/bin/sh") (which "sh")))
> + (substitute* "ext/fiddle/libffi-3.2.1/configure"
> +   (("SHELL = /bin/sh")
> +(string-append "SHELL = " (which "sh"
> + #t)
> +(inputs
> + `(("readline" ,readline)
> +   ("openssl" ,openssl)
> +   ("libffi" ,libffi)
> +   ("gdbm" ,gdbm)
> +   ("zlib" ,zlib)))
> +(native-search-paths
> + (list (search-path-specification
> +(variable "GEM_PATH")
> +(files (list (string-append "lib/ruby/gems/"
> +(version-major+minor version)
> +".0"))

This seems to be shared with 2.2.  Is it possible to avoid duplication
using something like:

  (define ruby
(package
  (inherit ruby-2.2)
  (version "2.3.4")
  ;; …
  ))

?

Thanks,
Ludo’.