[PATCH] Upgrade bedtools to 0.25.0.

[Ben Woodcroft]

A reasonably straightforward update.


Version 2.25.0 (3-Sept-2015)

1. Added new -F option that allows one to set the minimum fraction of
   overlap required for the B interval. This complements the
   functionality of the -f option.Available for intersect, coverage,
   map, subtract, and jaccard.
2. Added new -e option that allows one to require that the minimum
   fraction overlap is achieved in either A _OR_ B, not A _AND_ B which
   is the behavior of the -r option. Available for intersect, coverage,
   map, subtract, and jaccard.
3. Fixed a longstanding bug that prevented genomecov from reporting
   chromosomes that lack a single interval.
4. Modified a src directory called "aux" to "driver" to prevent
   compilation errors on Windows machines. Thanks very much to John
   Marshall.
5. Fixed a regression that caused the coverage tool to complain if BED
   files had less than 5 columns.
6. Fixed a variable overload bug that prevented compilation on Debian
   machines.
7. Speedups to the groupby tool.
8. New -delim option for the groupby tool.
9. Fixed a bug in map that prevented strand-specifc overlaps from being
   reported when using certain BEDPLUS formats.
10. Prevented excessive memory usage when not using pre-sorted input.

>From 0a1faccd5765b0005dccb4bdffb8badd87986286 Mon Sep 17 00:00:00 2001
From: Ben Woodcroft 
Date: Sat, 9 Jan 2016 16:27:37 +1000
Subject: [PATCH] gnu: bedtools: Update to 2.25.0.

* gnu/packages/bioinformatics.scm (bedtools): Update to 0.25.0.
[arguments]: Use 'modify-phases'.  Remove 'patch-makefile-SHELL-definition'
phase.
[source]: Remove patch merged upstream.
---
 gnu/packages/bioinformatics.scm | 35 ---
 1 file changed, 12 insertions(+), 23 deletions(-)

diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index c598327..55b7cb1 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -1,6 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2014, 2015 Ricardo Wurmus 
-;;; Copyright © 2015 Ben Woodcroft 
+;;; Copyright © 2015, 2016 Ben Woodcroft 
 ;;; Copyright © 2015 Pjotr Prins 
 ;;; Copyright © 2015 Andreas Enge 
 ;;;
@@ -207,7 +207,7 @@ computational cluster.")
 (define-public bedtools
   (package
 (name "bedtools")
-(version "2.24.0")
+(version "2.25.0")
 (source (origin
   (method url-fetch)
   (uri (string-append "https://github.com/arq5x/bedtools2/archive/v";
@@ -215,8 +215,7 @@ computational cluster.")
   (file-name (string-append name "-" version ".tar.gz"))
   (sha256
(base32
-"0lnxrjvs3nnmb4bmskag1wg3h2hd80przz5q3xd0bvs7vyxrvpbl"))
-  (patches (list (search-patch "bedtools-32bit-compilation.patch")
+"1ywcy3yfwzhl905b51l0ffjia55h75vv3mw5xkvib04pp6pj548m"
 (build-system gnu-build-system)
 (native-inputs `(("python" ,python-2)))
 (inputs `(("samtools" ,samtools)
@@ -224,25 +223,15 @@ computational cluster.")
 (arguments
  '(#:test-target "test"
#:phases
-   (alist-cons-after
-'unpack 'patch-makefile-SHELL-definition
-(lambda _
-  ;; patch-makefile-SHELL cannot be used here as it does not
-  ;; yet patch definitions with `:='.  Since changes to
-  ;; patch-makefile-SHELL result in a full rebuild, features
-  ;; of patch-makefile-SHELL are reimplemented here.
-  (substitute* "Makefile"
-(("^SHELL := .*$") (string-append "SHELL := " (which "bash") " -e \n"
-(alist-delete
- 'configure
- (alist-replace
-  'install
-  (lambda* (#:key outputs #:allow-other-keys)
-(let ((bin (string-append (assoc-ref outputs "out") "/bin/")))
-  (for-each (lambda (file)
-  (install-file file bin))
-(find-files "bin" ".*"
-  %standard-phases)
+   (modify-phases %standard-phases
+ (delete 'configure)
+ (replace 'install
+   (lambda* (#:key outputs #:allow-other-keys)
+ (let ((bin (string-append (assoc-ref outputs "out") "/bin/")))
+   (for-each (lambda (file)
+   (install-file file bin))
+ (find-files "bin" ".*")))
+ #t)
 (home-page "https://github.com/arq5x/bedtools2";)
 (synopsis "Tools for genome analysis and arithmetic")
 (description
-- 
2.6.3

Re: Ruby security updates

[Ben Woodcroft]

On 09/01/16 10:15, Thompson, David wrote:

On Fri, Jan 8, 2016 at 6:48 PM, Mark H Weaver  wrote:

Some of our ruby versions may need security updates.

   https://bugzilla.redhat.com/show_bug.cgi?id=1248935

Can someone who cares about ruby please investigate?

This particular issue is definitely fixed in Ruby 2.2.4 or later,
which we upgraded very recently in response to this.
Indeed, but seems it also affects 2.1 < 2.1.8, where we have 2.1.6. I've 
attached a trivial patch that updates it - ok to push?

Now, I suspect Pjotr will find issue with this, but I think we really
should drop the Ruby 1.8.7 package because it is end-of-life and will
*not* receive bug fixes or security updates.


In general though it is a shame to remove old packages, Guix seems well 
suited to keeping old software usable. Is there a more useful place for 
removed packages to go other than the trash? A collection of exported 
profiles perhaps?


ben
>From 4c40fa0229dc2cb479227c16f23abad703101b70 Mon Sep 17 00:00:00 2001
From: Ben Woodcroft 
Date: Sat, 9 Jan 2016 14:53:58 +1000
Subject: [PATCH] gnu: ruby-2.1: Update to 2.1.8.

* gnu/packages/ruby.scm (ruby-2.1): Update to 2.1.8.
---
 gnu/packages/ruby.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/ruby.scm b/gnu/packages/ruby.scm
index 4ac3385..577be18 100644
--- a/gnu/packages/ruby.scm
+++ b/gnu/packages/ruby.scm
@@ -97,7 +97,7 @@ a focus on simplicity and productivity.")
 
 (define-public ruby-2.1
   (package (inherit ruby)
-(version "2.1.6")
+(version "2.1.8")
 (source
  (origin
(method url-fetch)
@@ -106,7 +106,7 @@ a focus on simplicity and productivity.")
"/ruby-" version ".tar.bz2"))
(sha256
 (base32
- "1sbcmbhadcxk0509svwxbm2vvgmpf3xjxr1397bgp9x46nz36lkv"
+ "11rkbfc90cg9p9mzg32475alf3ddcn9q8a3ar3fwm5xskic0n395"
 (arguments
  `(#:test-target "test"
#:parallel-tests? #f
-- 
2.6.3

Re: Ruby security updates

[Pjotr Prins]

Ruby 1.8.7 is still being used.  For me one of the selling points of
GNU Guix is that we can retain older packages when they are still
useful. The switch from Ruby 1.8 to 1.9 was quite intrusive and not
all software made the switch (similar to the python 2 to 3
switch). Some people argue that the software should be updated, but it
sometimes proves to be (too) hard or not worth the effort. Ruby 1.8 is
still a nice interpreter (it was the original Ruby by Matz).

If you run Ruby 1.8 in user space the security concerns are not really
relevant. There is no magic, Ruby can not circumvent the Linux
kernel's permissions.

So, the question here is not about security per se, it is more about
what packages do we retain in Guix. I think in this case, because
there are users, Ruby 1.8 belongs in Guix. Guix' versioning and
isolation allows for using different versions of software and
retaining Ruby 1.8's incompatibility with later Ruby's makes it a
distinct selling point for Guix.

Of course I can do without. But now I can point to others at the incompatiple
versions of Ruby we support, as well as Python, Perl and samtools, for example.
If you ditch 1.8.7 I won't be upset, but I hope you see my point. There is no
real cost attached and plenty upside :)

Pj.

On Fri, Jan 08, 2016 at 07:15:53PM -0500, Thompson, David wrote:
> On Fri, Jan 8, 2016 at 6:48 PM, Mark H Weaver  wrote:
> > Some of our ruby versions may need security updates.
> >
> >   https://bugzilla.redhat.com/show_bug.cgi?id=1248935
> >
> > Can someone who cares about ruby please investigate?
> 
> This particular issue is definitely fixed in Ruby 2.2.4 or later,
> which we upgraded very recently in response to this.
> 
> Now, I suspect Pjotr will find issue with this, but I think we really
> should drop the Ruby 1.8.7 package because it is end-of-life and will
> *not* receive bug fixes or security updates.
> 
> Thoughts?
> 
> - Dave
> 

-- 

Re: Ruby security updates

[Thompson, David]

On Fri, Jan 8, 2016 at 6:48 PM, Mark H Weaver  wrote:
> Some of our ruby versions may need security updates.
>
>   https://bugzilla.redhat.com/show_bug.cgi?id=1248935
>
> Can someone who cares about ruby please investigate?

This particular issue is definitely fixed in Ruby 2.2.4 or later,
which we upgraded very recently in response to this.

Now, I suspect Pjotr will find issue with this, but I think we really
should drop the Ruby 1.8.7 package because it is end-of-life and will
*not* receive bug fixes or security updates.

Thoughts?

- Dave

Ruby security updates

[Mark H Weaver]

Some of our ruby versions may need security updates.

  https://bugzilla.redhat.com/show_bug.cgi?id=1248935

Can someone who cares about ruby please investigate?

  Mark

Re: 03/05: gnu: icedtea6, icedtea7: Rename package to "icedtea".

[Mark H Weaver]

Ricardo Wurmus  writes:

> rekado pushed a commit to branch master
> in repository guix.
>
> commit 5452588c1a662bdcee240342a9df87efa4b1c017
> Author: Ricardo Wurmus 
> Date:   Wed Dec 30 14:31:16 2015 +0100
>
> gnu: icedtea6, icedtea7: Rename package to "icedtea".
> 
> * gnu/packages/java.scm (icedtea6): Change package name to "icedtea".
> (icedtea7): Inherit package name.

This broke the builds for icedtea-1.13.9, see:

  http://hydra.gnu.org/eval/108721#tabs-new
  http://hydra.gnu.org/build/908975/log/tail-reload

The relevant excerpt is:

--8<---cut here---start->8---
In unknown file:
   ?: 0 [chdir "icedtea-1.13.9"]

ERROR: In procedure chdir:
ERROR: In procedure chdir: No such file or directory
builder for `/gnu/store/dl93r8kpd9g7g3iamwz2n05nxlwjsfsn-icedtea-1.13.9.drv' 
failed with exit code 1
--8<---cut here---end--->8---

I guess the problem is that the custom unpack phase does this:

   (chdir (string-append ,name "-" ,version))

and this patch changes the value of 'name'.

 Regards,
   Mark


> diff --git a/gnu/packages/java.scm b/gnu/packages/java.scm
> index c4ad45e..b4c7bc3 100644
> --- a/gnu/packages/java.scm
> +++ b/gnu/packages/java.scm
> @@ -186,7 +186,7 @@ build process and its dependencies, whereas Make uses 
> Makefile format.")
>  
>  (define-public icedtea6
>(package
> -(name "icedtea6")
> +(name "icedtea")
>  (version "1.13.9")
>  (source (origin
>(method url-fetch)
> @@ -514,7 +514,8 @@ build process and its dependencies, whereas Make uses 
> Makefile format.")
> (alist-replace
>  'install
>  (lambda* (#:key outputs #:allow-other-keys)
> -  (let ((doc (string-append (assoc-ref outputs "doc") 
> "/share/doc/" ,name))
> +  (let ((doc (string-append (assoc-ref outputs "doc")
> +"/share/doc/icedtea"))
>  (jre (assoc-ref outputs "out"))
>  (jdk (assoc-ref outputs "jdk")))
>  (copy-recursively "openjdk.build/docs" doc)
> @@ -585,7 +586,6 @@ build process and its dependencies, whereas Make uses 
> Makefile format.")
>   "/icedtea7/" version "/" name ".tar.bz2"))
> (sha256 (base32 hash))
>  (package (inherit icedtea6)
> -  (name "icedtea7")
>(version version)
>(source (origin
>  (method url-fetch)

Re: [PATCH] guix: build-system: python: Add an optional argument EXTENSION to pypi-uri

[swedebugia]

On 2016-01-08 19:20, l...@gnu.org wrote:

swedebu...@riseup.net skribis:


I corrected as suggested by cyril and efraim.

I hope it is acceptable now.


Oops, I was reading messages older-first, and I didn’t see that there
had been a followup to the previous discussion.  Sorry about that!


np!


Commit 17ad0a2 does the same as what you posted.


:)

Re: [PATCH] guix: build-system: python: Add an optional argument EXTENSION to pypi-uri

[Ludovic Courtès]

swedebu...@riseup.net skribis:

> I corrected as suggested by cyril and efraim.
>
> I hope it is acceptable now.

Oops, I was reading messages older-first, and I didn’t see that there
had been a followup to the previous discussion.  Sorry about that!
Commit 17ad0a2 does the same as what you posted.

Ludo’.

Re: [PATCH] Add rubygems updater.

[Ludovic Courtès]

Ben Woodcroft  skribis:

> On 04/01/16 00:06, Ludovic Courtès wrote:
>> Ben Woodcroft  skribis:
>>
>>> On 03/01/16 06:54, Ludovic Courtès wrote:
 Ben Woodcroft  skribis:

> On 02/01/16 04:17, Ludovic Courtès wrote:
>> Ben Woodcroft  skribis:
>> [...]
>>
>>> + `(#:phases
>>> +   (modify-phases %standard-phases
>>> + (replace 'check
>>> +   (lambda _
>>> + (zero? (system* "ruby" "-Ilib" "-r" "ansi")))

[...]

>>>  (build-system ruby-build-system)
>>>  (arguments
>>>   `(#:import "ansi"
>>> #:tests? #f)) ; tests require circular dependencies
>> The problem is that the “-Ilib” in the command above cannot be guessed,
>> can it?
> My understanding is that the the "-Ilib" is almost invariant because
> putting imported code in the lib subdirectory is a convention that
> most gems adhere to. In those cases where it fails, the 'check-import
> phase can be replaced or removed.

OK.  My point is that, since we’re talking about saving 4 lines of code,
we have to make sure that the default thing works for the vast majority
of Ruby packages.

> [..]
>>> We could even default this to the expected name of the library guessed
>>> from the name of the package when #:import is not given. However, this
>>> would unfortunately break packages that have been written outside of
>>> Guix, so I imagine you don't feel this is a good idea.
>> We could choose the package name as a default value, but often that’s
>> not going to work, notably because of the “ruby-” prefix.
>>
>> WDYT?
> Removing the "ruby-" from the package name sounds like a reasonable
> default, but won't work every time because some imports use
> underscores where some use dashes e.g. "minitest-pretty_diff".
>
> I'm keen to make sure you understand what I'm attempting to say
> though. By "default" I mean when the #:import flag is missing from
> arguments, "ruby -Ilib -r " will be run. So if I
> have previously packaged a rubygem outside Guix and it is working
> fine, implementing the default might break my package making me
> unhappy. If you instead interpreted "default" as the guessed value
> that "guix import" generates, then that is less likely to end in
> unhappiness.

I was thinking of the #:import default value in ‘ruby-build-system’.

Using properties as Ricardo suggests woudln’t be any more concise than
using #:import "the-right-name".

Thanks,
Ludo’.

Re: [PATCH] guix pypi importer: Add ending as an optional argument to pypi-uri.

[Ludovic Courtès]

I’ve pushed commit 17ad0a2, which does what you want but using an
optional parameter as discussed by Cyril and Efraim.

Thanks,
Ludo’.

Re: Error log analys from building ~2851 packages

[Ludovic Courtès]

swedebu...@riseup.net skribis:

> Get the package list:
> guix package -A|pick the first field with cut/gawk >list
>
> Then a bash for-loop along the lines of:
> for $i in `cat list` ; echo "building $i" && do guix build $i
> --verbosity=1 -c 2 1<&2 build.log ; done
>
> Then follow the build with:
> tail -F build.log

OK.

>> I would like to provide a more convenient interface, like
>> ‘guix build --all’ or something like that.
>
> Sounds like a good idea. I would prefer if guix build to be able to
> instruct the daemon to cache failures if possible.

The default is to not cache failures, on the grounds that we experience
non-deterministic failures occasionally.  But I think we might want to
change the default at some point, since it’s always possible to clear
cached failures anyway.

Besides, on GuixSD, it’s always possible to change the options passed to
guix-daemon (info "(guix) Using the Configuration System"):

   (modify-services %base-services
 (guix-service-type config =>
(guix-configuration
 (inherit config)
 (extra-options '("--cache-failures")

Thanks,
Ludo’.

Re: Error log analys from building ~2851 packages

[swedebugia]

On 2016-01-07 15:33, l...@gnu.org wrote:

swedebu...@riseup.net skribis:


Would you like to know the results of my quest to build all packages?


This is interesting, thanks for doing it!


:D


Out of curiosity, what command did you use to do that?


I did this on GuixSD:
stopped the daemon: sudo deco stop guix-daemon
started the daemon again manually with --cache-failures

Get the package list:
guix package -A|pick the first field with cut/gawk >list

Then a bash for-loop along the lines of:
for $i in `cat list` ; echo "building $i" && do guix build $i 
--verbosity=1 -c 2 1<&2 build.log ; done


Then follow the build with:
tail -F build.log


I would like to provide a more convenient interface, like
‘guix build --all’ or something like that.


Sounds like a good idea. I would prefer if guix build to be able to 
instruct the daemon to cache failures if possible.


cheers

Re: [PATCH] build: Speed up .go compilation.

[Ludovic Courtès]

taylanbayi...@gmail.com (Taylan Ulrich "Bayırlı/Kammer") skribis:

> Here's an updated version that uses the same strategy as the one we
> settled on for 'guix pull'.

So, what speedup to you get compared to ‘make -jN’?

> From 220a8caed6da22e349545899d5c51083bb3a8ac5 Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?Taylan=20Ulrich=20Bay=C4=B1rl=C4=B1/Kammer?=
>  
> Date: Thu, 5 Nov 2015 23:42:45 +0100
> Subject: [PATCH] build: Speed up .go compilation.
>
> * build-aux/compile-all.scm: New file.
> * Makefile.am: Call build-aux/compile-all.scm to compile many .scm files
>   in a single process.

Rather:

  * Makefile.am (%.go, make-go): New rules.

Also, the new script must be added to ‘EXTRA_DIST’.

> +++ b/build-aux/compile-all.scm

Please add a copyright/license header.

> +(define (file->module file)
> +  (map string->symbol
> +   (string-split (string-drop-right file 4) #\/)))

Does this work with out-of-tree builds?

> +(let* ((args (cdr (command-line)))
> +   (target (car args))
> +   (files (cdr args)))

Please use ‘match’ for the win!  :-)

> +  (for-each
> +   (lambda (file)
> + (let ((go (scm->go file)))
> +   (unless (and (file-exists? go)
> +(file-mtime + (let ((module (file->module file)))
> +   (format #t "  LOAD ~s~%" module)
> +   (resolve-interface module)
> +   files)

Make the ‘lambda’ a named top-level procedure, for clarity.  Also add a
reference to the evil bug that makes this hack necessary in the first
place.

Maybe it would be clearer to have:

  (define (file-needs-compilation? file)
(file-mtimego file) file))

and then to:

  (let ((files (filter file-needs-compilation? files)))
;; …
)

> +  (with-target target
> +(lambda ()
> +  (let ((mutex (make-mutex)))
> +(par-for-each
> + (lambda (file)
> +   (let ((go (scm->go file)))
> + (unless (and (file-exists? go)
> +  (file-mtime +   (with-mutex mutex
> + (format #t "  GUILEC ~s~%" file)
> + (force-output))
> +   (compile-file file #:output-file go #:opts compile-options)
> +   (with-mutex mutex
> + (format #t "  WROTE ~s~%" go)
> + (force-output)
> + files)

Ditto: name the lambda.

I would use ~a instead of ~s, to match the current output, and remove
the “WROTE” output.

Could you send an updated patch?

It would be awesome if you could check that ‘make distcheck’ still
passes, and also make sure things behave correctly when modifying just
one file and running ‘make’, things like that.

Thank you!

Ludo’.

Re: [PATCH] Help Ruby packages be reproducible

[Ludovic Courtès]

Ben Woodcroft  skribis:

> On 08/01/16 00:31, Ludovic Courtès wrote:
>> Hi Ben,
>>
>> Do you have an account on Savannah?  If not, I invite you to create one
>> so we can give you write access to the repo.  :-)
> Sweet, thanks muchly.
>
> My new savannah account name is benwoodcroft

I’ve added you, welcome aboard!

Please read the ‘HACKING’ file for commit access rules.  If in doubt
with Git or anything, don’t hesitate to ask here or on #guix on IRC.

Thanks,
Ludo’.

[PATCH] Update Ruby to 2.3.0

[Ludovic Courtès]

Ben Woodcroft  skribis:

> From 28dee96b0de99adf8657457660b9e3733434d2b4 Mon Sep 17 00:00:00 2001
> From: Ben Woodcroft 
> Date: Mon, 4 Jan 2016 09:38:42 +1000
> Subject: [PATCH 4/4] gnu: ruby: Update to 2.3.0.
>
> * gnu/packages/ruby.scm (ruby): Update to 2.3.0.

Please mention the switch to ‘modify-phases’ here.

> (ruby-2.2): New variable.

[...]

> + (substitute* "ext/fiddle/libffi-3.2.1/configure"
> +   (("SHELL = /bin/sh")
> +(string-append "SHELL = " (which "sh"

I overlooked this part before but: (1) this looks weird (the spaces
around ‘=’ suggest it’s Makefile syntax, not shell syntax), and (2) we
shouldn’t use the bundled libffi.

How much work would it be to fix #2?  It would be awesome if you could
do that.  Sorry for the extra work!

> +(define-public ruby-2.2
> +  (package (inherit ruby)
> +(version "2.2.4")
> +(source
> + (origin
> +   (method url-fetch)
> +   (uri (string-append "http://cache.ruby-lang.org/pub/ruby/";
> +   (version-major+minor version)
> +   "/ruby-" version ".tar.bz2"))
> +   (sha256
> +(base32
> + "0g3ps4q3iz7wj9m45n8xyxzw8nh29ljdqb87b0f6i0p3853gz2yj"
> +(arguments
> + `(#:test-target "test"
> +   #:parallel-tests? #f
> +   #:phases
> +   (alist-cons-before
> +'configure 'replace-bin-sh
> +(lambda _
> +  (substitute* '("Makefile.in"
> + "ext/pty/pty.c"
> + "io.c"
> + "lib/mkmf.rb"
> + "process.c"
> + "test/rubygems/test_gem_ext_configure_builder.rb"
> + "test/rdoc/test_rdoc_parser.rb"
> + "test/ruby/test_rubyoptions.rb"
> + "test/ruby/test_process.rb"
> + "test/ruby/test_system.rb"
> + "tool/rbinstall.rb")
> +(("/bin/sh") (which "sh"
> +%standard-phases)

AFAICS all of ‘arguments’ can be omitted once the above libffi
substitution is removed.

(If it there’s still a need for different phases, it should use
‘substitute-keyword-arguments’ to modify only #:phases, and then
‘modify-phases’ rather than ‘alist-cons-before’.)

Could you send an updated patch, hopefully the last one?  :-)

Thanks,
Ludo’.

Re: [PATCH] Help Ruby packages be reproducible

[Thompson, David]

On Wed, Jan 6, 2016 at 6:46 AM, Ricardo Wurmus
 wrote:
>
> Ben Woodcroft  writes:
>
>> On 02/01/16 01:10, Ludovic Courtès wrote:
> [...]
>>> So I guess you (Ricardo?) can push it now.
> [...]
>> Thanks for pushing the patch.
>
> I did this just now.  Sorry for the delay.  Pushed as 7266848.  Thanks,
> Ben, for helping to make Ruby packages reproducible!

Just catching up on this conversation now.  Thanks, Ben!

- Dave

[PATCH] build: Speed up .go compilation.

[Taylan Ulrich Bayırlı/Kammer]

Here's an updated version that uses the same strategy as the one we
settled on for 'guix pull'.

>From 220a8caed6da22e349545899d5c51083bb3a8ac5 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Taylan=20Ulrich=20Bay=C4=B1rl=C4=B1/Kammer?=
 
Date: Thu, 5 Nov 2015 23:42:45 +0100
Subject: [PATCH] build: Speed up .go compilation.

* build-aux/compile-all.scm: New file.
* Makefile.am: Call build-aux/compile-all.scm to compile many .scm files
  in a single process.
---
 Makefile.am   | 22 --
 build-aux/compile-all.scm | 44 
 2 files changed, 52 insertions(+), 14 deletions(-)
 create mode 100644 build-aux/compile-all.scm

diff --git a/Makefile.am b/Makefile.am
index 760caed..4aa459d 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -341,14 +341,6 @@ CLEANFILES =	\
   $(GOBJECTS)	\
   $(SCM_TESTS:tests/%.scm=%.log)
 
-AM_V_GUILEC = $(AM_V_GUILEC_$(V))
-AM_V_GUILEC_ = $(AM_V_GUILEC_$(AM_DEFAULT_VERBOSITY))
-AM_V_GUILEC_0 = @echo "  GUILEC" $@;
-
-# Flags passed to 'guild compile'.
-GUILD_COMPILE_FLAGS =\
-  -Wformat -Wunbound-variable -Warity-mismatch
-
 # Unset 'GUILE_LOAD_COMPILED_PATH' altogether while compiling.  Otherwise, if
 # $GUILE_LOAD_COMPILED_PATH contains $(moduledir), we may find .go files in
 # there that are newer than the local .scm files (for instance because the
@@ -358,14 +350,16 @@ GUILD_COMPILE_FLAGS =\
 #
 # XXX: Use the C locale for when Guile lacks
 # .
-.scm.go:
-	$(AM_V_GUILEC)$(MKDIR_P) `dirname "$@"` ;			\
+%.go: make-go ; @:
+make-go: $(MODULES)
+	for f in $^; do			\
+  $(MKDIR_P) `dirname "$$f"` ;	\
+	done ;\
 	unset GUILE_LOAD_COMPILED_PATH ;\
 	LC_ALL=C			\
 	$(top_builddir)/pre-inst-env	\
-	$(GUILD) compile -L "$(top_builddir)" -L "$(top_srcdir)"	\
-	  $(GUILD_COMPILE_FLAGS) --target="$(host)"			\
-	  -o "$@" "$<"
+	$(GUILE) -L "$(top_builddir)" -L "$(top_srcdir)"		\
+	  --no-auto-compile -s build-aux/compile-all.scm $(host) $^
 
 SUFFIXES = .go
 
@@ -457,6 +451,6 @@ assert-final-inputs-self-contained:
 	$(top_builddir)/pre-inst-env "$(GUILE)"\
 	  "$(top_srcdir)/build-aux/check-final-inputs-self-contained.scm"
 
-.PHONY: sync-descriptions gen-ChangeLog gen-AUTHORS clean-go
+.PHONY: sync-descriptions gen-ChangeLog gen-AUTHORS clean-go make-go
 .PHONY: assert-no-store-file-names assert-binaries-available
 .PHONY: assert-final-inputs-self-contained
diff --git a/build-aux/compile-all.scm b/build-aux/compile-all.scm
new file mode 100644
index 000..f546822
--- /dev/null
+++ b/build-aux/compile-all.scm
@@ -0,0 +1,44 @@
+(use-modules (system base target)
+ (ice-9 threads))
+
+(define compile-options '(format unbound-variable arity-mismatch))
+
+(define (file-mtimego file)
+  (string-append (string-drop-right file 4) ".go"))
+
+(define (file->module file)
+  (map string->symbol
+   (string-split (string-drop-right file 4) #\/)))
+
+(let* ((args (cdr (command-line)))
+   (target (car args))
+   (files (cdr args)))
+  (for-each
+   (lambda (file)
+ (let ((go (scm->go file)))
+   (unless (and (file-exists? go)
+(file-mtimemodule file)))
+   (format #t "  LOAD ~s~%" module)
+   (resolve-interface module)
+   files)
+  (with-target target
+(lambda ()
+  (let ((mutex (make-mutex)))
+(par-for-each
+ (lambda (file)
+   (let ((go (scm->go file)))
+ (unless (and (file-exists? go)
+  (file-mtime

Re: [PATCH] Help Ruby packages be reproducible

[Ben Woodcroft]

On 08/01/16 00:31, Ludovic Courtès wrote:

Hi Ben,

Do you have an account on Savannah?  If not, I invite you to create one
so we can give you write access to the repo.  :-)

Sweet, thanks muchly.

My new savannah account name is benwoodcroft

I'll try not to mess things up too badly. Great to be more officially a 
part of GNU, though I'm sure I'll never be able to repay all the 
goodness it has provided me over the years.

ben

Re: [PATCH] Update Ruby to 2.3.0 (was Re: Freezing core-updates soon)

[Ben Woodcroft]

On 06/01/16 08:24, Ludovic Courtès wrote:

Ben Woodcroft  skribis:


I had some trouble downloading dependencies because of inability to
download sources e.g. http://graphviz.org/ seems to be down. Ruby
seems to build though, can we just review as normal and then push to
core-updates?

I did rebuild all the packages in ruby.scm, and out of these about 1/3
of packages failed to build, because at least these three packages'
tests fail
* yard
* power_assert
* minitest_tu_shim

The first two presumably will be fixed upstream soon, but the third
hasn't seen a release since Aug 2013, and the homepage listed on
rubygems is dead. I guess we just have to encourage the only package
we have packaged that uses tu_shim (term-ansicolor) to stop relying on
it.

OK, thanks for testing this thoroughly.  It would be good to fix these 3
packages sometime before core-updates is merged.
In the attached patches I've disabled the failing test in both yard and 
power_assert. There's no fix for yard in upstream. For power_assert 
there is a fix, but unfortunately it seems Ruby packages cannot be 
patched in the usual way because .gem files are archives of archives so 
the unpacking/patch/repacking fails. I guess we fix that in future.


Turns out fixing minitest_tu_shim (and others) was easier than I thought 
as the problem was some hardcoded ruby 2.2 paths in package definitions. 
I created a new procedure to remove the hard-coding so the update to 2.4 
will work too, WDYT?


After all that, everything in gnu/packages/ruby.scm builds.

+   (modify-phases %standard-phases
+ (add-before 'configure 'replace-bin-sh
+   (lambda _
+ (substitute* '("Makefile.in"
+"ext/pty/pty.c"
+"io.c"
+"lib/mkmf.rb"
+"process.c"
+"test/rubygems/test_gem_ext_configure_builder.rb"
+"test/rdoc/test_rdoc_parser.rb"
+"test/ruby/test_rubyoptions.rb"
+"test/ruby/test_process.rb"
+"test/ruby/test_system.rb"
+"tool/rbinstall.rb")
+   (("/bin/sh") (which "sh")))
+ (substitute* "ext/fiddle/libffi-3.2.1/configure"
+   (("SHELL = /bin/sh")
+(string-append "SHELL = " (which "sh"
+ #t)
+(inputs
+ `(("readline" ,readline)
+   ("openssl" ,openssl)
+   ("libffi" ,libffi)
+   ("gdbm" ,gdbm)
+   ("zlib" ,zlib)))
+(native-search-paths
+ (list (search-path-specification
+(variable "GEM_PATH")
+(files (list (string-append "lib/ruby/gems/"
+(version-major+minor version)
+".0"))

This seems to be shared with 2.2.  Is it possible to avoid duplication
using something like:

   (define ruby
 (package
   (inherit ruby-2.2)
   (version "2.3.4")
   ;; …
   ))

?

[..]

Yes, I did this but forgot to delete some/most duplication. Now after 
the final patch ruby-2.2 inherits from ruby (not vice-versa as you 
suggest) as this was the previous convention in ruby.scm. I checked that 
the hashes in the store names for ruby 2.2, 2.1 and 1.8 did not change 
after applying the patch.


Better? Thanks for the review.
ben
>From 910c98d233a271bf4ee74ee9edf4c42fb090afd5 Mon Sep 17 00:00:00 2001
From: Ben Woodcroft 
Date: Wed, 6 Jan 2016 21:23:15 +1000
Subject: [PATCH 1/4] gnu: ruby-power-assert: Disable failing test.

* gnu/packages/ruby.scm (ruby-power-assert)[arguments]: Disable test which
fails on Ruby 2.3.0.
---
 gnu/packages/ruby.scm | 12 
 1 file changed, 12 insertions(+)

diff --git a/gnu/packages/ruby.scm b/gnu/packages/ruby.scm
index 47ac392..9706ec7 100644
--- a/gnu/packages/ruby.scm
+++ b/gnu/packages/ruby.scm
@@ -1067,6 +1067,18 @@ using Net::HTTP, supporting reconnection and retry according to RFC 2616.")
(base32
 "0gbj379jhnff8rbb6m3kzdm282szjz1a021xzxa38d1bnswj2jx3"
 (build-system ruby-build-system)
+(arguments
+ '(#:phases
+   (modify-phases %standard-phases
+ (add-before 'check 'patch-test
+   ;; Disable test that fails on Ruby 2.3.  This has been fixed
+   ;; upstream but the fix has not been released.
+   (lambda _
+ (substitute*
+ "test/test_power_assert.rb"
+   (("def test_inspection_failure")
+"def dont_test_inspection_failure"))
+ #t)
 (native-inputs
  `(("bundler" ,bundler)))
 (synopsis "Assert library with descriptive assertion messages")
-- 
2.6.3

>From 207d2fa7769a4fea5a3a32a7227b2c7a2e0293bb Mon Sep 17 00:00:00 2001
From: Ben Woodcroft 
Date: Wed, 6 Jan 2016 21:57:44 +1000
Subject: [PATCH 2/4] gnu: ruby-yard: Disable failing test.

* gnu/packages/ruby.sc

Re: dmd’s new name

[Ludovic Courtès]

Adam Pribyl  skribis:

> The only disadvantage is it is long and in the text may get confused
> with a ordinary word and I personaly always struggle with spelling.

Hopefully, you’ll rarely have to type it.  ;-)

> I hope, besides the name change, there is going to be some more
> development on Shepherd then...

Yup, it needs some love!

Ludo’.

Re: [PATCH] gnu: unison: Add "doc" output.

[Ludovic Courtès]

Eric Bavier  skribis:

> From c8a0086db4d78091e06326a5ffe55a7bf2543091 Mon Sep 17 00:00:00 2001
> From: Eric Bavier 
> Date: Fri, 18 Dec 2015 19:29:22 -0600
> Subject: [PATCH] gnu: unison: Add "doc" output.
>
> * gnu/packages/ocaml.scm (unison)[source]: Use svn-fetch with snippet.
>   [outputs]: New field.
>   [native-inputs]: Add ghostscript, texlive, hevea, and lynx.
>   [arguments]: Add 'install-doc phase.

[...]

> +(snippet
> + `(begin
> +;; The svn revision in the release tarball appears to be
> +;; artificially manipulated in order to set the desired point
> +;; version number.  Because the point version is calculated 
> during
> +;; the build, we can offset pointVersionOrigin by the desired
> +;; point version and write that into "Rev: %d".  We do this 
> rather
> +;; than hardcoding the necessary revision number, for
> +;; maintainability.

Fun.  :-)

> +(with-atomic-file-replacement "src/mkProjectInfo.ml"
> +  (lambda (in out)
> +(let ((pt-ver (string->number (third (string-split ,version 
> #\.
> +  (pt-rx  (make-regexp "^let pointVersionOrigin = 
> ([0-9]+)"))
> +  (rev-rx (make-regexp "Rev: [0-9]+")))
> +  (let loop ((pt-origin #f))
> +(let ((line (read-line in 'concat)))
> +  (cond
> +   ((regexp-exec pt-rx line)
> +=> (lambda (m)
> + (begin

‘begin’ can be omitted here…

> +   ((regexp-exec rev-rx line)
> +=> (lambda (m)
> + (begin

… and here.

> +(for-each (lambda (f)
> +(install-file f doc))
> +  (map (lambda (ext)
> + (string-append
> +  "doc/unison-manual." ext))
> +   '("ps" "pdf" "dvi" "html")))

What about installing only HTML?

I find that HTML (and Info, and man) is more convenient to read on-line
than PDFs.  We rarely include PDF documentation in packages.

Of course, avoiding PDF/PS/DVI allows us to remove the dependency on
TeX Live.  Last, from discussions I heard at the Reproducible Build
Summit, I think DVIs and maybe PS/PDFs are not bit-reproducible
out-of-the-box.

WDYT?

Maybe we should have a policy on how to choose the installed
documentation formats.

Thanks for finding out how to build documentation from source!

Ludo’.

Re: dmd’s new name

[Adam Pribyl]

On Thu, 7 Jan 2016, Ludovic Courtès wrote:


l...@gnu.org (Ludovic Courtès) skribis:


Walter Bright  skribis:


The D programming language compiler has been called dmd since about
2000 or so. In order to avoid confusion, may I request that the Gnu
dmd ("Daemons-managing Daemon") change its acronym?


For the record, GNU’s dmd has been around since ~2003¹.

Changing names is obviously an annoyance, technically and otherwise.
We’re not opposed to the idea, but that would take a bit more discussion
among developers.


You might remember this discussion from March 2015.

 https://lists.gnu.org/archive/html/guix-devel/2015-03/msg00176.html

I proposed to Richard Stallman that we change the name to “GNU Shepherd”
and recently got the green light.

Rationale:

 1. A shepherd looks after sheep (or gnus ;-)), and GNU dmd looks after
daemons.

 2. “Shepherd” ends with a ‘d’, which makes it a good name for a daemon.

 3. It’s a pun in the same spirit as “GNU Hurd”.

 4. We might even rename ‘deco’ to ‘herd’ because it’s the very command
that allows users to herd their daemons.

Hope you like it.  :-)


The only disadvantage is it is long and in the text may get confused with 
a ordinary word and I personaly always struggle with spelling.



Ludo’.


I hope, besides the name change, there is going to be some more 
development on Shepherd then...


Adam Pribyl