Re: [PATCH] Add preseq.
Mark H Weaver writes: > No. It's deterministic, and 'modify-phases' is in (guix build utils) so > it can't be changed without triggering a full rebuild. The edits are > done in the order shown, as if it was imperative code. For example: Thank you for confirming the behaviour. >> When I built it locally back then “use-samtools-headers” must have >> ended up after “enter-dir”.) > > I don't see how that could have happened. Maybe you made some changes > after your last test? Anyway, if 'use-samtools-headers' must come after > 'enter-dir' then it's surely better to make that explicit. In that case it must have been that I made a change (reordering the phases) and forgot to build preseq afterward. My apologies for the noise! ~~ Ricardo
Re: [PATCH] Add preseq.
Ricardo Wurmus writes: >> ERROR: In procedure mkstemp!: >> ERROR: In procedure mkstemp!: No such file or directory >> builder for `/gnu/store/2ll28ypms57h0k44f4mmclhw9dz3ccvm-preseq-1.0.2.drv' >> failed with exit code 1 >> @ build-failed >> /gnu/store/2ll28ypms57h0k44f4mmclhw9dz3ccvm-preseq-1.0.2.drv - 1 >> builder for >> `/gnu/store/2ll28ypms57h0k44f4mmclhw9dz3ccvm-preseq-1.0.2.drv' >> failed with exit code 1 >> guix build: error: build failed: build of >> `/gnu/store/2ll28ypms57h0k44f4mmclhw9dz3ccvm-preseq-1.0.2.drv' failed > > The problem was that the “enter-dir” and “use-samtools-headers” phases > were both added after “unpack”. (Has the order in which phases are > added with ‘modify-phases’ changed? No. It's deterministic, and 'modify-phases' is in (guix build utils) so it can't be changed without triggering a full rebuild. The edits are done in the order shown, as if it was imperative code. For example: (modify-phases %standard-phases (add-after 'unpack 'A ) (add-after 'unpack 'B )) expands into: (let* ((phases* %standard-phases) (phases* (alist-cons-after 'unpack 'A phases*)) (phases* (alist-cons-after 'unpack 'B phases*))) phases*) which is equivalent to: (alist-cons-after 'unpack 'B (alist-cons-after 'unpack 'A %standard-phases)) Since B is the last one added, it ends up after 'unpack' and before A. > When I built it locally back then “use-samtools-headers” must have > ended up after “enter-dir”.) I don't see how that could have happened. Maybe you made some changes after your last test? Anyway, if 'use-samtools-headers' must come after 'enter-dir' then it's surely better to make that explicit. > I just pushed a commit that explicitly moves “use-samtools-headers” > after the “enter-dir” phase, fixing this problem. Thank you! Mark
Re: [PATCH] Add preseq.
> ERROR: In procedure mkstemp!: > ERROR: In procedure mkstemp!: No such file or directory > builder for `/gnu/store/2ll28ypms57h0k44f4mmclhw9dz3ccvm-preseq-1.0.2.drv' > failed with exit code 1 > @ build-failed /gnu/store/2ll28ypms57h0k44f4mmclhw9dz3ccvm-preseq-1.0.2.drv - > 1 builder for `/gnu/store/2ll28ypms57h0k44f4mmclhw9dz3ccvm-preseq-1.0.2.drv' > failed with exit code 1 > guix build: error: build failed: build of > `/gnu/store/2ll28ypms57h0k44f4mmclhw9dz3ccvm-preseq-1.0.2.drv' failed The problem was that the “enter-dir” and “use-samtools-headers” phases were both added after “unpack”. (Has the order in which phases are added with ‘modify-phases’ changed? When I built it locally back then “use-samtools-headers” must have ended up after “enter-dir”.) I just pushed a commit that explicitly moves “use-samtools-headers” after the “enter-dir” phase, fixing this problem. Thank you, Mark, for the notification. ~~ Ricardo
Re: [PATCH] Add preseq.
l...@gnu.org (Ludovic Courtès) writes: > Ricardo Wurmus skribis: > >> From e3bd0a861121c441b44d77d30f2aaddb766b49c2 Mon Sep 17 00:00:00 2001 >> From: Ricardo Wurmus >> Date: Tue, 23 Jun 2015 10:54:00 +0200 >> Subject: [PATCH 2/2] gnu: Add preseq. >> >> * gnu/packages/bioinformatics.scm (preseq): New variable. > > Please add the .patch files to the commit log and to the .am file. > > OK with this change. preseq failed to build on Hydra on x86_64 and mips64el. As I write this, the results are not yet in for i686. http://hydra.gnu.org:3000/job/gnu/master/preseq-1.0.2.x86_64-linux http://hydra.gnu.org:3000/job/gnu/master/preseq-1.0.2.mips64el-linux http://hydra.gnu.org:3000/job/gnu/master/preseq-1.0.2.i686-linux Here's the relevant build log excerpt for x86_64: --8<---cut here---start->8--- starting phase `use-samtools-headers' Backtrace: In ice-9/boot-9.scm: 157: 15 [catch #t # ...] In unknown file: ?: 14 [apply-smob/1 #] In ice-9/boot-9.scm: 63: 13 [call-with-prompt prompt0 ...] In ice-9/eval.scm: 432: 12 [eval # #] In ice-9/boot-9.scm: 2401: 11 [save-module-excursion #] 4050: 10 [#] 1724: 9 [%start-stack load-stack ...] 1729: 8 [#] In unknown file: ?: 7 [primitive-load "/gnu/store/6pzf2jhf9syyc02wbgl9mxma4wf2qxj2-preseq-1.0.2-guile-builder"] In ice-9/eval.scm: 387: 6 [eval # ()] In srfi/srfi-1.scm: 830: 5 [every1 # ...] In /gnu/store/l1019wb5yzbar26cxxqrm88xkbn5gwv1-module-import/guix/build/gnu-build-system.scm: 588: 4 [# #] In ice-9/eval.scm: 432: 3 [eval # #] In srfi/srfi-1.scm: 619: 2 [for-each # #] In /gnu/store/l1019wb5yzbar26cxxqrm88xkbn5gwv1-module-import/guix/build/utils.scm: 486: 1 [with-atomic-file-replacement "smithlab_cpp/SAM.cpp" ...] In unknown file: ?: 0 [mkstemp! "smithlab_cpp/SAM.cpp.XX"] ERROR: In procedure mkstemp!: ERROR: In procedure mkstemp!: No such file or directory builder for `/gnu/store/2ll28ypms57h0k44f4mmclhw9dz3ccvm-preseq-1.0.2.drv' failed with exit code 1 @ build-failed /gnu/store/2ll28ypms57h0k44f4mmclhw9dz3ccvm-preseq-1.0.2.drv - 1 builder for `/gnu/store/2ll28ypms57h0k44f4mmclhw9dz3ccvm-preseq-1.0.2.drv' failed with exit code 1 guix build: error: build failed: build of `/gnu/store/2ll28ypms57h0k44f4mmclhw9dz3ccvm-preseq-1.0.2.drv' failed --8<---cut here---end--->8--- Mark
Re: [PATCH] Add preseq.
Ricardo Wurmus skribis: > From b1ba27153faca9bdf46d42046c43f8c60f945368 Mon Sep 17 00:00:00 2001 > From: Ricardo Wurmus > Date: Thu, 2 Jul 2015 15:53:47 +0200 > Subject: [PATCH 1/2] gnu: samtools: Install headers. > > * gnu/packages/bioinformatics.scm (samtools)[arguments]: Add phase > "install-headers" to install header files. OK. > From e3bd0a861121c441b44d77d30f2aaddb766b49c2 Mon Sep 17 00:00:00 2001 > From: Ricardo Wurmus > Date: Tue, 23 Jun 2015 10:54:00 +0200 > Subject: [PATCH 2/2] gnu: Add preseq. > > * gnu/packages/bioinformatics.scm (preseq): New variable. Please add the .patch files to the commit log and to the .am file. OK with this change. Thank you! Ludo’.
Re: [PATCH] Add preseq.
>>> Why leave the headers? Isn’t it risky to leave headers that may be
>>> different from the Samtools that we’re actually linking against?
>>
>> The problem is with our Samtools package(s). They don't install
>> headers. I'd better update them to install the headers; then I could
>> just delete the complete "samtools" directory.
>
> Yes that would be best, if that’s fine with you.
Attached is a patch to add a build phase to samtools to install the
headers into $out/include/samtools/.
The second patch is just the updated patch for adding preseq.
>From b1ba27153faca9bdf46d42046c43f8c60f945368 Mon Sep 17 00:00:00 2001
From: Ricardo Wurmus
Date: Thu, 2 Jul 2015 15:53:47 +0200
Subject: [PATCH 1/2] gnu: samtools: Install headers.
* gnu/packages/bioinformatics.scm (samtools)[arguments]: Add phase
"install-headers" to install header files.
---
gnu/packages/bioinformatics.scm | 17 -
1 file changed, 16 insertions(+), 1 deletion(-)
diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index 4a1804a..ce2f55b 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -1824,6 +1824,10 @@ distribution, coverage uniformity, strand specificity, etc.")
;; systems.
#:tests? ,(string=? (or (%current-system) (%current-target-system))
"x86_64-linux")
+ #:modules ((ice-9 ftw)
+ (ice-9 regex)
+ (guix build gnu-build-system)
+ (guix build utils))
#:make-flags (list "LIBCURSES=-lncurses"
(string-append "prefix=" (assoc-ref %outputs "out")))
#:phases
@@ -1848,7 +1852,18 @@ distribution, coverage uniformity, strand specificity, etc.")
(let ((lib (string-append (assoc-ref outputs "out") "/lib")))
(mkdir-p lib)
(copy-file "libbam.a" (string-append lib "/libbam.a"
- (alist-delete 'configure %standard-phases)
+ (alist-cons-after
+ 'install 'install-headers
+ (lambda* (#:key outputs #:allow-other-keys)
+(let ((include (string-append (assoc-ref outputs "out")
+ "/include/samtools/")))
+ (mkdir-p include)
+ (for-each (lambda (file)
+ (copy-file file (string-append include
+ (basename file
+(scandir "." (lambda (name) (string-match "\\.h$" name
+ #t))
+ (alist-delete 'configure %standard-phases))
(native-inputs `(("pkg-config" ,pkg-config)))
(inputs `(("ncurses" ,ncurses)
("perl" ,perl)
--
2.1.0
>From e3bd0a861121c441b44d77d30f2aaddb766b49c2 Mon Sep 17 00:00:00 2001
From: Ricardo Wurmus
Date: Tue, 23 Jun 2015 10:54:00 +0200
Subject: [PATCH 2/2] gnu: Add preseq.
* gnu/packages/bioinformatics.scm (preseq): New variable.
---
gnu/packages/bioinformatics.scm| 56 ++
.../patches/preseq-1.0.2-install-to-PREFIX.patch | 37 ++
.../patches/preseq-1.0.2-link-with-libbam.patch| 33 +
3 files changed, 126 insertions(+)
create mode 100644 gnu/packages/patches/preseq-1.0.2-install-to-PREFIX.patch
create mode 100644 gnu/packages/patches/preseq-1.0.2-link-with-libbam.patch
diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index ce2f55b..27b847e 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -2137,6 +2137,62 @@ subsequent visualization, annotation and storage of results.")
;; LGPLv2.1+
(license (list license:gpl2 license:lgpl2.1+
+(define-public preseq
+ (package
+(name "preseq")
+(version "1.0.2")
+(source (origin
+ (method url-fetch)
+ (uri
+ (string-append "http://smithlabresearch.org/downloads/preseq-";
+ version ".tar.bz2"))
+ (sha256
+ (base32 "0r7sw07p6nv8ygvc17gd78lisbw5336v3vhs86b5wv8mw3pwqksc"))
+ (patches (list (search-patch "preseq-1.0.2-install-to-PREFIX.patch")
+ (search-patch "preseq-1.0.2-link-with-libbam.patch")))
+ (modules '((guix build utils)))
+ (snippet
+ ;; Remove bundled samtools.
+ '(delete-file-recursively "preseq-master/samtools"
+(build-system gnu-build-system)
+(arguments
+ `(#:tests? #f ;no "check" target
+ #:phases
+ (modify-phases %standard-phases
+ (add-after
+ 'unpack 'enter-dir
+ (lambda _
+(chdir "preseq-master")
+#t))
+ (add-after
+ 'unpack 'use-samtools-headers
+ (lambda _
+(substitute* '("smithlab_cpp/SAM.cpp"
+ "smithlab_cpp/SAM.hpp")
+ (("sam.h") "samtools/sam.h"))
+
Re: [PATCH] Add preseq.
Ricardo Wurmus skribis: > Ludovic Courtès writes: > >>> + (snippet >>> + ;; Remove bundled samtools. Leave the headers. >>> + '(for-each delete-file >>> + (find-files "preseq-master/samtools" >>> + "\\.c$") >> >> Why leave the headers? Isn’t it risky to leave headers that may be >> different from the Samtools that we’re actually linking against? > > The problem is with our Samtools package(s). They don't install > headers. I'd better update them to install the headers; then I could > just delete the complete "samtools" directory. Yes that would be best, if that’s fine with you. Ludo’.
Re: [PATCH] Add preseq.
Ludovic Courtès writes: >> + (snippet >> + ;; Remove bundled samtools. Leave the headers. >> + '(for-each delete-file >> + (find-files "preseq-master/samtools" >> + "\\.c$") > > Why leave the headers? Isn’t it risky to leave headers that may be > different from the Samtools that we’re actually linking against? The problem is with our Samtools package(s). They don't install headers. I'd better update them to install the headers; then I could just delete the complete "samtools" directory.
Re: [PATCH] Add preseq.
> Attached is a new version that includes support for BAM files, but > without using the bundled version of samtools. Two patches were > necessary that I have submitted upstream. > > I'm still testing this with a user, and I'm waiting for upstream to > comment on the patches, so I'm not in a hurry to push this. Upstream merged the patches, but there's no new release yet. See https://github.com/smithlabcode/preseq/pull/15. It seems to work fine. ~~ Ricardo
Re: [PATCH] Add preseq.
Ricardo Wurmus skribis: > From 3ffe6847848f6ba107e0757728f1c6a6e7807d56 Mon Sep 17 00:00:00 2001 > From: Ricardo Wurmus > Date: Tue, 23 Jun 2015 10:54:00 +0200 > Subject: [PATCH] gnu: Add preseq. > > * gnu/packages/bioinformatics.scm (preseq): New variable. [...] > + (snippet > + ;; Remove bundled samtools. Leave the headers. > + '(for-each delete-file > + (find-files "preseq-master/samtools" > + "\\.c$") Why leave the headers? Isn’t it risky to leave headers that may be different from the Samtools that we’re actually linking against? Otherwise LGTM. Thanks, Ludo’.
Re: [PATCH] Add preseq.
Attached is a new version that includes support for BAM files, but
without using the bundled version of samtools. Two patches were
necessary that I have submitted upstream.
I'm still testing this with a user, and I'm waiting for upstream to
comment on the patches, so I'm not in a hurry to push this.
~~ Ricardo
>From 3ffe6847848f6ba107e0757728f1c6a6e7807d56 Mon Sep 17 00:00:00 2001
From: Ricardo Wurmus
Date: Tue, 23 Jun 2015 10:54:00 +0200
Subject: [PATCH] gnu: Add preseq.
* gnu/packages/bioinformatics.scm (preseq): New variable.
---
gnu/packages/bioinformatics.scm| 51 ++
.../patches/preseq-1.0.2-install-to-PREFIX.patch | 37
.../patches/preseq-1.0.2-link-with-libbam.patch| 33 ++
3 files changed, 121 insertions(+)
create mode 100644 gnu/packages/patches/preseq-1.0.2-install-to-PREFIX.patch
create mode 100644 gnu/packages/patches/preseq-1.0.2-link-with-libbam.patch
diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index 4eb977b..d6fb32c 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -2033,6 +2033,57 @@ Databases are stored in a portable image within the file system, and can be
accessed/downloaded on demand across HTTP.")
(license license:public-domain)))
+(define-public preseq
+ (package
+(name "preseq")
+(version "1.0.2")
+(source (origin
+ (method url-fetch)
+ (uri
+ (string-append "http://smithlabresearch.org/downloads/preseq-";
+ version ".tar.bz2"))
+ (sha256
+ (base32 "0r7sw07p6nv8ygvc17gd78lisbw5336v3vhs86b5wv8mw3pwqksc"))
+ (patches (list (search-patch "preseq-1.0.2-install-to-PREFIX.patch")
+ (search-patch "preseq-1.0.2-link-with-libbam.patch")))
+ (modules '((guix build utils)))
+ (snippet
+ ;; Remove bundled samtools. Leave the headers.
+ '(for-each delete-file
+ (find-files "preseq-master/samtools"
+ "\\.c$")
+(build-system gnu-build-system)
+(arguments
+ `(#:tests? #f ;no "check" target
+ #:phases
+ (modify-phases %standard-phases
+ (add-after
+ 'unpack 'enter-dir
+ (lambda _
+(chdir "preseq-master")
+#t))
+ (delete 'configure))
+ #:make-flags (list (string-append "PREFIX="
+ (assoc-ref %outputs "out"))
+ (string-append "LIBBAM="
+ (assoc-ref %build-inputs "samtools")
+ "/lib/libbam.a"
+(inputs
+ `(("gsl" ,gsl)
+ ("samtools" ,samtools-0.1)
+ ("zlib" ,zlib)))
+(home-page "http://smithlabresearch.org/software/preseq/";)
+(synopsis "Program for analyzing library complexity")
+(description
+ "The preseq package is aimed at predicting and estimating the complexity
+of a genomic sequencing library, equivalent to predicting and estimating the
+number of redundant reads from a given sequencing depth and how many will be
+expected from additional sequencing using an initial sequencing experiment.
+The estimates can then be used to examine the utility of further sequencing,
+optimize the sequencing depth, or to screen multiple libraries to avoid low
+complexity samples.")
+(license license:gpl3+)))
+
(define-public sra-tools
(package
(name "sra-tools")
diff --git a/gnu/packages/patches/preseq-1.0.2-install-to-PREFIX.patch b/gnu/packages/patches/preseq-1.0.2-install-to-PREFIX.patch
new file mode 100644
index 000..f8318ae
--- /dev/null
+++ b/gnu/packages/patches/preseq-1.0.2-install-to-PREFIX.patch
@@ -0,0 +1,37 @@
+This patch has been submitted upstream: https://github.com/smithlabcode/preseq/pull/15
+
+From 65387b0d766e6c06a15cf8d8f9183d9a4ba644bb Mon Sep 17 00:00:00 2001
+From: Ricardo Wurmus
+Date: Thu, 25 Jun 2015 15:01:53 +0200
+Subject: [PATCH] Install to PREFIX
+
+---
+ Makefile | 8 ++--
+ 1 file changed, 6 insertions(+), 2 deletions(-)
+
+diff --git a/Makefile b/Makefile
+index 4d0367a..e334f2f 100644
+--- a/preseq-master/Makefile
b/preseq-master/Makefile
+@@ -22,6 +22,10 @@ ifndef ROOT
+ ROOT = $(shell pwd)
+ endif
+
++ifndef PREFIX
++PREFIX = $(ROOT)
++endif
++
+ ifndef SMITHLAB_CPP
+ SMITHLAB_CPP=$(ROOT)/smithlab_cpp/
+ endif
+@@ -91,8 +95,8 @@ endif
+ $(CXX) $(CXXFLAGS) -o $@ $^ $(INCLUDEARGS) $(LIBS)
+
+ install: $(PROGS)
+- @mkdir -p $(ROOT)/bin
+- @install -m 755 $(PROGS) $(ROOT)/bin
++ @mkdir -p $(PREFIX)/bin
++ @install -m 755 $(PROGS) $(PREFIX)/bin
+
+ clean:
+ @-rm -f $(PROGS) *.o *~
diff --git a/gnu/packages/patches/preseq-1.0.2-link-with-libbam.patch b/gnu/packages/patches/preseq-1.0.2-link-with-libbam.patch
new file mode 100644
index 000..035a3b1
--- /dev/
