Re: [PATCH 3/3] gnu: icedtea-6: Generate keystore.
On Sun, Jul 24, 2016 at 09:43:59AM +0200, Ricardo Wurmus wrote: > Thanks again for reporting this! Thanks for fixing it so quickly! 2 more packages out of the 700 that currently fail :-) Andreas
Re: [PATCH 3/3] gnu: icedtea-6: Generate keystore.
Andreas Engewrites: > On Sat, Jul 23, 2016 at 11:33:12PM +0200, Ricardo Wurmus wrote: >> You’re welcome to give it a try yourself! (Who knows, maybe this change >> would also allow us to reinstate the phase in the latest icedtea >> version?) > > Unfortunately it fails for me with the following error message: > > starting phase `install-keystore' > Importing certificate ACCVRAIZ1:2.8.94.195.183.166.67.127.164.224.pem > Backtrace: > In ice-9/boot-9.scm: > 157: 17 [catch #t # ...] > In unknown file: >?: 16 [apply-smob/1 #] > In ice-9/boot-9.scm: > 63: 15 [call-with-prompt prompt0 ...] > In ice-9/eval.scm: > 432: 14 [eval # #] > In ice-9/boot-9.scm: > 2401: 13 [save-module-excursion # ()>] > 4050: 12 [#] > 1724: 11 [%start-stack load-stack # ice-9/boot-9.scm:4041:10 ()>] > 1729: 10 [#] > In unknown file: >?: 9 [primitive-load > "/gnu/store/i0322cxlmymrnxrlsnplh4894fzjbj9h-icedtea-1.13.11-guile-builder"] > In ice-9/eval.scm: > 387: 8 [eval # ()] > In srfi/srfi-1.scm: > 830: 7 [every1 # /gnu/store/vs75q74qsfr3h45vkbr559p5w6fn4mw2-module-import/guix/build/gnu-build-system.scm:589:9 > (expr)> ...] > In > /gnu/store/vs75q74qsfr3h45vkbr559p5w6fn4mw2-module-import/guix/build/gnu-build-system.scm: > 593: 6 [# /gnu/store/vs75q74qsfr3h45vkbr559p5w6fn4mw2-module-import/guix/build/gnu-build-system.scm:589:9 > (expr)> #] > In ice-9/eval.scm: > 432: 5 [eval # #] > In ice-9/boot-9.scm: > 768: 4 [for-each # #] > In ice-9/eval.scm: > 432: 3 [eval # #] > In ice-9/boot-9.scm: > 867: 2 [call-with-input-file > "/gnu/store/lp7s9x1llgw1rc675yvslxsnpcyy05ld-nss-certs-3.23/etc/ssl/certs/ACCVRAIZ1:2.8.94.195.183.166.67.127.164.224.pem" > ...] > 883: 1 [call-with-output-file # ...] > In unknown file: >?: 0 [open-file # "w" #:encoding #f] > > ERROR: In procedure open-file: > ERROR: Wrong type (expecting string): # Oh, “(tmpfile)” returns a port. Changed to a string (it can be the same temp file for each certificate) rebuilt all three icedtea packages successfully and pushed to master as 6af691723ed6c70fc468768e1e07b19b27c6f4d8. Thanks again for reporting this! ~~ Ricardo
Re: [PATCH 3/3] gnu: icedtea-6: Generate keystore.
On Sat, Jul 23, 2016 at 11:33:12PM +0200, Ricardo Wurmus wrote: > You’re welcome to give it a try yourself! (Who knows, maybe this change > would also allow us to reinstate the phase in the latest icedtea > version?) Unfortunately it fails for me with the following error message: starting phase `install-keystore' Importing certificate ACCVRAIZ1:2.8.94.195.183.166.67.127.164.224.pem Backtrace: In ice-9/boot-9.scm: 157: 17 [catch #t # ...] In unknown file: ?: 16 [apply-smob/1 #] In ice-9/boot-9.scm: 63: 15 [call-with-prompt prompt0 ...] In ice-9/eval.scm: 432: 14 [eval # #] In ice-9/boot-9.scm: 2401: 13 [save-module-excursion #] 4050: 12 [#] 1724: 11 [%start-stack load-stack #] 1729: 10 [#] In unknown file: ?: 9 [primitive-load "/gnu/store/i0322cxlmymrnxrlsnplh4894fzjbj9h-icedtea-1.13.11-guile-builder"] In ice-9/eval.scm: 387: 8 [eval # ()] In srfi/srfi-1.scm: 830: 7 [every1 # ...] In /gnu/store/vs75q74qsfr3h45vkbr559p5w6fn4mw2-module-import/guix/build/gnu-build-system.scm: 593: 6 [# #] In ice-9/eval.scm: 432: 5 [eval # #] In ice-9/boot-9.scm: 768: 4 [for-each # #] In ice-9/eval.scm: 432: 3 [eval # #] In ice-9/boot-9.scm: 867: 2 [call-with-input-file "/gnu/store/lp7s9x1llgw1rc675yvslxsnpcyy05ld-nss-certs-3.23/etc/ssl/certs/ACCVRAIZ1:2.8.94.195.183.166.67.127.164.224.pem" ...] 883: 1 [call-with-output-file # ...] In unknown file: ?: 0 [open-file # "w" #:encoding #f] ERROR: In procedure open-file: ERROR: Wrong type (expecting string): # Andreas
Re: [PATCH 3/3] gnu: icedtea-6: Generate keystore.
Ricardo Wurmuswrites: > Andreas Enge writes: > >> Hello, Ricardo! >> >> Icedtea@1 in master now fails to build in the install-keystore phase. >>http://hydra.gnu.org:3000/build/1309224 >>http://hydra.gnu.org:3000/build/1308950 >> Could you have a look, please? […] > The keytool from icedtea@1 doesn’t like this certificate. My hunch is > that we may need to remove comments from the certificate files, only > leaving the certificate block. > > I’ll fix this as soon as I can. Attached is an untested patch to fix this. I’m now building icedtea@1 again with this patch (on a remote machine). Not sure when I can check on the result as I’ll be out for the most part of tomorrow. You’re welcome to give it a try yourself! (Who knows, maybe this change would also allow us to reinstate the phase in the latest icedtea version?) ~~ Ricardo >From 04cafa35d7e226843cdccaf5a3ea5a82d9dc5d3e Mon Sep 17 00:00:00 2001 From: Ricardo Wurmus Date: Sat, 23 Jul 2016 23:25:11 +0200 Subject: [PATCH] gnu: icedtea-6: Narrow file to certificate block. * gnu/packages/java.scm (icedtea-6)[arguments]: Extract certificate blocks from pem files before importing. --- gnu/packages/java.scm | 39 ++- 1 file changed, 30 insertions(+), 9 deletions(-) diff --git a/gnu/packages/java.scm b/gnu/packages/java.scm index 2d50ad8..78e2143 100644 --- a/gnu/packages/java.scm +++ b/gnu/packages/java.scm @@ -535,17 +535,38 @@ build process and its dependencies, whereas Make uses Makefile format.") "/etc/ssl/certs")) (keytool (string-append (assoc-ref outputs "jdk") "/bin/keytool"))) + (define (extract-cert file target) + (call-with-input-file file + (lambda (in) + (call-with-output-file target + (lambda (out) + (let loop ((line (read-line in 'concat)) +(copying? #f)) + (cond +((eof-object? line) #t) +((string-prefix? "-BEGIN" line) + (display line out) + (loop (read-line in 'concat) #t)) +((string-prefix? "-END" line) + (display line out) + #t) +(else + (when copying? (display line out)) + (loop (read-line in 'concat) copying?) (define (import-cert cert) (format #t "Importing certificate ~a\n" (basename cert)) - (let* ((port (open-pipe* OPEN_WRITE keytool - "-import" - "-alias" (basename cert) - "-keystore" keystore - "-storepass" "changeit" - "-file" cert))) - (display "yes\n" port) - (when (not (zero? (status:exit-val (close-pipe port - (error "failed to import" cert + (let ((temp (tmpfile))) + (extract-cert cert temp) + (let ((port (open-pipe* OPEN_WRITE keytool + "-import" + "-alias" (basename cert) + "-keystore" keystore + "-storepass" "changeit" + "-file" temp))) + (display "yes\n" port) + (when (not (zero? (status:exit-val (close-pipe port + (error "failed to import" cert))) + (delete-file temp))) ;; This is necessary because the certificate directory contains ;; files with non-ASCII characters in their names. -- 2.9.0
Re: [PATCH 3/3] gnu: icedtea-6: Generate keystore.
Andreas Engewrites: > Hello, Ricardo! > > Icedtea@1 in master now fails to build in the install-keystore phase. >http://hydra.gnu.org:3000/build/1309224 >http://hydra.gnu.org:3000/build/1308950 > Could you have a look, please? Hmm, that’s strange. I ran “guix build icedtea” after removing the validation filter and built out all three versions of icedtea before pushing this. I don’t have the very same version of the “keytool” binary on my machine right now (with the very same version of nss-certs as on hydra), but in principle this works without errors: /gnu/store/r63vag0814nz79xr9g2ph6fvhq5xp2f3-icedtea-2.6.6/bin/keytool \ -import \ -alias ACCVRAIZ1:2.8.94.195.183.166.67.127.164.224.pem \ -keystore /tmp/keystore \ -storepass changeit \ -file /gnu/store/lp7s9x1llgw1rc675yvslxsnpcyy05ld-nss-certs-3.23/etc/ssl/certs/ACCVRAIZ1:2.8.94.195.183.166.67.127.164.224.pem … Trust this certificate? [no]: yes Certificate was added to keystore The pem file looks like a valid X.509 certificate to me. I cannot build icedtea@1 on my machine right now as I’m traveling, but I just started a build remotely on my workstation in the office and it failed. I used to have an additional stripping phase that I removed at some point. As I continued to refine the new phase I must have used the cached build of icedtea@1 without ever rebuilding it. Sorry! The keytool from icedtea@1 doesn’t like this certificate. My hunch is that we may need to remove comments from the certificate files, only leaving the certificate block. I’ll fix this as soon as I can. ~~ Ricardo
Re: [PATCH 3/3] gnu: icedtea-6: Generate keystore.
Hello, Ricardo! Icedtea@1 in master now fails to build in the install-keystore phase. http://hydra.gnu.org:3000/build/1309224 http://hydra.gnu.org:3000/build/1308950 Could you have a look, please? Andreas
Re: [PATCH 3/3] gnu: icedtea-6: Generate keystore.
Ludovic Courtèswrites: > Ricardo Wurmus skribis: > >> From: Ricardo Wurmus >> >> * gnu/packages/java.scm (icedtea-6)[arguments]: Add phase >> "install-keystore". >> [native-inputs]: Add nss-certs and openssl. > > [...] > >> + (add-after 'install 'install-keystore >> + (lambda* (#:key inputs outputs #:allow-other-keys) > > Could you add a comment to explain what’s going on here? > > Too bad IceTea’s build system doesn’t take care of that. > >> + (let* ((keystore "cacerts") >> +(certs-dir (string-append (assoc-ref inputs "nss-certs") >> + "/etc/ssl/certs")) >> +(keytool (string-append (assoc-ref outputs "jdk") >> + "/bin/keytool")) >> +(openssl (which "openssl")) >> +(recent(date->time-utc (string->date "2016-1-1" >> + "~Y-~m-~d" >> + (define (valid? cert) >> + (let* ((port (open-pipe* OPEN_READ openssl >> + "x509" "-enddate" "-in" cert >> "-noout")) >> +(str (read-line port)) >> +(end (begin (close-pipe port) >> + ;; TODO: use match? >> + (cadr (string-split str #\=) > > Why not use ‘match’, indeed. :-) No big deal though. > >> + (time>? (date->time-utc >> +(string->date end "~b ~d ~H:~M:~S ~Y")) >> recent))) >> + >> + (define (import-cert cert) >> + (format #t "Importing certificate ~a\n" (basename cert)) >> + (let* ((port (open-pipe* OPEN_WRITE keytool >> + "-import" >> + "-alias" (basename cert) >> + "-keystore" keystore >> + "-storepass" "changeit" >> + "-file" cert))) >> + (display "yes\n" port) >> + (when (not (eqv? 0 (status:exit-val (close-pipe port > > Maybe (zero? (status:exit-val …)). > >> + (format (current-error-port) >> + "Failed to import certificate.\n" > > Rather (error "failed to import" cert) so the process stops here. > >> + ;; This is necessary because the certificate directory >> contains >> + ;; files with non-ASCII characters in their names. >> + (setlocale LC_ALL "en_US.utf8") >> + (setenv "LC_ALL" "en_US.utf8") >> + >> + (for-each import-cert >> + (filter valid? (find-files certs-dir "\\.pem$"))) > > Why do we need to filter out invalid certificates? > > The problem I see is that the result of ‘valid?’, and thus the output of > the build process, depends on the build time, which isn’t great. > > I would prefer to unconditionally install all the certificates if that > doesn’t cause any problems. WDYT? I removed the validation (because I expect certs to be validated at runtime). I also added a comment explaining why this is needed and made the suggested changes. (I pushed from my workstation without signing key, because I forgot that I normally push from my laptop. Sorry, won’t happen again! Key replacement is on my list, and then I’ll get myself a subkey for the office workstation.) Thanks for the review! ~~ Ricardo
Re: [PATCH 3/3] gnu: icedtea-6: Generate keystore.
Ludovic Courtèswrites: > Ricardo Wurmus skribis: > >> From: Ricardo Wurmus >> >> * gnu/packages/java.scm (icedtea-6)[arguments]: Add phase >> "install-keystore". >> [native-inputs]: Add nss-certs and openssl. > > [...] > >> + (add-after 'install 'install-keystore >> + (lambda* (#:key inputs outputs #:allow-other-keys) > > Could you add a comment to explain what’s going on here? Okay, I’ll add a comment. > Too bad IceTea’s build system doesn’t take care of that. Yeah, there is an old bug report about this, but its resolution has been pushed to later releases repeatedly. All distributions have their own bash scripts to generate a keystore. >> + (let* ((keystore "cacerts") >> +(certs-dir (string-append (assoc-ref inputs "nss-certs") >> + "/etc/ssl/certs")) >> +(keytool (string-append (assoc-ref outputs "jdk") >> + "/bin/keytool")) >> +(openssl (which "openssl")) >> +(recent(date->time-utc (string->date "2016-1-1" >> + "~Y-~m-~d" >> + (define (valid? cert) >> + (let* ((port (open-pipe* OPEN_READ openssl >> + "x509" "-enddate" "-in" cert >> "-noout")) >> +(str (read-line port)) >> +(end (begin (close-pipe port) >> + ;; TODO: use match? >> + (cadr (string-split str #\=) > > Why not use ‘match’, indeed. :-) No big deal though. > >> + (time>? (date->time-utc >> +(string->date end "~b ~d ~H:~M:~S ~Y")) >> recent))) >> + >> + (define (import-cert cert) >> + (format #t "Importing certificate ~a\n" (basename cert)) >> + (let* ((port (open-pipe* OPEN_WRITE keytool >> + "-import" >> + "-alias" (basename cert) >> + "-keystore" keystore >> + "-storepass" "changeit" >> + "-file" cert))) >> + (display "yes\n" port) >> + (when (not (eqv? 0 (status:exit-val (close-pipe port > > Maybe (zero? (status:exit-val …)). Okay. >> + (format (current-error-port) >> + "Failed to import certificate.\n" > > Rather (error "failed to import" cert) so the process stops here. Yes, that’s better. I changed this for testing purposes and forgot to change it back. >> + ;; This is necessary because the certificate directory >> contains >> + ;; files with non-ASCII characters in their names. >> + (setlocale LC_ALL "en_US.utf8") >> + (setenv "LC_ALL" "en_US.utf8") >> + >> + (for-each import-cert >> + (filter valid? (find-files certs-dir "\\.pem$"))) > > Why do we need to filter out invalid certificates? > > The problem I see is that the result of ‘valid?’, and thus the output of > the build process, depends on the build time, which isn’t great. It actually depends on the arbitrary value of “recent”, which I set to 2016-1-1, but I must admit that I don’t know if we really must filter out invalid certs at all. I don’t know if it is a problem if invalid certs are part of the keystore. Maybe it’s not an issue. ~~ Ricardo
Re: [PATCH 3/3] gnu: icedtea-6: Generate keystore.
Ricardo Wurmusskribis: > From: Ricardo Wurmus > > * gnu/packages/java.scm (icedtea-6)[arguments]: Add phase > "install-keystore". > [native-inputs]: Add nss-certs and openssl. [...] > + (add-after 'install 'install-keystore > + (lambda* (#:key inputs outputs #:allow-other-keys) Could you add a comment to explain what’s going on here? Too bad IceTea’s build system doesn’t take care of that. > + (let* ((keystore "cacerts") > +(certs-dir (string-append (assoc-ref inputs "nss-certs") > + "/etc/ssl/certs")) > +(keytool (string-append (assoc-ref outputs "jdk") > + "/bin/keytool")) > +(openssl (which "openssl")) > +(recent(date->time-utc (string->date "2016-1-1" > + "~Y-~m-~d" > + (define (valid? cert) > + (let* ((port (open-pipe* OPEN_READ openssl > + "x509" "-enddate" "-in" cert > "-noout")) > +(str (read-line port)) > +(end (begin (close-pipe port) > + ;; TODO: use match? > + (cadr (string-split str #\=) Why not use ‘match’, indeed. :-) No big deal though. > + (time>? (date->time-utc > +(string->date end "~b ~d ~H:~M:~S ~Y")) recent))) > + > + (define (import-cert cert) > + (format #t "Importing certificate ~a\n" (basename cert)) > + (let* ((port (open-pipe* OPEN_WRITE keytool > + "-import" > + "-alias" (basename cert) > + "-keystore" keystore > + "-storepass" "changeit" > + "-file" cert))) > + (display "yes\n" port) > + (when (not (eqv? 0 (status:exit-val (close-pipe port Maybe (zero? (status:exit-val …)). > + (format (current-error-port) > + "Failed to import certificate.\n" Rather (error "failed to import" cert) so the process stops here. > + ;; This is necessary because the certificate directory > contains > + ;; files with non-ASCII characters in their names. > + (setlocale LC_ALL "en_US.utf8") > + (setenv "LC_ALL" "en_US.utf8") > + > + (for-each import-cert > + (filter valid? (find-files certs-dir "\\.pem$"))) Why do we need to filter out invalid certificates? The problem I see is that the result of ‘valid?’, and thus the output of the build process, depends on the build time, which isn’t great. I would prefer to unconditionally install all the certificates if that doesn’t cause any problems. WDYT? Thank you for working on it! Ludo’.
[PATCH 3/3] gnu: icedtea-6: Generate keystore.
From: Ricardo Wurmus* gnu/packages/java.scm (icedtea-6)[arguments]: Add phase "install-keystore". [native-inputs]: Add nss-certs and openssl. --- gnu/packages/java.scm | 63 +-- 1 file changed, 61 insertions(+), 2 deletions(-) diff --git a/gnu/packages/java.scm b/gnu/packages/java.scm index faa6e5b..95e273e 100644 --- a/gnu/packages/java.scm +++ b/gnu/packages/java.scm @@ -30,6 +30,7 @@ #:use-module (gnu packages autotools) #:use-module (gnu packages base) #:use-module (gnu packages bash) + #:use-module (gnu packages certs) #:use-module (gnu packages cpio) #:use-module (gnu packages cups) #:use-module (gnu packages compression) @@ -47,6 +48,7 @@ #:use-module (gnu packages pkg-config) #:use-module (gnu packages perl) #:use-module (gnu packages mit-krb5) + #:use-module (gnu packages tls) #:use-module (gnu packages xml) #:use-module (gnu packages xorg) #:use-module (gnu packages zip) @@ -262,7 +264,8 @@ build process and its dependencies, whereas Make uses Makefile format.") #:modules ((guix build utils) (guix build gnu-build-system) (ice-9 popen) - (ice-9 rdelim)) + (ice-9 rdelim) + (srfi srfi-19)) #:configure-flags (let* ((gcjdir (assoc-ref %build-inputs "gcj")) @@ -521,7 +524,58 @@ build process and its dependencies, whereas Make uses Makefile format.") (jdk (assoc-ref outputs "jdk"))) (copy-recursively "openjdk.build/docs" doc) (copy-recursively "openjdk.build/j2re-image" jre) - (copy-recursively "openjdk.build/j2sdk-image" jdk))) + (copy-recursively "openjdk.build/j2sdk-image" jdk + (add-after 'install 'install-keystore + (lambda* (#:key inputs outputs #:allow-other-keys) + (let* ((keystore "cacerts") +(certs-dir (string-append (assoc-ref inputs "nss-certs") + "/etc/ssl/certs")) +(keytool (string-append (assoc-ref outputs "jdk") + "/bin/keytool")) +(openssl (which "openssl")) +(recent(date->time-utc (string->date "2016-1-1" + "~Y-~m-~d" + (define (valid? cert) + (let* ((port (open-pipe* OPEN_READ openssl + "x509" "-enddate" "-in" cert "-noout")) +(str (read-line port)) +(end (begin (close-pipe port) + ;; TODO: use match? + (cadr (string-split str #\=) + (time>? (date->time-utc +(string->date end "~b ~d ~H:~M:~S ~Y")) recent))) + + (define (import-cert cert) + (format #t "Importing certificate ~a\n" (basename cert)) + (let* ((port (open-pipe* OPEN_WRITE keytool + "-import" + "-alias" (basename cert) + "-keystore" keystore + "-storepass" "changeit" + "-file" cert))) + (display "yes\n" port) + (when (not (eqv? 0 (status:exit-val (close-pipe port + (format (current-error-port) + "Failed to import certificate.\n" + + ;; This is necessary because the certificate directory contains + ;; files with non-ASCII characters in their names. + (setlocale LC_ALL "en_US.utf8") + (setenv "LC_ALL" "en_US.utf8") + + (for-each import-cert + (filter valid? (find-files certs-dir "\\.pem$"))) + (mkdir-p (string-append (assoc-ref outputs "out") + "/lib/security")) + (mkdir-p (string-append (assoc-ref outputs "jdk") + "/jre/lib/security")) + (install-file keystore + (string-append (assoc-ref outputs "out") +"/lib/security")) + (install-file keystore + (string-append (assoc-ref outputs "jdk") +"/jre/lib/security")) + #t)) (native-inputs `(("ant" ,ant) ("alsa-lib" ,alsa-lib) @@ -544,6 +598,7 @@ build process and its dependencies, whereas Make uses Makefile format.") ("libxslt" ,libxslt) ;for xsltproc ("mit-krb5" ,mit-krb5) ("nss" ,nss) +