bug#27157: [PATCH] Re: bug#27157: Building Guile 2.2 "times out"
Hi Maxim, Maxim Cournoyer skribis: > From ba078ca1b8efa17fb0b2ee7b4269611c6bb4c3d6 Mon Sep 17 00:00:00 2001 > From: Maxim Cournoyer > Date: Wed, 30 Aug 2017 23:12:12 -0400 > Subject: [PATCH] ui: Remove max-silent-time from the default options. > > Having a finite default `max-silent-time' value for scripts such as 'guix > environment' could lead to timeouts when building subtitutes; this was > undesirable. This fixes bug #27157. > > * guix/scripts/archive.scm (%default-options): Remove max-silent-time entry. > * guix/scripts/copy.scm (%default-options): Likewise. > * guix/scripts/environment.scm (%default-options): Likewise. > * guix/scripts/pack.scm (%default-options): Likewise. > * guix/scripts/package.scm (%default-options): Likewise. > * guix/scripts/pull.scm (%default-options): Likewise. > * guix/scripts/system.scm (%default-options): Likewise. Good catch! It was actually a bug, because the doc said that the daemon’s setting should be honored by default, which was obviously not the case. Pushed: https://git.savannah.gnu.org/cgit/guix.git/commit/?id=fe55199542590d16aa6ca6a8bfa0fda27903db5c Thanks, Ludo’.
bug#27157: [PATCH] Re: bug#27157: Building Guile 2.2 "times out"
Hi, Here's a patch that fixes the problem originally reported (guile was timing out while being built by 'guix environment guix', which was capped to 3600 s of silent time. I think the bug can now be closed. Maxim >From ba078ca1b8efa17fb0b2ee7b4269611c6bb4c3d6 Mon Sep 17 00:00:00 2001 From: Maxim Cournoyer Date: Wed, 30 Aug 2017 23:12:12 -0400 Subject: [PATCH] ui: Remove max-silent-time from the default options. Having a finite default `max-silent-time' value for scripts such as 'guix environment' could lead to timeouts when building subtitutes; this was undesirable. This fixes bug #27157. * guix/scripts/archive.scm (%default-options): Remove max-silent-time entry. * guix/scripts/copy.scm (%default-options): Likewise. * guix/scripts/environment.scm (%default-options): Likewise. * guix/scripts/pack.scm (%default-options): Likewise. * guix/scripts/package.scm (%default-options): Likewise. * guix/scripts/pull.scm (%default-options): Likewise. * guix/scripts/system.scm (%default-options): Likewise. --- guix/scripts/archive.scm | 1 - guix/scripts/copy.scm| 1 - guix/scripts/environment.scm | 1 - guix/scripts/pack.scm| 1 - guix/scripts/package.scm | 3 +-- guix/scripts/pull.scm| 1 - guix/scripts/system.scm | 1 - 7 files changed, 1 insertion(+), 8 deletions(-) diff --git a/guix/scripts/archive.scm b/guix/scripts/archive.scm index 5ea19784d..a569848ae 100644 --- a/guix/scripts/archive.scm +++ b/guix/scripts/archive.scm @@ -54,7 +54,6 @@ `((system . ,(%current-system)) (substitutes? . #t) (graft? . #t) -(max-silent-time . 3600) (verbosity . 0))) (define (show-help) diff --git a/guix/scripts/copy.scm b/guix/scripts/copy.scm index 32438b99d..9e8cc 100644 --- a/guix/scripts/copy.scm +++ b/guix/scripts/copy.scm @@ -151,7 +151,6 @@ Copy ITEMS to or from the specified host over SSH.\n")) `((system . ,(%current-system)) (substitutes? . #t) (graft? . #t) -(max-silent-time . 3600) (verbosity . 0))) diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scm index 95ba199d9..0d6921833 100644 --- a/guix/scripts/environment.scm +++ b/guix/scripts/environment.scm @@ -179,7 +179,6 @@ COMMAND or an interactive shell in that environment.\n")) `((system . ,(%current-system)) (substitutes? . #t) (graft? . #t) -(max-silent-time . 3600) (verbosity . 0))) (define (tag-package-arg opts arg) diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm index c269a1fef..21fea446a 100644 --- a/guix/scripts/pack.scm +++ b/guix/scripts/pack.scm @@ -263,7 +263,6 @@ the image." (system . ,(%current-system)) (substitutes? . #t) (graft? . #t) -(max-silent-time . 3600) (verbosity . 0) (symlinks . ()) (compressor . ,(first %compressors diff --git a/guix/scripts/package.scm b/guix/scripts/package.scm index fa45bd48a..9ec6950c4 100644 --- a/guix/scripts/package.scm +++ b/guix/scripts/package.scm @@ -377,8 +377,7 @@ ENTRIES, a list of manifest entries, in the context of PROFILE." (define %default-options ;; Alist of default option values. - `((max-silent-time . 3600) -(verbosity . 0) + `((verbosity . 0) (graft? . #t) (substitutes? . #t))) diff --git a/guix/scripts/pull.scm b/guix/scripts/pull.scm index a1deec804..b1c87c870 100644 --- a/guix/scripts/pull.scm +++ b/guix/scripts/pull.scm @@ -96,7 +96,6 @@ Install it by running: (system . ,(%current-system)) (substitutes? . #t) (graft? . #t) -(max-silent-time . 3600) (verbosity . 0))) (define (show-help) diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm index 8793c4092..773779318 100644 --- a/guix/scripts/system.scm +++ b/guix/scripts/system.scm @@ -864,7 +864,6 @@ Some ACTIONS support additional ARGS.\n")) (substitutes? . #t) (graft? . #t) (build-hook? . #t) -(max-silent-time . 3600) (verbosity . 0) (file-system-type . "ext4") (image-size . guess) -- 2.13.1
bug#27157: Building Guile 2.2 "times out"
Hi Mark, > I pushed this as commit 1577a65809f8ef1883d18cf81095f632b65c042a. > I'm not sure whether to close this bug or not. Can someone else do so > if appropriate? > > Mark > I wanted to give another shot at removing the the default max-silent-time options in our scripts, so that everything can be consistent, but haven't got around it yet. I believe we shouldn't close this bug yet as it is still possible to get into the problem by running "guix environment", for example, which has a default max-silent-time value set to 1 hour. Feel free to beat me to it if you have the bandwith. Maxim
bug#27157: Building Guile 2.2 "times out"
Ricardo Wurmus writes: > Mark H Weaver writes: > >> I think we should further increase the 'max-silent-time' value for >> Guile-2.2. I've had to restart that build several times on ARM. 3 >> hours might be enough on a machine that's not doing anything else, but >> our build slaves are often building multiple packages. >> >> I would increase it to 6 hours. What do you think? > > I have no objections. We should assume busy build machines. It’s > unfortunate to have no output whatever for up to six hours :-/ I pushed this as commit 1577a65809f8ef1883d18cf81095f632b65c042a. I'm not sure whether to close this bug or not. Can someone else do so if appropriate? Mark
bug#27157: [PATCH] Re: bug#27157: Building Guile 2.2 "times out"
Maxim Cournoyer skribis: > l...@gnu.org (Ludovic Courtès) writes: > > [...] > At any rate, it would be good to remove ‘max-silent-time’ from all the ‘%default-options’ variables, for consistency. Ludo’. >>> >>> I agree! In my view, interactive user commands should expect the users >>> to take action themselves (e.g.; user notice no output for 3 hours; hits >>> C-c. Or, user notice no output for 3 hours, check the process in top, >>> sees it's still doing something, decides to leave it running a couple >>> more hours). >> >> Yes. Would you like to prepare a patch to remove ‘max-silent-time’ from >> all the ‘%default-options’? > > I gave this a quick try, but got confused upon encountering the > guix-offload script; it has a max-silent-time option, but doesn't build > anything, it simply copies stuff IIUC. > > Should we remove it there as well? No: ‘guix offload’ gets max-silent-time systematically from its command-line arguments (it is invoked by guix-daemon). So this one can remain as-is. Ludo’.
bug#27157: [PATCH] Re: bug#27157: Building Guile 2.2 "times out"
l...@gnu.org (Ludovic Courtès) writes: [...] >>> At any rate, it would be good to remove ‘max-silent-time’ from all the >>> ‘%default-options’ variables, for consistency. >>> >>> Ludo’. >> >> I agree! In my view, interactive user commands should expect the users >> to take action themselves (e.g.; user notice no output for 3 hours; hits >> C-c. Or, user notice no output for 3 hours, check the process in top, >> sees it's still doing something, decides to leave it running a couple >> more hours). > > Yes. Would you like to prepare a patch to remove ‘max-silent-time’ from > all the ‘%default-options’? I gave this a quick try, but got confused upon encountering the guix-offload script; it has a max-silent-time option, but doesn't build anything, it simply copies stuff IIUC. Should we remove it there as well? Maxim
bug#27157: [PATCH] Re: bug#27157: Building Guile 2.2 "times out"
Maxim Cournoyer skribis: > l...@gnu.org (Ludovic Courtès) writes: > >> Maxim Cournoyer skribis: >> >>> Guix build doesn't come with default option value for the >>> max-silent-time, but guix environment does, and it is set to 3600 >>> s. This is defined under (guix scripts environment): >>> >>> (define %default-options >>> `((system . ,(%current-system)) >>> (substitutes? . #t) >>> (graft? . #t) >>> (max-silent-time . 3600) >>> (verbosity . 0))) >>> >>> If this is by design, I guess this issue can be closed. >> >> Good catch! >> >> However, according to ‘set-build-options’ in (guix store), >> #:max-silent-time #f is equivalent to #:max-silent-time 3600. > > IIUC, this should only be true when (< (nix-server-minor-version > server) #x61), as can be seen in (guix store): > > (when (< (nix-server-minor-version server) #x61) > (let ((max-build-jobs (or max-build-jobs 1)) > (xomax-silent-time (or max-silent-time 3600))) > (send (integer max-build-jobs) (integer max-silent-time Oooh, indeed, sorry for the confusion. So what happens when #:max-silent-time is #f is that we use the daemon’s default value, which is infinite (zero) by default (see libstore/globals.cc). > Given that the the nix worker protocole PROTOCOL_VERSION was bumped to > 0x161 in commit deac976d3d26c7b85b9c90efb424b0aa94f1027c last January, I > would *not* expect this condition to be evaluated. > > By the way, I find it weird that these conditions be expressed in terms > of the minor version number only. The correctness of those is thus only > guaranteed as long as the major number version is not bumped; this seems > unwise. Historical baggage: the major number was never bumped, I think. :-) You’re right though. >> At any rate, it would be good to remove ‘max-silent-time’ from all the >> ‘%default-options’ variables, for consistency. >> >> Ludo’. > > I agree! In my view, interactive user commands should expect the users > to take action themselves (e.g.; user notice no output for 3 hours; hits > C-c. Or, user notice no output for 3 hours, check the process in top, > sees it's still doing something, decides to leave it running a couple > more hours). Yes. Would you like to prepare a patch to remove ‘max-silent-time’ from all the ‘%default-options’? In the meantime I’ll add --timeout and --max-silent-time as options to guix-daemon. Thanks! Ludo’.
bug#27157: [PATCH] Re: bug#27157: Building Guile 2.2 "times out"
l...@gnu.org (Ludovic Courtès) writes:
> Maxim Cournoyer skribis:
>
>> Guix build doesn't come with default option value for the
>> max-silent-time, but guix environment does, and it is set to 3600
>> s. This is defined under (guix scripts environment):
>>
>> (define %default-options
>> `((system . ,(%current-system))
>> (substitutes? . #t)
>> (graft? . #t)
>> (max-silent-time . 3600)
>> (verbosity . 0)))
>>
>> If this is by design, I guess this issue can be closed.
>
> Good catch!
>
> However, according to ‘set-build-options’ in (guix store),
> #:max-silent-time #f is equivalent to #:max-silent-time 3600.
IIUC, this should only be true when (< (nix-server-minor-version
server) #x61), as can be seen in (guix store):
(when (< (nix-server-minor-version server) #x61)
(let ((max-build-jobs (or max-build-jobs 1))
(xomax-silent-time (or max-silent-time 3600)))
(send (integer max-build-jobs) (integer max-silent-time
Which seems to be matched with the following in nix-daemon.cc:
if (GET_PROTOCOL_MINOR(clientVersion) < 0x61) {
settings.set("build-max-jobs", std::to_string(readInt(from)));
settings.set("build-max-silent-time",
std::to_string(readInt(from)));
}
Given that the the nix worker protocole PROTOCOL_VERSION was bumped to
0x161 in commit deac976d3d26c7b85b9c90efb424b0aa94f1027c last January, I
would *not* expect this condition to be evaluated.
By the way, I find it weird that these conditions be expressed in terms
of the minor version number only. The correctness of those is thus only
guaranteed as long as the major number version is not bumped; this seems
unwise.
> Perhaps the build remains silent for just about an hour and you were
> lucky on your second try?
See explanation above. I don't think this .
> At any rate, it would be good to remove ‘max-silent-time’ from all the
> ‘%default-options’ variables, for consistency.
>
> Ludo’.
I agree! In my view, interactive user commands should expect the users
to take action themselves (e.g.; user notice no output for 3 hours; hits
C-c. Or, user notice no output for 3 hours, check the process in top,
sees it's still doing something, decides to leave it running a couple
more hours).
Maxim
bug#27157: Building Guile 2.2 "times out"
Mark H Weaver writes: > I think we should further increase the 'max-silent-time' value for > Guile-2.2. I've had to restart that build several times on ARM. 3 > hours might be enough on a machine that's not doing anything else, but > our build slaves are often building multiple packages. > > I would increase it to 6 hours. What do you think? I have no objections. We should assume busy build machines. It’s unfortunate to have no output whatever for up to six hours :-/ -- Ricardo GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC https://elephly.net
bug#27157: Building Guile 2.2 "times out"
Ricardo Wurmus writes: > Hi Maxim, > >> I'm not sure where this timeout value can be configured, but clearly it >> shouldn't be 1 hour since 2 cores of an i5 intel class processor can't >> manage to build it under 1 hour. > > for the guile-2.2 package I see that this is part of the package > definition: > > (properties '((timeout . 72000) ;20 hours > (max-silent-time . 10800))) ;3 hours (needed on ARM) I think we should further increase the 'max-silent-time' value for Guile-2.2. I've had to restart that build several times on ARM. 3 hours might be enough on a machine that's not doing anything else, but our build slaves are often building multiple packages. I would increase it to 6 hours. What do you think? Mark
bug#27157: Building Guile 2.2 "times out"
Maxim Cournoyer skribis: > Guix build doesn't come with default option value for the > max-silent-time, but guix environment does, and it is set to 3600 > s. This is defined under (guix scripts environment): > > (define %default-options > `((system . ,(%current-system)) > (substitutes? . #t) > (graft? . #t) > (max-silent-time . 3600) > (verbosity . 0))) > > If this is by design, I guess this issue can be closed. Good catch! However, according to ‘set-build-options’ in (guix store), #:max-silent-time #f is equivalent to #:max-silent-time 3600. Perhaps the build remains silent for just about an hour and you were lucky on your second try? At any rate, it would be good to remove ‘max-silent-time’ from all the ‘%default-options’ variables, for consistency. Ludo’.
bug#27157: Building Guile 2.2 "times out"
Hi Ludo and Ricardor, l...@gnu.org (Ludovic Courtès) writes: > Hello, > > Ricardo Wurmus skribis: > >>> I'm not sure where this timeout value can be configured, but clearly it >>> shouldn't be 1 hour since 2 cores of an i5 intel class processor can't >>> manage to build it under 1 hour. >> >> for the guile-2.2 package I see that this is part of the package >> definition: >> >> (properties '((timeout . 72000) ;20 hours >> (max-silent-time . 10800))) ;3 hours (needed on ARM) >> >> We do this so that we can build it on Hydra. >> You can override the max silent time: >> >> guix build --max-silent-time=999 guile >> >> Does “guix build” not respect these build properties? > > It does not. Only build-aux/hydra/gnu-system.scm does. Apparently the problem was due to the build action being triggered as part of a "guix environment" command. I could launch the command using "guix build guile@2.2" and it completed successfully. Guix build doesn't come with default option value for the max-silent-time, but guix environment does, and it is set to 3600 s. This is defined under (guix scripts environment): (define %default-options `((system . ,(%current-system)) (substitutes? . #t) (graft? . #t) (max-silent-time . 3600) (verbosity . 0))) If this is by design, I guess this issue can be closed. Maxim
bug#27157: Building Guile 2.2 "times out"
Hello, Ricardo Wurmus skribis: >> I'm not sure where this timeout value can be configured, but clearly it >> shouldn't be 1 hour since 2 cores of an i5 intel class processor can't >> manage to build it under 1 hour. > > for the guile-2.2 package I see that this is part of the package > definition: > > (properties '((timeout . 72000) ;20 hours > (max-silent-time . 10800))) ;3 hours (needed on ARM) > > We do this so that we can build it on Hydra. > You can override the max silent time: > > guix build --max-silent-time=999 guile > > Does “guix build” not respect these build properties? It does not. Only build-aux/hydra/gnu-system.scm does. Ludo’.
bug#27157: Building Guile 2.2 "times out"
Hi Maxim, > I'm not sure where this timeout value can be configured, but clearly it > shouldn't be 1 hour since 2 cores of an i5 intel class processor can't > manage to build it under 1 hour. for the guile-2.2 package I see that this is part of the package definition: (properties '((timeout . 72000) ;20 hours (max-silent-time . 10800))) ;3 hours (needed on ARM) We do this so that we can build it on Hydra. You can override the max silent time: guix build --max-silent-time=999 guile Does “guix build” not respect these build properties? -- Ricardo GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC https://elephly.net
bug#27157: Building Guile 2.2 "times out"
The build errors out with: --8<---cut here---start->8--- SNARF weak-set.doc SNARF weak-table.doc SNARF weak-vector.doc SNARF posix.doc SNARF net_db.doc SNARF socket.doc SNARF regex-posix.doc CCLD libguile-2.2.la .libs/libguile_2.2_la-posix.o: In function `scm_tmpnam': /tmp/guix-build-guile-2.2.2.drv-0/guile-2.2.2/libguile/posix.c:1574: warning: the use of `tmpnam' is dangerous, better use `mkstemp' CCLD guile GEN guile-procedures.texi make[3]: Leaving directory '/tmp/guix-build-guile-2.2.2.drv-0/guile-2.2.2/libguile' make[2]: Leaving directory '/tmp/guix-build-guile-2.2.2.drv-0/guile-2.2.2/libguile' Making all in bootstrap make[2]: Entering directory '/tmp/guix-build-guile-2.2.2.drv-0/guile-2.2.2/bootstrap' BOOTSTRAP GUILEC ice-9/eval.go wrote `ice-9/eval.go' BOOTSTRAP GUILEC ice-9/psyntax-pp.go BOOTSTRAP GUILEC language/cps/intmap.go building of `/gnu/store/i2p0gqxm378ydlh58qpy6jas7rdk79jg-guile-2.2.2.drv' timed out after 3600 seconds of silence cannot build derivation `/gnu/store/c34w733549bjvcdixb0yj306ydhwv8c3-guile2.2-gnutls-3.5.9.drv': 1 dependencies couldn't be built --8<---cut here---end--->8--- I'm not sure where this timeout value can be configured, but clearly it shouldn't be 1 hour since 2 cores of an i5 intel class processor can't manage to build it under 1 hour. Maxim
