Re: [PATCH] build: pull: Compile .scm files in one process.

l...@gnu.org (Ludovic Courtès) writes: > Go for it! Done! Thanks for the review. > Thank you, and apologies for the looong delay! No problem, and apologies on my side too for the delays. > Ludo’. Taylan

Re: [PATCH] build: pull: Compile .scm files in one process.

872c477bbbd18eb2465c76030 Mon Sep 17 00:00:00 2001 > From: =?UTF-8?q?Taylan=20Ulrich=20Bay=C4=B1rl=C4=B1/Kammer?= > > Date: Fri, 27 Nov 2015 09:27:55 +0100 > Subject: [PATCH] build: pull: Compile .scm files in one process. > > * guix/build/pull.scm (call-with-process, report-build

Re: [PATCH] build: pull: Compile .scm files in one process.

taylanbayi...@gmail.com (Taylan Ulrich "Bayırlı/Kammer") writes: > After some tinkering around I realized that the problem is that our > workaround of loading files explicitly causes the package record > type to be redefined after some packages have already been defined. > More gen

Re: [PATCH] build: pull: Compile .scm files in one process.

;, and the former cannot be done in parallel, so we could again cut the time to below half of what it is now, if we solved that.) Tell me if you see any problems with this variant. >From 77ac65593a94673872c477bbbd18eb2465c76030 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Taylan=20Ulrich=20Bay=C4=B1rl=

Re: [PATCH] build: pull: Compile .scm files in one process.

Mark H Weaver skribis: > taylanbayi...@gmail.com (Taylan Ulrich "Bayırlı/Kammer") writes: > >> It would be great if the whole circular import problem could somehow be >> solved by Guile (no idea how feasible it is). > > I think we should eliminate circular module dependencies. They cause > nasty

Re: [PATCH] build: pull: Compile .scm files in one process.

Mark H Weaver writes: > taylanbayi...@gmail.com (Taylan Ulrich "Bayırlı/Kammer") writes: > >> It would be great if the whole circular import problem could somehow be >> solved by Guile (no idea how feasible it is). > > I think we should eliminate circular module dependencies. They cause > nasty

Re: [PATCH] build: pull: Compile .scm files in one process.

taylanbayi...@gmail.com (Taylan Ulrich "Bayırlı/Kammer") writes: > It would be great if the whole circular import problem could somehow be > solved by Guile (no idea how feasible it is). I think we should eliminate circular module dependencies. They cause nasty problems, and there's no compellin

Re: [PATCH] build: pull: Compile .scm files in one process.

taylanbayi...@gmail.com (Taylan Ulrich "Bayırlı/Kammer") skribis: > l...@gnu.org (Ludovic Courtès) writes: > >> taylanbayi...@gmail.com (Taylan Ulrich "Bayırlı/Kammer") skribis: >> >>> After some tinkering around I realized that the problem is that our >>> workaround of loading files explicitly ca

Re: [PATCH] build: pull: Compile .scm files in one process.

l...@gnu.org (Ludovic Courtès) writes: > taylanbayi...@gmail.com (Taylan Ulrich "Bayırlı/Kammer") skribis: > >> After some tinkering around I realized that the problem is that our >> workaround of loading files explicitly causes the package record type to >> be redefined after some packages have a

Re: [PATCH] build: pull: Compile .scm files in one process.

taylanbayi...@gmail.com (Taylan Ulrich "Bayırlı/Kammer") skribis: > After some tinkering around I realized that the problem is that our > workaround of loading files explicitly causes the package record type to > be redefined after some packages have already been defined. More > generally, we for

Re: [PATCH] build: pull: Compile .scm files in one process.

ad a file again if it was already loaded would do. That's basically what importing a module does, so scanning for module definitions in files and importing them might work, but seems somewhat hacky... For now, here's the patch that just loads package.scm first. >From dcb563f611c4fb

Re: [PATCH] build: pull: Compile .scm files in one process.

taylanbayi...@gmail.com (Taylan Ulrich "Bayırlı/Kammer") skribis: > l...@gnu.org (Ludovic Courtès) writes: [...] >>?: 2 [primitive-load >> "/gnu/store/d51z2xkwp1vh0dh6gqadyyzv21m0b772-guix-latest/guix/scripts/import/hackage.scm"] >> In ice-9/eval.scm: >> 453: 1 Exception thrown while print

Re: [PATCH] build: pull: Compile .scm files in one process.

gt; Date: Thu, 5 Nov 2015 23:43:20 +0100 >> Subject: [PATCH] build: pull: Compile .scm files in one process. >> >> * guix/build/pull.scm (call-with-process, report-build-progress) >> (p-for-each): Remove. >> (thread-safe-port): New procedure. >> (build-guix): Load and

Re: [PATCH] build: pull: Compile .scm files in one process.

taylanbayi...@gmail.com (Taylan Ulrich "Bayırlı/Kammer") skribis: > From 78be6d09d2d4c0a563be14c66ac2a1a345ff9b1d Mon Sep 17 00:00:00 2001 > From: =?UTF-8?q?Taylan=20Ulrich=20Bay=C4=B1rl=C4=B1/Kammer?= > > Date: Thu, 5 Nov 2015 23:43:20 +0100 > Subject: [PATCH] build:

Re: [PATCH] build: pull: Compile .scm files in one process.

*seems* reasonable. Andy, Mark? > From 78be6d09d2d4c0a563be14c66ac2a1a345ff9b1d Mon Sep 17 00:00:00 2001 > From: =?UTF-8?q?Taylan=20Ulrich=20Bay=C4=B1rl=C4=B1/Kammer?= > > Date: Thu, 5 Nov 2015 23:43:20 +0100 > Subject: [PATCH] build: pull: Compile .scm files in one process. > >

Re: [PATCH] build: pull: Compile .scm files in one process.

gt; Date: Thu, 5 Nov 2015 23:43:20 +0100 >> Subject: [PATCH] build: pull: Compile .scm files in one process. >> >> * guix/build/pull.scm (call-with-process): Removed procedure. >> (report-build-progress): Removed procedure. >> (p-for-each): Removed procedure. > >

Re: [PATCH] build: pull: Compile .scm files in one process.

. ;-) Nice. :-) > From 0eaf7d4c264dc531718a4c0b933323f48ea91930 Mon Sep 17 00:00:00 2001 > From: =?UTF-8?q?Taylan=20Ulrich=20Bay=C4=B1rl=C4=B1/Kammer?= > > Date: Thu, 5 Nov 2015 23:43:20 +0100 > Subject: [PATCH] build: pull: Compile .scm files in one process. > > * guix/build/p

Re: [PATCH] build: pull: Compile .scm files in one process.

>From 0eaf7d4c264dc531718a4c0b933323f48ea91930 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Taylan=20Ulrich=20Bay=C4=B1rl=C4=B1/Kammer?= Date: Thu, 5 Nov 2015 23:43:20 +0100 Subject: [PATCH] build: pull: Compile .scm files in one process. * guix/build/pull.scm (call-with-process): Removed proced

Re: [PATCH] build: pull: Compile .scm files in one process.

On Thu 12 Nov 2015 12:27, taylanbayi...@gmail.com (Taylan Ulrich "Bayırlı/Kammer") writes: > First attempt: when I simply put > > (for-each (lambda (file) > (display file) > (newline) > (primitive-load file)) > files) > >

Re: [PATCH] build: pull: Compile .scm files in one process.

Andy Wingo writes: > On Thu 12 Nov 2015 10:23, taylanbayi...@gmail.com (Taylan Ulrich > "Bayırlı/Kammer") writes: > >> +;; Compile the .scm files. Also load every compiled file after writing >> it >> +;; to work around (FIXME). > > Hi :) Quick question. Wh

Re: [PATCH] build: pull: Compile .scm files in one process.

On Thu 12 Nov 2015 10:23, taylanbayi...@gmail.com (Taylan Ulrich "Bayırlı/Kammer") writes: > +;; Compile the .scm files. Also load every compiled file after writing > it > +;; to work around (FIXME). Hi :) Quick question. Why not primitive-load each file i

Re: [PATCH] build: pull: Compile .scm files in one process.

r import issue. I think it's worrying that a mere change in order of compilation can cause errors, but the speed gains of this hopefully make it an acceptable temporary solution. Thanks for the help! >From ef1a1c6cb3f2abd7faa147468d2f062f10adf2b2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Taylan=20

Re: [PATCH] build: pull: Compile .scm files in one process.

taylanbayi...@gmail.com (Taylan Ulrich "Bayırlı/Kammer") skribis: > l...@gnu.org (Ludovic Courtès) writes: [...] >> (gnu scripts environment) imports (gnu packages bash), so we end up >> loading a big bunch of (gnu packages …) modules. >> >> The problem is that (gnu packages commencement) refers

Re: [PATCH] build: pull: Compile .scm files in one process.

l...@gnu.org (Ludovic Courtès) writes: > taylanbayi...@gmail.com (Taylan Ulrich "Bayırlı/Kammer") skribis: > >> compiling >> '/gnu/store/dbkpm8rqjz096w0bchmnlj8gw15q2riy-guix-latest/guix/scripts/environment.scm'... >> Backtrace: >> In ice-9/boot-9.scm: >> 2401: 19 [save-module-excursion #> ice-9/

Re: [PATCH] build: pull: Compile .scm files in one process.

taylanbayi...@gmail.com (Taylan Ulrich "Bayırlı/Kammer") skribis: > compiling > '/gnu/store/dbkpm8rqjz096w0bchmnlj8gw15q2riy-guix-latest/guix/scripts/environment.scm'... > Backtrace: > In ice-9/boot-9.scm: > 2401: 19 [save-module-excursion # ice-9/boot-9.scm:3066:17 ()>] > 3085: 18 [#] > In unkno

Re: [PATCH] build: pull: Compile .scm files in one process.

On Mon 09 Nov 2015 09:50, taylanbayi...@gmail.com (Taylan Ulrich "Bayırlı/Kammer") writes: > Andy Wingo writes: > >> On Mon 09 Nov 2015 08:51, taylanbayi...@gmail.com (Taylan Ulrich >> "Bayırlı/Kammer") writes: >> >>> The relevant bug report is . >>> >>> According to

Re: [PATCH] build: pull: Compile .scm files in one process.

Andy Wingo writes: > On Mon 09 Nov 2015 08:51, taylanbayi...@gmail.com (Taylan Ulrich > "Bayırlı/Kammer") writes: > >> The relevant bug report is . >> >> According to Ludo's explanation, compiling a module file leads to the >> module being created in the runtime, but w

Re: [PATCH] build: pull: Compile .scm files in one process.

On Mon 09 Nov 2015 08:51, taylanbayi...@gmail.com (Taylan Ulrich "Bayırlı/Kammer") writes: >> I do not understand what you mean by "degenerate" modules :) An >> interpreted module should act the same as a compiled module. I am >> interested to hear what difference you can perceive between the t

Re: [PATCH] build: pull: Compile .scm files in one process.

Andy Wingo writes: > On Fri 06 Nov 2015 16:41, taylanbayi...@gmail.com (Taylan Ulrich > "Bayırlı/Kammer") writes: > >> Andy Wingo writes: >> >>> On Thu 05 Nov 2015 17:10, taylanbayi...@gmail.com (Taylan Ulrich >>> "Bayırlı/Kammer") writes: >>> It used to max out every CPU core, now just o

Re: [PATCH] build: pull: Compile .scm files in one process.

On Fri 06 Nov 2015 16:41, taylanbayi...@gmail.com (Taylan Ulrich "Bayırlı/Kammer") writes: > Andy Wingo writes: > >> On Thu 05 Nov 2015 17:10, taylanbayi...@gmail.com (Taylan Ulrich >> "Bayırlı/Kammer") writes: >> >>> It used to max out every CPU core, now just one. :-) >>> It used to take ~18

Re: [PATCH] build: pull: Compile .scm files in one process.

l...@gnu.org (Ludovic Courtès) writes: > Could you do ‘guix pull --verbose’ so we see the order in which files > are compiled? Here's the output: compiling... 0.0% of 463 files compiling '/gnu/store/dbkpm8rqjz096w0bchmnlj8gw15q2riy-guix-latest/guix.scm'... ;;; Failed to autoload make-sessio

Re: [PATCH] build: pull: Compile .scm files in one process.

taylanbayi...@gmail.com (Taylan Ulrich "Bayırlı/Kammer") skribis: > Sadly, when I reintroduce sorting, I get a different error: > > === snip === > compiling... 17.9% of 463 filesBacktrace: > In ice-9/boot-9.scm: > 2401: 19 [save-module-excursion # ice-9/boot-9.scm:3066:17 ()>] > 3085: 18 [#] > I

Re: [PATCH] build: pull: Compile .scm files in one process.

taylanbayi...@gmail.com (Taylan Ulrich "Bayırlı/Kammer") skribis: > We need to ensure that as soon as a module file is compiled, it's also > explicitly loaded before anything else is compiled (which might import > it), otherwise that compilation will import the "degenerate" version of > the module

Re: [PATCH] build: pull: Compile .scm files in one process.

Andy Wingo writes: > On Thu 05 Nov 2015 17:10, taylanbayi...@gmail.com (Taylan Ulrich > "Bayırlı/Kammer") writes: > >> CPU: >> >> It used to max out every CPU core, now just one. :-) >> >> >> Time: >> >> It used to take ~18 minutes on my machine, now less than 3. > > If you compile within a par-

Re: [PATCH] build: pull: Compile .scm files in one process.

On Thu 05 Nov 2015 17:10, taylanbayi...@gmail.com (Taylan Ulrich "Bayırlı/Kammer") writes: > CPU: > > It used to max out every CPU core, now just one. :-) > > > Time: > > It used to take ~18 minutes on my machine, now less than 3. If you compile within a par-for-each you should be able to peg yo

Re: [PATCH] build: pull: Compile .scm files in one process.

p 17 00:00:00 2001 >> From: =?UTF-8?q?Taylan=20Ulrich=20Bay=C4=B1rl=C4=B1/Kammer?= >> >> Date: Thu, 5 Nov 2015 16:21:48 +0100 >> Subject: [PATCH] build: pull: Compile .scm files in one process. >> >> * guix/build/pull.scm (call-with-process): Removed procedure

Re: [PATCH] build: pull: Compile .scm files in one process.

4=B1rl=C4=B1/Kammer?= > > Date: Thu, 5 Nov 2015 16:21:48 +0100 > Subject: [PATCH] build: pull: Compile .scm files in one process. > > * guix/build/pull.scm (call-with-process): Removed procedure. > (report-build-progress): Removed procedure. > (p-for-each): Removed procedure. >

[PATCH] build: pull: Compile .scm files in one process.

:21:48 +0100 Subject: [PATCH] build: pull: Compile .scm files in one process. * guix/build/pull.scm (call-with-process): Removed procedure. (report-build-progress): Removed procedure. (p-for-each): Removed procedure. (build-guix): Don't create subprocesses to compile the .scm file