Re: "guix pack -f docker" does too much work

[Andy Wingo]

On Sat 01 Jun 2024 15:58, Ludovic Courtès  writes:

>> I think it would be great if "guix pack -f docker" could avoid building
>> all these identical layers again and again.  Perhaps it would be
>> possible to have a single derivation for each layer?  This way we
>> wouldn't have to recreate the same layer archives every time.
>
> That sounds nice in terms of saving CPU time.  It’s less nice in terms
> of disk usage: a single ‘guix pack -f docker’ run would populate the
> store with roughly twice the size of the closure.

If the concern is CPU time, I would make sure you have switched to zstd
or some other faster codec, via `guix pack -f docker -C zstd`.

You probably already knew but if you haven't tried, it's quite
surprising :)

Andy

Re: Guile CSE elimination of record accessor?

[Andy Wingo]

Hi :)

On Sat 27 Apr 2024 19:04, Simon Tournier  writes:

> In Guile module (ice-9 vlist), one reads:
>
> ;; Asserting that something is a vlist is actually a win if your next
> ;; step is to call record accessors, because that causes CSE to
> ;; eliminate the type checks in those accessors.
> ;;
> (define-inlinable (assert-vlist val)
>   (unless (vlist? val)
> (throw 'wrong-type-arg
>#f
>"Not a vlist: ~S"
>(list val)
>(list val
>
> [...]
>
> (define (vlist-head vlist)
>   "Return the head of VLIST."
>   (assert-vlist vlist)
>   (let ((base   (vlist-base vlist))
> (offset (vlist-offset vlist)))
> (block-ref (block-content base) offset)))
>
>
> Other said, the argument ’vlist’ is “type-checked” with ’assert-vlist’
> and thus that is exploited by Guile compiler, if I understand correctly
> the comment.
>
> The first question is: is it still correct?  Because this module had
> been implemented before many Guile compiler improvements.

No, the comment is incorrect.  The type check on whatever accessor is
called first (unspecified in scheme; probably we should just bite the
bullet and do predictable left-to-right semantics, as racket does) will
dominate the rest and eliminate those checks.  The assert-type is
unnecessary.

To see this, do ,optimize-cps at the repl, and count the number of
e.g. struct? checks with and without the assert-vlist.  There is only
one, either way.  (A type check is a heap-object? check, then struct?,
then get the vtable, then check against the global variable .
All of these duplicates get eliminated.)

> PS: Raining day and weird pastime… diving into Guile source code. ;-)

:)

Cheers

Andy

Re: using shepherd's (shepherd service repl) in guix system

[Andy Wingo]

On Tue 09 Jan 2024 23:36, Ludovic Courtès  writes:

> scheme@(shepherd-user)> (+ 2 3)
> $6 = 5
> scheme@(shepherd-user)> (lookup-service 'repl)
> ;;; socket:56:1: warning: possibly unbound variable `lookup-service'
> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
> Unbound variable: lookup-service
>
> Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
> scheme@(shepherd-user) [1]> While reading expression:
> Attempt to suspend fiber within continuation barrier
> scheme@(shepherd-user) [1]> While reading expression:
> Attempt to suspend fiber within continuation barrier
>
>
> The problem here is that the exception is raised C code (in
> libguile/modules.c, then indirectly calling ‘scm_error’), which makes it
> a “continuation barrier” (meaning that it prevents Fibers from switching
> contexts among fibers in the shepherd process, hence the error above.)

Interesting.  I guess the short-term fix is to only enter the debugger
if suspendable-continuation? is true for the fibers prompt, and
otherwise just print a backtrace.

Andy

Re: better error messages through assertions

[Andy Wingo]

On Wed 30 Mar 2022 11:37, Ludovic Courtès  writes:

>> scheme@(guile-user)> (container-contents '())
>> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
>> In procedure struct-vtable: Wrong type argument in position 1
> scheme@(guile-user)> ,use(srfi srfi-9)
> scheme@(guile-user)> (define-record-type 
>  (make-foo x)
>  foo?
>  (x foo-x))
> scheme@(guile-user)> ,optimize (foo-x '())
> $9 = (if (eq? (struct-vtable '()) )
>   (struct-ref '() 0)
>   (throw 'wrong-type-arg
>  'foo-x
>  "Wrong type argument: ~S"
>  (list '())
>  (list '(
>
> With Guile 3, it might be that adding an extra ‘struct?’ test would have
> little effect on performance; we’d need to check.

Would have no effect.

Incidentally, you might want to use ,optimize-cps;

  scheme@(guile-user)> ,optimize (foo-x '())
  $9 = (if (eq? (struct-vtable '()) )
(struct-ref '() 0)
(throw 'wrong-type-arg
   'foo-x
   "Wrong type argument: ~S"
   (list '())
   (list '(
  scheme@(guile-user)> ,optimize-cps (foo-x '())
  L0:   ;   at 
:15:14
v0 := self
L1(...)
  L1:
receive()
v1 := const ()  ; arg   at 
:15:21
throw throw/value+data[#(wrong-type-arg "struct-vtable" "Wrong type 
argument in position 1 (expecting struct): ~S")](v1) ;  at :15:14

L1(...) means, pass all values to L1.  In this case because there are
varargs on the stack from the procedure call.  L1 parses them with the
receive().  Anyway, here we see that with respect to the immediate '(),
that all the tests folded.  If we instead lift to a procedure:

  scheme@(guile-user)> ,optimize-cps (lambda (x) (foo-x x))
  L0:   ;   at 
:16:14
v0 := self
L1(...)
  L1:
receive()
v1 := current-module()  ; moduleat 
:16:14
cache-set![0](v1)   ;   at 
:16:14
v2 := const-fun L7  ; _ 
return v2   ;   at 
:16:14

  L7:   ;   at 
:16:14
v3 := self
L8(...)
  L8:
v4 := receive(x); x 
heap-object?(v4) ? L10() : L38();   at 
:16:26
  L10():
struct?(v4) ? L11() : L38() ;   at 
:16:26
  L38():
throw throw/value+data[#(wrong-type-arg "struct-vtable" "Wrong type 
argument in position 1 (expecting struct): ~S")](v4) ;  at :16:26
  L11():
v5 := scm-ref/tag[struct](v4)   ; vtableat 
:16:26
v6 := cache-ref[(0 . )]()  ; cachedat 
:10:20
heap-object?(v6) ? L19() : L14();   at 
:10:20
  L19():
L20(v6) ;   at 
:10:20
  L14():
v7 := cache-ref[0](); mod   at 
:10:20
v8 := const; name  at 
:10:20
v9 := lookup-bound(v7, v8)  ; var   at 
:10:20
cache-set![(0 . )](v9) ;   at 
:10:20
L20(v9) ;   at 
:10:20
  L20(v10): ; box 
v11 := scm-ref/immediate[(box . 1)](v10); arg   at 
:10:20
eq?(v5, v11) ? L22() : L37();   at 
:16:26
  L37():
throw throw/value+data[#(wrong-type-arg foo-x "Wrong type argument: 
~S")](v4) ;  at :16:26
  L22():
v12 := word-ref/immediate[(struct . 6)](v5) ; rfields   at 
:16:26
v13 := v12  ; nfields   at 
:16:26
imm-u64-<[0](v13) ? L25() : L35()   ;   at 
:16:26
  L35():
v21 := const 0  ; _ at 
:16:26
throw throw/value+data[#(out-of-range "struct-ref/immediate" "Argument 2 
out of range: ~S")](v21) ;  at :16:26
  L25():
v14 := pointer-ref/immediate[(struct . 7)](v5) ; ptrat 
:16:26
v15 := load-u64[0](); word  at 
:16:26
v16 := u32-ref[bitmask](v5, v14, v15)   ; bits  at 
:16:26
v17 := load-u64[1](); mask  at 
:16:26
v18 := ulogand(v17, v16); res   at 
:16:26
u64-imm-=[0](v18) ? L31() : L33()   ;   at 
:16:26
  L33():
v20 := const 0  ; _ at 
:16:26
throw throw/value+data[#(wrong-type-arg "struct-ref/immediate" "Wrong type 
argument in position 2 (expecting boxed field): ~S")](v20) ;  at :16:26
  L31():
v19 := scm-ref/immediate[(struct . 1)](v4)  ; val   at 
:16:26
return v19  ;   at 
:16:26

Here we see the first procedure which 

Re: [minor patch] Amend CoC

[Andy Wingo]

On Tue 22 Feb 2022 18:16, Taylan Kammer  writes:

> If anyone's annoyed by this thread, please tell, and let us move it off-list.

I am annoyed by it I think it should be off-list :)

Andy

Re: [bootstrappable] Re: Can Guile be bootstrapped from source without psyntax-pp.scm?

[Andy Wingo]

On Mon 15 Mar 2021 20:50, Michael Schierl  writes:

> Am 15.03.2021 um 18:09 schrieb Ludovic Courtès:
>> Woow, this is great news!  I think it would be great towards importing
>> it in Guile proper.
>>
>> To do that, I think we should first get Andy’s opinion on the approach.
>
> I don't think upstream is very interested in having psyntax-pp.scm
> bootstrappable.

Why do you say that? :)

> In Guile 3.0.3 they broke even the `make ice-9/psyntax-pp.scm.gen`
> target, and did not repair it even in Guile 3.0.5, that's why I used
> 3.0.2 for the bootstrap.

Strange -- I used it many times over the past couple months without
problems.  But I see now from your patches what the issue was -- and omg
how embarrassing, I must have a stale canonicalize.go file hanging
around in the tree.  Goes to show how important bootstrapping is!

Andy

Re: The size of ‘.go’ files

[Andy Wingo]

Hi :)

On Tue 09 Jun 2020 18:09, Ludovic Courtès  writes:

> Andy Wingo  skribis:
>
>> The guile.arities section starts with a sorted array of fixed-size
>> headers, then is followed by a sequence of ULEB128 references to local
>> variable names, including non-arguments.  The size is a bit perplexing,
>> I agree.  I can think of a number of ways to encode that section
>> differently but we'd need to understand a bit more about it and why the
>> baseline compiler is significantly different.
>
> ‘.guile.arities’ size should be proportional to the number of
> procedures, right?  Additionally, if there are only/mostly thunks, the
> string table for argument names should be small if not empty.  For N
> thunks, I would expect roughly N 28-byte headers + NxM UL128, say 100
> bytes per thunk; there’s 1000 of them, so we should be ~100,000 bytes.
> This is roughly what we get observe with the baseline compiler.

Yes but that doesn't mean that you can directly compare baseline to CPS
-- CPS has many more intermediate names than baseline for non-argument
locals, all of which end up getting entries in the arities section.

Andy

Re: The size of ‘.go’ files

[Andy Wingo]

Hi :)

A few points of information :)

On Fri 05 Jun 2020 22:50, Ludovic Courtès  writes:

> [Sorting] the ELF sections of a .go file by size; for ‘python-xyz.go’,
> I get this:
>
> $13 = ((".rtl-text" . 3417108)
>  (".guile.arities" . 1358536)
>  (".data" . 586912)
>  (".rodata" . 361599)
>  (".symtab" . 117000)
>  (".debug_line" . 97342)
>  (".debug_info" . 54519)
>  (".guile.frame-maps" . 47114)
>  ("" . 1344)
>  (".guile.arities.strtab" . 681)
>  ("" . 232)
>  (".shstrtab" . 229)
>  (".dynamic" . 112)
>  (".debug_str" . 87)
>  (".strtab" . 75)
>  (".debug_abbrev" . 65)
>  (".guile.docstrs.strtab" . 1)
>  ("" . 0)
>  (".guile.procprops" . 0)
>  (".guile.docstrs" . 0)
>  (".debug_loc" . 0))
>
> More than half of those 6 MiB is code, and more than 1 MiB is
> “.guile.arities” (info "(guile) Object File Format"), which is
> surprisingly large; presumably the file only contains thunks (the
> ‘thunked’ fields of ).

The guile.arities section starts with a sorted array of fixed-size
headers, then is followed by a sequence of ULEB128 references to local
variable names, including non-arguments.  The size is a bit perplexing,
I agree.  I can think of a number of ways to encode that section
differently but we'd need to understand a bit more about it and why the
baseline compiler is significantly different.

> Stripping the .debug_* sections (if that works) clearly wouldn’t help.

I believe that it should eventually be possible to strip guile.arities,
fwiw.

> So I guess we could generate less code (reduce ‘.rtl-text’), perhaps by
> tweaking ‘define-record-type*’, but I have little hope there.

Hehe :)  As you mention later:

> With 3.0.3-to-be and -O1, python-xyz.go weighs in at 3.4 MiB instead of
> 5.9 MiB!  Here’s the section size distribution:
>
> $4 = ((".rtl-text" . 2101168)
>  (".data" . 586392)
>  (".rodata" . 360703)
>  (".guile.arities" . 193106)
>  (".symtab" . 117000)
>  (".debug_line" . 76685)
>  (".debug_info" . 53513)
>  ("" . 1280)
>  (".guile.arities.strtab" . 517)
>  ("" . 232)
>  (".shstrtab" . 211)
>  (".dynamic" . 96)
>  (".debug_str" . 87)
>  (".strtab" . 75)
>  (".debug_abbrev" . 56)
>  (".guile.docstrs.strtab" . 1)
>  ("" . 0)
>  (".guile.procprops" . 0)
>  (".guile.docstrs" . 0)
>  (".debug_loc" . 0))
> scheme@(guile-user)> (stat:size (stat go))
> $5 = 3519323
>
> “.rtl-text” is 38% smaller and “.guile.arities” is almost a tenth of
> what it was.

The difference in the text are the new baseline intrinsics,
e.g. $vector-ref.  It goes in the opposite direction from instruction
explosion, which sought to (1) make the JIT compiler easier by
decomposing compound operations into their atomic parts, (2) make the
optimizer learn more information from flow rather than type-checking
side effects, and (3) allow the optimizer to eliminate / hoist / move
the component pieces of macro-operations.

However in the baseline compiler (2) and (3) aren't possible because
there is no optimizer on that level, and therefore the result is
actually a lose -- 10 micro-ops cost more than 1 macro-op because of
stack traffic overhead, which isn't currently mitigated by the JIT (1).

So instruction explosion is residual code explosion, which should pay
off in theory, but not for the baseline compiler.  So I added new
intrinsics for e.g. $vector-ref et al.  Thus the smaller code size.

I am not sure what causes the significantly different .guile.arities
size!

> Something’s going on here!  Thoughts?

There are more possibilities for making code size smaller, e.g. having
two equivalent encodings for bytecode, where one is smaller:

  https://webkit.org/blog/9329/a-new-bytecode-format-for-javascriptcore/

Or it could be that if we could do register allocation for a
target-dependent fixed set of registers in bytecode already, that could
decrease minimum instruction size, making more instructions fit into
single 32-bit words.  Would be nice if the JIT could rely on the
bytecode compiler to already have done register allocation, and reify
corresponding debug information.  Just a thought though, and not really
appropriate to the baseline compiler.

Cheers,

Andy

Re: Compilation time with Guile 3.0.3-to-be

[Andy Wingo]

Hi :)

On Thu 04 Jun 2020 09:50, Ludovic Courtès  writes:

> With the attached patch I’ve run ‘make as-derivation’ (equivalent to
> ‘guix pull’) and timed the builds of guix-packages-base.drv (279 files)
> and guix-packages.drv (217 files) on my 4-core i7 laptop:
>
>   • guix-packages-base.drv: 1m30s (was 4m)
>
>   • guix-packages.drv: 30s (was 1m8s)

Nice!!  Thanks for testing :-)

> For the record, the optimizations currently used in (guix build compile)
> are between the new -O0 and -O1:
>
>   (cond ((or (string-contains file "gnu/packages/")
>  (string-contains file "gnu/tests/"))
>  ;; Level 0 is good enough but partial evaluation helps preserve the
>  ;; "macro writer's bill of rights".
>  (override-option #:partial-eval? #t
>   (optimizations-for-level 0)))

Here fwiw I would use -O1.  It is basically the same as -O0 except that
it adds partial evaluation and it inlines primcalls.  If you are willing
to do partial evaluation, you are probably willing to inline primcalls
too; I think it typically leads to less code and the compilation time is
similar to -O0.

> ((string-contains file "gnu/services/")
>  ;; '-O2 -Ono-letrectify' compiles about ~20% faster than '-O2' for
>  ;; large files like gnu/services/mail.scm.
>  (override-option #:letrectify? #f
>   (optimizations-for-level 2)))

Interesting.  I think this is probably a bug of some sort that we'll
have to keep working on.

> With the new -O1, the scheme->tree-il conversion (mostly macro
> expansion) accounts for half of the build time on large files:
>
> scheme@(guile-user)> ,use(system base optimize)
> scheme@(guile-user)> ,time (compile-file "gnu/packages/python-xyz.scm" #:opts 
> (optimizations-for-level 1))
> $1 = 
> "/data/src/guile-3.0/cache/guile/ccache/3.0-LE-8-4.3/home/ludo/src/guix/gnu/packages/python-xyz.scm.go"
> ;; 4.154311s real time, 5.604945s run time.  2.538106s spent in GC.

Interesting data :)

Note that in 3.0.3 there is also a new phase called "lowering".  Before
a compiler from Tree-IL to language X is called, the tree-IL program is
"lowered" -- meaning canonicalized and optionally optimized.

(define (lower-exp exp env optimization-level opts)
  (let ((make-lowerer (language-lowerer (lookup-language 'tree-il
((make-lowerer optimization-level opts) exp env)))

Similarly there is an analysis pass for warnings, which runs before
lowering:

(define (analyze-exp exp env warning-level warnings)
  (let ((make-analyzer (language-analyzer (lookup-language 'tree-il
((make-analyzer warning-level warnings) exp env)))

These can be interesting to test different phases of the tree-il ->
bytecode path.

> The profile looks like this:
>
> scheme@(guile-user)> ,pr (define t (call-with-input-file  
> "gnu/packages/python-xyz.scm" (lambda (port) (read-and-compile port #:to 
> 'tree-il
> % cumulative   self 
> time   seconds seconds  procedure
>  13.16  0.45  0.40  anon #x1136458
>  10.53  0.35  0.32  ice-9/popen.scm:168:0:reap-pipes
>   7.89  0.24  0.24  anon #x1132af8
>   6.14  0.35  0.19  ice-9/boot-9.scm:3128:0:module-gensym
>   5.26  0.21  0.16  ice-9/boot-9.scm:2201:0:%load-announce
>   4.39  0.19  0.13  ice-9/psyntax.scm:749:8:search
>   3.51  0.69  0.11  ice-9/psyntax.scm:2964:6:match*
>   3.51  0.11  0.11  anon #x11334e8
>   3.51  0.11  0.11  anon #x1136428
>   2.63  0.08  0.08  anon #x113a258
>   1.75  0.05  0.05  ice-9/psyntax.scm:3017:12:$sc-dispatch
>   1.75  0.05  0.05  anon #x1139e68
>   0.88277.30  0.03  ice-9/boot-9.scm:220:5:map1
>
> It’d be great to waive the anonymity of that first lambda.  :-)

I think I just fixed it :)

> I was wondering what fraction of that time was spent running Guix macros
> (‘package’, ‘base32’, and so on), but it’s difficult to answer that
> question here.  Probably something to investigate so we can make further
> progress!

I think the reap-pipes call is a pretty bad sign, incidentally!

Cheers,

Andy

Re: Medium-term road map

[Andy Wingo]

On Sat 25 Apr 2020 15:37, Ludovic Courtès  writes:

>   2. Performance.  There are many things we can improve there, first and
>  foremost: the “Computing derivation” part of ‘guix pull’, Guile’s
>  compiler terrible time and space requirements

I think I have a solution here, as discussed on IRC.  Basic idea is to
make a direct compiler from Tree-IL to bytecode, skipping the CPS step.
The result won't be optimal in terms of generated code quality (I
estimate on average 2x slowdown) but it will be very cheap to produce (I
estimate 10x improvement in time, and going from O(n log n) space to
O(n), with better constant factors.  We'd use this compiler for -O0 or
maybe even -O1 code.

Got to spend a few days working it up tho!

Andy

Re: Speeding up “guix pull”: splitting modules

[Andy Wingo]

Hi,

Just a couple notes here regarding the concrete questions.  I'm sure
there are good solutions but I don't know what they are yet!

On Sun 05 Jan 2020 21:37, Ricardo Wurmus  writes:

> Or could we improve compilation
> times by disabling optimizations?

I think Guix does this already; see %lightweight-optimizations in (guix
build compile).

> Or by marking the modules as declarative and thus unlock more
> optimizations (with Guile 3)?

Not really; the declarative modules optimization only really helps cases
where inlining would help.  For package definitions, there's not much to
inline, I don't think.  Given that the consideration is compile-time
rather than run-time I would not think that declarative modules would
help.

Andy

Re: bug#38529: Make --ad-hoc the default for guix environment proposed deprecation mechanism

[Andy Wingo]

On Fri 20 Dec 2019 22:08, Ricardo Wurmus  writes:

> zimoun  writes:
>
>>  - I propose the name "guix shell"
>
> This is not a bad idea, especially considering that “guix environment”
> was meant to get shebang support, so that you could use it as the
> interpreter in a script that handles the environment configuration.
>
> It is also shorter.

I like this idea.  It would also allow us to deprecate "guix
environment" over a period of a year or so, and we can probably show an
equivalent "guix shell" invocation in the deprecation message.

Andy

Re: Profiles/manifests-related command line interface enhancements

[Andy Wingo]

On Sun 10 Nov 2019 10:36, Konrad Hinsen  writes:

> One direction could be to add a sandboxing feature to Guile, which would
> be nice-to-have for other uses as well if Guile is to become a
> general-purpose systems scripting language. There are some interesting
> ideas in shill (http://shill.seas.harvard.edu/) for this scenario.

I wrote this for that purpose:

  https://www.gnu.org/software/guile/manual/html_node/Sandboxed-Evaluation.html

However I can't recommend it as a robust security layer because of the
weaknesses in the heap allocation limit; discussed in the page above.

I agree that Shill has some great patterns that go beyond what Guile or
Guix has, and that adopting some of them is a really interesting idea
:-)

I admit that I was a bit depressed at the impact that Spectre et al has
had on language-level sandboxing abstractions :-( and haven't much
pursued this line since then.  In practice Guix's "containerized" build
jobs are much more effective than in-language barriers.

Cheers,

Andy

Re: smallerizing bootstrap guile?

[Andy Wingo]

On Mon 04 Nov 2019 11:57, Efraim Flashner  writes:

> Is there a good way to actually compile the .go files and use them to
> replace the ones shipped in the guile-static tarball?
> Can we do this on purpose and ship just the bin/ output, grab guile
> sources and compile it for real? Or can I delete the .go files, take the
> .scm files and compile them? Any hints on the guild invocation?

Not entirely sure what you mean, but it would be possible to unpack
Guile's sources, include as a build input a built version of that same
Guile, then do

  make GUILE_FOR_BUILD=[path to native guile] -C prebuilt/32-bit-big-endian

then take the .go files from prebuilt/32-bit-bit-endian and define them
as an output for some guile-32-bit-big-endian bootstrap package.  Then
you could use that package as an input to another package that builds a
native PPC Guile, but having previously unpacked the .go files into
prebuilt/ on the PPC package, after the step in the existing Guile
recipes that scrubs prebuilt/.

Andy

Re: Preliminary IceCat 68.2 package

[Andy Wingo]

Heya Mark :)

On Wed 23 Oct 2019 12:32, Mark H Weaver  writes:

> I wrote:
>> * Many earlier attempts to build it have failed due to non-deterministic
>>   failures in the build system, possibly due to a bug in the Cargo tool.
>>   I'm not sure how much luck was involved in my successful build.  Your
>>   mileage may vary.
>
> Having now done a few more test builds, I'm sorry to say that the build
> seems to often fail non-deterministically.  Most of my attempts have
> failed so far.
>
> The most common failure mode is described in these bug reports for
> Gentoo and BLFS:
>
>   https://bugs.gentoo.org/show_bug.cgi?id=680934
>   http://wiki.linuxfromscratch.org/blfs/ticket/11975#comment:6
>
> Those errors, where a backtrace is printed by cargo, happen at a
> different place in the build every time.  If you're lucky, you might get
> through the entire build without hitting that bug.
>
> I'd be very grateful for help debugging these failures or finding a
> workaround.

I don't know what this is, but at work I have to build Firefox a lot,
and sometimes the rust bits die in the middle of compilation.  Is it a
resource exhaustion issue?  I am not sure.  In any case, continuing to
"make" always solves it for me.  I know that's not very reassuring, but
if you're looking for a quick workaround, running your "make -jN" in a
loop 5 times or something may be good enough.

Cheers,

Andy

Re: Joint statement on the GNU Project

[Andy Wingo]

On Tue 08 Oct 2019 18:38, Dimakakos Dimos  writes:

> So to sum up it would be nice to expand a bit on the announcement. My
> main questions are:
>
>  1. How does the GNU project operate, in the light of all the
>  different projects, and what role and influence had rms as the head
>  of GNU?
>
>  2. How does rms with his afformentioned role undermine diversity
>  and inclusivity of people working in GNU projects?
>
>  3. What would be your suggestion on the next day of GNU when rms
>  hypothetically steps down? Is he a single issue or are the other
>  issues that our community shall overcome.
>
>  4. In the case that rms is actually harming to our community how do
>  we protect free software ideals while removing a person so closely
>  related to their creation?

For what it is worth, I have some personal answers to some of these
questions here:

  https://wingolog.org/archives/2019/10/08/thoughts-on-rms-and-gnu

Regards,

Andy

Re: Magit is very slow with Guix's Git checkout

[Andy Wingo]

On Tue 18 Jun 2019 06:04, Chris Marusich  writes:

> I've been learning to use Magit more and more, and I really enjoy it.
> However, I've noticed that it takes a very long time (minutes) to give
> control back to me when I run "M-x magit-status" in my Guix checkout.
>
> Am I doing something wrong?  Is there a way to avoid this slowdown?
> Magit is great, but it's painfully slow when I try to use it to interact
> with the Guix's Git repository.

Apparently issues like https://github.com/magit/magit/issues/3808 are
being systematically closed in favor of a plan to compute diffs in two
steps.  However I don't entirely understand the argument as magit-status
on a clean tree should be free.  I share the frustration.

Andy

Re: [PATCH][WIP] Elogind update

[Andy Wingo]

On Sat 10 Nov 2018 17:00, Stefan Stefanović  writes:

> But the problem is with the stale PID found after a forced/abrupt reboot.
> As far as I know other distributions do not have to deal with this problem,
> because they mount tmpfs to /run and /var/run.
> We (Guix) need to find the 2nd part of the solution.

We do this too, right?  At least there's a subtree of /run that we mount
fresh at boot-time.  See %elogind-file-systems in (gnu system
file-systems).  Perhaps we could put the PID file in /run/systemd or
something.

Andy

Re: FSDG status of chromium

[Andy Wingo]

On Tue 25 Sep 2018 21:36, Clément Lassieur  writes:

> Thank you very much for letting us know about these issues!  I'm glad
> Marius put it in a channel then, it was a wise decision.  I hope we'll
> make it free at some point, so that it can be integrated into Guix.  But
> there is Icecat 60 now (thanks to everyone involved!), so it's not that
> urgent anymore.

Unless I misunderstand, we have only been notified about the history of
a discussion.  Are you aware of any concrete issue that has been raised?
Any non-free file or incompatible in our distribution?

Andy

Re: FOSDEM 2019

[Andy Wingo]

On Tue 21 Aug 2018 19:57, Ricardo Wurmus  writes:

> If we base our application on this proposal I think we should make these
> changes:
>
> - mention the work on adding JIT

Yes definitely!  This will be Guile 3, so we can call it "Just-in-time
code generation for Scheme: Speed for free in Guile 3" or something.
I'd be happy to give this talk.

Andy

Re: IRC freenode issue

[Andy Wingo]

Hi,

On Thu 02 Aug 2018 15:04, Clément Lassieur  writes:

> There seems to be an issue with #guix: I can't join anymore.  The error
> is:
>
>   Cannot join channel (+i) - you must be invited 
>
> I think it also happens to other people (ng0, marusich).

This is a configuration error I think.  There was a similar one in
Guile.  The fundamental issue is that we are not irc experts :)  The
derived issue is that someone suggested the "/mode +r" command to remove
a spam problem.  But for some reason that is preventing people from
joining.  Goran Weinholt then suggested "/mode -r" but "/mode +q $~a",
as what #scheme and #chez were using; that lets anyone join but only
registered users speak.  However in the meantime the only ops in #guix
were unavailable.  I think this will get fixed whenever Mark, Ricardo,
or Ludo can get around to it.

Visit #guile for technical support in the meantime.

Andy

Re: PYTHONPATH woes

[Andy Wingo]

On Tue 20 Feb 2018 16:01, Pjotr Prins  writes:

> On Tue, Feb 20, 2018 at 11:53:54AM +0100, Ricardo Wurmus wrote:
>> Would it be good to make the wrappers for Python scripts stricter and
>> not accept any user-set PYTHONPATH?
>
> I think that is a bad idea. You need to be able to opt out.

Why?  I am not sure this is the case for programs that just happen to be
written in Python.

> That should come with a health warning ;). Similarly we should allow
> for LD_LIBRARY_PATH etc. It is what they exist for, even if it is
> dangerous.

In Guix we don't set LD_LIBRARY_PATH but we do set PYTHONPATH, so it's
not quite the same I don't think.

Andy

Re: Improving Shepherd

[Andy Wingo]

On Wed 14 Feb 2018 14:10, l...@gnu.org (Ludovic Courtès) writes:

> Christopher Lemmer Webber  skribis:
>
>> Ludovic Courtès writes:
>>
>>> Hopefully it’s nothing serious: Fibers doesn’t rely on anything
>>> architecture-specific.
>>
>> I think it relies on epoll currently?  But I think there should be no
>> reason other architectures couldn't also be supported.
>
> Ooh good point, that may rule out GNU/Hurd.  :-/

You can always replace the epoll module :)

Andy

idea: "guix cargo", "guix npm" etc

[Andy Wingo]

A little idea inspired by Eelco's talk at the Guix miniconference.

The "problem" is that users are often more used to their
language-specific package manager than Guix.  They find "cargo install
foo" to be convenient.  However this rarely results in a reproducible
build in the sense of Guix or Nix.

So, an idea: why not offer a subset of the command-line interface of
"cargo" et al?  Specifically the commands to install a package.  The
"guix cargo" command would translate "cargo install" instantiations to
Guix commands.  If the package is in Guix, then it installs directly.
If not, maybe it runs the importer and prepares a package submission to
Guix?

Just an idea!  I don't plan on working on this in the short term.

Andy

Re: [Geiser-users] geiser-xref-callers does not seem to work

[Andy Wingo]

On Mon 29 Jan 2018 18:52, "Jose A. Ortega Ruiz"  writes:

> i've figured out how to find the filename for a procedure, but i'm
> missing what is probably the easier part: give the latter, how do i get
> hold of the module object?

On the one side there's no 100% reliable way.  E.g. some files don't
declare what module they're in, and so if they are loaded (e.g. via
"include") from within some other module, then they inherit the current
module from the caller.

However there is the very common case in which each file defines a
module -- there, I would actually go about this in the other way.  Walk
the module tree and record source file names that map to modules.
However like source-procedures, probably Guile should bake this facility
into (system xref).  Anyway here is a procedure that will do it:

  ;; Return hash table mapping filename to list of modules defined in that
  ;; file.
  (define (compute-source->module-mapping)
(let ((ret (make-hash-table)))
  (define (record-module m)
(let ((f (module-filename m)))
  (hash-set! ret f (cons m (hash-ref ret f '())
  (define (visit-module m)
(record-module m)
(hash-for-each (lambda (k v) (visit-module v))
   (module-submodules m)))
  (visit-module (resolve-module '() #f))
  ret))

Cheers,

Andy

Re: Cuirass news

[Andy Wingo]

On Sun 28 Jan 2018 22:47, l...@gnu.org (Ludovic Courtès) writes:

> interned symbols are potentially not GC’d (though I think with Guile
> 2.2 and its weak sets they may be subject to GC.)

Symbols are collectable.  They are collectable in 2.0 as well.

Depending on the context, it may be worth considering using non-symbols
if all you need is a unique object -- some freshly allocated object will
do.  That's for when you know that X is just an identifier of some kind
(you don't have to dispatch on its type, etc).

Andy

Re: [Geiser-users] geiser-xref-callers does not seem to work

[Andy Wingo]

Hi!

Great to hear from you jao :)

On Sat 27 Jan 2018 17:41, "Jose A. Ortega Ruiz"  writes:

> hmmm, i was investigating this.  the cause geiser fails is that, in the
> process of looking for other things, it's not able to find
> `program-arities', exported by (system vm program).  i am not sure why:

Weird.  Apologies for this, the function just doesn't exist.  While
program-arity exists and calls it, program-arity itself isn't called.  I
think this is detritus of refactoring :(

The facility that Guile uses now for arities is find-program-arity from
(system vm debug).  It takes an address as an argument.  Pass
(program-code prog) if you know you have a program.  The issue is that
with Guile 2.2 there is code that doesn't necessarily have a
corresponding program object.  For example a function defined in a local
context whose callers can be enumerated by the compiler can be called by
label instead of by value, and in that case the "closure" of the
function can be specialized to be e.g. a vector instead of a closure
object.  So while you always have an address, you don't always have a
program object.

Relatedly, you ask about program-module.  This one is sadly no longer
available.  I suspect it is not coming back either :/  I think you have
to fall back on mapping procedures to files, and files to modules.

You might find a look to (system vm debug) to be useful for Geiser
purposes.  Note also that the interface to local variables and stack
frames changed slightly too; see (system vm frame).

Cheers,

Andy

Re: question regarding substitute* and #t

[Andy Wingo]

On Thu 25 Jan 2018 06:31, Maxim Cournoyer  writes:

> Where does this `invoke' comes from? Geiser is unhelpful at finding it,
> and it doesn't seem to be documented in the Guile Reference?

https://lists.gnu.org/archive/html/guix-devel/2018-01/msg00163.html

Re: question regarding substitute* and #t

[Andy Wingo]

On Wed 24 Jan 2018 15:45, Hartmut Goebel <h.goe...@crazy-compilers.com> writes:

> Am 24.01.2018 um 13:14 schrieb Andy Wingo:
>> On Wed 24 Jan 2018 13:06, Mark H Weaver <m...@netris.org> writes:
>>
>>> + ;; Install to the right directory
>>> + (substitute* '("Makefile"
>>> +"Qsci/Makefile")
>>> +   (("\\$\\(INSTALL_ROOT\\)/gnu/store/[^/]+")
>>> +(assoc-ref outputs "out")))
>>> + #t)
>> I guess once we switch over all instances of "system" and "system*" to
>> use invoke, does that mean we will also be able to remove these
>> vestigial "#t" returns?
> I wonder why substitute* not simply returns #t?!

There was a proposal to make it return #t!  However then someone pointed
out that actually instead of making phases return boolean results, we
should instead signal problems via exceptions, and that drove the shift
from system / system* to invoke.  In the future world where completion
means success, it doesn't matter what substitute* returns.

However!  Because it doesn't matter, perhaps in the interest of
transition we should make substitute* return #t, so that once we switch
to the new exception-based error signalling, that we have less code to
clean up later.

Andy

Re: question regarding substitute* and #t

[Andy Wingo]

On Wed 24 Jan 2018 14:28, Mark H Weaver <m...@netris.org> writes:

> Andy Wingo <wi...@igalia.com> writes:
>
>> On Wed 24 Jan 2018 13:06, Mark H Weaver <m...@netris.org> writes:
>>
>>> + ;; Install to the right directory
>>> + (substitute* '("Makefile"
>>> +"Qsci/Makefile")
>>> +   (("\\$\\(INSTALL_ROOT\\)/gnu/store/[^/]+")
>>> +(assoc-ref outputs "out")))
>>> + #t)
>>
>> I guess once we switch over all instances of "system" and "system*" to
>> use invoke, does that mean we will also be able to remove these
>> vestigial "#t" returns?
>
> After we switch to using 'invoke' everywhere, or more precisely, after
> we arrange to never return #false from any phase or snippet, then there
> should be one more step before removing the vestigial #true returns: we
> should change the code that calls phases or snippets to ignore the
> value(s) returned by those procedures.  When that is done, then the #t's
> will truly be vestigial.  Does that make sense?

Sure, makes sense.  Thanks for thinking it through with me :)

Andy

question regarding substitute* and #t (was: Simplifications enabled by switching to 'invoke')

[Andy Wingo]

Hi!

On Wed 24 Jan 2018 13:06, Mark H Weaver  writes:

> + ;; Install to the right directory
> + (substitute* '("Makefile"
> +"Qsci/Makefile")
> +   (("\\$\\(INSTALL_ROOT\\)/gnu/store/[^/]+")
> +(assoc-ref outputs "out")))
> + #t)

I guess once we switch over all instances of "system" and "system*" to
use invoke, does that mean we will also be able to remove these
vestigial "#t" returns?

Andy

Re: What do Meltdown and Spectre mean for libreboot x200 user?

[Andy Wingo]

Greets,

On Mon 15 Jan 2018 12:32, Leah Rowe  writes:

> The implications [of Meltdown/Spectre] at firmware level are
> non-existent (for instance, these attacks can't, to my knowledge, be
> used to actually run/modify malicious code, just read memory, so it's
> not as if some evil site could install malicious boot firmware in your
> system).

I agree that it's unlikely that a site could install boot firmware, but
AFAIU it's not out of the realm of possibility.  The vector I see would
be using Meltdown/Spectre to read authentication/capability tokens which
could be used to gain access, either via some other RCE vuln or possibly
via remote access.  Maybe evil code could find an SSH private key in a
mapped page, for example, which the evil server could use to SSH
directly to your machine.  But I admit that it's a bit farfetched :)

Andy

Re: Why should build phases not return unspecified values?

[Andy Wingo]

On Sun 17 Dec 2017 00:28, Arun Isaac  writes:

> Whenever we have a build phase that ends with a call (for example, to
> substitute, chdir, symlink, etc) that returns an unspecified value, we
> append a #t so that the return value is a boolean. However, the build
> system, as it stands currently, does not mind an unspecified value, and
> treats it as a success. As a result, forgetting to add a #t at the end
> of custom phases is a common mistake. To fix this, I have submitted a
> patch at https://debbugs.gnu.org/cgi/bugreport.cgi?bug=29745 that
> modifies the build system to reject unspecified values as
> failures.
>
> However, IMO, the addition of #t at the end of certain phases, does not
> contribute anything of value and we should simply be at peace with
> phases returning unspecified values. Am I missing something here?

I agree entirely.

IMO the continuation of a build phase should be:

  (define build-phase-cont
(case-lambda
  (() #t)
  ((ret . _) (and ret #t

I.e. the phase could return 0 values, that's fine, we count as success.
Quite possible if your build phase ends up tail-calling to some
procedure you don't care about which returns 0 values for its own
reasons (arguably nicer than returning "the unspecified value", blah);
currently this doesn't work though.  Similarly for build phases that
return more than 1 value; extra values should be ignored (this is
currently the case).  Making a build phase return a single boolean value
in a language like Scheme is busy-work IMO.

Andy

Re: geiser-xref-callers does not seem to work

[Andy Wingo]

On Sat 16 Dec 2017 08:55, Chris Marusich  writes:

> * Place point on symbol expression->derivation-in-linux-vm on line 203
>   (in the definition of the iso9660-image procedure), and press "C-c <".
>
> When I do this, I receive the following message in the minibuffer:
>
>   No callers found for ’expression->derivation-in-linux-vm’
>
> And yet, if I run "M-x rgrep" on the same symbol, there are obviously
> many call sites:
>
>   ./gnu/system/vm.scm:68:  #:export (expression->derivation-in-linux-vm
>   ./gnu/system/vm.scm:105:(define* (expression->derivation-in-linux-vm name 
> exp
>   ./gnu/system/vm.scm:203:  (expression->derivation-in-linux-vm
>   ./gnu/system/vm.scm:274:  (expression->derivation-in-linux-vm
>
> What might the problem be?  When I run "M-x geiser-doc-symbol-at-point"
> on this symbol, it also claims that there is no documentation
> available.  I feel like maybe I've misconfigured something, but I don't
> know what.

It appears to be a Geiser problem and not a Guile problem:

wingo@rusty:~/src/guix$ ./pre-inst-env guile
GNU Guile 2.2.2
Copyright (C) 1995-2017 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> (use-modules (gnu system vm))
scheme@(guile-user)> expression->derivation-in-linux-vm 
$1 = #derivation-in-linux-vm (name exp #:key system 
linux initrd qemu env-vars guile-for-build single-file-output? make-disk-image? 
references-graphs memory-size disk-image-format disk-image-size)>
scheme@(guile-user)> (use-modules (system xref))
scheme@(guile-user)> (procedure-callers $1)
ERROR: In procedure scm-error:
ERROR: expected a variable, symbol, or (modname . sym) #derivation-in-linux-vm (name exp #:key system linux initrd qemu 
env-vars guile-for-build single-file-output? make-disk-image? references-graphs 
memory-size disk-image-format disk-image-size)>

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
scheme@(guile-user) [1]> ,q
scheme@(guile-user)> (procedure-callers '((gnu system vm) . 
expression->derivation-in-linux-vm))
$2 = (((gnu system vm) # #))

I assume your Scheme files are indeed compiled?

Andy

Re: How to customize the kernel

[Andy Wingo]

On Sat 16 Dec 2017 03:37, Chris Marusich  writes:

> Is anyone building custom kernels?  How are you doing it?  Are you just
> using the "make-linux-libre" procedure anyway, despite the fact that it
> supposedly isn't public?  Did you copy it and just modify it to suit
> your needs?  Or did you do something totally different?

To build upstream kernels I just do a (package (inherit linux-libre)
...) and include a customized (source) pointing to the upstream kernel.
I don't change the config though.  For what it's worth :)

Andy

Re: MIME database

[Andy Wingo]

On Tue 28 Nov 2017 12:23, Alex Vong  writes:

> julien lepiller  writes:
>
>> Le 2017-11-28 10:24, l...@gnu.org a écrit :
>>> brendan.tildes...@openmailbox.org skribis:
>>>
>>> Right, so I think the immediate course of action here would be to patch
>>> GIMP’s MIME data so that it does not register as a PDF viewer.
>>>
>>> If we stumble upon other issues like that, we can fix them similarly.
>>
>> Thanks for investigating that.
>>
>> Inkscape also wants to open pdf files ;)
>>
> On my laptop, the recommended programs include evince, libreoffice draw,
> gimp, inkscape, imagemagick and winebrowser. I am not running guixsd
> though.

I think it's reasonable to want to be able to open PDFs in inkscape or
GIMP (e.g. via the "Open With" menu in a file browser like Nautilus),
just that they shouldn't be the default option.  I think removing the
association would be a not-so-good option; the blessed way to fix this
is apparently to install a set of defaults.

Specifically we should add to this package from gnome.scm to include the
PDF -> evince association:

(define-public gnome-default-applications
  (package
(name "gnome-default-applications")
(version "0")
(build-system trivial-build-system)
(source #f)
(propagated-inputs
 `(("nautilus" ,nautilus)))
(arguments
 `(#:modules ((guix build utils))
   #:builder
   (begin
 (use-modules (guix build utils))
 (let* ((out (assoc-ref %outputs "out"))
(apps (string-append out "/share/applications")))
   (mkdir-p apps)
   (call-with-output-file (string-append apps "/defaults.list")
 (lambda (port)
   (format port "[Default Applications]\n")
   (format port 
"inode/directory=org.gnome.Nautilus.desktop\n")))
   #t
(synopsis "Default MIME type associations for the GNOME desktop")
(description
 "Given many installed packages which might handle a given MIME type, a
user running the GNOME desktop probably has some preferences: for example,
that folders be opened by default by the Nautilus file manager, not the 
Baobab
disk usage analyzer.  This package establishes that set of default MIME type
associations for GNOME.")
(license license:gpl3+)
(home-page #f)))

Possibly we could have a set of defaults for XFCE as well.  See also
https://wiki.archlinux.org/index.php/default_applications#XDG_standard.

Andy

Re: MIME database

[Andy Wingo]

On Tue 28 Nov 2017 10:24, l...@gnu.org (Ludovic Courtès) writes:

> Hello,
>
> (Moving the discussions to guix-devel.)
>
> brendan.tildes...@openmailbox.org skribis:
>
>> I was wondering why my pdf documents were getting opened in Gimp by
>> default instead of Evince, so I investigated xdg-open and found it
>> uses mimeopen as a fallback, and packaged it. Turns out it didn't make
>> a difference and the issue was simply that the generated
>> mimedata.cache in xdg-mime-database had Gimp listed before Evince.
>
> Oh, good catch.  Profile generation is sensitive to package order; so
> indeed, if GIMP comes first, it “wins.”

I had this problem with e.g. Nautilus vs Baobab on directories.
Apparently the right solution is to install a desktop-specific package
that manually does "tie-breaking" for MIME types that have multiple
handlers.  See 96d36f385cb1de83f95dd0404dc2166d6f877389.  You might try
just installing gnome-default-applications, or we might have a similar
package.

Andy

Re: [PATCH] scripts: hash: Add --git option. WIP

[Andy Wingo]

On Thu 23 Nov 2017 04:54, Jan Nieuwenhuizen  writes:

> Hi!
>
> Attached is a patch to get the hash of a git archive without having to
> clean the tree or do a clean checkout.
>
> Using
>
> guix hash -gr .
>
> procudes the same hash as doing something like
>
> git clone . tmp && guix hash -rx tmp && rm -r tmp
>
> I marked it as WIP because while it is already "handy" as it is, I
> consider adding a commit argument and imply --recursive, like so
>
>guix hash --git HEAD
>guix hash --git v0.13
>
> WDYT?

Weird that we have done the same thing :)

https://git.savannah.gnu.org/gitweb/?p=guix.git;a=commit;h=572907daff98a77a4215861a88b81d2f30542c09

Andy

ATI radeon + modesetting

[Andy Wingo]

Hi,

I just upgraded my desktop machine for the first time in about 3
months.  It has this graphics card from about 5 years ago:

   01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] 
Turks PRO [Radeon HD 7570]

After the upgrade, I was experiencing horrible tearing and corrupted
graphics.  (I do install the binary firware blobs, for what that's
worth, but that wasn't the source of the problem.)  This reminded me of
something I had experienced in the past with Intel on laptops, though
with less intense effects:

  https://bugs.freedesktop.org/show_bug.cgi?id=101620

The ultimate problem in that case was that when running an
OpenGL-enabled composited window manager like gnome-shell, some updates
would not be synchronized to the display.  There were various things you
could do to ungarble the display on the application level, but it was
very irritating.  The origin of the problem is that the synchronization
facilities exposed by OpenGL / GLX weren't playing well with the
acceleration architecture in the Intel xorg driver.  To explain this,
some background follows.

Historically there were different acceleration paths for 2D and 3D
graphics on Linux/X.  Some graphics people work to accelerate 3D, and
some do 2D.  If you have an intel card you might remember XAA, EXA, UXA,
SNA, etc; those are all 2d acceleration paths.  Now these days there is
convergence to accelerate 2D in terms of 3D -- i.e. translate 2D
operations to 3D operations, as much as possible.  Then you just need to
have one good generic 3d acceleration backend.

The new "modesetting" xorg driver is just that: a generic xorg graphics
driver incorporating a generic 2D-in-terms-of-OpenGL acceleration
architecture called "glamor".  The modesetting xorg driver needs basic
facilities from the kernel to get the card into the right mode, then it
can hook into the card-specific support for OpenGL/etc provided by Mesa.
One of these kernel facilities is called "modesetting", hence the name
of the driver.  Glamor and the modesetting driver are now shipped as
part of the X server itself, not as separate modules.

Switching from the Intel driver to the modesetting driver was thus a
switch from a separate 2D and 3D accel path to a unified path.
Additionally, the 2d accel path in the intel driver was not very well
maintained.  There had been no release of xf86-video-intel for a couple
years; distros that used that driver (a dwindling number; Debian
switched over last year) used Git snapshots.  The "SNA" acceleration
architecture that the Intel devs came up with was indeed fast, but had
some problems that couldn't be resolved related to buffer
synchronization.  Normally SNA worked fine but for some reason emacs
under a composited WM exhibited corruption, so in guix we switched back
to UXA, an earlier 2D accel architecture, but that was quite slow.  So
in c68c201fdd429140da1c606861c9296b9cb01265, we switched over to prefer
the modesetting driver for Intel cards that are recent enough to have
enough OpenGL support for the glamor pipeline to be usable.

Recalling all of this history, I thought that perhaps a switch to
modesetting could fix my radeon problems as well, and indeed, voilà,
this patch fixes it:

--- a/gnu/services/xorg.scm
+++ b/gnu/services/xorg.scm
@@ -117,7 +117,7 @@ Section \"Files\"
   FontPath \"" font-adobe75dpi "/share/fonts/X11/75dpi\"
   ModulePath \"" xf86-video-vesa "/lib/xorg/modules/drivers\"
   ModulePath \"" xf86-video-fbdev "/lib/xorg/modules/drivers\"
-  ModulePath \"" xf86-video-ati "/lib/xorg/modules/drivers\"
+  # ModulePath \"" xf86-video-ati "/lib/xorg/modules/drivers\"
   ModulePath \"" xf86-video-cirrus "/lib/xorg/modules/drivers\"
   ModulePath \"" xf86-video-intel "/lib/xorg/modules/drivers\"
   ModulePath \"" xf86-video-mach64 "/lib/xorg/modules/drivers\"

Removing the ATI driver from the X server makes modesetting the "best"
available driver; from Xorg.0.log:

  [   465.481] (==) Matched ati as autoconfigured driver 0
  [   465.483] (==) Matched ati as autoconfigured driver 1
  [   465.485] (==) Matched modesetting as autoconfigured driver 2
  [   465.487] (==) Matched fbdev as autoconfigured driver 3
  [   465.489] (==) Matched vesa as autoconfigured driver 4
  [   465.491] (==) Assigned the driver to the xf86ConfigLayout
  [   465.491] (II) LoadModule: "ati"
  [   465.494] (WW) Warning, couldn't open module ati
  [   465.494] (II) UnloadModule: "ati"
  [   465.494] (II) Unloading ati
  [   465.498] (EE) Failed to load module "ati" (module does not exist, 0)
  [   465.498] (II) LoadModule: "modesetting"
  [   465.500] (II) Loading 
/gnu/store/2ka3fa3rkxglhn88pqfv8gpcw0ai8fjk-xorg-server-1.19.5/lib/xorg/modules/drivers/modesetting_drv.so
  [   465.502] (II) Module modesetting: vendor="X.Org Foundation"
  [   465.502]compiled for 1.19.5, module version = 1.19.5
  [   465.504]Module class: X.Org Video Driver
  [   465.504]ABI class: X.Org Video Driver, version 23.0

I think I would like to 

Re: GDM status (again)

[Andy Wingo]

Hi!

On Sun 29 Oct 2017 16:30, l...@gnu.org (Ludovic Courtès) writes:

> Timothy Sample <samp...@ngyro.com> skribis:
>
>> Two months ago, Andy Wingo did a bunch of work on getting GDM working as
>> a display manager for Guix [1]. Unfortunately, Andy had to step away
>> from the task before getting everything working.
>
> Thanks for taking it over!

OMG yes, thank you!!  Excellent work!

>> After clearing that up, I found out that GDM was unable to find the
>> Gnome Shell. This is pretty tricky because Gnome Shell already depends
>> on GDM, so you have to avoid a circular dependency. Other distros (I
>> looked at Debian and Fedora) solve this by slicing out a “libgdm” output
>> from the GDM sources, so that you can build “libgdm” without Gnome
>> Shell, have Gnome Shell depend on “libgdm”, and then have GDM depend on
>> Gnome Shell.
>
> Do we really need a compile-time dependency (GDM in master doesn’t
> depend on gnome-shell, after all), or can we just have a hard-coded
> /run/current-system/bin/gnome-shell somewhere?  Granted, that’s not very
> elegant, but it might be good enough?
>
> Also, one can use GDM without using GNOME, so it would be best if GDM
> didn’t depend on GNOME.

This might be fine (and is what I was trying to do).  However this would
mean that the GDM service should ensure that gnome-shell is there, which
in practice is like having a dependency on GNOME or at least on
gnome-shell.  My understanding is that GDM uses gnome-shell as the login
greeter process, so you really can't use GDM without GNOME -- of course
you can log in to some non-GNOME session but GDM itself will use
gnome-shell.  Given that's the case, might as well depend on gnome-shell
explicitly.

> With this and the other changes you described, it looks like there’s
> already a bunch of patches we could apply.  Would you like to send them?
> :-)

I'd be happy to review.  I'd also be happy to apply them :)  There's no
currently-working thing to break -- we can only get better -- so I think
it's OK to noodle on the problem in master.

> Thanks for the detailed report, and thanks in advance for the patches
> you described!  :-)

Yes, looking forward to the patches :-))

Andy

Re: gnu/services/shepherd.scm:90:2: In procedure allocate-struct: Wrong type argument in position 2: 5

[Andy Wingo]

On Fri 29 Sep 2017 13:21, "Huang, Ying"  writes:

> I use the guix git.  After not upgrade the code for quite some time
> (several months?), today after `git pull`, `guix environment guix`
> report the following error.  How to deal with it?

Remove .go files from your git checkout.  A change in Scheme files
caused a change in layout of objects, but some bits were baked into .go
files, and we are missing the ability to know when .go files need to be
rebuilt.  To work around this, remove .go files from your checkout.

Andy

Re: does a shebang-patched shebang of autotools leak into my 'make dist'?

[Andy Wingo]

On Thu 21 Sep 2017 09:05, ng0  writes:

> If you do:
> git clone https://git.taler.net/gnurl.git
> cd gnurl
> guix environment gnurl -- ./buildconf
> sed 's/\/usr\/bin\/file/\/usr\/bin\/env file/' configure
> guix environment gnurl -- ./configure --enable-ipv6 --with-gnutls
> --without-libssh2 --without-libmetalink --without-winidn
> --without-librtmp --without-nghttp2 --without-nss --without-cyassl
> --without-polarssl --without-ssl --without-winssl --without-darwinssl
> --disable-sspi --disable-ntlm-wb --disable-ldap --disable-rtsp
> --disable-dict --disable-telnet --disable-tftp --disable-pop3
> --disable-imap --disable-smtp --disable-gopher --disable-file
> --disable-ftp --disable-smb
> guix environment gnurl -- make
> guix environment gnurl zip xz perl git -- ./maketgz 7.55.1-4
>
> and the extract the resulting gnurl-7.55.1-4.tar.xz somewhere
> you can cat ltmain.sh:
>
> #!/gnu/store/kpxi8h3669afr9r1bgvaf9ij3y4wdyyn-bash-minimal-4.4.12/bin/sh
> ## DO NOT EDIT - This file generated from ./build-aux/ltmain.in
> ##   by inline-source v2014-01-03.01

This used to be the case but should have been fixed in
92e779592d269ca1924f184496eb4ca832997b12.

Andy

Re: guix system init: failed to get path of `/boot/efi'

[Andy Wingo]

On Mon 18 Sep 2017 10:04, Hartmut Goebel  writes:

> Hi,
>
> For installing a GuixSD in e fresh system I followed the installation
> instructions in the manual. As described in the manual, I did set
>
>   (bootloader (bootloader-configuration
>     (bootloader grub-efi-bootloader)
>     (target "/boot/efi")))
>
> But running
>
> root@gnu ~# ls -d /mnt/boot/efi
> /mnt/boot/efi/
> root@gnu ~# guix system init /mnt/etc/config.scm /mnt
>
> failed with:
>
> populating '/mnt'...
> Installing for x86_64-efi platform.
> /gnu/store/…-grub-efi-2.02/sbin/grub-install: error: failed to get
> canonical path of `/boot/efi'.
>
> Obviously grub-install is not "chroot"-ing to /mnt.
>
> Is this expected behavior? And what is the correct solution?

You need to mount the EFI partition to /boot/efi in the installation
system.  As the manual says in "Preparing for Installation" (perhaps not
clearly enough):

   If you instead wish to use EFI-based GRUB, a FAT32 “EFI System
   Partition” (ESP) is required.  This partition should be mounted at
   ‘/boot/efi’ and must have the ‘esp’ flag set.

Andy

Re: Question about multiple licenses

[Andy Wingo]

On Mon 11 Sep 2017 13:29, Alex Vong  writes:

>>> Well, from what I know about copyright, that isn't the licence of glibc,
>>> which is the sum of all the licences involved, and you'd have to know
>>> how to find them if you didn't just unpack the tarball.  With pack
>>> output in a lot of cases you don't have the information.
>>
>> Right, ‘guix pack’ makes things more complicated—although I would argue
>> that, contrary to Dockerfiles and the like (which nobody seems to
>> complain about), Guix makes it easier to do provenance tracking since
>> there’s an unambiguous source → binary mapping.
>>
> Does 'guix pack' currently included the source that uses to build the
> pack? Will including the source signaficantly increases the size of the
> pack? Or should we add a flag for building a "source pack"?

It does not.  Guix's idea of "source" is larger than copyright's idea of
source I think -- i.e. the compiler doesn't impose additional copyright
concerns on binary products, but it does form part of what Guix
considers to be source.

More concretely... if this is necessary (and I suspect but don't know
that it is,) probably the easiest thing would be for each package to
install a copyright file in its output derivations.  Then a "guix pack"
would include them automatically.  It would be good to symlink/dedup
common copyright files of course, but that can be a later step.

Andy

Re: NetworkManager instead of Wicd in ‘%desktop-services’?

[Andy Wingo]

On Mon 31 Jul 2017 03:35, Kei Kebreau  writes:

> l...@gnu.org (Ludovic Courtès) writes:
>
>> Hello Guix!
>>
>> Shouldn’t we replace Wicd with NetworkManager in ‘%desktop-services’?
>>
>> I’ve given it a try on a GuixSD GNOME installation and it appears to
>> work well, and it’s obviously well-integrated with GNOME, whereas Wicd
>> is a bit hidden (one has to know what to look for) and “foreign.”
>>
>> Thoughts?
>>
>> Now, as someone who doesn’t use GNOME, I wonder if NM would work well
>> for me.  Last time I tried it’s CLI was too low-level to be usable, and
>> I don’t remember seeing a curses interface.  Suggestions?
>>
>> Thanks,
>> Ludo’.
>
> I'm in favor of the change as long as NetworkManager's equivalents to
> wicd-gtk and wicd-curses are installed by default. Mark brought up
> nmtui; that sounds like something we should enable. Also, it would be
> nice to have notifications work (if they don't already) when using
> nm-applet. I hear that this depends on xfce4-notifyd?

It seems like all GNOME users are ready for this change.  I tried it out
locally and it's quite nice.  Patch like this:

diff --git a/gnu/services/desktop.scm b/gnu/services/desktop.scm
index 0509bd8a4..f12fe78e5 100644
--- a/gnu/services/desktop.scm
+++ b/gnu/services/desktop.scm
@@ -810,7 +810,8 @@ with the administrator's password."
 
  ;; The D-Bus clique.
  (avahi-service)
- (wicd-service)
+ (service network-manager-service-type)
+ (service wpa-supplicant-service-type)
  (udisks-service)
  (upower-service)
  (accountsservice-service)

The network manager command also seems to include `nmtui' for a
curses-based interface, and `nmcli' for a command-line interface.  I
don't know if there's an applet or something like that, but of course
that could be installed as part of the xfce-desktop-service package.
Seems to me like network manager could be a good default for Guix
desktop systems.

Andy

Re: ‘core-updates’ is back!

[Andy Wingo]

On Tue 29 Aug 2017 00:01, l...@gnu.org (Ludovic Courtès) writes:

> Yup, I just created a new ‘core-updates’ branch by pushing
> .  Enjoy!
>
> Let’s freeze in one month, say Oct. 1st?

It would be pretty cool if we could fix our O(n^2) problems in search
paths in this core-updates -- basically whenever you go to create an
environment, instead of making e.g. VAR=A:B:C:..., for all VARs
(LIBRARY_PATH, PKG_CONFIG_PATH, etc), instead we make a union directory
Z containing the union of A, B, C, etc and set VAR=Z.  The goal would be
to fix quadratic run-time lookup costs by replacing it with a
compile-time computation.  This applies to many lookups: PATH, -rpath,
etc.

For specific variables we might also have to rewrite some internal
references, for example to replace the -L line in pkg-config files to
point to the union directory.  Dunno if there are more examples or if
PKG_CONFIG_PATH is the only one.

Andy

Re: GSoC final update

[Andy Wingo]

On Tue 29 Aug 2017 09:44, Caleb Ristvedt  writes:

>  I found myself checking the guile reference quite
> frequently. One time I happened upon a part describing the ECMAScript
> implementation (curiosity and all that), and noticed that several times
> it was mentioned how irresponsible the implementor was. That scared me
> probably more than it should have. By all objective criteria, I have
> failed in this project, and it frightens me to think that, nice as the
> community is, I could screw up badly enough to end up a footnote
> somewhere as an example of what not to be. That fear pretty much sums up
> my mental state during the second half of the summer.

LOL I wrote that about myself :-) As in, the ECMAScript implementation
was mostly mine, and its faults were mine, and I documented that and
moved on.  It would be nice to fix some day but it takes time; in the
meantime I just wanted to moderate peoples' expectations.  Apologies if
that language made you feel intimidated, the intention was precisely the
opposite.

Happy hacking! :-)

Andy

GDM status

[Andy Wingo]

Hi!

I gave a poke to getting the GNOME Display Manager (GDM) to work.  The
idea would be to replace SLiM, at least for GNOME workstations.  My goal
is to get native GNOME screen locking to work.  I would also like to get
Wayland to work at some point as well.

The upstream state of GDM right now is a little weird -- I feel like it
was being developed mostly by Fedora GNOME folks who were racing towards
a new future of Wayland and more secure logins but who haven't had time
to collect and document things.  The documentation that exists is
extensive but completely out of date, corresponding to an older
architecture.  But I think the general idea is for GDM to spawn a
limited GNOME session, including GNOME shell.  That is the UI for the
greeter.  The actual PAM authentication is handled by GDM via a worker
process (GdmSessionWorker).  GDM is also involved in screen lock --
specifically it handles unlocking of sessions.  I don't know how screen
lock works yet though.

The current Guix status of GDM is that we have a package and a service,
but the combination doesn't really work.  You can build this VM and
test:

(use-modules (gnu))
(use-service-modules desktop xorg)
(use-package-modules gnome linux)

(operating-system
  (host-name "gnome-test")
  (timezone "Europe/Paris")
  (locale "en_US.utf8")

  (bootloader (bootloader-configuration
   (bootloader grub-bootloader)
   (target "/dev/sda")))
  (file-systems (cons* (file-system
 (device "root")
 (type 'label)
 (mount-point "/")
 (type "ext4"))
   %base-file-systems))

  (packages (cons strace %base-packages))

  (services (cons* (gnome-desktop-service)
   (gdm-service)
   (filter (lambda (x)
 (not (eq? (service-kind x) slim-service-type)))
   %desktop-services

I have the GDM shepherd service marked as (auto-start #f), so you'll
have to go in manually and do "herd start xorg-server" as root.  What
happens is that GDM blanks the screen, starts a new session, then
somehow fails, blinks to the terminal, and restarts.  I have a really
hard time debugging it; I spent two or three days on it!

My biggest problem was being able to get out good log information.  With
the VM I wasn't able to launch multiple VT's, so I couldn't tail
/var/log/debug from other terminals; and trying to do anything from
terminal 1 was impossible, and of course you have no scrollback there.
Literally at one point I was taking dozens of screenshots in a row,
hoping one of them captured the part when the VT wasn't blank.

I got the same results on "bare metal", with the unhappy bonus that
other terminals slowed down and stopped processing input events, to the
point that I could barely log in.

In theory the configuration file that I write should cause GDM to log
more information to syslog but I am not sure if it's working.  I got the
best results by patching out the part of GDM that enables logging to
syslog, instead logging to the terminal, and setting the
G_MESSAGES_DEBUG=all environment variable (so that I would get g_debug()
output).

So that's where GDM is.  I need to step back for a while, and I would be
happy if anyone else wants to take a stab at it :)  I feel like it's
actually not that far away.  Godspeed!

Andy

Re: GRUB EFI installation breakage

[Andy Wingo]

On Wed 23 Aug 2017 00:25, l...@gnu.org (Ludovic Courtès) writes:

> Marius Bakke  skribis:
>
>> Ludovic Courtès  writes:
>>
>>> Hello,
>>>
>>> To reconfigure my system on UEFI, I had to apply this patch:
>>>
>>> modified   gnu/bootloader/grub.scm
>>> @@ -401,7 +401,8 @@ submenu \"GNU system, old configurations...\" {~%")
>>>  ;; root partition.
>>>  (setenv "GRUB_ENABLE_CRYPTODISK" "y")
>>>  (unless (zero? (system* grub-install "--boot-directory" install-dir
>>> -"--efi-directory" efi-dir))
>>> +;; "--efi-directory" efi-dir
>>> +))
>>>(error "failed to install GRUB (EFI)")
>>>  
>>>
>>> Before that ‘grub-install’ would fail because ‘efi-dir’ would actually
>>> be “/dev/sda”, which is what I have in the ‘device’ field of
>>> ‘grub-configuration’.
>>>
>>> Removing the “--efi-directory” solves the problem because ‘grub-install’
>>> automatically determines that the EFI directory is mounted at /boot/efi.
>>>
>>> I think 2941b347b664a3d3114de0ac95e28db78db66144 is bogus because it
>>> assumes that the second argument of the gexp’d lambda is ‘efi-dir’,
>>> where in fact it is the ‘device’ field of the bootloader config.
>>>
>>> So what is the preferred fix?  Simply remove “--efi-directory” like I
>>> did above, and rename ‘efi-dir’ to ‘device’ to avoid the ambiguity?
>>> Thoughts?
>>
>> Maybe we could rename "device" to something like "target" and update the
>> documentation to mention that "target" means the _mounted_ EFI System
>> Partition for grub-efi, but is typically a block device.
>
> You mean it’s a block device except for EFI, right?
> So yes, we can rename ‘device’ to ‘target’, and we should at least
> document what you wrote.

Hi!  Sorry for causing problems.  For EFI, we shouldn't specify a block
device AFAIU -- if anything, we should specify the location of the EFI
directory (though it defaults to /boot/efi).  My intention was to change
the meaning of "device" for EFI to be the mounted EFI directory.  But
really I think we should recommend getting rid of this configuration
option, at least in our EFI examples, as a default mount path of
/boot/efi will be good enough I think.

I can update the doc and examples -- will do that now.

Andy

Re: Docker export: add /etc/environment?

[Andy Wingo]

On Mon 21 Aug 2017 12:02, Ricardo Wurmus  writes:

> Especially for “guix pack -f docker” it makes sense to me to include a
> default “/etc/environment” that includes things we set up in GuixSD by
> default, such as GUIX_LOCPATH, and that sources the packed profile’s
> “etc/profile”.
>
> What do you think?

Go for it!! :)

Andy

Re: MATE applications only fit for a running MATE Environment?

[Andy Wingo]

On Mon 14 Aug 2017 22:12, ng0  writes:

> user@abyayala ~$ mate-terminal
>
> (mate-terminal:841): GLib-GIO-ERROR **: Settings schema 'org.mate.interface' 
> is not installed

I believe you need to use glib-or-gtk-build-system instead of
gnu-build-system.  That will make sure that binaries built by your
packages have wrappers that set environment variables ensuring that the
program's schemas are visible to the program.

Andy

Re: Grafting fails for latest Go release candidate

[Andy Wingo]

On Mon 07 Aug 2017 22:08, Marius Bakke  writes:

> I've caught a couple other elusive grafting errors:
>
> grafting '/gnu/store/ccw7wzh9rbflc0fl968dbj2x0x9dn4y5-chromium-60.0.3112.90' 
> -> '/gnu/store/b3f2d6l1c546xwcv4hg8619qh87cixhb-chromium-60.0.3112.90'...
> ERROR: Wrong type to apply: "der */\n  max-width: 638px;\n  min" 

This looks like a Guile 2.2 GC error somehow :-(

Andy

Re: compiling guix is too slow?

[Andy Wingo]

On Sat 01 Jul 2017 15:33, l...@gnu.org (Ludovic Courtès) writes:

> Leo Famulari  skribis:
>
>> My understanding is that Guile 2.2 traded slower compilation for faster
>> execution of compiled programs. Hopefully the performance of the
>> compiler will be improved in later updates to Guile.
>
> Yes, that’s a good summary.
>
> Most of the code we compile is package definitions that don’t benefit
> from all the fancy optimizations, so build-aux/build-self.scm (run by
> ‘guix pull’) turns those off.
>
> Unfortunately even with this change compilation is slower than with
> Guile 2.0, and IMO unacceptably slow for ‘guix pull’ (it takes only a
> few minutes at most on my laptop, but that’s a lot if we compare to
> ‘apt-get update’.)
>
> Andy, do you have other approaches in mind that we could explore to
> improve on this?  Evaluating all of gnu/packages instead of compiling it
> AOT would probably be bad for memory and CPU consumption.

I agree that this is a correct summary.  I think it's possible that even
with improvements that 2.2 would remain slower than 2.0 to compile, but
the current situation is clearly not good and needs improvement.  I
think we need to revisit that synthetic python.scm test you had (can't
find it atm), see where the compiler is spending time, and focus effort
there.

As far as what Guix can do -- there I don't know :/  Nothing immediate
comes to mind.

I would like to help here but can't promise much over the summertime...

Andy

Re: Xorg tearing fix on Intel HD Graphics 4000

[Andy Wingo]

On Tue 27 Jun 2017 19:43, Mark H Weaver <m...@netris.org> writes:

> Andy Wingo <wi...@igalia.com> writes:
>
>> On Wed 21 Jun 2017 09:55, Roel Janssen <r...@gnu.org> writes:
>>
>>> For a long time now, I have a tearing issue on GuixSD (parts of the
>>> screen do not get updated while others do, resulting in dissapearing
>>> text in Emacs).
>>
>> I wonder if it makes sense to update the intel driver to see if it fixed
>> it?  Then perhaps we could avoid the hack.  There are commits here that
>> touch SNA and things that look like they could be relevant:
>>
>>   https://cgit.freedesktop.org/xorg/driver/xf86-video-intel/log/
>
> It's a good idea, but we're already using HEAD (6babcf1) from their
> repo, and have been since June 11 (guix commit e2de6bbdd5).  I can
> confirm that this didn't solve the problem for me.
>
> In the meantime, how would you feel about pushing the following commit
> to master?  It would still allow the use of SNA by explicitly asking for
> it in xorg.conf, but auto-configuration would choose UXA by default.
>
> What do you think?

Looks good to me.  We should report the problem though... ok I did that
here:

  https://bugs.freedesktop.org/show_bug.cgi?id=101620

Looking forward to having a normal Emacs again :)

Andy

Re: RPC performance

[Andy Wingo]

On Mon 26 Jun 2017 13:54, ludovic.cour...@inria.fr (Ludovic Courtès) writes:

> Andy Wingo <wi...@igalia.com> skribis:
>
>> Hi!
>>
>> On Fri 23 Jun 2017 11:24, ludovic.cour...@inria.fr (Ludovic Courtès) writes:
>>
>>> With the current protocol, often we’re just reading a handful of bytes.
>>> Full buffering would mean that Guile would block on an 8K read or so
>>> that will never be fulfilled.
>>
>> That's not how it works :)  The "read" function of a port should only
>> block if no byte can be read.  If 1K bytes are available for an 8K
>> buffer, then the read function should return after filling only 1K
>> bytes; looping to fill at least 8K is some other code's responsibility.
>
> I must be missing something.  With full buffering, when my code does:
>
>   (get-bytevector-n port 8)

Note that get-bytevector-n will block until there are 8 bytes.

> I see read(2) hanging on an 8K read:
>
> #0  0x7fb0b36baaed in read () at ../sysdeps/unix/syscall-template.S:84
> #1  0x7fb0b3b91c47 in fport_read (port=, dst= out>, start=, 
> count=8192) at fports.c:604
> #2  0x7fb0b3bbed77 in scm_i_read_bytes (port=port@entry=0x194f700, 
> dst=0x195c000, start=start@entry=0, 
> count=8192) at ports.c:1544
> #3  0x7fb0b3bc25fe in scm_fill_input (port=port@entry=0x194f700, 
> minimum_size=1, minimum_size@entry=0, 
> cur_out=cur_out@entry=0x7ffd7eee5f30, 
> avail_out=avail_out@entry=0x7ffd7eee5f38) at ports.c:2677

Here this indicates that the buffer is empty, and that it's blocking on
receiving *any* bytes at all.

> (That’s not Guile-specific.)  I agree that read(2) could return less
> than 8K and not block, but it doesn’t have to.

I think this is incorrect.  Read returns when it has any bytes at all.
>From read(2):

   It is not an error if this number is smaller than the number of
   bytes requested; this may happen for example because fewer bytes
   are actually available right now (maybe because we were close to
   end-of-file, or because we are reading from a pipe, or from a
   terminal), or because read() was interrupted by a signal.  See
   also NOTES.

In short the reason read is blocking for you is that there are no bytes
available -- if there were 8 bytes and only 8 bytes, the read(2) would
return directly.

If you have blocking problems related to 8K buffers, it's likely related
to using get-bytevector-n inside CBIP read() functions.

Andy

Re: Xorg tearing fix on Intel HD Graphics 4000

[Andy Wingo]

On Wed 21 Jun 2017 09:55, Roel Janssen  writes:

> For a long time now, I have a tearing issue on GuixSD (parts of the
> screen do not get updated while others do, resulting in dissapearing
> text in Emacs).

Thank you for this work!  I notice this since a month ago and haven't
had time to delve into it.  Horrible stuff.

I wonder if it makes sense to update the intel driver to see if it fixed
it?  Then perhaps we could avoid the hack.  There are commits here that
touch SNA and things that look like they could be relevant:

  https://cgit.freedesktop.org/xorg/driver/xf86-video-intel/log/

As you probably know, the intel driver doesn't get proper releases :(
You just have to take a tag from git.  That's what we do now and perhaps
updating the driver could fix this bug.

Andy

Re: RPC performance

[Andy Wingo]

Hi!

On Fri 23 Jun 2017 11:24, ludovic.cour...@inria.fr (Ludovic Courtès) writes:

> With the current protocol, often we’re just reading a handful of bytes.
> Full buffering would mean that Guile would block on an 8K read or so
> that will never be fulfilled.

That's not how it works :)  The "read" function of a port should only
block if no byte can be read.  If 1K bytes are available for an 8K
buffer, then the read function should return after filling only 1K
bytes; looping to fill at least 8K is some other code's responsibility.

In particular, "read" functions should not use get-bytevector-n, as
get-bytevector-n is defined to block until N bytes are available.
Instead they should use get-bytevector-some.  See:

  
https://git.savannah.gnu.org/cgit/guile.git/commit/?h=stable-2.2=0c102b56e98da39b5a3213bdc567a31ad8ef3e73

Andy

Re: RPC performance

[Andy Wingo]

On Thu 22 Jun 2017 18:05, ludovic.cour...@inria.fr (Ludovic Courtès) writes:

> Andy Wingo <wi...@igalia.com> skribis:
>
>> Why not just set to _IOFBF and let Guile 2.2's buffering handle it?
>
> Because we want controlled buffering when writing (we need to flush
> pending output when we’re done writing the RPC request), and no
> buffering at all when reading.

For controlling output buffering, there is the setvbuf buffer size, and
"force-output".  In Guile 2.2 the CBOP's "write" function is really a
"flush" function -- it only gets called when the internal buffer is
filled, or when flush-output is called, or (for line-buffered ports)
when a newline is written.

Why do you not want buffering when reading?  Do you need to hand off
this FD to some other process?

Andy

Re: RPC performance

[Andy Wingo]

On Mon 19 Jun 2017 10:15, ludovic.cour...@inria.fr (Ludovic Courtès) writes:

> +(define (buffering-output-port port buffer)
> +  ;; Note: In Guile 2.2.2, custom binary output ports already have their own
> +  ;; 4K internal buffer.
> +  (define size
> +(bytevector-length buffer))
> +
> +  (define total 0)
> +
> +  (define (flush)
> +(put-bytevector port buffer 0 total)
> +(set! total 0))
> +
> +  (define (write bv offset count)
> +(if (zero? count) ;end of file
> +(flush)
> +(let loop ((offset offset)
> +   (count count)
> +   (written 0))
> +  (cond ((= total size)
> + (flush)
> + (loop offset count written))
> +((zero? count)
> + written)
> +(else
> + (let ((to-copy (min count (- size total
> +   (bytevector-copy! bv offset buffer total to-copy)
> +   (set! total (+ total to-copy))
> +   (loop (+ offset to-copy) (- count to-copy)
> + (+ written to-copy
> +
> +  (let ((port (make-custom-binary-output-port "buffering-output-port"
> +  write #f #f flush)))
> +(setvbuf port _IONBF)
> +port))
> +

Why not just set to _IOFBF and let Guile 2.2's buffering handle it?

Andy

Re: (guix git) and guile-git finalizers.

[Andy Wingo]

Hi :)

Just some code review.

On Mon 19 Jun 2017 18:01, Mathieu Othacehe  writes:

> The idea here is to create a guardian per pointer-type to finalize. A
> pumper function that operates on this guardian is also created. This
> pumper function knows the git_libgit2_xxx function to call to free the
> pointers stored in guardians.

Neat.  A couple problems though.

  (1) All finalizable objects will be kept alive until they are removed
  from the guardian.  With guardians, you usually want to incrementally
  visit it during program execution: say, every time you allocate a new
  repository object.  Or via after-gc-hook.

  (2) Whether an object is visible in the guardian is not a very
  deterministic property.  It won't be returned from a guardian until
  all other references are gone, but there's no guarantee about when it
  will be returned -- depends on GC and many other factors.  So it's
  possible that repository objects are still "alive" after you shut down
  libgit.

Why do you feel the need to "shut down" libgit?  Honestly I would punt.
Most other libraries aren't like that.

Andy

Re: (guix git) and guile-git finalizers.

[Andy Wingo]

On Mon 12 Jun 2017 14:44, Mathieu Othacehe  writes:

> Maybe the right answer is to disable auto finalization and call
> run-finalizers as described here :
>
> https://www.gnu.org/software/guile/manual/html_node/Foreign-Object-Memory-Management.html#Foreign-Object-Memory-Management
>
> However I can't find guile counterparts for those libguile functions.

Nice sleuthing!

If you want to manually handle finalization at 'safe' points from
Scheme, guardians are the things to use.  Add your "finalizable" object
to a guardian, and periodically "pump" the guardian to see if anything
is finalizable; you can run the finalizer there.

Andy

Re: shepherd -> guile 2.2?

[Andy Wingo]

On Tue 16 May 2017 10:42, Ricardo Wurmus  writes:

> ;;; WARNING: compilation of 
> /gnu/store/5zx29y44nrqj0s8h3jlvlj82k8hj4dxs-guile-2.2.2/bin/guild failed:
> ;;; ERROR: failed to create path for auto-compiled file 
> "/gnu/store/5zx29y44nrqj0s8h3jlvlj82k8hj4dxs-guile-2.2.2/bin/guild"

This is because the build does not set XDG_CACHE_HOME.  See:

  https://lists.gnu.org/archive/html/guix-devel/2017-02/msg00939.html

> Some deprecated features have been used.  Set the environment
> variable GUILE_WARN_DEPRECATED to "detailed" and rerun the
> program to get more information.  Set it to "no" to suppress
> this message.

Here you will get more info if you set the
GUILE_WARN_DEPRECATED=detailed variable.

> shepherd/support.scm:209:2: warning: non-literal format string

Not a deal-breaker; worth having a look at though.

> shepherd/runlevel.scm:106:6: warning: possibly unbound variable 
> `next-services'

Something to look into.

> shepherd/service.scm:606:8: warning: possibly unbound variable `apply-to-args'

Replace with "apply".

> And lastly: should we keep a guile-2.0 variant of the shepherd, or is it
> fine to push this tiny change?

I would say don't bother keeping it.

Andy

Re: Guix publish pipe error from #{read pipe}#

[Andy Wingo]

On Tue 16 May 2017 08:29, Pjotr Prins  writes:

> On a recent guix-publish server I get many of these
>
>   gzip: stdin: invalid compressed data--format violated
> r-annotationdbi-1.36.0
> 1.7MiB/s 00:02 | 3.0MiB transferredguix substitute: error: corrupt
>   input while restoring
> 
> '/gnu/store/g33ns71m61zs5sn758smmc4lnv6h4c4q-r-annotationdbi-1.36.0/site-library/AnnotationDbi/extdata/HG-U95Av2_probe_tab.gz'
>  from #{read pipe}#
>
> I don't think this is a guix problem per se. I have a hunch the people
> changed the firewall which may cause this.  Anyone any other ideas?

IME this means that the .nar or whatever was truncated by some
intermediate cache.  Like if you wget the file and try to gunzip it, it
will give the same error AFAIU.

Andy

Re: guix pull timing out on low resource servers

[Andy Wingo]

On Mon 15 May 2017 12:12, ng0  writes:

> Since the switch to Guile 2.2 and guix pull taking much more
> computing time and resources, all my virtual hosting servers
> time out when I run guix pull on them without using mosh
> and just openssh, because the process just takes much too
> long.
>
> I doubt I'm the only one who experiences this, and just
> increasing the server specs for this process is pointless
> for what they run.
> It's even worse than that, in the future there will be just
> one DNS server running GuixSD, this requires minimal resources.
> Should I just never update these devices or "just get a real
> server"?

I have the smallest DigitalOcean droplet size with a swap partition, and
things work for me I think.  Takes a while though.

There will always be some machines that won't be able to "guix pull"
because of resource constraints.  But I think they are in a minority and
they don't include e.g. the smallest DigitalOcean droplets.

In the meantime we need to rework "build-all.scm" I think to not incur
an O(N) memory usage in the size of the guix package set by
topologically sorting the files (I know there are cycles, but the
general approach should improve things), and by forking off and
compiling the files in separate processes instead of doing everything
from one process (albeit with many threads).  Additionally there are
some compiler speedups in Guile to be had (notably the "basic register
allocation" task from
https://wingolog.org/archives/2016/02/04/guile-compiler-tasks).

And in the meantime-meantime, the workaround is to use a swap file that
is as large as necessary.

Andy

Re: cross-compiling in core-updates

[Andy Wingo]

On Tue 02 May 2017 21:03, Sergei Trofimovich  writes:

> Yay! The following patch makes bash-minimal compile fine!
>
> diff --git a/gnu/packages/bash.scm b/gnu/packages/bash.scm
> index ef22728a9..38aa1786e 100644
> --- a/gnu/packages/bash.scm
> +++ b/gnu/packages/bash.scm
> @@ -213,7 +213,7 @@ without modification.")
>  (arguments
> - (let ((args `(#:modules ((guix build gnu-build-system)
> + (let ((args `(,@(package-arguments bash)
> +   #:modules ((guix build gnu-build-system)
>(guix build utils)
>(srfi srfi-1)
> -  (srfi srfi-26))
> -   ,@(package-arguments bash
> +  (srfi srfi-26)
> (substitute-keyword-arguments args

Excellent!  Thank you and Rennes and Manolis for the excellent bug
sleuthing.

> I'm afraid I understood almost nothing about your comments of 
> visibility. The only thing I've got is that keyword argument order
> matters :)

Sorry for the digression!  I didn't help things by using the term
"keyword" in two different ways :P

> Would the similar ordering change have the no-op effect in master
> branch or does it mean core-updates should do this reordering while
> master should not?

Yes, it should have no effect.  On the other hand I don't know if it
would cause a whole-world rebuild; maybe that would be a reason to not
apply it to master?  I defer to Ludovic here.

Andy

Re: certbot service experience

[Andy Wingo]

On Sat 29 Apr 2017 23:33, Christopher Allan Webber  
writes:

> I'm crossposting this to guix-devel, even though it's in reply to
> guix-patches bug #26685 adding the cerbot service, because I think it's
> more about my experiences with workflow and less about what might affect
> that specific bug.  (If you reply on guix-devel, maybe remove the
> debbugs address.)

Oh, I think this does bear on the service.  Thanks for the detailed
report!

>  - I was surprised that I was prompted for an email while doing guix
>system reconfigure (and in being asked if I was willing to supply
>that info to the EFF so they can put me on their mailing lists, which
>I'm not against but was surprising to encounter in a reconfigure).
>IIRC this is the first time anything like that has happened.  I
>generally anticipate such a reconfigure to be "prompt-less".
>I wonder if there should be an email field we can provide, so that it
>doesn't do this prompt?  Granted, it's for the first time run only.
>Maybe it's okay to do things as they are, but we should document it?
>I dunno?

Yeah I don't know :)  I mean, good that it worked.  I hadn't tested this
bit entirely -- I didn't have a spare source of domain names handy (!).
And I forgot about it asking me for my email.

However apparently what happens is that certbot will prompt you for
things if you run it with an interactive terminal.  We can pass
--no-interactive, but AFAIU that will require you to go later and
"register" your account with an after-step -- and if you miss that you
have to do that, perhaps your cert stops working soon.  Not sure what
the best practice is here.  In any case we need to document this!

>  - Not specifically something to do with this package, but nginx wasn't
>coming up at a few points (btw, default nginx configuration won't
>work, because it points at certificate filepaths that we don't
>"automatically" put in place).  When nginx doesn't come up, there's
>pretty much no information as to why; nothing in the shepherd logs,
>nothing useful from shepherd itself ("shepherd: Service nginx could
>not be started."), and nothing in the nginx logs either.

I think the default nginx configuration works, because it defines no
servers.  If you run just (nginx-configuration), then certbot extends it
correctly to allow it to request a certificate.  You are right that the
default server config does want to listen on HTTPS and it will balk
if/when there is no cert.  (Does that prevent the whole nginx server
from running?  That would be a bummer.)

>  - Getting the certbot stuff to work basically involved three stages:
>
>1) Enable nginx without cert certbot, and point the root somewhere
>   which will be the same place you'll set certbot-configuration's
>   webroot at in the next step.  This looks something like the
>   following:
>
> (service nginx-service-type
>   (nginx-configuration
>(server-blocks
> (list
>  (nginx-server-configuration
>   ;; Replace these with your own domain and web root
>   (server-name '("test.activitypub.rocks"))
>   (root "/srv/activitypub.rocks/site/")
>   ;; Note that you have to disable https and any certificates
>   ;; esp because our nginx config defaults to pointing at keys
>   ;; that probably aren't there.  Without doing this nginx
>   ;; will die mysteriously.
>   (https-port #f)
>   (ssl-certificate #f)
>   (ssl-certificate-key #f))

Pretty sure you could do this with just the default nginx-configuration
and no sites configured and having the certbot service.

>   3) Okay hopefully that went successfully!  It should say.  Assuming it
>  did, *now* we can add the keys appropriately to the nginx config.
>
>(service nginx-service-type
> (nginx-configuration
>  (server-blocks
>   (list
>(nginx-server-configuration
> ;; Again, adjust to your site
> (server-name '("test.activitypub.rocks"))
> (root "/srv/activitypub.rocks/site/")
> (ssl-certificate
>  
> "/etc/letsencrypt/live/test.activitypub.rocks/fullchain.pem")
> (ssl-certificate-key
>  
> "/etc/letsencrypt/live/test.activitypub.rocks/privkey.pem"))

Now it works yes :)

>   4) At this point I was surprised that it seemed like nginx should have
>  been working with https since everything was in place, but I
>  couldn't access it from my browser over https.  Frustrated, I
>  restarted the server.
>
>  And then it worked! :)

Incidentally I found this a few times too.  Like you change the
nginx config but the nginx server never restarts because somehow
shepherd wasn't updated 

Re: cross-compiling in core-updates

[Andy Wingo]

On Fri 28 Apr 2017 21:04, Manolis Ragkousis  writes:

> The reason for the cascading errors is bash-minimal.
>
> Trying to build `./pre-inst-env guix build bash-minimal'
> on core-updates ends up with :
>
> The following derivation will be built:
>/gnu/store/1r0fpfcik796g2b5v9ns163sgan1rkmz-bash-minimal-4.4.12.drv
> @ build-started
> /gnu/store/1r0fpfcik796g2b5v9ns163sgan1rkmz-bash-minimal-4.4.12.drv -
> x86_64-linux
> /usr/local/var/log/guix/drvs/1r//0fpfcik796g2b5v9ns163sgan1rkmz-bash-minimal-4.4.12.drv.bz2
> ice-9/psyntax.scm:1534:32: In procedure expand-macro:
> ice-9/psyntax.scm:1534:32: Syntax error:
> /gnu/store/k84sww1zzh33a5hw8bcmsa5yp7w628a8-bash-minimal-4.4.12-guile-builder:1:2285:
> source expression failed to match any pattern in form (%modify-phases
> phases* (delete (quote move-development-files)))
> builder for
> `/gnu/store/1r0fpfcik796g2b5v9ns163sgan1rkmz-bash-minimal-4.4.12.drv'
> failed with exit code 1
> @ build-failed
> /gnu/store/1r0fpfcik796g2b5v9ns163sgan1rkmz-bash-minimal-4.4.12.drv - 1
> builder for
> `/gnu/store/1r0fpfcik796g2b5v9ns163sgan1rkmz-bash-minimal-4.4.12.drv'
> failed with exit code 1
> guix build: error: build failed: build of
> `/gnu/store/1r0fpfcik796g2b5v9ns163sgan1rkmz-bash-minimal-4.4.12.drv' failed
>
> It's the same issue Sergei reported and commit 78dea6f1d4a did not fix it.

Hi!

I suspect I know what this is.  First, the package in question:

  (define-public bash-minimal
;; A stripped-down Bash for non-interactive use.
(package (inherit bash)
  (name "bash-minimal")
  (inputs '()); no readline, no curses

  ;; No "include" output because there's no support for loadable modules.
  (outputs (delete "include" (package-outputs bash)))

  (arguments
   (let ((args `(#:modules ((guix build gnu-build-system)
(guix build utils)
(srfi srfi-1)
(srfi srfi-26))
 ,@(package-arguments bash
 (substitute-keyword-arguments args
   ((#:configure-flags flags)
`(list "--without-bash-malloc"
   "--disable-readline"
   "--disable-history"
   "--disable-help-builtin"
   "--disable-progcomp"
   "--disable-net-redirections"
   "--disable-nls"

   ;; Pretend 'dlopen' is missing so we don't build loadable
   ;; modules and related code.
   "ac_cv_func_dlopen=no"

   ,@(if (%current-target-system)
 '("bash_cv_job_control_missing=no"
   "bash_cv_getcwd_malloc=yes")
 '(
   ((#:phases phases)
`(modify-phases ,phases
   ;; No loadable modules.
   (delete 'move-development-files

So you are aware of the change in Guile regarding "keyword" matching in
macros.  In this case the "delete" in the modify-phases macro is a
keyword.  Whereas before in Guile, they were only matched by name, now
they are matched by binding.  If the keyword was bound where the macro
was defined, then it will only be matched to identifiers bound to the
same variable when the macro is used.  Otherwise if the keyword was
unbound when the macro was defined, then it will match by name, but only
if the name is also unbound at the macro use.

In (guix build utils), `delete' is bound to "delete" from (srfi srfi-1).
In the bash package definition, I am not sure what the scope is because
it's staged.  However let's assume that the #:modules bit intends to
make it so that srfi-1 is present also in the bash-minimal module.  That
should work, right?  Except I think because the bash package *also*
defines a #:modules argument, that will result in:

  #:modules the-bash-minimal-modules ... #:modules the-bash-modules

and the last keyword wins.  So, perhaps try putting the
,@(package-arguments bash) first in this list?

(#:modules ((guix build gnu-build-system)
(guix build utils)
(srfi srfi-1)
(srfi srfi-26))
 ,@(package-arguments bash))

Andy

Re: potluck status

[Andy Wingo]

Greets,

Just a quick yes-and note:

On Tue 02 May 2017 04:26, Katherine Cox-Buday <cox.katherin...@gmail.com> 
writes:

> Andy Wingo <wi...@igalia.com> writes:
>> & as for dev/testing/prod/etc -- I have no idea :)  I'm not really an
>> ops person, so I can only speculate, and anyone can do that as well as I
>> can :)  I think with guix-potluck.org my main focus is to let people
>> share work-in-progress Guix packages immediately.  I can imagine many
>> ways this could relate to a sort of devopsy workflow but I can't pretend
>> to be an expert here :)
>
> Sorry, I didn't mean to be too dev-opsy. All I really meant is just like
> we have a staging branch/channel now that eventually gets rolled into
> master, we could have a dev branch/channel which is really unstable and
> experimental, a staging branch/channel (which is test in my vernacular),
> and master branch/channel (which is prod in my vernacular).
>
> Potluck seemed appropriate for dev for the same reasons it seemed
> appropriate for scala/sbt.

Oh I didn't mean to disparage this kind of workflow, rather the
opposite!  Apologies if I didn't communicate that right -- I think
there's lots of cool stuff to do in this area, and I look forward to
seeing what people come up with.

As far as having dev and stable channels of Guix itself -- you're right,
having a channel facility could be a step towards Guix itself having a
stable branch and a development branch and perhaps something in-between
too.  Really interesting to explore this space mentally.  In the past,
Guix developers have always said "sure but probably now isn't the right
time" when the topic of production vs development branches was brought
up -- and that could be the answer now too, but when channels are
implemented that certainly does seem like a good mechanism for it.

One thing about Guix packages is that currently every package should
have a unique "bindings" -- I mean, that you can't usefully have a
package bound to variable V in module M, and have different versions of
module M in A/M.scm and B/M.scm.  The potluck facility gives somewhat
unique names M to the modules, "solving" this problem there.  You can of
course have as many (package (name "foo")) values as you like, and Guix
will know about all of them and their versions.  But to be loadable they
need to have different bindings.  This makes channels inappropriate for
directly implementing Guix dev/testing/prod workflows, I think -- you
want to have the same module M, but three versions of it to choose
from.

So implementing a dev/testing/prod thing in Guix would require solving
this problem.  It's not so much a Guix problem as a Guile problem, in a
way, or a mismatch between how Guix wants to think of available packages
and how Guile wants to think about bindings.  Something to solve!

Andy

Re: potluck status

[Andy Wingo]

Hi :)

On Fri 28 Apr 2017 15:41, Katherine Cox-Buday <cox.katherin...@gmail.com> 
writes:

> Andy Wingo <wi...@igalia.com> writes:
>
>> With that I think I'd like to move to a "use" phase where I just sit
>> back and see how people use the thing :)  WDYT?
>
> I am very excited about this functionality for all of the reasons you
> enumerated. I don't have any time at the moment, but I was (and still
> am) interested in getting Scala and sbt packaged, and I think this is
> how I would begin. I could also see packaging a few private work-related
> things as potluck packages for use by myself and others.
>
> A couple of questions:
>
> 1. When Guix grows channel support, will it have a concept of
>dev->test->prod?
> 2. Would it make sense for the dev channel to be expressed in terms of
>potluck packages?
>
> I'm not very active in the community at the moment, but I wanted to
> chime in and say +1, and thanks for the effort.

Great to hear the interest.  There are many many many unknowns at this
point.  The basic "channel" facility could be just like "git remote" --
you do "guix channel add testing https:///testing.git;.  "git
channel pull" or so could update that checkout.  "git channel enable"
makes a channel active for you by default.  I guess you would probably
also want to to specify also a set of active channels for a given guix
command; e.g. "guix package --channels=testing --install my-package".

Concretely you could "git channel add potluck
https://guix-potluck.org/git/target.git; to get the potluck channels.
Currently what you have to do is manually check out that repo and do
"guix package -L /path/to/checkout --install my-package".  So an initial
"guix channel" would just pave that guix-packages-in-a-git-repository
cowpath.

As for Scala and sbt and everything -- I think potluck packages are most
appropriate for "leaf" packages.  For packages that form
"infrastructure" like sbt and all, I think you will probably want to
integrate more closely in Guix.  But I don't know.

& as for dev/testing/prod/etc -- I have no idea :)  I'm not really an
ops person, so I can only speculate, and anyone can do that as well as I
can :)  I think with guix-potluck.org my main focus is to let people
share work-in-progress Guix packages immediately.  I can imagine many
ways this could relate to a sort of devopsy workflow but I can't pretend
to be an expert here :)

Cheers,

Andy

Re: potluck status

[Andy Wingo]

On Fri 28 Apr 2017 14:42, Hartmut Goebel <h.goe...@crazy-compilers.com> writes:

> Am 28.04.2017 um 14:05 schrieb Andy Wingo:
>>   5.15 Invoking ‘guix potluck’
>
> Please think about an other name for this command. "potlouk"  may be
> common to native speakers but I never heard this word. Thanks.

I thought about many things :)  Do you have a suggestion?
https://guix-potluck.org/ does provide the definition.

Andy

potluck status

[Andy Wingo]

Hi!

To recall, "guix potluck" is a packaging facility that is a decoupled
from mainline Guix development.  The latest patches are in wip-potluck,
and a recent copy of the patches is here:

  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=26645

Here is part of the documentation in that branch:

  5.15 Invoking ‘guix potluck’
  

   Note: As of version 0.12.0, this tool is experimental.  The
   interface is subject to radical change in the future.

 Guix is developed as a unified project composed of both the package
  manager and the set of packages.  This allows Guix to evolve while
  remaining healthy and coherent.  If there is a change that needs to be
  done across Guix’s entire tree, Guix developers can make it happen.  One
  way in which this principle manifests itself is in the ‘package’ data
  type, where input packages are directly specified by value in the
  ‘inputs’, ‘native-inputs’, and ‘propagated-inputs’ fields, instead of
  being specified as some abstract package name and version constraint
  that Guix would have to solve for.  *Note package Reference::, for more
  on the ‘package’ data type.

 However it is sometimes desirable to develop a package or set of
  packages in a more decoupled way, for example when a package set is
  still incubating or when a package is inappropriate for sending upstream
  for some reason.  Such packages use Guix and extend Guix but are not a
  part of the Guix project, properly speaking.  As such, they need to be
  resilient to changes in upstream Guix.  It would be brittle if such a
  package definition had to reference a Guix package by value; the Scheme
  variable denoting the upstream Guix package might move to a different
  module, or be renamed, or changed in some unexpected way.

 Guix’s “potluck” facility fills this gap.  A “potluck package” is
  like a normal Guix package, except it expresses its inputs in the form
  of package specifications instead of direct references.  *Note
  potluck-package Reference::.  Potluck packages also have a simpler
  package structure with fewer fields; compared to normal Guix packages,
  they are less expressive but more isolated from details of upstream
  Guix.

 The user interface to potluck packages is concentrated in the ‘guix
  potluck’ command.  To begin, let’s say you are a developer of the ‘foo’
  project, and you’d like to package ‘foo’ for use in your Guix system and
  maybe also that of a friend.  You’re not sure if you want to support it
  yet so you don’t want to make a proper release, but there should be
  something in the middle between that and not packaging it at all.  You
  decide to give ‘guix potluck’ a go.

 So in that git checkout, you run ‘guix potluck init URL’, where URL
  is a publicly accessible git URL at which ‘foo’ is hosted.  ‘guix
  potluck init’ takes the following options:

  ‘--build-system=SYS’
  ‘--build-system=help’
  ‘--autotools’
   Indicate that the package uses the build system named SYS.  Pass
   ‘help’ as the build system to see available options.  ‘--autotools’
   is like the common ‘--build-system=gnu’, but additionally
   indicating that an ‘autoreconf’ step is needed before building.
  ‘--license=LICENSE’
  ‘--license=help’
   Specify the license of the project.

 Calling ‘guix potluck init’ will result in the creation of a
  ‘guix-potluck’ directory in your git checkout, containing a brief
  overview ‘README.md’ file as well as a ‘foo.scm’ potluck package
  definition.  *Note potluck-package Reference::.  Just fill in the
  synopsis and description and add the inputs and you have the beginnings
  of a potluck package.

 You can try building your new package by running ‘guix build -f
  guix-potluck/foo.scm’.  Once that works, you can share the file with
  your friends and they can build your package too.

 Of course, it would be nice if you could share that package with the
  world.  And it would be nice if your potluck package definitions could
  augment the set of available packages and versions.  And it would be
  nice if your potluck package could serve as a first draft of a proper
  Guix package definition.  We agree completely!

 Guix’s potluck facility also implements a kind of registry of potluck
  recipes, as if it were hosting an actual potluck.  This ‘host-channel’
  facility takes requests to add potluck packages and translates that into
  a git repository of all potluck packages, as well as a git repository of
  Guix packages compiled from those potluck packages.

 To inform a channel host of the presence of fresh tasty potluck
  dishes, run ‘guix potluck update URL BRANCH’.  URL should be the URL of
  a git repository containing a ‘guix-potluck’ directory, and BRANCH is a
  ref in that repository.  By default, the request is made to add the
  package to the default ‘guix-potluck.org’ host; pass ‘--host=HOST’ to
  specify an alternate registry.

 Running ‘guix 

Re: Guile 2.2 .go files are larger

[Andy Wingo]

On Mon 24 Apr 2017 10:24, Andy Wingo <wi...@igalia.com> writes:

> On Sat 22 Apr 2017 15:19, l...@gnu.org (Ludovic Courtès) writes:
>
>> The closure of Guix built with 2.0 is 193.8 MiB; when built with 2.2,
>> it’s 311.8 MiB.  Guix itself goes from 66 to 150 MiB:
>>
>> $ du -ms 
>> /gnu/store/jh07pwbyf5dbpdd5q0nvgagqkgmh76nh-guix-0.12.0-9.25a4/lib/guile/2.2
>> 101  
>> /gnu/store/jh07pwbyf5dbpdd5q0nvgagqkgmh76nh-guix-0.12.0-9.25a4/lib/guile/2.2
>> $ du -ms 
>> /gnu/store/rnpz1svz4aw75kibb5qb02hhccy2m4y0-guix-0.12.0-7.aabe/lib/guile/2.0
>> 24   
>> /gnu/store/rnpz1svz4aw75kibb5qb02hhccy2m4y0-guix-0.12.0-7.aabe/lib/guile/2.0
>
> Before we begin, some general notes.  My understanding is that the heap
> usage corresponding to an individual Guix process will be lower, both
 ^
  lower than Guix-on-Guile-2.0, I mean

> due to allocation of read-only data in read-only, shareable sections of
> the .go ELF files, allocation of read-write data in packed sections
> known to the GC but not managed by GC, and various optimizations that
> can eliminate or simplify some heap allocations (closure optimization
> among them).  In short my understanding is that Guile 2.2 (and systems
> built on it) should have a smaller run-time footprint than Guile 2.2.

Er, than smaller footprint than Guile 2.0.

Sorry for the confusion :)

Re: XPATH queries and manipulation in guile?

[Andy Wingo]

On Sun 23 Apr 2017 19:10, Hartmut Goebel  writes:

> Yes, I've seen SXPath [4], but IMHO this usign this would *not* be a
> good choice: It would require packages to learn yet another path
> language, while when using XPath, the packagers could simply copy
> expressions from some fedora .spec-file. Additionally I find the
> documentation of SXpath hard to understand - for be frank: I did not get
> it at all.

The documentation is indeed terrible.  This library is used by other
Schemes; we should see if we can steal their docs, if they are better.

However it's possible to use SXPath with the string syntax.  I seem to
recall there's code out there to do that.  We could incorporate into
Guile to just support the standard syntax.  I think the way forward here
is to improve Guile's sxpath documentation, fixing whatever interfaces
are needed, and adding the xpath->selector function.

Andy

Re: Guile 2.2 .go files are larger

[Andy Wingo]

On Sat 22 Apr 2017 15:19, l...@gnu.org (Ludovic Courtès) writes:

> The closure of Guix built with 2.0 is 193.8 MiB; when built with 2.2,
> it’s 311.8 MiB.  Guix itself goes from 66 to 150 MiB:
>
> $ du -ms 
> /gnu/store/jh07pwbyf5dbpdd5q0nvgagqkgmh76nh-guix-0.12.0-9.25a4/lib/guile/2.2
> 101   
> /gnu/store/jh07pwbyf5dbpdd5q0nvgagqkgmh76nh-guix-0.12.0-9.25a4/lib/guile/2.2
> $ du -ms 
> /gnu/store/rnpz1svz4aw75kibb5qb02hhccy2m4y0-guix-0.12.0-7.aabe/lib/guile/2.0
> 24
> /gnu/store/rnpz1svz4aw75kibb5qb02hhccy2m4y0-guix-0.12.0-7.aabe/lib/guile/2.0

Before we begin, some general notes.  My understanding is that the heap
usage corresponding to an individual Guix process will be lower, both
due to allocation of read-only data in read-only, shareable sections of
the .go ELF files, allocation of read-write data in packed sections
known to the GC but not managed by GC, and various optimizations that
can eliminate or simplify some heap allocations (closure optimization
among them).  In short my understanding is that Guile 2.2 (and systems
built on it) should have a smaller run-time footprint than Guile 2.2.

> Would you have any suggestions to shrink the ELF files a bit?

Why?  Have you compared gzipped or lzipped sizes?  I don't want to put
effort in here that's not worth it :)

Part of it is that our page alignment is 64 kB and so average wasteage
per .go is 32kB, and there are a lot of .go files.  These are just zero
bytes though.

If you look at an individual file, you can use readelf to see things, or
that old ELF visualizer I wrote:

  https://wingolog.org/elf-mapper/

Here's a map of gnu/packages/curl.go:

  https://wingolog.org/pub/elf/elf-JWXYrI.png

I think stripping the .debug_* stuff won't get you much of anywhere.

Stripping arities info could reduce size by 10%.  We could figure out a
different way to represent arities that takes less space.

Compiling all the .go files together (requires Guile hacking) could
remove that per-go 64kB alignment overhead.  Alternately we could do
what GNU tools / ld.so do which maps the linear sequence of bytes in the
file to aligned segments in memory.

Honestly though I would punt.  It's not a run-time issue.  I don't think
it's a transport issue.  It's only a disk space issue.  I personally
don't plan to spend time on it but am happy to point out possibilities
to people that will do so.

Andy

Re: Ready for Guile 2.2!

[Andy Wingo]

On Thu 20 Apr 2017 14:35, l...@gnu.org (Ludovic Courtès) writes:

> l...@gnu.org (Ludovic Courtès) skribis:
>
>> ;; 2.724686s real time, 3.117062s run time.  0.880827s spent in GC.
>> scheme@(guile-user)> (version)
>> $1 = "2.0.13"
>>
> scheme@(guile-user)> ,use(guix scripts build)
> scheme@(guile-user)> ,time (guix-build "libreoffice" "certbot" "xmonad" "-n" 
> "--no-substitutes" "--no-build-hook")
>
> [...]
>
> ;; 1.826528s real time, 1.994426s run time.  0.382750s spent in GC.
> scheme@(guile-user)> (version)
> $1 = "2.2.1"
>
> That’s a 33% speedup compared to 2.0.

That is a 50% speedup compared to 2.0 :)  If we consider its speed as
being how many times you could do this per second, then 2.0 speed is
1/2.72, and 2.2.1 speed is 1/1.82.  Speed ratio is then
2.72/1.82=1.4945.  So 2.2.1 is 1.5x the speed of 2.0, or 50% faster :)

Andy, who is not looking for praise, but who likes perf numbers :)

Re: Extending the mysql configuration

[Andy Wingo]

On Wed 19 Apr 2017 19:45, Christopher Baines  writes:

> I also spotted the define-configuration syntax, which looks like it
> might work well, but I wanted to check if this was definitely a
> direction more services were heading before attempting to write out a
> large part of the supported configuration options.

MHO is this is the direction we should go in.  Having a configuration
defined in a data type that Guix can understand makes it easier to
operate on the system as a whole -- the system can see your mysql
configuration and introspect on it, it's easy to define extension
points, etc.  There are a few services that use this approach and I
expect the number to grow over time.  At the same time many of these
services have the option to fall back on an opaque configuration when
you have special needs or a different workflow -- see dovecot for a very
long example.

Andy

Re: potluck in the house

[Andy Wingo]

On Fri 14 Apr 2017 14:54, l...@gnu.org (Ludovic Courtès) writes:

>> I think we should make it so that the lower-potluck-package process
>> prefers "core" packages if available, and only goes to the channel if
>> the core does not provide a package matching the specification.
>>
>> I think this is a question for the design of channels though: how to
>> resolve dynamically scoped (specification->package) links in the
>> presence of channels.
>
> ‘specification->package’ already transparently handles things in
> GUIX_PACKAGE_PATH, so I suspect it wouldn’t be very different.
>
> WDYT?

I don't know.  I see a danger if person A makes a potluck package for
that depends on Guile, and person B makes a potluck package for some
development version of Guile with a later version number.  Person A (and
person A's users) probably don't expect to be using a development
Guile, so the specifications in person A's package should probably *not*
resolve person B's guile as the "best" one.  See what I'm saying?
Tricky stuff.  A hierarchy of potential sources sounds best to me so
that specification->package only looks for "guile" in the potluck
GUIX_PACKAGE_PATH if it's not found (possibly with the version
constraint) in the "main" GUIX_PACKAGE_PATH.

Andy

Re: potluck in the house

[Andy Wingo]

Hi :)

On Thu 13 Apr 2017 17:42, l...@gnu.org (Ludovic Courtès) writes:

> Turns out Guile-Present is already in Guix proper, but Guile-JPEG isn’t!
> :-)

Oh neat :)

>> (The wip-potluck branch extends guix build -f to recognize potluck
>> packages.)
>
> Oh I see, I was wondering why that was working.  :-)
>
> Maybe we could have ‘guix potluck build’ or something?

We could, but I was getting a bit overloaded with all the config options
you might want (substitutes, grafts, input replacements, etc) so I just
extended "build".  To do so, you just have to make the environment in
which the file is loaded also have the "potluck-package" and
"potluck-source" bindings, and then add another special case along with
package? / derivation? / etc.  WDYT?  OK or not the right thing?

>> Anyway, fine.  So the server received this request, checked out that git
>> repo and branch, looked for potluck packages in guix-potluck/, and added
>> those packages to a "source" git repo:
>
> Woow, that’s the really cool part.
>
>>   (define-module (gnu packages potluck 
>> https%3A%2F%2Fgitlab.com%2Fwingo%2Fguile-present master guile-present)
>> #:pure
>> #:use-module ((guile) #:select (list quote define-public))
>> #:use-module ((guix packages) #:select (package origin base32))
>> #:use-module ((guix git-download) #:select (git-fetch git-reference))
>> #:use-module ((guix licenses) #:select ((lgpl3+ . license:lgpl3+)))
>> #:use-module ((guix build-system gnu) #:select (gnu-build-system))
>> #:use-module ((gnu packages pkg-config) #:select (%pkg-config))
>> #:use-module ((gnu packages texinfo) #:select (texinfo))
>> #:use-module ((gnu packages autotools) #:select (libtool automake 
>> autoconf))
>> #:use-module ((gnu packages gtk) #:select (guile-cairo))
>> #:use-module ((gnu packages guile) #:select (guile-lib guile-2.0)))
>
> Since #:select doesn’t play well with circular dependencies among
> modules, we should probably avoid it for the (gnu packages potluck …)
> modules.

Is this still the case?  With Guile 2.2 we fixed
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=15540 so maybe this isn't
a problem.

> Also, I wonder if this should use ‘specification->package’ for packages
> provided by Guix?

It certainly could, and indeed it could do so for all packages.
Currently it uses specification->package only if it can't resolve the
module in which a package is exported at compile-time.  I think this
probably leads to better run-time overhead when the files are compiled
to .go and a package graph that makes more sense but I don't know.

>> Now this git repo should be able to be added to the GUIX_PACKAGE_PATH.
>> It's effectively a "channel" managed by "guix potluck host-channel"
>> running on guix-potluck.org.  If you have a checkout of this repo, you
>> can build Guile-Present just via "guix build -L /path/to/target/checkout
>> guile-jpeg".  Pretty cool, right??!!!?!?!
>
> Pretty cool, indeed!
>
> The nice thing is that we can pretty much unleash it on Guile users very
> soon!

Guix users I guess you mean? :)  There is of course nothing
Guile-specific in this facility.  But yeah as a solution to the Guile
"CPAN problem", yes definitely!

>> Incidentally here is the configuration for that server:
>
> We should add a potluck server service in GuixSD, too, so that this
> config can be shrinked, and to make it easier to run a potluck server.

Yes, and improve the nginx configuration, and add a fcgiwrap service,
and add a git http service... :)

>>   (2) The host-channel facilities run user-supplied Scheme code.  To do
>>   so safely, that needs sandbox facilities from Guile.  That will
>>   need Guile 2.2.1.  For the time being I have disabled the server.
>
> Given that ‘potluck-package’ is very declarative, I was wondering
> whether it would be simpler to simply interpret it as data (have a
> ‘potluck-sexp->package’ compiler), as opposed to setting up a sandbox
> with a white-list of bindings.  Of course inert data is not as nice as
> something programmable, but in this case it looks like users would not
> really depart from what the default template.  Thoughts?
>
> Anyway, that’s really an “implementation detail.”

Yeah I don't know.  I thought that would maybe be good but then it's
also kinda cool that we provide a capability for abstraction in these
package definitions.  I can imagine interesting extensions given that
it's a programming language rather than data.  And this way we do take
advantage of cheap syntax checking via macros, pushing people to program
Scheme, and it's nice to be able to paste these definitions into a REPL
too... and we certainly want the record type!

I see what you are saying but since it doesn't seem to be necessary I am
tempted to punt, given that we can get Guix-on-Guile-2.2.1 out very soon
(a couple weeks maybe?).

> Now, I realize that none of it is Guile-specific, is it?  So there’s the
> question of the effect it can have on contributions to Guix 

Re: how to "install" guixsd on a digitalocean server

[Andy Wingo]

On Thu 13 Apr 2017 17:28, ng0 <contact@cryptolab.net> writes:

> Andy Wingo transcribed 6.6K bytes:
>> I just "installed" GuixSD on a DigitalOcean droplet.  You can't actually
>> install GuixSD; you have to mutate an existing installation into
>> GuixSD.  But fine.
>
> Could this be applied to linode as well? Or does linode function
> differently?

Hi,

I think it can apply directly to linode as well:

  
http://www.linode.com/docs/tools-reference/custom-kernels-distros/run-a-distribution-supplied-kernel-with-kvm

Apparently there is the option to use DHCP as well, which is nice:

  https://www.linode.com/docs/networking/linux-static-ip-configuration

Andy

potluck in the house

[Andy Wingo]

:Freetype
gfl1.0 
http://www.gust.org.pl/projects/e-foundry/licenses/GUST-FONT-LICENSE.txt
giftware   http://liballeg.org/license.html
gpl1   
https://www.gnu.org/licenses/old-licenses/gpl-1.0.html
gpl1+  
https://www.gnu.org/licenses/old-licenses/gpl-1.0.html
gpl2   
https://www.gnu.org/licenses/old-licenses/gpl-2.0.html
gpl2+  
https://www.gnu.org/licenses/old-licenses/gpl-2.0.html
gpl3   https://www.gnu.org/licenses/gpl.html
gpl3+  https://www.gnu.org/licenses/gpl.html
ibmpl1.0   http://directory.fsf.org/wiki/License:IBMPLv1.0
ijghttp://directory.fsf.org/wiki/License:JPEG
imlib2 http://directory.fsf.org/wiki/License:Imlib2
ipa
http://directory.fsf.org/wiki/License:IPA_Font_License
ischttp://directory.fsf.org/wiki/License:ISC
lgpl2.0
https://www.gnu.org/licenses/old-licenses/lgpl-2.0.html
lgpl2.0+   
https://www.gnu.org/licenses/old-licenses/lgpl-2.0.html
lgpl2.1
https://www.gnu.org/licenses/old-licenses/lgpl-2.1.html
lgpl2.1+   
https://www.gnu.org/licenses/old-licenses/lgpl-2.1.html
lgpl3  https://www.gnu.org/licenses/lgpl.html
lgpl3+ https://www.gnu.org/licenses/lgpl.html
mpl1.0 http://www.mozilla.org/MPL/1.0/
mpl1.1 http://directory.fsf.org/wiki/License:MPLv1.1
mpl2.0 http://directory.fsf.org/wiki/License:MPLv2.0
ms-pl  http://directory.fsf.org/wiki/License:MsPL
ncsa   http://directory.fsf.org/wiki/License:IllinoisNCSA
nmap   https://svn.nmap.org/nmap/COPYING
openldap2.8http://directory.fsf.org/wiki/License:OpenLDAPv2.8
opensslhttp://directory.fsf.org/wiki/License:OpenSSL
opl1.0+http://opencontent.org/openpub/
psfl   http://docs.python.org/license.html
public-domain  http://directory.fsf.org/wiki/License:PublicDomain
qplhttp://directory.fsf.org/wiki/License:QPLv1.0
repoze http://repoze.org/LICENSE.txt
ruby   http://directory.fsf.org/wiki/License:Ruby
sgifreeb2.0http://directory.fsf.org/wiki/License:SGIFreeBv2
silofl1.1  http://scripts.sil.org/OFL_web
sleepycat  http://directory.fsf.org/wiki/License:Sleepycat
tcl/tk http://www.tcl.tk/software/tcltk/license.html
unlicense  https://unlicense.org/
vimhttp://directory.fsf.org/wiki/License:Vim7.2
wtfpl2 http://www.wtfpl.net
x11http://directory.fsf.org/wiki/License:X11
zlib   http://www.gzip.org/zlib/zlib_license.html
zpl2.1 
http://directory.fsf.org/wiki?title=License:ZopePLv2.1

  If your package's license is not in this list, add it to Guix first.

What is this package's license anyway? I look and it's LGPLv3+
apparently.  So now:

  guix potluck init --autotools --license=lgpl3+ 
https://gitlab.com/wingo/guile-present

And voilà:

  Cloning into '/tmp/guix-directory.861r57/git-checkout'...  done.
  Creating guix-potluck/
  Creating guix-potluck/README.md
  Creating guix-potluck/guile-present.scm

  Done.  Now open guix-potluck/guile-present.scm in your editor, fill out its 
"synopsis"
  and "description" fields, add dependencies to the 'inputs' field, and try to
  build with

guix build --file=guix-potluck/guile-present.scm

  When you get that working, commit your results to git via:

git add guix-potluck && git commit -m 'Add initial Guix potluck files.'

  Once you push them out, add your dish to the communal potluck by running:

guix potluck update https://gitlab.com/wingo/guile-present master
  
OK that's interesting!  What I get in the file looks like this:

  ;;; guix potluck package
  ;;; Copyright (C) 2017 Andy Wingo

  ;;; This file is free software: you can redistribute it and/or modify
  ;;; it under the terms of the GNU General Public License as published by
  ;;; the Free Software Foundation, either version 3 of the License, or
  ;;; (at your option) any later version.  No warranty.  See
  ;;; https://www.gnu.org/licenses/gpl.html for a copy of the GPLv3.

  (potluck-package
(name "guile-present")
(version "v0.3.0-14-g3b0d2d9")
(source
  (potluck-source
(git-uri "https://gitlab.com/wingo/guile-present;)
(git-commit "3b0d2d967909dabdd3f86d3a50da79c6b1a8aeb2")
(sha256 "0sr5snj921k91frh5dx1gsqrc3679liydv3b6nwxkl48m7h5mals")))
(build-system 'gnu)
(inputs '())
(native-inputs '("a

Re: how to "install" guixsd on a digitalocean server

[Andy Wingo]

Hi :)

On Fri 07 Apr 2017 16:04, myglc2 <myg...@gmail.com> writes:

> On 04/07/2017 at 14:07 Andy Wingo writes:
>
>> I just "installed" GuixSD on a DigitalOcean droplet.  You can't actually
>> install GuixSD; you have to mutate an existing installation into
>> GuixSD.  But fine.
> [...]
>
> I upgraded Debian to GuixSD on a physical server in a similar way ...
>
> https://lists.gnu.org/archive/html/guix-devel/2016-03/msg00354.html
>
> ... but I used 'guix system init' instead of 'guix system reconfigure'.
>
> I wonder, could that approach have been used in this situation to avoid
> the need to "clean up the remaining Debian bits"?

Neat.  For me the answer is, I don't know :)  I thought with guix system
init you had to do a bunch of partitiony type things?  Certainly if I
had a blank scratch space I could do that.

In hindsight it is something of a miracle that "reconfigure" worked on a
previously non-GuixSD system.  Strange.  I will accept it though :)

Andy

how to "install" guixsd on a digitalocean server

[Andy Wingo]

Hi,

I just "installed" GuixSD on a DigitalOcean droplet.  You can't actually
install GuixSD; you have to mutate an existing installation into
GuixSD.  But fine.

So I installed the latest Debian x86_64 image that they offer.  All fine
and easy.  You tell DigitalOcean your SSH key, then you can SSH directly
to root@your-ip.

Having done this, I proceeded to the binary Guix installation:

wget ftp://alpha.gnu.org/gnu/guix/guix-binary-0.12.0.x86_64-linux.tar.xz
gpg --keyserver pgp.mit.edu --recv-keys 
BCA689B636553801C3C62150197A5888235FACAC
wget ftp://alpha.gnu.org/gnu/guix/guix-binary-0.12.0.x86_64-linux.tar.xz.sig
gpg --verify guix-binary-0.12.0.x86_64-linux.tar.xz.sig

cd /tmp
tar --warning=no-timestamp -xf ~/guix-binary-0.12.0.x86_64-linux.tar.xz
mv var/guix/ /var/
mv gnu/ /
ln -sf /var/guix/profiles/per-user/root/guix-profile ~root/.guix-profile
groupadd --system guixbuild
for i in `seq -w 1 10`; do useradd -g guixbuild -G guixbuild -d /var/empty 
-s `which nologin` -c "Guix build user $i" --system guixbuilder$i; done

Debian uses systemd so as the manual tells me, I did:

ln -s ~root/.guix-profile/lib/systemd/system/guix-daemon.service 
/etc/systemd/system/
systemctl start guix-daemon && systemctl enable guix-daemon

This gave an error for the "enable" part; apparently that doesn't work.
But starting the daemon worked fine.  In the future when I would reboot
the machine I would have to manually do a "systemctl start guix-daemon"
again.

So then:

~root/.guix-profile/bin/guix archive --authorize < 
~root/.guix-profile/share/guix/hydra.gnu.org.pub

But at this point it started carping about locales.  The manual doesn't
mention what you need to do until later on, but what you do is this:

~/.guix-profile/bin/guix package -i glibc-utf8-locales
export GUIX_LOCPATH=$HOME/.guix-profile/lib/locale

Then what init file do you add to for the environment variables?  I
think it's ~/.profile but IDK.  I did this:

echo 'source ~/.guix-profile/etc/profile' >> ~/.profile

However this didn't include the LOCPATH thing, so you have to do this
too:

echo 'export GUIX_LOCPATH=$HOME/.guix-profile/lib/locale' >> ~/.profile

OK.  Only problem now is that ~/.guix-profile/etc/profile has absolute
paths in it, instead of pointers into the user's $HOME.  Humm.

Anyway!  Off to update my Guix:

guix pull

Unfortunately this failed due to lack of memory.  I had installed on the
smallest instance with 512MB memory.  So for now I bumped it up to 1G
and retried and was able to upgrade.  Then:

guix package -i emacs

but I found that was installing X things and cancelled; had to ask on
IRC what was the right thing because guix package --search=emacs showed
too much info for me to find emacs-no-x :)

guix package -i emacs-no-x

OK.  So I edit a new os-config.scm because I'm going to be trying to
install GuixSD.  Here is what I ended up with.  Three notes:

 (1) I used the stock template but you have to customize to use
 "/dev/vda", which is what DigitalOcean gives you.

 (2) DigitalOcean doesn't use DHCP, and instead fills in some static
 information in /etc/networking/interface apparently.  I read that
 and made the static networking config.

 (3) Why do we promote lsh by default?  It took me quite some looking to
 figure out why my authorized_keys wasn't working.  The fix was to
 just use OpenSSH.

(use-modules (gnu))
(use-service-modules networking ssh)
(use-package-modules admin)

(operating-system
  (host-name "guix-potluck")
  (timezone "Europe/Berlin")
  (locale "en_US.UTF-8")

  ;; Assuming /dev/sdX is the target hard disk, and "my-root" is
  ;; the label of the target root file system.
  (bootloader (grub-configuration (device "/dev/vda")))
  (file-systems (cons (file-system
(device "/dev/vda1")
(mount-point "/")
(type "ext4"))
  %base-file-systems))

  ;; This is where user accounts are specified.  The "root"
  ;; account is implicit, and is initially created with the
  ;; empty password.
  (users (cons (user-account
(name "wingo")
(group "users")
;; Adding the account to the "wheel" group
;; makes it a sudoer.
(supplementary-groups '("wheel"))
(home-directory "/home/wingo"))
   %base-user-accounts))

  ;; Globally-installed packages.
  (packages (cons tcpdump %base-packages))

  (services (cons* (static-networking-service "eth0" "46.101.231.54"
#:netmask "255.255.192.0"
#:gateway "46.101.192.1"
#:name-servers '("8.8.8.8" "8.8.4.4"))
   (service openssh-service-type
(openssh-configuration
 

Re: [PATCH] guix hash: -g hashes a git repository

[Andy Wingo]

On Tue 04 Apr 2017 14:21, l...@gnu.org (Ludovic Courtès) writes:

>> +For example:
>> +@example
>> +$ git clone http://example.org/foo.git
>> +$ guix hash -g foo
>> +@end example
>
> In this case -g is equivalent to -rx.

My main use case is when I am in a git checkout that has build products
or other stuff.  Then I can "guix hash -g .".  Easier than making a temp
dir, clone, hash, then delete.

>> +Or even:
>> +@example
>> +$ guix hash -g http://example.org/foo.git
>> +@end example
>>  @end table
>
> This one is indeed simpler.  However, one typically needs to get the
> commit id in addition to the hash, so it seems that in practice, most of
> the time, we’d still need to do:
>
>   git clone http://…
>   cd foo
>   git log | head -1
>   guix hash -rx .
>
> so we have both the commit id and the content hash.
>
> WDYT?

Could be!  Or "git rev-parse HEAD".  I'll do that.

Andy

[PATCH] guix hash: -g hashes a git repository

[Andy Wingo]

* guix/scripts/hash.scm (show-help, %options): Add -g option.
(guix-hash): Support hashing of Git URLs.
* doc/guix.texi (Invoking guix hash): Document guix hash --git.
---
 doc/guix.texi | 18 ++
 guix/scripts/hash.scm | 37 +
 2 files changed, 47 insertions(+), 8 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 8da82b4d8..7d35d9f45 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -5281,6 +5281,24 @@ $ git clone http://example.org/foo.git
 $ cd foo
 $ guix hash -rx .
 @end example
+
+Hashing a git repository is so common that it has its own alias:
+
+@item --git
+@itemx -g
+Clones the git repository at @var{file} into a temporary directory and
+recursively hashes it, excluding the @file{.git} subdirectory.
+
+For example:
+@example
+$ git clone http://example.org/foo.git
+$ guix hash -g foo
+@end example
+
+Or even:
+@example
+$ guix hash -g http://example.org/foo.git
+@end example
 @end table
 
 @node Invoking guix import
diff --git a/guix/scripts/hash.scm b/guix/scripts/hash.scm
index a048b5346..104a6a864 100644
--- a/guix/scripts/hash.scm
+++ b/guix/scripts/hash.scm
@@ -25,6 +25,7 @@
   #:use-module (guix ui)
   #:use-module (guix scripts)
   #:use-module (guix base16)
+  #:use-module (guix utils)
   #:use-module (ice-9 binary-ports)
   #:use-module (rnrs files)
   #:use-module (ice-9 match)
@@ -52,6 +53,9 @@ and 'hexadecimal' can be used as well).\n"))
   (format #t (_ "
   -x, --exclude-vcs  exclude version control directories"))
   (format #t (_ "
+  -g, --git  clone the git repository at FILE and hash it
+ (implies -r)"))
+  (format #t (_ "
   -f, --format=FMT   write the hash in the given format"))
   (format #t (_ "
   -r, --recursivecompute the hash on FILE recursively"))
@@ -68,6 +72,10 @@ and 'hexadecimal' can be used as well).\n"))
   (list (option '(#\x "exclude-vcs") #f #f
 (lambda (opt name arg result)
   (alist-cons 'exclude-vcs? #t result)))
+(option '(#\g "git") #f #f
+(lambda (opt name arg result)
+  (alist-cons 'git? #t
+  (alist-cons 'exclude-vcs? #t result
 (option '(#\f "format") #t #f
 (lambda (opt name arg result)
   (define fmt-proc
@@ -133,18 +141,31 @@ and 'hexadecimal' can be used as well).\n"))
   (negate vcs-file?)
   (const #t
 
+(define (recursive-hash file)
+  (let-values (((port get-hash) (open-sha256-port)))
+(write-file file port #:select? select?)
+(force-output port)
+(get-hash)))
+
 (define (file-hash file)
   ;; Compute the hash of FILE.
   ;; Catch and gracefully report possible '' conditions.
   (with-error-handling
-(if (assoc-ref opts 'recursive?)
-(let-values (((port get-hash) (open-sha256-port)))
-  (write-file file port #:select? select?)
-  (force-output port)
-  (get-hash))
-(match file
-  ("-" (port-sha256 (current-input-port)))
-  (_   (call-with-input-file file port-sha256))
+(cond
+ ((assoc-ref opts 'git?)
+  (call-with-temporary-directory
+   (lambda (dir)
+ (let ((checkout (in-vicinity dir "git-checkout")))
+   (unless (zero? (system* "git" "clone" "--" file checkout))
+ (leave (_ "git clone failed~%")))
+   (pk "git" "clone" file checkout)
+   (recursive-hash checkout)
+ ((assoc-ref opts 'recursive?)
+  (recursive-hash file))
+ (else
+  (match file
+("-" (port-sha256 (current-input-port)))
+(_   (call-with-input-file file port-sha256)))
 
 (match args
   ((file)
-- 
2.12.2

Re: "guix potluck", a moveable feast

[Andy Wingo]

Hi :)

Thanks all for review; comments and suggestions very welcome.  Choosing
this message to reply to.

On Sun 02 Apr 2017 01:05, l...@gnu.org (Ludovic Courtès) writes:

> Andy Wingo <wi...@igalia.com> skribis:
>
>>   (1) Install Guix as a user.  (This needs to be easier.)
>>   (2) guix channel add potluck https://gitlab.com/potluck/potluck master
>>   (3) guix channel enable potluck
>
> So users would see the union of independent potluck “dishes”, right?

Yes I think so: a union of all potluck "dishes" with the Guix package
set as well.

Christopher Webber asks about breakage due to version skew between peer
channels and channels and Guix itself.  I think I would like to just
ignore this problem for now: if you add channels and things break
somehow due to an update in Guix or an update in some channel, then the
workaround is to disable channels until developers fix things.

> The sandbox would have transitive access to a lot of modules; I wonder
> if this might somehow make it easier to escape the sandbox, by
> increasing the attack surface.  For instance,
>
>   (source-module-closure '((guix packages)) #:select? (const #t))

I think the strategy here would be to avoid making a sandbox binding set
that is "unsafe".  Having source-module-closure in that binding set
would seem to make it unsafe.

> I think the server should resolve package specifications when the
> potluck.scm file is submitted, and insert each package in the Guix
> package graph of the moment.  Does that make sense?  Maybe that’s what
> you were describing when you talk about rewriting potluck.scm files
> so?

Yes I think this is a good idea.

Incidentally I am now thinking that all the potluck stuff should be in a
potluck dir; you run "guix potluck init" and it makes

  potluck/README.md
  potluck/mypackage.scm

and the .scm files should evaluate to a single package, like:

  (import-packages ...)
  (package
   ...)

The rewrite would create files like:

  gnu/packages/potluck/gitlab-com-wingo-foo-master/mypackage.scm
  gnu/packages/potluck/gitlab-com-wingo-foo-master/mypackage2.scm

These files would look like:

  (define-module (gnu packages potluck gitlab-com-wingo-foo-master mypackage)
#:pure
;; The sandbox.  We've already verified that the user code works in
;; this sandbox when we rewrite the package, so this allows us to
;; provide a stable language for sandbox packages
#:use-module (guix potluck environment)
;; The individual module imports, resolved by channel manager.
#:use-module ((gnu packages guile) #:select (guile))
...
#:export (mypackage))

  (define mypackage
(package ))

You can compile files from the channel, so guix startup time will be
only minimally affected.

>> There is a particular concern about staging: there is staged Scheme code
>> in these modules that runs inside build processes in guix-daemon.  I
>> don't have any nice solution here.
>
> What’s the problem anyway?  The build environment is a “sandbox” so it’s
> not a problem if staged code attempts to do nasty things.

I guess so, yeah.

Andy

"guix potluck", a moveable feast

[Andy Wingo]

Hi!

Following on thoughts from earlier this month that Guix is the guildhall
that we always wanted, I think I have more of a plan.


s/guildhall/potluck/


Firstly, a name change: the guidhall was a somewhat unified, curated
thing that was Guile-specific, whereas I think I think we need to take
this feast outdoors and move it around a bit.  I would like to use the
term "potluck" to refer to the offering of tasty home-hacked goods, the
composition of which is but loosely planned.  This is inspired by the
"potluck" dishes that we would hack up to celebrate the Guile 2.0
release anniversary.

I don't want to take the metaphor too far of course; I think there was a
forced aspect around the guildhall name that never really became
natural.

Also, the potluck facility is just one part of the system.  Firstly we
use Guix itself as the package manager.  Secondly we build on the
upcoming "guix channel" facility.  (In its most basic form, a channel is
just a git branch somewhere that contains package files.)  The job of
the potluck facility is just cultivating the git branch that is the
"potluck channel".

As an interlude, here is how a user would enter an environment that has
a potluck package "foo" using Guix (using a pack is also possible).  We
start with setup steps:

  (1) Install Guix as a user.  (This needs to be easier.)
  (2) guix channel add potluck https://gitlab.com/potluck/potluck master
  (3) guix channel enable potluck

Then:

  (4) guix environment --ad-hoc foo

Assuming "foo" is a package that's in the potluck channel.  Basically
the potluck channel augments the set of available packages.  This is
handled by the channel mechanism.


A packaging language for stability and security
---

So how do packages enter the potluck channel?  Good question, fictional
reader!  This is the tricky bit.  There are some concerns here:

  (1) The Guix API is not stable and has no plans to be stable.  This
  works great for now because all packages are in one atomic
  repository and people work on making the whole thing make sense
  together.  One of the goals of the potluck effort is to
  decentralize things a bit, so we have an impedance mismatch
  between potluck packages and Guix itself.

  (2) Potluck package definitions will live in many different git
  repositories across the internet, and anyone should be able to
  make a potluck package.  Some potluck package authors will be
  malicious.  They could:

   1. Damage the server that manages the potluck channel

   2. Damage the users that run Guix commands with the potluck
  channel enabled

   3. Damage the users that install potluck packages

  I think we need to forget about 3, for now at least.  (Flatpak
  solves this, more or less; Guix has ongoing work to do here I
  think.)

Both of these large issues point to the need for careful design of the
language that potluck packages are written in.  The language that Guix
packages are written in is inappropriate because of (1).  In particular
we should not depend on which module a package comes from, and what
identifier binds any given package.  For (2), packages are currently
written in full Scheme, staged between the Guix command itself and the
sandbox that runs inside guix-daemon.  Full Scheme might be OK in the
daemon but it's not OK in the Guix command itself.

Concretely I would propose that the language that potluck files are
written in is like this:

  (1) It's code, not inert data.

  (2) It's a subset of Scheme, like core Guix packages.

  (3) The general structure looks like this:

  (import-guix-packages ((guile "guile@2.0")
 (glibc "glibc")))
  (import-potluck-packages ((foo "foo")))

  (define bar
(package
  (name "guile-bar")
  (version "1.0.0")
  (build-system gnu-build-system)
  (inputs `(("guile" ,guile))
  )))

I.e. we have a special mechanism to import packages by name using
specification->package.  We can define packages using Guix's "package"
form, and a number of standard Guix bindings are available to the code
(license:gplv3+, gnu-build-system, etc).

There is a particular concern about staging: there is staged Scheme code
in these modules that runs inside build processes in guix-daemon.  I
don't have any nice solution here.  Simply sandboxing the host Scheme
will be fine enough for me, using (ice-9 sandbox) if that lands soon
(see
https://lists.gnu.org/archive/html/guile-devel/2017-03/msg00111.html).
This requires Guile 2.2.1 (when it is released).

I can see an argument to support more simple package definition,
specifically that it should be easier to build Guile-only projects from
git -- skipping autotools, skipping tarballs.  But we should be able to
support tarball builds containing C or anything else, so the full
package capabilities are needed.  I think a first 

Re: guix is the guildhall that we always wanted!

[Andy Wingo]

On Fri 17 Mar 2017 14:54, Christopher Allan Webber <cweb...@dustycloud.org> 
writes:

> Andy Wingo writes:
>
>> On Thu 16 Mar 2017 23:01, Mark H Weaver <m...@netris.org> writes:
>>
>>> If [Guix] starts encouraging a decentralized approach, that would
>>> result in strong pressure on us to freeze our API, which includes even
>>> such details as which module each package is exported from.  This
>>> would drastically reduce the freedom Guix has to evolve the way its
>>> packages are specified.
>>
>> I get what you are saying.  I think that if a future guildhall is
>> decentralized but uses Guix it needs to minimize its burden on Guix.
>> That could mean that the packages are actually specified in a different
>> DSL with different stability characteristics -- for example that DSL
>> could call specification->package under the hood for example, like
>> Ludovic mentions.  (I should mention that this idea of using Guix and
>> especially all its errors are my own -- haven't talked to others about
>> it yet!)
>>
>> Which module a package definition is in is a good example of something
>> not to depend on.
>
> This makes sense to me... if it really is true that our scheme'y
> Guildhall-style packages are so simple they're more data than code,
> maybe we could even restrict them to... just data.  Just a list of what
> files are being provided, etc.  That could easily be stored in some
> minimal database.

Concretly I would propose something like this in a package.scm in the
git repo:

  (use-modules (guildhall)) ; or whatever we call it
  (import-guix-packages ((a "a")
 (b "b@5.2")))

  (define foo
(guildhall-package
  (name "foo")
  (inputs
   `(("a" ,a)
 ("b" ,b)))
  (git-url "https://github.com/foo/foo;)
  (home-page "https://github.com/foo/foo;)
  (synopsis "Foo is a thing")
  (build-system simple-guile-build-system)
  (description
   "It's a thing")
  (license license:expat)))

I guess you would not want to load this file from the web service as it
has arbitrary Scheme code inside it.  I could see solutions where only
the end-user tool loads this file and exports data to the server, and
the server creates an appropriate "normal" package definition using just
the Guix API.  It can write those definitions out to disk.  It would
export a progressing git repository of Guix package definitions.  As
Guix API changes, that server could re-render those package
descriptions.  Of course that only works in some limited cases.
Otherwise you as a user could do this rendering process for packages you
are developing.  Maybe it is a good thing to encourage nontrivial
packages to go upstream to Guix.

There's not a whole lot that is Guile-specific there of course.  Maybe
this is just an exurb of Guix; I don't know.  Probably "foo" might need
a local build story as well and this doesn't naturally solve that.  More
thinking to do here!

Andy

Re: guix is the guildhall that we always wanted!

[Andy Wingo]

On Fri 17 Mar 2017 12:30, l...@gnu.org (Ludovic Courtès) writes:

>> So!  My proposal for this new "guildhall" would be:
>>
>> 1. a web service
>>
>> 2. on which users registers projects
>>
>> 3. a project is a name + a git repository with a /package.scm file
>>
>> 4. the package.scm contains Guix package definitions for that project
>>
>> 5. the web service administers a git repository collecting those
>>packages
>>- without any hydra.gnu.org overhead
>>- without any manual checks
>>- in a form that you can just check out once and add to 
>> GUIX_PACKAGE_PATH
>>
>> 6. adding a new tag to a project's git repo of the form vX makes a
>>release X and updates the guildhall package
>>- it could be the web service has to poll git repos
>>- or maybe you have to invoke some command to update guildhall
>>
>> 7. probably we need to steal many ideas from npm.org
>
> So are you suggesting that the new guildhall would provide
> Guix-generated packs for those who don’t use Guix, and otherwise regular
> Guix packages?

No, I think the guildhall is simply another source of package
definitions for Guix.  Pack creation would be done by the user as
needed -- if I want to ship packages A, B, and C, I can just do that.
Some of those packages might be defined in Guix proper and some from
guildhall.

The interesting bit I think is simply being able to add to the set of
guildhall packages in a decentralized way, to complement the set of
packages in core Guix.  I imagine the package flow over time would be
from guildhall to core guix.

Andy

Re: grammar usage

[Andy Wingo]

Hi,

I agree that we need to drop this subject.  This will be my last mail in
this thread as well.

On Fri 17 Mar 2017 12:12, John Darrington  writes:

> Sometime ago, due to a misunderstanding I received a series of emails
> addressing me as Frau Darrington.  I did not feel offended or insulted
> at all, and I cannot imagine anyone would feel that way.

I understand you feel this way, however to others the effect is
important.  To trans and non-binary people, being at the receiving end
of a stream of misgenderings (intentional or not) is a constant
irritation, reminding them that they are not welcome in that part of the
world.  We want Guix to be welcoming to everyone, so we should be
attentive to hints about what pronoun people would like others to use
when referring to them.

> Why is it only advocacy of *my* opinion which is unwelcome whereas
> that of others is perfectly acceptable?  I feel that I am being
> victimised here.

To create a welcoming environment for more people and especially people
who are marginalized, we must make some behaviors unwelcome.  In this
case, advocating this opinion of what is correct English contributes to
the marginalization of trans and non-binary people, and for that reason
I think it is appropriate for the Guix project to discourage this
behavior in its spaces.

Andy

Re: grammar usage

[Andy Wingo]

Hello John,

Misgendering a guix contributor is rude.  On the grand scale of things
it is worse than what you perceive to be misuse of language.  Continuing
to advocate for your idea on what good English is is unwelcome behavior,
especially when this advocacy ignores the issue of misgendering.

Andy

Re: guix is the guildhall that we always wanted!

[Andy Wingo]

On Fri 17 Mar 2017 10:01, Florian Paul Schmidt  writes:

> On 03/16/2017 11:24 PM, Ludovic Courtès wrote:
>> I think having repos maintained elsewhere is OKish, but it’s true that
>> it requires people who maintain those repos to follow closely what’s
>> going on in Guix proper because we’re not guaranteeing API stability.
>
> Wouldn't taking the functional/reproducibility aspect one step further
> migitate this issue? I.e. maintainers of decentralized repos just need
> to have as input the precise guix version they are using?

This is certainly possible.  Currently the Guix revision is an implicit
input to everything; a guix package "bar" may depend on "foo@2.1", but
the precise built derivation of "foo" on which you depend will evolve
over time as the dependencies change (glibc for example).

Relying on a fixed Guix version fixes this, but at a cost of adding a
new dimension of maintenance complexity that is not reducible in an
automatic way at run-time.  I explain.

For complexity, in the Guix project itself we have one git repo with N
packages; right now packages for a given revision of git, packages only
depend on each other in that revision.  But if you start to build
systems that have packages depending on packages *at M specific
revisions in Git*, you no longer have N nodes in your graph, you have
N*M.  That's a serious increase in maintenance cost for the system as a
whole (if indeed you do decide to maintain it).

However when you actually instantiate a given application (process) it
is only going to depend on one libc, and one "libfoo".  (I know Rust's
cargo has some facility to import multiple versions of libraries.  It
does so by making the types that those versions work on distinct.  But
in Guile there's no easy facility here.)  So in practice you do have to
resolve the M revisions of Guix into a single revision.  How do you do
that in a system that depends on multiple libc versions?  In many ways
it is better to avoid the issue and have one system that evolves over
time.

When you are building an operating system as a collection of somewhat
independent packages that communicate over shared standard protocols and
not shared ABI, version skew is more manageable.  But when building
composite processes that need to agree on API and ABI internally, the
problem is hard.  A guildhall module needs to be able to compose with
other modules and for that reason I think specifying the Guix revision
is not the right thing.

Just my humble opinion of course :)

Andy

Re: guix is the guildhall that we always wanted!

[Andy Wingo]

On Thu 16 Mar 2017 23:01, Mark H Weaver  writes:

> If [Guix] starts encouraging a decentralized approach, that would
> result in strong pressure on us to freeze our API, which includes even
> such details as which module each package is exported from.  This
> would drastically reduce the freedom Guix has to evolve the way its
> packages are specified.

I get what you are saying.  I think that if a future guildhall is
decentralized but uses Guix it needs to minimize its burden on Guix.
That could mean that the packages are actually specified in a different
DSL with different stability characteristics -- for example that DSL
could call specification->package under the hood for example, like
Ludovic mentions.  (I should mention that this idea of using Guix and
especially all its errors are my own -- haven't talked to others about
it yet!)

Which module a package definition is in is a good example of something
not to depend on.

Basically I think Guix should be able to do what it wants to.  The
stability characteristics that Guix already has are sufficient for a
Guildhall -- no additional maintenance burden intended and I hope no
additional burden imposed.

WDYT?

Andy

Re: guix is the guildhall that we always wanted!

[Andy Wingo]

On Thu 16 Mar 2017 20:26, Amirouche Boubekki  writes:

>> So!  My proposal for this new "guildhall" would be:
>>
>> 1. a web service
>>
>
> What would be the cli interface associated with that web service?
> Some thing like the following will do:
>
>   $ guildhall register

So I think we could re-use the "guild" command; see the original
"guildhall" on how this is done.  I dunno though, maybe that's not
desireable.  If it's not, maybe guildhall isn't the right name?  I don't
know.

> Register a package against the guildhall repository.
> You must have a package.scm in the current working
> directory that follows package specification of guix.
> http://link/to/guix/doc/that/I/dont/know

I think you would have to specify a git repo and it would have to be
public, I think.  (Should the web service mirror the repo?  I don't
know.)

> Does it require login/password?

Yes I think.  Or at least email/token, where the service sends you the
token to your email address.  Dunno.  We could use GPG but that is
complicated too.

>> 2. on which users registers projects
>>
>> 3. a project is a name + a git repository with a /package.scm file
>
> We can have that using guile-git: current commit + remote origin.

Yes :)

>> 4. the package.scm contains Guix package definitions for that
>> project
>>
>> 5. the web service administers a git repository collecting those
>>packages
>>- without any hydra.gnu.org overhead
>
> What do you mean, by no hydra.gnu.org overhead? Where are
> done the 'guix pack' command?

I mean that when a package is part of Guix it becomes part of Guix's
continuous integration and build process, and Hydra will build it all
the time, and it's a burden of sorts onto Guix.  A value as well of
course but a burden too.  So what I am saying here is that we avoid
that.

>>- without any manual checks
>>- in a form that you can just check out once and add to
>> GUIX_PACKAGE_PATH
>>
>
> Ok, the 'guix pack' can be done by the maintainer and
> 'guix register' does upload the lz file to guildhall
> website.

"Guix pack" is orthogonal to this; it's just a nice way to deliver
software.  But for registering a package with the guildhall and its
dependencies and arranging that you can just "git pull" the guildhall
git repo which contains all the package.scm files in the guildhall --
that's what I mean.

>> But!  Who is going to build the guildhall v2.2? :-)
>
> idk ;)

:)

Andy

guix is the guildhall that we always wanted!

[Andy Wingo]

Hi all!

I've always wondered about the right solution for getting Guile users to
share code with each other.  At one point I thought that we would have a
Guile-specific packaging system like CPAN or NPM, but one with GNU
stow-like characteristics.  We had problems with C extensions though:
how do you build one?  Where do you get the compilers?  Where do you get
the libraries?

Guix (https://guixsd.org/) solves this in a comprehensive way.  From the
four or five bootstrap binaries, Guix can download and build the world
from source, for any of its supported architectures.  The result is a
farm of weirdly-named files in /gnu/store, but the transitive closure of
a store item works on any distribution of that architecture.

I got the feeling that Guix was what the guildhall should be long ago,
but something recent made it clear to me.  For background, consider
https://www.gnu.org/software/guix/manual/html_node/Binary-Installation.html,
the binary installation instructions for Guix.  These are directions for
how to install Guix on a "foreign" distro like Fedora.  Basically you
just untar a /gnu/store onto your system, make sure that your /var/guix
has some metadata so that Guix knows what's in the store, and there you
are.

The process of building that weird binary installation tarball was
always a bit ad-hoc though, geared to Guix's installation needs.

But it turns out that we can use the same strategy to distribute
reproducible binaries for any package that Guix includes.  Notably, if
you run:

  guix pack -S /opt/guile-fibers-1.0.0=/ guile-next guile-fibers 
glibc-utf8-locales

Then what you get is a tarball that has Guile, Fibers, and everything
they depend on.  It's safe to extract in / because it only adds files to
/gnu/store, and then it adds a symlink for /opt/guile-fibers-1.0.0.

Crucially, this "guix pack" is really an SDK for Guile and Fibers.
Fibers is a library.  You can now make an SDK for anything!!!  I
mean, I know, that's a lot of exclamation marks, but check it: you can
develop locally, say using

  /opt/guile-fibers-1.0.0/bin/guile -L /path/to/my/code

and it just works.  If you package up your code, you can make a Guix
package out of it and make a "guix pack" out of that closure and
distribute it to your users: REGARDLESS OF WHAT OPERATING SYSTEM THEY
RUN, AND WITH FULL SOURCE REPRODUCIBILITY.  It's huge.  It's the missing
piece of docker: not only do you get space-sharing (installing
guile-fibers-1.0.0 and my-app-built-on-guile-fibers-2.0 can share common
libraries as needed, instead of many huge image files), you also have
the keys to rebuild that image in a standard way.

So:

  * local development using packaged libraries: check.
  * distribution of your work to people without $package-manager: check.
  * easy addition of your code to a common NPM-like registry: ?

For this last bit, we have two options.  One is to add your package to
Guix.  It's relatively easy; here's what I added for Fibers:

(define-public guile-fibers
  (package
(name "guile-fibers")
(version "1.0.0")
(source (origin
  (method url-fetch)
  (uri (string-append "https://wingolog.org/pub/fibers/fibers-;
  version ".tar.gz"))
  (sha256
   (base32
"0vjkg72ghgdgphzbjz9ig8al8271rq8974viknb2r1rg4lz92ld0"
(build-system gnu-build-system)
(native-inputs
 `(("texinfo" ,texinfo)
   ("pkg-config" ,pkg-config)))
(inputs
 `(("guile" ,guile-next)))
(synopsis "Lightweight concurrency facility for Guile")
(description
 "Fibers is a Guile library that implements a a lightweight concurrency
facility, inspired by systems like Concurrent ML, Go, and Erlang.  A fiber 
is
like a \"goroutine\" from the Go language: a lightweight thread-like
abstraction.  Systems built with Fibers can scale up to millions of 
concurrent
fibers, tens of thousands of concurrent socket connections, and many 
parallel
cores.  The Fibers library also provides Concurrent ML-like channels for
communication between fibers.

Note that Fibers makes use of some Guile 2.1/2.2-specific features and
is not available for Guile 2.0.")
(home-page "https://github.com/wingo/fibers;)
(license license:lgpl3+)))

OK, so this uses gnu-build-system, which requires a tarball; we need to
extend this to have a "guile-build-system" for modules that are just
Scheme, in which we just "guild compile" everything.  That way you can
have a repo on gitlab or whatever and you just specify the URL and the
revision and you are done.  I don't know if we can get around specifying
the sha256 when we specify the git revision; probably not a good idea in
light of the SHA1 breakage.  Anyway, that's a thing.

But while getting your package into guix is easier than you think, it's
not automatic.  I think there's room for a 

Re: delete profile

[Andy Wingo]

On Tue 14 Mar 2017 17:43, Federico Beffa  writes:

> Alex Kost  writes:
>
>> Federico Beffa (2017-03-14 09:42 +0100) wrote:
>>
>>> I run 'guix gc' and now I find dangling symlinks in my $HOME.
>>
>> What dangling symlinks?  Just remove them :-)
>
> Of this type:
>
> $HOME/guix-test-profile-XXX-link -> /gnu/store/...
>
> Yes, sure, I deleted them.  It just seems strange that a GC leaves
> dangling links around.

Currently, profiles that aren't in your ~/.guix-profile are not treated
as GC roots (AFAIR).  I think this is a bug -- albeit a known bug. :)

Andy

Re: Introducing ‘guix pack’

[Andy Wingo]

Hey :)

On Tue 14 Mar 2017 14:42, l...@gnu.org (Ludovic Courtès) writes:

> If we remove /var/guix/profiles, users will have to actually type
> /gnu/store/asasdfadfgsadfa-profile/bin/guile.  This is not great, but I
> don’t know what else could be done.  We could profile a
> /bin/guile → /gnu/store/asasdfadfgsadfa-profile/bin/guile symlink, but
> perhaps that’s risky too.

As we were discussing in the channel, maybe it's OK for these binary
installs to claim "/opt/gnu".  Then we expose the profile in /opt/gnu,
so you would run Guile as /opt/gnu/bin/guile.  Additionally you could
actually build against that Guile, which would be pretty neat.  If the
user untars multiple guix packs, /gnu/store easily absorbs the union,
and /opt/gnu will adjoin any new profile directories/files and replace
any overwritten links.

We would have to make sure the union directory in /opt/gnu has all real
directories and only symlink files, as per the recent patch on
guix-devel.

Andy

Re: [GSoC] Development of Cuirass.

[Andy Wingo]

On Sun 12 Mar 2017 15:49, Mathieu Lirzin  writes:

> Here is my proposal for the Google Summer of Code 2017.

Looks great to me.  FWIW I think you may want to use Fibers in Cuirass.
Sometimes a web API request might need to "fork" off a number of tasks,
and Fibers lets you do that pretty easily, and provides nice
communications mechanisms for inter-fiber communication like channels
and condition variables.  It also prevents one long API request from
starving other API users.  Also its abstractions are thread-safe, and it
enables parallel speedups by using all available cores.

Right now Fibers doesn't have explicit support for subprocess events
like child-died, etc, though it can do concurrent access to multiple
pipes at once.  So there's some work to do here.

A reference:

  https://github.com/wingo/fibers/wiki/Manual

Specifically see the "web server" notes in the Examples section.  Fibers
is in Guix as "guile-fibers".

Happy hacking,

Andy

Re: Introducing ‘guix pack’

[Andy Wingo]

On Fri 10 Mar 2017 22:50, l...@gnu.org (Ludovic Courtès) writes:

> I had it on my to-do list and Andy said he’d like to have something like
> that to publish Guile 2.2 binaries: the ‘guix pack’ command below is a
> generalization of the code that builds the Guix binary tarball¹.  It
> creates a bundle of the closure of the given packages, with a profile
> containing all the packages.

Very cool, thank you!!!

> Andy, does this correspond to what you have in mind?

Yes, though I hadn't thought everything out.  I guess my mail question
is about user experience -- this is going to be a gateway for people to
get Guix and Guile and we should make sure there are no rough edges.  I
guess in particular I have a concern about users overwriting their
/var/guix.  (No worries about overwriting /gnu/store of course.)

Particularly if a user installs one "guix pack" then installs another
"guix pack", what happens?  Does the /var/guix tarball include the
sqlite db?  Could it be overwritten?  What if the user had already
installed Guix already; does this silently trash their Guix install?

A bug report: I just tried it but it seems guix pack doesn't respect
--no-build-hook for some reason, and also for some reason on this
machine my Guix daemon fails with "offload: command not found", which I
was getting around via --no-build-hook.

I guess what would be ideal would be:

  cd /
  sudo tar xvf aasdfafasdfjasdaldfhasdfh-guile.tar.xz

and then telling the user to run via
/gnu/store/asasdfadfgsadfa-profile/bin/guile.  That way there is very
little risk of trashing the user's system.

Of course we could also provide them a README of sorts for all the load
paths, but in the end this is a gateway to the real experience :)

I guess for binary installs you would of course want /var/guix, the
database, and the profiles.  I think in that case it makes sense to add
an option to --pack about including them, and have it default to off
(given the potential to trample a user's store).

WDYT?  Any of this make sense? :)

Andy

Re: bug#24544: 2.1.4 tarball install fails on GuixSD

[Andy Wingo]

On Sat 11 Mar 2017 12:33, l...@gnu.org (Ludovic Courtès) writes:

> Andy Wingo <wi...@pobox.com> skribis:
>
>> On Sun 25 Sep 2016 21:22, Jan Nieuwenhuizen <jann...@gnu.org> writes:
>>
>>> ld-wrapper: error: attempt to use impure library 
>>> "/home/janneke/guile-2.1.4/lib/libguile-2.2.so"
>>> collect2: error: ld returned 1 exit status
>>> libtool:   error: error: relink 'guile-readline.la' with the above 
>>> command before installing it
>>
>> This is because Guix is silly and thinks that even when you are
>> installing to /rando/prefix/that/you/like that it's bad to link to
>> thinks outside /gnu/store.
>>
>> The workaround is to export GUIX_LD_WRAPPER_ALLOW_IMPURITIES=yesplease.
>> (Actually any value.)
>>
>> The real fix is to prevent ld-wrapper from carping for normal installs
>> to non-store prefixen!
>
> Do you mean we should change the default?  That is,
> GUIX_LD_WRAPPER_ALLOW_IMPURITIES=yes by default, and gnu-build-system
> would set it to “no”.

I think I didn't know precisely what I meant :) However!  That sounds
like a good idea -- the benefits of the check are only intended for Guix
builds, so builds outside Guix should probably not go through that
check.

Andy

Re: gnu-patches back log

[Andy Wingo]

On Wed 01 Mar 2017 09:17, Pjotr Prins  writes:

> I would like to ask the Guix mailing list members whether it is
> *acceptable* that a good looking patch has not been touched for two
> weeks. Like this one
>
>   https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25725

FWIW -- I accept this situation.  I have limited bandwidth and can't do
everything and am not always in a very Guixy place.  If I felt that I
could not accept this, my life would be much worse -- stress, burnout,
etc.

Hopefully the patch situation will improve over time as more people
become committers, and we improve our processes (for example the tags
from your great suggestions).

Andy