SWH: fallback error

[zimoun]

Hi,

In order to check if the SWH fallback works, time to time, I test it.  I
have a variant package of hello defined here:

   

The source had been archived on SWH long time ago (via their Web
submission or via “guix lint -c archival”).  

Using Guix 16ef7b4, I get this:

--8<---cut here---start->8---
$ guix build -L /tmp/channel-example/ hi -S
The following derivation will be built:
   /gnu/store/bii72cavifd0ll151vjiz3brmdfxjdyp-git-checkout.drv
building /gnu/store/bii72cavifd0ll151vjiz3brmdfxjdyp-git-checkout.drv...
guile: warning: failed to install locale
environment variable `PATH' set to 
`/gnu/store/378zjf2kgajcfd7mfr98jn5xyc5wa3qv-gzip-1.10/bin:/gnu/store/sf3rbvb6iqcphgm1afbplcs72hsywg25-tar-1.32/bin'
hint: Using 'master' as the name for the initial branch. This default branch 
name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint: 
hint:   git config --global init.defaultBranch 
hint: 
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint: 
hint:   git branch -m 
Initialized empty Git repository in 
/gnu/store/884nsva9r8wkp40kbqyvpj1ad57jc5dd-git-checkout/.git/
fatal: could not read Username for 'https://github.com': No such device or 
address
Failed to do a shallow fetch; retrying a full fetch...
fatal: could not read Username for 'https://github.com': No such device or 
address
git-fetch: 
'/gnu/store/5vai7bfrfkzv22dx13bxpszjrqyi78x6-git-minimal-2.33.0/bin/git fetch 
origin' failed with exit code 128
Trying content-addressed mirror at berlin.guix.gnu.org...
Trying content-addressed mirror at berlin.guix.gnu.org...
Trying to download from Software Heritage...
SWH: found revision e1eefd033b8a2c4c81babc6fde08ebb116c6abb8 with directory at 
'https://archive.softwareheritage.org/api/1/directory/c3e538ed2de412d54c567ed7c8cfc46cbbc35d07/'
Backtrace:
   6 (primitive-load "/gnu/store/xq0z66k209klw42wwssgww5mq6y?")
In ice-9/boot-9.scm:
152:2  5 (with-fluid* _ _ _)
In guix/swh.scm:
622:8  4 (call-with-temporary-directory #)
632:5  3 (_ "/tmp/guix-build-git-checkout.drv-0/guix-directory.a?")
543:0  2 (vault-fetch _ _ #:log-port _)
   261:30  1 (call "https://archive.softwareheritage.org/api/1/vaul?; ?)
390:0  0 (json->vault-reply _)

guix/swh.scm:390:0: In procedure json->vault-reply:
In procedure string->symbol: Wrong type argument in position 1 (expecting 
string): #
builder for `/gnu/store/bii72cavifd0ll151vjiz3brmdfxjdyp-git-checkout.drv' 
failed with exit code 1
build of /gnu/store/bii72cavifd0ll151vjiz3brmdfxjdyp-git-checkout.drv failed
View build log at 
'/var/log/guix/drvs/bi/i72cavifd0ll151vjiz3brmdfxjdyp-git-checkout.drv.bz2'.
guix build: error: build of 
`/gnu/store/bii72cavifd0ll151vjiz3brmdfxjdyp-git-checkout.drv' failed
--8<---cut here---end--->8---

The revision is found but then an error.  Hum?

Any clue?


All the best,
simon

Why linux-libre source code is not in sources.json

[zimoun]

Hi,

On Thu, 09 Sep 2021 at 00:00, Ludovic Courtès  wrote:

> Now we have to see what’s available on berlin & co., and the extent to
> which SWH can help with this situation.  So far I’m not sure about the
> tarball contents, but the repo seems to be archived:

As we know, the issue with tarball on SWH is that we cannot guarantee
the map between the information stored at package time and the
information SWH serves at fetch time.  Metadata, etc. so SWH cooking can
return a tarball having the checksum Guix that expects or something
different.

We (at least me :-)) need to invest energy into Disarchive.
Definitively! 

> For some reason, we seem to be exporting only one tarball in
> sources.json currently (the file that SWH periodically reads):
>
> --8<---cut here---start->8---
> $ wget -qO - https://guix.gnu.org/sources.json|jq |grep linux-libre.fsfla.org
> 
> "https://linux-libre.fsfla.org/pub/linux-libre/releases/5.4.20-gnu/linux-libre-5.4.20-gnu.tar.xz;,
> 
> "https://linux-libre.fsfla.org/pub/linux-libre/releases/5.4.20-gnu/linux-libre-5.4.20-gnu.tar.xz;,
> --8<---cut here---end--->8---
>
> We should check why we’re not providing all the URLs and fix it.

Interesting… :-)

The file ’https://guix.gnu.org/sources.json’ is basically built using
’fold-packages’ (see ’all-packages’ in
guix-artwork/website/apps/packages/data.scm) and ’package-source’ (see
’sources-json-builder’ in
guix-artwork/website/apps/packages/builder.scm).  And indeed,
’linux-libre’ is missing.

Let consider this snippet as ’/tmp/foo.scm’ mimicking the builder of
JSON files:

--8<---cut here---start->8---
(use-modules (guix packages)
 (gnu packages))

(define all-packages
  (sort
   (fold-packages (lambda (package lst)
(if (or (package-superseded package)
(package-replacement package))
lst
(cons package lst)))
  '())
   (lambda (p1 p2)
 (string8---

Then indeed, only 5.4.20 is seen.

--8<---cut here---start->8---
$ guix repl /tmp/foo.scm | grep 'linux-libre.fsfla'

#https://linux-libre.fsfla.org/pub/linux-libre/releases/5.4.20-gnu/linux-libre-5.4.20-gnu.tar.xz;
 "ftp://alpha.gnu.org/gnu/guix/mirror/linux-libre-5.4.20-gnu.tar.xz; 
"mirror://gnu/linux-libre/5.4.20-gnu/linux-libre-5.4.20-gnu.tar.xz") 
# () 
7faf439cfba0>
#https://linux-libre.fsfla.org/pub/linux-libre/releases/5.4.20-gnu/linux-libre-5.4.20-gnu.tar.xz;
 "ftp://alpha.gnu.org/gnu/guix/mirror/linux-libre-5.4.20-gnu.tar.xz; 
"mirror://gnu/linux-libre/5.4.20-gnu/linux-libre-5.4.20-gnu.tar.xz") 
# () 
7faf439cfba0>
--8<---cut here---end--->8---

The difference is that this package ’linux-libre-headers-5.4.20’ is
created with ’make-linux-libre-headers’ and the others with
’make-linux-libre-headers*’.  Subtle. ;-)  In other words,

--8<---cut here---start->8---
$ guix repl
scheme@(guix-user)> ,use(guix packages)
scheme@(guix-user)> ,use(gnu packages linux)
scheme@(guix-user)> (package-source linux-libre-headers-5.4.20)
$1 = #https://linux-libre.fsfla.org/pub/linux-libre/releases/5.4.20-gnu/linux-libre-5.4.20-gnu.tar.xz;
 "ftp://alpha.gnu.org/gnu/guix/mirror/linux-libre-5.4.20-gnu.tar.xz; 
"mirror://gnu/linux-libre/5.4.20-gnu/linux-libre-5.4.20-gnu.tar.xz") 
# () 
7f881c1ec4e0>
scheme@(guix-user)> (package-source linux-libre-headers-5.13)
$2 = #> # (#http://www.fsfla.org/svn/fsfla/software/linux-libre/lemote/gnewsense/branches/3.16/100gnu+freedo.patch;
 # () 
7f881c1ec840> #https://salsa.debian.org/kernel-team/linux/raw/34a7d9011fcfcfa38b68282fd2b1a8797e6834f0/debian/patches/bugfix/arm/arm-mm-export-__sync_icache_dcache-for-xen-privcmd.patch;
 # () 
7f881c1ec7e0> 
"/gnu/store/yrqr7syxbm4pddzlgc4pwn9wixmpy9xh-guix-module-union/share/guile/site/3.0/gnu/packages/patches/linux-libre-arm64-generic-pinebook-lcd.patch")
 7f881c1ec780>
--8<---cut here---end--->8---

Therefore, the builder of JSON (mainly ’origin->json’) does not consider
such cases and assume that ’origin-uri’ can be applied.  Well, I will
try to improve the situation if no one beats me. :-)


Cheers,
simon

Re: Rethinking propagated inputs?

[zimoun]

On Thu, 09 Sep 2021 at 00:12, Ludovic Courtès  wrote:
> Maxime Devos  skribis:

>>   (define glib
>> (package
>>   (name "glib")
>>   (outputs '("out" ; everything
>>  "bin" ; glib-mkenums ... depends on Python
>>  "build")) ; glib-2.0.pc
>>   (propagated-inputs
>> `(("pcre" ,pcre #:propagate-for-output "build") ; in the 
>> Requires.private field of glib-2.0.pc
>>   ("libffi" ,libffi #:propagate-for-output "build") ; ditto, for 
>> gobject-2.0.pc
>>   ;; etc.
>>   ))
>>   (native-inputs '()) ; not relevant to this e-ail
>>   ...))
>
> Good’ol ‘ROADMAP’ file has a half-baked proposal along these lines.

I think you mean the ’TODO’ file:



Cheers,
simon

Re: Rethinking propagated inputs?

[Liliana Marie Prikler]

Hi,

Am Donnerstag, den 09.09.2021, 00:12 +0200 schrieb Ludovic Courtès:
> Hi,
> 
> Maxime Devos  skribis:
> 
> > For packages using pkg-config, I had the following scheme in mind:
> 
> As a reminder to everyone, pkg-config is just one of the reasons for
> propagating inputs, probably not the main one numerically speaking.
That is true, pkg-config propagations are merely closest to the "core"
of Guix and thus (have the potential affect to) larger parts, whereas other 
forms are typically – though perhaps not necessarily – limited to some language 
ecosystem or similar.

> [...]
> It would be great if the ‘Requires’ field of .pc files could specify
> absolute file name; it would no longer be necessary to set
> PKG_CONFIG_PATH, and thus propagation wouldn’t be needed in this
> case.
In that regard, would symlinking other pkg-config files as proposed
somewhere in passing in [1] also work?  We would have to add a phase,
that parses the Requires field from installed pkg-config files, and
then symlinks the respective files from the inputs to gnu-build-system, 
but since they would then exist in PKG_CONFIG_PATH and point to the
library in question, theoretically we could get around this limitation
of pkg-config without requiring propagation.

> Regarding outputs, Nixpkgs introduced a “dev” output a while back
> that lumps together our “lib” and “include” outputs, roughly.  I
> think that’s a good idea (for other reasons too).
Naming-wise I'd still prefer the more concrete “lib” as “dev” to me is
a weird umbrella term that doesn't really tell me what I'm getting
(also it probably contributes to the distinction of users and devs, and
as some distros like Debian have demonstrated often becomes
pointless[2] the minute you introduce language bindings which require
dev inputs anyway).  

As far as the lib/include split in Guix is concerned, there appear to
be few packages that split them, so “lib” effectively is “dev” in Guix
when “out” isn't.  So apart from those outliers, is there something our
“lib” outputs are missing that Nix' “dev” outputs include?

Regards

[1] 
https://yhetil.org/guix-devel/997473f47b882a670e4b9bf6fe3fc66e66ba6911.ca...@gmail.com/
[2] Pointless in the sense, that “users” need to install those “dev”
packages anyway if an application requires them.  Consider for example
a Guile application built on guile-git, which therefore requires
libgit2-dev to be installed. 

Re: Can we find a better idiom for unversioned packages?

[Leo Famulari]

I think Ludo meant difftastic:

https://github.com/Wilfred/difftastic

On Wed, Sep 8, 2021, at 18:21, Jonathan McHugh wrote:
> Hi Ludo,
> 
> Just checking:
> 
> Is Diffstatic a real tool? It wasnt quite clear to me (and I fancy 
> finding a new diff tool).
> 
> 
> Jonathan McHugh
> indieterminacy@libre.brussels
> 
> September 8, 2021 11:31 PM, "Ludovic Courtès"  wrote:
> 
> > Hello!
> > 
> > Sarah Morgensen  skribis:
> > 
> >> Currently, there are about 1500 packages defined like this:
> >> 
> >> (define-public sbcl-feeder
> >> (let ((commit "b05f517d7729564575cc809e086c262646a94d34")
> >> (revision "1"))
> >> (package
> >> [...])))
> >> 
> >> I feel like there are some issues with this idiom (in no particular
> >> order):
> > 
> > I’m late to the party but I’ll complement previous answers. :-)
> > 
> >> 1. When converting between this idiom and regularly versioned packages,
> >> the git diff shows the whole package changing because of the indentation
> >> change.
> > 
> > One can use ‘git diff -w’ to work around that (or the newfangled
> > Diffstatic tool.)
> > 
> >> 3. Packages inheriting from it lose the definitions. For actual fields,
> >> we have e.g. `(package-version this-package)`, but we have no equivalent
> >> for these.
> > 
> > Right, these pieces of information are not “first-class”, except in the
> > ‘git-reference’ record (or similar) for the commit ID. Do you have
> > examples in mind where it’s insufficient?
> > 
> > [...]
> > 
> >> 5. The closest thing we have to a standardized way of generating
> >> versions for these packages is `(version (git-version "0.0.0" revision
> >> commit))`. We can do better than that boilerplate.
> > 
> > I can sympathize with the feeling, but I’m not sure what to do. A
> > ‘vcs-version’ record as Maxime proposes seems a bit overkill to me (and
> > it would probably have an impact on performance, build times, and
> > whatnot.)
> > 
> >> 6. Not a direct complaint, but I feel like the overall package interface
> >> was designed before straight-from-vcs unversioned packages were so
> >> common, and so this idiom developed organically to work around that.
> > 
> > Sure, though “straight-from-vcs” and “unversioned” are two different
> > things: I’m fine with the former, but the latter equates to upstream
> > telling its users “go find a revision that works for you”. I think
> > releases still make sense for any non-trivial piece of software.
> > 
> > As noted in the manual (info "(guix) Version Numbers"), packages built
> > from arbitrary commits were supposed to be exceptional. Perhaps the
> > reason we’re having this conversation now is that development practices
> > are evolving towards what looks like chaos. :-)
> > 
> > Thanks,
> > Ludo’.
> 
> 

Transform options: check if applied or not

[zimoun]

Hi Ludo,

This thread is initially about 2 issues:

 1) apply a transformation to a non-existent package,
 2) check if the transformation makes sense or not.

Here, I will comment only about #2.

On Wed, 08 Sep 2021 at 22:55, Ludovic Courtès  wrote:

>> The other issue is to list if the transformation is applied or not.  I
>> think it is possible by traversing again the graph and check if a
>> property appears at least once; well it should be better to warn if
>> the 'mapping-property' is not found at least once.  I had some
>> headaches to implement it... and I moved to other "urgent" stuff. :-)
>
> Hmm the ‘mapping-property’ is not enough.  I think you pretty much have
> to compute the derivations of the new and old packages and compare them.

>From my understanding, the transformation can replace something and then
return a new package but this new package can be the same as the old
one.  I agree that the detection of such cases requires to compute the
both derivation––which is probably too expensive / complex, IMHO.

However, the transformation might replace nothing and so it is sure that
the new and old packages are the same. :-)

Other said,

  a) there is no guarantee that if the new graph contains a least one
replacement then the new package is different than the old one.

But,

  b) there is a guarantee that if the new graph does not contains any
replacement, then for sure the new package is the same than the old one.

For instance,

 a) guix build hello@2.10 
--with-source=hello=https://ftpmirror.gnu.org/gnu/hello/hello-2.10.tar.gz
 b) guix build hello@2.10 --with-latest=python-scipy

and it would be nice to warn for the case b-.  After digging again in
the code, the ’mapping-property’ is probably not the right way, even if
a ’properties’ could be added when the rewrite happens and this
’properties’ can be checked at the end of the traversal.  WDYT?


Cheers,
simon

Well well, I miss a point:

--8<---cut here---start->8---
$ guix build hello@2.10 
--with-source=hello=https://ftpmirror.gnu.org/gnu/hello/hello-2.10.tar.gz
/gnu/store/zfwhbbknkhxi3yqmp0qgh1l1crljgbm6-hello-2.10

$ guix build hello@2.10 
/gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10
--8<---cut here---end--->8---

Where https://ftpmirror.gnu.org/gnu/hello/hello-2.10.tar.gz is an
official mirror.  It was expecting the same store item so why not a
different one.

It is because the derivations are different.  The first call contains
the source tarball as an “input”; that’s fine.  The second call contains
the source tarball as a derivation; again that’s fine.  The corresponding
first and second derivations are:

--8<---cut here---start->8---
Derive
([("out","/gnu/store/zfwhbbknkhxi3yqmp0qgh1l1crljgbm6-hello-2.10","","")]
 
,[("/gnu/store/101ny738l311p4fm3cas7jgnkzagjv6a-module-import-compiled.drv",["out"])

[...]

   ,("/gnu/store/z4s27gwacbw8f38andfsh21r8v330dag-xz-5.2.4.drv",["out"])]
 ,["/gnu/store/8a0wry8cvr405ha8d8bpjyzj5dzghigd-module-import",
 "/gnu/store/chariqd6k0sli3s7vcl4q3al0crirz5v-hello-2.10.tar.gz",
 "/gnu/store/rndq9g8877l29ha41dvsl3aj1z0gw0ng-hello-2.10-guile-builder"]

[...]

--8<---cut here---end--->8---

and

--8<---cut here---start->8---
Derive
([("out","/gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10","","")]
 
,[("/gnu/store/101ny738l311p4fm3cas7jgnkzagjv6a-module-import-compiled.drv",["out"])
   
,("/gnu/store/1a7xfcqcxj0pqi4f81x1agcxa46v2bbm-hello-2.10.tar.gz.drv",["out"])

[...]

   ,("/gnu/store/z4s27gwacbw8f38andfsh21r8v330dag-xz-5.2.4.drv",["out"])]
 ,["/gnu/store/8a0wry8cvr405ha8d8bpjyzj5dzghigd-module-import",
 "/gnu/store/kql8b2hbsabcmany4m3hfm3wzdiymliy-hello-2.10-guile-builder"]

[...]

--8<---cut here---end--->8---

Therefore, I understand why the two store items have different hashes.


What puzzled me are the corresponding ’guile-builder’s:

--8<---cut here---start->8---
  (define %build-inputs
(quote
 (("source" . 
"/gnu/store/chariqd6k0sli3s7vcl4q3al0crirz5v-hello-2.10.tar.gz")
--8<---cut here---end--->8---

and

--8<---cut here---start->8---
  (define %build-inputs
(quote
 (("source" . 
"/gnu/store/hbdalsf5lpf01x4dcknwx6xbn6n5km6k-hello-2.10.tar.gz")
--8<---cut here---end--->8---

but these 2 items have the same hash:

--8<---cut here---start->8---
$ guix hash /gnu/store/chariqd6k0sli3s7vcl4q3al0crirz5v-hello-2.10.tar.gz
0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i

$ guix hash /gnu/store/hbdalsf5lpf01x4dcknwx6xbn6n5km6k-hello-2.10.tar.gz
0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i
--8<---cut here---end--->8---

Why?  It is 

Re: Can we find a better idiom for unversioned packages?

[Jonathan McHugh]

Hi Ludo,

Just checking:

Is Diffstatic a real tool? It wasnt quite clear to me (and I fancy finding a 
new diff tool).


Jonathan McHugh
indieterminacy@libre.brussels

September 8, 2021 11:31 PM, "Ludovic Courtès"  wrote:

> Hello!
> 
> Sarah Morgensen  skribis:
> 
>> Currently, there are about 1500 packages defined like this:
>> 
>> (define-public sbcl-feeder
>> (let ((commit "b05f517d7729564575cc809e086c262646a94d34")
>> (revision "1"))
>> (package
>> [...])))
>> 
>> I feel like there are some issues with this idiom (in no particular
>> order):
> 
> I’m late to the party but I’ll complement previous answers. :-)
> 
>> 1. When converting between this idiom and regularly versioned packages,
>> the git diff shows the whole package changing because of the indentation
>> change.
> 
> One can use ‘git diff -w’ to work around that (or the newfangled
> Diffstatic tool.)
> 
>> 3. Packages inheriting from it lose the definitions. For actual fields,
>> we have e.g. `(package-version this-package)`, but we have no equivalent
>> for these.
> 
> Right, these pieces of information are not “first-class”, except in the
> ‘git-reference’ record (or similar) for the commit ID. Do you have
> examples in mind where it’s insufficient?
> 
> [...]
> 
>> 5. The closest thing we have to a standardized way of generating
>> versions for these packages is `(version (git-version "0.0.0" revision
>> commit))`. We can do better than that boilerplate.
> 
> I can sympathize with the feeling, but I’m not sure what to do. A
> ‘vcs-version’ record as Maxime proposes seems a bit overkill to me (and
> it would probably have an impact on performance, build times, and
> whatnot.)
> 
>> 6. Not a direct complaint, but I feel like the overall package interface
>> was designed before straight-from-vcs unversioned packages were so
>> common, and so this idiom developed organically to work around that.
> 
> Sure, though “straight-from-vcs” and “unversioned” are two different
> things: I’m fine with the former, but the latter equates to upstream
> telling its users “go find a revision that works for you”. I think
> releases still make sense for any non-trivial piece of software.
> 
> As noted in the manual (info "(guix) Version Numbers"), packages built
> from arbitrary commits were supposed to be exceptional. Perhaps the
> reason we’re having this conversation now is that development practices
> are evolving towards what looks like chaos. :-)
> 
> Thanks,
> Ludo’.

Re: "guix copy" cannot deal with > 32-bit integers

[Ludovic Courtès]

Hi,

zimoun  skribis:

> $ guix copy --from=chouib 
> /gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10
> retrieving 1 store item from 'x.x.x.x'...
> guix copy: error: implementation cannot deal with > 32-bit integers

This is ‘guix copy’’s very own way of saying that the remote machine is
lacking /etc/guix/signing-key.{pub,sec}, or something along these lines.

Could you check that everything is in order?

Then we’ll take a look to see how we can, ahem, improve diagnostics.

Thanks,
Ludo’.

Re: Rethinking propagated inputs?

[Ludovic Courtès]

Hi,

Maxime Devos  skribis:

> For packages using pkg-config, I had the following scheme in mind:

As a reminder to everyone, pkg-config is just one of the reasons for
propagating inputs, probably not the main one numerically speaking.

> A package that has .pc files puts them into a separate "build" output.
> The code handling 'propagated-inputs' is modified such that an entry in
> 'propagated-inputs' can have an additional 'propagated-for-output' component.
>
> (Feel free to suggest a more concise name.)
>
> I.e., something like:
>
>   (define glib
> (package
>   (name "glib")
>   (outputs '("out" ; everything
>  "bin" ; glib-mkenums ... depends on Python
>  "build")) ; glib-2.0.pc
>   (propagated-inputs
> `(("pcre" ,pcre #:propagate-for-output "build") ; in the 
> Requires.private field of glib-2.0.pc
>   ("libffi" ,libffi #:propagate-for-output "build") ; ditto, for 
> gobject-2.0.pc
>   ;; etc.
>   ))
>   (native-inputs '()) ; not relevant to this e-ail
>   ...))

Good’ol ‘ROADMAP’ file has a half-baked proposal along these lines.
It’s interesting, but I’m a bit concerned about the maintenance
cost/benefit ratio of this approach.  (It also goes against work on
simplifying the surface syntax of inputs.)

It would be great if the ‘Requires’ field of .pc files could specify
absolute file name; it would no longer be necessary to set
PKG_CONFIG_PATH, and thus propagation wouldn’t be needed in this case.

Regarding outputs, Nixpkgs introduced a “dev” output a while back that
lumps together our “lib” and “include” outputs, roughly.  I think that’s
a good idea (for other reasons too).

Ludo’.

Re: Linux-libre source code will be taken offline

[Ludovic Courtès]

Hi!

Leo Famulari  skribis:

> According to the linux-libre team in the #gnu-linux-libre IRC channel on
> Libera.chat, all releases of linux-libre before 4.4.282, 4.9.281,
> 4.14.245, 4.19.205, 5.4.143, 5.10.61, and 5.13.13 will be deleted from
> their servers, and their Git repo is also going to be rewritten to
> remove them.
>
> So, if anybody wanted to build old versions of Guix System (or
> linux-libre based on old revisions of guix.git), some third party should
> preserve and mirror those files and the Git repo:
>
> https://linux-libre.fsfla.org/pub/linux-libre/releases/
> git://linux-libre.fsfla.org/releases.git

At the time you wrote this message (Sep. 4), I and others who were
present on IRC downloaded the tarballs to be on the safe side, so thanks
for the heads-up!

Now we have to see what’s available on berlin & co., and the extent to
which SWH can help with this situation.  So far I’m not sure about the
tarball contents, but the repo seems to be archived:

--8<---cut here---start->8---
scheme@(guile-user)> ,m (guix swh)
scheme@(guix swh)> (lookup-origin "https://guix.gnu.org/sources.json;)
$2 = #< visits-url: 
"https://archive.softwareheritage.org/api/1/origin/https://guix.gnu.org/sources.json/visits/;
 type: # url: "https://guix.gnu.org/sources.json;>
scheme@(guix swh)> (car (origin-visits $2))
$3 = #< date: # origin: 
"https://guix.gnu.org/sources.json; url: 
"https://archive.softwareheritage.org/api/1/origin/https://guix.gnu.org/sources.json/visit/313/;
 snapshot-url: 
"https://archive.softwareheritage.org/api/1/snapshot/bfdb775f4768dd3d20231effbf29b96cd7184985/;
 status: partial number: 313>
scheme@(guix swh)> (define s (visit-snapshot $3))
scheme@(guix swh)> (car ((@@ (gnu packages linux) linux-libre-urls) "5.14.1" 
"gnu"))
$4 = 
"https://linux-libre.fsfla.org/pub/linux-libre/releases/5.14.1-gnu/linux-libre-5.14.1-gnu.tar.xz;
scheme@(guix swh)> (lookup-snapshot-branch s $4)
$5 = #f
scheme@(guix swh)> (lookup-snapshot-branch s (car ((@@ (gnu packages linux) 
linux-libre-urls) "5.13.14" "gnu1")))
$6 = #f
scheme@(guix swh)> (lookup-snapshot-branch s (car ((@@ (gnu packages linux) 
linux-libre-urls) "5.10.62" "gnu1")))
$7 = #f
scheme@(guix swh)> (lookup-origin "git://linux-libre.fsfla.org/releases.git")
$8 = #< visits-url: 
"https://archive.softwareheritage.org/api/1/origin/git://linux-libre.fsfla.org/releases.git/visits/;
 type: # url: "git://linux-libre.fsfla.org/releases.git">
scheme@(guix swh)> (car (origin-visits $8))
$9 = #< date: # origin: 
"git://linux-libre.fsfla.org/releases.git" url: 
"https://archive.softwareheritage.org/api/1/origin/git://linux-libre.fsfla.org/releases.git/visit/4/;
 snapshot-url: 
"https://archive.softwareheritage.org/api/1/snapshot/cbf57c5bd0c5161c8df8853c64e96040b5c9cd9c/;
 status: full number: 4>
--8<---cut here---end--->8---

For some reason, we seem to be exporting only one tarball in
sources.json currently (the file that SWH periodically reads):

--8<---cut here---start->8---
$ wget -qO - https://guix.gnu.org/sources.json|jq |grep linux-libre.fsfla.org

"https://linux-libre.fsfla.org/pub/linux-libre/releases/5.4.20-gnu/linux-libre-5.4.20-gnu.tar.xz;,

"https://linux-libre.fsfla.org/pub/linux-libre/releases/5.4.20-gnu/linux-libre-5.4.20-gnu.tar.xz;,
--8<---cut here---end--->8---

We should check why we’re not providing all the URLs and fix it.

> The reason for the removal is that some nonfree things were accidentally
> not removed by the previous releases of the deblob scripts.

IMO such issues should be treated as regular bugs, without compromising
on transparency and availability.

Thanks,
Ludo’.

Re: Can we find a better idiom for unversioned packages?

[Ludovic Courtès]

Hello!

Sarah Morgensen  skribis:

> Currently, there are about 1500 packages defined like this:
>
> (define-public sbcl-feeder
>   (let ((commit "b05f517d7729564575cc809e086c262646a94d34")
> (revision "1"))
> (package
>   [...])))
>
> I feel like there are some issues with this idiom (in no particular
> order):

I’m late to the party but I’ll complement previous answers.  :-)

> 1. When converting between this idiom and regularly versioned packages,
> the git diff shows the whole package changing because of the indentation
> change.

One can use ‘git diff -w’ to work around that (or the newfangled
Diffstatic tool.)

> 3. Packages inheriting from it lose the definitions.  For actual fields,
> we have e.g. `(package-version this-package)`, but we have no equivalent
> for these.

Right, these pieces of information are not “first-class”, except in the
‘git-reference’ record (or similar) for the commit ID.  Do you have
examples in mind where it’s insufficient?

[...]

> 5. The closest thing we have to a standardized way of generating
> versions for these packages is `(version (git-version "0.0.0" revision
> commit))`.  We can do better than that boilerplate.

I can sympathize with the feeling, but I’m not sure what to do.  A
‘vcs-version’ record as Maxime proposes seems a bit overkill to me (and
it would probably have an impact on performance, build times, and
whatnot.)

> 6. Not a direct complaint, but I feel like the overall package interface
> was designed before straight-from-vcs unversioned packages were so
> common, and so this idiom developed organically to work around that.

Sure, though “straight-from-vcs” and “unversioned” are two different
things: I’m fine with the former, but the latter equates to upstream
telling its users “go find a revision that works for you”.  I think
releases still make sense for any non-trivial piece of software.

As noted in the manual (info "(guix) Version Numbers"), packages built
from arbitrary commits were supposed to be exceptional.  Perhaps the
reason we’re having this conversation now is that development practices
are evolving towards what looks like chaos.  :-)

Thanks,
Ludo’.

Re: Can we find a better idiom for unversioned packages?

[Ludovic Courtès]

Hi,

Xinglu Chen  skribis:

> Well, seeing
>
>   foo-1.0.0-1.2021-01-31
>
> gives a user more useful information than something like
>
>   foo-1.0.0-1.cabba9e
>
> With the former, I can quickly see that the version is from 2021-01-31,
> whereas with the latter, I would have to either find the VCS repo online
> or go to my local checkout of it and browse the logs.

I agree with Leo here.  IMO what matters with version strings is that
they be comparable: users (and UIs) should be able to tell which string
denotes a newer version.  That’s exactly what the ‘revision’ bit does in
the example above.  The commit shorthand is an additional bit to
uniquely identify the snapshot.

Knowing the date of a release, or that of a commit, may also be useful,
but that’s not what version strings are about generally.

Thanks,
Ludo’.

Re: gcc: build all languages at once?

[Ludovic Courtès]

Hi Sarah,

Sarah Morgensen  skribis:

> I notice that out of GCC's supported languages (ada, c, c++, d, fortran,
> go, jit, lto, objc, obj-c++) we currently build all except ada, and five
> of them (d, fortran, go, objc, and obj-c++) are built separately.  Most
> of GCC's build time is spent bootstrapping and building the actual
> compiler, rather than building the frontends, so we would save a lot of
> build time by building them all together.

Yes, but at the expense of the build time of the main GCC package and of
its size.

> We could also possibly reuse the 'core' parts of GCC between language
> frontends, saving some space.  (This is what distros seem to do.)

That’d be ideal, but I’m not sure it’s feasible.  It’s feasible for FHS
distros that “just” need to shuffle files around in various binary
packages, but here we’d have to have several outputs or something, and
most likely they’d refer to one another.

> If we do this, I'm not sure whether it would be better to have each
> additional language as an output for gcc, or as a separate package.
>
> Currently, the only build-time difference between our GCC packages is
> that our package for 'jit' uses '--enable-host-shared', which
> "[specifies] that the host code should be built into
> position-independent machine code (with -fPIC), allowing it to be used
> within shared libraries, but yielding a slightly slower compiler" [0].
> I don't think it would be too big of a hit to just turn that on
> unconditionally, but we could also keep 'jit' as a separately-built
> package.

Yeah, dunno.

> What do you all think?

There’s definitely room for improvement in this area, but it’s not clear
to me what can be done within the constraints that we have, in
particular the closure size of gcc (gcc:lib in particular) and having a
reasonably small set of dependencies for ‘gcc-final’.  I suppose
experimentation could tell us which approach to follow!

Thanks,
Ludo’.

Re: delete-generations or --delete-generations?

[Ludovic Courtès]

Hi,

zimoun  skribis:

> I speculate too. :-) I guess because the idea behind “guix system” is
> one action at a time however “guix package” can compose actions in one
> transaction (guix package --install=foo --remove=bar).  Using the
> default CLI parser (SRFI-37), it seems easier for composing to use the
> options (--) than to extract actions.  For instance, if it was
> “guix package ” as it is with “guix system”, then it is harder
> to parse the composition, i.e.,

Yeah, long ago Alex Kost and others proposed turning some of the current
‘guix package’ options into sub-sub-commands of a new ‘guix profile’
sub-command: ‘guix profile install foo’, ‘guix profile
delete-generations’, etc.

It may be consistent but maybe a bit far-fetched and annoying to type…

Ludo’.

Re: Transform options should error on nonexistant targets

[Ludovic Courtès]

Hello!

zimoun  skribis:

> However, from my opinion, it is easy to check if the package-target is
> a package or not, i.e.
>
>   $ guix build foo --=package-target=new
>   guix build: error: package-target: unknown package
>
> For instance by using 'specification->package'; see attached patch.
> But then, the test suite fails; I guess because 'dummy-package' and I
> have not found the time to investigate.  From my point of view, this
> kind of patch will fix one part of the initial issue reported by Ryan.

Right.

> The other issue is to list if the transformation is applied or not.  I
> think it is possible by traversing again the graph and check if a
> property appears at least once; well it should be better to warn if
> the 'mapping-property' is not found at least once.  I had some
> headaches to implement it... and I moved to other "urgent" stuff. :-)

Hmm the ‘mapping-property’ is not enough.  I think you pretty much have
to compute the derivations of the new and old packages and compare them.

> Last, speaking about transformations, the graph is walked too much
> when several transformations is applied:
>
>guix build hello --with-latest=foo --with-input=bar=baz 
> --with-latest=chouib
>
> then the graph is walked 3 times, IIUC.  The options needs a rewrite
> to pass a list of specs to 'package-with-latest-upstream' and not
> twice a list with only one element.  This would reduce to 2 walks.
> Then it could be nice to compose the transformation and then walk only
> once (apply 'package-mapping' only once).
> Well, maybe I miss something.

Right, I guess it could work.  It’s the same complexity anyway, but at
least it would lower the constant costs.

> From c0fa86d316c91044630b85c9e789f9a455fd29f4 Mon Sep 17 00:00:00 2001
> From: zimoun 
> Date: Fri, 27 Aug 2021 18:15:16 +0200
> Subject: [PATCH] transformations: Error when incorrect specifications.
>
> * guix/transformations.scm (transform-package-with-debug-info,
> transform-package-latest, transform-package-tests)[rewrite]: Raise when
> incorrect specification.
> (options->transformation)[package-name?]: New procedure.
> [applicable]: Use it.

[...]

> -(package-input-rewriting/spec (map (lambda (spec)
> - (cons spec package-with-debug-info))
> -   specs)))
> +(package-input-rewriting/spec
> + (map (lambda (spec)
> +(match (string-tokenize spec %not-equal)
> +  ((spec)
> +   (cons spec package-with-debug-info))
> +  (_
> +   (raise
> +(formatted-message (G_ "~a: invalid specification") spec)
> +  specs)))
>  
>(lambda (obj)
>  (if (package? obj)
> @@ -451,9 +458,15 @@ to the same package but with #:strip-binaries? #f in its 
> 'arguments' field."
>   ((#:tests? _ #f) #f)
>  
>(define rewrite
> -(package-input-rewriting/spec (map (lambda (spec)
> - (cons spec package-without-tests))
> -   specs)))
> +(package-input-rewriting/spec
> + (map (lambda (spec)
> +(match (string-tokenize spec %not-equal)
> +  ((spec)
> +   (cons spec package-without-tests))
> +  (_
> +   (raise
> +(formatted-message (G_ "~a: invalid specification") spec)

The goal here is to catch mistakes like ‘--with-debug-info=foo=bar’,
right?  Is that a plausible typo?  :-)

>  Each symbol names a transformation and the corresponding string is an 
> argument
>  to that transformation."
> +  (define (package-name? value)

Rather ‘assert-package-specification’, since it’s used for control
effects (exception raised by ‘specification->package’).

> +;; Return an error if value does not correspond to a package.
> +(match (string-tokenize value %not-equal)
> +  ((name _ ...)
> +   (specification->package name

The problem I see is that it prevents rewrites where the package to be
rewritten is not public.  Maybe that’s an acceptable tradeoff though,
I’m not sure.

Thoughts?

Thanks,
Ludo’.

Re: PATCH: clarify some of the Guix Linode recipe

[Ludovic Courtès]

Hi Grant,

Grant Shoshin Shangreaux  skribis:

> Hello! I hope I'm not double posting, but I think the email I intended
> to send before never made it out of my outbox. I recently set up GuixSD
> on Linode following the cookbook. I hit a few snags along the way where
> I either missed something or misunderstood, so I thought I'd offer a
> patch with a few clarifications to the article. Some of it is likely due
> to changes in the Linode UI. Hopefully this patch is helpful.

Looks like this message fell through the summer cracks…

> From 446bf79c3733936306e9fd95c46cbfd495f6bc2a Mon Sep 17 00:00:00 2001
> From: Grant Shangreaux 
> Date: Mon, 23 Aug 2021 22:47:05 -0500
> Subject: [PATCH] doc: add clarifications to Linode cookbook recipe
>
> * doc/guix-cookbook.texi: clarify Linode recipe
>
> Reword paragraph about adding the Guix device disk to Debian config.
> Improve example commands for sftp-ing files to the server.
> Minor wording fixes

This looks like a welcome improvement, so I went ahead and applied it.
(Cc’ing Joshua, who originally wrote this section.)

Thanks!

Ludo’.

Re: Batching world-rebuilding changes for the core-updates-frozen branch

[Guillaume Le Vaillant]

Maxim Cournoyer  skribis:

> Hello Guix!
>
> Since there's going to be at least this change [0] causing a world
> rebuild of the core-updates-frozen branch, I'd like to know if there are
> other world-rebuilding but low-risk changes you'd like to see integrated
> into the branch.
>
> If you do, feel free to 'block' the 50358 issue with the changes, so
> that I can review them and push them at the same time as [0].
>
> Thank you!
>
> Maxim
>
> [0]  https://issues.guix.gnu.org/50358

Hi,

I just pushed a world-rebuilding fix for a bug in binutils (see [1])
2 hours ago.
It's evaluation 15282 on ci.guix.gnu.org. Maybe someone with admin
access to the CI could cancel it, in order not to rebuild everything
twice.

[1] https://lists.gnu.org/archive/html/guix-devel/2021-09/msg00077.html


signature.asc
Description: PGP signature

Re: [fontconfig]: Default Font

[Maxim Cournoyer]

Hi Raghav,

Raghav Gururajan  writes:

> Hello Guix!
>
> In the package-definition of fontconfig, the default font is currently
> set to font-dejavu. I was wondering if we could change it to 
> font-gnu-freefont, for following reasons:
>
> [1] Provides multiple formats (ttf, otf and woff).
> [2] Is a GNU package.

Does it have as good of a glyph coverage as font-dejavu?  Is it as
pleasing to the eye?  I'm not against the change, but we'd have to
review if it's equivalent to deja-vu.

Thanks,

Maxim

Batching world-rebuilding changes for the core-updates-frozen branch

[Maxim Cournoyer]

Hello Guix!

Since there's going to be at least this change [0] causing a world
rebuild of the core-updates-frozen branch, I'd like to know if there are
other world-rebuilding but low-risk changes you'd like to see integrated
into the branch.

If you do, feel free to 'block' the 50358 issue with the changes, so
that I can review them and push them at the same time as [0].

Thank you!

Maxim

[0]  https://issues.guix.gnu.org/50358

"guix copy" cannot deal with > 32-bit integers

[zimoun]

Hi,

I do not understand why I get this:

--8<---cut here---start->8---
$ ssh chouib -t guix build hello
/gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10
Connection to x.x.x.x closed.

$ guix copy --from=chouib /gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10
retrieving 1 store item from 'x.x.x.x'...
guix copy: error: implementation cannot deal with > 32-bit integers
--8<---cut here---end--->8---

What do I miss?  This message comes from 'serialise.cc' therefore is
it a daemon issue?

Thanks in advance for any explanations.

All the best,
simon

Re: [core-updates-frozen] Bug in binutils 2.37

[Guillaume Le Vaillant]

Guillaume Le Vaillant  skribis:

> Leo Famulari  skribis:
>
>> On Mon, Sep 06, 2021 at 03:39:52PM +, Guillaume Le Vaillant wrote:
>>> There's a bug in binutils 1.37, which we are using on the
>>> core-updates-frozen branch.
>>
>> 2.37, right? :)
>>
>
> Indeed. :)
>
>
>>> It's a file descriptor leak that can lead to 'malformed archive' errors
>>> when linking libraries. We get this problem at least when building
>>> qtwekbit and qtwebengine. A workaround allows us to compile qtwebkit
>>> (see [1]), but it doesn't work for qtwebengine.
>>> 
>>> The bug was discussed at [2] and upstream has a patch to fix it at [3].
>>> However, adding this patch to our binutils rebuilds the world.
>>> I'm currently trying to build things with the patched binutils.
>>> If everything works, should I push this fix on core-updates-frozen, or
>>> does someone have an idea that would lead to less rebuilds?
>>
>> There are a few options:
>>
>> 1) Apply the patch in the normal way and rebuild the world. If the
>> changes in the patch are limited to fixing this bug, then we can be
>> confident that changing binutils will not break other packages, which is
>> the main goal behind "freezing" the core-updates branch. Do you think
>> that expectation is reasonable? Otherwise, the branch is frozen except
>> for bug fixes; we'd like to avoid rebuilding the world but it's not a
>> problem if we have to.
>> 2) Create a binutils-fixed package and only use it for qtwebkit and
>> qtwebengine
>> 3) Try to work around the bug in the qtwebkit and qtwebengine packages
>>
>> 2 and 3 are not great because maybe the bug affects other packages in
>> some situations. Do you know if it manifests deterministically?
>
> The problem happens when linking a library with a lot of members (I
> don't know exactly how many), which is the case at least for qtwebkit
> and qtwebengine. Users creating such libraries in their projects will
> also have the problem.
>
> Increasing the maximum number of open file descriptors seems not be
> a very robust workaround. Setting it at 4096 for qtwebkit works,
> but I tried 4096, 8192 and 16384 for qtwebengine and it didn't work.
>
> I have built many packages with the patched binutils and I didn't get
> any new failure so far. I'm not yet at qtwebkit and qtwebengine though
> (compiling the 20 versions of rust took a while).
> So when I get there, and if the patch really solves the issue, pushing
> it looks like the best solution.
>
> Do you know if there are other world-rebuilding pending fixes that could
> go in at the same time?

I confirm that the patch fixed the compilation issues for qtwebkit and
qtwebengine, and it didn't cause new build failures, so I pushed it as
de8e2a699c0219f5ea86f6bbfff4d5ee35104738.


signature.asc
Description: PGP signature

Re: avoid wrapper scripts when possible

[Maxime Devos]

Attila Lendvai schreef op wo 08-09-2021 om 09:20 [+]:
> thanks for the ideas Maxime!
> 
> 
> > You could patch trezor-agent with something like
> > 
> > Python syntax might be incorrect
> > 
> > 
> > if sys.argv[0] == ".trezor-gpg'
> > sys.argv[0] = "trezor-gpg"
> > device_name = os.path.basename(sys.argv[0]).rsplit('-',1)[0]
> > 
> > Would that work?
> 
> most probably, but i thought first i'll pursue a more general fix for
> this. i will fall back to just patching upstream if i fail.
> 
> 
> > FWIW, there's 'wrap-program' and there is 'wrap-script'.
> > 
> > wrap-script is less likely to cause errors here (it doesn't rename the 
> > executable)
> > but less general. Maybe wrap-script can be used instead of wrap-program?
> 
> not sure how to do this. looking at python-build-system.scm it looks
> like there is no way to tell it to use wrap-script, so i went ahead
> and just changed the wrap-program call to wrap-script globally, and
> gave it a try.

Are you referring to the lines

(for-each (lambda (dir)
(let ((files (list-of-files dir)))
  (for-each (cut wrap-program <> var)
files)))
  bindirs)

in the procedure 'wrap' from (guix build python-build-system)?

> the first error i encountered was in the package called scons. it has
> .bat files that wrap-script fails on. if i delete them in a snippet,
> then scons' own build system errors out.

The .bat are only for Windows, so you could remove them after the 'install'
phase or before the 'wrap' phase.  They seem to be useless on GNU/{Linux,Hurd}.

> any hints on how to proceed from here?

Instead of modifying 'wrap', you could replace the 'wrap' phase with
something like

  `(#:phases
(modify-phases %standard-phases
  (replace 'wrap
(lambda _
  ... customised wrapping code ...

Also, replacing wrap-program with wrap-script globally won't work
because wrap-script is less general, it only spports python, R,
perl and bash.  So something like

  (define (wrap-script-or-program ...)
(if it-uses-python-or-ruby-or...
(wrap-script ...)
(wrap-program ...)))

may be required.

> was this what you meant at all?
> 
> would it be a worthwhile improvement to hack my way through this path
> at all?

There were some changes on core-updates to reduce double-wrapping,
so possibly it's already solved.

> shall i try to make it configurable from the package's definition,
> change the default, and force it back to wrap-program for packages
> that fail with wrap-script?

Perhaps try something like 'wrap-script-or-program' first?  That should
reduce the number of package definitions that need to be adjusted.

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: avoid wrapper scripts when possible

[Attila Lendvai]

thanks for the ideas Maxime!


> You could patch trezor-agent with something like
>
> Python syntax might be incorrect
> 
>
> if sys.argv[0] == ".trezor-gpg'
> sys.argv[0] = "trezor-gpg"
> device_name = os.path.basename(sys.argv[0]).rsplit('-',1)[0]
>
> Would that work?


most probably, but i thought first i'll pursue a more general fix for
this. i will fall back to just patching upstream if i fail.


> FWIW, there's 'wrap-program' and there is 'wrap-script'.
>
> wrap-script is less likely to cause errors here (it doesn't rename the 
> executable)
> but less general. Maybe wrap-script can be used instead of wrap-program?


not sure how to do this. looking at python-build-system.scm it looks
like there is no way to tell it to use wrap-script, so i went ahead
and just changed the wrap-program call to wrap-script globally, and
gave it a try.

the first error i encountered was in the package called scons. it has
.bat files that wrap-script fails on. if i delete them in a snippet,
then scons' own build system errors out.

any hints on how to proceed from here?

was this what you meant at all?

would it be a worthwhile improvement to hack my way through this path
at all?

shall i try to make it configurable from the package's definition,
change the default, and force it back to wrap-program for packages
that fail with wrap-script?

- attila

Re: Rethinking propagated inputs?

[iskarian]

Hi,

September 8, 2021 12:18 AM, "Liliana Marie Prikler"  
wrote:

> Hi,
> 
> Am Dienstag, den 07.09.2021, 12:01 -0700 schrieb Sarah Morgensen:
> 
>> [...]
>> 
>> However, IMO it doesn't make sense to automatically install these in
>> a profile when installing that package. If you're installing a
>> source-only package, you likely don't want all the transitive
>> dependencies automatically installed--what if you're just inspecting
>> the source, or want to try building it with different
>> dependencies? If you want all the build dependencies as well, you
>> would just do
>> 
>> guix environment go-foo --ad-hoc go-foo
>> 
>> like you would do for other Guix packages.
> 
> I think if you wanted to build go-foo locally, you should go with just
> `guix environment go-foo' anyway. For the source code, you might be
> interested in `guix build --source'.

guix build --source doesn't capture any modifications made in the dependency 
package, such as
rewriting paths or other patching.

You're right, though, in that (for now) it's a rare use-case, and probably 
doesn't warrant the
added complexity and the extra field. It just feels... messy as it is now.

>> In the context of this discussion, I can think of three types of
>> solution:
>> 
>> 1. Add a modifier like #:profile? (defaulting to #f) to
>> propagated-inputs entries
>> (propagated-inputs
>> `(("foo" #:profile? #t)))
> 
> This is basically a weaker linked-inputs, because linked-inputs could
> be sanitized while these propagated inputs could not.

Good point. I included it because there seemed to be pushback on adding a field.

>> 2. Rename "propagated-inputs" to e.g. "profile-propagated-inputs",
>> and then introduce a new "propagated-inputs", which only propagates
>> into the inputs of dependents, and not into built profiles.
> 
> Again same as introducing a new field, but also changing semantics for
> everything in the tree. I think having "propagated-inputs" mean what
> it means is fine. If we do find that we're lacking a field with
> certain semantics, we should add that or change existing fields in
> backwards-compatible ways.

I agree; I should have stated this differently ("Add a new field"). The name is 
just bikeshedding.

>> 3. Introduce a way to prevent an input from propagating anything into
>> the current package/profile
>> (inputs
>> `(("foo" #:propagate? #f)))
>> and then provide CLI options (transformations) to control this.
> 
> Your example should have propagated-inputs. I'm not sure what benefit
> there is to this solution that isn't given by adding a field, except
> that it can't be sanitized. Now that I think about it, this seems to
> be a different syntax to 1.

Ah, I forgot to state that in this example, "foo" is the package with a 
propagated-inputs field,
and this is a dependent package (or a profile) which does not want "foo"'s 
propagated-inputs to be
propagated to it.

This way a packager has an "escape hatch" if something unwanted got propagated. 
Thinking about it
some more, though, in such a case it's more likely to want to disallow a 
specific package than a
whole set of propagated inputs. If such a feature would actually be useful 
(questionable), it's
probably better implemented differently than this (disallowed-inputs, perhaps, 
to mirror
disallowed-references)?

I hope I made some more sense this time!

--
Sarah

Re: Rethinking propagated inputs?

[Liliana Marie Prikler]

Hi,

Am Dienstag, den 07.09.2021, 12:01 -0700 schrieb Sarah Morgensen:
> [...]
> 
> However, IMO it doesn't make sense to automatically install these in
> a profile when installing that package.  If you're installing a
> source-only package, you likely don't want all the transitive
> dependencies automatically installed--what if you're just inspecting
> the source, or want to try building it with different
> dependencies?  If you want all the build dependencies as well, you
> would just do
> 
>   guix environment go-foo --ad-hoc go-foo
> 
> like you would do for other Guix packages.
I think if you wanted to build go-foo locally, you should go with just
`guix environment go-foo' anyway.  For the source code, you might be
interested in `guix build --source'.

> In the context of this discussion, I can think of three types of
> solution:
> 
> 1. Add a modifier like #:profile? (defaulting to #f) to
> propagated-inputs entries
>   (propagated-inputs
> `(("foo" #:profile? #t)))
This is basically a weaker linked-inputs, because linked-inputs could
be sanitized while these propagated inputs could not.

> 2. Rename "propagated-inputs" to e.g. "profile-propagated-inputs",
> and then introduce a new "propagated-inputs", which only propagates
> into the inputs of dependents, and not into built profiles.
Again same as introducing a new field, but also changing semantics for
everything in the tree.  I think having "propagated-inputs" mean what
it means is fine.  If we do find that we're lacking a field with
certain semantics, we should add that or change existing fields in
backwards-compatible ways.

> 3. Introduce a way to prevent an input from propagating anything into
> the current package/profile
>   (inputs
> `(("foo" #:propagate? #f)))
> and then provide CLI options (transformations) to control this.
Your example should have propagated-inputs.  I'm not sure what benefit
there is to this solution that isn't given by adding a field, except
that it can't be sanitized.  Now that I think about it, this seems to
be a different syntax to 1.

> I think the first two are clear and make the packager consider
> exactly what they're going to propagate into user profiles.  Of
> course, both also rely on the packager always knowing what should be
> propagated to a user's profile or not.  The third option allows more
> control over that (and could be combined with one of the first two),
> but feels a bit rough to use...
W.r.t. relying on the packager to know that, we could document such use
cases in the manual (similar to how they are documented currently) and
we still have a review process to fall back on.

Regards