Re: python-pytest in references graph

[Lars-Dominik Braun]

Hi,

> It should, but sometimes there are bugs in the package definition or 
> build system, in this case causing python-rdflib to refer to the 
> native-input python-pytest.  Likely it's the 'add-install-to-path' phase 
> adding too much, a known issue, which could be solved by separating 
> inputs and native-inputs on the build side when compiling natively (and 
> not only when cross-compiling) (currently they are merged together into 
> 'inputs'), though non-trivial.
the issue is indeed a known bug https://issues.guix.gnu.org/25235

Lars

Re: python-pytest in references graph

[Roel Janssen]

On Sun, 2022-07-24 at 23:01 +0200, Maxime Devos wrote:
> 
> On 24-07-2022 22:25, Roel Janssen wrote:
> > I'm trying to understand the output of:
> > $ guix graph --type=references python-rdflib | dot -Tsvg -o rdflib.svg
> > 
> > Particularly, I'm looking at why python-pytest has an input arrow from 
> > python-rdflib, while it's
> > "only" a native-input?  I thought the "references" graph type would only 
> > include run-time
> > references, but I don't know what happens in this case.
> > 
> > What am I missing?
> 
> It should, but sometimes there are bugs in the package definition or 
> build system, in this case causing python-rdflib to refer to the 
> native-input python-pytest.  Likely it's the 'add-install-to-path' phase 
> adding too much, a known issue, which could be solved by separating 
> inputs and native-inputs on the build side when compiling natively (and 
> not only when cross-compiling) (currently they are merged together into 
> 'inputs'), though non-trivial.

Thanks for your explanation! I see indeed that in the build output a couple
of programs include pytest in their "GUIX_PYTHONPATH".

Kind regards,
Roel Janssen

Re: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.

[Julien Lepiller]

I don't like the wording at all. You're mixing too many things together.

I think it would be better to first document the guiding principles (eg. the 
goal that there are no non-free software in Guix, going for the simplest thing, 
etc) and then derive rules for specific cases, based on these principles:

How do I remove non-free software? -> snippet because …

How do I remove bundled libraries? -> snippet or phase because …

How do I fix a build issue? -> patch or snippet if this affects building from 
source, can also be a phase if the result of --sources can still build

A test issue?

…

This leaves some cases up to interpretation, but that's probably not so 
different from "it's not an absolute rule". It's also much clearer and quicker 
to figure out in which case you are. If not documented as a case, you can fall 
back to the general principles.

Le 25 juillet 2022 05:17:33 GMT+02:00, Maxime Devos  a 
écrit :
>Context: it's currently a mess:, and at times contradictory
>
> * There is policy involving those three, as can be seen from the
>   shepherd mess.
> * This policy is partially secret, as can be seen by some people
>   treating some things as policy even if it's not in the manual.
> * Some versions of the policy are based on archeology, e.g. see the
>   'snippets were introduced for this particular purpose, so don't use
>   it for other things’ which is not documented in the manual as sole
>   legitimate reason and asking contributors to read all past
>   discussions seems too much for me.
> * Sometimes, people refer to the manual (Snippets versus Phases) for
>   how things (should) work, but what they say is not actually present
>   in that section of the manual.
> * Some variants of the policy are contradictory with each other (IIRC)
> * Some of the policies are contradictory with current practice in
>   other Guix packages.
> * '(guix)Snippets versus Phases says Phases' states that it is elusive.
> * The section name implies it's a ‘X versus Y’, which seems
>   polarizing? (Maybe?)
> * The section neglects the is/ought-distinction -- it just says what
>   is typically used for what, not whether they should be used for them
>   and whether they are allowed to be used for other things, so that
>   section does not seem policy to me (except for the single 'should
>   produce' and 'must not' line), only matters of fact.
>
>I can't work with such a mess. As such, I've a proposal for a consistent, 
>clear and non-elusive set of rules and guidelines, based on the following 
>principles:
>
> * It appears we cannot agree on what exactly the policy should be, but
>   having a single policy everyone can use even if some would rather
>   have the specifics be a tiny bit different, is much better than the
>   mess of everyone having their own policy.
> * There are no absolutes, except that the result of "guix build
>   --source" must be free software;
> * There can be more than one (acceptable) way to do things, but this
>   doesn't make things elusive, this just means there are multiple
>   acceptable options and you should probably go for the simplest.
>
>More concretely, I propose the following new contents for (guix)Snippets 
>versus Phases (the phrasing could use some work for smooth reading), which I 
>believe to be sufficiently clear (except for some phrasing that could be 
>tweaked, e.g. the phrases are currently rather long), covers a sufficient 
>amount of cases (feel free to respond if you see a missing case), free of 
>contradictions (likewise) and mostly in line with current practice:
>
>[start]
>
>@c: There is no opposition or such, so no versus, let's not start with 
>polarisation.
>
>20.4.5 Snippets, phases and patches
>
>Snippets, phases and patches at times serve overlapping purposes. To decide 
>between the three, there are several considerations to keep in mind:
>
> * Patches must not be used to remove non-free files, because a patch
>   by construction contains the non-free file itself so the patch would
>   be non-free, which would not be acceptable to Guix. Likewise,
>   patches should not be used to remove bundled libraries, to avoid
>   large space usage, but this is not an absolute rule unlike as for
>   non-free files.
> * Snippets are often convenient for removing unwanted files such as
>   bundled libraries, non-free sources and binaries. It is technically
>   also possible to use phases for this, albeit slightly less
>   convenient at times. However, phases must not be used to remove
>   non-free sources, as then the output of "guix build --source" would
>   still contain the non-free sources, which is incompatible with Guix'
>   stance on free software. Likewise, phases should not be used to
>   remove binaries; however, this is not strictly forbidden.
> * Snippets must not embed store items in the source, as this is
>   incompatible with cross-compilation and prevents effectively sharing
>   the source code produced with "guix build --source" with people
>   using non-Gui

A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.

[Maxime Devos]

Context: it's currently a mess:, and at times contradictory

 * There is policy involving those three, as can be seen from the
   shepherd mess.
 * This policy is partially secret, as can be seen by some people
   treating some things as policy even if it's not in the manual.
 * Some versions of the policy are based on archeology, e.g. see the
   'snippets were introduced for this particular purpose, so don't use
   it for other things’ which is not documented in the manual as sole
   legitimate reason and asking contributors to read all past
   discussions seems too much for me.
 * Sometimes, people refer to the manual (Snippets versus Phases) for
   how things (should) work, but what they say is not actually present
   in that section of the manual.
 * Some variants of the policy are contradictory with each other (IIRC)
 * Some of the policies are contradictory with current practice in
   other Guix packages.
 * '(guix)Snippets versus Phases says Phases' states that it is elusive.
 * The section name implies it's a ‘X versus Y’, which seems
   polarizing? (Maybe?)
 * The section neglects the is/ought-distinction -- it just says what
   is typically used for what, not whether they should be used for them
   and whether they are allowed to be used for other things, so that
   section does not seem policy to me (except for the single 'should
   produce' and 'must not' line), only matters of fact.

I can't work with such a mess. As such, I've a proposal for a 
consistent, clear and non-elusive set of rules and guidelines, based on 
the following principles:


 * It appears we cannot agree on what exactly the policy should be, but
   having a single policy everyone can use even if some would rather
   have the specifics be a tiny bit different, is much better than the
   mess of everyone having their own policy.
 * There are no absolutes, except that the result of "guix build
   --source" must be free software;
 * There can be more than one (acceptable) way to do things, but this
   doesn't make things elusive, this just means there are multiple
   acceptable options and you should probably go for the simplest.

More concretely, I propose the following new contents for (guix)Snippets 
versus Phases (the phrasing could use some work for smooth reading), 
which I believe to be sufficiently clear (except for some phrasing that 
could be tweaked, e.g. the phrases are currently rather long), covers a 
sufficient amount of cases (feel free to respond if you see a missing 
case), free of contradictions (likewise) and mostly in line with current 
practice:


[start]

@c: There is no opposition or such, so no versus, let's not start with 
polarisation.


20.4.5 Snippets, phases and patches

Snippets, phases and patches at times serve overlapping purposes. To 
decide between the three, there are several considerations to keep in mind:


 * Patches must not be used to remove non-free files, because a patch
   by construction contains the non-free file itself so the patch would
   be non-free, which would not be acceptable to Guix. Likewise,
   patches should not be used to remove bundled libraries, to avoid
   large space usage, but this is not an absolute rule unlike as for
   non-free files.
 * Snippets are often convenient for removing unwanted files such as
   bundled libraries, non-free sources and binaries. It is technically
   also possible to use phases for this, albeit slightly less
   convenient at times. However, phases must not be used to remove
   non-free sources, as then the output of "guix build --source" would
   still contain the non-free sources, which is incompatible with Guix'
   stance on free software. Likewise, phases should not be used to
   remove binaries; however, this is not strictly forbidden.
 * Snippets must not embed store items in the source, as this is
   incompatible with cross-compilation and prevents effectively sharing
   the source code produced with "guix build --source" with people
   using non-Guix systems.
 * In principle, you can apply a patch from a phase. However, this
   causes the result of "guix build --source" to not correspond to the
   actual source code anymore (i.e., it doesn't act as corresponding
   source anymore), so consider this a last resort for situations such
   as avoiding causing a world-rebuild for a patch fixing a
   target-specific bug by making the patching conditional upon
   target-foo?. If you apply a patch from a phase, make sure that the
   patch appears in the inputs or native-inputs, such that "guix build
   --source=all" will include the patch.

   @c this relaxes the old rule a little

 * Ideally, the source derived from the origin should be usable for
   building on any system that the upstream package supports (even if
   Guix does not support that system), as a courtesy to the people that
   the source code is shared with. However, this is not an absolute
   rule, most important is that it is usable on Guix and it is allowed
   to neglect

Re: xpip install -U 'xonsh[full]'

[Maxime Devos]

On 25-07-2022 00:29, jgart wrote:

When installing xonsh I get the following after starting:

```
You are currently using the readline backend. For interactive tab-completion, 
on-the-fly syntax highlighting, and more, install prompt_toolkit by running:

   xpip install -U 'xonsh[full]'

```

How should Guix handle optionally installing xonsh[full]?


I guess it could ask to install it with Guix instead. It's a bit 
complicated with the multiple mechanisms (guix install, guix home, "guix 
shell foo bar -- baz"), but that can be avoided by patching it to 
rephrase it a bit:


```
You are currently using the readline backend. For interactive 
tab-completion, on-the-fly syntax highlighting, and more, install the 
'python-prompt-toolkit' package.

```

OTOH, from what I've gathered from previous discussions, some people 
want "guix build --source" to be the unmodified source code, with as 
only exception non-freeness removal, so patching would be out. E.g., 
doing the "-O1" bugfix in the shepherd package as a patch (well, 
snippet, but same thing basically) was rejected by some people in 



But at the same time, we patch glibc, gcc and guile-fibers without any 
complaints while those are more complicates patches and (in case of 
glibc and gcc), even Guix-specific, while the shepherd patch is simple 
and non-Guix specific.


(I don't understand anything from those discussions)

Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

xpip install -U 'xonsh[full]'

[jgart]

Hi Guixers,

When installing xonsh I get the following after starting:

```
You are currently using the readline backend. For interactive tab-completion, 
on-the-fly syntax highlighting, and more, install prompt_toolkit by running:

  xpip install -U 'xonsh[full]'

```

How should Guix handle optionally installing xonsh[full]?

all best,

jgart

Re: python-pytest in references graph

[Maxime Devos]

On 24-07-2022 22:25, Roel Janssen wrote:

I'm trying to understand the output of:
$ guix graph --type=references python-rdflib | dot -Tsvg -o rdflib.svg

Particularly, I'm looking at why python-pytest has an input arrow from 
python-rdflib, while it's
"only" a native-input?  I thought the "references" graph type would only 
include run-time
references, but I don't know what happens in this case.

What am I missing?


It should, but sometimes there are bugs in the package definition or 
build system, in this case causing python-rdflib to refer to the 
native-input python-pytest.  Likely it's the 'add-install-to-path' phase 
adding too much, a known issue, which could be solved by separating 
inputs and native-inputs on the build side when compiling natively (and 
not only when cross-compiling) (currently they are merged together into 
'inputs'), though non-trivial.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

python-pytest in references graph

[Roel Janssen]

Dear Guix,

I'm trying to understand the output of:
$ guix graph --type=references python-rdflib | dot -Tsvg -o rdflib.svg

Particularly, I'm looking at why python-pytest has an input arrow from 
python-rdflib, while it's
"only" a native-input?  I thought the "references" graph type would only 
include run-time
references, but I don't know what happens in this case.

What am I missing?

Thank you for your time.

Kind regards,
Roel Janssen

Re: User group during build process

[Tobias Geerinckx-Rice]

Hiya,

On 24 July 2022 19:17:29 UTC, Maya  wrote:
> shouldn’t opensmtpd-service-type already extend setuid-program-service-type 
> with appropriate setuids?

A matter of taste (I disagree with your characterisation 'useless'!), but I 
certainly wouldn't mind.  Plenty of services do this for convenience nowadays.

You seem to be familiar with the mechanisms needed; would you like to propose a 
patch?

Kind regards,

T G-R

Sent on the go.  Excuse or enjoy my brevity.

Re: User group during build process

[Maya]

Hi!

Thank you both for the responses. Although I wanted to ask you, can I use the 
setuid-program-service-type?

If yes, shouldn’t opensmtpd-service-type already extend 
setuid-program-service-type with appropriate setuids? As it right now renders 
smtpctl unusable.

With all the best wishes,
Maya

>

Re: User group during build process

[Development of GNU Guix and the GNU System distribution.]

Hi,

On Sun, Jul 24, 2022 at 7:14 AM Tobias Geerinckx-Rice  wrote:
>
> The correct way to make smtpctl setgid is to use Guix System's 
> setuid-programs field, and use (setgid? #t) (group "smtpq") for smtpctl.

In line with TGR's recommendation, the snippet below works locally.

Kind regards
Felix Lechner

* * *

  (setuid-programs
   (append (list

(setuid-program
 (program (file-append opensmtpd "/sbin/smtpctl"))
 (setuid? #false)
 (setgid? #true)
 (group "smtpq"))

(setuid-program
 (program (file-append opensmtpd "/sbin/sendmail"))
 (setuid? #false)
 (setgid? #true)
 (group "smtpq"))

(setuid-program
 (program (file-append opensmtpd "/sbin/send-mail"))
 (setuid? #false)
 (setgid? #true)
 (group "smtpq"))

(setuid-program
 (program (file-append opensmtpd "/sbin/makemap"))
 (setuid? #false)
 (setgid? #true)
 (group "smtpq"))

(setuid-program
 (program (file-append opensmtpd "/sbin/mailq"))
 (setuid? #false)
 (setgid? #true)
 (group "smtpq"))

(setuid-program
 (program (file-append opensmtpd "/sbin/newaliases"))
 (setuid? #false)
 (setgid? #true)
 (group "smtpq")))

 %setuid-programs))

Re: User group during build process

[Tobias Geerinckx-Rice]

Hi Maya!

So... You cannot change group ownership in the store.  There are no additional 
user groups in the build environment.  The chgrp failing is by design ('it's a 
feature, not a bug'), and there's no way to subvert this.  The group is not 
required during the build process.

The correct way to make smtpctl setgid is to use Guix System's setuid-programs 
field, and use (setgid? #t) (group "smtpq") for smtpctl.

Kind regards,

T G-R

Sent on the go.  Excuse or enjoy my brevity.

User group during build process

[Maya]

Hi,

I have been setting up mail server on my guix machine and I have found a bug. 
The smtpctl command from the opensmptd package requires to be owned by the 
smtpq group.

The problem is, that this group is added by opensmtp-service, but it is 
required during the build process as can be seen here 
http://ci.guix.gnu.org/build/748270/log/raw (search for chgrp)

I write to you since I’m not sure how to fix it idiomatically. Here are my 
solutions:

a) smtpq can be a system group that exists on all guix machines
b) smtpq will get a static gid, the build process will create a dummy group 
with that gid, chgrp the file to that group and the opensmtp-service-type will 
create user-group smtpq with that static gid (i dont know if there’s an index 
of reserved guix gids)
c) patch the upstream opensmtpd

With all the best,

Maya