Re: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.
Currently writing a v2 with a structure like Julien proposed. Greetings, Maxime. OpenPGP_0x49E3EE22191725EE.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature
Re: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.
On 05-08-2022 05:23, Philip McGrath wrote: On Sun, Jul 24, 2022, at 11:17 PM, Maxime Devos wrote: * Patches must not be used to remove non-free files, because a patch by construction contains the non-free file itself so the patch would be non-free, which would not be acceptable to Guix. Likewise, patches should not be used to remove bundled libraries, to avoid large space usage, but this is not an absolute rule unlike as for non-free files. It is possible to create patches that do not contain the deleted file, e.g. with `git format-patch --irreversible-delete`. That said, I don't know if the version of `patch` we use to patch origins is able to apply such patches—but maybe it would be a useful feature? -Philip Right, this is possible though it would have to be checked whether it is supported, so that statement should be weakened a bit -- to allow deleting non-free files with a patch, but noting that you'll have to set the right options to avoid including the deleted file in the patch. I would recommend a (delete-file-recursively ".") over a patch here though, to avoid having to remember the --irreversible-delete option and in case there is not a git repo. Greetings, Maxime. OpenPGP_0x49E3EE22191725EE.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature
Re: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.
On 05-08-2022 05:38, Philip McGrath wrote: On Sun, Jul 24, 2022, at 11:17 PM, Maxime Devos wrote: * In principle, you can apply a patch from a phase. However, this causes the result of "guix build --source" to not correspond to the actual source code anymore (i.e., it doesn't act as corresponding source anymore), so consider this a last resort for situations such as avoiding causing a world-rebuild for a patch fixing a target-specific bug by making the patching conditional upon target-foo?. If you apply a patch from a phase, make sure that the patch appears in the inputs or native-inputs, such that "guix build --source=all" will include the patch. Should we have an option for "guix build --source=all" to also include the Guix "scripts used to control compilation and installation"? What do you mean with "Guix scripts" here? I don't think Guix has a notion of 'scripts' (except stuff like 'wrap-script', but that doesn't seem relevant here). Do you mean the phases code? Greetings, Maxime. OpenPGP_0x49E3EE22191725EE.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature
Re: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.
On Sun, Jul 24, 2022, at 11:17 PM, Maxime Devos wrote: > * In principle, you can apply a patch from a phase. However, this causes the > result of "guix build --source" to not correspond to the actual source code > anymore (i.e., it doesn't act as corresponding source anymore), so consider > this a last resort for situations such as avoiding causing a world-rebuild > for a patch fixing a target-specific bug by making the patching conditional > upon target-foo?. If you apply a patch from a phase, make sure that the patch > appears in the inputs or native-inputs, such that "guix build --source=all" > will include the patch. Should we have an option for "guix build --source=all" to also include the Guix "scripts used to control compilation and installation"? -Philip
Re: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.
On Sun, Jul 24, 2022, at 11:17 PM, Maxime Devos wrote: > * Patches must not be used to remove non-free files, because a patch by > construction contains the non-free file itself so the patch would be > non-free, which would not be acceptable to Guix. Likewise, patches should not > be used to remove bundled libraries, to avoid large space usage, but this is > not an absolute rule unlike as for non-free files. It is possible to create patches that do not contain the deleted file, e.g. with `git format-patch --irreversible-delete`. That said, I don't know if the version of `patch` we use to patch origins is able to apply such patches—but maybe it would be a useful feature? -Philip
Re: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.
Hello! Maxime Devos skribis: > Context: it's currently a mess:, and at times contradictory As a rule of thumb, I’d suggest avoiding denigrating wording like this, in an effort to keep communication smooth and effective. > * There is policy involving those three, as can be seen from the >shepherd mess. What is “the shepherd mess”? Realize also that not everyone may agree that there is “a mess” in the first place. The ‘shepherd’ package uses a snippet to fix a bug. I think that’s akin to applying a patch: the intent is that ‘guix build -S’ gives you the code that’s actually built, with patches applied. > * This policy is partially secret, as can be seen by some people >treating some things as policy even if it's not in the manual. There’s no secret, but there might be unwritten rules. I think what we need to do is improve the “Snippets” section of the manual, as you propose, so we don’t have unwritten rules and misunderstandings based on hearsay. [...] > 20.4.5 Snippets, phases and patches > > Snippets, phases and patches at times serve overlapping purposes. To > decide between the three, there are several considerations to keep in > mind: > > * Patches must not be used to remove non-free files, because a patch >by construction contains the non-free file itself so the patch would >be non-free, which would not be acceptable to Guix. Likewise, >patches should not be used to remove bundled libraries, to avoid >large space usage, but this is not an absolute rule unlike as for >non-free files. > * Snippets are often convenient for removing unwanted files such as >bundled libraries, non-free sources and binaries. It is technically >also possible to use phases for this, albeit slightly less >convenient at times. However, phases must not be used to remove >non-free sources, as then the output of "guix build --source" would >still contain the non-free sources, which is incompatible with Guix' >stance on free software. Likewise, phases should not be used to >remove binaries; however, this is not strictly forbidden. > * Snippets must not embed store items in the source, as this is >incompatible with cross-compilation and prevents effectively sharing >the source code produced with "guix build --source" with people >using non-Guix systems. > * In principle, you can apply a patch from a phase. However, this >causes the result of "guix build --source" to not correspond to the >actual source code anymore (i.e., it doesn't act as corresponding >source anymore), so consider this a last resort for situations such >as avoiding causing a world-rebuild for a patch fixing a >target-specific bug by making the patching conditional upon >target-foo?. If you apply a patch from a phase, make sure that the >patch appears in the inputs or native-inputs, such that "guix build >--source=all" will include the patch. > >@c this relaxes the old rule a little > > * Ideally, the source derived from the origin should be usable for >building on any system that the upstream package supports (even if >Guix does not support that system), as a courtesy to the people that >the source code is shared with. However, this is not an absolute >rule, most important is that it is usable on Guix and it is allowed >to neglect this recommendation when it is tricky to follow or a >large amount of work. For example, if some Windows-specific source >files are non-free, you can simply remove them without replacing >them by a free implementation, even if that would reduce the set of >systems the package can be built on. > > Sometimes, there remains more than one acceptable way to accomplish > the goal. In that case, choose whatever appears to be most convenient. I kinda agree with what Julien wrote. I’d suggest starting with a patch against that section to address one specific point that you think is the most pressing one. From there we can continue the discussion. WDYT? Thanks, Ludo’.
Re: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.
I agree that we should better document that policy. I think we could go further, but it's ok to document the common denominator first. To me, the structure you suggest is not great, but I don't have issues with the content. Le 25 juillet 2022 13:18:28 GMT+02:00, Maxime Devos a écrit : > >On 25-07-2022 07:21, Julien Lepiller wrote: >> I don't like the wording at all. You're mixing too many things together. >Feel free to try to separate the things, but going previous discussions, many >tings are important, and they appear all to be inseparable. >> >> I think it would be better to first document the guiding principles (eg. the >> goal that there are no non-free software in Guix, going for the simplest >> thing, etc) and then derive rules for specific cases, based on these >> principles: >> >> How do I remove non-free software? -> snippet because … >> >> How do I remove bundled libraries? -> snippet or phase because … >> >> How do I fix a build issue? -> patch or snippet if this affects building >> from source, can also be a phase if the result of --sources can still build >> >> A test issue? >> >> … >> >> This leaves some cases up to interpretation, but that's probably not so >> different from "it's not an absolute rule". It's also much clearer and >> quicker to figure out in which case you are. If not documented as a case, >> you can fall back to the general principles. > >TBC, is the issue here the structure of the section, or some individual rules? >In the former case, I could try rewriting it a bit to follow your proposed >structure. > >Greetings, >Maxime.
Re: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.
On 25-07-2022 07:21, Julien Lepiller wrote: I don't like the wording at all. You're mixing too many things together. Feel free to try to separate the things, but going previous discussions, many tings are important, and they appear all to be inseparable. I think it would be better to first document the guiding principles (eg. the goal that there are no non-free software in Guix, going for the simplest thing, etc) and then derive rules for specific cases, based on these principles: How do I remove non-free software? -> snippet because … How do I remove bundled libraries? -> snippet or phase because … How do I fix a build issue? -> patch or snippet if this affects building from source, can also be a phase if the result of --sources can still build A test issue? … This leaves some cases up to interpretation, but that's probably not so different from "it's not an absolute rule". It's also much clearer and quicker to figure out in which case you are. If not documented as a case, you can fall back to the general principles. TBC, is the issue here the structure of the section, or some individual rules? In the former case, I could try rewriting it a bit to follow your proposed structure. Greetings, Maxime. OpenPGP_0x49E3EE22191725EE.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature
Re: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.
I don't like the wording at all. You're mixing too many things together. I think it would be better to first document the guiding principles (eg. the goal that there are no non-free software in Guix, going for the simplest thing, etc) and then derive rules for specific cases, based on these principles: How do I remove non-free software? -> snippet because … How do I remove bundled libraries? -> snippet or phase because … How do I fix a build issue? -> patch or snippet if this affects building from source, can also be a phase if the result of --sources can still build A test issue? … This leaves some cases up to interpretation, but that's probably not so different from "it's not an absolute rule". It's also much clearer and quicker to figure out in which case you are. If not documented as a case, you can fall back to the general principles. Le 25 juillet 2022 05:17:33 GMT+02:00, Maxime Devos a écrit : >Context: it's currently a mess:, and at times contradictory > > * There is policy involving those three, as can be seen from the > shepherd mess. > * This policy is partially secret, as can be seen by some people > treating some things as policy even if it's not in the manual. > * Some versions of the policy are based on archeology, e.g. see the > 'snippets were introduced for this particular purpose, so don't use > it for other things’ which is not documented in the manual as sole > legitimate reason and asking contributors to read all past > discussions seems too much for me. > * Sometimes, people refer to the manual (Snippets versus Phases) for > how things (should) work, but what they say is not actually present > in that section of the manual. > * Some variants of the policy are contradictory with each other (IIRC) > * Some of the policies are contradictory with current practice in > other Guix packages. > * '(guix)Snippets versus Phases says Phases' states that it is elusive. > * The section name implies it's a ‘X versus Y’, which seems > polarizing? (Maybe?) > * The section neglects the is/ought-distinction -- it just says what > is typically used for what, not whether they should be used for them > and whether they are allowed to be used for other things, so that > section does not seem policy to me (except for the single 'should > produce' and 'must not' line), only matters of fact. > >I can't work with such a mess. As such, I've a proposal for a consistent, >clear and non-elusive set of rules and guidelines, based on the following >principles: > > * It appears we cannot agree on what exactly the policy should be, but > having a single policy everyone can use even if some would rather > have the specifics be a tiny bit different, is much better than the > mess of everyone having their own policy. > * There are no absolutes, except that the result of "guix build > --source" must be free software; > * There can be more than one (acceptable) way to do things, but this > doesn't make things elusive, this just means there are multiple > acceptable options and you should probably go for the simplest. > >More concretely, I propose the following new contents for (guix)Snippets >versus Phases (the phrasing could use some work for smooth reading), which I >believe to be sufficiently clear (except for some phrasing that could be >tweaked, e.g. the phrases are currently rather long), covers a sufficient >amount of cases (feel free to respond if you see a missing case), free of >contradictions (likewise) and mostly in line with current practice: > >[start] > >@c: There is no opposition or such, so no versus, let's not start with >polarisation. > >20.4.5 Snippets, phases and patches > >Snippets, phases and patches at times serve overlapping purposes. To decide >between the three, there are several considerations to keep in mind: > > * Patches must not be used to remove non-free files, because a patch > by construction contains the non-free file itself so the patch would > be non-free, which would not be acceptable to Guix. Likewise, > patches should not be used to remove bundled libraries, to avoid > large space usage, but this is not an absolute rule unlike as for > non-free files. > * Snippets are often convenient for removing unwanted files such as > bundled libraries, non-free sources and binaries. It is technically > also possible to use phases for this, albeit slightly less > convenient at times. However, phases must not be used to remove > non-free sources, as then the output of "guix build --source" would > still contain the non-free sources, which is incompatible with Guix' > stance on free software. Likewise, phases should not be used to > remove binaries; however, this is not strictly forbidden. > * Snippets must not embed store items in the source, as this is > incompatible with cross-compilation and prevents effectively sharing > the source code produced with "guix build --source" with people > using
