Re: Cuirass 1.2.0 released

[Katherine Cox-Buday]

On 10/30/23 10:42 AM, Ludovic Courtès wrote:

Hello Guix!

It’s a bit of a non-event since we’ve been packaging and using snapshots
of Cuirass all along, but nevertheless, I’m totally thrilled to announce
that Cuirass 1.2.0 is out!


Congratulations! It might be my holiday project this year to stand up a 
local instance :)

Re: Guix Survey (follow up on "How can we decrease the cognitive overhead for contributors?")

[Katherine Cox-Buday]

On 10/2/23 5:24 AM, Wilko Meyer wrote:


- Where solicitations to complete the survey are broadcast is very
   important. E.g. if we only send it to guix-dev, this skews the
   responses to questions like "where do you talk about Guix".


Definitely, I'm not entirely sure on how to solve this; publishing
surveys on as many channels that seem fitting could maybe mitigate this?
Then again the selection of communication channels is highly subjective
as well.


I don't think we have to be very selective about where to broadcast the 
survey. I think the answer is: anywhere Guix people hang out or anyone 
feels it might be useful to do so.


I think the only danger here is missing some popular hangout spots due 
to ignorance of their existence. We should enumerate them to ensure we 
catch them all.



- When soliciting responses to the survey, it's very important to set
   expectations about the survey in the solicitation. It is important
   to briefly describe what the survey is like and how long the survey
   will take. Without this, some people will have uncertainty about
   what they're committing to and not even try.

- The survey should endeavor to remain on the shorter end; many will
   not complete longer surveys.


This is another good reason to start with a narrow focus on questions
regarding contributions instead of a general survey.


- Does the survey need translation to eliminate language barriers?


Most FLOSS surveys I've looked at were english only; which comes with a
certain language bias. Realistically I'd say that, given a survey may
contain free form questions, translation also seems to be a resource
issue when it comes to analyzing the results.


Does the GNU project have a "general translation" team?

Maybe some of our Guix community members who speak multiple languages 
would be willing to translate the survey into their primary language?



- The survey should use a uniform measurement system throughout. Don't
   use scales with different magnitudes in different questions, and
   don't suddenly invert whether higher is better or worse.


Good point, this also means that questions should be asked in a way,
that they can be answered using the same metrics/scale?


I think that's the idea and should be the rule for which there are 
exceptions.



- As you've already mentioned, free-form questions are very difficult
   to quantify, and I think we should use them with caution.
   Communities rooted in philosophical values, as Guix is, have
   impassioned people and resolving a large number of free-form
   responses to a quantitative statement may be difficult.


My approach to free form questions is to, on one hand try to quantify
trends (things that are mentioned often, key topics that are mentioned),
on the other try to derrive actionable items/issues from them that can
be worked on. Quantifiying qualitative responses is cumbersome, and as
you've also pointed out, quite difficult; but identifying trends/key
topics and maybe actionable items/issues from those could work. WDYT?


My opinion is that we should not do free form questions for this first 
time. We're new at this, we have enough topics to cover, and the topics 
we are covering seem to cause a lot of discussion (that's good) which 
could lead to a lot of text to read through.



- Up front, it may be difficult to identify all the root-causes of
   something the project wants to know about. Instead of trying to
   infer these, ask the questions directly. E.g. instead of questions
   about liking crunchy vegetables, orange vegetables, and root
   vegetables, ask whether they like carrots.

   However, if you think you have some idea of the root-causes, you can
   ask those as well to see if the correlation you think exists does.


If we've a first draft of a prospective, narrowed-down on contributions,
survey, the questions should probably be benchmarked against these
criteria. I revisted my loose collection of survey questions I posted
earlier on here and realized that I probably have to rephrase a vast
majority of those, to be consistent with this.


- You may want to ensure the survey has "marker questions" which
   clearly categorize your responder for you to make it easier to make
   the statements you'd like to make. E.g. if you're interested in
   analyzing what vegetarians vs omnivores think of carrots, ask that
   so you don't have to try and infer it later.


I'll revisit the original thread on how to decrease cognitive overhead
for contributors to see what good markers could be. With a grain of
salt, I think we should figure out ways to identify participants that:

- were contributors to guix before but stopped contributing

Maybe:

 (1) Have you made contributions to Guix in the past?

This is our marker question.

combined with:

 (2) How many contributions to Guix per month would you estimate
 you've made in the past year? (identify ranges we're interested in)

This is a dimension that would be useful to hav

Re: is core-updates still a thing?

[Katherine Cox-Buday]

On 9/24/23 1:27 AM, Andy Tai wrote:

Hi, curious if core-updates still a thing?There seems a branch by
that name and the manual still says patches causing large number of
rebuilds should go to core-updates, at least for these not aiming at a
specific feature branch.

I have the same question.

My understanding is that feature-branches are now the canonical way to 
work on things, and core-updates has become just another kind of 
feature-branch for landing groups of large changes that don't really fit 
anywhere; the difference being that it's now more ephemeral and "as 
needed" instead of the default for large changes.


Do I have that right?

I guess we need to update the manual?

Re: How can we decrease the cognitive overhead for contributors?

[Katherine Cox-Buday]

Imran Iqbal  writes:

> Personally I don't think its fair to ask Guix to move away from emails
> because folks are more familiar with using web browsers for everything.

Imran, you bring up good points. I wanted to state that I share this opinion: 
Guix should not move away from emails.

I view this conversation as discussing root-causes and ways to address those 
root causes through addition, not subtraction.

I hope that helps to assuage your concerns.

Re: Guix Survey (follow up on "How can we decrease the cognitive overhead for contributors?")

[Katherine Cox-Buday]

This is awesome, thanks Wilko!

I've been talking with my wife who is the field of psychology. As part 
of her degree, and as part of her ongoing education, she's studied how 
to design studies/surveys so that they're not biased and don't produce 
contaminated data. She's not an expert at this, but she knows more about 
it than me :)


The things we could come up with which we thought were important to 
consider are:


- You must first define your goals for the survey. Is it meant to see
  who is using Guix? Who is contributing? How they find the
  contribution process? How they find using Guix? There are many
  dimensions, and we may need to create more than one survey.

- The medium of the survey is very important. E.g. some people won't
  reply to a survey served something that uses JavaScript. Some people
  may not be able to reply to a survey unless accessibility concerns
  are met. Some people won't overcome the barrier of having to log in
  to respond.

- Where solicitations to complete the survey are broadcast is very
  important. E.g. if we only send it to guix-dev, this skews the
  responses to questions like "where do you talk about Guix".

- When the solicitations are made is very important. Some religions do
  not allow use of electronics on certain days, or times of the year.
  Some people are away on holiday during parts of the year. Some
  people are trying to meet deadlines during fiscal quarters. It may
  be impossible to accommodate everyone, but giving a little
  consideration to the issue and a sufficient window of time may cover
  most cases.

- When soliciting responses to the survey, it's very important to set
  expectations about the survey in the solicitation. It is important
  to briefly describe what the survey is like and how long the survey
  will take. Without this, some people will have uncertainty about
  what they're committing to and not even try.

- The survey should endeavor to remain on the shorter end; many will
  not complete longer surveys.

- Does the survey need translation to eliminate language barriers?

- The survey should use a uniform measurement system throughout. Don't
  use scales with different magnitudes in different questions, and
  don't suddenly invert whether higher is better or worse.

- As you've already mentioned, free-form questions are very difficult
  to quantify, and I think we should use them with caution.
  Communities rooted in philosophical values, as Guix is, have
  impassioned people and resolving a large number of free-form
  responses to a quantitative statement may be difficult.

- Questions which are intended to solicit a agree/disagree should be
  phrased as "I" questions, e.g.:

On a scale of 1-5, how much do you agree with the phrase "I
like carrots"?

- Questions should not be leading, and be biased towards the positive.
  E.g., with the carrots example, don't do this:

   Carrots are disgusting. How much do you agree with this?

  and don't do this:

On a scale of 1-5, how much do you agree with the phrase "I
think carrots are disgusting!"

- Up front, it may be difficult to identify all the root-causes of
  something the project wants to know about. Instead of trying to
  infer these, ask the questions directly. E.g. instead of questions
  about liking crunchy vegetables, orange vegetables, and root
  vegetables, ask whether they like carrots.

  However, if you think you have some idea of the root-causes, you can
  ask those as well to see if the correlation you think exists does.

- You may want to ensure the survey has "marker questions" which
  clearly categorize your responder for you to make it easier to make
  the statements you'd like to make. E.g. if you're interested in
  analyzing what vegetarians vs omnivores think of carrots, ask that
  so you don't have to try and infer it later.

- We were unable to resolve the question of astroturfing wherein one
  malicious party responds many times to skew the data. This might be
  difficult to address without relying on a vendor who has solved this
  concern somehow, but requires logins, JavaScript, or something else
  people won't use.

And finally, I'd like to suggest:

I think since this is the result of a discussion about how to lower the 
cognitive overhead of contributing, the goal of this initial survey 
should be:


1. To quantify how easy it is to contribute to Guix.
2. To quantify how easy it is to maintain Guix.
3. To correlate (1) and (2) with people's opinion of using email for 
contributions.
4. To correlate (1) and (2) with people's opinion of using a forge for 
contributions.

5. To correlate (1) and (2) with people's opinion on only improving tooling.
6. To be able to do trend-analysis year-over-year on these issues.

I would suggest adding these questions to a survey exploring the 
contribution process:


On a scale of 1-5, 1 being "Strongly Disagree" and 5 being
"Strongly Agree", how much do you agree with the st

Re: OCI-backed Guix System Services

[Katherine Cox-Buday]

On 9/20/23 4:12 PM, Ricardo Wurmus wrote:


To conclude, I'm not advocating for adding OCI-backed services to Guix 
mainstream: in my opinion they should be bootstrapped and
built from source, but I believe the actual "backend" implementation for such 
services could be useful to have in Guix. What do you
think?


I think this could still be a valuable addition to Guix, because it
integrates with Shepherd and thus unifies management of system services
independent of whether they came from Guix or from elsewhere.


I completely agree. It would be so nice to have a unified interface to 
manage everything!

Re: Swineherd: Guix System container manager

[Katherine Cox-Buday]

On 9/13/23 3:06 AM, Ricardo Wurmus wrote:

Hi there,

you know the Shepherd: it is an elegant service manager looking after a
herd of daemons.  Since it can be extended with Guile, I decided to do
just that to add an extra skill to the Shepherd, turning it into the
Swineherd.

The Swineherd is a manager of Guix System containers.  It is implemented
as a Shepherd service called “swineherd” that lets you manage Guix
System containers as Shepherd services.  These system container services
implement a few common features to bring up bridge networking, execute
commands inside the containers, peek at files, and get simple usage
stats.

Swineherd also provides an optional HTTP API server to launch, manage,
and destroy containers through an HTTP client.  This makes the Swineherd
useful for building web applications that launch Guix System containers
on demand.  It comes with an Info manual describing the assumptions, the
required setup, as well as the implemented commands and the HTTP API.

The Swineherd was designed to be used with Shepherd on foreign distros,
so it does not assume to be running on top of Guix System (for better or
worse).

You can find the code here:

 https://github.com/BIMSBbioinfo/swineherd

Of course the Swineherd is also available as a Guix package called
“swineherd”.

If this tickles your interests I’d be very happy to hear your comments
and incorporate patches to extend it beyond my original requirements.


This is really cool stuff! I'd love to see Guix continue to eat the 
dev-ops space, which, in my opinion, loves to stack complexity on top of 
complexity.


Can you share any ways you're currently using this?

Any plans/interest in hooking it into `guix deploy`?

Re: How can we decrease the cognitive overhead for contributors?

[Katherine Cox-Buday]

On 9/8/23 2:37 PM, Ricardo Wurmus wrote:


Liliana Marie Prikler  writes:


Am Freitag, dem 08.09.2023 um 17:27 +0200 schrieb Ricardo Wurmus:

I have the same positive view on our faux ChangeLogs commit messages,
though I also would like to have them generated.  The benefit is
still there: I still get to *review* an effective summary of the
changes before pushing or sending them off for review.  But at least
I don’t have to write them myself.

Now, this is no longer a problem for me because I’ve been writing so
many commit messages over the years (and because I no longer try to
adhere to some poorly specified format), but it *is* a problem for
people that I’ve mentored.

etc/committer.scm and the yasnippets are supposed to alleviate some
of the pain, but I don’t need to think for a long time to come up
with a number of improvements in this area.

Can I assume this to mean it'd take you some short time to think of
snippets that we're currently lacking?  If so, please do contribute
them.  If not, what do you mean then?


I mean that they have plenty of defects.

When I wrote the first few iterations of etc/committer.scm it was only
really meant and good for bulk package updates (= lots of changes across
files, all upgrades).  It couldn’t (and maybe still can’t) reliably
detect added or removed package definitions.  It doesn’t handle changes
to the arguments field.  It’s also terribly slow because it naively
recomputes information for every hunk in the diff, reading package
definitions from the old vs the changed file after every commit.

The update yasnippet repeatedly gets the order of lines wrong when
adding a patch to dist_patch_DATA in gnu/local.mk; it also doesn’t do
what etc/committer.scm is already able to do: detecting changes to
inputs.  Configuring yasnippet is also not trivial for people who don’t
regularly use Emacs (the snippets are tied to modes set by magit).

I think in light of these defects “Uhm, we have snippets?” isn’t a
satisfying response.


Also, from my original message:

   I use the templates provided, but those don't cover all cases, 
and I've

   even gotten feedback in a review to change a message it created.

Re: How can we decrease the cognitive overhead for contributors?

[Katherine Cox-Buday]

On 9/8/23 5:28 AM, Ricardo Wurmus wrote:


Katherine, I’m very happy you brought this up and continue to respond in
this thread to clarify and steer the discussion into a fruitful
direction.  I know I couldn’t do it.  I thank you for this work, and I
hope that the project can come up with ways to lower the barriers to
entry.


Thanks for saying so, Ricardo, and thank you for all of your 
contributions to Guix. When I think fondly of the project, yours is one 
of the names that springs to mind.

Re: How can we decrease the cognitive overhead for contributors?

[Katherine Cox-Buday]

On 9/8/23 6:40 AM, Giovanni Biscuolo wrote:

Ricardo,

Ricardo Wurmus  writes:


Giovanni,


You are obviously free not to contribute your patches upstream but the
fact that you decided not to because it's "too hard" (my executive
summary about your complaints about Change Log content rules) to write
commit messages suitable for contribution it _not_ a Guix maintainers
fault, not at all.


As a former Guix co-maintainer I disagree with this take.  (Nobody even
brought up the word “fault”, which is a particularly unhelpful lens for
understanding social issues, in my opinion.)


sorry for using "fault", I can't find a better term


“too hard” sounds (perhaps unintentionally) derisive.


the complete sentence is: «"too hard" (my executive summary about your
complaints about Change Log content rules)»

what can I add about my intentions?

[...]


It’s not that writing commit messages is hard.  It’s one of many
obstacles,


IMO one of the very little ones


Taking a single step is trivial. Walking a thousand miles is difficult. 
The aggregate is the thing that matters most!


And these are subjective statements, which are bad to rest decisions on. 
I have the opinion that this style of commit message is difficult and 
doesn't have a lot of value; others think it's easy and find a lot of 
value in it.


I don't place much emphasis on my opinion or others' on this, but I 
place an enormous emphasis on the existence of the two groups. We should 
be curious why the two groups hold their opinions, and curious about a 
mutual path forward.


Instead of setting up camp and throwing rocks, let's share a meal and 
create a better way for everyone.



We can’t blame anyone for seeing these two piles and come to the
conclusion that it’s not worth the hassle — especially when operating
your own channel is so easy in comparison.


I'm not blaming Katherine, I respect her decision; I just wanted to say:
please don't blame the guidelines about ChangeLog for the two (or more)
piles.


No hard feeling, Gio. I want to echo what Ricardo is saying: viewing 
these conversations through the lens of blame and fault is not the 
intent, and is not helpful. In my original message I intentionally said 
that this was not a list of grievances, but an attempt to describe the 
"shape" of the problem.


While I do search for root-causes to issues, it's not done with the 
intent to cast blame on something. It's done with curiosity, and the 
desire to make the situation better for everyone.

Re: How can we decrease the cognitive overhead for contributors?

[Katherine Cox-Buday]

On 9/11/23 6:19 AM, Giovanni Biscuolo wrote:


If you want I can add a little bit of Italian attitide at discussing in
detail tiny variations of the same thing :-O... just joking, eh! ;-)


One of the reasons I love working with people from around the world is 
the delight in discovering that despite real cultural differences, we 
are all human.


I think discussing minute details in a search to be perfectly understood 
is a very human thing that transcends cultures :)

Re: How can we decrease the cognitive overhead for contributors?

[Katherine Cox-Buday]

On 9/6/23 3:07 AM, Simon Tournier wrote:


As you said, we are all different, thus it means that any collaboration
cannot be full-frictionless.  Because any social interaction implies
norms and standards.  Norms and standards are by their definition
excluding.

For example, we communicate in English.  It appears to me impossible to
send a contribution without having some basic knowledge of English.  How
do I know that the English I wrote in the docstring, or in the comments
of code, or the name of the procedures, or in the commit message, etc.
how do I know that English is meeting the standard?  There is an
expectation about the English we are using for communicating and that
expectation is complicated enough that it’s easy to get it wrong.  What
is the thing that will tell me that the English I wrote is not meeting
the standard?

Why do we accept this “friction” about English filtering people?


This is an excellent analogy, and a very good parallel to the 
conversation about the Changelog format. It really made me stop and 
think! Thank you!


I can think of a rebuttal, but I'm going to drop this line of 
conversation, as you suggest, since it's not really the point.



I hope that I am demonstrating to always choose kindness.

Well, if we do not have a common understanding about something, then we
cannot communicate about this something, IMHO.  Sharing a common
understanding about something is a core principle to establish
communication and collaboration.

If group A says ’foo’ and group B does not understand ’foo’, this ’foo’
is real for group A but is it real for group B?  Group A and group B
needs to have a common understanding about ’foo’ in order to agree on
how to deal with ’foo’.

My messages in this thread show, I hope, that I am taking seriously this
discussion.  I am doing my best to be empathetic and I am considering
all the concerns.  However, raising a concern does not make it real or
automatically equal with all the others.

 ( Do not take me wrong, I am not saying that for example commit
message format could not be a real friction for some people, I am sure
it is; as using in English is a real friction for some people.  Instead,
I am saying that I fail to get why is it or what makes this commit
message format a real problem. )


Simon, for whatever it's worth, I think you're doing an amazing job. I 
think few people are able to simultaneously not understand something, 
but still engage in thoughtful and empathetic conversation. Really, well 
done.



- Easy is relative: https://youtu.be/SxdOUGdseq4?t=497


Somehow, that’s the remark by Liliana [1],

 Maybe it's time to take a step back and instead of asking “How can we
 decrease the cognitive overhead for contributors?”, we should perhaps
 ask “For which contributors do we want to/can we decrease the cognitive
 overhead?”


That's interesting, because I view this as the antithesis of what Rich 
was trying to convey.


That quote is at the end of a dismissive ad hominem response which has 
grossly misinterpreted this discussion, even attributes it to malice, 
seems to draw the conclusion that contributing to Guix should be left to 
those for whom the current situation is fine, and even intimates that 
those who would like to improve the situation are incompetent.


Here's the quote from Rich's talk:

   The fact that we throw these things around sort of casually 
saying, "Oh I
   like to use that technology because it's simple. And when I say 
simple I

   mean easy. *And when I'm saying easy, I mean, because I already know
   something that looks very much like that*" is how this whole thing
   degrades, and we can never have objective discussion about the 
qualities

   that matter to us in our software.

Rich is saying that there are intrinsic properties to approaches that 
make them simple, but possibly not easy, and that we shouldn't rest our 
arguments on claiming something is "easy", because that term is 
relative, and often related to familiarity. Familiarity is a hard bias 
to overcome.


I'm here to discuss those intrinsic properties, the contributor 
experience, and see where that leads us.


Contextualized, this quote is insinuating that I'm trying many different 
arguments in an attempt to push an agenda, and that because of this, any 
of the points I've made are suspect and should be dismissed.


Read charitably, this quote suggests that there is a singular, best, way 
to do things, and that if it doesn't work for some, the problem is not 
the process, but that "those people" are incompetent.


This is classic gatekeeping.


which is another way, IMHO, to express what I have tried to say with
“range of contributions” in my first message [2].


- Differentiating the types of complexity (importantly defining
incidental complexity): https://youtu.be/SxdOUGdseq4?t=1173


It appears to me that it is also what I have tried to say in my very
first message [2]. :-)

   

Re: Next action, survey?

[Katherine Cox-Buday]

On 9/6/23 3:34 AM, Simon Tournier wrote:


Well, from my point of view, the next steps are:

  + Propose a survey (questions), open a new thread (or a bug report) for
iterating.


I'll open a bug report and start logging notes/suggestions against it. 
In it, I'll include your points:



  + Review how to distribute the survey >   + Prepare the skeleton for 
generating a report about the stats from the
survey.

Maybe the person or group of people in charge of this survey could be in
touch with Emacs folks.  They did a survey on 2022 [1].

1: https://lists.gnu.org/archive/html/emacs-devel/2022-10/msg01824.html


I'll start reaching out to people, specifically people who have done 
surveys for projects similar to Guix.


I'm not an expert in how to design surveys, and I assume there are some 
out there who could help us design one to do things like avoid skewed 
data (e.g. your point about how the survey is distributed), and not 
frame questions in a way that elicits a certain response.



A survey and the associated report will act similarly: list what we more
or less already know, and then make easier the path for improving.


Agreed! I think the trend-analysis of survey responses over time will be 
most interesting.

Re: How can we decrease the cognitive overhead for contributors?

[Katherine Cox-Buday]

On 9/5/23 2:43 PM, Liliana Marie Prikler wrote:

Am Dienstag, dem 05.09.2023 um 19:40 +0100 schrieb (:

Liliana Marie Prikler  writes:

Uhm, we have snippets?


Well, those are exclusive to Emacs :)  And without regard to /that/
issue, I do think that there's a problem if the commit format is so
complex that it's not trivial for anyone new to the project to write
them out manually.

By definition, no amount of typing is non-trivial, safe for the empty
amount, which good luck trying to commit your changes by pure mouse
movements, I guess?

Now, if you excuse my French, I think the problem isn't really as much
that people struggle to type out the perfect ChangeLog on the first
try, which also makes it odd to request a linter.  Bear in mind that
committers will sign off anything that appears convincing enough, even
if there are smaller mistakes in the message.  Trust me, I've been
there and seen that; and also done it myself.

Instead, we have seen in this thread appeals to age, appeals to
perceived lack of personal benefit, and now appeals to typing effort,
none of which really make that great of an argument against the
ChangeLog style, especially when they come in combination with a
refusal to make use of already provided tools.  I think we're starting
to see the moving of the goal post as the actual game here.

Maybe it's time to take a step back and instead of asking “How can we
decrease the cognitive overhead for contributors?”, we should perhaps
ask “For which contributors do we want to/can we decrease the cognitive
overhead?”  We have drifted much from the original post that discussed
moms with full-time jobs, who struggle to do “difficult” tasks
(simplified wording; may change the meaning of the OP a little).  Now,
I personally struggle to see how your personal preference for
communication media, commit message style, and other things that were
discussed in any of the preceding threads actually correlate with being
a parent.  However, I do know that with its 150 million users, most
people of the world don't have a Github account.  Being one of the 4
billion email users out there is a comparably low barrier of entry
imho.  So, whose cognitive overhead do you want to reduce (besides the
obvious "my own", which everyone always tries)?


Liliana, many of your responses in this thread are not OK. They have 
been dismissive and have strayed into personal attacks. At times I'm not 
sure if you're intentionally misinterpreting the conversation to try and 
make a point.


If you disagree with something, or have a point to make, please do so 
directly, and without attacking anyone's character or making veiled 
innuendos about their motives or capabilities.


Speaking for myself, I think if you asked me questions instead of 
inferring narratives, you'd find that our values and preferences are 
probably more closely aligned than you realize.


We're all here to try and make Guix better, but we can't do that in a 
hostile environment.

Re: How can we decrease the cognitive overhead for contributors?

[Katherine Cox-Buday]

On 9/5/23 5:55 PM, Simon Tournier wrote:

Hi Katherine,

I am fully aligned with the Survey proposal; at some past Guix Days I
remember discussing a kind of survey á la Haskell or Emacs surveys.
Well, such appears to me very informative to better know how to improve,
from user to contributor.


OK, great! What are next steps, and how can I help?


On Tue, 05 Sep 2023 at 12:00, Katherine Cox-Buday  
wrote:


1. A list of the issues
2. How we'll measure that they're issues
3. How we'll measure improvement against the issues
4. How we'll address the issues

So often in my long career I've worked with organizations/people that
really want to skip to (5): implement something.

Implement a vertically-integrated solution to gather feedback against
reality: yes. Jump straight to the "final solution" with all the details
managed: no.


Since I am French and it is easier for me to comment about my
disagreements than the converse. ;-)


You keep mentioning you're French as some kind of disparaging thing, but 
all the French people I've met (especially our Guix friends) have been 
kind and thoughtful :)



In my career, I have seen the converse: spending plenty of time cooking
all the details of the plan and then being bitten by implementation
“details” that were impossible to predict beforehand.

Instead, I often see the incremental improvements more productive.
Especially when people are volunteers for their contributions and so
their motivation is variable and non-fungible.


I think you've just said the same thing I've said: implement a 
vertically integrated proof of concept and start iterating.


But you need to know the requirements first because even the basic shape 
of your implementation can swing wildly depending on these. E.g. a 
service with three nines of availability can be a fundamentally 
different service than one with four.



I think we have

  #1. a list of issues; some points discussed in this thread
  could be reported as bugs.
  #4. an hope for addressing some of them using a plan (scripts, fix guix
  edit, etc.)

As we discussed earlier, the ’measures’ are the difficult part and there
is too many bias that it’s almost impossible to measure.  At best, the
number of contributors (#2) and some stats from some survey (#3) would
provide some indicators.


I think almost all surveys ask subjective questions. The bias is OK, 
because in aggregate, it at least points towards a consensus or 
indicates that there are undiscovered problems.


For example: "The Guix logo should be blue: agree/disagree?". This is 
completely subjective, but if 95% of responses disagree, and they 
disagree year-over-year, that's an indication that there's some reason 
it shouldn't be blue.


*Maybe* we even find out that a lot of Guix users are color-blind and 
blue just looks gray.


The point isn't that you practice direct democracy and subject yourself 
to the shifting opinions of the responders. The point is to gather data 
and try and draw conclusions based on what you see.


We should probably go ahead and do what we think is best to improve the 
situation, but we should probably also have a standing question about 
this that we ask in our survey to see if the situation is getting better 
or worse.



Maybe I misread, I think we have a consensus about the issues and
about the concrete actionable next steps, no?

  a. Prepare a survey
  b. Report bugs about “guix style” and “guix edit” (and I am probably
 missing other reported in the thread :-))
  c. Look to Build Coordinator and QA [A,B] (ask Chris?) and try to
 extract what this CI does.

Well, from what my opinion is worth here.

A: https://git.savannah.gnu.org/cgit/guix/build-coordinator.git
B: https://git.savannah.gnu.org/cgit/guix/qa-frontpage.git/


That all sounds good to me.

Before I opened this can of worms, I was planning on trying to work on 
the Go branch/packages 
(https://lists.gnu.org/archive/html/guix-devel/2023-08/msg00058.html). 
But since I'm the cause of all this, and it will indirectly help me with 
Go things (which is why I brought it up), how can I help?

Re: How can we decrease the cognitive overhead for contributors?

[Katherine Cox-Buday]

On 9/5/23 4:57 PM, Simon Tournier wrote:

Hi,

On Tue, 05 Sep 2023 at 11:01, Katherine Cox-Buday  
wrote:


Well, somehow, I consider the commit message format similarly as coding
style.  We can discuss which one is better than the other when at the
end it only reflects some artificial preferences and for the sake of any
project one needs to be arbitrarily picked.  Why not ChangeLog?

The distinction I draw is that I can usually run a linter against a
coding style.

I don't care very much what the standard for commit messages is other
than if it has an expectation of structure, I be able to run a tool to
tell me if it's wrong.

In other words, the overhead isn't "I don't like this standard", it's "I
can't find a way to reliably adhere to the standard".

Well, I am not sure to fully understand what you have in mind with the
term “standard“.


I'm sorry if I've not explained this well, Simon. Thank you for trying 
to understand.


By "standard" I mean the GNU Changelog format 
(https://www.gnu.org/prep/standards/standards.html#Change-Logs). As in: 
it's expected that commit messages use this format.



   To me, coding style or similarly commit message format
are about standard or norm, meaning they respect a set of rules.

The question is then: is it possible to explicitly write down all the
rules?  Are all the rules all well-defined or are some ambiguous?  etc.

For some norm or standard, it is possible to have a checker because all
the rules are explicitly well-defined.  For many norms/standards, we do
not have any checker.


This is my point. There is an expectation, but no way to check against 
that expectation, and the expectation is complicated enough that it's 
easy to get wrong. The Changelog format is like a little language with 
conditionals about how "simple" the change is, and line length, and 
blank lines.


In my response I was trying to point out a flaw in your comparison: that 
with style guidelines, which are also complicated, there is usually a 
formatter that will do it for me, or a linter that will tell me that 
something is not meeting the standard. This is because languages have 
grammars, and linters have higher-order system grammars.



Now, I have read the thread and I hear the comments about the commit
message format as ChangeLog.  To be honest, I am somehow surprised.  If
after being enough annoyed by something that then one clones the Guix
repository, finds how to improve and last drops all because writing the
commit message is too “complex” or because one does not know if the
commit message correctly adhere to the standard… Sorry, I do not buy.


I don't think anyone has said this. To be explicit: I am not saying 
this. What my original message said, and what others have said, is that 
in aggregate, the steps to contribute produce enough friction so as to 
keep contributions from being proposed and merged. The commit messages 
are part of that aggregate.


Here are others saying this in this thread:

    - https://lists.gnu.org/archive/html/guix-devel/2023-09/msg00040.html

    - https://lists.gnu.org/archive/html/guix-devel/2023-09/msg00051.html

Here is my channel with things I intend to upstream, but haven't, 
largely because of this friction. It includes services and packages.


https://github.com/kat-co/guix-channels/tree/upstream-staging/upstream

For whatever else has been brought up in this thread, I started with this:

    I have given a list of issues to a group of people who are presumably
    analytical, and I think the natural inclination is to go 
point-by-point and
    make arguments for/against. Instead of that[*], I invite you to 
address the

    more abstract issue: (should/how do) we reduce friction for making
    contributions?


And I do not buy either an issue when resuming after an interruption
because writing commit message can be done from the diff.  More than
often, I tweak stuff, then commit with the oneline subject ’DRAFT foo’,
continue to tweak, commit ’DRAFT bar’.  Days or weeks (or months) later,
I resume my work and run “git rebase” for polishing the commit ’DRAFT
foo’ and preparing it for submission.

Again I hear all the comments and I am trying hard to understand.  From
my point of view and from where I stand, my understanding is that the
core point of commit message format is about 1. discipline – the quality
of being able to behave and work in a controlled way which involves
obeying particular rules or standards – and 2. confidence – the willing
to send the perfect message on the first try.  And there is no tool for
fixing these both issues.


In the US, the phrase "I don't buy it" is usually the response to 
someone trying to trick you into something. This is a little hurtful 
because it's either saying:


"You have an ulterior motive and are trying to trick me into doing 
something."


or

"I don

Re: How can we decrease the cognitive overhead for contributors?

[Katherine Cox-Buday]

Thank you for your thoughtful comments, Giovanni!

On 9/2/23 5:16 AM, Giovanni Biscuolo wrote:


1. We should use sourcehut or continue to improve mumi


Please forgive me if I insist, but the one and _only_ benefit of using
SourceHut is the web-UI /helper/ to prepare an email message to send,
it's "just" a web-UI version of the "git format-patch" CLI; the rest of
the "patch management workflow" is email **and** CLI (git am) based;
it's documented.


I enumerated the underlying issues that both my suggestions would address:

   - QA status should be visible from a patch's page
   - It should be possible to interact with the issue through the page

I know everyone is focusing on email vs. web-forge, but I am trying to 
draw attention to the root causes of the complexity, and enumerating 
possible solutions to these.


Please notice that under my first suggestion, I was calling out points 
about how QA status shows up, and switching between tools to interact 
with issues, not specifically how to prepare/apply patches.



Furthermore, users that are comfortable with the SourceHut web UI are
free to use that as their personal working repo, there is no need for
Guix to use a SourceHut remote as the official one.


Defaults matter, and I don't view it as a valid starting-point to state 
that contributors can manage intrinsic complexity on their own. I think 
that's where we're at right now, and it's why I created this thread.



     - QA status should be visable from a patch's page


On mumi web interface, in each issue page related to a patch, there is a
"badge" linking to the QA status for that patch, right below the issue
title; i.e.:

https://issues.guix.gnu.org/65694

have a link to https://qa.guix.gnu.org/issue/65694

QA (and relates services, like data.qa) is a great project that could
greatly improve current situation when completed!


Agreed! I called out Mumi as helpful but also highlighted some areas for 
improvement:


In a web-forge, I generally have a URL I can go to and see everything 
about my patch. I think we have that with https://issues.guix.gnu.org 
with two exceptions: (1) QA is a click away, or if you're using email, 
you're not even aware that there's a QA process failing (2) If you're 
not using email, context-switching between this page and email to respond.



- It should be possible to interact with the issue through the page


I don't exactly understand: what do you mean with "interact"?


It should be possible to perform almost all meta-activities against an 
issue through its page:


- Manage status (close, re-open)
- Manage tags
- Request reviewers
- See CI status including high-level reason(s) for failure
- Apply to parent branch
- Respond to comments


...and what page?  https://issues.guix.gnu.org/ or
https://qa.guix.gnu.org/issue/65694 (or any other issue)


Right now I'd hold up https://issues.guix.gnu.org/ as our best example.


2. We should create scripts/sub-commands to lift contribution activities
into
     higher-order concepts:
     - Prepare a new submission
     - Run pre-checks on a submission
     - Submit a patch
     - Status of patch


AFAIU you already use some of this "lifting" scripts od commands: can
you please send patches so thay could possibly be included in Guix
proper or in some section of the Cookbook?


It's not a stand-alone script that I can share unfortunately. It's 
functions integrated into a more complicated system. However, it's 
pretty straightforward to recreate: anywhere there's a decision-point, 
infer the correct decision (e.g. check dependencies on a package to 
determine which branch a commit should go in), execute the list of 
steps, give a human-friendly description of anything that went wrong, 
and offer suggestions on how to correct it.



For me, steps 20-23 are bothersome. There's a lot of "if" statements
that lead
to branching operations, and a lot of commands and flags to get
right.


oh yes, CLI is a cognitive overhead sometimes, so we need better
interfaces, some have found them


I loathe working with workflows that don't have integration points, and 
so I *love* CLI tools. But I don't like having to be the computer and 
walking through a workflow while filling all the blanks in.



actually, point 21 "Run `./pre-inst-env ./etc/teams.scm cc-members
` to get the CC flags for Git" is bothersome and we should find a
way to better integrate that in "git format-patch" (so that will be
automatically used in all the git interfaces we use)


The extra step to get a debbugs ID is annoying.


have you tried mumi CLI with the new feature?


No, I've never used the mumi CLI.


Forgive me if I insist: that forge site is _not_ SourceHut

Second: each forge web site have a custom (not standard) way to manage
pull-requests.


OK. I am not particularly attached to any one tool. I only care about 
reducing the complexity. I was responding to a message requesting 
specific suggestions, and Sourcehut has been brought up.



Third: git have a pul

Re: How can we decrease the cognitive overhead for contributors?

[Katherine Cox-Buday]

On 9/3/23 1:36 AM, Ricardo Wurmus wrote:


Why are we trying to compete with Sourcehut?  What's the end goal?


I can’t make myself even clearer about this, but I’ll repeat myself
again: we are not.

“Competition” is a foreign concept to me.  Mumi exists only because I
found the Debbugs web interface kinda ugly and alienating.  It’s a hack
that obviously outlived its welcome.  I have to admit that messages like
this make me regret scratching this itch that gave rise to Mumi in 2016.
(Sourcehut started in 2018.)


I think you've already addressed this, but I wanted to plainly state 
that I appreciate your work on Mumi. Even if we decide to move on, it 
fulfilled a need for this period of time in Guix's life, and I thank you 
for it.

Re: How can we decrease the cognitive overhead for contributors?

[Katherine Cox-Buday]

On 9/5/23 8:01 AM, Simon Tournier wrote:

Hi Katherine,

Thank you for your extensive analysis.  I concur.

On Wed, 30 Aug 2023 at 10:11, Katherine Cox-Buday  
wrote:


3. We should reify a way for Guix, the project, to measure, and track
progress,
     against long-term goals. Particularly when they're social and not
strictly
     technical.


That is the most difficult part, IMHO.  Well, what are the long-term
goals? :-)

I am almost sure we will get various answers depending on people.  Let
say the long-term goals of the Guix project are: Liberating, Dependable
and Hackable.  Then how do you give concrete quantities that we can
measure or track?


I think starting at the top and trying to derive concrete quantities 
from values is a healthy exercise. I agree that it's difficult, but 
without doing that it's easy to be left in an echo-chamber where your 
project isn't actually accomplishing any of the things you'd like it to.


However, my point (3) above is a little different and easier. Here we 
have a lower-level goal that is a closer to a concrete quantity: "The 
overhead of contributing to Guix should be low." There are various ways 
to reduce this further, but they are tightly coupled to the method of 
measurement. In other words, what you measure will be what you manage, 
so choose what you measure carefully.



And it is always difficult, if not impossible, to measure or track some
goals that are not technical but social.  For example, how do you
measure being welcoming or being a safe place for all?


I think the easiest way to start, and something that's actually pretty 
effective, is to start doing annual surveys, e.g.:


- https://discourse.nixos.org/t/2022-nix-survey-results/18983
- https://survey.stackoverflow.co/2023/
- https://tip.golang.org/blog/survey2023-q1-results

This thread turned out to be an informal survey, and I think it's easy 
to see that some people are happy with how things are, and some people 
would like to see change.


With a survey you can quantify these opinions and say things like "X% of 
people would like the current contribution process to remain the same. 
Y% of those are committers."


This can help reveal larger patterns, and over time, trends.


Do not take me wrong, I strongly think we must think again and again on
that point for improving.  It’s just easier to tackle technical bug. :-)


It's definitely not as easy as tackling a technical bug. I think as 
engineers we often have a difficult time admitting to ourselves that 
efficacy is much more than the perfect algorithm or the mechanical 
shuffling of things around. Do we want Guix to be a fun technical 
exercise? Or do we want it to fulfill its stated goals? Why not both! :)



Here I see two annoyances:

  1. The number of subcommands and steps.
  2. Each subcommand has a list of options to digest.

Well, CI is helpful here, for sure.  However, it would be helpful to
have a script similar as etc/teams.scm or etc/committer.scm that would
help to run all these steps.


Yes, and commonly whatever you would use for CI is the same thing you 
would run locally. This is intentional so that you can have some 
confidence that CI will pass.



It does not mean that all these steps need to be run before each
submission.  However having a tool would help irregular contributors or
newcomers; it would decrease the cognitive overhead i.e., that overhead
would be pushed to some script and it would reinforce confidence.


And then although we've reduced the cognitive overhead globally, we've 
increased it locally by introducing another conditional: "do I need to 
run the CI script? how do i know?" The script could probably decide this 
too.



Now someone™ needs to implement this script. ;-)


Collectively, I don't think we've arrived at a consensus on

1. A list of the issues
2. How we'll measure that they're issues
3. How we'll measure improvement against the issues
4. How we'll address the issues

So often in my long career I've worked with organizations/people that 
really want to skip to (5): implement something.


Implement a vertically-integrated solution to gather feedback against 
reality: yes. Jump straight to the "final solution" with all the details 
managed: no.



To be fair, here you forget one important blocker: having an account to
the forge website.

I do not speak about freedom issues.  Just about the fact to open an
account to the forge website.  For example, let consider this project:

 https://framagit.org/upt/upt

And if I want to contribute with a Merge-Request, I need to open an
account to the Gitlab instance of FramaGit and push my code to this
instance, even if I already have my fork living on my own Git
repository and I have no plan to use this FramaGit forge.


You're correct, you usually have to have an account on some forge 
website. Think of how various s

Re: How can we decrease the cognitive overhead for contributors?

[Katherine Cox-Buday]

On 9/4/23 7:32 PM, Maxim Cournoyer wrote:


   13. Run `guix style` (this is still in the manual although I have since
   learned this is not actually advisable).


The intent is for it to be advisable -- when it's not, a bug should
be/have been reported against it to track its resolution.


Thanks, Simon. I only became aware that it has quirks and should not be 
blindly trusted by reading through this thread. In the future, I'll 
modify my workflow to still run it, but treat it as untrustworthy, check 
its results, and report bugs against it.

Re: How can we decrease the cognitive overhead for contributors?

[Katherine Cox-Buday]

On 9/5/23 10:01 AM, Simon Tournier wrote:


Well, somehow, I consider the commit message format similarly as coding
style.  We can discuss which one is better than the other when at the
end it only reflects some artificial preferences and for the sake of any
project one needs to be arbitrarily picked.  Why not ChangeLog?


The distinction I draw is that I can usually run a linter against a 
coding style.


I don't care very much what the standard for commit messages is other 
than if it has an expectation of structure, I be able to run a tool to 
tell me if it's wrong.


In other words, the overhead isn't "I don't like this standard", it's "I 
can't find a way to reliably adhere to the standard".

Re: How can we decrease the cognitive overhead for contributors?

[Katherine Cox-Buday]

Summary of my conclusions:

1. We should use sourcehut or continue to improve mumi
   - QA status should be visable from a patch's page
   - It should be possible to interact with the issue through the page

2. We should create scripts/sub-commands to lift contribution activities 
into

   higher-order concepts:
   - Prepare a new submission
   - Run pre-checks on a submission
   - Submit a patch
   - Status of patch

3. We should reify a way for Guix, the project, to measure, and track 
progress,
   against long-term goals. Particularly when they're social and not 
strictly

   technical.

On 8/28/23 4:17 AM, Simon Tournier wrote:
> Hi Katherine,
>
> On Fri, 25 Aug 2023 at 19:02, Katherine Cox-Buday 
 wrote:

>
>> I think there's utility in distinguishing between familiarity and
>> eliminating toil. I think it was incorrect of me to suggest forming
>> habits in my original message. I think it's better to focus on
>> eliminating toil so that whatever workflow remains can more easily be
>> habituated.
>
> Since I am French, I am good as moaner and I would like to avoid the
> ramblings of some grumble party. :-)
>
> I agree with you that it is important to reflect ourselves about our
> practises and we need to be critical about what does not suit us.
>
> In order to be pragmatical and list actionable items, could you
> specifically list what you consider as a toil or cognitive overhead?
> Maybe you could share your script helping you.

Yes, great point! Let's try to distill all this conversation down into the
salient points and see if we can't agree on some actionable items.

Here's my understanding of the process to contribute a patch:

  1. Check out main, and run `./bootstrap`, then `./configure 
--localstatedir=/var --sysconfdir=/etc`

  2. Run `make`
  3. You need to determine whether the change can be targeted against 
main or

 needs to target a feature branch, so you go read about that.

[I'm usually starting here]

  4. Run `./pre-inst-env guix refresh --list-dependent `
  5. Create a git worktree for your patch
  6. Run `./bootstrap`, then `./configure --localstatedir=/var 
--sysconfdir=/etc`

  7. Run `make`
  8. Make your changes
  9. Build to ensure your changes are workable.
  10. Try and determine how your changes should be factored into individual
  commits (sometimes it's not always so clear when changing two 
things might

  need to be done atomically).
  11. Try and get each commit message close to correct and commit.
  12. Run `guix lint`
  13. Run `guix style` (this is still in the manual although I have since
  learned this is not actually advisable).
  14. Review the changes these tools have made, and fix things.
  15. Run `guix build --rounds=2 ` to check for idempotency.
  16. Run `make` to ensure you didn't break anything elsewhere in Guix.
  17. Run `guix size` to ensure the closure isn't becoming bloated.
  18. Build all the packages that depend on this package.
  19. Run `guix pull --url=/path/to/your/checkout 
--profile=/tmp/guix.master` to

  ensure you didn't break Guix in a different way.
  20. Run `git format-patch -1 --cover-letter [--reroll-count]`
  21. Run `./pre-inst-env ./etc/teams.scm cc-members ` to get 
the CC flags for Git
  22. Remember that if you're sending multiple patches, an email first 
has to be

  sent to `guix-patc...@gnu.org` to get an ID, and then...
  23. Run `git send-email --to guix-patches  `

I view steps 1-10 as pretty standard "development" steps common to most
projects, although 11 compounds the effort in 10.

In other projects I've worked with, steps 12-19 are commonly done in a CI
pipeline, and courteous people will try to save CI resources by running 
these

steps locally first using some kind of environment identical to what CI runs
(sometimes a container is used for this. I think Guix has better options!).
Sometimes this is not feasible due to asymmetric resources. But having the
option to let CI manage this process is very nice.

For me, steps 20-23 are bothersome. There's a lot of "if" statements 
that lead
to branching operations, and a lot of commands and flags to get right. 
The extra

step to get a debbugs ID is annoying.

If I compare this workflow to the workflow of other contributions I make:

  1-10 as usual
  11. Write a more commonly accepted commit message with no special 
formatting.

  12. Run `git push` (subsequent changes are still just `git push`).
  13. Go to forge website, click button to open a pull-request.
  14. Wait for CI to tell you if anything is wrong.

So now that I've enumerated the steps, compared them to contributing to 
projects
I don't find difficult, and reflected on the difference for awhile, I 
think, at

least for me, the highest friction comes from:

- Steps 11-19, or (to assign it a name for easie

Re: Updates for Go

[Katherine Cox-Buday]

On 8/27/23 9:41 AM, wolf wrote:

Sure, golang compiles faster than C++ for example, but anecdotal data point: at
$DAYJOB we had to start persisting the compiler cache to make CI fast enough.


I've seen similar things done at companies. This is perhaps an 
interesting avenue to pursue later: if Go doesn't support 
linked-libraries by that time, maybe our "linked libraries" could be to 
store what the compiler caches. More investigation needed obviously.


But, Go does compile very fast, and that's a metric the Go team tracks 
and tries to improve.



- IIUC currently the go build system in guix does not reuse build artifacts, 
i.e. it recompiles everything for each leaf package.


That is my understanding as well. We bring in the source for any package 
that requires it.


--

Katherine

Re: Updates for Go

[Katherine Cox-Buday]

On 8/25/23 6:29 PM, John Kehayias wrote:

I've not been following in detail this discussion, but where do we currently 
stand? Is the proposed Go 1.21 patch basically ready?


As far as I know, yes. I've been using it locally since I submitted the 
patch, and things seem to be working as expected. I think the Guile code 
still needs a review, but as far as I know, it's a pretty standard package.



Should we create a branch and build job to start seeing how far we get in 
making 1.21 the default Go in Guix?


I think we should do a two-phase process:

1. Check the package in.

2. Begin working on making it default.

There is utility in having the package available for people to use prior 
to it being the default.



Like others, I have a few random Go packages (a bunch locally I really need to 
clean up too) and am not familiar with the language and our packaging much. 
Still, if I can help review/push some patches and get things moving, please let 
me know.


The following would help:

1. Testing that the new package behaves correctly for them

2. Submitting individual patches for packages they work with to manually 
specify Go v1.21.0 as the version to use. We can then remove this later.


3. Investigating what work will need to be done for our larger Go 
packages with many leaves.


4. Setting up a Go branch to do the work to cut everything over (I've 
volunteered to do this, but I don't know much about it, so if someone 
does and gets to it first that will be a pleasant surprise!).



And thanks for all your work here, it is appreciated!


You're welcome! Thanks for asking!

--

Katherine

Re: How can we decrease the cognitive overhead for contributors?

[Katherine Cox-Buday]

On 8/24/23 12:53 PM, Simon Tournier wrote:

Hi,

At some point, I sympathize.

On Wed, 23 Aug 2023 at 10:25, Katherine Cox-Buday  
wrote:


   I don't use the email-based patch workflow day-to-day, so this is
another area where I spend a lot of time trying to make sure I'm doing
things correctly.


I agree that Debbugs is not handy at all for submitting patches.  Send
the cover letter, wait, refresh email, wait, refresh email, loop until
an issue number is assigned, and then send the series.


Yes, and imagine that between every step, it is likely that something 
pulls your attention away. Maybe you have ADHD, maybe a kiddo is asking 
you endless (but interesting) questions.


This is where the overhead begins to erode contributions. People can end 
up spending their limited resources managing the workflow instead of 
improving Guix.



Reduce friction for making contributions?  Yes, we all want that, I
guess.  The question is how?  What is the right balance between the
change of habits and the satisfaction of people used to these habits?


I think a good start is to eliminate toil for everyone. It doesn't 
matter how something is done if it doesn't need doing! Manually 
organizing/pruning imports is a good example.


The good news is that we are working with computers which, among other 
things, are purpose-built for automating away toil :)



Well, from my point of view, we are using here the term “contribution”
as it was one homogeneous thing.  Instead, I think the term refers to a
range with a gradual complexity.  And the improvements or tools maybe
also need to be gradual depending on this range.

For example, a two-line patch trivially updating one package is not the
same contribution as several-to-many two-line patches more some package
additions for updating all the R ecosystem.  And that’s not the same as
rewriting the Python build system or as packaging the last version
TensorFlow.

The cognitive overhead for these 3 contributions is not the same.
Therefore, the way to reduce the friction will not be the same, IMHO.


That's a good point, but even along that gradient, there's a baseline of 
toil which is not a function of the complexity of the code.


But also, even for complicated things, a tight loop of exploration is 
useful for getting things right. For some, as I alluded to above, it's 
even a critical component for even completing the task.



* We could support a managed web-based workflow


Here, I am very doubtful that it would help.

For instance, Debian is based on Gitlab since their switch from Alioth
to Salsa.  It would be interesting to know if this “new” web-based
workflow using Merge Request is increasing the number of submissions
and/or increasing the number of occasional contributors.


It's a good question.


Another example is Software Heritage (SWH).  Their web-based workflow is
a Gitlab instance [1].  As an occasional person who deal with the SWH
archive, I am never able to find my way and I just roam on #swh-devel
IRC channel asking for help.


I think there's utility in distinguishing between familiarity and 
eliminating toil. I think it was incorrect of me to suggest forming 
habits in my original message. I think it's better to focus on 
eliminating toil so that whatever workflow remains can more easily be 
habituated.



Another example: the channel guix-science [2] based on GitHub.  Well, I
am able to count using one of my hands the number of PRs. ;-) (And I do
not speak about other channels as nonguix)
  
Well, reading the item above about mistake and the item below about

"Guix 'R Us", and maybe I am wrong, somehow I feel that one “cognitive
overhead” is the willing to submit a perfect patch.  Again, maybe I am
wrong, somehow I feel that one part of the issue is a lack of
self-confidence.  Do not take me wrong, I am speaking about submitting a
patch by occasional contributor and not about the personality of person
that I do not personally know. :-)

This “that’s not perfect so I postpone the submission” is something I
often hear by people attending to Café Guix [3].  Hum, I do not know
what we could do differently for reducing this barrier.

The idea of the team Mentor is in this direction but it does not seem
working… :-(

1: https://gitlab.softwareheritage.org/explore
2: https://github.com/guix-science/guix-science
3: https://hpc.guix.info/events/2022/caf%C3%A9-guix/


Similarly, I think there's utility in distinguishing between learning 
and cognitive overhead. I can learn anything, but there are factors 
beyond my control that cause high levels of cognitive overhead to cause 
enough friction to stop me.


Mentorship is a great thing, and I'm sure I"d learn a lot from a mentor, 
but it wouldn't solve the problem I've raised unless the mentor somehow 
showed me how to eliminate the cognitive overhead, in which case: let's 
just do that for everyone.



For sure, I thi

Re: How can we decrease the cognitive overhead for contributors?

[Katherine Cox-Buday]

On 8/24/23 12:33 AM, ( wrote:


* We could support a managed web-based workflow


The problem with this is that it would not be possible without changing
the git hosting entirely to something like Gitea.  I'm personally a fan
of the email-based workflow; what, specifically, is it that bothers you
about it?  If it's:


I can envision some kind of upstream branch that's blessed and merged 
daily. The web crowd commits to that, the email crowd commits to main.



- Setting it up: Yes, this is annoying.  Sadly, our mighty oligarchal
   masters have taken it upon themselves to make it as annoying as
   possible to use email from anywhere but their web or mobile clients.


This isn't it at all, but I agree with your comment. I'm fond of email, 
and it's distressing how centralized it's become.



- Sending the emails: This isn't that bad once you get used to it;
   sadly most Git clients (magit sadly included) don't support send-email
   well or at all.  But on the command line, all you need to do is:

   # for a single commit
   $ git send-email --to=guix-patc...@gnu.org -1 --base=master -a
   # for several commits
   $ git send-email --to=guix-patc...@gnu.org -$N_COMMITS --base=master 
--cover-letter -a

   Or, if sending an amended series:
   $ git send-email --to=$bug_...@debbugs.gnu.org -$N_COMMITS --base=master -a 
-v$VERSION


It's this. Having to:

1. Remember the flags and their values
2. Remember the email address (it might seem silly unless you have forms 
of dyslexia. is it guix-patches? or patches-guix? Wait, what was I doing?)

3. And then the whole deal with what to do with follow ups.

I feel like I know my way around git pretty well, but I struggle with 
how those concepts map onto sending emails.


I have only been able to surmount this by lifting these concepts through 
scripts into higher-order concepts with less cognitive overhead.



- Switching between branches: The best way to handle this is with
   subtrees; see `git subtree --help`.


Interesting! I use worktrees, but maybe subtrees are easier? I'll have 
to read up on this. Thank you!



- Applying patches: This is a bit annoying.  Most email clients won't
   let you set up commands to pipe mailboxes to, unlike aerc.  Perhaps we
   could have a `mumi apply` command to fetch a patch series from debbugs
   and apply it to the checkout.


I wrote some elisp to one-key apply patches from GNUS, but I guess my 
point is: not everyone can do that. How are we to expect more 
contributors if that, or something similar, is the barrier to entry?


--
Katherine

Re: How can we decrease the cognitive overhead for contributors?

[Katherine Cox-Buday]

On 8/23/23 9:33 PM, Ahmed Khanzada via Development of GNU Guix and the 
GNU System distribution. wrote:

My wife and I are currently trying, so I hope to be a busy parent soon too!


Good luck to you!


The debate comes down to: the people contributing the most code already
have a very familiar workflow that they have automated (all probably within
Emacs). Why should they change their contribution model for those who
don't contribute much currently, and may never do so? (Not implying this
is you! Just recounting the debate).


I know it's not your argument, but do you see the circular logic?

This type of "reasoning" is used to great effect in society to maintain 
an "in group" and an "out group". E.g. "There aren't a lot of paralyzed 
people at the top of these stairs, so why should we build a ramp if 
there's no one to use it?"



But you're not asking that; you just want to lower the cognitive overhead.


Thank you for acknowledging my central point.


I can't do much about the brutal learning curve of Emacs, Guix, and GNU,
but I certainly can just package it so it's all ready to go with fancy
scripts for the most common workflows.


I don't mind the learning. I actually think I know everything that 
should be done, it's just very cumbersome and error-prone to do 
everything, every time.


--
Katherine

Re: How can we decrease the cognitive overhead for contributors?

[Katherine Cox-Buday]

On 8/24/23 6:10 PM, Ekaitz Zarraga wrote:


   Lots of important use cases that
Guix could serve are ignored because the people who need them are not
represented in our community and/or they can't contribute and no one is
able/willing to write code for them.


Yes, and even if you manage to write something yourself, many times you
get now answer to your patches because no one else is interested on what
you did. It's perfectly understandable but also discouraging.

Example: I wanted to push Zig support in guix a while ago. Made a build
system and got no answer. 
The feeling here is that the code proposed is not good enough, but I don't
know how to improve it so I'm stuck. It would be great to feel comfortable
enough with the code to be sure that it can be merged, but I can't find
the resources to make it better. If it was clearer or if it was easier
both sides, maintainers and contributors, would be more effective and
happier.


Yes, a point I'm seeing echoed a lot here which I can't highlight enough is:

Any effort made to make it easier to contribute helps everyone, not just 
those having issues.


Your point about your patch going unanswered is indeed a good example. 
Here we sit with no Zig build system. It can't be proven, but maybe if 
contributing and reviewing were easier, we'd have more hands, and 
getting something like that merged would be easier.



* It's OK to make lots of mistakes

The people who have reviewed my code have been generous both with their
time and fixing my mistakes and then applying. Maybe this model is OK? I
still feel guilty every time a reviewer has to correct an oversight I've
made. I also want to become a committer, but I don't know how that would
work if I'm regularly making mistakes. Obviously people would still be
reviewing my commits, but presumably a committer should not regularly be
making mistakes.


Exactly my feeling. And I've been working with Guix for a while...


Same here. My first commit was in 2018.

Re: How can we decrease the cognitive overhead for contributors?

[Katherine Cox-Buday]

On 8/23/23 6:18 PM, Csepp wrote:


* Contributing to Guix is not for you

     I would be really sad if someone ever said this, but I guess it's a
     possibility. Maybe someone like me just can't be a contributor to
Guix until
     I have the bandwidth to manage all the details. I would
preemptively argue
     that diversity of thought and experiences usually leads to better
things.


I really hope we can lower the barrier to entry without sacrificing code
quality precisely because of this.  Lots of important use cases that
Guix could serve are ignored because the people who need them are not
represented in our community and/or they can't contribute and no one is
able/willing to write code for them.


Yes, the goal has to be to lower the cognitive overhead while 
maintaining the level of quality.





* It's OK to make lots of mistakes

     The people who have reviewed my code have been generous both with
their time
     and fixing my mistakes and then applying. Maybe this model is OK?
I still
     feel guilty every time a reviewer has to correct an oversight I've
made. I
     also want to become a committer, but I don't know how that would
work if
     I'm regularly making mistakes. Obviously people would still be
reviewing my
     commits, but presumably a committer should not regularly be making
     mistakes.


In a sense I agree with this, but if mistakes are this easy to make,
then I think something is wrong with the project, not with the
contributor.  Instead of making people learn tightrope walking, maybe we
should be building actual bridges.


Yes! For the same reason that focusing on accessibility helps everyone, 
focusing on making it easy to avoid mistakes helps everyone.



Guix actually fares pretty well in this regard compared to some other
projects I tried contributing to (*stares at Plan 9*), but there is
still a lot of knowledge that experienced developers take for granted
and don't actually document.  Writing new packages is mostly documented
well, but as soon as something breaks, you are thrown into the deep end,
having to dissect logs, bisect commit ranges, learn strace, gdb (which
still doesn't work well on Guix), learn how to compile packages with
debug info (and actually waste some more CPU time and IO on rebuilding a
package you already have), learn how to adapt docs from other distros,
etc, etc, etc.
I've been trying to document these at least for myself, but haven't yet
had time to put them together into something others could read.


A lot of the activities you mentioned fall into the "learn to be a 
developer" category, and I think it's a little too broad of a target, at 
least for what I was trying to point out.



By the way, that's another issue.  Using a TeX based document format for
the docs is, uuuh, maybe not the best idea.  Info is a pretty okayish
documentation reader, but it's a relatively big barrier to entry
compared to what you need to know to make a small edit to the Arch wiki.
This way mostly just experienced contributors write docs, not the
users who just want to document how they made some weird use case
possible.


Another great example. I don't write much documentation because of this.

--
Katherine

Re: How can we decrease the cognitive overhead for contributors?

[Katherine Cox-Buday]

On 8/25/23 3:57 AM, Attila Lendvai wrote:


Otherwise I do not get your point: I keep untreated messages with the latest
patch version in my Guix inbox, and file away the others in a separate mbox.
So things are not flat, but have two levels: "to be treated" or "done".



my point is that in a PR based model/workflow things like this is done by a 
program. and each brain cycle you spend on maintaining the sanity of your local 
inbox, is not spent on hacking, and the results of your effort is not even 
mirrored into the inbox of the other contributors.


I was reflecting on what it is about the email-based workflow that I 
find difficult, and I think you've highlighted one thing:


With a PR based workflow, there is a program essentially figuring out 
the equivalent of the `git send-email` flags and doing the submission 
for me. I generally go to the site for a repo's main branch, get 
prompted about my recent branch, click a button, and it's submitted.


And you've also highlighted the core of my original message: it's 
frustrating to spend effort on the meta of changing code instead of 
changing code. There will always be ancillary effort, but it can be 
greatly reduced.



this seems like a small thing, but multiply this with every message, and every 
potential contributor and maintainer... and then consider its cumulative effect 
on the emergent order that we call the Guix community.


A thousand times this.



meta:

the reason i'm contributing to this discussion is not that i'm proposing to 
move to some specific other platform right now. it's rather to nudge the 
consensus away from the conclusion that the email based workflow is good and is 
worth sticking with.

once/if we get closer that consensus, only then should the discussion move on 
to collect our requirements and evaluate the free sw solutions that are 
available today. which again could be organized much better in a wiki than in 
email threads, but that's yet another topic...


I just want to call out that my original message was not strictly about 
the email based workflow. I want reduction of cognitive overhead to be 
an ongoing goal, whatever that comes to mean.


--
Katherine

Re: How can we decrease the cognitive overhead for contributors?

[Katherine Cox-Buday]

On 8/23/23 12:03 PM, Andreas Enge wrote:

Hello,

Am Wed, Aug 23, 2023 at 10:27:31AM -0700 schrieb Felix Lechner via Development 
of GNU Guix and the GNU System distribution.:

  I can't ever seem to get the GNU style commit messages correct.

Neither can I. The style apparently helps with automated maintenance
of the changelog, but I do not understand why a changelog is useful
for a rolling release model.


personally, I find them super helpful to grep through commit messages
to find changes, like when a file was touched for the last time (now
I think that git wizards will have a better solution; but you get the
idea). Or when a package was added. Or updated to a specific version.


I have no love for Git's CLI (that's one of the reasons the email-based 
workflow grates on me; I use magit for everything), but I found it 
interesting that here you present a valid argument against having to 
learn Git in one way, and elsewhere people are making arguments for 
learning the git send-email command. I draw no conclusions from that, 
but it caught my eye!


FWIW, the git command you could use is:

git log --grep=foo -- the/file/path.scm


* Contributing to Guix is not for you
* It's OK to make lots of mistakes


Definitely "no" to the first one, and "yes" to the second one!
I think that even when one only contributes from time to time, but
regularly, habits will form and mistakes disappear.


I've been contributing to Guix since 2018. I've definitely learned a lot 
about Guix idiosyncrasies, but until I wrote my script, I'd forget to do 
something every time I submitted a patch.


--
Katherine

Re: How can we decrease the cognitive overhead for contributors?

[Katherine Cox-Buday]

On 8/23/23 11:27 AM, Felix Lechner via Development of GNU Guix and the 
GNU System distribution. wrote:



* Encourage upstream communities like "Guix 'R Us"


Every contributor should have their own channels for packages [1] and
for Guix. [2] Testing patches before they are submitted would vastly
improve the code quality in Guix.

Just fork my repos on Codeberg and use the 'prebuilt' branches. (Also,
please tell me when to advance them to more recent commits.) Here is
how you use them via Guix Home. [3]


I do exactly this! My channels can be found here: 
https://github.com/kat-co/guix-channels/tree/master

How can we decrease the cognitive overhead for contributors?

[Katherine Cox-Buday]

Summary: for people who don't contribute to Guix a lot, each 
contribution has

very high cognitive overhead. Can we work to reduce that?

Hey all,

Contributing to Guix is currently pretty difficult for me. I'm a Mom with a
full-time job, and anything outside of my job that requires a lot of 
executive
functioning (e.g. lots of manual, detailed, steps) is very difficult for 
me to
get correct. That last part is what I wanted to discuss, because that's 
the part

that prevents me from contributing more than I do, and I think there are
probably others whom are impacted by this.

When I have some time to contribute, I usually stall out at the "submit 
this for

review" because of the amount of cognitive overhead required. I've written a
script for myself that tries to perform all the steps including running 
the git
command to submit the patch, and this has helped me, but that this is 
necessary

for me to do might be something that, if addressed, could help others.

Here are some examples of things that cause issues for me. This is not a 
list of
grievances; it's my way of attempting to demonstrate the "shape" of the 
issue:


    I signed up on Savannah with the intention of applying to be a 
committer.

    Savannah closed my account one or two days later due to inactivity.

    I can't ever seem to get the GNU style commit messages correct. I 
use the
    templates provided, but those don't cover all cases, and I've even 
gotten

    feedback in a review to change a message it created.

    I don't use the email-based patch workflow day-to-day, so this is 
another

    area where I spend a lot of time trying to make sure I'm doing things
    correctly.

    My script runs `guix style` and `guix lint`, but its suggestions aren't
    always correct. I suspect I've submitted some patches with nonsensical
    changes due to implicitly trusting these tools.

    Maybe 
https://lists.gnu.org/archive/html/guile-devel/2023-02/msg00030.html

    addresses this, but manually managing Guile imports is laborious. As a
    fledgling schemer, I often forget which imports I needed just for
    development.

I think I would summarize the core of these examples as:

    "At every step of the contribution process, I must manually check that
    multiple things are correct. With limited available executive 
functioning,
    and no automation, this is very difficult to do correctly, and an 
easy place

    for contributors to stop."

I think that if I were working on Guix more frequently, I would build 
habits I

didn't have to think about, and these wouldn't be the large issues they are
for me. But because of the nature of a project like Guix, we want 
contributions
from people of all kinds of backgrounds, not just people who have the 
privilege

to work on Guix a lot.

I have given a list of issues to a group of people who are presumably
analytical, and I think the natural inclination is to go point-by-point 
and make

arguments for/against. Instead of that[*], I invite you to address the more
abstract issue: (should/how do) we reduce friction for making contributions?

Here are some things I've considered:

* Contributing to Guix is not for you

    I would be really sad if someone ever said this, but I guess it's a
    possibility. Maybe someone like me just can't be a contributor to 
Guix until
    I have the bandwidth to manage all the details. I would 
preemptively argue
    that diversity of thought and experiences usually leads to better 
things.


* It's OK to make lots of mistakes

    The people who have reviewed my code have been generous both with 
their time
    and fixing my mistakes and then applying. Maybe this model is OK? I 
still
    feel guilty every time a reviewer has to correct an oversight I've 
made. I
    also want to become a committer, but I don't know how that would 
work if
    I'm regularly making mistakes. Obviously people would still be 
reviewing my

    commits, but presumably a committer should not regularly be making
    mistakes.

* We could support a managed web-based workflow

    I think this has been brought up a lot of times, and I don't have a 
clear
    idea of what's been said. But I think a lot of developers these 
days are
    more familiar with this style, and we could still maintain 
self-sovereignty

    if we hosted something.

* Encourage upstream communities like "Guix 'R Us" 
(https://sr.ht/~whereiseveryone/guixrus/)


    Maybe Guix proper continues as is and we foster various upstream 
communities

    that utilize different styles and batch their contributions?

What do you all think? Does this affect anyone else?

[*] But if you have workflow suggestions, I'm all ears!

--
Katherine

Re: Updates for Go

[Katherine Cox-Buday]

On 8/22/23 8:24 AM, Felix Lechner via Development of GNU Guix and the 
GNU System distribution. wrote:

Hi Attila,

On Tue, Aug 22, 2023 at 6:14 AM Attila Lendvai  wrote:


currently the go build system in guix does not reuse build artifacts


Can Golang reuse build artifacts?


I don't think it's recommended right now. See discussion here:

- https://github.com/golang/go/issues/47788#issuecomment-1019562695
- https://github.com/golang/go/issues/47788#issuecomment-1258773167

Summary:

It sounds like due to the way the compiler can optimize code (e.g. 
inlining), `buildmode=shared` is not recommended, but in the future they 
are looking at allowing linking against a single shared library.


--
Katherine

Re: Updates for Go

[Katherine Cox-Buday]

On 8/22/23 12:06 PM, david larsson wrote:

Im not a fan of Go, but I've wanted to package some Go packages. Ive 
only managed to write 2 packages for my private channel so far, but they 
were simple. If there is a guide or so somewhere explaining how to do 
this, then maybe I could complete and contribute a package that I am 
working on (with its dependencies).


The best we have currently are:

- https://guix.gnu.org/manual/en/guix.html#index-go_002dbuild_002dsystem
- https://guix.gnu.org/manual/en/guix.html#index-go

Guix packages are always welcome! The biggest help at the moment would 
be helping to bring existing packages onto Go v1.21.0 so that we can 
make that the default, and improving the Go importer so that people like 
you can just run `guix import go ` most of the time and get a 
working package out of it.



Thank you for giving Go in Guix some attention.


You're welcome! I love Guix and appreciate the work of all the other 
maintainers. Together, we have something special :)


--
Katherine

Re: A certain new commiter

[Katherine Cox-Buday]

On 8/19/23 11:40 AM, Hilton Chain wrote:

Hello Guix,

With the commit [1] made hours ago, I have been granted commit access
to Guix repository.


Congratulations and welcome!

Re: Updates for Go

[Katherine Cox-Buday]

On 8/21/23 11:53 AM, Felix Lechner wrote:

Hi,

On Mon, Aug 21, 2023 at 9:11 AM Katherine Cox-Buday
 wrote:

the immediate emphasis should be on making bringing
our Go ecosystem onto a supported version of Go

 From my experience of packaging Gocryptfs in Debian and here, perhaps
some reconsideration should be given to the widely unpopular idea of
using more package functions in Guix. Ending in asterisks, they would
allow packagers, and perhaps even automated tools, to select exactly
the particular versions or commits specified for each prerequisite in
go.mod.
I have questions about this, but for now, I'm going to try and keep 
bringing the conversation back to just getting our ecosystem compiling 
on a supported version of Go.

Re: Updates for Go

[Katherine Cox-Buday]

Summary: these are good things to talk about. I think we should focus on 
using the current approach to get our Go ecosystem into a supported 
state before we talk about these things.


On 8/19/23 5:31 AM, Attila Lendvai wrote:

at one point i tried to compile some large projects written in golang in a 
reproducible way, and making sure that they use the exact same versions of all 
their dependencies.

in short: there's a philosophical mismatch between how guix and the golang crowd looks at 
building go apps. guix likes to explicitly represent every dependency in the guix 
namespace. golang has its own way of gathering the dependencies (and the nixos crowd 
don't mind "vendoring" the dependencies).


Yeah, there's a lot of unpack in this space for sure. To repeat what I 
think you're saying to ensure I understand it, the way I would 
characterize it is that there are two concerns:


1. The version of module dependencies a module is built with

2. How the toolchain resolves dependencies


(1) is not a new or unique problem to Go. Distros and software have 
always had mis-aligned views on what specific versions a package should 
use. Distros want to maintain fewer packages, and developers want to 
reference specific versions of dependencies. Guix is in a a better 
position in that it's theoretically possible to carry many different 
versions of the same package without conflicts. My understanding is that 
the limiting factor now are the resources that go into doing that (e.g. 
build-farms, cognitive overhead, etc.).


I don't think (2) is actually a problem. A lot of projects like to 
"vendor" their dependencies (i.e. check in the version of their 
dependency in the same repository as the host module), and there's valid 
reasons to do that. Fortunately, Go's vendoring is strongly tied to 
specific versions and the code is checksummed to ensure the vendored 
copy isn't modified. So it follows that all Guix has to do is ignore 
vendored dependencies and refer to the exact same version of the Guix 
package.


That is to say, at least the way I've framed it, I think (1) is the root 
of any issue that exists.



i've also worked on the golang importer (https://issues.guix.gnu.org/55242 
which needs a bit more care). once it works reliably enough, then it'd be 
possible to import the entire transitive closure of the dependencies into an 
isolated namespace in guix, which would be a halfway solution between the two 
conflicting philosophies.
Very nice! Our Go importer has improved so much, and further 
improvements such as these will make managing the Go ecosystem even easier.

for now i've abandoned that endeavour in favor of adding binary packages to my 
channel, but i made some notes on the way:

https://issues.guix.gnu.org/43872

a go-build-system patch
http://issues.guix.gnu.org/50227
discussion with iskarian
https://logs.guix.gnu.org/guix/2021-08-31.log#024401


I don't think this is an issue specifically with Go. As long as I've 
been involved in software, there has been an issue with N software 
depending on M versions of the same thing.



https://logs.guix.gnu.org/guix/2021-08-31.log#180940
iskarian: If you search for my name and "go" or "golang" in the 
mailing list archives, you should find a lot of discussion
https://savannah.gnu.org/users/lfam
Here's a more recent message from me: 
https://lists.gnu.org/archive/html/guix-devel/2019-03/msg00227.html
Ah, I see I've unknowingly repeated you! 
https://lists.gnu.org/archive/html/guix-patches/2021-08/msg01222.html
Heh, it's gratifying that someone else came to the same conclusion. 
It means I wasn't totally in the weeds


The smallest divisible unit of a Go repository that is independently 
distributable is now a Go module. Modules are what are resolvable, 
versioned, and check-summed. As a rule that may have exceptions: Guix 
packages should neither encapsulate anything larger nor smaller than 
that. Some of the messages you're referencing are right around the time 
modules were being reified into the Go ecosystem.


I think there is an opportunity and need for Guix to try and reach a 
consensus on what our primitives and approaches are for the Go ecosystem 
and write them down in the manual. I think our current approach is 
workable, but there's obviously still some confusion and maybe debate to 
be had.


Having said all of that, I think we should focus on using our current 
approach to get everything compiling on a supported version of Go. I 
think our packages and substitutes are probably carrying CVEs that have 
been fixed upstream, and, in my opinion, we need to resolve that ASAP.


WDYT?

Re: Updates for Go

[Katherine Cox-Buday]

On 8/17/23 3:54 PM, Wilko Meyer wrote:

That being said, I'd actually be willing to put some time and effort
into Guixes Go ecosystem; even though I haven't been on Guix for that
long and would probably have to read through prior contributions to
golang.scm to get the gist on how the go-build-system and packaging all
things go work and to contribute something useful.

Is there a list of current TODOs somewhere? Or would one start by
bumping packages to build with a more recent/non-EoL go version and see
if that works out?


Thank you for volunteering!

I'm not aware of a TODO list anywhere other than the issue tracker 
(https://issues.guix.gnu.org/search?query=golang+is%3Aopen).


Personally, I think the immediate emphasis should be on making bringing 
our Go ecosystem onto a supported version of Go (ideally 1.21.0). If 
there is consensus on that, then ensuring that some of our packages with 
larger dependency graphs compile would be a good place to start.


We also really need a Go branch to host a lot of this work. I can start 
looking at what's needed for that.


It would also be useful to get https://issues.guix.gnu.org/65317 (add 
go-1.21) reviewed, even if you don't have commit access. I've been 
exercising the package since I sent the patch, and I think v3 is correct 
(at least functionally), but it could use more exercising and a review 
of the scheme code.


--

Katherine

Re: Updates for Go

[Katherine Cox-Buday]

On 8/16/23 11:25 AM, Felix Lechner via Development of GNU Guix and the 
GNU System distribution. wrote:

Hi Katherine,

On Tue, Aug 15, 2023 at 12:59 PM Katherine Cox-Buday
 wrote:


There's also no one on Guix's Go team. I've created a patch to add
myself[1]


Your courage and initiative are inspiring. (Unfortunately, my skills
are lousy.) Anybody with an interest in Golang should please speak up!


Thank you, but I'm just another hacker trying to do her part to maintain 
software she loves. I am always grateful to the other Guix hackers doing 
amazing work!


I tried to emphasize in my message that even if you dislike Go, there 
are some critical packages like Docker that are essentially going 
unmaintained because they are written in Go. *This includes security 
fixes*. As it stands, I cannot recommend people use our packages which 
are compiled by Go v1.17.3 in production. I think every release of Go 
(43 of them!) since that one has contained a security patch.


Even if you dislike Go, but can work your way through a package, please 
consider signing up!


--
Katherine

Updates for Go

[Katherine Cox-Buday]

Hey all,

Our Go ecosystem is currently in need of a lot of love.

* The Go Team

There is currently no branch for Go updates. I know Leo had tried to get 
one setup at one point[0] but ran into issues. I'm unclear if they were 
ever resolved, but the branch isn't there, and we need one.


There's also no one on Guix's Go team. I've created a patch to add 
myself[1], although my availability has historically been very 
intermittent. Please consider joining since the Go ecosystem contains 
some pretty critical packages (e.g. Docker, syncthing, etc.).


* Go Versions

Go v1.21.0 has just been released. Go's release policy[2] states that 
releases are supported until two major releases have been made. We're 
building everything in Go 1.17.3 by default right now which puts our Go 
ecosystem at risk since our packages won't be using security fixes that 
have since been released.


To help address this, I've proposed[3] a Go v1.21.0 package, but not 
made it the default.


Next, we should get started on compiling everything with a supported 
version of Go. It would be nice to switch the default to Go v1.21.0 
since it has some nice performance improvements. To do this, we'll need 
the feature branch so we can begin working on what's broken.


What do you all think? If it's just me trying to fix broken packages, 
we're going to be on Go v1.17 for a long time!


[0] https://lists.gnu.org/archive/html/guix-devel/2023-01/msg00097.html
[1] https://issues.guix.gnu.org/65314
[2] https://go.dev/doc/devel/release#policy
[3] https://issues.guix.gnu.org/65317

--
Katherine

Re: Branch (and team?) for mesa updates

[Katherine Cox-Buday]

On 7/2/23 4:30 AM, Andreas Enge wrote:


So I would suggest to delete every branch right after merging, and then
branch off of master later when a new patch is going to be applied.
This will also create a cleaner history by avoiding a merge.


I disagree with this because it seems like Mesa moves along at a pretty 
brisk pace and I feel like we'd be constantly recreating the same branch:


23.1.3, 2023-06-22 (14 days)
23.1.2, 2023-06-08 ( 9 days)
23.0.4, 2023-05-30 ( 5 days)
23.1.1, 2023-05-25 (15 days)
23.1.0, 2023-05-10

And so on.

>> Am Mon, Jun 19, 2023 at 06:25:04PM + schrieb John Kehayias:
>> Master can be merged into this branch just prior to a patches going 
to this branch with the expectation merging back to master will be soon 
after and changes are only affecting packages that won't be touched on 
master anyway. I think this should be relatively clean and 
straightforward, a good use of our new branching/building strategy.


This seems more reasonable to me and like it's standard Git practice.

Re: Welcome to Simon as a new committer

[Katherine Cox-Buday]

On 5/11/23 8:44 AM, Maxim Cournoyer wrote:

Hello,

I'd like to welcome Simon (aka zimoun) as a new committer.  They have
made many contributions along the years, both in code and in community
management.  I'm sure they'll make good use of their new
responsibility/privilege.

Congrats, Simon!


Congratulations, Simon!

Re: gcc-toolchain is missing libstdc++.so

[Katherine Cox-Buday]

On 5/4/23 11:33 AM, Kaelyn wrote:


Regarding solutions, I would prefer to have libstdc++ in it's own package or 
output rather than bundled into gcc-toolchain:out; it feels messy and against 
the grain of isolating programs in containers if I have to make the gcc and g++ 
compilers available in the container in order to run a program that needs 
libstdc++.


+1. I recently ran into this as well and went looking for it.

I think a good reason to give libstdc++ its own output is that this 
question continually gets asked.


--
Katherine

Re: Core-updates merge

[Katherine Cox-Buday]

On 4/25/23 8:40 AM, Felix Lechner via Development of GNU Guix and the 
GNU System distribution. wrote:

Hi Andreas,

On Tue, Apr 25, 2023 at 7:09 AM Andreas Enge  wrote:


many thanks to
everyone who contributed to the branch, be it by commits, discussions or
by working on the infrastructure.


I'd like to please also acknowledge your pivotal role in shepherding
that monumental merge.

With grace and patience you took on an unrewarding legacy task that
had hardly any potential for glory. For your dedication in advancing
the codebase, you earned my sincere admiration and gratitude. Thank
you!


An emphatic +1.

Re: Latest news on core-updates

[Katherine Cox-Buday]

On 4/19/23 4:48 AM, Andreas Enge wrote:


just a quick update to share good news and to heap praise on people who are
not rewarded by seeing their name in a git commit.


Consider my praise added to the heap (and hopefully not GCed). As 
always, I am deeply appreciative to all of our maintainers.


Thank you, thank you, thank you!

--
Katherine

Re: [RFC] Cosmetic changes to define-configuration usage

[Katherine Cox-Buday]

On 3/24/23 6:33 AM, Bruno Victal wrote:


I'd like to propose for field specifications in define-configuration to be 
separated with a
blank line between them. Reason for this is that it makes it much easier on the 
eyes
to read, rather than being presented with a dense hunk of text to sift through.

I tend to always insert these blank lines when making changes in these parts to 
aid reading,
even if they weren't originally present and were not planned to be committed. 
I'd be happy if
I could simply keep the line separations and avoid the tedious insert-erase 
ritual.

I think I'm not alone in this opinion, consider the following snippets:


It's funny how sometimes this works out. I was just hacking on a 
service's configuration and had the same thought. I had spaced 
everything out so that it wasn't as overwhelming, and with a sigh 
grouped it all back together because that's how all existing services work.


I would definitely appreciate this change in the style standard.

--
Katherine

Re: Rust team branch (was Re: Discussion notes on releases and branches)

[Katherine Cox-Buday]

On 2/14/23 1:08 PM, Efraim Flashner wrote:


Also I don't think there's a lot of overlap between go and rust so we
can probably have both going at the same time.


Oh, yes, I didn't mean to imply that the two were related, or 
conflicted. I just wanted to point it out in case it was of any help.


Best of luck to the rust team!

--
Katherine

Re: Rust team branch (was Re: Discussion notes on releases and branches)

[Katherine Cox-Buday]

Efraim Flashner  writes:

> As a project we haven't setup anything for starting the team-based
> branches and upgrades, and the Rust Team volunteers to go first.

Leo has attempted[1] to setup a branch for Go, but we're waiting to hear from
sysadmins on why there's a build error.

[1] https://lists.gnu.org/archive/html/guix-devel/2023-01/msg00131.html

-- 
Katherine

Re: Merging core-updates?

[Katherine Cox-Buday]

Efraim Flashner  writes:

> On Sun, Feb 12, 2023 at 10:05:40AM +0100, Julien Lepiller wrote:
>> Hi Guix!
>> 
>> As discussed at Guix Days before Fosdem, we haven't merged core-updates
>> in a very long time. I'd volunteer to lead this effort, but I don't
>> know what steps I should follow. Do we have some documentation about
>> that?
>
> I think we normally have a 2 week last-chance window to get all sorts of
> last minute packages bumped and then we freeze it and try to build
> "everything".


On that note, could we possibly give Mesa one final bump? The latest
version is 22.3.5, and the version in core-updates is 22.2.4.

I also hope to be able to test, but my schedule tends to run away from
me.

-- 
Katherine

Re: Request for review of: [bug#60899] [PATCH 00/25] gnu: golang: Add gopls

[Katherine Cox-Buday]

Katherine Cox-Buday  writes:

> Katherine Cox-Buday  writes:
>
>> This is a patch series to add the gopls package.
>>
>> I haven't contributed to many projects which use the e-mail flow, so
>> hopefully I'm doing this correctly. Please feel free to make
>> suggestions if not!
>>
>> Some of the diffs are a little busier than I'd like for version bumps.
>> This is due to running `guix style` over everything.
>>
>> For all of the packages I have:
>>
>> . Run guix style
>> . Run guix lint
>> . Built 2x
>> . Checked that the change is in the correct branch
>> . Built all dependencies
>> . Built the repository
>>
>> Katherine Cox-Buday (25):
>>   gnu: go-golang-org-x-sync: Update to 0.1.0-1.8fcdb60.
>>   gnu: go-golang-org-x-mod: Update to 0.7.0.
>>   gnu: Add go-golang-org-x-exp.
>>   gnu: Add go-github-com-jba-printsrc.
>>   gnu: Add go-github-com-google-safehtml.
>>   gnu: Add go-github-com-jba-templatecheck.
>>   gnu: go-github-com-google-go-cmp-cmp: Update to 0.5.9.
>>   gnu: go-github-com-pkg-diff: Update to
>> 0.0.0-20210226163009-20ebb0f2a09e.
>>   gnu: go-github-com-rogpeppe-go-internal: Update to 1.9.0.
>>   gnu: gopkg-in-errgo-fmt-errors: Rename package to
>> go-gopkg-in-errgo-fmt-errors.
>>   gnu: go-golang-org-x-tools: Update to 0.5.0.
>>   gnu: Add xurls.
>>   gnu: Add go-mvdan-cc-xurls.
>>   gnu: Add misspell.
>>   gnu: Add go-github-com-client9-misspell.
>>   gnu: Add go-github-com-google-go-cmdtest.
>>   gnu: Add unparam.
>>   gnu: Add go-mvdan-cc-unparam.
>>   gnu: Add govulncheck.
>>   gnu: Add go-golang-org-x-vuln.
>>   gnu: go-github-com-burntsushi-toml: Update to 1.2.1.
>>   gnu: go-honnef-co-go-tools: Update to 0.3.3.
>>   gnu: Add gofumpt.
>>   gnu: Add go-mvdan-cc-gofumpt.
>>   gnu: Add gopls.
>>
>>  gnu/packages/configuration-management.scm |   2 +-
>>  gnu/packages/golang.scm   | 695 ++
>>  2 files changed, 578 insertions(+), 119 deletions(-)
>>
>>
>> base-commit: 5c921977179489caef4a9e54ada6696fc86d2f0b
>
> Hello Guix! Acknowledging that everyone are volunteers, and busy (guilty
> myself), this is a humble request for a review of this patch-series I
> made two weeks ago so that it doesn't bit-rot.
>
> I have a `gopls` home service[1] waiting to be proposed after this
> patch-series is merged.
>
> Thank you very much!
>
> [1]
> https://github.com/kat-co/guix-channels/blob/upstream-staging/upstream/home/services/gopls.scm
>
> P.S. I think I am following etiquette for review reminders (i.e. waiting
> long enough, bringing this over to guix-devel), but I was having trouble
> finding examples. If I haven't waited long enough, or have reminded in
> the wrong way, please give me feedback.

Hello, this is another humble request for review :)

It looks like QA is passing, and I think I did a pretty thorough job of
making sure things follow standards and will apply cleanly.

-- 
Katherine

Request for review of: [bug#60899] [PATCH 00/25] gnu: golang: Add gopls

[Katherine Cox-Buday]

Katherine Cox-Buday  writes:

> This is a patch series to add the gopls package.
>
> I haven't contributed to many projects which use the e-mail flow, so
> hopefully I'm doing this correctly. Please feel free to make
> suggestions if not!
>
> Some of the diffs are a little busier than I'd like for version bumps.
> This is due to running `guix style` over everything.
>
> For all of the packages I have:
>
> . Run guix style
> . Run guix lint
> . Built 2x
> . Checked that the change is in the correct branch
> . Built all dependencies
> . Built the repository
>
> Katherine Cox-Buday (25):
>   gnu: go-golang-org-x-sync: Update to 0.1.0-1.8fcdb60.
>   gnu: go-golang-org-x-mod: Update to 0.7.0.
>   gnu: Add go-golang-org-x-exp.
>   gnu: Add go-github-com-jba-printsrc.
>   gnu: Add go-github-com-google-safehtml.
>   gnu: Add go-github-com-jba-templatecheck.
>   gnu: go-github-com-google-go-cmp-cmp: Update to 0.5.9.
>   gnu: go-github-com-pkg-diff: Update to
> 0.0.0-20210226163009-20ebb0f2a09e.
>   gnu: go-github-com-rogpeppe-go-internal: Update to 1.9.0.
>   gnu: gopkg-in-errgo-fmt-errors: Rename package to
> go-gopkg-in-errgo-fmt-errors.
>   gnu: go-golang-org-x-tools: Update to 0.5.0.
>   gnu: Add xurls.
>   gnu: Add go-mvdan-cc-xurls.
>   gnu: Add misspell.
>   gnu: Add go-github-com-client9-misspell.
>   gnu: Add go-github-com-google-go-cmdtest.
>   gnu: Add unparam.
>   gnu: Add go-mvdan-cc-unparam.
>   gnu: Add govulncheck.
>   gnu: Add go-golang-org-x-vuln.
>   gnu: go-github-com-burntsushi-toml: Update to 1.2.1.
>   gnu: go-honnef-co-go-tools: Update to 0.3.3.
>   gnu: Add gofumpt.
>   gnu: Add go-mvdan-cc-gofumpt.
>   gnu: Add gopls.
>
>  gnu/packages/configuration-management.scm |   2 +-
>  gnu/packages/golang.scm   | 695 ++
>  2 files changed, 578 insertions(+), 119 deletions(-)
>
>
> base-commit: 5c921977179489caef4a9e54ada6696fc86d2f0b

Hello Guix! Acknowledging that everyone are volunteers, and busy (guilty
myself), this is a humble request for a review of this patch-series I
made two weeks ago so that it doesn't bit-rot.

I have a `gopls` home service[1] waiting to be proposed after this
patch-series is merged.

Thank you very much!

[1] 
https://github.com/kat-co/guix-channels/blob/upstream-staging/upstream/home/services/gopls.scm

P.S. I think I am following etiquette for review reminders (i.e. waiting
long enough, bringing this over to guix-devel), but I was having trouble
finding examples. If I haven't waited long enough, or have reminded in
the wrong way, please give me feedback.

-- 
Katherine

Re: Compiler bootstrapping requirements

[Katherine Cox-Buday]

Robby Zambito  writes:

> Hi,
>
> I remember seeing some details about certain compilers not being
> allowed in Guix due to having an inadequate bootstrapping process, but
> I can't seem to find it right now. I'm looking to write a package for
> the Cyclone Scheme[1] implementation. The compiler requires a copy of
> itself to build from the main sources, but there is a secondary
> repository which includes some C files that normally require a copy of
> the compiler to generate. I'm wondering: is the second repository[2]
> sufficient for including in Guix?
>
> Footnotes:
> [1]  https://justinethier.github.io/cyclone/
> [2]  https://github.com/justinethier/cyclone-bootstrap

Without having looked at the specifics:

Generally what we do is find an early version of the compiler that was
bootstrapped not using itself, package that, and then use that package
to bootstrap the later versions.

Sometimes that earlier version is never published as the author(s) was
quickly iterating. In that scenario, I don't know what we do.

I hope that helps.

-- 
Katherine

Re: Packaging Grafana

[Katherine Cox-Buday]

Katherine Cox-Buday  writes:

Sorry, I had a bug in my format one-liner which gave versions parenthesis. This 
should be copy/pastable:

--8<---cut here---start->8---
scheme@(guix import go)> (for-each (lambda (p) (format #t "guix import go -r 
~a@~a >> grafana.scm~%" (car p) (second p))) $11)
guix import go -r google.golang.org/genproto@v0.0.0-20220421151946-72621c1f0bd3 
>> grafana.scm
guix import go -r google.golang.org/grpc@v1.45.0 >> grafana.scm
guix import go -r 
github.com/grafana/prometheus-alertmanager@v0.24.1-0.20221012142027-823cd9150293
 >> grafana.scm
guix import go -r github.com/grafana/xorm@v0.8.3-0.20220614223926-2fcda7565af6 
>> grafana.scm
guix import go -r github.com/hashicorp/go-hclog@v0.16.1 >> grafana.scm
guix import go -r github.com/grafana/saml@v0.4.9-0.20220727151557-61cd9c9353fc 
>> grafana.scm
guix import go -r github.com/moby/moby@v0.7.3-0.20190826074503-38ab9da00309 >> 
grafana.scm
guix import go -r github.com/gomodule/redigo@v1.8.9 >> grafana.scm
guix import go -r github.com/russellhaering/goxmldsig@v1.1.1 >> grafana.scm
guix import go -r 
github.com/grafana/go-mssqldb@v0.0.0-20210326084033-d0ce3c521036 >> grafana.scm
guix import go -r gopkg.in/warnings.v0@v0.1.2 >> grafana.scm
guix import go -r golang.org/x/mod@v0.7.0 >> grafana.scm
guix import go -r go.opentelemetry.io/proto/otlp@v0.16.0 >> grafana.scm
guix import go -r go.opentelemetry.io/otel/exporters/otlp/internal/retry@v1.7.0 
>> grafana.scm
guix import go -r github.com/yudai/pp@v2.0.1+incompatible >> grafana.scm
guix import go -r github.com/xlab/treeprint@v1.1.0 >> grafana.scm
guix import go -r github.com/xanzy/ssh-agent@v0.3.0 >> grafana.scm
guix import go -r github.com/wk8/go-ordered-map@v1.0.0 >> grafana.scm
guix import go -r github.com/valyala/fasttemplate@v1.2.1 >> grafana.scm
guix import go -r github.com/pierrec/lz4/v4@v4.1.12 >> grafana.scm
guix import go -r github.com/mschoch/smat@v0.2.0 >> grafana.scm
guix import go -r github.com/mitchellh/go-wordwrap@v1.0.1 >> grafana.scm
guix import go -r github.com/mitchellh/go-homedir@v1.1.0 >> grafana.scm
guix import go -r github.com/labstack/gommon@v0.3.1 >> grafana.scm
guix import go -r github.com/labstack/echo/v4@v4.9.0 >> grafana.scm
guix import go -r github.com/kylelemons/godebug@v1.1.0 >> grafana.scm
guix import go -r github.com/klauspost/compress@v1.15.5 >> grafana.scm
guix import go -r 
github.com/kevinburke/ssh_config@v0.0.0-20201106050909-4977a11b4351 >> 
grafana.scm
guix import go -r 
github.com/jbenet/go-context@v0.0.0-20150711004518-d14ea06fba99 >> grafana.scm
guix import go -r github.com/imdario/mergo@v0.3.12 >> grafana.scm
guix import go -r github.com/grpc-ecosystem/grpc-gateway/v2@v2.10.3 >> 
grafana.scm
guix import go -r github.com/google/go-github@v17.0.0+incompatible >> 
grafana.scm
guix import go -r github.com/golang-jwt/jwt@v3.2.2+incompatible >> grafana.scm
guix import go -r github.com/go-logr/stdr@v1.2.2 >> grafana.scm
guix import go -r github.com/go-logr/logr@v1.2.3 >> grafana.scm
guix import go -r github.com/go-git/go-billy/v5@v5.3.1 >> grafana.scm
guix import go -r github.com/go-git/gcfg@v1.5.0 >> grafana.scm
guix import go -r github.com/ghodss/yaml@v1.0.1-0.20190212211648-25d852aebe32 
>> grafana.scm
guix import go -r github.com/emirpasic/gods@v1.12.0 >> grafana.scm
guix import go -r github.com/elazarl/goproxy@v0.0.0-20220115173737-adb46da277ac 
>> grafana.scm
guix import go -r 
github.com/dgryski/go-metro@v0.0.0-20211217172704-adc40b04c140 >> grafana.scm
guix import go -r github.com/coreos/go-semver@v0.3.0 >> grafana.scm
guix import go -r 
github.com/chromedp/cdproto@v0.0.0-20220208224320-6efb837e6bc2 >> grafana.scm
guix import go -r github.com/caio/go-tdigest@v3.1.0+incompatible >> grafana.scm
guix import go -r github.com/blugelabs/ice@v1.0.0 >> grafana.scm
guix import go -r github.com/blevesearch/vellum@v1.0.7 >> grafana.scm
guix import go -r github.com/blevesearch/snowballstem@v0.9.0 >> grafana.scm
guix import go -r github.com/blevesearch/segment@v0.9.0 >> grafana.scm
guix import go -r github.com/blevesearch/mmap-go@v1.0.4 >> grafana.scm
guix import go -r github.com/blevesearch/go-porterstemmer@v1.0.3 >> grafana.scm
guix import go -r github.com/bits-and-blooms/bitset@v1.2.0 >> grafana.scm
guix import go -r 
github.com/axiomhq/hyperloglog@v0.0.0-20191112132149-a4c4c47bc57f >> grafana.scm
guix import go -r github.com/acomagu/bufpipe@v1.0.3 >> grafana.scm
guix import go -r github.com/RoaringBitmap/roaring@v0.9.4 >> grafana.scm
guix import go -r 
github.com/ProtonMail/go-crypto@v0.0.0-20210428141323-04723f9f07d7 >> 
grafana.scm
guix import go -r github.com/Micr

Re: Getting tree-sitter grammars in Guix

[Katherine Cox-Buday]

Demis Balbach  writes:

> Hello, I was wondering what the "correct" way would be to get grammars
> for the tree-sitter library available in Guix when using tree-sitter
> with Emacs 29+.
>
> There is https://github.com/emacs-tree-sitter/tree-sitter-langs, but I
> don't think it has been packaged in Guix yet. I started packaging the
> grammar for JS/TS:
>
> (define-public tree-sitter-javascript
>   (let ((commit "7a29d06274b7cf87d643212a433d970b73969016")
> (revision "0")
> (version "0.20.0"))
> (package
>  (name "tree-sitter-javascript")
>  (version (git-version version revision commit))
>  (source
>   (origin
>(method git-fetch)
>(uri (git-reference
>  (url "https://github.com/tree-sitter/tree-sitter-javascript";)
>  (commit commit)))
>(sha256
> (base32 "1pk6d9g6a7bzhxmwnvfiycarcgz76wq2rgfqr0xjh7y7swfw5hvw"))
>(file-name (git-file-name name version
>  (build-system gnu-build-system)
>  (native-inputs
>   (list gcc))
>  (arguments
>   `(#:tests?
> #f
> #:phases
> (modify-phases
>  %standard-phases
>  (delete 'configure)
>  (delete 'install)
>  (delete 'build)
>  (add-after 'unpack 'create-dirs
> (lambda _ (mkdir "lib")))
>  (add-after 'create-dirs 'compile
> (lambda* (#:key inputs outputs #:allow-other-keys)
>   (let* ((out (assoc-ref outputs "out"))
>  (gcc (string-append (assoc-ref inputs "gcc") 
> "/bin")))
> (copy-file "grammar.js" "src/grammar.js")
> (with-directory-excursion
>  (string-append "./src")
>  (invoke (string-append gcc "/gcc") "-fPIC" "-c" 
> "-I." "parser.c")
>  (invoke (string-append gcc "/gcc") "-fPIC" "-c" 
> "-I." "scanner.c")
>  (invoke (string-append gcc "/gcc")
>  "-fPIC" "-shared" "parser.o" "scanner.o" "-o"
>  "libtree-sitter-javascript.so")
>  (add-after 'compile 'symlink
> (lambda* (#:key inputs outputs #:allow-other-keys)
>   (let* ((out (assoc-ref outputs "out"))
>  (lib (string-append out "/lib")))
> (install-file "src/libtree-sitter-javascript.so" 
> lib)))
>   (home-page "https://github.com/tree-sitter/tree-sitter-javascript";)
>  (synopsis "JavaScript and JSX grammar for tree-sitter.")
>  (description "JavaScript and JSX grammar for tree-sitter.")
>  (license license:expat
>
> But I'm not sure if packaging each grammar is the correct way. I
> hesitate to continue working on this because there are not other
> grammars packaged in Guix (yet). With tree-sitter being built into
> Emacs 29+ I expected grammars to be packaged already (hence why I'm
> asking if this approach is correct).


This seems correct to me, but you might get more informed opinions on
guix-devel@gnu.org (CCed), since this is a packaging question.

>
> Thanks in advance!

-- 
Katherine

Re: Packaging Grafana

[Katherine Cox-Buday]

phodina via  writes:

> Hi Guix,

Hey Petr, thanks for having a go at packaging this! It would be awesome
to have Grafana in Guix!

I was able to troubleshoot this a little. I agree with the sentiment[1][2]
that the Guix importer should just use Go tooling; however, I don't
think it's at fault here.

By default, the importer uses https://proxy.golang.org to fetch a
module's go.mod file. Let's see what that says:

--8<---cut here---start->8---
$ curl 
https://proxy.golang.org/github.com/grafana/grafana/@v/v5.4.5+incompatible.mod
module github.com/grafana/grafana
--8<---cut here---end--->8---

Weird.

Also, pkg.go.dev versions seem to disagree with both Git tags and
Grafana's stated latest version, v9.3.2:

--8<---cut here---start->8---
$ git ls-remote --tags g...@github.com:grafana/grafana.git |awk '{print $2}' 
|sort |tail -4
refs/tags/v9.3.2
refs/tags/v9.3.2^{}
refs/tags/vtest-new-release-pipeline
refs/tags/vtest-new-release-pipeline^{}
--8<---cut here---end--->8---

Then there's this commit[3], stating:

**Warning:** Do not use `go get` to download Grafana. Recent
  versions of Go have added behavior which isn't compatible with
  the way the Grafana repository is structured.

OK, so I think even Go's toolchain is having issues with this.

If I start a REPL, we can test out theory:

--8<---cut here---start->8---
$ guix repl
scheme@(guile-user)> ,m (guix import go)
scheme@(guix import go)> (http-fetch* 
"https://raw.githubusercontent.com/grafana/grafana/v9.3.2/go.mod";)
$9 = [removed lengthy go.mod output]
scheme@(guix import go)> (parse-go.mod $9)
$10 = [removed lengthy parsed representation]
scheme@(guix import go)> (go.mod-requirements $10)
$11 = (("google.golang.org/genproto" "v0.0.0-20220421151946-72621c1f0bd3") 
("google.golang.org/grpc" "v1.45.0") [etc..])
scheme@(guix import go)> (length $11)
$12 = 319
--8<---cut here---end--->8---

That looks plausibly correct.

So to work around Grafana's weirdness, we can just manually import all
of its requirements, recursively. I think you should be able to
cut-paste this and, if there are no more weird packages, you'll be off
to the races :)

--8<---cut here---start->8---
scheme@(guix import go)> (for-each (lambda (p) (format #t "guix import go -r 
~a@~a >> grafana.scm~%" (car p) (cdr p))) $11)
guix import go -r 
google.golang.org/genproto@(v0.0.0-20220421151946-72621c1f0bd3) >> grafana.scm
guix import go -r google.golang.org/grpc@(v1.45.0) >> grafana.scm
guix import go -r 
github.com/grafana/prometheus-alertmanager@(v0.24.1-0.20221012142027-823cd9150293)
 >> grafana.scm
guix import go -r 
github.com/grafana/xorm@(v0.8.3-0.20220614223926-2fcda7565af6) >> grafana.scm
guix import go -r github.com/hashicorp/go-hclog@(v0.16.1) >> grafana.scm
guix import go -r 
github.com/grafana/saml@(v0.4.9-0.20220727151557-61cd9c9353fc) >> grafana.scm
guix import go -r github.com/moby/moby@(v0.7.3-0.20190826074503-38ab9da00309) 
>> grafana.scm
guix import go -r github.com/gomodule/redigo@(v1.8.9) >> grafana.scm
guix import go -r github.com/russellhaering/goxmldsig@(v1.1.1) >> grafana.scm
guix import go -r 
github.com/grafana/go-mssqldb@(v0.0.0-20210326084033-d0ce3c521036) >> 
grafana.scm
guix import go -r gopkg.in/warnings.v0@(v0.1.2) >> grafana.scm
guix import go -r golang.org/x/mod@(v0.7.0) >> grafana.scm
guix import go -r go.opentelemetry.io/proto/otlp@(v0.16.0) >> grafana.scm
guix import go -r 
go.opentelemetry.io/otel/exporters/otlp/internal/retry@(v1.7.0) >> grafana.scm
guix import go -r github.com/yudai/pp@(v2.0.1+incompatible) >> grafana.scm
guix import go -r github.com/xlab/treeprint@(v1.1.0) >> grafana.scm
guix import go -r github.com/xanzy/ssh-agent@(v0.3.0) >> grafana.scm
guix import go -r github.com/wk8/go-ordered-map@(v1.0.0) >> grafana.scm
guix import go -r github.com/valyala/fasttemplate@(v1.2.1) >> grafana.scm
guix import go -r github.com/pierrec/lz4/v4@(v4.1.12) >> grafana.scm
guix import go -r github.com/mschoch/smat@(v0.2.0) >> grafana.scm
guix import go -r github.com/mitchellh/go-wordwrap@(v1.0.1) >> grafana.scm
guix import go -r github.com/mitchellh/go-homedir@(v1.1.0) >> grafana.scm
guix import go -r github.com/labstack/gommon@(v0.3.1) >> grafana.scm
guix import go -r github.com/labstack/echo/v4@(v4.9.0) >> grafana.scm
guix import go -r github.com/kylelemons/godebug@(v1.1.0) >> grafana.scm
guix import go -r github.com/klauspost/compress@(v1.15.5) >> grafana.scm
guix import go -r 
github.com/kevinburke/ssh_config@(v0.0.0-20201106050909-4977a11b4351) >> 
grafana.scm
guix import go -r 
github.com/jbenet/go-context@(v0.0.0-20150711004518-d14ea06fba99) >> grafana.scm
guix import go -r github.com/imdario/mergo@(v0.3.12) >> grafana.scm
guix import go -r github.com/grpc-ecosystem/grpc-gateway

Re: Packages grow, no longer fit on a 💾

[Katherine Cox-Buday]

Ludovic Courtès  writes:

> Even from a purely practical perspective, it’s not convenient when
> ‘guix pack’ creates big images that you have to carry around and your
> colleague laughs at you because their Alpine image of “the same thing”
> (quotes!) is half the size. ;-)

I know this pain!

-- 
Katherine

Re: Exception: srfi-35 vs (ice-9 exceptions (was Re: [bug#60802] [PATCH v2 1/2] platforms: Raise an exception when no suitable platform is found.)

[Katherine Cox-Buday]

Ludovic Courtès  writes:

> Hi,
>
> Josselin Poiret  skribis:
>
>> I also don't see how Guix would gain anything from moving to Guile's
>> native exceptions. However, one thing that would be nice to have is a
>> description of all such "implementation choices" in Guix, perhaps in
>> the "Coding style" section of the manual. Having a definitive
>> reference to what is the current accepted coding style would probably
>> help newcomers, more than the actual choice of one specifi
>> implementation over another.
>
> Agreed. Let’s add exceptions under “Coding Style”, and if anything
> else comes to mind, we can add it too!

And a lint check too? :)

-- 
Katherine

Re: Packages grow, no longer fit on a 💾

[Katherine Cox-Buday]

John Kehayias  writes:

>>> Efraim Flashner  skribis:
>>>
>>> I've made some progress on LLVM and I think I have a working LLVM
>>> that can be used as an input for mesa.

This is awesome! Thank you Efraim!

> I don't think there were any errors in building mesa with older LLVM,
> but on current hardware (unfortunately brings in non-free
> considerations) this was necessary. I believe this is the summary:
> 

I can confirm that 3D acceleration would not work until LLVM-15 was
used. I don't understand the entire rendering pipeline anymore, but I
think it had something to do with Vulkan.

> Props to katco on IRC for going through some long building and
> debugging to track this down.

Thanks, John!

Part of the reason this was difficult to test (aside from having to
discover the LLVM part) was discovering that the XORG_DRI_DRIVER_PATH
environment variable wasn't being set correctly despite doing package
rewriting to use a different version of Mesa.

I ended up doing this:

--8<---cut here---start->8---
(define-public (xorg-with-mesa xorg-server mesa)
  "Returns an xorg-server package which will wrap the call to Xorg to include 
the
correct XORG_DRI_DRIVER_PATH for the mesa package provided."
  (package
   (inherit xorg-server)
   (arguments
(substitute-keyword-arguments
 (package-arguments xorg-server)
 ((#:phases child-phases '%standard-phases)
  #~(modify-phases #$child-phases
   (add-after 'install 'wrap
  (lambda* (#:key outputs #:allow-other-keys)
(let* ((out (assoc-ref outputs "out"))
   (bin (string-append out "/bin"))
   (xorg (string-append bin "/Xorg")))
  (wrap-program xorg
'("XORG_DRI_DRIVER_PATH" 
":" = (#$(file-append mesa "/lib/dri")
--8<---cut here---end--->8---

But something like this (I'm sure I've gotten the gexps wrong) would have been 
made it much easier:

--8<---cut here---start->8---
diff --git a/gnu/services/xorg.scm b/gnu/services/xorg.scm
index 5f073d05d3..d626e87073 100644
--- a/gnu/services/xorg.scm
+++ b/gnu/services/xorg.scm
@@ -196,7 +196,9 @@ (define-record-type* 
   (server   xorg-configuration-server ;file-like
 (default xorg-server))
   (server-arguments xorg-configuration-server-arguments ;list of strings
-(default %default-xorg-server-arguments)))
+(default %default-xorg-server-arguments))
+  (mesa xorg-configuration-mesa ; package
+(default mesa)))
 
 (define (xorg-configuration->file config)
   "Compute an Xorg configuration file corresponding to CONFIG, an
@@ -362,7 +364,7 @@ (define* (xorg-wrapper #:optional (config 
(xorg-configuration)))
   (define exp
 ;; Write a small wrapper around the X server.
 #~(begin
-(setenv "XORG_DRI_DRIVER_PATH" (string-append #$mesa "/lib/dri"))
+(setenv "XORG_DRI_DRIVER_PATH" (string-append 
#$(xorg-configuration-mesa config) "/lib/dri"))
 (setenv "XKB_BINDIR" (string-append #$xkbcomp "/bin"))
 
 (let ((X (string-append #$(xorg-configuration-server config) 
"/bin/X")))
--8<---cut here---end--->8---

--
Katherine

Re: Ability to specify compiler package for Go build system.

[Katherine Cox-Buday]

Hilton Chain  writes:

> Hi Guix,
>
> I'm trying to package wakatime-cli[1], while Guix uses Go 1.17 as the
> default, the package requires Go 1.19 at the current version
> (v1.60.4).
>
> Though it's possible to package the last version supports Go 1.17
> (v1.38.0), I wonder if we can adjust the build system so that a Go
> package could be specified via an argument in the package definition.

As a temporary work-around, you can do this in a channel:

https://github.com/kat-co/guix-channels/blob/5c17bdd6c4ae801effdf1069df69ec4e2ce0a0dd/upstream/packages/golang.scm#L58-L59

https://github.com/kat-co/guix-channels/blob/5c17bdd6c4ae801effdf1069df69ec4e2ce0a0dd/upstream/packages/golang.scm#L818

Or maybe Ekaitz's suggestion might work.

At any rate, work is underway to bring Guix's default Go up to 1.19:
https://lists.gnu.org/archive/html/guix-devel/2023-01/msg00097.html

-- 
Katherine

Re: Golang go-updates feature branch?

[Katherine Cox-Buday]

Leo Famulari  writes:

> Now that our build farm is running smoothly, I propose we revive the
> practice of feature branches, when appropriate.

It was on my TODO list to bring this up on the mailing list! Thank you, Leo!

I also think this is a fantastic idea, and like John, I think mesa is another 
good candidate (and not only because I recently would have benefited from this).

> The first one that I propose is a go-updates branch, where we update
> the Go compilers. This will affect ~500 packages.
>
> If I understand correctly, Go is a relatively self-contained reverse
> dependency graph within Guix and thus a good candidate for this
> approach. It contains the Go libaries, and then some end-user
> applications, and they don't really affect other packages. So, it
> should be easy to understand when the update is ready to push.
>
> I can set up a jobset on ci.guix.gnu.org if people like the idea.

This was also on my TODO list for this week, and I would love to
coordinate! I'm still pretty unfamiliar with the email based workflow
and managing our CI, so this could be a great mentoring opportunity!

The things I wanted to do:

* Dust off 55210 to support a `go-next` package.

There are two uses for compilers: (1) Building Guix packages (2)
Installing in profiles for use. For (2), in addition to feature
branches, it would be nice to be able to quickly release the latest Go
package without having to immediately worry about testing all of Guix.
The idea with 55210 was to try and do this. I think this is the first
thing we should do, and we should land Go 1.19 as the first `go-next`.

We should also try and 

* Bring the default Go in Guix up to Go 1.9

* Update/create some packages

I have these in my channel I use for staging upstream changes. You can
see them from this line down:

https://github.com/kat-co/guix-channels/blob/5c17bdd6c4ae801effdf1069df69ec4e2ce0a0dd/upstream/packages/golang.scm#L429

- go-golang-org-x-vuln
- go-mvdan-cc-unparam
- go-github-com-google-go-cmdtest
- go-github-com-client9-misspell
- go-mvdan-cc-xurls-v2
- go-mvdan-cc-gofumpt
- go-github-com-google-go-cmp-cmp
- go-github-com-jba-templatecheck
- go-github-com-google-safehtml
- go-github-com-jba-printsrc
- go-golang-org-x-exp
- go-golang-org-x-mod
- go-golang-org-x-sync

There are more packages there I need to upstream, but my goal was to get
the following bullet-points into master:

* Create a package for `gopls` (Go's Language Server Protocol daemon)

I've already done the work here:

https://github.com/kat-co/guix-channels/blob/5c17bdd6c4ae801effdf1069df69ec4e2ce0a0dd/upstream/packages/golang.scm#L817-L860

* Create a home service for running gopls

I've already done the work here:

https://github.com/kat-co/guix-channels/blob/upstream-staging/upstream/home/services/gopls.scm

Go 1.20 is supposed to be released in February, so we'd have a chance to 
exercise our new Go feature-branch for 1.19 and then quickly 1.20.

I'll reach out on IRC to coordinate!

-- 
Katherine

Re: Potential minor API change notice

[Katherine Cox-Buday]

Maxim Cournoyer  writes:

> This is a heads up to let you know that the
> %BASE-PACKAGES-DISK-UTILITIES public variable exported from (gnu
> system) may be removed in the near future.
>
> For the rationale and more details, see the proposed change [0].
>
> [0]  https://issues.guix.gnu.org/59661#1

Maxim, I just wanted to say thank you for taking the feedback from the
last breaking change, and not only graciously listening, but
implementing it so quickly. You are a true leader!

Sincerely,
-- 
Katherine

Re: Feature Request: Sort inputs in style

[Katherine Cox-Buday]

jgart  writes:

> Could we support sorting inputs with guix style?

And for that matter, auto-add and auto-remove unusued imports. I'm so
bad at this kind of manual book-keeping.

But maybe that's more of a geiser/scheme-lsp feature request?

-- 
Katherine

Re: Layout of ‘define-configuration’ records

[Katherine Cox-Buday]

Maxim Cournoyer  writes:

> Apologies for causing grief!

No worries at all, Maxim! The good you do far outweighs any grief, and
even if that weren't the case, we're only human :)

> I'm taking yours and Ludovic's feedback into account for the future
> and will reach out to guix-devel with heads-up when introducing
> breaking changes to Guix APIs.

This seems like the logical place to me, but it's also a bit busy for
announcements! I'm sure I'm not alone, but sometimes I'm very busy and
distracted and I need a very loud signal to let me know I need to take
action :) That's why I suggested info-guix. Its description reads:

#+begin_quote
Subscribe to the info-guix low-traffic mailing list to receive important
announcements sent by the project maintainers (in English).
#+end_quote

Would it make sense to publish there? I don't think we break APIs very often?

Also, do we need any kind of formalization around lead-time for these
announcements?

> A side-note, it seems that Ludovic has been working toward eliminating
> the use of match patterns matching the fields directly, instead
> encouraging the use of 'match-record', see
> https://issues.guix.gnu.org/59390. I haven't checked if this is
> compatible with define-configuration records though.

Thanks, I'll have a look!

-- 
Katherine

Re: Layout of ‘define-configuration’ records

[Katherine Cox-Buday]

Maxim Cournoyer  writes:

>> Moving the field last is problematic as we’ve seen for any user that
>> uses ‘match’ on records—something that’s not recommended but still
>> used a lot.

Some feedback I hope is useful:

I'm one such newbie, and had to stumble my way into discovering why some
of my services suddenly broke. All I had to go on was with "invalid
G-expression input" at a location in an `operating-system` declaration.

Initially, because of the reference to gexps, I thought something had
changed with `local-file` which I am using to deploy source code off of
a local branch. Through trial and error and reading git logs, I
discovered this was not the case, and saw that `define-configuration`
had changed. From there I was able to discover the problem.

> Yep. I had that on mind when I made the change, though I still missed
> a few occurrences.

I came looking for an announcement of this change somewhere on a mailing
list (info-guix maybe?) or the Guix blog but couldn't find anything. I
understand the focus is on in-repo code, but could we consider
announcing changes which might affect channels and such? I don't always
have time to stay up to date with the changes to the project :(

-- 
Katherine

Re: Is Guix suitable for large monorepos?

[Katherine Cox-Buday]

jgart  writes:

> Is Guix suitable for large monorepos?

I use Guix against a large mono-repo, and there are a few
difficulties.

I should mention that I am not using Guix in the typical fashion with
fixed commits (packages are pointed at HEAD and I don't use checksums).
For my purposes, it's useful for me to build HEAD, although sometimes I
use transformation options if I'm interested in a specific branch or
commit.

I've defined a package for the entire repo and then individual packages
for the constituent projects which inherit from the repo package.

For new commits:

- Updating the source take a long time.
- Indexing the entire repo takes a long time.
- Copying over the entire tree to the build directory is unnecessary and
  takes time.

Sometimes I want to build a local change I haven't committed anywhere,
so I use the --with-source option. The problems are the same, but
they're exacerbated.

A happy medium I've landed on is to build some binaries outside of Guix
and then use packages that are defined in terms of the binaries. These
packages use patchelf to get everything linked correctly. This allows me
to reify these packages into Guix while keeping the standard workflow
from a programming language.

I've also considered a phase that, after checkout, deletes all files not
relevant to the project to lessen the burden of copying things over for
building.

So, in my opinion, these issues are not really Guix's fault, and it does
a pretty good job all things considered. I think Guix used in a CI/CD
pipeline (which I guess means cuirass) would handle mono-repositories
wonderfully and do exactly what I want: reproducible, verifiable,
builds. Guix as a development tool in these situations is still useful,
but has some things that make it a little difficult to work with.

Maybe someone has some ideas to improve things?

-- 
Katherine

Re: Problem with emacs-libgit

[Katherine Cox-Buday]

Michael Rohleder writes:

> This looks like https://issues.guix.gnu.org/55427

I tried to update emacs yesterday and ran into this issue, and a few
other issues with emacs packages as well. I was surprised that we merged
an update to emacs when some packages (well used ones at that) were
failing to build.

I realize emacs has a large tree of packages depending on it, but I've
always been taught to check to ensure everything remains building. This
makes sense to me because I thought one of the reasons a distribution
exists is to handle the integration of everything for its users. This
seems like it breaks that ethos.

Can anyone point me to some documentation or explain how these decisions
are made? All I could find was this:

  For important changes, check that dependent packages (if applicable)
  are not affected by the change
  https://guix.gnu.org/en/manual/devel/en/guix.html#Submitting-Patches

Thank you to all the contributors.

-- 
Katherine

Re: Hardened toolchain

[Katherine Cox-Buday]

zimoun  writes:

> On Tue, 29 Mar 2022 at 12:15, Ludovic Courtès  wrote:
>
>> Stack smashing protection (SSP) may incur measurable run-time
>> overhead though so enabling that one by default may be less
>> consensual.
>
> That’s true and it could be an issue for HPC practitioners.  However,
> quoting Wikipedia [1], for what it is worth:
>
> All Fedora packages are compiled with -fstack-protector since Fedora
> Core 5, and -fstack-protector-strong since Fedora 20.[19][20] Most
> packages in Ubuntu are compiled with -fstack-protector since 6.10.[21]
> Every Arch Linux package is compiled with -fstack-protector since
> 2011.[22] All Arch Linux packages built since 4 May 2014 use
> -fstack-protector-strong.[23] Stack protection is only used for some
> packages in Debian,[24] and only for the FreeBSD base system since
> 8.0.[25] Stack protection is standard in certain operating systems,
> including OpenBSD,[26] Hardened Gentoo[27] and DragonFly BSD.

For me at least, this is a compelling argument for also defaulting to more 
secure, but possibly slower, build flags. (Full disclosure: I would personally 
benefit from the security over performance model of defaults).

But I think we should state our reasons plainly in the documentation, and 
provide an easy way for those who need performance to "recompile the world".

-- 
Katherine

Re: Hardened toolchain

[Katherine Cox-Buday]

Aurora  writes:

> Katherine Cox-Buday  writes:
>
>> Everyone has different threat models and needs. A lot of computers
>> have CPU speculative execution attack mitigation disabled because
>> those types of attacks will never affect those computers, and it
>> reduces the performance of the CPU a lot.
>
> There are multicore processors made in the last decade or two that
> aren't affected by speculative execution vulnerabilities?

They are vulnerable to them, but not necessarily affected by them.
Consider a computer not networked to the internet and only running
trusted workloads (e.g. scientific HPC, etc.). This is why acknowledging that 
everyone has a different threat model is important.

I hope this helps to clarify.

Sincerely,
-- 
Katherine

Re: Hardened toolchain

[Katherine Cox-Buday]

raingloom  writes:

> People shouldn't have to take extra steps and burn extra CPU cycles
> for security.

To be clear, I don't have a strong opinion on this, but I wanted to give an 
alternative viewpoint: people shouldn't have to take extra steps and burn extra 
CPU cycles for performance.

Everyone has different threat models and needs. A lot of computers have CPU 
speculative execution attack mitigation disabled because those types of attacks 
will never affect those computers, and it reduces the performance of the CPU a 
lot.

I suggest we pick our default with care, and if possible with data about what 
most users would like.

-- 
Katherine

Re: go importer documentation/exceptions

[Katherine Cox-Buday]

I agree with the gist of what you're saying, but I first need to establish that 
Guix is actually not doing anything wrong before we can talk about how to make 
things better.

> On Saturday, April 23rd, 2022 at 9:51 AM, jgart  wrote:
>
> The Guix docs do not explicitly state that a uri like
> https://git.sr.ht/~emersion/chathistorysync@0.1.0 is not allowed:
>
> https://guix.gnu.org/manual/devel/en/html_node/Invoking-guix-import.html
>
> This leaves footguns for new users trying to use the go importer for
> the first time since a new user might want to just copy paste the go
> package's url into the terminal.

I'll explain more below, but a "package's URL" is a misnomer.

> Should we be explicit in the docs about what uri syntax is and is not
> allowed

Here is the output from `guix import go --help`:

Usage: guix import go PACKAGE-PATH[@VERSION] Import and convert the Go
module for PACKAGE-PATH. Optionally, a version can be specified after
the arobas (@) character.

And here is the definition of a package path from
https://go.dev/ref/mod#glos-package-path:

#+begin_quote

  package path: The path that uniquely identifies a package. A package
  path is a module path joined with a subdirectory within the module.
  For example "golang.org/x/net/html" is the package path for the
  package in the module "golang.org/x/net" in the "html" subdirectory.
  Synonym of import path.
  
#+end_quote

So I disagree that Guix doesn't explicitly state that a URI like the one you've 
given is not allowed, because it doesn't say to pass in a URI, it says to pass 
in a PACKAGE-PATH, which has a very specific meaning in Go.

Package paths happen to look like URLs, but they're not. They're URIs with 
special properties that go far beyond where to fetch code. As an example, you 
could fetch the package foo.bar/baz from 
https://my-repository.com/katco/my-lib. There is an algorithm[1] for resolving 
a module path to a SCS to fetch from, and Guix re-implements[2] this.

Now, having said all that, I can see how someone familiar with Guix, but not 
Go, would make this mistake. And since we're here to help humans, but not 
pedants, and not computers, we should keep that in mind when thinking about 
designing the tool.

In my opinion, we should assume the person importing Go packages knows what a 
Go package is, and therefore knows what a package path is. If they get it wrong 
and try and use a URL, that's when we should surface the difference between the 
two, and suggest they try again without any URL syntax.

WDYT?

[1] - https://go.dev/doc/modules/managing-source#tools
[2] - https://git.savannah.gnu.org/cgit/guix.git/tree/guix/import/go.scm#n474

-- 
Katherine

Re: Updating from Go 1.17 to 1.18

[Katherine Cox-Buday]

Pier-Hugues Pellerin  writes:

> We should probably also try to mimic how their bootstrap and have the
> following:
>
>- Bootstrap 1.17 with 1.4
>- Bootstrap 1.18 with 1.4
>- Have 1.19 bootstrap with 1.17.
>
> So maybe we should do something like this.
>
> - Add go-1.18 build from 1.4.
> - Add go-next that point to go-1.18
> - Refactor 1.17 package definition that is shared between go-1.18. and
> go-1.17.
> - Move  go to go-1.18  in another branch
> - Remove 1.16?

Thanks for looking into how the Go team will be bootstrapping, and I
agree with your plan. Is there anything I can do to help?

-- 
Katherine

Re: Updating from Go 1.17 to 1.18

[Katherine Cox-Buday]

>>> Pier-Hugues Pellerin  writes:
>> Ludovic Courtès writes:

>>> I am trying to update Go to 1.18, I do have a *working* patch that defines
>>> a package that inherits from 1.17 and that adjusts the inputs.
>>
>> Nice!

Yes, thank you! I just found out I need this and came to see if anyone had
started on it.

>> You can define Go 1.18 inheriting from 1.17; that’ll allow us to have both
>> versions, and eventually we’ll remove the older one.

I suggest inverting this: copy/paste go-1.17 to go-1.18, and then make go-1.17
inherit from go-1.18. This means that when it's time to sunset a version, it's
a simple delete and not something that cascades through all recent versions.

> I think it makes sense, looking at the number of impacted packages and from
> my experience working in go, even if the contract is the same, sometimes it
> does break on minor.

I was wondering if we don't want to start publishing a go-next package like we 
do with emacs-next? That would allow us to publish the latest version of Go 
without needing to immediately address building all the packages that depend on 
it.

Kindest regards,
-- 
Katherine

Re: Building a software toolchain that works

[Katherine Cox-Buday]

I run Guix everywhere I can, and it's now the only way I develop software. 
Having said that, I have thought about this issue a little bit, and here's my 
opinion on why this happens.

Pjotr Prins  writes:

>  And they start out as the next new thing to solve all problems! If
> they would only would have used Guix to create a coherent build
> system...

Is Guix not a new thing designed to solve all problems? Why aren't we all just 
using Nix! Or Make!

The reality is that every new system is created because its creators feel that 
the available systems are deficient in some way. That opinion may not be shared 
by everyone.

In addition, because free software is largely developed in people's spare time, 
they're going to use whatever tools make them most productive or even just 
happy. They're probably not thinking about their software against the backdrop 
of the larger software ecosystem.

Finally, build tools which are bespoke to a language or use-case are usually 
going to be easier to work with because they don't have to consider the general 
or aggregate cases. So if I only live in 1 ecosystem, I'm going to learn that 
tool and use it any time I can.

Guix solves a lot of issues, and is wonderful to use, but I don't think it 
solves the most difficult issues: human issues :)

-- 
Katherine

Re: GNU Guix maintainer rotation

[Katherine Cox-Buday]

Maxim Cournoyer  writes:

> I'd like to bring your attention to a change to the current Guix
> maintainers collective; in a nutshell, Ludovic and Marius are stepping
> down from maintainer-ship while Efraim is joining.

Thank you very much for your guidance, support, and hard work, Ludovic and 
Marius. Likewise, Efraim, thank you for your contributions, and for stepping up!

-- 
Katherine

Re: Guix Documentation Meetup

[Katherine Cox-Buday]

adriano  writes:

> Il giorno dom, 12/12/2021 alle 21.50 -0600, Katherine Cox-Buday ha scritto:

>> - In geiser, run =,a thing-i-want-to-look-for= (this is supposedly an
>> apropos command that is supposed to search symbols for you). The command
>> returns nothing.

> about this, I want to report an example
>
> scheme@(guile-user)> (help tail)
> Did not find any object named `tail'
> scheme@(guile-user)> 
>
> BUT
>
> scheme@(guile-user)> (help "tail") <-- notice the quotation marks !
> Documentation found for:

> That is, help with the quotation marks reports about things even if
> they're in namespaces you haven't imported yet !

Thank you very much for the tip! I will definitely be trying this next time I'm 
hacking on Guile!

-- 
Katherine

Re: Guix "R" Us - GNU's joy store!

[Katherine Cox-Buday]

jgart  writes:

> On Wed, 29 Dec 2021 00:27:04 +0100 raingloom  wrote:
>> TLDR how much of the effort spent on this channel is really justified
>> compared to making the underserved use-cases easier in upstream Guix?

I also have my own personal "upstream staging" channel so that I can continue 
contributing to Guix at my own pace, in my own way, until I can get enough 
velocity to really internalize contributing in the way Guix would like me to. I 
also tend to hack on things there, and then batch my changes for upstream so 
that I can keep everything consistent and make sure all my code meets upstream 
standards. In the meantime, I'm able to actually use the code.

My only concern with organizing this activity into a group is that it is a kind 
of "community fork". I hope we can take the reasons people make these channels 
and bring them back to Guix proper to make contributing here just as easy.

At any rate, any energy put into the ecosystem is good in my book. Cheers! :)

--
Katherine

Re: Guix Documentation Meetup

[Katherine Cox-Buday]

Luis Felipe  writes:

> Hi,

Hello! Thank you for adding some context around why things are laid out the way 
they are.

I just wanted to respond again with gratitude for your work, and to plainly 
state that I view this work as difficult, with no clear best practice.

Sincerely,
-- 
Katherine

Re: Guix Documentation Meetup

[Katherine Cox-Buday]

Here is some analysis on various popular languages' documentation landing 
pages. I did this on Sunday, but I held it back from my previous message 
because it was already quite long, and I was worried this would be seen as 
over-critical and maybe raise the ire of various language fans.

But I think it might add to the conversation and help create better Guile 
documentation, so I'll risk it :)

For context, the only language on this list I really know well is Go.

Guile:
==
- Tutorial for extending a C program with Guile
- The manual
- Scheme Resources
  - "The Revised^6 Report on the Algorithmic Language Scheme"
  - "" but 7
  - "" but 5
  - (more links that IMO are only meaningful if you already know Scheme)

Overall, I'm left feeling like I have a clear sense of direction because it's 
apparent to me that I should click through to the reference manual. Experience 
with the GNU ecosystem helps here. I am left wondering whether I should click 
on any of the other links, and what I might be missing by not doing so. But 
clicking through reveals lengthy documents that I don't know if I should read 
yet or not.

Once I do click through to the manual, I am in a pretty familiar GNU document, 
and the other comments about this document become applicable.

The one thing I would add here is that other languages have a separate 
symbol/library explorer with a few more bells and whistles to support jumping 
around. It would be nice if Guile not only curated opinions on which modules to 
use, but used a separate, more featureful site for that.

Go (https://go.dev/doc/):
=
- A blurb trying to sell Go
- A link on how to install Go
- Links to tutorials and opinions on how to write Go.
- Links for very specific topics like editor plugins and fuzzing.

Overall, this seems cluttered and disorganized. For some reason there are 
several links to tutorials on very specific topics before there is a link to 
look at the packages in the standard library. These tutorials seem to be 
interspersed with more fundamental, important, beginner topics, and I'm not 
sure why.

The link to look at the documentation for the standard library -- something a 
developer will be working with a lot -- is titled "Package Documentation", too 
generic a term.

Once I do click through, there is a great site for navigating the standard 
library which explains what packages are for, and provides various navigation 
elements for jumping around.

Rust (https://www.rust-lang.org/learn):
===
- A link to a book
- A link to a course
- A link to rust by example
(note the support for 3 different kinds of learning styles)
- Core documentation
  - The standard library (here is, by way of being the standard library, an
opinion I can borrow for how things are done).
  - (more links I skipped because I don't know rust, and I"d start with the
above)

Overall, I am left feeling like I know where I would start depending on how I 
want to learn, and have a quick reference to the standard library's API.

Once I click through to the standard library, it suggests to me how to read it, 
and addresses the meta-question of what I'm looking at. I haven't used this 
site, but clicking around, it appears to have decent navigation elements for 
jumping around.

Python (https://www.python.org/doc/):
=
- A blurb on how the different ways to consume the documentation.
- Links grouped by self-reported expertise with the language

If I click on Beginner's guide:

- Link to explanation of what Python is
- Blurb on how to get Python
- Links for non-programmers to learn
- Links for programmers to learn
  - Acknowledgment that I've been reading wiki pages. I now have an
explanation for why what I've been reading feels cobbled together.
  - A link to a separate beginner's guide overview with more general
information about the language.
  - A long list of books, websites, and tutorials (note there is no opinion
given on which I should begin with)

Overall, I am left feeling overwhelmed, and like I must independently find a 
curated tutorial which will give me opinions I can use until I can form my own.

But wait! Had I clicked on "Python Docs" instead of "Beginner's Guide", I get 
something much easier to consume:

- A Tutorial which simply states "start here". As a newbie, OK!
  - This looks like a book
- Library reference
- Language reference
- General links to support usage

This is not overwhelming, and I feel like I have a small enough set of options 
to click through that I can find what I need. Further, it looks like a curated 
opinion of how I should do things, and in what order.

-- 
Katherine

Re: core-updates-frozen branch merged

[Katherine Cox-Buday]

Congratulations, everyone!

I love Guix, and I really appreciate all the work that everyone puts into it. 
I'm updating now :)

Sincerely,
-- 
Katherine

Re: Guix Documentation Meetup

[Katherine Cox-Buday]

Blake Shaw  writes:

> Katherine Cox-Buday  writes:
>
> Katherine, reading a big deeper into your user experience report, its really
> so so helpful to have this concrete sort of step-by-step user experience
> report.

Definitely! The first step to resolution is a common understanding.

> Would you mind if I solicit the list for more reports like this for anyone
> who might feel like offering them?

I don't think I'm in a position to give you authority to do this, but I 
certainly don't mind :)

> And would you mind if I cite this in the presentation I'm gathering?

Not at all. I posted this to the public list for that reason.

Thanks for trying to take this on!

-- 
Katherine

Re: Guix Documentation Meetup

[Katherine Cox-Buday]

Blake Shaw  writes:

> I'd love to know where you found trouble so that I can take that into
> consideration when developing a design strategy for the makeover.

When reading this, please keep in mind that I have a lot of gratitude for the 
various maintainers of this software and its respective documentation. 
Communication is very difficult, and relaying my experience and opinions is 
meant to convey that experience, and not to criticize anyone's efforts.

I think my frustration stems from the confluence of me not being familiar with 
scheme, how Guix writes scheme, how scheme the language and ecosystem work, the 
tools we have available to us, and then finally the documentation.

Not being a schemer (yet?), I can't speak with much authority, aside from the 
fact that I've written code in many languages, and most of these pain points 
are well addressed in others.

Usually, my frustrated workflow goes like this (warning: ignorance on display! 
:p):

- Identify something in Guix I want to do

- Open (or create) a file

- Start geiser

- Start writing Guile

- Come upon a semantic structure I want to write, and realize I don't know the
  Guile way to do it.
  
- In geiser, run =,a thing-i-want-to-look-for= (this is supposedly an apropos
  command that is supposed to search symbols for you). The command returns
  nothing. I realize I have to import the module implementing the thing I'm
  looking for first, thus defeating the purpose.

- Before, I would then go here[1] and try and textually search for the symbol.
  I have recently discovered the emacs function: =helm-info-guile= which makes
  searching a bit faster.

- Find that there are multiple, competing, modules which do the same thing in
  different ways.

- Realize I don't know which one to use, or which one Guix would prefer.

- Sometimes, realize that Guix has their own wrapper around one of the
  modules, and learn that something I thought was standard scheme is actually
  a Guix neologism.

- Import one and try it. Maybe import another and try it. Lose track of why
  I've imported things as I'm trying them because the module names have no
  indication of what they implement.

- Wish that Guix had a standard to prefix function calls with their module, as
  we do with =license:=.

- Have no way of automatically and programatically cleaning up imports to work
  around this.

- Forget what I was even trying to accomplish. Go to bed and try again
  tomorrow!

Specific to Guile documentation, I guess if I boil the above down, it would be 
something like:

Early in the documentation, directly address the intrinsic fragmentation of the 
scheme ecosystem, and how a new Guiler is meant to navigate that. Do so in 
terms that don't rely on experience with the Scheme ecosystem.

The Guile reference manual has this[2] section which, to me, is a paragraph 
which touches on this, and then much later has a blurb[3] on what SRFI is. I 
had no idea what =ice-9= was, and the first mention[5] of it in the manual is 
one of its modules. To my knowledge, what =ice-9= is, or why it's called that, 
is never addressed. It appears[5] I am not the first to be confused by this.

As a newcomer to Guile, and to scheme, I would expect Guile to give me its 
opinion of how to do things (i.e. which modules to prefer, what is considered 
"standard" in most schemes) until I've written enough Guile to form my own. As 
it's written, my impression is that the documentation relies on the reader 
knowing a lot about scheme coming in.

I feel there just needs to be an easier gradient for people trying to use Guile 
as their first scheme. It is not specifically Guile's problem, but as a service 
to its developers, the meta-topic should be addressed so a new developer is not 
left confused.

When we put a lot of work into something, it's really easy to fall in love with 
the idiosyncrasies of the thing, and to lose sight of what it's like to come 
into an ecosystem with no context. This can result in addressing things that 
feel important to us, the expert, but are really much further up the ladder of 
complexity.

When writing a landing page for a language, I feel the following is important:

*Don't overwhelm your newcomers.*
Your landing page should have few links, and they should directly address 
high-level questions:

- What am I looking at?
- How is what I'm looking at organized?
- What should people like me (e.g. non-programmers, programmers, schemers,
  people who don't speak English) be reading?
- A link to a site which provides easy to use symbol lookup.

The documentation can fan out in complexity from there, but the landing page 
must not be overwhelming.

Anyway, those are my opinions :)

[1] - https://www.gnu.org/software/guile/manual/guile.html
[2] - https://www.gnu.org/software/guile/manual/guile.html#Guile-Scheme
[3] - https://www.gnu.org/software/guile/manual/guile.html#SRFI-Support
[4] - https://www.gnu.org/software/guile/manual/guile.html#ice_002d9-optargs
[5] - https://l

Re: Guix Documentation Meetup

[Katherine Cox-Buday]

Ryan Prior  writes:

> I've found the Guile IRC channel to be polite and encouraging, but also very
> self-satisfied, which makes it hard to feel heard as a Guile hacker who's
> struggling.

Consider reaching out to Andy Wingo (wingo in IRC in #guile). I hope I am not 
laying this issue at his feet by suggesting this, but I have interacted with 
him a handful of times, and he is a genuinely nice and helpful person, and also 
a guile maintainer. I am pretty confident he would want to help make this 
problem better -- especially if there are those willing to do some work.

Good luck! I have also found myself hunting around for basic functionality, and 
not being a guiler, nor even a schemer (I love lisps, but I mostly write 
emacs-lisp and common-lisp), wondering why things are fragmented into weird 
modules. It's a shame, because I feel like I would be contributing bigger 
features to Guix if I could stop bouncing off the language.

-- 
Katherine

Re: Common Lisp package: all test cases pass but 'check' phase fails

[Katherine Cox-Buday]

Foo Chuan Wei  writes:

> In lisp-xyz.scm, I see that the "cl-locale" package has the same problem
> with its tests.
>
> Does anyone here know the cause of the error above?

Without having time to look at the code, but with the hope that this points you 
in the right direction:

It probably has something to do with the separate test .asd file. The 
asdf-build-system is probably copying that file to a place asdf can't find it 
from the production .asd file. You can probably briefly scan the code and get 
an idea of what's going on:

https://git.savannah.gnu.org/cgit/guix.git/tree/guix/build/asdf-build-system.scm?h=master#n1

I will do my best to find some time to answer definitively, but I am 
particularly busy at the moment. Hopefully someone else can chime in with a 
more specific answer.

-- 
Katherine

Guix services, logging, and log rotation

[Katherine Cox-Buday]

Hey Guix!

I'm slowly working on contributing rsyslog with a Guix service. I have just 
arrived at trying to allow the user to specify where the Shepherd service sends 
its log file, and I began thinking about log rotation.

In the manual, SS10.8.3, it says:

> (usually, services that produce log files already take care of that)

I found an excellent example in =hpcguix-web-service-type=. It looks like you 
can achieve this by extending the =rottlog-service-type=?

#+BEGIN_SRC scheme
  (define hpcguix-web-service-type
(service-type
 (name 'hpcguix-web)
 (description "Run the hpcguix-web server.")
 (extensions
  (list (service-extension account-service-type
   (const %hpcguix-web-accounts))
(service-extension activation-service-type
   (const %hpcguix-web-activation))
(service-extension rottlog-service-type
   (const %hpcguix-web-log-rotations))
(service-extension shepherd-root-service-type
   (compose list hpcguix-web-shepherd-service))
#+END_SRC

However, I noticed not all services allow users to specify where log files go, 
or setup log rotation. E.g.:

#+BEGIN_SRC scheme
(define syncthing-service-type
  (service-type (name 'syncthing)
(extensions (list (service-extension shepherd-root-service-type
 
syncthing-shepherd-service)))
(description
 "Run @uref{https://github.com/syncthing/syncthing, Syncthing}
decentralized continuous file system synchronization.")))
#+END_SRC

Are these bugs?

Why don't all services allow you to specify where logs go?

I guess if no log location is specified, it just goes in the master shepherd 
log which is rotated?

Thank you,
--
Katherine

Re: Recommend order for package fields?

[Katherine Cox-Buday]

zimoun  writes:

> And obviously, it could be nice to have an automatic tool for formatting; > 
> something similar as etc/indent-code.el for ordering packages. ;-)

And cleaning up unused imports too! Does such a thing exist for Guile in 
general?

-- 
Katherine

Re: Incentives for review

[Katherine Cox-Buday]

zimoun  writes:

>> I have often seen folks on various projects worried about the size of
>> various backlogs: bugs, issues, etc. I think it is human to want to
>> try and contain something that appears to be growing, unbounded. 
>
> …about patches only.  Bug is another story. :-)

Sorry, I meant to speak to both and instead repeated bugs with a different 
word, issues! I think patches and bugs are similar in this context.

> Just number to fix the idea about large backlog.

I think it's really great that you went through the trouble to quantify this. 
Thank you.

>> I think the thing that bothers us is a sense that the backlog is
>> becoming unmanageable, or too large to triage. I submit that this is
>> actually a tooling and organizational issue, and not an intrinsic
>> issue to be solved. Bugs may still be valid; patches may still have
>> valuable bones to modify.
>
> This is the point.  What do you do?  What could we improve about tooling
> and organisation to better scale and deal with this “becoming
> unmanageable backlog”?

I tried to give some ideas here[1].

> From my point of view, it is good to have this issue.  It means that
> Guix is becoming more popular.  And we – regular user, contributor,
> committer – have to adapt to this increasing workload, IMHO.

I totally agree!

> The question is how.  And how to invite people to complete review. :-)

Humans usually enjoy community. I think the group activities are really great.

>> I think the real issue is that as a backlog grows, the tools we're
>> used to using cannot answer the questions we want to ask: what is most
>> relevant to me or the project right now?
>
> If it is relevant to the project then it is also relevant to me as an
> user.  And vice-versa. ;-)

I think I did not give the proper context. I meant relevant as in "I am working 
on this package. Is anyone else? What tickets might I update? What other 
trivial bugs might I fix while I'm looking at this?"

I.e. relevant in the temporal sense.

> When something relevant to me is not making progress, it often means
> people are busy elsewhere, so I try to comment (review?) about patches
> or bugs.  It is a Sisyphean task because the workload never
> decreases. :-)  Or maybe structured procrastination. ;-)

I find it helpful to not think of it as a discrete task, but work along a 
continuum -- a joyful habit of collectively helping everyone to have something 
nice :)

"A society grows great when old (wo)men plant trees whose shade they know they 
shall never sit in."

[1] - https://lists.gnu.org/archive/html/guix-devel/2021-10/msg00158.html

-- 
Katherine

Re: Accuracy of importers?

[Katherine Cox-Buday]

Ludovic Courtès  writes:

>go   (Sarah? Leo? Raghav?)

I have only used this a few times so far, but the quality seems to have gotten 
a lot better. My impression, though, due to the nature of how we have to 
generate packages so as to not be reliant on a centralized GOPROXY server 
(namely one controlled by Google), is that we stumble dealing with the 
heterogeneity of the internet. There are a few things which could make this 
situation better:

There is an open issue[1] for a better API to https://pkg.go.dev which may 
eventually allow us to query for things like license, VCS path, etc. This could 
obviate Guix's need to crawl the internet.

I was also discussing[2] the pros/cons of relying on the Go tool-chain to do 
most of the work for us. I think doing so might be making the right trade-offs, 
but it sounds like[3] we are blocked by cgit's ability to work with shallow 
checkouts. Since Guix has a build environment, maybe we could just use Git the 
CLI instead of a scheme library when necessary.

I hope this helps, and good luck with your talk!

[1] - https://github.com/golang/go/issues/36785
[2] - https://lists.gnu.org/archive/html/guix-devel/2021-09/msg00344.html
[3] - https://lists.gnu.org/archive/html/guix-devel/2021-10/msg00020.html

-- 
Katherine

Re: Time for a request-for-comments process?

[Katherine Cox-Buday]

Ludovic Courtès  writes:

> I think a major goal of the process would be to formalize a minimum
> and a maximum duration under which an RFC is under evaluation, and a
> mechanism to determine whether it’s accepted or withdrawn.

I think it's a good idea. The only contribution I can give is that I would not 
have known =guix shell= was coming had I not been watching the patches list. So 
in addition to formalizing what you've mentioned, I suggest we also formalize a 
location to watch (guix-devel seems logical?).

-- 
Katherine

Re: Incentives for review

[Katherine Cox-Buday]

Ricardo Wurmus  writes:

> Katherine Cox-Buday  writes:
>
>>> It’s not about urgency but rather about not contributing to the growth
>>> of our patch backlog, which is a real problem.
>>
>> I have often seen folks on various projects worried about the size of
>> various backlogs: bugs, issues, etc. I think it is human to want to
>> try and contain something that appears to be growing, unbounded.
>>
>> I think the thing that bothers us is a sense that the backlog is
>> becoming unmanageable, or too large to triage. I submit that this is
>> actually a tooling and organizational issue, and not an intrinsic
>> issue to be solved. Bugs may still be valid; patches may still have
>> valuable bones to modify.
>>
>> I think the real issue is that as a backlog grows, the tools we're used to
>> using cannot answer the questions we want to ask: what is most relevant to
>> me or the project right now?
>>
>> To me, this sounds like a search and display problem.

> I would be happy if people used this opportunity to change mumi (the tool
> behind issues.guix.gnu.org) to present the backlog in more helpful ways.

I don't have time to work on this, but here are some ideas. Some of these 
capabilities are present, but maybe not discoverable or a pre-built clickable 
link while viewing a patch/issue.

- Contextual search based on a path.
  - Show me issues/patches for this file/directory
  - Show me the rate of change of this file/directory

- Contextual search based on a patch
  - Show me bugs which mention any top-level public symbols changing in this
patch, or if packages, the package name.
  - Show me patches which conflict with this one.
  
- Contextual search based on author.
  - Show me other patches by this author
  - Show me the median time-to-commit for this author's patches

- Show me patches/issues, grouped by author, sorted by median time-to-commit,
  descending.
- Show me the paths/files with the highest number of bugs reported.

A lot of this requires static analysis which may not be trivial to implement. 
Still, I think being able to say "we don't have time to build what would fix 
this" is a helpful progression from "we don't know how to manage this backlog".

Thanks for pointing out the source code to mumi!
  
-- 
Katherine

Re: Incentives for review

[Katherine Cox-Buday]

Ludovic Courtès  writes:

>> On Tue, 19 Oct 2021 at 14:56, Ludovic Courtès 

> But I also view things from a different angle: everyone contributes in their
> own way, and each contribution is a gift.

Maybe selfishly, but I really agree with this. I think this is just the nature 
of community-based projects: people are going to scratch their own itch, and 
when time allows, do altruistic things for other people.

Some people (e.g. me) don't have very much time at all to do the altruistic 
things (which gnaws at me). I do what I can, when I can, and hope that someone 
else benefits.

> A good middle ground may be to provide incentives for review.  How?  I’m
> not sure exactly, but first by making it clear that review is makes the
> project move forward and is invaluable.
>
>>> I think it’s about finding the right balance to be reasonably efficient
>>> while not compromising on quality.
>>
>> I totally agree.  And I do not see nor understand where is the
>> inefficiency here when asking to go via guix-patches and wait two weeks
>> for adding a new package.

Often I find that people on projects/teams have fundamentally different 
understandings of what reviews are for. Are they quality control? Mentoring 
opportunities? Opportunities to teach others how something new works? A way to 
encourage cohesiveness in the project?

It can help to publicly state the intent somewhere. I think the word "review" 
is mentioned in the manual 11 times, but nowhere does it say what the review's 
purpose is.

Large, public projects like Guix are a bit different, so I'm not sure this 
applies, but reviews meant to be gates on quality are my least favorite:

(Please note: these are general observations about the industry and not 
necessarily specific to Guix)

- The reviewers are human too, and there are various circumstances where they
  will miss things. Some of the most insidious forms of this is are: tragedy
  of the commons, i.e.

  > Submitter: They always do such a good job catching things. I think this is
  > good, but I know they'll catch any issues.

  > Reviewer: I feel bad this has sat for so long, this person usually does a
  > good job. +1 without a detailed review.

  > Submitter: A +1! It must not have had any issues.

- Unavoidably, because of human nature, groups form, and certain people
  experience less friction getting patches in. See the last point.

- There is a feedback loop present: those who have committed earlier have
  control and are more likely to reject later commits which don't do things as
  they would have. Sometimes "quality" is abused as a cover for opinion. Very
  few people are doing this maliciously, but it still happens.

- As I mentioned in another thread[1], trying to chase the ideal of quality
  may actually be worse in the end, or be a local maxima for quality or
  utility. Focusing on velocity may help achieve the global maxima for both.
  As always, there is a balance.

> It’s not about urgency but rather about not contributing to the growth
> of our patch backlog, which is a real problem.

I have often seen folks on various projects worried about the size of various 
backlogs: bugs, issues, etc. I think it is human to want to try and contain 
something that appears to be growing, unbounded.

I think the thing that bothers us is a sense that the backlog is becoming 
unmanageable, or too large to triage. I submit that this is actually a tooling 
and organizational issue, and not an intrinsic issue to be solved. Bugs may 
still be valid; patches may still have valuable bones to modify.

I think the real issue is that as a backlog grows, the tools we're used to 
using cannot answer the questions we want to ask: what is most relevant to me 
or the project right now?

To me, this sounds like a search and display problem.

[1] - https://lists.gnu.org/archive/html/guix-devel/2021-10/msg00081.html

-- 
Katherine

Could the Go importer use the Go toolchain? (was Re: Go importer and packages with version flags)

[Katherine Cox-Buday]

Sarah Morgensen  writes:

>> As an aside, when I started writing the importer, I didn't know it was a
>> possibility to just use the Go toolchain to help us generate packages. I
>> thought "the Guix way" was to do it all in scheme. It's nice that it's in
>> scheme, but I would love to leverage the Go toolchain instead.
>>
>> IMO, module resolution and graph dependencies in Go are complicated enough
>> that I'd much rather take the effort we put in trying to replicate and keep
>> up with the Go toolchain's behavior, and spend that effort elsewhere.
>>
>> Does anyone have opinions on this?
>
> Hmmm.  Setting aside whether or not we want to use external tools in
> general...
>
> If we use the Go tool, we shift the problem domain into "translating between
> `go' and Guix."
>
> For example: when Go downloads a module, it only saves the module, not the
> whole repository*, so we can't use that to generate the hash.  (* Except it
> does if you used GOPROXY=direct, but that's in the module cache, and only a
> bare repository.)

We could use the ~GOMODCACHE~ environment variable[0] to specify where these 
files are placed.

> Or, the fact that we import the latest available version of packages (unless
> --pin-versions is used), but Go uses exact versions selected with MVS [0].

Won't the importer import the import path you specify? The Go toolchain can 
also specify a ~@latest~ "version" to fetch the latest.

> You might also be interested in taking a look at Gentoo dealing with this
> translation problem [1].
>
> I'm not saying that this would necessarily be a bad tradeoff either, but it's
> definitely a tradeoff.

Agreed. I think the problem space of managing the translation between the 
toolchain and Guix is probably both smaller and less volatile.

I.e., the toolchain operations that dump dependency graph type stuff tend to 
support JSON output, and the download bits have options for where stuff gets 
stored on disk. As you know, the steps to go from an import path to a 
repository on-disk is complicated. If we could keep that complexity 
encapsulated upstream, which also happens to be the reference implementation, 
it could probably make the importer resilient to change and maybe even faster.

> Did you have something particular in mind as far as leveraging the Go tooling?

I haven't though too much about it, but ~GOPROXY=direct~ was my guess too. 
Thinking about this, would a shallow fetch into a bare repository be so bad?

It made me start wondering why our ~git-download~ type /doesn't/ have this 
behavior. Are we unnecessarily pulling down the entire commit history with all 
tags and branches of a large repository?

I don't even know what the philosophy is being doing this or not. Isn't the 
only commitment a Guix package makes for software at a particular 
version/commit that the store contains that immutable version of the source, 
not the complete repository?

Also, Guix supports all kinds of download types, one of which is zip files. The 
rust importer generates packages that fetch ~.tar.gz~'s from https://create.io. 
When I started writing the importer, I specifically kept the packages it 
generated decoupled from Google's module cache, but to make the comparison: the 
Go importer could do the same thing as the Rust importer: download zips from 
https://proxy.golang.org.

If this line of reasoning is wrong, it also looks like the toolchain respects 
standard VCS configuration. I wonder if there's a way to manipulate the fetches 
so that they're not shallow or bare.

TLDR: A shallow fetch into a bare repository might actually be /ideal/! It's 
small, and still directly connected to the package's actual source.

> [0] 
>
> [1]  cmd/go: allow extraction of
> urls used to download dependencies
>
> --
> Sarah

[0] - https://golang.org/ref/mod#environment-variables
[1] - https://golang.org/ref/mod#private-module-repo-auth

-- 
Katherine

Re: Merging wip-guix-home to master

[Katherine Cox-Buday]

Xinglu Chen  writes:

> I disagree, I think it’s OK for things like (guix git), which are mainly
> used by developers, to not be documented in the manual.

I strongly disagree with this. As a long-time developer, I have used 
documentation both as a user and as a developer many times.

Documentation is for everyone.

-- 
Katherine

Re: Go importer and packages with version flags

[Katherine Cox-Buday]

Sarah Morgensen  writes:

> Hi Katherine, Jack,
>
> Katherine Cox-Buday  writes:
>
>> Jack Hill  writes:
>>
>>> Hi Guix,
>>
>> Hey, Jack, a few thoughts.
>>
>>> While I was was working with the go importer today, it suggested I package
>>> go-github-com-russross-blackfriday-v2. Fair enough, except we already have a
>>> package for go-github-com-russross-blackfriday.
>>
>> I was poking around a rust code-base the other day and I noticed our crate
>> importer, and thus a lot of crate packages, have major-version suffixes. I
>> think one of the unique benefits of Guix is that it can simultaneously have
>> multiple versions of libraries installed, and I think we should allow this
>> for library packages.
>>
>> I know that leads to dependency graph explosion, but perhaps we only commit
>> to substitutes for the latest version, and thus any packages using old
>> versions. It should converge over time unless packages go unmaintained.
>>
>> I thought our current stance was to only allow one version at a time, but
>> the crate packages made me question this. I'd like clarity too.
>
> I think there's a bit of a difference between (our packages for) the Rust and
> Go ecosystems here.
>
> In the Go ecosystem, a module is uniquely identified by its module path,
> e.g. "github.com/russross/blackfriday/v2".  According to Go's major version
> suffix rules [0], "[s]tarting with major version 2, module paths must have a
> major version suffix like /v2 that matches the major version."  Therefore,
> each major version is logically a different module according to Go, and so I
> think we should treat them as separate packages as well.  (Note that in many
> cases we can use 'inherit' for brevity.)

That's a great point! I hadn't considered that we could leverage this to 
consider major versioned Go modules as separate packages. That's great!

> Additionally, the major version suffix rules dictate that "[i]f an old package
> and a new package have the same import path, the new package must be backwards
> compatible with the old package."  Assuming upstream sources follow the rules,
> we should be able to update each Go package within each major version without
> breaking dependencies.
>
> (A corollary to that is that packages often break if you try to use a v2 when
> it is expecting a v1.)
>
> I think this differs from Rust, where we have, for example, package-0.1 and
> package-0.2 coexisting.  This difference should prevent dependency graph
> explosion for Go.

It's nice that our Rust packages are enjoying the same stance, but I'm still 
not clear on why? Does Rust have the same guarantees?

> There are some caveats with "major version suffixes":
>
> * Major versions 0 and 1 don't get a version suffix (so no /v1)...
>
> * ...except for module paths starting with "gopkg.in/", which always have
>   a major version suffix, but theirs is styled ".v1" rather than "/v1".
>
> * Modules may either be located in the repository root, or in a "/v2"
>   subdirectory (for major version 2).  This makes things difficult for our
>   importer, because we can't know whether the unpack path should include "/v2"
>   without looking at the repository contents.
>
> This is why Jack had to manually add "/v2" to the unpack path.  I recently
> made some changes to the importer to better handle, for example,
> "github.com/example/repository/subproject", but it doesn't yet discriminate
> between "/subproject" and "/v2", so it treated "/v2" like a subdirectory of
> the repository.  (Until we fix this properly, the importer should probably not
> count major version suffixes when calculating the unpack path, since most
> projects don't use a "/v2" subdirectory.)

As an aside, when I started writing the importer, I didn't know it was a 
possibility to just use the Go toolchain to help us generate packages. I 
thought "the Guix way" was to do it all in scheme. It's nice that it's in 
scheme, but I would love to leverage the Go toolchain instead.

IMO, module resolution and graph dependencies in Go are complicated enough that 
I'd much rather take the effort we put in trying to replicate and keep up with 
the Go toolchain's behavior, and spend that effort elsewhere.

Does anyone have opinions on this?

> All that to say... I think we should definitely maintain coexisting Go v2, v3,
> etc. package definitions.  We should probably go the way of Rust though, so we
> have them all in the same package, at different ve

Re: Go importer and packages with version flags

[Katherine Cox-Buday]

Jack Hill  writes:

> Hi Guix,

Hey, Jack, a few thoughts.

> While I was was working with the go importer today, it suggested I package
> go-github-com-russross-blackfriday-v2. Fair enough, except we already have a
> package for go-github-com-russross-blackfriday.

I was poking around a rust code-base the other day and I noticed our crate 
importer, and thus a lot of crate packages, have major-version suffixes. I 
think one of the unique benefits of Guix is that it can simultaneously have 
multiple versions of libraries installed, and I think we should allow this for 
library packages.

I know that leads to dependency graph explosion, but perhaps we only commit to 
substitutes for the latest version, and thus any packages using old versions. 
It should converge over time unless packages go unmaintained.

I thought our current stance was to only allow one version at a time, but the 
crate packages made me question this. I'd like clarity too.

> The packages differ in their version (2.1.0 verses 2.0.1), their imputs (the
> imported package definition is missing the inputs and native-inputs that we
> have in the Guix package), and the presense of the v2 versioning
> disambiguation tag.

The importer suggested package is correct. It looks like the listed 
dependencies dropped >= v1.5. See:

- 
https://pkg.go.dev/github.com/russross/blackfriday@v2.0.0+incompatible?tab=imports
vs
- https://pkg.go.dev/github.com/russross/blackfriday/v2?tab=imports

I wasn't sure if pkg.go listed test dependencies, so I ran this command to be 
sure:

#+begin_example
$ go list -deps -test -f '{{.ImportPath}} {{.Standard}}' |awk '{if ($2 == 
"false")  print $1}'
github.com/russross/blackfriday/v2
github.com/russross/blackfriday/v2.test
#+end_example

I think those were just dependencies incorrectly carried over as the package 
changed.

I hope that helps.

-- 
Katherine

Re: Merging wip-guix-home to master

[Katherine Cox-Buday]

Andrew Tropin  writes:

> The core part of Guix Home project has been moved from rde
> repository[fn:1] to wip-guix-home branch of guix repository.
>
> I'm about a week on wip-guix-home branch completely and Guix Home works
> fine.  There are no any major issues on rde-devel and guix-devel mailing
> lists and it seems that branch is ready to be merged.

I just want to thank you for the work. I don't think I'll use this everywhere, 
but it is definitely going to be helpful in some environments. Thank you!

-- 
Katherine

Re: “What’s in a package”

[Katherine Cox-Buday]

Ludovic Courtès  writes:

> I agree.  Like Konrad and you wrote, it’s good that we can all have our
> quick-and-dirty packages in personal channels, and it’s good that these
> are separate channels.

Definitely! I would also encourage Guix to adopt some of the tools that make 
creating quick, but bad, packages easier. E.g. builds that patch elfs with the 
library paths of their dependencies.

> I’m not sure how tooling could help in the way of making quick packages,
> but it’s worth exploring.  Examples that come to mind are: merging the
> npm importer that currently lives in a branch, and providing a generic
> “guix import upstream” importer that would figure out as much as
> possible so that one doesn’t have to start from a blank page.

That sounds like a great start. I tossed out some other ideas elsewhere in the 
thread. Most of them involve meta-inspection of the package, Guix ecosystem, 
runtime environment and logs. It would be nice in general to have a kind of 
"agent" that you could run repeatedly over the course of packaging that would 
suggest next steps on ~stderr~ and next logical packaging definition on 
~stdout~. Kind of like pair-programming with Guix :)

It would perform different operations dependent on what stage in the life-cycle 
the package is at, i.e. ~import~ when no package definition exists, build when 
one does, and possibly running the result in a container when the package build 
succeeds.

E.g. your PyTorch example, starting from scratch (note: ~guix import~ may not 
always feel like the right command to invoke in this example. This may be some 
larger concept than import; also, the example always redirects to package.scm 
for brevity, but the user would probably want to look at it first):

#+begin_example
  $ guix import upstream pytorch

  stderr: This looks like it might be python package (heuristics.scm:123 - 
package name starts with py), try this instead:
  stdout: guix import upstream pypi pytorch

  $ guix import upstream pypi pytorch | tee package.scm

  $ guix import upstream package.scm | tee package.scm

  stderr: downloading...
  stderr: It looks like this fails to build because it's missing autoconf 
(heuristics.scm:133 - grepping build output found a missing autoconf error). 
Try adding it as a native-input.
  stdout: (package definition with imports defined and native-input modified)

  $ guix import upstream package.scm

  stderr: downloading...
  stderr: It looks like this package comes with binaries that are available as 
Guix packages (heuristics.scm:143 - unpacking source includes binary or object 
files, heuristics.scm:153 - bundled files match output of known packages). Try 
this package definition instead:
  stdout: (package definition with suggested inputs and overridden phases to 
remove the binaries from the download)

  $ guix import upstream package.scm | tee package.scm

  stderr: It looks like this package vendors libraries that are available as 
Guix packages (heuristics.scm:163 - unpacking source includes vendored 
libraries, heuristics.scm:153 - bundled files match output of known packages). 
Try this package definition instead:
  stdout: (package definition with suggested inputs and overridden phases to 
remove the vendored libraries from the download)

  $ guix import upstream package.scm | tee package.scm
  
  stderr: It looks like this package searches XDG_DATA_DIRS for some files 
(heuristics.scm:163 - grep an strace of a containerized run of the output). Try 
this package definition instead:
  stdout: (package definition with ~native-search-paths~ defined)
#+end_example

etc., etc. Typing that out, it feels dangerously close to Microsoft's Clippy, 
but hopefully more helpful :)

Heuristics, by definition, wouldn't be correct all the time, but this kind of 
thing could help new contributors (or experienced contributors with bad 
memories like me!), and in some cases actually do some of the programming.

And every time someone comes to the mailing list or IRC with a question, we can 
ask ourselves if this is a common question, and maybe create a new heuristic.

-- 
Katherine

Re: [Spam:]Re: “What’s in a package”

[Katherine Cox-Buday]

Konrad Hinsen  writes:

> What is so far insufficiently supported by computing technology is the
> necessary transition from "fast" to "robust".

This is really a large problem in the industry. Especially since in most 
circles moving fast is considered the preferred way to do things. SaaS and 
abstractions are endemic, and while helpful to get things going, it can lead to 
precarious systems with interdependencies and risks that are not fully 
understood or appreciated.

The "fast" path does allow people to test out new ideas very quickly, but there 
is a hidden cost. As we've seen these past years with COVID-19 and the world's 
supply chains, efficiency has some kind of inverse relationship with 
robustness. If you go too far down the path of efficiency, you are not very 
flexible, and you're building sand castles.

It's for this reason I appreciate having "robust" software underneath my sand 
castle. At least I know only so much can crumble :)

> There are a few exceptions, such as programming language with gradual typing.
> In most situations, moving software from exploratory to robust involves a lot
> of rewriting, often manually, with no tooling support.

I really like this framing. How can we support every step of the continuum with 
a gentle pull towards robustness? That sounds like something to strive for.

>> Bringing this back to Guix, and maybe the GNU philosophy, it has been
>> very helpful for me to be able to leverage the flexibility of Guix to
>> occasionally do things the "fast" way, perhaps by packaging a
>> binary. Paradoxically, it has allowed me to stay within the Guix and
>> free software ecosystem. In my opinion, flexibility is key to growing
>> the ecosystem and community, and I would encourage Guix as a project
>> to take every opportunity to give the user options.
>
> +100 :-)
>
> There is a lot we can improve here. Tutorials would be a good start.
> Example: How do you package a binary in Guix? In particular, how do you
> deal with binaries that have binary dependencies that they expect in
> /lib etc.? A next step would be tool support: Grab whatever PyPI offers,
> even if it's only binary wheels, and turn that into a Guix package.

I want to be careful here in what I suggest. I think it is very important that 
Guix remain a bastion of robust software with very high standards. I don't want 
to see the PyPi PyTorch packages of the world in Guix. I /do/ want to see 
tooling in Guix that allows users to package and utilize these things as 
first-class primitives in the Guix world.

In other words, let me create beautiful and terrible things, but don't let me 
unleash them on the world.

So with your example: make it really easy to transform that PyPi package into a 
terrible Guix primitive of some kind, but don't let me commit it to Guix proper.

> Another aspect would be supporting software development moving from fast
> to robust. Suppose I have software I compile by hand, or via a simple
> Makefile, somewhere in my home directory. How do I go from there to (1)
> a quick-and-dirty Guix package, then (2) a very basic publishable Guix
> package and finally (3) a Guix package with tests and documentation?
> The path should be supported by various tools, from automatic rewriting
> to debugging. As an example, something I have wished for more than once
> is the possibility to run the individual build steps of a Guix package
> under my own account in my home directory, for debugging purposes.

This kind of stuff really excites me. If we could build tooling that somehow 
moves things along the continuum, that would really be something. Maybe 
interactive software that introspects how a package is written and behaves at 
runtime (in a container?) and utilizes the homoiconicity of scheme to suggest 
modifications of the package, or next steps. E.g. expand the linter to suggest 
things like documentation, or to identify at what point on the continuum the 
package might currently be, and how to move forward. Does the package vendor 
binaries? Does Guix have any packages that look like those binaries? What does 
the packages binaries want to link to? What paths does it try and access when 
run?

Speaking of industry, I don't think we leverage software to build software 
enough.

And by the way, none of those ideas would be possible if Guix weren't such a 
robust and sane ecosystem.

-- 
Katherine

Re: “What’s in a package”

[Katherine Cox-Buday]

Ludovic Courtès  writes:

> Hello Guix!
>
> I and others are often disappointed (or angry!) when looking at the
> weaknesses of the most popular software deployment tools.  I felt that
> acutely after packaging PyTorch last month and felt the need to look
> more closely at what others are doing and to document our motivation,
> having put so much sweat in all these packages:
>
>   https://hpc.guix.info/blog/2021/09/whats-in-a-package/
>
> It’s probably no news to people here, but the packaging approach has a
> direct impact on verifiability, and thus on security and transparency,
> as expected from a scientific process.  The idea is to explain all that
> looking at the contents of packages, in particular for pip and CONDA.
>
> Feel free to share with non-Guix people and to comment!
>
> Ludo’.

I appreciate this post very much. Setting aside questions of freedom, and 
security -- both of which I value a lot -- the main benefit of Guix has, for 
me, been: simplicity (but not always ease)[1]. I.e., when trying to achieve a 
goal, it is a pain to package things that aren't yet packaged, but what I get 
in return are sane environments, deployments, and meta-data about all of these.

This is perhaps a rehash of the "worse is better"[2] conversation, but I often 
struggle with deciding whether to do things the "fast" way, or the "correct" 
way. I think when your path is clear, the correct way will get you farther, 
faster. But when you're doing experiments, or exploratory programming, being 
bogged down with the "correct" way of doing things (i.e. Guix packages) might 
take a lot of time for no benefit. E.g. maybe you end up packaging a cluster of 
things that you find out don't work out for you. Of course the challenge is: if 
you choose the fast way, and it works out, do you got back to do it the correct 
way so that you're on sound footing?

Bringing this back to Guix, and maybe the GNU philosophy, it has been very 
helpful for me to be able to leverage the flexibility of Guix to occasionally 
do things the "fast" way, perhaps by packaging a binary. Paradoxically, it has 
allowed me to stay within the Guix and free software ecosystem. In my opinion, 
flexibility is key to growing the ecosystem and community, and I would 
encourage Guix as a project to take every opportunity to give the user options.

[1] - https://www.infoq.com/presentations/Simple-Made-Easy/
[2] - https://en.wikipedia.org/wiki/Worse_is_better

-- 
Katherine

Re: On the naming of System and Home services modules.

[Katherine Cox-Buday]

Xinglu Chen  writes:

> On Wed, Sep 15 2021, Andrew Tropin wrote:

>> Records even for the same services have slightly different fields and
>> because of macro nature can't be reused between Home and System
>> services. In more details I mentioned this problem here:
>> https://lists.sr.ht/~abcdw/rde-devel/%3C87y2cqifpx.fsf%40yoctocell.xyz%3E#%3c878s4l8kai@trop.in%3E
>
> Some services might be useful to have in both Guix System and Guix Home;
> for instance, Guix System currently has a service for configuring
> Syncthing, and I think it makes sense to also have one for Guix Home,
> this would mean that people not using Guix System (me :-)) could also
> have Guix manage Syncthing.  With the current approach, we would have to
> copy and paste quite a bit of code, and if the Syncthing service for
> Guix System changes, then the one for Guix Home might have to change as
> well.

I agree with this point. I have several Guix systems, and several
non-guix systems with Guix managing some services. In the past, I have
had to write my own Shepherd services for things already written as
system services.

>> The intersection of home and system services should be very low, so
>> there is not much benifit here as well.
>
> Quite the opposite, I think it would be great if home and system
> services could integrate more with each other.  In NixOS, the NixOS
> modules and Home Manager modules feel like two very distinct things, and
> it’s not really easy to share things between them.

I agree.

>> ** Summary
>> Let's keep System and Home services separate for the sake of clarity,
>> reuse code via shared modules or just exports in (gnu services ...).

[...]

>> However, ~(gnu home services ...)~ also looks cool, but it would be a
>> little inconsistent with system services, which will have one level of
>> nestiness less: ~(gnu services)~.
>>
>> IMO, ~(gnu home services ...)~ would be a good choice if we use ~(gnu
>> system services)~ for system services.
>
> Yeah, having both (gnu system service) and (gnu home service) could make
> sense, but since we only have (gnu services), I don’t think it makes
> much sense.

I haven't put in the energy to follow the rational behind the proposed
naming schemas, but I'd like to suggest this as well: =(gnu services
home)=. This namespace increases in specificity, and I think it would
easily follow that things in =(gnu services home)= might utilize things
in =(gnu services)=. Or I also like the idea of =home= and =system=
being sibling namespaces.

-- 
Katherine