Re: Split (gnu packages suckless) module

[Liliana Marie Prikler]

Hi Guix,

Am Donnerstag, den 28.10.2021, 15:55 + schrieb Mekeor Melire:

> The reasons for this proposal are:
> 
>   1. We generally do not create modules according to the groups of
>  developers, but rather package declarations are grouped into
>  modules according to their function (e.g. "wm") or according to
>  some technical or factual realm they belong to (e.g. "haskell" 
>  or "music"). There is no reason why software from the Suckless 
>  group should not follow this rule.
Actually, the software from the suckless group follows this rule in a
sense, as we also have a module for freedesktop, gnome, kde, etc.

>   2. There are hints indicating that the Suckless group sympathizes 
>  to oppressive Nazi ideology. The Suckless group is a registered
>  association in Munich, Germany.
This is a somewhat weak argument, as filenames only matter when editing
– you'd still be linking to a Nazi group in the homepage, which matters
much more.

On the other hand, Guix users often complain about suckless-style
software as it is in fact not the most Guix-friendly to handle.  Simply
renaming the file to "suckmore" would in my opinion be adequate, and
switching to maintained forks of their stuff ethical.

Regards,
Liliana

Re: Split (gnu packages suckless) module

[Leo Famulari]

On Thu, Oct 28, 2021 at 03:55:26PM +, Mekeor Melire wrote:
> I would like to propose to split the (gnu packages suckless) module,
> located in the /gnu/packages/suckless.scm source file.

I agree. There are already a handful of "suckless" packages scattered in
other modules.

> The reasons for this proposal are:
> 
>   1. We generally do not create modules according to the groups of
>  developers, but rather package declarations are grouped into
>  modules according to their function

Sometimes we do, as with (gnu packages freedesktop), but those packages
tend to be more tightly coupled. Anyways, that's a digression.

Thanks for bringing this up.

Split (gnu packages suckless) module

[Mekeor Melire]

Hi Guix :)

I would like to propose to split the (gnu packages suckless) module,
located in the /gnu/packages/suckless.scm source file. It contains
package declarations for software written by the Suckless group, namely
following packages:

blind, colors, dmenu, dwm, fortify-headers, human, lchat, libutf,
noice, prout, sbm, scron, sent, skroll, slock, slscroll, slstatus,
spoon, st, surf, tabbed, wificurse, xbattmon

The reasons for this proposal are:

  1. We generally do not create modules according to the groups of
 developers, but rather package declarations are grouped into
 modules according to their function (e.g. "wm") or according to
 some technical or factual realm they belong to (e.g. "haskell" or
 "music"). There is no reason why software from the Suckless group
 should not follow this rule.

  2. There are hints indicating that the Suckless group sympathizes to
 oppressive Nazi ideology. The Suckless group is a registered
 association in Munich, Germany.

 
https://web.archive.org/web/20210817073134/https://ev.suckless.org/impressum/

 - It has been reported that one of their members (L.H.) called
   their mail server "Wolfsschanze" which was the name of Hitlers
   last bunker.

   https://twitter.com/pid_eins/status/1113738766471057408

 - In 2017, they did a torch march during their conference which is
   a Nazi symbol.

   
https://web.archive.org/web/20210910103557/http://suckless.org/conferences/2017/

 - In 2018, on the Devuan mailing list, one of their members (E.W.)
   depicts the defeat of Nazi Germany as a genocide on Germans.

   
https://web.archive.org/web/20210613185451/https://lists.dyne.org/lurker/message/20181010.120415.e5f96d11.en.html

What do you think? Maybe we should start discussing where each of these
suckless software package declarations should be moved to? My
suggestions follow:

#+begin_src
blind:
Command line video editing utilities
→ (gnu packages video)

colors:
Extract colors from pictures
→ (gnu packages image)

dmenu:
Dynamic menu
→ (gnu packages xorg)

dwm:
Dynamic window manager
→ (gnu packages wm)

fortify-headers:
Standalone fortify-source implementation
→ ?

human:
Convert bytes to human readable formats
→ (gnu packages textutils)

lchat:
Line chat is a frontend for the irc client ii from suckless
→ (gnu packages irc)

libutf:
Plan 9 compatible UTF-8 library
→ (gnu packages c)

noice:
Small file browser
→ (gnu packages disk)

prout:
Smaller lp command
→ ?

sbm:
Simple bandwidth monitor
→ (gnu packages monitoring)

scron:
Simple cron daemon
→ ?

sent:
Plain-text presentation tool
→ (gnu packages textutils)

skroll:
Commandline utility which scrolls text
→ (gnu packages textutils)

slock:
Simple X session lock
→ (gnu packages xorg)

slscroll:
Scroll-back buffer program for st
→ (gnu packages terminals)

slstatus:
Status monitor for window managers
→ (gnu packages wm)

spoon:
Set dwm status
→ (gnu packages wm)

st:
Simple terminal emulator
→ (gnu packages terminals)

surf:
Simple web browser
→ (gnu packages web-browsers)

tabbed:
Tab interface for application supporting Xembed
→ (gnu packages xorg)

wificurse:
Wifi DoS attack tool
→ (gnu packages networking)

xbattmon:
Simple battery monitor for X
→ (gnu packages xorg)
#+end_src

Ciao Guix :)

Re: Time for a request-for-comments process?

[Tobias Platen]

GNUnet has something similar called the GANA (GNUnet Assigned Numbers
Authority)

https://git.gnunet.org/gana.git/


On Thu, 2021-10-28 at 12:33 +0200, Bengt Richter wrote:
> Hi Zimoun, Ludo,
> 
> On +2021-10-28 10:42:02 +0200, zimoun wrote:
> > Hi Ludo,
> > 
> > On Wed, 27 Oct 2021 at 23:22, Ludovic Courtès  wrote:
> > 
> > > The recent ‘guix shell’ addition is almost anecdotal technically
> > > yet
> > > important for the project because users interact with Guix
> > > primarily
> > > through the CLI.  Adding a new command is a commitment (our users
> > > must
> > > trust it won’t change overnight), and getting the details wrong
> > > could
> > > make us fail to honor that commitment.
> > > 
> > > For ‘guix shell’ I left time for comments and repeatedly asked
> > > people to
> > > comment; yet pushing it was a bit stressful: Did I make a
> > > mistake?  Did
> > > everyone with a stake in this really have a chance to comment?
> > 
> > Note that the patch received many comments; especially v1.  Then,
> > only
> > two people commented for v2.  And v3 did not receive any general
> > LGTM –
> > I sent one for the two trivial parts I reviewed.
> > 
> > For me, one important root of the issue is the review process.  I
> > feel
> > the balance described in thread «Incentives for review» [1],
> > 
> >     There’s a balance to be found between no formal commitment
> > on
> >     behalf of committers, and a strict and codified commitment
> >     similar to what is required for participation in the
> > distros
> >     list¹.
> > 
> > is hard to found.  Because, on one hand, the project has to honor
> > commitments, and on the other hand, no one as team is committed to
> > do
> > it.
> > 
> > From my understanding, your message here is interesting because
> > somehow
> > you did a similar experience as maintainer of what is an usual
> > non-committer contributor experience; somehow explained by some of
> > my
> > soft ramblings from the thread «Incentives for review» [1]. :-)
> > Another
> > meaningful because similar, IMHO, failure of the review process is
> > patch#45692 [4].
> > 
> > As you know, I did some stats in order to find, or at least
> > discuss, how
> > to improve the situation grounded on current facts.  Aside, Debbugs
> > already provides insightful numbers [2], especially this one [3]:
> > 
> >     
> > 
> > The traffic on guix-patches is quite high and I do not know how
> > many
> > people subscribe – I guess few.  I hope the discussed improvements
> > of
> > Mumi will help.  Or perhaps if someone is willing to setup a Guix
> > official public-inbox; for example, the instance
> > https://yhetil.org/guix
> > is providing helpful tools for easily filtering, IMHO.
> > 
> > 1: 
> > 2: 
> > 3: 
> > 4: 
> > 
> > Closing parenthesis, back to your question. :-)
> > 
> > > That makes me think it’s perhaps time for a formalized
> > > request-for-comments (RFC) kind of process for such “major
> > > changes”.  We
> > > could draw inspiration from one of the many existing processes:
> > > Python’s
> > > PEPs, Scheme’s SRFIs, Nix’s RFCs, Rust’s MCPs, etc.  I think a
> > > major
> > > goal of the process would be to formalize a minimum and a maximum
> > > duration under which an RFC is under evaluation, and a mechanism
> > > to
> > > determine whether it’s accepted or withdrawn.
> > 
> > Aside the usual review process, at least my understanding what the
> > review process should be, you are asking for a special flag then
> > expose
> > materials to various channels of communication, IIUC.
> > 
> > For sure, it appears a good idea. :-)
> > 
> > Concretely, what does it mean “major changes”?  How many of these
> > do you
> > consider that happened in the recent two past years?
> > 
> > For example, the recent label-less input style [5] is one instance,
> > IMHO.  However, I do not remember* if it was discussed outside
> > guix-patches.
> > 
> > In addition to the change itself sent to guix-patches with an
> > associated
> > number, it could be worth to send that information elsewhere.
> > 
> > What would be this elsewhere?  Create another dedicated (low-
> > traffic)
> > list would scatter the information and I am not convinced it would
> > help
> > to gain attraction at the moment.  However, it would ease digging
> > in the
> > future because all would be in only one archive.
> 
>  Wherever "elsewhere" might be, I'd like notification when there is
> something
>  new to read.
> 
> I'm visualizing a screensaver hook where the screen is restored after
> being locked,
> like logging in the first or subsequent times, to show an
> intermediate popup
> before going on as usual. Sort of a dynamic motd (message of the
> day).
> 
> What I'd like then, to find out details, is acc

Re: Incentives for review

[Katherine Cox-Buday]

zimoun  writes:

>> I have often seen folks on various projects worried about the size of
>> various backlogs: bugs, issues, etc. I think it is human to want to
>> try and contain something that appears to be growing, unbounded. 
>
> …about patches only.  Bug is another story. :-)

Sorry, I meant to speak to both and instead repeated bugs with a different 
word, issues! I think patches and bugs are similar in this context.

> Just number to fix the idea about large backlog.

I think it's really great that you went through the trouble to quantify this. 
Thank you.

>> I think the thing that bothers us is a sense that the backlog is
>> becoming unmanageable, or too large to triage. I submit that this is
>> actually a tooling and organizational issue, and not an intrinsic
>> issue to be solved. Bugs may still be valid; patches may still have
>> valuable bones to modify.
>
> This is the point.  What do you do?  What could we improve about tooling
> and organisation to better scale and deal with this “becoming
> unmanageable backlog”?

I tried to give some ideas here[1].

> From my point of view, it is good to have this issue.  It means that
> Guix is becoming more popular.  And we – regular user, contributor,
> committer – have to adapt to this increasing workload, IMHO.

I totally agree!

> The question is how.  And how to invite people to complete review. :-)

Humans usually enjoy community. I think the group activities are really great.

>> I think the real issue is that as a backlog grows, the tools we're
>> used to using cannot answer the questions we want to ask: what is most
>> relevant to me or the project right now?
>
> If it is relevant to the project then it is also relevant to me as an
> user.  And vice-versa. ;-)

I think I did not give the proper context. I meant relevant as in "I am working 
on this package. Is anyone else? What tickets might I update? What other 
trivial bugs might I fix while I'm looking at this?"

I.e. relevant in the temporal sense.

> When something relevant to me is not making progress, it often means
> people are busy elsewhere, so I try to comment (review?) about patches
> or bugs.  It is a Sisyphean task because the workload never
> decreases. :-)  Or maybe structured procrastination. ;-)

I find it helpful to not think of it as a discrete task, but work along a 
continuum -- a joyful habit of collectively helping everyone to have something 
nice :)

"A society grows great when old (wo)men plant trees whose shade they know they 
shall never sit in."

[1] - https://lists.gnu.org/archive/html/guix-devel/2021-10/msg00158.html

-- 
Katherine

Re: Accuracy of importers?

[Katherine Cox-Buday]

Ludovic Courtès  writes:

>go   (Sarah? Leo? Raghav?)

I have only used this a few times so far, but the quality seems to have gotten 
a lot better. My impression, though, due to the nature of how we have to 
generate packages so as to not be reliant on a centralized GOPROXY server 
(namely one controlled by Google), is that we stumble dealing with the 
heterogeneity of the internet. There are a few things which could make this 
situation better:

There is an open issue[1] for a better API to https://pkg.go.dev which may 
eventually allow us to query for things like license, VCS path, etc. This could 
obviate Guix's need to crawl the internet.

I was also discussing[2] the pros/cons of relying on the Go tool-chain to do 
most of the work for us. I think doing so might be making the right trade-offs, 
but it sounds like[3] we are blocked by cgit's ability to work with shallow 
checkouts. Since Guix has a build environment, maybe we could just use Git the 
CLI instead of a scheme library when necessary.

I hope this helps, and good luck with your talk!

[1] - https://github.com/golang/go/issues/36785
[2] - https://lists.gnu.org/archive/html/guix-devel/2021-09/msg00344.html
[3] - https://lists.gnu.org/archive/html/guix-devel/2021-10/msg00020.html

-- 
Katherine

Patches that should be applied in the Future

[Jelle Licht]

Hello there guix,

All of this is about core-updates-frozen at revision dac8d013bd1.

`c-ares' in gnu/packages/adns.scm has its tests disabled; On my x86_64
machine, it seems the tests for c-ares pass just fine because of the
defined GTEST_FILTER when I re-enable them;

With my limited git-foo, it seems that this particular change comes from
merging signed/master back into c.u.f. in e486b2b674b. To me it seems
that (re-?)introducing the `tests? #f' might have been something silly
done by git-merge.

This is still a pretty good situation to be in; we have working tests
that we simply don't run :). Practically, I wonder what the next step
would be to make sure things go well in the future;

We can't 'fix' c.u.f. this late in the cycle without rebuilding pretty
much everything. In my limited understanding of juggling many branches,
it also seems that merging c.u.f. into master (and later, core-updates)
will have this `tests? #f' being propagated.

What can we do to make sure we won't simply forget to apply this and
other such changes? 

- Jelle

Re: Accuracy of importers?

[Ricardo Wurmus]

Ludovic Courtès  writes:


Hello Guix!

As I’m preparing my PackagingCon talk and wondering how language 
package
managers could make our lives easier, I thought it’d be 
interesting to

know how well our importers are doing.

My understanding is that most of them require manual 
intervention—i.e.,

one has to tweak what ‘guix import’ produces, even if we ignore
synopsis/description/license, to set the right inputs, etc.  If 
we were
to estimate the fraction of imported packages for which manual 
changes

are needed, what would it look like?

   importer fraction of imported packages needing changes

[…]
   cran 5% (Ricardo? Simon? seems to almost always 
   work?)


Like Lars and Simon wrote: the importers work *really* well for 
both CRAN and Bioconductor, so much so that I’m using them in the 
background here:


https://git.elephly.net/gitweb.cgi?p=software/r-guix-install.git;a=blob;f=guix-install.R;h=2766aa1f2d248a8ed2a4eb4c3244b85574d326e2;hb=HEAD

The biggest annoyance is the missing “license:” prefix when 
packaging things for gnu/packages/cran.scm or 
gnu/packages/bioconductor.scm.  Descriptions need regular clean- 
up work (e.g. to complete sentences), even though we’re using some 
heuristics to fix the most common stylistic problems.  It’s really 
not a big deal, though.


The biggest missing feature is recursive import of dependencies 
hosted on Github or Mercurial (with “-r -a git” or “-r -a hg”). 
I.e. a package on Github that declares a dependency on another 
package that’s also only hosted on Github will fail to import that 
dependency.  This is pretty rare, but it happens with experimental 
bioinfo software.



   texlive  (Ricardo? Thiago? Marius?)


This one is not usable.  I’d even add “at all”.  I keep announcing 
that one day I’ll replace it with a new importer, but that new 
importer just isn’t ready yet.


What about licensing info: which ones provide accurate licensing 
info?

My guess:

   gnu
   pypi
   cpan
   cran


The CRAN importer is as accurate as upstream allows.  CRAN 
requires a free license, Bioconductor requires a license 
declaration (there have been very few cases where the license was 
not correct, but a number of cases where the license was non-free, 
such as the Artistic 1.0 license.  Bioconductor sometimes is 
sneaky and the R code is free but a necessary library is not.



   texlive


Pretty terrible.  The license declaration is generally too vague. 
Licenses are often declared without version number, and sometimes 
it’s just some generic “free” license.  A new importer based on 
texlive.tlpdb would not improve this by much, because the upstream 
declarations are just spotty and unreliable.


--
Ricardo


PS: attached is a rough WIP patch of what I had been using to 
import new texlive stuff.


diff --git a/guix/import/texlive.scm b/guix/import/texlive.scm
index 18d8b95ee0..b94aa1cf40 100644
--- a/guix/import/texlive.scm
+++ b/guix/import/texlive.scm
@@ -19,10 +19,12 @@
 
 (define-module (guix import texlive)
   #:use-module (ice-9 match)
+  #:use-module (ice-9 rdelim)
   #:use-module (sxml simple)
   #:use-module (sxml xpath)
   #:use-module (srfi srfi-11)
   #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-2)
   #:use-module (srfi srfi-26)
   #:use-module (srfi srfi-34)
   #:use-module (web uri)
@@ -125,9 +127,9 @@ (define (fetch-sxml name)
   (xml->sxml (http-fetch url)
  #:trim-whitespace? #t
 
-(define (guix-name component name)
+(define (guix-name name)
   "Return a Guix package name for a given Texlive package NAME."
-  (string-append "texlive-" component "-"
+  (string-append "texlive-"
  (string-map (match-lambda
(#\_ #\-)
(#\. #\-)
@@ -186,12 +188,123 @@ (define (sxml-value path)
  ((lst ...) `(list ,@lst))
  (license license)))
 
+(define tlpdb
+  (memoize
+   (lambda ()
+ (let ((file "/home/rekado/dev/gx/branches/master/texlive.tlpdb")
+   (fields
+'((name . string)
+  (shortdesc . string)
+  (longdesc . string)
+  (catalogue-license . string)
+  (catalogue-ctan . string)
+  (srcfiles . list)
+  (runfiles . list)
+  (docfiles . list)
+  (depend   . list)))
+   (record
+(lambda* (key value alist #:optional (type 'string))
+  (let ((new
+ (or (and=> (assoc-ref alist key)
+(lambda (existing)
+  (cond
+   ((eq? type 'string)
+(string-append existing " " value))
+   ((eq? type 'list)
+(cons value existing)
+ (cond
+  ((eq? type 'string)
+   val

Re: Accuracy of importers?

[Julien Lepiller]

Le 28 octobre 2021 03:02:27 GMT-04:00, "Ludovic Courtès" 
 a écrit :
>Hello Guix!
>
>As I’m preparing my PackagingCon talk and wondering how language package
>managers could make our lives easier, I thought it’d be interesting to
>know how well our importers are doing.
>
>My understanding is that most of them require manual intervention—i.e.,
>one has to tweak what ‘guix import’ produces, even if we ignore
>synopsis/description/license, to set the right inputs, etc.  If we were
>to estimate the fraction of imported packages for which manual changes
>are needed, what would it look like?
>
>   importer fraction of imported packages needing changes
>
>   gnu  90% (doesn’t know about dependencies)
>   pypi 50% (some miss source distro, “sdist”; some have
> non-Python deps)
>   cpan ?
>   hackage  ?
>   stackage (Lars?)
>   egg  (Xinglu?)
>   elpa (Nicolas?)
>   gem  ?
>   go   (Sarah? Leo? Raghav?)
>   cran 5% (Ricardo? Simon? seems to almost always work?)
>   crate10% (Efraim?)
>   texlive  (Ricardo? Thiago? Marius?)
>   opam (Julien?)

I find it pretty good, when importing huge numbers of packages recently, I was 
able to build all of them without modification. However, lots rely on a github 
tarball, so I would change the source in these cases before sending them to 
guix.

>   minetest (Maxime? Vivien?)
>   julia (WIP)  (Simon?)
>   npm (WIP)(Jelle? Timothy?)
>
>(Lower is better.)  What would be your estimate?  
>
>Among those, which importers provide source that differs from what you’d
>get from upstream’s checkout or release tarballs?  My guess:
>
>   pypi (see LastPyMile paper)
>   elpa (gives hosted tarballs that can differ from upstream repo)
>   gem (similar to PyPI)
>   npm (ditto)
>
>What about licensing info: which ones provide accurate licensing info?
>My guess:
>
>   gnu
>   pypi
>   cpan
>   cran
>   elpa
>   go (?)
>   cran
>   crate (?)
>   texlive
>   opam (?)
>   minetest (?)
>
>TIA! :-)
>
>Ludo’.
>

Re: Time for a request-for-comments process?

[Bengt Richter]

Hi Zimoun, Ludo,

On +2021-10-28 10:42:02 +0200, zimoun wrote:
> Hi Ludo,
> 
> On Wed, 27 Oct 2021 at 23:22, Ludovic Courtès  wrote:
> 
> > The recent ‘guix shell’ addition is almost anecdotal technically yet
> > important for the project because users interact with Guix primarily
> > through the CLI.  Adding a new command is a commitment (our users must
> > trust it won’t change overnight), and getting the details wrong could
> > make us fail to honor that commitment.
> >
> > For ‘guix shell’ I left time for comments and repeatedly asked people to
> > comment; yet pushing it was a bit stressful: Did I make a mistake?  Did
> > everyone with a stake in this really have a chance to comment?
> 
> Note that the patch received many comments; especially v1.  Then, only
> two people commented for v2.  And v3 did not receive any general LGTM –
> I sent one for the two trivial parts I reviewed.
> 
> For me, one important root of the issue is the review process.  I feel
> the balance described in thread «Incentives for review» [1],
> 
> There’s a balance to be found between no formal commitment on
> behalf of committers, and a strict and codified commitment
> similar to what is required for participation in the distros
> list¹.
> 
> is hard to found.  Because, on one hand, the project has to honor
> commitments, and on the other hand, no one as team is committed to do
> it.
> 
> From my understanding, your message here is interesting because somehow
> you did a similar experience as maintainer of what is an usual
> non-committer contributor experience; somehow explained by some of my
> soft ramblings from the thread «Incentives for review» [1]. :-) Another
> meaningful because similar, IMHO, failure of the review process is
> patch#45692 [4].
> 
> As you know, I did some stats in order to find, or at least discuss, how
> to improve the situation grounded on current facts.  Aside, Debbugs
> already provides insightful numbers [2], especially this one [3]:
> 
> 
> 
> The traffic on guix-patches is quite high and I do not know how many
> people subscribe – I guess few.  I hope the discussed improvements of
> Mumi will help.  Or perhaps if someone is willing to setup a Guix
> official public-inbox; for example, the instance https://yhetil.org/guix
> is providing helpful tools for easily filtering, IMHO.
> 
> 1: 
> 2: 
> 3: 
> 4: 
> 
> Closing parenthesis, back to your question. :-)
> 
> > That makes me think it’s perhaps time for a formalized
> > request-for-comments (RFC) kind of process for such “major changes”.  We
> > could draw inspiration from one of the many existing processes: Python’s
> > PEPs, Scheme’s SRFIs, Nix’s RFCs, Rust’s MCPs, etc.  I think a major
> > goal of the process would be to formalize a minimum and a maximum
> > duration under which an RFC is under evaluation, and a mechanism to
> > determine whether it’s accepted or withdrawn.
> 
> Aside the usual review process, at least my understanding what the
> review process should be, you are asking for a special flag then expose
> materials to various channels of communication, IIUC.
> 
> For sure, it appears a good idea. :-)
> 
> Concretely, what does it mean “major changes”?  How many of these do you
> consider that happened in the recent two past years?
> 
> For example, the recent label-less input style [5] is one instance,
> IMHO.  However, I do not remember* if it was discussed outside
> guix-patches.
> 
> In addition to the change itself sent to guix-patches with an associated
> number, it could be worth to send that information elsewhere.
> 
> What would be this elsewhere?  Create another dedicated (low-traffic)
> list would scatter the information and I am not convinced it would help
> to gain attraction at the moment.  However, it would ease digging in the
> future because all would be in only one archive.

 Wherever "elsewhere" might be, I'd like notification when there is something
 new to read.

I'm visualizing a screensaver hook where the screen is restored after being 
locked,
like logging in the first or subsequent times, to show an intermediate popup
before going on as usual. Sort of a dynamic motd (message of the day).

What I'd like then, to find out details, is access (CLI or Web browser) to a 
relational DB
like the ones supporting online shopping, but in this case I am shopping for 
info, and filtering
by e.g. zimoun or ludo instead of Asus or Lenovo, and similarly to narrow or 
widen context
for OS or achitecture etc. (I am obviously suggesting something broader than 
just "shopping"
for RFC info :)

The shopping interface could be used to select what info to subscribe for,
to get notifications about different info "products" or categories.

> Maybe info-guix co

Re: Accuracy of importers?

[Lars-Dominik Braun]

Hi Ludo’,

> Right.  PyPI/setup.py/.whl doesn’t contain info as to how to run tests,
> right?
technically setup.py has a standard test target, but it’s been
deprecated for years and it must be enabled manually by the project. I’m
not aware of any standard pyproject.toml approach to this. It might be
possible to parse tox.ini.

> >>hackage  ?
> >>stackage (Lars?)
> > I’ve mostly used the updater, not the importer, so I can’t say a
> > number unfortunately.
> Did the updater suggest input changes?
yes, I added it in 127828ddd74fc950c0403ca58a6f650355e3d67d, but it
cannot update #:cabal-revision, which is a common source for errors. Would
be nice if updaters could just return an entirely new package and the
generic updater code would modify/merge the existing package definition
as needed.

Cheers,
Lars

Re: Accuracy of importers?

[zimoun]

Re,

On Thu, 28 Oct 2021 at 11:07, zimoun  wrote:

> For instance, filtered on build-system.  For sure, all
> python-build-system packages do not come from PyPI, r-build-system from
> CRAM/Bioconductor, etc. but, IMHO, such stats would provide a good
> estimation for how upstream archives ELPA, PyPI, CRAN/Bioconducor,
> Hackage/Stackage, TexLive, etc. are ready for Guix without manual
> intervention.

Ah, better: filtering on uris, e.g., pypi-uri, cran-uri,
biocondutor-uri, crate-uri, and so on, i.e., where each importer
looks.  And compare how many 'arguments' is not the default one.  This
should get an estimation on the accuracy of importers, IMHO.

Cheers,
simon

Re: Accuracy of importers?

[zimoun]

Hi,

On Thu, 28 Oct 2021 at 09:02, Ludovic Courtès  wrote:

> My understanding is that most of them require manual intervention—i.e.,
> one has to tweak what ‘guix import’ produces, even if we ignore
> synopsis/description/license, to set the right inputs, etc.  If we were
> to estimate the fraction of imported packages for which manual changes
> are needed, what would it look like?

Manual intervention depends on how it is packaged upstream, i.e., the
availability of metadata.  Therefore, it depends on the upstream
archive.  PyPI is messier than CRAN for instance but I find hard to back
this claim with numbers – just intuition. :-)


>importer fraction of imported packages needing changes
>
>gnu  90% (doesn’t know about dependencies)
>pypi 50% (some miss source distro, “sdist”; some have
>  non-Python deps)
>cpan ?
>hackage  ?
>stackage (Lars?)
>egg  (Xinglu?)
>elpa (Nicolas?)
>gem  ?
>go   (Sarah? Leo? Raghav?)
>cran 5% (Ricardo? Simon? seems to almost always work?)
>crate10% (Efraim?)
>texlive  (Ricardo? Thiago? Marius?)
>opam (Julien?)
>minetest (Maxime? Vivien?)
>julia (WIP)  (Simon?)
>npm (WIP)(Jelle? Timothy?)

For the ones I use “cran” and “cran -a bioconductor“, and from the
feedback I get from users in my lab, one regular complaint is the
missing prefix ’license:’ – if that’s the issue, it means the importer
works pretty well. :-)

About Julia, it is often not clear how to extract “dependencies”, which
means the run-time ones vs the test-time other ones.


> (Lower is better.)  What would be your estimate?

For all cases, to have a good estimation, I would examine how many
packages already in Guix have a non-default ’argument’ and modified
phases.  It means that these packages require manual fix.

Missing or incorrect dependencies happen.  But they are impossible to
evaluate.  However, special ’argument’ are something eval-able and for
now, none importer tweaks that, IIUC, thus it would sketch the picture
«how well our importers are doing».

For instance, filtered on build-system.  For sure, all
python-build-system packages do not come from PyPI, r-build-system from
CRAM/Bioconductor, etc. but, IMHO, such stats would provide a good
estimation for how upstream archives ELPA, PyPI, CRAN/Bioconducor,
Hackage/Stackage, TexLive, etc. are ready for Guix without manual
intervention.


Cheers,
simon

Re: Accuracy of importers?

[Ludovic Courtès]

Hi!

Lars-Dominik Braun  skribis:

>> My understanding is that most of them require manual intervention—i.e.,
>> one has to tweak what ‘guix import’ produces, even if we ignore
>> synopsis/description/license, to set the right inputs, etc.  If we were
>> to estimate the fraction of imported packages for which manual changes
>> are needed, what would it look like?
>> 
>>importer fraction of imported packages needing changes
>>pypi 50% (some miss source distro, “sdist”; some have
>>  non-Python deps)
> that seems right, although the most common modification I do nowadays
> is replacing 'check with a pytest phase.

Right.  PyPI/setup.py/.whl doesn’t contain info as to how to run tests,
right?

>>hackage  ?
>>stackage (Lars?)
> I’ve mostly used the updater, not the importer, so I can’t say a
> number unfortunately.

Did the updater suggest input changes?

>>cran 5% (Ricardo? Simon? seems to almost always work?)
> In my experience the number of interventions here goes towards zero
> actually, except for description. It’s pretty good :)

Yay!

>>npm (WIP)(Jelle? Timothy?)
> Maybe 5%? But the imported packages do not build anything and don’t
> run tests either, so chances for failure are pretty low.

Yeah.

> Would it be possible to just run the importer again for existing packages
> and compare the result (minus synopsis/description) with what’s
> available in Guix? That should give you much more accurate numbers than
> our guesswork.

That’s a good idea.  I can try and do that on a sample of packages.

Thanks!

Ludo’.

Re: Time for a request-for-comments process?

[zimoun]

Hi Ludo,

On Wed, 27 Oct 2021 at 23:22, Ludovic Courtès  wrote:

> The recent ‘guix shell’ addition is almost anecdotal technically yet
> important for the project because users interact with Guix primarily
> through the CLI.  Adding a new command is a commitment (our users must
> trust it won’t change overnight), and getting the details wrong could
> make us fail to honor that commitment.
>
> For ‘guix shell’ I left time for comments and repeatedly asked people to
> comment; yet pushing it was a bit stressful: Did I make a mistake?  Did
> everyone with a stake in this really have a chance to comment?

Note that the patch received many comments; especially v1.  Then, only
two people commented for v2.  And v3 did not receive any general LGTM –
I sent one for the two trivial parts I reviewed.

For me, one important root of the issue is the review process.  I feel
the balance described in thread «Incentives for review» [1],

There’s a balance to be found between no formal commitment on
behalf of committers, and a strict and codified commitment
similar to what is required for participation in the distros
list¹.

is hard to found.  Because, on one hand, the project has to honor
commitments, and on the other hand, no one as team is committed to do
it.

>From my understanding, your message here is interesting because somehow
you did a similar experience as maintainer of what is an usual
non-committer contributor experience; somehow explained by some of my
soft ramblings from the thread «Incentives for review» [1]. :-) Another
meaningful because similar, IMHO, failure of the review process is
patch#45692 [4].

As you know, I did some stats in order to find, or at least discuss, how
to improve the situation grounded on current facts.  Aside, Debbugs
already provides insightful numbers [2], especially this one [3]:



The traffic on guix-patches is quite high and I do not know how many
people subscribe – I guess few.  I hope the discussed improvements of
Mumi will help.  Or perhaps if someone is willing to setup a Guix
official public-inbox; for example, the instance https://yhetil.org/guix
is providing helpful tools for easily filtering, IMHO.

1: 
2: 
3: 
4: 

Closing parenthesis, back to your question. :-)

> That makes me think it’s perhaps time for a formalized
> request-for-comments (RFC) kind of process for such “major changes”.  We
> could draw inspiration from one of the many existing processes: Python’s
> PEPs, Scheme’s SRFIs, Nix’s RFCs, Rust’s MCPs, etc.  I think a major
> goal of the process would be to formalize a minimum and a maximum
> duration under which an RFC is under evaluation, and a mechanism to
> determine whether it’s accepted or withdrawn.

Aside the usual review process, at least my understanding what the
review process should be, you are asking for a special flag then expose
materials to various channels of communication, IIUC.

For sure, it appears a good idea. :-)

Concretely, what does it mean “major changes”?  How many of these do you
consider that happened in the recent two past years?

For example, the recent label-less input style [5] is one instance,
IMHO.  However, I do not remember* if it was discussed outside
guix-patches.

In addition to the change itself sent to guix-patches with an associated
number, it could be worth to send that information elsewhere.

What would be this elsewhere?  Create another dedicated (low-traffic)
list would scatter the information and I am not convinced it would help
to gain attraction at the moment.  However, it would ease digging in the
future because all would be in only one archive.

Maybe info-guix could be used.  But it would mean that everybody would
be allowed to this list, when currently the messages landing there are
somehow “highly filtered”.  However, an announce there pointing where
and how to comment could be something helping to get more attention.
Adding a section under Contributing about the process too.

Last, the core question is formalization.  Formalize the process (min,
max duration, expectations of evaluation, mechanism to accept or
withdraw, i.e., how to revolve different points of views, etc.) strongly
depends on what “major changes” means and how often that happens.  Could
you provide examples of such “major changes”?  It would help for drawing
a sketch of such formalization grounded on concrete examples.


Cheers,
simon

5: 


*remember discussion: Personally, I receive all emails to all lists. All
in my Inbox.  Thus, the channel does not mind for my workflow. :-)
However, dealing with Guix traffic is a daily task – if I am off for a
couple of days or holidays or busy by day job, then I skip s

Re: Tricking peer review

[Ludovic Courtès]

Hi!

Christine Lemmer-Webber  skribis:

> Here's another way to think of it, borrowing from some ocap systems: the
> hash is the actual, canonical identifier of this package revision.  The
> URL to get the package is merely a "hint" as to where to get it.

Yes, definitely.  It remains we need to educate ourselves to not give
too much credit to this hint.

Thanks,
Ludo’.

Re: Accuracy of importers?

[Lars-Dominik Braun]

Hi Ludo’,

> My understanding is that most of them require manual intervention—i.e.,
> one has to tweak what ‘guix import’ produces, even if we ignore
> synopsis/description/license, to set the right inputs, etc.  If we were
> to estimate the fraction of imported packages for which manual changes
> are needed, what would it look like?
> 
>importer fraction of imported packages needing changes
>pypi 50% (some miss source distro, “sdist”; some have
>  non-Python deps)
that seems right, although the most common modification I do nowadays
is replacing 'check with a pytest phase.

>hackage  ?
>stackage (Lars?)
I’ve mostly used the updater, not the importer, so I can’t say a
number unfortunately.

>cran 5% (Ricardo? Simon? seems to almost always work?)
In my experience the number of interventions here goes towards zero
actually, except for description. It’s pretty good :)

>npm (WIP)(Jelle? Timothy?)
Maybe 5%? But the imported packages do not build anything and don’t
run tests either, so chances for failure are pretty low.

Would it be possible to just run the importer again for existing packages
and compare the result (minus synopsis/description) with what’s
available in Guix? That should give you much more accurate numbers than
our guesswork.

Cheers,
Lars

Accuracy of importers?

[Ludovic Courtès]

Hello Guix!

As I’m preparing my PackagingCon talk and wondering how language package
managers could make our lives easier, I thought it’d be interesting to
know how well our importers are doing.

My understanding is that most of them require manual intervention—i.e.,
one has to tweak what ‘guix import’ produces, even if we ignore
synopsis/description/license, to set the right inputs, etc.  If we were
to estimate the fraction of imported packages for which manual changes
are needed, what would it look like?

   importer fraction of imported packages needing changes

   gnu  90% (doesn’t know about dependencies)
   pypi 50% (some miss source distro, “sdist”; some have
 non-Python deps)
   cpan ?
   hackage  ?
   stackage (Lars?)
   egg  (Xinglu?)
   elpa (Nicolas?)
   gem  ?
   go   (Sarah? Leo? Raghav?)
   cran 5% (Ricardo? Simon? seems to almost always work?)
   crate10% (Efraim?)
   texlive  (Ricardo? Thiago? Marius?)
   opam (Julien?)
   minetest (Maxime? Vivien?)
   julia (WIP)  (Simon?)
   npm (WIP)(Jelle? Timothy?)

(Lower is better.)  What would be your estimate?  

Among those, which importers provide source that differs from what you’d
get from upstream’s checkout or release tarballs?  My guess:

   pypi (see LastPyMile paper)
   elpa (gives hosted tarballs that can differ from upstream repo)
   gem (similar to PyPI)
   npm (ditto)

What about licensing info: which ones provide accurate licensing info?
My guess:

   gnu
   pypi
   cpan
   cran
   elpa
   go (?)
   cran
   crate (?)
   texlive
   opam (?)
   minetest (?)

TIA! :-)

Ludo’.