Astronomy 2023/08 updates
Hi Guix! The end of August Astronomy related packages udpates and new packages. Hope QA will green them up soon :) * Updates - date-time :: 20230905T163102+0100 - commit :: 8d19ff23052bffb1c43f0d39f543eb0b1e363074 - issues :: https://issues.guix.gnu.org/65814 - quick [16/16][100%] - [X] libxisf :: would be upgraded from 0.2.8 to 0.2.9. - [X] python-astropy-healpix :: would be upgraded from 0.7 to 1.0.0. - [X] python-astropy :: would be upgraded from 5.3.1 to 5.3.2 - [X] python-bayesicfitting :: would be upgraded from 3.1.1 to 3.2.0 - [X] python-cdflib :: would be upgraded from 1.0.5 to 1.1.0. - [X] python-crds :: would be upgraded from 11.17.0 to 11.17.4. - [X] python-fitsio :: would be upgraded from 1.1.10 to 1.2.0 - [X] python-jwst :: would be upgraded from 1.10.2 to 1.11.4 - [X] python-photutils :: would be upgraded from 1.7.0 to 1.9.0 - [X] python-pyvo :: would be upgraded from 1.4.1 to 1.4.2 - [X] python-rad :: would be upgraded from 0.15.0 to 0.17.1 - [X] python-roman-datamodels :: would be upgraded from 0.15.0 to 0.17.1 - [X] python-stdatamodels :: would be upgraded from 1.7.1 to 1.8.0 - [X] python-sunpy :: Enable more tests. - [X] python-tslearn :: would be upgraded from 0.6.1 to 0.6.2 - [X] splash :: would be upgraded from 3.8.2 to 3.8.4. - complex [0/8][0%] - [ ] python-asdf :: would be upgraded from 2.15.0 to 2.15.1. Requires lower version of python-jsonschema which and also contain vendorized version which breaks all tests now, https://github.com/asdf-format/asdf-standard/releases/tag/1.0.3 - [ ] cfitsio :: would be upgraded from 4.2.0 to 4.3.0. Building the following 22 packages would ensure 66 dependent packages are rebuilt: alfa@2.2 splash@3.8.2 sextractor@2.28.0 imppg@0.6.5 stellarium@23.2 python-fitsio@1.1.10 glnemo2@1.21.0 siril@1.0.6 python-tslearn@0.6.1 python-regions@0.7 python-jwst@1.10.2 python-astroalign@2.4.2 python-sunpy@5.0.0 python-poliastro@0.17.0 julia-wcs@0.6.2 phd2@2.6.11 gnuastro@0.20 aoflagger@3.3.0 swarp@2.41.5 julia-fitsio@0.17.0 gwenview@23.04.3 labplot@2.9.0. Need more love with dependents as some of them failed to build. - [ ] python-hvpy :: would be upgraded from 1.0.1 to 1.1.0. Requires more fresh version of python-pydantic, which depends on python-pydantic-core which require brand new Rust based package. - [ ] python-spherical-geometry :: would be upgraded from 1.2.22 to 1.2.23. Can't be updated to the latest see: https://github.com/spacetelescope/spherical_geometry/issues/227 - [ ] aoflagger :: would be upgraded from 3.2.0 to 3.3.0. Build failerur, more work requried. - [ ] casacore :: would be upgraded from 3.4.0 to 3.5.0. Build failerur, more work requried. - [ ] indi :: would be upgraded from 1.9.9 to 2.0.3. stellarium failing. - [ ] libpasastro :: would be upgraded from 1.4.0-1.e3c218d to 1.4.1. Full package reformat is requried. * New packages - [PATCH] gnu: Add python-casa-formats-io: https://issues.guix.gnu.org/65571 - [PATCH] gnu: Add WCSTools: https://issues.guix.gnu.org/65880 -- … наш разум - превосходная объяснительная машина которая способна найти смысл почти в чем угодно, истолковать любой феномен, но совершенно не в состоянии принять мысль о непредсказуемости.
comparing commit-relation using Scheme+libgit2 vs shellout plumbing Git
Hi, On Mon, 11 Sep 2023 at 14:26, Maxim Cournoyer wrote: > In the grand scheme of things (pun intended), we'd like every > programming to be feasible via nice Scheme APIs, which is what Guile-Git > provides to work with git repositories. The appeal is to have a single > language to rule them all, reducing friction among Guix contributors. > The alternative here is to have an API reduced to invoking system > commands with string arguments, which is less expressive and lacks > elegance. As Maxim noticed in the message that I am proposing to revisit, it seems that libgit2 comes with some performance penalties. As wolf is illustrating in the message: bug#65720: Guile-Git-managed checkouts grow way too much wolf Mon, 11 Sep 2023 16:42:59 +0200 id:ZP8nc1m8rN_34XV-@ws https://issues.guix.gnu.org//65720 https://issues.guix.gnu.org/msgid/ZP8nc1m8rN_34XV-@ws https://yhetil.org/guix/ZP8nc1m8rN_34XV-@ws it might be possible to use an invocation of plain Git command which is much faster in this case. Well, that’s need to be investigated, IMHO. For instance, instead of the current ’commit-relation’ implementation, (define (commit-relation old new) "Return a symbol denoting the relation between OLD and NEW, two commit objects: 'ancestor (meaning that OLD is an ancestor of NEW), 'descendant, or 'unrelated, or 'self (OLD and NEW are the same commit)." (if (eq? old new) 'self (let ((newest (commit-closure new))) (if (set-contains? newest old) 'ancestor (let* ((seen (list->setq (commit-parents new))) (oldest (commit-closure old seen))) (if (set-contains? oldest new) 'descendant 'unrelated)) which relies on ’commit-closure’, they propose to use a plumbing Git commands, as: (define (shelling-commit-relation old new) (let ((h-old (oid->string (commit-id old))) (h-new (oid->string (commit-id new (cond ((eq? old new) 'self) ((zero? (git-C %repo "merge-base" "--is-ancestor" h-old h-new)) 'ancestor) ((zero? (git-C %repo "merge-base" "--is-ancestor" h-new h-old)) 'descendant) (else 'unrelated Well, this needs to be checked (read the Git documentation which is probable harder than read some Scheme implementation ;-)) in order to see if these invocations are doing the same. >> I’m quite confident this would be slow > > My version is ~2000x faster compared to (guix git): > > Guix: 1048.620992ms > Git: 0.532143ms On my machine, I get something less spectacular for a history with 1000 commits in between. --8<---cut here---start->8--- scheme@(guix-user)> ,time (commit-relation* 1000th newest) $1 = ancestor ;; 0.128948s real time, 0.082921s run time. 0.046578s spent in GC. scheme@(guix-user)> ,time (commit-relation 1000th newest) $2 = ancestor ;; 4.588075s real time, 5.521358s run time. 1.404764s spent in GC. --8<---cut here---end--->8--- I did something very similar as wolf is proposing and named it ’commit-relation*’. Well, considering the implementation of ’commit-relation’, I think the slowness is expected compared to the plain plumbing Git command. Basically, ’commit-closure’ walks the Git history and for sure the loop cannot be as efficient as an optimized Git specific implementation. Hum, I think the most annoying is the time spent in GC. Basically, ’commit-closure’ is building a set with many visited elements and that set must be garbage collected. And this GC time is not nothing compared to the whole time, IMHO. I agree with the grand scheme of things and that’s why I started this thread. :-) However, for what it is worth, today I am less convinced that manipulating libgit2 is able to provide “not-so-worse” performance compared to what plain plumbing Git commands could offer. Cheers, simon
Re: How can we decrease the cognitive overhead for contributors?
Giovanni Biscuolo writes: > [[PGP Signed Part:Undecided]] > Hi Liliana, > > Liliana Marie Prikler writes: > > [...] > >> For example, whenever people say that "forges would improve stuff", my >> reply is (modulo phrasing) "yes, for the people who are already used to >> forges". > > I just want to point out that actually Guix _do_have_ a forge. the > software is Savane and it's hosted on savannah.gnu.org: > https://savannah.gnu.org/projects/guix > > This is just to remind everyone that there are very different forges out > there, and: > > All forges suck, _no one_ sucks less To quote the elementary HIG: "Design is not just, like, your opinion, man" https://docs.elementary.io/hig/design-philosophy#design-is-not-just-like-your-opinion-man Just because two alternatives are both imperfect does not mean they are equally bad. Statistically speaking, 10 forges having the same "suckiness" score has an infinitesimal chance. A more meaningful argument would be that the alternatives have different goals, but even that's shaky, since most of their goals are shared. If it takes me 30 minutes to find a commit in a repo on one forge and 5 minutes on another, then one forge is *objectively* worse in *that aspect*. I don't think repeating that no forge sucks less advances the conversation towards any solution other than keeping the status quo, which can't really be called a solution.
Re: hard dependency on Git? (was bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts)
Vagrant Cascadian writes: > [[PGP Signed Part:Undecided]] > On 2023-09-11, Simon Tournier wrote: >> On Mon, 11 Sep 2023 at 16:23, Ludovic Courtès wrote: >>> Note that the patch series adds a hard dependency on Git. >>> This is because the existing ‘git-fetch’ code depends on Git, >>> which is itself motivated by the fact that Git supports >>> shallow clones and libgit2/Guile-Git doesn’t. > ... >> Personally, I do not have a strong opinion about the Big Plan™. I note >> that the introduction of Git as a hard dependency is a slippery slope >> considering the current state of libgit2. Here, it starts with “git >> clone”, then “git gc” (unsupported by libgit2) is also in the pipes >> (#65720 [1]). > > What about making git an optional dependency, and only calling out to > "git gc" if git is available in PATH? Maybe possible also with shallow > clones? > > Then you have the best/worst of both worlds! Speaking to the worst, you > have at least two disparate codepaths for a seemingly similar operation, > and that might be annoying... > > live well, > vagrant > > [[End of PGP Signed Part]] For what it's worth, I wrote a small (incomplete) tool for some commit analysis that used specific --format arguments that were easy to parse. It's not especially difficult.
Re: Closing submission incomplete since years?
Hi Simon, Simon Tournier writes: > Hi Maxim, > > On Mon, 11 Sep 2023 at 13:54, Maxim Cournoyer > wrote: > >> If you want to package elasticsearch, know it's not in the Guix as we >> speak, and see it as the first item when browsing >> https://issues.guix.gnu.org/search?query=elasticsearch, I have some hope >> that someone could find it. Not to mention search engines, which are >> also good at this. > > Well, maybe Xapian is enough? > > >> That said, I get your point. Perhaps a 'rejected' user tag would be >> useful? I doubt most users would know to use them though... but maybe >> if it's exposed via Mumi it could be helpful. > > Well, if there is a tag and it is documented in the manual, then I think > it is easier. For example, assume ’moreinfo’ is this tag. And consider > I am interested in looking for some packages but I am hesitant about the > name and let say I think it is phylogeny. Then: > > https://issues.guix.gnu.org/search?query=moreinfo+phylogeny > > and I find ’mrbayes’ which is the one I wanted. So I resume. > > Well, instead of ’rejected’, I propose ’unfinished’ or ’incomplete’ or > reuse ’moreinfo’ for that. I feel the ’rejected’ as “no way!” when it > is not the idea. I like the user tag 'incomplete', as moreinfo would yield many false positives, I think. -- Thanks, Maxim
Re: [workflow] Automatically close bug report when a patch is committed
Hi Liliana, Liliana Marie Prikler writes: > Am Montag, dem 11.09.2023 um 14:36 -0400 schrieb Maxim Cournoyer: >> Hi, >> >> Liliana Marie Prikler writes: >> >> [...] >> >> > Maybe make that bug-guix so that it matches with the mailing list >> > name, >> > though we also need a wording for when a patch is not a bug, e.g. a >> > simple package upgrade. >> > >> > WDYT about the following >> > Applies: [patch] >> > Closes: [patch] >> > Resolves: [patch] >> > Done: [patch] >> >> I don't follow; why do we need 'Applies' ? Why do we need a >> 'namespace'. Are these things the user would need to manually know >> and enter themselves in their commit messages? > I'm just asking which wording you prefer. For the tracker, they'd mean > the same as "Fixes", but fixes imho sounds like a bug, which "Update > Emacs to 29.2" isn't. Thus, something with a more neutral meaning like > "Resolves" might apply better here. If we choose this simple scheme where the top commit of a series can be annotated with Debbugs control commands, I'd opt for: --8<---cut here---start->8--- Applies: #bug-number --8<---cut here---end--->8--- I'm not sure what [patch] or namespace add (is it for a fancy URL?), so I'd drop them. >> If so, that's adding rather than reducing friction, and I'm not sure >> it'd gain much traction. The way I see it, it needs to happen >> automatically. > I mean, the way I imagine is that you type this as part of your message > and then debbugs would do the work of closing the bug. In short, "git > push" saves you the work of writing a mail because there's a hook for > it. Perhaps both approach could be combined. I still see value in a general scheme to automate closing applied series that linger on in Debbugs. [0] https://lists.gnu.org/archive/html/guix-devel/2023-09/msg00138.html Change-Ids would also add the benefit that any commit found in Git could easily be traced back to their submission on the guix-patches or guix trackers or vice-versa (git log --grep='Change-Id='), as noted by Giovanni. The process could go like this: 1. commits of a series pushed to master 2. Savannah sends datagram to a remote machine to trigger the post-commit job, with the newly pushed commits 'Change-Id' values (a list of them). 3. The remote machine runs something like 'mumi close-issues [change-id-1 change-id-2 ...]' In case it couldn't close an issue, it could send a notification to the submitter: "hey, I've seen some commits of series landing to master, but not all of the commits appears to have been pushed, please check" What mumi does internally would be something like: a) Check in its database to establish the Change-Id <-> Issue # relation, if any. b) For each issue, if issue #'s known Change-Ids are all covered by the change-ids in the arguments, close it This is a bit more complex (UDP datagram, mumi database) but it does useful work for us committers (instead of simply changing the way we currently do the work). When not provided any change-id argument, 'mumi close-issues' could run the process on its complete list of issues. Since it'd be transparent and requires nothing from a committer, it'd provide value without having to document yet more processes. -- Thanks, Maxim
Re: hard dependency on Git? (was bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts)
On 2023-09-11, Simon Tournier wrote: > On Mon, 11 Sep 2023 at 16:23, Ludovic Courtès wrote: >> Note that the patch series adds a hard dependency on Git. >> This is because the existing ‘git-fetch’ code depends on Git, >> which is itself motivated by the fact that Git supports >> shallow clones and libgit2/Guile-Git doesn’t. ... > Personally, I do not have a strong opinion about the Big Plan™. I note > that the introduction of Git as a hard dependency is a slippery slope > considering the current state of libgit2. Here, it starts with “git > clone”, then “git gc” (unsupported by libgit2) is also in the pipes > (#65720 [1]). What about making git an optional dependency, and only calling out to "git gc" if git is available in PATH? Maybe possible also with shallow clones? Then you have the best/worst of both worlds! Speaking to the worst, you have at least two disparate codepaths for a seemingly similar operation, and that might be annoying... live well, vagrant signature.asc Description: PGP signature
Re: How can we decrease the cognitive overhead for contributors?
Hi Liliana, On Mon, 11 Sep 2023 at 19:53, Liliana Marie Prikler wrote: > For "patch does not apply", the forge solution is typically to send a > notification to the issuer. No, that does not match my small experience. Because often the issuer is gone or not responding. As a reviewer using the forge solution, I am still able to pull the issuer branch and then resolve the conflicts if any. Using “our” workflow, I fail earlier in the process. I am not able to apply the patches against any branches (pull the PR somehow). An example: [bug#62202] [PATCH 0/21] Juliahub import script. https://issues.guix.gnu.org/msgid/871qlq89kz@ngraves.fr Even before looking at it, I have to spend some time to find a way to manually apply the patches. Then, rebasing on the top of master could lead to conflict but that another story and the same appears whatever the workflow. Nicolas did a cool job with this Julia importer, and because of this first boring task, I am procrastinating and delaying to the eternal tomorrow the review of the work. Whatever the complexity of this task, there is no value for the project. I read Ricardo’s message [1] as: our workflow is far to help us for having smooth reviews so it’s hard to convince folks already familiar with other workflows to adopt our. Not saying that these other workflows are better either. Cheers, simon 1: Re: How can we decrease the cognitive overhead for contributors? Ricardo Wurmus Fri, 08 Sep 2023 22:24:06 +0200 id:87tts44d2y@elephly.net https://yhetil.org/guix/87tts44d2y@elephly.net https://lists.gnu.org/archive/html/guix-devel/2023-09
Re: Closing submission incomplete since years?
Hi Maxim, On Mon, 11 Sep 2023 at 13:54, Maxim Cournoyer wrote: > If you want to package elasticsearch, know it's not in the Guix as we > speak, and see it as the first item when browsing > https://issues.guix.gnu.org/search?query=elasticsearch, I have some hope > that someone could find it. Not to mention search engines, which are > also good at this. Well, maybe Xapian is enough? > That said, I get your point. Perhaps a 'rejected' user tag would be > useful? I doubt most users would know to use them though... but maybe > if it's exposed via Mumi it could be helpful. Well, if there is a tag and it is documented in the manual, then I think it is easier. For example, assume ’moreinfo’ is this tag. And consider I am interested in looking for some packages but I am hesitant about the name and let say I think it is phylogeny. Then: https://issues.guix.gnu.org/search?query=moreinfo+phylogeny and I find ’mrbayes’ which is the one I wanted. So I resume. Well, instead of ’rejected’, I propose ’unfinished’ or ’incomplete’ or reuse ’moreinfo’ for that. I feel the ’rejected’ as “no way!” when it is not the idea. Cheers, simon
Re: [workflow] Automatically close bug report when a patch is committed
Am Montag, dem 11.09.2023 um 14:36 -0400 schrieb Maxim Cournoyer: > Hi, > > Liliana Marie Prikler writes: > > [...] > > > Maybe make that bug-guix so that it matches with the mailing list > > name, > > though we also need a wording for when a patch is not a bug, e.g. a > > simple package upgrade. > > > > WDYT about the following > > Applies: [patch] > > Closes: [patch] > > Resolves: [patch] > > Done: [patch] > > I don't follow; why do we need 'Applies' ? Why do we need a > 'namespace'. Are these things the user would need to manually know > and enter themselves in their commit messages? I'm just asking which wording you prefer. For the tracker, they'd mean the same as "Fixes", but fixes imho sounds like a bug, which "Update Emacs to 29.2" isn't. Thus, something with a more neutral meaning like "Resolves" might apply better here. > If so, that's adding rather than reducing friction, and I'm not sure > it'd gain much traction. The way I see it, it needs to happen > automatically. I mean, the way I imagine is that you type this as part of your message and then debbugs would do the work of closing the bug. In short, "git push" saves you the work of writing a mail because there's a hook for it. As for the namespace: you would have to type it on your own – hence why I prefer the URL approach as that can more easily be copied. I think we had a discussion that we don't want to involuntarily trigger stuff elsewhere, hence why we're marking our own bugs as our own. Cheers
Re: [workflow] Automatically close bug report when a patch is committed
Hi, Liliana Marie Prikler writes: [...] > Maybe make that bug-guix so that it matches with the mailing list name, > though we also need a wording for when a patch is not a bug, e.g. a > simple package upgrade. > > WDYT about the following > Applies: [patch] > Closes: [patch] > Resolves: [patch] > Done: [patch] I don't follow; why do we need 'Applies' ? Why do we need a 'namespace'. Are these things the user would need to manually know and enter themselves in their commit messages? If so, that's adding rather than reducing friction, and I'm not sure it'd gain much traction. The way I see it, it needs to happen automatically. -- Thanks, Maxim
Re: hard dependency on Git? (was bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts)
Hi wolf, wolf writes: [...] >> Personally, I do not have a strong opinion about the Big Plan™. I note >> that the introduction of Git as a hard dependency is a slippery slope >> considering the current state of libgit2. Here, it starts with “git >> clone”, then “git gc” (unsupported by libgit2) is also in the pipes >> (#65720 [1]). And after timing, I am almost sure that many operations >> using Guile-Git will be slower than their plain Git counter-parts. And >> we will start to parse the output of ’git’ plumbing commands. > > If you don't mind me asking, why is that so problematic approach? Git's > plumbing commands are intended to be used in scripts, so I am unsure what the > problem is. In the grand scheme of things (pun intended), we'd like every programming to be feasible via nice Scheme APIs, which is what Guile-Git provides to work with git repositories. The appeal is to have a single language to rule them all, reducing friction among Guix contributors. The alternative here is to have an API reduced to invoking system commands with string arguments, which is less expressive and lacks elegance. -- Thanks, Maxim
Seeking panelists for a Reproduciblity panel: Spack + Nix + Guix
Hi Guix developers, I'm hosting a panel for the University of California Santa Cruz's Open Source symposium. The panel will be on "Reproducibility" --- My goal is to try and get representatives from Guix, Spack & Nix to talk about various aspects of the store-model, reproducibility challenges and more! I have a few willing participants from Spack & Nix and reaching out here to find someone from the Guix ecosystem :) The panel (https://ucospo23.sched.com/) is on Thurdsay September 28th 2023. We have support for either in-person (Santa Cruz, CA) or remote. Please reach out to me if you are interested.
Re: hard dependency on Git? (was bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts)
Hi, Simon Tournier writes: > Hi, > > On Mon, 11 Sep 2023 at 16:23, Ludovic Courtès wrote: > >> Note that the patch series adds a hard dependency on Git. >> This is because the existing ‘git-fetch’ code depends on Git, >> which is itself motivated by the fact that Git supports >> shallow clones and libgit2/Guile-Git doesn’t. That's no longer true, as of libgit2 v1.7.0 [0]. I already have it packaged in a branch, but I need to iron out dependent breakages. [0] https://github.com/libgit2/libgit2/releases/tag/v1.7.0 So given there's no technical reasons not to use libgit2, I'd use that and keep the closure size down. -- Thanks, Maxim
Re: [workflow] Automatically close bug report when a patch is committed
Am Montag, dem 11.09.2023 um 10:09 +0200 schrieb Giovanni Biscuolo: > Hi! > > Liliana Marie Prikler writes: > > > Am Donnerstag, dem 07.09.2023 um 09:12 -0700 schrieb Vagrant > > Cascadian: > > > I am much more comfortable with the "Fixes" convention of: > > > > > > Fixes: https://issues.guix.gnu.org/NNN > > OK, I understand Vagrant's concerns: we need a _namespaced_ URI, but > there is no need that URI is the URL of **one** of our current web > interfaces, why not the other one? ;-) Well, I like cool URLs, but maybe that's a nice benefit here. > > I propose: > > Fixes: [optional bug description] > > where namespace is the package name, in our case "guix"; fo example: > > Fixes: Emacs hangs when I press a key Maybe make that bug-guix so that it matches with the mailing list name, though we also need a wording for when a patch is not a bug, e.g. a simple package upgrade. WDYT about the following Applies: [patch] Closes: [patch] Resolves: [patch] Done: [patch]
Re: Closing submission incomplete since years?
Hi Simon, Simon Tournier writes: > Hi Maxim, > > For reference [1]. > > On Sat, 09 Sep 2023 at 18:14, Maxim Cournoyer > wrote: > >> On a related note sometimes we have WIP kind of work that stays on our >> tracker with deeper questions / problems to solve, and I don't think >> it's fair for our reviewers to have these linger on for years on the >> tracker (they take a lot of time to get familiar with, and would then >> require quit more investment to be completed, sometimes with the >> original submitter no longer active in the discussion) -- I think for >> these situations it's fair to close it. An interested person can >> hopefully find these in the archives and resume work on it if they are >> so inclined. > > I am more or less agree. Especially for keeping the tracker > healthy. However, I am missing how “an interested person can hopefully > find these in the archives and resume work on it if they are so > inclined.“ > > Maybe we could have another usertags for tagging this case or another > tag. Else, it appears to me unpractical to find these in the archives. > Well, I personally do not even know how or where to start for finding > these. > > Exercise: find the patches that someone pinged their status from the 10 > Years of Guix event and another person then closed. :-) > > Found it? If no, > > Hint 1: someone was me. ;-) > > Found it? If no, > > Hint 2: another person was you. ;-) > > Found it? If no, > > Hint 3: it was about elasticsearch. > > https://issues.guix.gnu.org/search?query=elasticsearch > > Found it? If no, > > Hint 4: tag ’moreinfo’. > > Well, if just marked as done, then it appears to me unpractical to find > these in the archives. If you want to package elasticsearch, know it's not in the Guix as we speak, and see it as the first item when browsing https://issues.guix.gnu.org/search?query=elasticsearch, I have some hope that someone could find it. Not to mention search engines, which are also good at this. That said, I get your point. Perhaps a 'rejected' user tag would be useful? I doubt most users would know to use them though... but maybe if it's exposed via Mumi it could be helpful. -- Thanks, Maxim
Re: How can we decrease the cognitive overhead for contributors?
Hi Simon, Am Montag, dem 11.09.2023 um 12:36 +0200 schrieb Simon Tournier: > Today, the review and commit process have many steps, more or less > simple, not all sure!, well at the end, we have something complex. > > One trivial step is to apply the patch (or series) to your local > checkout and so many things can lead to some useless frictions. For > example, the patch does not apply because the base commit is not > provided, or because it is badly formatted, or because…. And one > ends to fix themself, e.g., probably using some manual trick just for > applying the patch. No value for the project. > > Yes, QA is currently helping for that specific example about applying > patches but the solution depends on why the patch does not apply. As a committer, I've already shared my tool for applying patches from mumi. It is currently broken due to yet another bug in mumi that I can't figure out the cause of, but assuming that gets fixed, it is indeed a trivial step with the added value that I can sign off the commit as I do (and signed commits gives the benefit of a trusted upgrade chain, as detailed in the introductory blog). Now granted, there is no benefit if you do this "checking out for review work" as a non-committer, because your signature, whether you apply it or not, holds no value for the project. However, there are issues one can spot simply "by looking" as well as others that do require actual research beyond building the package (such as "is this software actually GPL'd" for example). As for actually checking whether the package builds, we could move more of that to QA, but I'm not sure whether that'd be a benefit. For "patch does not apply", the forge solution is typically to send a notification to the issuer. Sure, we could do that, we could even do so automatically, but fixing one's own patch over and over against a running target when there's no hope it'll get merged (for other issues that are never pointed out, because you're coding for a bot) induces even more cognitive overhead. > Well, I am not saying that we should switch to PR model using > GitHub. :-) Let just recognize that our current workflow has many > drawbacks that make some frictions (steps with no value) for > reviewing. And PR model à la GitHub has many other drawbacks but at > least it reduces some frictions (technical steps with no value); > mainly because some people are paid for removing (or hide) these > useless friction. > > Don’t you agree that the review process implies many manual steps > that have no value for the project? I might actually disagree on the "many", but let's agree on "some" and move on from there. > Last, I agree that the main issue when speaking about the reviewing > process is not about tooling. The lack of a smooth technical > solution is just acting as a chemical developing solution in > photography, it increases the contrast and makes the poor image > apparent. Yeah, I don't get that metaphor. Cheers
Re: hard dependency on Git? (was bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts)
Re, On Mon, 11 Sep 2023 at 17:17, Simon Tournier wrote: > On Mon, 11 Sep 2023 at 16:23, Ludovic Courtès wrote: > >> Note that the patch series adds a hard dependency on Git. >> This is because the existing ‘git-fetch’ code depends on Git, >> which is itself motivated by the fact that Git supports >> shallow clones and libgit2/Guile-Git doesn’t. [...] > Personally, I do not have a strong opinion about the Big Plan™. I note > that the introduction of Git as a hard dependency is a slippery slope > considering the current state of libgit2. Here, it starts with “git > clone”, then “git gc” (unsupported by libgit2) is also in the pipes > (#65720 [1]). And after timing, I am almost sure that many operations > using Guile-Git will be slower than their plain Git counter-parts. And > we will start to parse the output of ’git’ plumbing commands. Slippery > slope for pushing Guile-Git out, no? For example, having plain Git command with shallow clone would allow to save resource when cloning the first time the Guix checkout cache. Compare, --8<---cut here---start->8--- $ git clone https://git.savannah.gnu.org/git/guix.git guix-full Cloning into 'guix-full'... remote: Counting objects: 696917, done. remote: Compressing objects: 100% (143179/143179), done. remote: Total 696917 (delta 552872), reused 696909 (delta 552867) Receiving objects: 100% (696917/696917), 347.14 MiB | 29.31 MiB/s, done. Resolving deltas: 100% (552872/552872), done. --8<---cut here---end--->8--- and, --8<---cut here---start->8--- $ git clone --shallow-since=2019-04-30 https://git.savannah.gnu.org/git/guix.git guix-oldest Cloning into 'guix-oldest'... remote: Counting objects: 426879, done. remote: Compressing objects: 100% (87246/87246), done. remote: Total 426879 (delta 342111), reused 423970 (delta 339518) Receiving objects: 100% (426879/426879), 259.75 MiB | 11.26 MiB/s, done. Resolving deltas: 100% (342111/342111), done. Checking connectivity: 426863, done. --8<---cut here---end--->8--- (Here, 2019-04-30 is the date that contains the %oldest-possible-commit "6298c3ffd9654d3231a6f25390b056483e8f407c" v1.0.) Well, ’shallow’ probably implies an overload on server side. But that is far less than the bits it saves: 87.39 MiB (= 347.14 - 259.75). It saves something like 25% to download, if I read correctly. I let you do some maths for the improvement you will get. :-) And there is no integration with Guile-Git, if I read correctly. The job is done by the procedure ’clone*’ from the module (guix git). This procedure call the Guile-Git procedure ’clone’ which could drop-in replaced by (invoke git-command "clone --shallow-since=2019-04-30" …). Cheers, simon
Re: hard dependency on Git? (was bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts)
On 2023-09-11 17:17:24 +0200, Simon Tournier wrote: > Hi, > > On Mon, 11 Sep 2023 at 16:23, Ludovic Courtès wrote: > > > Note that the patch series adds a hard dependency on Git. > > This is because the existing ‘git-fetch’ code depends on Git, > > which is itself motivated by the fact that Git supports > > shallow clones and libgit2/Guile-Git doesn’t. > > Going this path, I appears to me worth to revisit the proposal: > > RFC: libgit2 is slow/inefficient; switch to git command? > Maxim Cournoyer > Mon, 21 Nov 2022 21:21:02 -0500 > id:87cz9fpw4x@gmail.com > https://yhetil.org/guix/87cz9fpw4x@gmail.com > https://lists.gnu.org/archive/html/guix-devel/2022-11 > > I know it is not an option for now to parse the output of ’git’ commands > in order to keep the features of (guix git), (guix channels), etc. > > However, this discussion was mentioning an implementation of > clone/checkout in pure Racket supporting shallow checkout. Considering > the current level of integration, I thought the next Big Plan™ was to > gradually move bits of Guile-Git to being pure Scheme, maybe based on > the Racket implementation of ’clone’ as a starting point. > > Personally, I do not have a strong opinion about the Big Plan™. I note > that the introduction of Git as a hard dependency is a slippery slope > considering the current state of libgit2. Here, it starts with “git > clone”, then “git gc” (unsupported by libgit2) is also in the pipes > (#65720 [1]). And after timing, I am almost sure that many operations > using Guile-Git will be slower than their plain Git counter-parts. And > we will start to parse the output of ’git’ plumbing commands. If you don't mind me asking, why is that so problematic approach? Git's plumbing commands are intended to be used in scripts, so I am unsure what the problem is. I cannot recall a single instance when some tooling I have at home or wrote for work stopped working due to git changing. I guess there likely are such instances, but would be interested in examples if someone has a list. > Slippery slope for pushing Guile-Git out, no? Guile-git does not seem to be in the best place currently, when I was putting together the script I sent to #65720, I tried to use the info manual, and the best way to describe it is "incomplete". Some behaviors are... surprising. Arguably that is the impression I got based on one morning of trying to use it, so it is probably inaccurate description and lacks some details. > > And I do not speak about the closure. Is it possible to extract the > command ’git-clone’ from git-minimal? It would reduce the size, no? > > > Cheers, > simon > > 1: https://issues.guix.gnu.org/65720 > -- There are only two hard things in Computer Science: cache invalidation, naming things and off-by-one errors. signature.asc Description: PGP signature
Re: Cross compilation status
Am Montag, dem 11.09.2023 um 18:19 +0200 schrieb Simon Tournier: > Hi Mathieu, > > On Sun, 10 Sep 2023 at 11:14, Mathieu Othacehe > wrote: > > > Here is a status of cross-compilation in Guix. For cross- > > compilation to > > work, the build-system needs to support cross-compilation. > > > > The following build-systems explicitly refuse cross-compiling > > packages: > > > > haskell, agda, waf, chicken, rakudo, julia, python, emacs, rebar, > > cargo, ruby, renpy, dub, android-ndk, scons, dune, ant, pyproject, > > maven, asdf, r, ocaml, node > > > > [...] > > > --8<---cut here---start->8--- > > Total packages: 28076 > > > > clojure: 13 > > qt: 317 > > copy: 134 > > minetest-mod: 22 > > tree-sitter: 35 > > raw: 2 > > linux-module: 17 > > glib-or-gtk: 139 > > asdf/source: 794 > > go: 632 > > cmake: 1278 > > minify: 12 > > perl: 839 > > trivial: 250 > > guile: 49 > > elm: 30 > > font: 101 > > gnu: 4178 > > asdf/ecl: 753 > > asdf/sbcl: 814 > > meson: 504 > > mozilla: 2 > > texlive: 4150 > > Cross-compilation OK: 15065 > > > > ocaml: 61 > > haskell: 812 > > maven: 2 > > chicken: 12 > > node: 57 > > emacs: 1370 > > dune: 289 > > android-ndk: 12 > > waf: 21 > > julia: 300 > > pyproject: 433 > > r: 2441 > > cargo: 3535 > > ruby: 597 > > rebar: 22 > > scons: 15 > > rakudo: 22 > > agda: 6 > > ant: 559 > > python: 2488 > > Cross-compilation KO: 13054 > > --8<---cut here---end--->8--- > > The build-systems renpy, dub is listed in “refuse“ but not then in > Cross-compilation KO. Is it expected? On the renpy side, there currently are no packages to cross-compile. "The Question", which ships with renpy, is built as an output rather than with renpy-build-system and was only used as a proof of concept back then. If I do encounter a renpy-based game that I actually can get the sources for, I'll gladly package it. HTH
Re: [workflow] Automatically close bug report when a patch is committed
Hi Maxim, Maxim Cournoyer writes: [...] >> If there is enough consensus I volunteer to collect ideas and send a >> feature request to the mumi and/or Debbugs devels (if we need Debbugs >> patches I guess it will be a long term goal) > > I don't think any changes to Debbugs would be necessary. Mumi is > already able to parse mail headers -- parsing a git trailer should be > just as simple. You are right, I'll try to file some feature request for mumi. Thanks, Gio' -- Giovanni Biscuolo Xelera IT Infrastructures signature.asc Description: PGP signature
Re: [workflow] Triaging issues (was Automatically close bug report when a patch is committed)
Hi Simon Simon Tournier writes: [...] >> is enough, but (is:open and tag:patch,moreinfo) is better: [...] >> We could also add a feature to have "saved searches" in mumi web and CLI >> interfaces to help with this task. > > Well, the Mumi CLI, “guix shell mumi” and then “mumi search”, should do > act as “saved searches”. I don't understand; I mean for example having a configuration file where we can save searches, something like: --8<---cut here---start->8--- [patches-to-be-checked] "is:open and tag:patch,moreinfo" --8<---cut here---end--->8--- and then "mumi search --saved patches-to-be-checked" ...something like notmuch, I mean. > Although it does not work for me. Please help improving mumi with bug reports or patches if you have not already done > Note that Debian provides some BTS tools, as pointed here, Yes I saw your message, useful thanks; we should package it and maybe add that functions to mumi, one by one Thanks, Gio' -- Giovanni Biscuolo Xelera IT Infrastructures signature.asc Description: PGP signature
hard dependency on Git? (was bug#65866: [PATCH 0/8] Add built-in builder for Git checkouts)
Hi, On Mon, 11 Sep 2023 at 16:23, Ludovic Courtès wrote: > Note that the patch series adds a hard dependency on Git. > This is because the existing ‘git-fetch’ code depends on Git, > which is itself motivated by the fact that Git supports > shallow clones and libgit2/Guile-Git doesn’t. Going this path, I appears to me worth to revisit the proposal: RFC: libgit2 is slow/inefficient; switch to git command? Maxim Cournoyer Mon, 21 Nov 2022 21:21:02 -0500 id:87cz9fpw4x@gmail.com https://yhetil.org/guix/87cz9fpw4x@gmail.com https://lists.gnu.org/archive/html/guix-devel/2022-11 I know it is not an option for now to parse the output of ’git’ commands in order to keep the features of (guix git), (guix channels), etc. However, this discussion was mentioning an implementation of clone/checkout in pure Racket supporting shallow checkout. Considering the current level of integration, I thought the next Big Plan™ was to gradually move bits of Guile-Git to being pure Scheme, maybe based on the Racket implementation of ’clone’ as a starting point. Personally, I do not have a strong opinion about the Big Plan™. I note that the introduction of Git as a hard dependency is a slippery slope considering the current state of libgit2. Here, it starts with “git clone”, then “git gc” (unsupported by libgit2) is also in the pipes (#65720 [1]). And after timing, I am almost sure that many operations using Guile-Git will be slower than their plain Git counter-parts. And we will start to parse the output of ’git’ plumbing commands. Slippery slope for pushing Guile-Git out, no? And I do not speak about the closure. Is it possible to extract the command ’git-clone’ from git-minimal? It would reduce the size, no? Cheers, simon 1: https://issues.guix.gnu.org/65720
Re: Cross compilation status
Hi Mathieu, On Sun, 10 Sep 2023 at 11:14, Mathieu Othacehe wrote: > Here is a status of cross-compilation in Guix. For cross-compilation to > work, the build-system needs to support cross-compilation. > > The following build-systems explicitly refuse cross-compiling packages: > > haskell, agda, waf, chicken, rakudo, julia, python, emacs, rebar, cargo, > ruby, renpy, dub, android-ndk, scons, dune, ant, pyproject, maven, asdf, r, > ocaml, node > [...] > --8<---cut here---start->8--- > Total packages: 28076 > > clojure: 13 > qt: 317 > copy: 134 > minetest-mod: 22 > tree-sitter: 35 > raw: 2 > linux-module: 17 > glib-or-gtk: 139 > asdf/source: 794 > go: 632 > cmake: 1278 > minify: 12 > perl: 839 > trivial: 250 > guile: 49 > elm: 30 > font: 101 > gnu: 4178 > asdf/ecl: 753 > asdf/sbcl: 814 > meson: 504 > mozilla: 2 > texlive: 4150 > Cross-compilation OK: 15065 > > ocaml: 61 > haskell: 812 > maven: 2 > chicken: 12 > node: 57 > emacs: 1370 > dune: 289 > android-ndk: 12 > waf: 21 > julia: 300 > pyproject: 433 > r: 2441 > cargo: 3535 > ruby: 597 > rebar: 22 > scons: 15 > rakudo: 22 > agda: 6 > ant: 559 > python: 2488 > Cross-compilation KO: 13054 > --8<---cut here---end--->8--- The build-systems renpy, dub is listed in “refuse“ but not then in Cross-compilation KO. Is it expected? The build-system asdf is listed as refuse but appears in the list Cross-compilation OK. Is it expected? > Over the 28076 packages in Guix, 15065 are part of build-systems with > cross-compilation support and 13054 are part of build-systems without > cross-compilation support. > > Overall 46.5% of our packages will refuse to cross-compile with errors > such as: > > --8<---cut here---start->8--- > mathieu@meije ~$ guix build --target=aarch64-linux-gnu librsvg > guix build: error: gnu/packages/gnome.scm:3500:2: librsvg@2.54.5: build > system `cargo' does not support cross builds > --8<---cut here---end--->8--- > > I'd like to help converge towards the situation where all build-systems > support cross-compilation. Considering Julia, the “julia compiler” support of upstream [1] reads: + ARMv8 (64-bit): Tier 2 + ARMv7 (32-bit): Tier 3 + PowerPC (64-bit): Tier 3 + then stuff about CUDA, ROCM, oneAPI where: + Tier 2: Julia is guaranteed to build from source using the default build options, but may or may not pass all tests. Official binaries are available on a case-by-case basis. + Tier 3: Julia may or may not build. If it does, it is unlikely to pass tests. Binaries may be available in some cases. When they are, they should be considered experimental. Ongoing support is dependent on community efforts. But I guess, these are about native compilation. To my tiny knowledge on that topic, Julia supports some cross-compilation. However, Julia depends on heavy packages as ’suitesparse’ or ’openblas’ and I do not know if we already cross-compile them. Since Julia is used for scientific computations, I do not know if the effort is worth. 1: https://julialang.org/downloads/ > I have CC'ed members of the python, java, ruby, rust, r, haskell and > emacs teams. Any plans adding cross-compilation support to your > build-system, barriers to overcome? About Haskell, from experience, we are already not able to maintain an usable ecosystem for i686, but that’s another story. :-) Well, I do not know if cross-compilation is well-supported by the Haskell compiler GHC. Maybe some resources seems there: https://github.com/input-output-hk/haskell.nix About R, similarly as Julia, since they are mainly used for scientific computations, I do not know if supporting cross-compilation is worth our effort. Cheers, simon
Re: [workflow] Triaging issues (was Automatically close bug report when a patch is committed)
Hi, On Mon, 11 Sep 2023 at 09:37, Giovanni Biscuolo wrote: > A more useful mumi (web or CLI) query to search for duplicates would be: > > is:open tag:patch subject: [...] > is enough, but (is:open and tag:patch,moreinfo) is better: > > https://issues.guix.gnu.org/search?query=is%3Aopen+tag%3Apatch%2Cmoreinfo > > or even filtered if older than 5m, because "The bug will be closed if > the submitter doesn't provide more information in a reasonable (few > months) timeframe." [3] > > We could also add a feature to have "saved searches" in mumi web and CLI > interfaces to help with this task. Well, the Mumi CLI, “guix shell mumi” and then “mumi search”, should do act as “saved searches”. Although it does not work for me. Note that Debian provides some BTS tools, as pointed here, Debbugs CLI client (was Re: Mumi CLI client (was: How can we decrease the cognitive overhead for contributors?))) Simon Tournier Tue, 05 Sep 2023 10:58:50 +0200 id:86msy1uhbp@gmail.com https://yhetil.org/guix/86msy1uhbp@gmail.com https://lists.gnu.org/archive/html/guix-devel/2023-09 Cheers, simon
Re: [workflow] Automatically close bug report when a patch is committed
Hi,
Simon Tournier writes:
> Hi Maxim,
>
> Thanks for the explanations.
>
> On Mon, 11 Sept 2023 at 15:47, Maxim Cournoyer
> wrote:
>
>> > 2. How is Change-Id linked to #65280?
>>
>> Each patch submission ("issue") can have one or multiple commits. We'd
>> know for sure the series was merged (and thus can be closed) when the
>> set of 'Change-Id's its commits contains have all appeared in the master
>> branch. The mapping of Debbugs ID <-> set(change-ids) would need to be
>> established by an external tool such as Mumi (which I think is in a good
>> position to do so).
>
> I think it is not straightforward to maintain such mapping. Because
> Mumi needs to implement a way to extract patches; and there is many
> corner-cases.
It wouldn't need to cover *all* cover cases to be useful; even if it
only worked for 'git send-email' series, it'd still be useful.
> For instance, I am already using a pre-commit hook, how would it work
> in this case?
I didn't think about this case, but if it's common, we could have our
own default pre-commit hook include an escape hatch (e.g., run all
scripts under ~/.config/git/hooks/pre-commit.d) and document that, so
that users could still plug their very own pre-commit hooks.
--
Thanks,
Maxim
Re: [workflow] Automatically close bug report when a patch is committed
Hi Maxim,
Thanks for the explanations.
On Mon, 11 Sept 2023 at 15:47, Maxim Cournoyer
wrote:
> > 2. How is Change-Id linked to #65280?
>
> Each patch submission ("issue") can have one or multiple commits. We'd
> know for sure the series was merged (and thus can be closed) when the
> set of 'Change-Id's its commits contains have all appeared in the master
> branch. The mapping of Debbugs ID <-> set(change-ids) would need to be
> established by an external tool such as Mumi (which I think is in a good
> position to do so).
I think it is not straightforward to maintain such mapping. Because
Mumi needs to implement a way to extract patches; and there is many
corner-cases.
For instance, I am already using a pre-commit hook, how would it work
in this case?
Cheers,
simon
Re: [workflow] Automatically close bug report when a patch is committed
Hi Giovanni, Giovanni Biscuolo writes: > Hi Maxim, > > Maxim Cournoyer writes: > > [...] > >>> c. how do we get the issue number of a patch containing "Change-Id"? [1] >> >> We'd have to search through the currently opened patches issues; I >> assume using a tool like the 'mumi' command we already have could do >> that. > > It would be fantastic if we find a way for mumi to index (via xapian) > the "Change-Id", enabling us to provide a query like this: (is:open and > change-id:). I don'r know if this is doable by mumi alone or if it > needs Debbugs to be able to manage the new "Change-Id" attribute. > > If there is enough consensus I volunteer to collect ideas and send a > feature request to the mumi and/or Debbugs devels (if we need Debbugs > patches I guess it will be a long term goal) I don't think any changes to Debbugs would be necessary. Mumi is already able to parse mail headers -- parsing a git trailer should be just as simple. -- Thanks, Maxim
Re: [workflow] Automatically close bug report when a patch is committed
Hi Giovanni, Giovanni Biscuolo writes: > Hi! > > Liliana Marie Prikler writes: > >> Am Donnerstag, dem 07.09.2023 um 09:12 -0700 schrieb Vagrant Cascadian: >>> I am much more comfortable with the "Fixes" convention of: >>> >>> Fixes: https://issues.guix.gnu.org/NNN > > OK, I understand Vagrant's concerns: we need a _namespaced_ URI, but > there is no need that URI is the URL of **one** of our current web > interfaces, why not the other one? ;-) > IMO this is an implementation detail we can easily fix once we find a > consensus on introducing this requirement in the Guix guidelines on > committing. Agreed. I think a simple git trailer such as: --8<---cut here---start->8--- Fixes: bug#65738 (java-ts-mode tests) --8<---cut here---end--->8--- Would be nice, where bug#N is already known and shown as a URL in Emacs when the bug-reference minor mode is enabled (it's the case working with Magit), so bugs can be consulted easily without any context switching. -- Thanks, Maxim
Re: [workflow] Automatically close bug report when a patch is committed
Hi Simon,
Simon Tournier writes:
> Hi Maxim,
>
> On Sat, 09 Sep 2023 at 19:50, Maxim Cournoyer
> wrote:
>
--8<---cut here---start->8---
random=$({ git var GIT_COMMITTER_IDENT ; echo "$refhash" ; cat "$1"; } |
git hash-object --stdin)
--8<---cut here---end--->8---
>>>
>>> That seems like it would only work if the patch was identical, as
>>> opposed to a slightly rebased patch on top of newer patches on master?
>>>
>>> How can you correlate Change-Id to a patch in the tracker?
>>
>> The Change-Id stays the same unless you manually edit it out of your
>> commit message when amending / rebasing, so the commit hash may change
>> while the Change-Id stays the same. So you can rebase your feature
>> branch on master and share a v2, whose existing commits will have the
>> same Change-Ids (newly added commits would get their own Change-Id
>> trailer).
>
> I am sorry if I am slow but I do not understand.
>
> $ git var GIT_COMMITTER_IDENT
> Simon Tournier 1694428674 +0200
> $ git var GIT_COMMITTER_IDENT
> Simon Tournier 1694428800 +0200
>
> Therefore this Change-Id can be different for the same series, depending
> when I am locally committing. No?
>
> And sorry if I am slow but I am also missing your answer about “How can
> you correlate Change-Id to a patch in the tracker?”. How is this
> Change-Id correlated to the Debbugs number?
The commit hook job is simple: generate a unique ID. The magic is that
it only adds it *once* to a commit message; if it finds the 'Change-Id:'
line already present in it, it won't refresh its value or add a new one.
So it's immutable once added, preserved in the Git commit message
itself.
> Let take an example. :-) Assume Change-Id was used for your submission
> bug#65280 about Qt. It reads many patches and we have:
>
> 02/59
> 1717c8a233b7fda3a10aabc061168c71317f883e
> AuthorDate: Fri Aug 11 15:26:14 2023 -0400
>
> 59/59
> 0a77b869322274030c4c0e8315ddea802da44c92
> AuthorDate: Tue Aug 15 16:20:10 2023 -0400
>
> From my understanding,
>
> 1. GIT_COMMITTER_IDENT depends on time so these two commits would have
> a different Change-Id, no?
Correct. The commit hook adds a unique ID *per* commit.
> 2. How is Change-Id linked to #65280?
Each patch submission ("issue") can have one or multiple commits. We'd
know for sure the series was merged (and thus can be closed) when the
set of 'Change-Id's its commits contains have all appeared in the master
branch. The mapping of Debbugs ID <-> set(change-ids) would need to be
established by an external tool such as Mumi (which I think is in a good
position to do so).
I hope that helps!
--
Thanks,
Maxim
Re: How can we decrease the cognitive overhead for contributors?
Hi Liliana, Liliana Marie Prikler writes: [...] > For example, whenever people say that "forges would improve stuff", my > reply is (modulo phrasing) "yes, for the people who are already used to > forges". I just want to point out that actually Guix _do_have_ a forge. the software is Savane and it's hosted on savannah.gnu.org: https://savannah.gnu.org/projects/guix This is just to remind everyone that there are very different forges out there, and: All forges suck, _no one_ sucks less > Now, forges might indeed be familiar to many, What kind of forge? Savannah, GitHub, GitLab, SourceHut, Codeberg (you name it) Just to bring a simple, when talking about the "Pull Request" workflow to manage merges, they are not even interoperable (while git request-pull /is/) ...not to talk about issue management features, when present. [...] > Bear in mind, that contributing already has at least one degree of > complexity baked right into itself on the basis of being a feedback > loop. Wow. food for thought! Thanks! Gio' -- Giovanni Biscuolo Xelera IT Infrastructures signature.asc Description: PGP signature
Re: How can we decrease the cognitive overhead for contributors?
Hello, (I find it difficult to efficiently follow this thread and to keep up to date with reading it, so please forgive me if someone else already addressed my considerations) Simon Tournier writes: [...] > « For which contributors do we want to/can we decrease the cognitive > overhead? », so I read it as: do we discuss about someone who is already > playing guitar or someone who is knowing nothing about music. > > We already have the answer: we are speaking about someone who already > plays guitar (a skilled programmer). There are many ways to contribute to Guix: --8<---cut here---start->8--- - Project Management - Art - Documentation - Packages - Programming - System Administration - Test and Bug Reports - Translation --8<---cut here---end--->8--- (https://guix.gnu.org/en/contribute/) and just a few of them requires to be a skilled programmer :-) But you are absolutely right, we are talking about someone who already have: --8<---cut here---start->8--- (skill (or project-management user-interface-design graphical-design multimedia-design technical-documentation-writing guix-programming guile-programming program-debugging system-administration translation-of-tachnical-documents)) --8<---cut here---end--->8--- I'd also say that other "low level" skills are dependencies for some or all of the above mentioned skills, like: git-dvcs-usage, text-mua-usage As already mentioned, a conditio sine qua non (hard dependency) to contribute to Guix (as to as many many other international distributed projects) is to have "high level" skill named manage-communications-in-EN. Last but not least, a "meta skill" is that you accept to do all of this as a volunteer in a large group of volunteers, with very few /direct/ rewards - the most important one being to improve the best ever free software distro [1] - and many many issues to address... Quite a lot of skills to be able to contribute, I'd say. Furthermore, not a skill but another requirement not to be underestimated is you need a certain amount of time and unfortunately many people can only subtract that from their (often already scarce) free time. Probably we should find a way to /introduce/ old and new contributors to this concepts since I feel someway sometimes they are forgotten or underestimated. [...] > Somehow, now we have to discuss about specific task, task by task, and > propose how to improve. Survey is one next action for collecting > data. My 2 cents: surveys should be _carefully_ designed or the resulting data would be useless at best, misleading at worst [...] > The improvement had been the removal of the friction by switching to > some web interface. Now, the process is probably not easy for people > like me that are not used to web interface, although interacting with > web interface is a simpler task than configuring some tools for > editing translation files. There is a weblate CLI we should probably package in Guix: https://docs.weblate.org/en/latest/wlc.html I just hope that changing from a git-commit-based approach to a weblate-tool approach have helped find many more active translators: https://translate.fedoraproject.org/projects/guix/#information > we are far from the initial discussion. ;-) Sorry: OT... OT?!? :-O > I do not see “the practice of controlling access to information, > advanced levels of study, elite sections of society, etc“. Well, are > you French? ;-) Because I feel we are discussing unrelated points > emerging although we are agree on the core and we just detail tiny > variations of the same thing. :-) If you want I can add a little bit of Italian attitide at discussing in detail tiny variations of the same thing :-O... just joking, eh! ;-) [...] Ciao! Gio' [1] well, I'm biased :-D -- Giovanni Biscuolo Xelera IT Infrastructures signature.asc Description: PGP signature
Re: [workflow] Automatically close bug report when a patch is committed
Hi Maxim,
On Sat, 09 Sep 2023 at 19:50, Maxim Cournoyer wrote:
>>> --8<---cut here---start->8---
>>> random=$({ git var GIT_COMMITTER_IDENT ; echo "$refhash" ; cat "$1"; } |
>>> git hash-object --stdin)
>>> --8<---cut here---end--->8---
>>
>> That seems like it would only work if the patch was identical, as
>> opposed to a slightly rebased patch on top of newer patches on master?
>>
>> How can you correlate Change-Id to a patch in the tracker?
>
> The Change-Id stays the same unless you manually edit it out of your
> commit message when amending / rebasing, so the commit hash may change
> while the Change-Id stays the same. So you can rebase your feature
> branch on master and share a v2, whose existing commits will have the
> same Change-Ids (newly added commits would get their own Change-Id
> trailer).
I am sorry if I am slow but I do not understand.
--8<---cut here---start->8---
$ git var GIT_COMMITTER_IDENT
Simon Tournier 1694428674 +0200
$ git var GIT_COMMITTER_IDENT
Simon Tournier 1694428800 +0200
--8<---cut here---end--->8---
Therefore this Change-Id can be different for the same series, depending
when I am locally committing. No?
And sorry if I am slow but I am also missing your answer about “How can
you correlate Change-Id to a patch in the tracker?”. How is this
Change-Id correlated to the Debbugs number?
Let take an example. :-) Assume Change-Id was used for your submission
bug#65280 about Qt. It reads many patches and we have:
02/59
1717c8a233b7fda3a10aabc061168c71317f883e
AuthorDate: Fri Aug 11 15:26:14 2023 -0400
59/59
0a77b869322274030c4c0e8315ddea802da44c92
AuthorDate: Tue Aug 15 16:20:10 2023 -0400
>From my understanding,
1. GIT_COMMITTER_IDENT depends on time so these two commits would have
a different Change-Id, no?
2. How is Change-Id linked to #65280?
Cheers,
simon
Re: How can we decrease the cognitive overhead for contributors?
Hi Liliana, On Sun, 10 Sep 2023 at 00:20, Liliana Marie Prikler wrote: >> > > Must we force a single workflow on everyone, even if our track >> > > record in reviewing and merging doesn’t clearly show that our way >> > > is superior >> > >> > Again, define superior. >> >> No, I won’t. I think it’s obvious that our review process isn’t >> working *well*. So the argument that our current workflow allows for >> effective review is dubious. Not saying that you made that claim, >> just that it’s hard to convince others of adopting our ways when the >> results just aren’t great. > > What do you consider "the results" here? The rate at which patches are > merged? This is hardly an issue our project alone is fighting and I'm > not convinced that technology, more or less, will shift it in either > direction. That’s not about “result” here. That’s about “simple vs easy” or “complex can be easy” or etc. Similarly as submitting patches means that many steps are more or less simple, then the complete process can be considered as relatively complex. To be explicit, I do not speak about being “easy” which is subjective and instead I am speaking about some objective criteria (e.g., the number of steps even if they are trivial). Some part of that complexity has some value for the project and some other part has no value. The question is thus to identify and then remove (or hide) the complexity that has no value for the project. Today, the review and commit process have many steps, more or less simple, not all sure!, well at the end, we have something complex. One trivial step is to apply the patch (or series) to your local checkout and so many things can lead to some useless frictions. For example, the patch does not apply because the base commit is not provided, or because it is badly formatted, or because…. And one ends to fix themself, e.g., probably using some manual trick just for applying the patch. No value for the project. Yes, QA is currently helping for that specific example about applying patches but the solution depends on why the patch does not apply. Well, I am not saying that we should switch to PR model using GitHub. :-) Let just recognize that our current workflow has many drawbacks that make some frictions (steps with no value) for reviewing. And PR model à la GitHub has many other drawbacks but at least it reduces some frictions (technical steps with no value); mainly because some people are paid for removing (or hide) these useless friction. Don’t you agree that the review process implies many manual steps that have no value for the project? Last, I agree that the main issue when speaking about the reviewing process is not about tooling. The lack of a smooth technical solution is just acting as a chemical developing solution in photography, it increases the contrast and makes the poor image apparent. Cheers, simon
Closing submission incomplete since years?
Hi Maxim, For reference [1]. On Sat, 09 Sep 2023 at 18:14, Maxim Cournoyer wrote: > On a related note sometimes we have WIP kind of work that stays on our > tracker with deeper questions / problems to solve, and I don't think > it's fair for our reviewers to have these linger on for years on the > tracker (they take a lot of time to get familiar with, and would then > require quit more investment to be completed, sometimes with the > original submitter no longer active in the discussion) -- I think for > these situations it's fair to close it. An interested person can > hopefully find these in the archives and resume work on it if they are > so inclined. I am more or less agree. Especially for keeping the tracker healthy. However, I am missing how “an interested person can hopefully find these in the archives and resume work on it if they are so inclined.“ Maybe we could have another usertags for tagging this case or another tag. Else, it appears to me unpractical to find these in the archives. Well, I personally do not even know how or where to start for finding these. Exercise: find the patches that someone pinged their status from the 10 Years of Guix event and another person then closed. :-) Found it? If no, Hint 1: someone was me. ;-) Found it? If no, Hint 2: another person was you. ;-) Found it? If no, Hint 3: it was about elasticsearch. https://issues.guix.gnu.org/search?query=elasticsearch Found it? If no, Hint 4: tag ’moreinfo’. Well, if just marked as done, then it appears to me unpractical to find these in the archives. Cheers, simon 1: [bug#31444] 'guix health': a tool to report vulnerable packages Maxim Cournoyer Sat, 09 Sep 2023 18:14:13 -0400 id:871qf7xadm@gmail.com https://yhetil.org/guix/871qf7xadm@gmail.com https://issues.guix.gnu.org/msgid/871qf7xadm@gmail.com
Re: How can we decrease the cognitive overhead for contributors?
Hi Efraim, Efraim Flashner writes: [...] > On the other hand, if we do manage to automate writing of commit > messages, it makes one less thing for committers to manually fix before > pushing the commit. It would be lovely! It could also be done by a client-side git hook, provided in the Guix repo and automatically installed when following the instructions on the Guix manual (sorry I miss the pointer now) so that not only committers can benefit using that script but also contributors. As usual: patches wellcome! :-) Sorry I can't contribute with this task, I really don't know ho to program such a script. All I can do is suggesting to add a git commit message template (see message id:87y1hhdnzj@xelera.eu point 4. for details) Anyway, automation does't mean that the contributor/committer can ignore the commit message content conforms to the Guix guidelines: IMO a human supervising activity is always needed, be it done by contributors before submiting a patch or by a reviewer before committing. > The last couple of contributions I pushed had green checks on > qa.guix.gnu.org and I felt at a bit of a loss what to do while > checking it over. Sorry feel I don't fully understand what do you mean. I'm guessing here... AFAIU having a green light on QA means one of the build farms succesfully built the package, I guess this is a "gree check" on a "Committer check list" before committing: actually I can't find such check list but probably it can be "extrapolated" from the checklist documented for patch submissions: https://guix.gnu.org/en/manual/devel/en/html_node/Submitting-Patches.html So, if all the items in the check list are OK, the package sould be committed to the appropriate branch. Lastly, IMO if you are committer you can go on, if you are not committer you should notify a suitable committer that all is ready for commitment. Maybe if QA would send an email notification to the bug owner (every bug related to a patch set should have an owner) about thay "green light" it could be of some help with keeping track of what can be actually merged. > After checking that it was in fact green I double-checked the > commit message and then also some of the layout of the commit and the > package itself, and made sure it passed `guix lint`. More resources for > qa.guix.gnu.org would let us build more patches more quickly. I agree, QA is a critical resource in this phase of Guix evolution. More resources IMO also means documentation... and maybe more features? I feel like we should find a way to sponsor the work on QA. Happy hacking! Gio' -- Giovanni Biscuolo Xelera IT Infrastructures signature.asc Description: PGP signature
Re: [workflow] Automatically close bug report when a patch is committed
Hi! Liliana Marie Prikler writes: > Am Donnerstag, dem 07.09.2023 um 09:12 -0700 schrieb Vagrant Cascadian: >> I am much more comfortable with the "Fixes" convention of: >> >> Fixes: https://issues.guix.gnu.org/NNN OK, I understand Vagrant's concerns: we need a _namespaced_ URI, but there is no need that URI is the URL of **one** of our current web interfaces, why not the other one? ;-) IMO this is an implementation detail we can easily fix once we find a consensus on introducing this requirement in the Guix guidelines on committing. > I like the idea, but we should also consider the bugs.gnu.org address > here as well as the convention of putting it into angular brackets. In > fact, I might even prefer it if the convention was > Fixes: Bug description > where bug description is a (possibly empty) name for the bug such as > "Emacs hangs when I press a key" or something. I agree that an (optional) bug description might be useful (could also be automatically added by some cool etc/committer.scm funcion?) I propose: Fixes: [optional bug description] where namespace is the package name, in our case "guix"; fo example: Fixes: Emacs hangs when I press a key WDYT? > As for when to send it, remember that we already send a bunch of mails > to guix-comm...@gnu.org as our commit hook? I think it shouldn't be > too hard to search for the fixes line and send it to debbugs control. Do you please know whare to get that scripts, just to have a quick look and understand how we could eventually add a function for automatic bug closing? Thanks! Gio' -- Giovanni Biscuolo Xelera IT Infrastructures signature.asc Description: PGP signature
Re: [workflow] Automatically close bug report when a patch is committed
Hi Maxim, Maxim Cournoyer writes: [...] >> c. how do we get the issue number of a patch containing "Change-Id"? [1] > > We'd have to search through the currently opened patches issues; I > assume using a tool like the 'mumi' command we already have could do > that. It would be fantastic if we find a way for mumi to index (via xapian) the "Change-Id", enabling us to provide a query like this: (is:open and change-id:). I don'r know if this is doable by mumi alone or if it needs Debbugs to be able to manage the new "Change-Id" attribute. If there is enough consensus I volunteer to collect ideas and send a feature request to the mumi and/or Debbugs devels (if we need Debbugs patches I guess it will be a long term goal) >> [1] right now how do we get the issue number of a committed patch? > > There's no direct mapping. You have to search by subject name. IMO a link like this is _very_ useful in helping bug tracking management (and also for historians :-) ) and we should find a way to manage that. Also for this reason (other that possibly automated bug closing) IMO committers should add a "Fixes: #" signature (or what is called in Git) to each commit with an associeted bug report. WDYT? -- Giovanni Biscuolo Xelera IT Infrastructures signature.asc Description: PGP signature
Re: [workflow] Triaging issues (was Automatically close bug report when a patch is committed)
Hello Vagrant, sorry for the delay with this reply (maybe meanwhile someone else have already done all or some of my considerations) Vagrant Cascadian writes: [...] >> The point is that triaging is a (boring) activity that Someone™ should >> perform, sooner or later (as Vagrant did with the bug reports mentioned >> above). > > I was definitely in the mood for "let me get some relatively easy, if > boring things done" the other day. :) boring but very much useful: thank you! ...and thanks to all the other people that sometimes are doing this job! >> Obviously a contrubutor could (should) also be a self-triager, if she >> wants help making the review process more efficient. > > FWIW, I think I used the search: > > https://issues.guix.gnu.org/search?query=is%3Aopen+tag%3Apatch > > Sorted by date, and searched for the phrase "update" in the subject, as > old bugs proposing to update to a newer version were and are, well, > likely candidates for culling. :) > Other tooling that could further help with the process would be > valuable. A more useful mumi (web or CLI) query to search for duplicates would be: is:open tag:patch subject: With a caveat: searched subject term could not be in the search results, since it gives bug titles, but in the /thread/ of a found bug, see bug #65809 for details WDYT? [...] > There were sometimes some things that were not merged, and I made > judgement calls on weather they still needed to be, such as a tweak to > the packaging that was maybe only needed to get an older version to > build, but the newer version was building correctly. I see and for this reason I feel triaging (for example to merge or close bugs) cannot be automated, it needs judgement [...] >> When sending a series of patches, it’s best to send a Git “cover letter” >> first, to give reviewers an overview of the patch series. >> >> --8<---cut here---end--->8--- >> >> Missing a cover letter means that triaging is harder. > > Indeed. Retitling can be used to help after the fact, at least. Right: retitling could be one of the (possibly early) triaging actions >> The issue title is from the first patch (gnu: rofi: Update to 1.7.5.) >> and IMO is somewhat confusing because the title is what appears in >> search results (Mumi, Debbugs, Emacs Debbugs). > > I retitled several bugs as well, to at least update the current status, > as a few had some patches merged or superseded, but there were > outstanding issues not yet merged. I recently learned not to confuse "subject" with "bug title"... I feel this is something contributors should always consider >> If the contrubutor sent a cover letter with subject "gnu: Update rofi >> and Add rofi-wayland (inherinting)", possibly with a little bit of >> explanation in the message body, the (now undone) early triaging would >> have been easier. > > Yes, cover letters would help significantly with triaging these more > complicated cases. > > That said, sometimes over the course of review, it becomes clear that > additional changes are needed, and it would be helpful to retitle the > bug in these cases. > > I saw at least one bug which started out as "Add XYZ" and evolved into > "Update ZXY, ... Add ABC, Add XYZ" and it would not have made sense to > make them separate patch submissions. yes: retitling is an important (re)triaging activity >> How do we solve such bug management class of problems? WDYT? >> >>> One improvement I can think of here is that QA should highlight that >>> some of the changes in each of those patch series can be found in >>> another patch series. >> >> ...and tag both bugs as related on Debbugs? > > Not sure how to mark related, Uh I see [1]: AFAIU the only available links between bugs are [2]: 1. merge bugnumber bugnumber ... --8<---cut here---start->8--- Merges two or more bug reports. When reports are merged opening, closing, marking or unmarking as forwarded and reassigning any of the bugs to a new package will have an identical effect on all of the merged reports. [...] Merger is like equality: it is reflexive, transitive and symmetric. Merging reports causes a note to appear on each report's logs; on the WWW pages this includes links to the other bugs. Merged reports are all expired simultaneously, and only when all of the reports each separately meet the criteria for expiry. --8<---cut here---end--->8--- 2. block|unblock bugnumber by|with bug [ bug ... ] --8<---cut here---start->8--- Use to note that one bug blocks another bug from being fixed. The first listed bug is the one being blocked, and it is followed by the bug or bugs that are blocking it. Use unblock to unblock a bug. --8<---cut here---end--->8--- Unfortunately "merge" is not good for two or more bugs containing "duplicated" patches. Could "Usertags" pseudo-header be used
