On +2021-07-06 15:28:43 -0300, Nathan Benedetto Proença wrote: > Bengt Richter <b...@bokr.com> writes: > > > Hi Nathan, > > Nice writeup! > > Thank you! > > > On +2021-07-05 11:03:46 -0300, Nathan Benedetto Proença wrote: > >> Hello! > >> > >> I am trying to upgrade the package texlive, first for me, but hopefully > >> for a patch, and I have a question regarding Guix policies. > >> > >> As you can see on > >> > >> > >> https://git.savannah.gnu.org/cgit/guix.git/tree/guix/build-system/texlive.scm?id=f7692617b624a01615cf412773c4dad9a2deeb68 > >> > >> the file guix/build-system/texlive.scm exposes two variables: > >> > >> (define %texlive-tag "texlive-2019.3") > >> (define %texlive-revision 51265) > >> > >> These variables are used throughout gnu/packages/tex.scm, as you can see > >> on > >> > >> > >> https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/tex.scm?id=f7692617b624a01615cf412773c4dad9a2deeb68 > >> > >> An example is the following code: > >> > >> (define hyph-utf8-scripts > >> (origin > >> (method svn-fetch) > >> (uri (texlive-ref "generic" "hyph-utf8")) > >> (file-name (string-append "hyph-utf8-scripts-" > >> (number->string %texlive-revision) > >> "-checkout")) > >> (sha256 > >> (base32 > >> "0lk7shx768sxvgr85y8bnmmnj8x4bbkgpxrz3z8jp8avi33prw83")))) > >> > >> Grep tells me there are 290+ occurrences of `%texlive-revision`. > >> What is the purpose of these variables? > >> > >> You see, they give me the impression that Guix is really concerned about > >> upgrading *all* of texlive at once. > >> These variables tell me I should go to the file texlive.scm and bump the > >> tag and revision, and then handle all the broken hashes. > >> > >> Hence, it seems to me that any attempt to upgrade the texlive package > >> would have to be done in a separate branch, which would only be merged > >> into master when all the packages are upgraded. > >> > >> Is this the case? > >> And if so, why? > >> > >> I have the impression that if such "monolithic" upgrade is not a goal, > >> and "partial" our "per-package" upgrades are desirable, there may be > >> better solutions. > >> > >> For example, we could add keyword arguments to texlive-ref and > >> texlive-origin, so the code above becomes something like this > >> > >> (define hyph-utf8-scripts > >> (origin > >> (method svn-fetch) > >> (uri (texlive-ref "generic" "hyph-utf8" > >> #:texlive-tag "texlive-2019.3" > >> #:texlive-revision 51265)) > >> (file-name "hyph-utf8-scripts-51625-checkout") > >> (sha256 > >> (base32 > >> "0lk7shx768sxvgr85y8bnmmnj8x4bbkgpxrz3z8jp8avi33prw83")))) > >> > >> This would work right now, and we could eventually remove every use of > >> %texlive-revision and %texlive-tag, so they become implementation > >> details of the build-system texlive.scm; a fallback version. > >> And further down the road we may even decide to remove this fallback, > >> and make developers be explicit about their tags and revisions; this > >> could amount to a refactor which makes the keyword arguments into > >> required arguments, for example. > >> > >> I also like the second version of the code because the hash already > >> pinpoints the tag and revision: both texlive-ref and texlive-origin use > >> these variables to find the correct files to download. > >> This just makes this dependency explicit. > >> > >> In any case, as this may be a choice between shipping stable and > >> up-to-date packages, and as I am new to contributing to Guix, I found > >> fitting to ask. > >> > >> Thanks in advance! > >> Nathan > >> > > > > I am wondering about guaranteeing generic behaviour by > > guaranteeing program source and toolchain source hash > > equivalences vs ignoring sources and guaranteeing end > > results by testing results. > > It seems to me that you are talking about my email regarding using > hashing in Scheme refactoring, right? > This one: > > https://lists.gnu.org/archive/html/guix-devel/2021-07/msg00023.html > > I will assume this is the case, even though I we can actually simply > talk about what you wrote. > > First thoughts: I think that what we really want to guarantee is > "correctness", and that both hashes or tests are mere proxies for this. > > I see them as useful in distinct moments of package development. > > For example, lets say that I want to partially upgrade TeXLive (which I > am already convinced is not a good idea, TBH, but serves as an > example). > To be able to do so, I may think that I want to refactor some function > to pinpoint the version I will download. > I can then make a single commit which updates the signature and all the > call sites of the function. > I can then actually upgrade the package I care about in a second commit, > and do some "proper testing" in it, like producing some pdf with latex. > > My interest in checking hashes in the first commit is to get some > confidence that this step, which may change hundred of places, did not > introduce a new bug, so I can focus any debugging on the hopefully more > local change made in the second commit. > > > I.e., if you want to print the sum of x and y passed as > > strings to a program, output as a string to stdout, it > > doesn't matter (other than optimization and debuggability) > > what language the program was written in, so long as it was > > compiled into a form that execve and co can launch and the > > end result is the same. > > > > As part of testing, maybe strace could be used to generate > > some kind of canonical kernel transaction trace that could > > be used to compare behaviours for equivalency of executing > > different-language programs? > > > > This would be a radical change in the approach to > > reproducibility, maybe dynamically selecting from a > > whitelist of trusted/tested substitutable executables with > > hash names in /gnu but not necessarily (though not > > excluding) binaries produced with guix source guarantees. > > > > Seems like guix is turing-complete enough to provide this > > kind of substitutable foreign functions already, so might > > this be a way to avoid mass recompilations? > > > > Or is this already available, but not so much used? > > > > I am not sure where to contibute thoughts like these, where > > they would be of interest rather than distracting. (Pls > > excuse the noise, if that's what this is to you). > > I am not sure I follow, although I do not regard this as noise. > > You are saying that one could use strace as a check that some program > behaves the same as it did before, similar to the use that I suggested > for hashing, right? > > I do not understand how this would be used, and how this avoids mass > recompilations. > I mean, to have binaries to inspect with strace I must build the code > anyway, right? > Am I missing something? >
I had in mind expanding on and polishing something like $ strace -s 72 ./bash-hello-strace |& grep ^write write(1, "Hello strace!\n", 14Hello strace! $ strace -s 72 ./guile-hello-strace.scm |& grep ^write write(1, "Hello strace!\n", 14Hello strace! Where $ cat -n ./guile-hello-strace.scm |gxsnip --8<---------------cut here---------------start------------->8--- 1 #! /usr/bin/guile -s 2 !# 3 (display "Hello strace!\n") --8<---------------cut here---------------end--------------->8--- and $ cat -n ./bash-hello-strace |gxsnip --8<---------------cut here---------------start------------->8--- 1 #! /usr/bin/bash 2 echo 'Hello strace!' --8<---------------cut here---------------end--------------->8--- I had in mind expanding on and polishing something like $ strace -s 72 ./bash-hello-strace |& grep ^write write(1, "Hello strace!\n", 14Hello strace! $ strace -s 72 ./guile-hello-strace.scm |& grep ^write write(1, "Hello strace!\n", 14Hello strace! Where $ cat -n ./guile-hello-strace.scm |gxsnip --8<---------------cut here---------------start------------->8--- 1 #! /usr/bin/guile -s 2 !# 3 (display "Hello strace!\n") --8<---------------cut here---------------end--------------->8--- and $ cat -n ./bash-hello-strace |gxsnip --8<---------------cut here---------------start------------->8--- 1 #! /usr/bin/bash 2 echo 'Hello strace!' --8<---------------cut here---------------end--------------->8--- I had in mind expanding on and polishing something like $ strace ./guile-hello-strace.scm |& grep ^write write(1, "Hello strace!\n", 14Hello strace! $ strace ./bash-hello-strace |& grep ^write write(1, "Hello strace!\n", 14Hello strace! Where $ cat -n ./guile-hello-strace.scm |gxsnip --8<---------------cut here---------------start------------->8--- 1 #! /usr/bin/guile -s 2 !# 3 (display "Hello strace!\n") --8<---------------cut here---------------end--------------->8--- and $ cat -n ./bash-hello-strace |gxsnip --8<---------------cut here---------------start------------->8--- 1 #! /usr/bin/bash 2 echo 'Hello strace!' --8<---------------cut here---------------end--------------->8--- Thus one could write something like this, compiling only it in whatever environment, so that updates to the tool chain which might otherwise have to be recompiled can be ignored. -- Regards Bengt Richter