Re: Do substitutes download slowly for you? / Speeding up substitute delivery/mirrors
Hi, On Thu, May 25, 2023, at 5:04 PM, Felix Lechner wrote: > Hi Philip, > > On Thu, May 25, 2023 at 1:49 PM Philip McGrath > wrote: >> >> Here are results from Florida, US: >> >> 2023-05-25 16:28:07 (14.9 MB/s) >> 2023-05-25 16:28:53 (52.9 MB/s) >> 2023-05-25 16:29:37 (10.5 MB/s) > > Wow, those are the best Guix speeds I have seen yet, perhaps aside > from folks connected to DFN. [1] Would you please share some details > about your ISP? From a peering perspective, I think the most > interesting part is your ASN. [2] Thanks! > This was on a residential Comcast connection which seems to be AS7922. I tried to avoid any local slowdowns by doing this with a wired connection at a time when there was little other activity on the machine or the LAN and downloading to a tmpfs. Anecdotally, the speeds for servers *other* than bordeaux-us-east-mirror were faster in this trial than feels typical here. The results I reported at https://lists.gnu.org/archive/html/guix-devel/2022-07/msg00320.html (e.g. 8.17MB/s for bordeaux.guix.gnu.org) might be more usual. Subjectively, my impression of the non-mirrored substitute servers is that often the speeds is ok, but when they are slow it is very, very slow. Philip
Re: Do substitutes download slowly for you? / Speeding up substitute delivery/mirrors
Hi Kaelyn & everyone, On Fri, May 26, 2023 at 11:29 AM Kaelyn wrote: > > Below are my results In order to make it easier for Christopher—and all of us, really—to think about the results of the poll, I put together a Framacalc spreadsheet: https://lite.framacalc.org/pm6mbqubqr-a16h Perhaps you could briefly look over your data for errors I may have made. You should be able to make corrections on your own. You are also welcome to contact me privately if the spreadsheet is not available to you, for example from China. Kind regards & please have a good weekend (a long one in the US)! Felix
Re: Do substitutes download slowly for you? / Speeding up substitute delivery/mirrors
> France: wget > https://bordeaux.guix.gnu.org/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 > US: wget > https://bordeaux-us-east-mirror.cbaines.net/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 > Singapore: wget > https://bordeaux-singapore-mirror.cbaines.net/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 > > > So please share the output from wget and if you're comfortable doing so, > the rough real world location of where the computer doing the > downloading is. I'm in Salem, Oregon with Comcast/Xfinity home internet, and while the speeds I get from sites varies a lot, the mirrors are kind of all equally slow for me compared to what I usually see. The Singapore mirror was a little slower on average than France or US east, but all three showed fluctuations throughout the transfer ranging roughly from 650KB/s to 3+MB/s. Below are my results, along with a 4th result downloading a kernel tarball from kernel.org which shows a more typical speed for me. Cheers, Kaelyn $ wget -O/dev/null https://bordeaux.guix.gnu.org/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 --2023-05-26 11:13:25-- https://bordeaux.guix.gnu.org/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 Resolving bordeaux.guix.gnu.org (bordeaux.guix.gnu.org)... 185.233.100.56, 2a0c:e300::58 Connecting to bordeaux.guix.gnu.org (bordeaux.guix.gnu.org)|185.233.100.56|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 208615205 (199M) [text/plain] Saving to: ‘/dev/null’ /dev/null100%[>] 198.95M 1.07MB/sin 1m 57s 2023-05-26 11:15:23 (1.70 MB/s) - ‘/dev/null’ saved [208615205/208615205] $ wget -O/dev/null https://bordeaux-us-east-mirror.cbaines.net/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 --2023-05-26 11:16:09-- https://bordeaux-us-east-mirror.cbaines.net/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 Resolving bordeaux-us-east-mirror.cbaines.net (bordeaux-us-east-mirror.cbaines.net)... 5.161.49.48 Connecting to bordeaux-us-east-mirror.cbaines.net (bordeaux-us-east-mirror.cbaines.net)|5.161.49.48|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 208615205 (199M) [text/plain] Saving to: ‘/dev/null’ /dev/null100%[>] 198.95M 1.65MB/sin 2m 2s 2023-05-26 11:18:12 (1.62 MB/s) - ‘/dev/null’ saved [208615205/208615205] $ wget -O/dev/null https://bordeaux-singapore-mirror.cbaines.net/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 --2023-05-26 11:18:30-- https://bordeaux-singapore-mirror.cbaines.net/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 Resolving bordeaux-singapore-mirror.cbaines.net (bordeaux-singapore-mirror.cbaines.net)... 64.176.80.78, 2401:c080:1400:71df:5400:4ff:fe73:757d Connecting to bordeaux-singapore-mirror.cbaines.net (bordeaux-singapore-mirror.cbaines.net)|64.176.80.78|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 208615205 (199M) [text/plain] Saving to: ‘/dev/null’ /dev/null100%[>] 198.95M 760KB/sin 2m 26s 2023-05-26 11:20:57 (1.37 MB/s) - ‘/dev/null’ saved [208615205/208615205] $ wget -O/dev/null https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.1.30.tar.xz --2023-05-26 11:21:17-- https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.1.30.tar.xz Resolving cdn.kernel.org (cdn.kernel.org)... 151.101.1.176, 151.101.65.176, 151.101.193.176, ... Connecting to cdn.kernel.org (cdn.kernel.org)|151.101.1.176|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 134914908 (129M) [application/x-xz] Saving to: ‘/dev/null’ /dev/null100%[>] 128.66M 10.2MB/sin 12s 2023-05-26 11:21:34 (10.5 MB/s) - ‘/dev/null’ saved [134914908/134914908]
Re: Should commit signing always be required for local work? [was Re: bug#63261: Recent changes to git config cause errors for non-committers]
On Fri, May 19, 2023 at 11:34:35AM +0200, Josselin Poiret wrote: > I'm curious Leo, in general (not Guix because we have a pre-push hook), > how do you make sure you always publish signed commits? I don't want to > put unsigned commits anywhere except locally, but it feels like I might > just forget to sign them before pushing. In general, I don't rigorously sign Git commits for projects that aren't Guix. You could set "gpgsign = true" in '~/.gitconfig'. I do sign commits sometimes for non-Guix projects, but without a code-authentication system like Guix's, I don't perceive a strong reason to always sign commits. There is *some* reason to always sign commits, which is to provide an unambiguous statement of authorship / provenance. But, it doesn't seem like most projects have a mechanism with which to derive value from the signatures. Also, it doesn't seem like there is much demand for this, in general. Git itself offers nothing, so each project has to design their own solution. I doubt many projects would consider that effort to be worthwhile. Instead they rely on the access controls of their centralized repo, typically Github, and Github's security seems fine in practice. I think that Guix is pushing the state of the art here. signature.asc Description: PGP signature
Re: Do substitutes download slowly for you? / Speeding up substitute delivery/mirrors
Another (faster) test from a different machine with Guix System 020184f, same place and network (60 Mbps): El 25/05/23 a las 15:49, Luis Felipe escribió: I'm in Colombia, Aburrá Valley, Guix System 5eb1d1b, home network, single user (as far as I know). France: #+begin_example wget https://bordeaux.guix.gnu.org/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 --2023-05-25 10:13:45-- https://bordeaux.guix.gnu.org/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 Resolviendo bordeaux.guix.gnu.org (bordeaux.guix.gnu.org)... 2a0c:e300::58, 185.233.100.56 Conectando con bordeaux.guix.gnu.org (bordeaux.guix.gnu.org)[2a0c:e300::58]:443... conectado. Petición HTTP enviada, esperando respuesta... 200 OK Longitud: 208615205 (199M) [text/plain] Grabando a: «078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0» 078vr3r8mn3yrwzwxw6 100%[===>] 198,95M 985KB/s en 7m 25s 2023-05-25 10:21:12 (457 KB/s) - «078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0» guardado [208615205/208615205] #+end_example US: #+begin_example wget https://bordeaux-us-east-mirror.cbaines.net/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 --2023-05-25 10:23:22-- https://bordeaux-us-east-mirror.cbaines.net/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 Resolving bordeaux-us-east-mirror.cbaines.net (bordeaux-us-east-mirror.cbaines.net)... 5.161.49.48 Connecting to bordeaux-us-east-mirror.cbaines.net (bordeaux-us-east-mirror.cbaines.net)|5.161.49.48|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 208615205 (199M) [text/plain] Saving to: '078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0.1' 078vr3r8mn3yrwzwxw6 100%[===>] 198.95M 835KB/s in 3m 25s 2023-05-25 10:26:49 (993 KB/s) - '078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0.1' saved [208615205/208615205] #+end_example Singapore: #+begin_example wget https://bordeaux-singapore-mirror.cbaines.net/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 --2023-05-25 10:27:36-- https://bordeaux-singapore-mirror.cbaines.net/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 Resolving bordeaux-singapore-mirror.cbaines.net (bordeaux-singapore-mirror.cbaines.net)... 2401:c080:1400:71df:5400:4ff:fe73:757d, 64.176.80.78 Connecting to bordeaux-singapore-mirror.cbaines.net (bordeaux-singapore-mirror.cbaines.net)|2401:c080:1400:71df:5400:4ff:fe73:757d|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 208615205 (199M) [text/plain] Saving to: '078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0.2' 078vr3r8mn3yrwzwxw6 100%[===>] 198.95M 1.26MB/s in 3m 30s 2023-05-25 10:31:08 (970 KB/s) - '078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0.2' saved [208615205/208615205] #+end_example France: #+begin_example wget https://bordeaux.guix.gnu.org/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 --2023-05-26 12:14:42-- https://bordeaux.guix.gnu.org/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 Resolviendo bordeaux.guix.gnu.org (bordeaux.guix.gnu.org)... 2a0c:e300::58, 185.233.100.56 Conectando con bordeaux.guix.gnu.org (bordeaux.guix.gnu.org)[2a0c:e300::58]:443... conectado. Petición HTTP enviada, esperando respuesta... 200 OK Longitud: 208615205 (199M) [text/plain] Grabando a: «078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0» 078vr3r8mn3yrwzwxw6 100%[===>] 198,95M 2,29MB/s en 92s 2023-05-26 12:16:15 (2,17 MB/s) - «078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0» guardado [208615205/208615205] #+end_example US: #+begin_example wget https://bordeaux-us-east-mirror.cbaines.net/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 --2023-05-26 12:19:45-- https://bordeaux-us-east-mirror.cbaines.net/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 Resolviendo bordeaux-us-east-mirror.cbaines.net (bordeaux-us-east-mirror.cbaines.net)... 5.161.49.48 Conectando con bordeaux-us-east-mirror.cbaines.net (bordeaux-us-east-mirror.cbaines.net)[5.161.49.48]:443... conectado. Petición HTTP enviada, esperando respuesta... 200 OK Longitud: 208615205 (199M) [text/plain] Grabando a: «078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0.1» 078vr3r8mn3yrwzwxw6 100%[===>] 198,95M 3,41MB/s en 82s 2023-05-26 12:21:08 (2,44 MB/s) - «078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0.1» guardado [208615205/208615205] #+end_example Singapore: #+begin_example wget https://bordeaux-singapore-mirror.cbaines.net/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 --2023-05-26 12:22:16-- https://bordeaux-singapore-mirror.cbaines.net/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 Resolviendo bordeaux-singapore-mirror.cbaines.net (bordeaux-singapore-mirror.cbaines.net)... 2401:c080:1400:71df:5400:4ff:fe73:757d, 64.176.80.78 Conectando con bordeaux-singapore-mirror.cbaines.net
Re: How many bytes do we add (closure of guix) when adding one new package?
Hello! Thanks for the detailed analysis! Simon Tournier skribis: > Conclusions: > > 1. the addition of one package leads to an increase of ~ 12 KiB > > 2. the core of Guix is about ~ 62 MiB > > 3. doubling the number of packages is doubling the size to download at > “guix pull” time. I agree that .go files are quite big (.scm files as well, but we’ve improved information density somewhat by removing input labels :-)). The size of .go files went down when we switch to the baseline compiler (aka. -O1): https://lists.gnu.org/archive/html/guix-devel/2020-06/msg00071.html That thread has ideas of things to do to further reduce .go size. Download size has to be treated separately though. For example, ‘git pull’ doesn’t redownload all of the repo or directory, and it uses compression heavily. Thus, a few hundred bytes of additional .scm text translate in less than that. As for the rest, download size can be reduced for example by choosing a content-address transport, like something based on ERIS. I think we must look precisely at what we want to optimize—on-disk size, or bandwidth requirement, in particular—and look at the whole solution space. My 2¢! Ludo’.
Re: Do substitutes download slowly for you? / Speeding up substitute delivery/mirrors
Andreas Enge skribis: > I am in France with a 100Mb/s FTTH link, and download is fast from all > of the mirrors. > FR 5,59MB/sin 39s Only? I’m in Bordeaux :-) and from my workplace’s fast network I get more than 10x more: --8<---cut here---start->8--- $ wget -O/dev/null https://bordeaux.guix.gnu.org/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 --2023-05-26 17:59:38-- https://bordeaux.guix.gnu.org/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 Resolving bordeaux.guix.gnu.org (bordeaux.guix.gnu.org)... 185.233.100.56, 2a0c:e300::58 Connecting to bordeaux.guix.gnu.org (bordeaux.guix.gnu.org)|185.233.100.56|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 208615205 (199M) [text/plain] Saving to: ‘/dev/null’ /dev/null100%[>] 198.95M 75.0MB/sin 2.7s 2023-05-26 17:59:41 (75.0 MB/s) - ‘/dev/null’ saved [208615205/208615205] --8<---cut here---end--->8--- The others are below but not too bad: --8<---cut here---start->8--- $ wget -O/dev/null https://bordeaux-us-east-mirror.cbaines.net/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 --2023-05-26 18:01:03-- https://bordeaux-us-east-mirror.cbaines.net/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 Resolving bordeaux-us-east-mirror.cbaines.net (bordeaux-us-east-mirror.cbaines.net)... 5.161.49.48 Connecting to bordeaux-us-east-mirror.cbaines.net (bordeaux-us-east-mirror.cbaines.net)|5.161.49.48|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 208615205 (199M) [text/plain] Saving to: ‘/dev/null’ /dev/null100%[>] 198.95M 17.6MB/sin 12s 2023-05-26 18:01:16 (16.1 MB/s) - ‘/dev/null’ saved [208615205/208615205] $ wget -O /dev/null https://bordeaux-singapore-mirror.cbaines.net/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 --2023-05-26 18:02:14-- https://bordeaux-singapore-mirror.cbaines.net/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 Resolving bordeaux-singapore-mirror.cbaines.net (bordeaux-singapore-mirror.cbaines.net)... 2401:c080:1400:71df:5400:4ff:fe73:757d, 64.176.80.78 Connecting to bordeaux-singapore-mirror.cbaines.net (bordeaux-singapore-mirror.cbaines.net)|2401:c080:1400:71df:5400:4ff:fe73:757d|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 208615205 (199M) [text/plain] Saving to: ‘/dev/null’ /dev/null100%[>] 198.95M 6.63MB/sin 34s 2023-05-26 18:02:49 (5.88 MB/s) - ‘/dev/null’ saved [208615205/208615205] --8<---cut here---end--->8--- Substitution is often not network-bound though: https://guix.gnu.org/en/blog/2021/getting-bytes-to-disk-more-quickly/ Ludo’.
Re: Format specification issue in the translations...
Hi, Sebastian Rasmussen skribis: > I'm involved in the translation of Shepherd to Swedish, and > poedit complains that the format specifications for the singular > and plural forms of one of the strings are not the same: > https://git.savannah.gnu.org/cgit/shepherd.git/tree/modules/shepherd/service.scm#n1107 This one is definitely a bug. I’ve now fixed it: https://git.savannah.gnu.org/cgit/shepherd.git/commit/?id=300fa7cf5c09b6022432e5450bcb12aadad41335 In the meantime you can do the same as Florian. Thanks for bringing it up! Ludo’.
Re: Transformations Shell Syntax
Hello! "jgart" skribis: > Uses specified commit hash: > > guix build emacs-ement@8b56efa9387262514daf63151d41c9e111e79567 > > Uses specified commit hash (short): > > guix build emacs-ement@8b56efa > > Uses latest upstream release: > > guix build emacs-ement@latest > > Uses upstream version 0.8.2 if not packaged: > > guix build emacs-ement@0.8.2 > > Uses the latest commit in the wip/find-room branch: > > guix build emacs-ement@wip/find-room I sympathize with the will to get a more compact way to express transformations. Right now, command-line tools parse package specs by calling ‘specification->package+output’. There are no restrictions on version fields: “8b56efa9387262514daf63151d41c9e111e79567” and “latest” are perfectly valid version fields. Thus, if the syntax above was implemented, we’d introduce ambiguity. Consequently, rather than overload “@”, I believe another syntax would need to be found. Thanks, Ludo’.
Re: Guidelines for pre-trained ML model weight binaries (Was re: Where should we put machine learning model parameters?)
Hello, Simon Tournier skribis: > On sam., 13 mai 2023 at 12:13, 宋文武 wrote: > >> Hello, zamfofex submited a package 'lc0', Leela Chess Zero” (a chess >> engine) with ML model, also it turn out that we already had 'stockfish' >> a similiar one with pre-trained model packaged. Does we reached a >> conclusion (so lc0 can also be accepted)? Or should we remove 'stockfish'? > > Well, I do not know if we have reached a conclusion. From my point of > view, both can be included *if* their licenses are compatible with Free > Software – included the weights (pre-trained model) as licensed data. We discussed it in 2019: https://issues.guix.gnu.org/36071 This LWN article on the debate that then took place in Debian is insightful: https://lwn.net/Articles/760142/ To me, there is no doubt that neural networks are a threat to user autonomy: hard to train by yourself without very expensive hardware, next to impossible without proprietary software, plus you need that huge amount of data available to begin with. As a project, we don’t have guidelines about this though. I don’t know if we can come up with general guidelines or if we should, at least as a start, look at things on a case-by-case basis. Ludo’.
Re: Patch graphviz in emacs-guix
"jgart" skribis: > Hi should emacs-guix be patched and the graphviz program add as an input to > emacs-guix? > > https://github.com/alezost/guix.el/blob/c9aef52121b458297e70bb50f49f7276b4a8d759/elisp/guix-external.el#L51 > > If I try to run `guix-package-graph` I get the following error: > > guix-dot-arguments: Couldn’t find ’dot’. > Set ‘guix-dot-program’ to a proper value Hi! I guess we could do that in the package definition of ‘emacs-guix’. Thanks, Ludo’.
Re: Tooling for branch workflows
Hello, Josselin Poiret skribis: > Maybe it would be a good time to create an infrastructure/CI team, and > set some clear goals for it? I think it might motivate people (me > included) to get more involved in GBC development if there is something > to build towards. Good idea! > Also, I kind of agree with Andreas about the bus factor of Cuirass and > GBC, and wonder whether it makes sense to duplicate development effort > across both. It seems to me that Cuirass is working properly as a > generic CI tool, but it's not super well adapted to Guix, something > which the GBC seems to be much better at. Is replacing Cuirass by GBC > on berlin for Guix CI something that is a future goal, or is it > understood that Cuirass will be running there for the foreseeable > future? Cuirass started as a replacement for Hydra, which is what Nix has been using, so it’s very similar. What’s nice is that it’s relatively easy to set it up to get a bunch of packages built continuously, which is the major use case for people out there (beyond the project’s own build farms). Because the Coordinator itself does nothing but build derivations, one has to set up some other tool such as the Data Service for that use case, which may be more difficult (I’m probably biased because I’ve only set up Cuirass instances). OTOH, the modularity that the Coordinator/Data Service approach offers is appealing to me. It can make it more approachable: one can hack the Coordinator without worrying about the Data Service. Anyhow, my suggestion would be to make our infrastructure more robust. I had to hot-fix Cuirass last week and it was cumbersome because the test suite only covers basic functionality. I believe the Coordinator lacks a test suite, which means it may be harder to modify it (harder to tell if you broke something) and to get confidence. So my personal wish list :-) would be improved testing and documentation. A longer-term wish: using Spritely Goblins and a modular actor architecture for those services so we can interact with them through a rich programming interface and distribute appropriate “capabilities” to designated developers (a (re)build capability, an “evaluate branch” capability, etc.). Ludo’.
Re: [PATCH glibc] Stop checking if MiG supports retcode.
Hey guix people! The Hurd developers having a 64 bit Hurd that can run
/bin/sh. The below are some tips for how to set up such a thing if you
were so inclined. The Debian people are providing 64-bit hurd
applications here for now:
https://people.debian.org/~sthibault/tmp/hurd-amd64
Flávio Cruz writes:
> Hi Sergey
>
> On Fri, May 19, 2023 at 4:02 AM Sergey Bugaev wrote:
>
> Hi,
>
> On Fri, May 19, 2023 at 9:43 AM Flávio Cruz
> wrote:
> > I have made changes so that it does daily builds and I'm able to boot
> small programs. However, I haven't had the time to boot programs built
> against Glibc. How do you package and boot the static binaries using a
> ramdisk? I've been reading the other threads about the Guix/rumpkernel
> so I might be able to piece something together and try it this weekend.
>
> You just put the entirety of the root filesystem (containing /usr,
> /bin, /lib, /hurd, and so on) as an ext2 image into a *file* that you
> place onto the actual drive (a CD disk in my case), and then you ask
> GRUB to load the file from the drive into memory, tell gnumach to make
> a ramdisk device out of it (you'll need to apply [0]), and tell ext2fs
> to use that device. Here's the relevant piece of my grub config
> script:
>
> [0]:
>
> https://salsa.debian.org/hurd-team/gnumach/-/blob/master/debian/patches/50_initrd.patch
>
>
> multiboot /boot/gnumach console=com0
> module /boot/initrd.ext2 initrd.ext2 '$(ramdisk-create)'
> module /sbin/ext2fs.static ext2fs
> --multiboot-command-line='${kernel-command-line}' --readonly
> --host-priv-port='${host-port}' --device-master-port='${device-port}'
> --exec-server-task='${exec-task}' --kernel-task='${kernel-task}' -T
> device rd0 '$(fs-task=task-create)' '$(prompt-task-resume)'
> module /lib/ld.so.1 ld.so.1 /hurd/exec
> --device-master-port='${device-port}' '$(exec-task=task-create)'
> boot
>
> (I should probably change it to not hardcode 'rd0', but whatever).
> Note that /boot/gnumach, /boot/initrd.ext2, /sbin/ext2fs.static, and
> /lib/ld.so.1 are all paths inside the CD image (those are going to be
> loaded by GRUB), and /boot/initrd.ext2 is the ext2 filesystem image
> containing the actual Hurd root. /hurd/exec however is already a path
> inside the fs image -- this is where ld.so (not grub) is going to load
> the exec server from. The only static binary here is ext2fs.static,
> the rest are all dynamically linked.
>
> Then in /libexec/console-run (inside the filesystem image), I have
> written the following:
>
> #! /bin/sh
>
> settrans -ac /dev/mach-console /hurd/streamio console
> exec <>/dev/mach-console >&0 2>&0
> echo Hello from /bin/sh!
> exec /bin/sh -i
>
> (If you're going to do the same, don't forget to create the
> /dev/mach-console node beforehand, since the fs is read-only.) I also
> had to patch streamio a little to do the \r -> \n conversion like
> glibc already does in devstream:
>
> diff --git a/trans/streamio.c b/trans/streamio.c
> index 272a002c..0af1aea3 100644
> --- a/trans/streamio.c
> +++ b/trans/streamio.c
> @@ -500,6 +500,9 @@ trivfs_S_io_read (struct trivfs_protid *cred,
>cred->po->openmodes & O_NONBLOCK);
> pthread_mutex_unlock (_lock);
> *data_len = data_size;
> + for (size_t i = 0; i < data_size; i++)
> +if ((*data)[i] == '\r')
> + (*data)[i] = '\n';
> return err;
> }
>
> (maybe I should also add echoing of input characters in the same way,
> which is also what glibc's devstream does -- otherwise currently I
> don't see what I'm typing on the console).
>
> Make sure to use the very latest glibc (Samuel has already pushed all
> of my patches upstream!) + the BRK_START hack.
>
> Thanks for the instructions. I was able to make it work and pushed my
> changes to Github.
>
> For people that might want to try out the new port using
> https://github.com/flavioc/cross-hurd,
> the following will download the packages and build a disk image with the ram
> disk:
>
> $ export CPU=x86_64
> $ bash download.sh && bash bootstrap.sh && bash compile.sh && bash
> create-initrd.sh
>
> Then, to run qemu:
>
> $ bash start-qemu-debug.sh
>
> Sergey
>
--
Joshua Branson
Sent from the Hurd
Re: Do substitutes download slowly for you? / Speeding up substitute delivery/mirrors
Hi Chris, Christopher Baines writes: [...] > France:wget > https://bordeaux.guix.gnu.org/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 > US:wget > https://bordeaux-us-east-mirror.cbaines.net/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 > Singapore: wget > https://bordeaux-singapore-mirror.cbaines.net/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 Here's the data, from an office connected to fiber in Montréal, Canada: Bordeaux (France): --8<---cut here---start->8--- $ wget https://bordeaux.guix.gnu.org/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 --2023-05-26 08:42:56-- https://bordeaux.guix.gnu.org/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 Resolving bordeaux.guix.gnu.org (bordeaux.guix.gnu.org)... 2a0c:e300::58, 185.233.100.56 Connecting to bordeaux.guix.gnu.org (bordeaux.guix.gnu.org)|2a0c:e300::58|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 208615205 (199M) [text/plain] Saving to: ‘078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0’ 078vr3r8mn3yrwzwxw64hmcysh 100%[==>] 198.95M 15.4MB/sin 14s 2023-05-26 08:43:11 (14.3 MB/s) - ‘078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0’ saved [208615205/208615205] --8<---cut here---end--->8--- US East: --8<---cut here---start->8--- $ wget https://bordeaux-us-east-mirror.cbaines.net/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 --2023-05-26 08:43:18-- https://bordeaux-us-east-mirror.cbaines.net/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 Resolving bordeaux-us-east-mirror.cbaines.net (bordeaux-us-east-mirror.cbaines.net)... 5.161.49.48 Connecting to bordeaux-us-east-mirror.cbaines.net (bordeaux-us-east-mirror.cbaines.net)|5.161.49.48|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 208615205 (199M) [text/plain] Saving to: ‘078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0.1’ 078vr3r8mn3yrwzwxw64hmcysh 100%[==>] 198.95M 33.7MB/sin 6.4s 2023-05-26 08:43:24 (31.0 MB/s) - ‘078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0.1’ saved [208615205/208615205] --8<---cut here---end--->8--- Singapore: --8<---cut here---start->8--- $ wget https://bordeaux-singapore-mirror.cbaines.net/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 --2023-05-26 08:43:31-- https://bordeaux-singapore-mirror.cbaines.net/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0 Resolving bordeaux-singapore-mirror.cbaines.net (bordeaux-singapore-mirror.cbaines.net)... 2401:c080:1400:71df:5400:4ff:fe73:757d, 64.176.80.78 Connecting to bordeaux-singapore-mirror.cbaines.net (bordeaux-singapore-mirror.cbaines.net)|2401:c080:1400:71df:5400:4ff:fe73:757d|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 208615205 (199M) [text/plain] Saving to: ‘078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0.2’ 078vr3r8mn3yrwzwxw64hmcysh 100%[==>] 198.95M 10.1MB/sin 30s 2023-05-26 08:44:02 (6.72 MB/s) - ‘078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0.2’ saved [208615205/208615205] --8<---cut here---end--->8--- The fastest is logically US East, which is twice as fast as the current Bordeaux France link for my location. -- Thanks, Maxim
Re: Do substitutes download slowly for you? / Speeding up substitute delivery/mirrors
Christopher Baines writes: > While I did stop running a mirror in Singapore, it's now back and from > the discussion on IRC today [3] there was some anecdotal evidence that > this helps with fetching substitutes from China. Yes, only the Singapore IPv4 mirror is usable for me in China. [berdeaux] wget 'https://bordeaux.guix.gnu.org/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0' Connecting to bordeaux.guix.gnu.org (bordeaux.guix.gnu.org)|2a0c:e300::58|:443... connected. .. 159.69K 8.27KB/seta 6h 50m wget -4 'https://bordeaux.guix.gnu.org/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0' Connecting to bordeaux.guix.gnu.org (bordeaux.guix.gnu.org)|185.233.100.56|:443... connected. .. 31.69K 5.60KB/seta 10h 6m ^C [us-east-mirror] wget 'https://bordeaux-us-east-mirror.cbaines.net/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0' Connecting to bordeaux-us-east-mirror.cbaines.net (bordeaux-us-east-mirror.cbaines.net)|5.161.49.48|:443... connected. Saving to: ‘078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0.2’ .. 639.69K 26.8KB/seta 1h 57m ^C [singapore-mirror] wget 'https://bordeaux-singapore-mirror.cbaines.net/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0' Connecting to bordeaux-singapore-mirror.cbaines.net (bordeaux-singapore-mirror.cbaines.net)|2401:c080:1400:71df:5400:4ff:fe73:757d|:443... connected. .. 415.69K 22.1KB/seta 2h 9m ^C wget -4 'https://bordeaux-singapore-mirror.cbaines.net/nar/lzip/078vr3r8mn3yrwzwxw64hmcyshic9p3q-stellarium-0.21.0' Connecting to bordeaux-singapore-mirror.cbaines.net (bordeaux-singapore-mirror.cbaines.net)|64.176.80.78|:443... connected. .. 14.59M 199KB/seta 11m 22s
Re: How Can We Make G-Expressions Even More Interactive At The REPL?
"(" writes:
> "jgart" writes:
>> Hi Guixers,
>>
>> How can we make g-expressions even more interactive at the REPL?
>>
>> For example, thing to explore is what can we print instead of
>> '(*approximate*)?
>
> It's physically impossible to print anything other than a placeholder
> there without actually building any derivations, which is the thing
> GEXP->APPROXIMATE-SEXP is supposed to /avoid/ :)
>
>> WDYT if we had even more convenience meta-commands for interactive
>> development with g-expressions?
>
> You'd need to give some concrete examples; what can't you do now that
> you think could be a good REPL command?
>
> -- (
I would very much welcome if the REPL commands (like ,build) were actual
first class functions, because right now it's quite a hassle to write
Guile scripts that depend on the output of a build.
Motivation: I'm trying to hack together a helper script that detects
generated OPAM dependency lists.
Re: Do substitutes download slowly for you? / Speeding up substitute delivery/mirrors
Hello, Am Thu, May 25, 2023 at 02:52:24PM +0100 schrieb Christopher Baines: > So please share the output from wget and if you're comfortable doing so, > the rough real world location of where the computer doing the > downloading is. I am in France with a 100Mb/s FTTH link, and download is fast from all of the mirrors. FR 5,59MB/sin 39s US 4,98MB/sin 53s SG 3,56MB/sin 73s The ordering looks consistent to me, but I am still surprised how good the connection is to Singapore! Andreas
Re: How Can We Make G-Expressions Even More Interactive At The REPL?
"jgart" writes: > Hi Guixers, > > How can we make g-expressions even more interactive at the REPL? > > For example, thing to explore is what can we print instead of > '(*approximate*)? It's physically impossible to print anything other than a placeholder there without actually building any derivations, which is the thing GEXP->APPROXIMATE-SEXP is supposed to /avoid/ :) > WDYT if we had even more convenience meta-commands for interactive > development with g-expressions? You'd need to give some concrete examples; what can't you do now that you think could be a good REPL command? -- (
