shepherd: failing test: should `herd stop` stop a respawning process?
i have a daemon process that quits soon after it starts (when it has some issue with its configurations). then i usually fix the config, and do a guix system reconfigure. but i have noticed from the logs that this process often remains in a resawn loop, even if i herd stop and herd disable it after the reconfigure (i.e. a shepherd service upgrade). i have attached a respawn2.sh that when put under tests/ reproduces the issue in a shepherd checkout (see the TODO notes): $ guix shell $ make check TESTS="tests/respawn2.sh" what's wrong? - is it my expectation that herd stop should stop the respawning loop? - do i have a bug in my test.sh? - is this a shepherd bug? if so, then shall i finish up this test case as a proper patch for shepherd? -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “In all history there is no war which was not hatched by the governments, the governments alone, independent of the interests of the people, to whom war is always pernicious even when successful.” — Leo Tolstoy (1828–1910), 'On Patriotism' (1894) respawn2.sh Description: application/shellscript
Re: status of guixrus
> If your client is confused by an Internationalized Domain Name (IDN), > you can use Punycode to access it, in this case « whereis.みんな ». which sure looks like some phishing attempt... but to increase the confusion, something in the sw stack has undone your conversion: i see 3 blocks above here in chromium+protonmail web client reply window (but not when your email is displayed). -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “If a man knows more than others, he becomes lonely.” — Carl Jung (1875–1961)
Re: Updating rclone and non-free dependencies
> The go-team branch contains changes to importer which makes recursive > import much easier. i have some extensive changes to the go importer that is on my TODO to update and re-submit: https://issues.guix.gnu.org/55242 i made these commits when i was importing some projects with some 100+ dependencies in their transitive closures. could you please say a few words about the plans/status of the go-team branch? and if i get to working on this again, then i guess i should rebase it on the go-team branch, right? -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “If you suffer it is because of you, if you feel blissful it is because of you. Nobody else is responsible – only you and you alone. You are your hell and your heaven too.” — Osho (1931–1990)
Re: Reproducer for failing shepherd startup
> Okay, then there is still another problem. I am unable to reconfigure > my systems via 'guix deploy' when a timer is enabled already. I am > trying to add a second timer. sadly, i cannot test your reproducer without substantial work. my patches that clean up error handling in shepherd are from before Ludo added timers; i.e. it's once again bitrotten. contributing to guix feels like an uphill battle. i stopped rebasing my shepherd commits. abandoning them does feel like an enormous waste, but i'm cutting my losses at this point. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Individualism is not a kind of opposition to community. It is only the insistence that we choose those communities.” — Peter Jaworski
shepherd: PKG_PROG_PKG_CONFIG is m4_require'd but not m4_defun'd
i've pulled and reconfigured into the recently merged core-updates changes, and then i tried to compile shepherd from a git checkout of the devel branch. $ ./configure [...] ./configure: line 7268: PKG_PROG_PKG_CONFIG: command not found configure: error: pkg-config is missing, please install it $ pkg-config --version 0.29.2 then i tried to autoconf it: $ autoconf configure.ac:52: warning: PKG_PROG_PKG_CONFIG is m4_require'd but not m4_defun'd aclocal.m4:49: GUILE_PKG is expanded from... configure.ac:52: the top level the referenced line is this: GUILE_PKG([3.0]) i.e. guile seems to be involved somehow. any ideas? -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “The very possibility of conflict over a resource renders it scarce, giving rise to the need for ethical rules to govern its use. Thus, the fundamental social and ethical function of property rights is to prevent interpersonal conflict over scarce resources.” — Stephan Kinsella (1965–), 'Against Intellectual Property' (2008)
Re: Error in bootloader configuration
> Attached is my current system config that I'm trying to do `guix system > reconfigure system.scm` with. But I get this error: > > https://paste.debian.net/1328778 admittedly, this is not a very useful error message. but looking at where it comes from, it suggests that you have something strange among your services. and indeed, this looks fishy: (set-xorg-configuration (xorg-configuration (keyboard-layout keyboard-layout))) remove that and try again. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “It is easy to be conspicuously 'compassionate' if others are being forced to pay the cost.” — Murray N. Rothbard (1926–1995)
Re: Simplify contribution by using user-friendly bug tracker and git interface
> I'm apparently smart enough to install Guix as my daily driver distro, > learn Emacs with Paredit and hack together packages, and then an > email-based system filters me out... i often feel that getting some of my (non-controversial) contributions merged is an equivalent effort compared to creating the contribution itself. that should bother any reasonable engineer. and if someone feels the urge to dismiss it as just another inexperienced guix user, i already have non-trivial fixes in shepherd, and i'm coding for 30+ years, and most of it in opensource contexts, just not the GNU flavour. and yet, i still remember the effort it took to send my first few patches to guix, and it's still a source of frustration even now that i already learned the necessary incantations. a lot of it feels like jumping through hoops. such things always serve as "elaborate filters"; that part is inevitable. the only question is what gets filtered out and what gets through. and what gets filtered out here is mostly unobservable, while it has the power to make or break a project. i'm writing this with two minds: as a very enthusiastic user, and as a reluctant contributor. the fact that this topic comes up repeatedly shows that i'm probably not alone with this split mind. the enthusiasm makes some of the frustraton observable (as opposed to silently walking away), which should be seen as a blessing here. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “The most urgent necessity is, not that the State should teach, but that it should allow education. All monopolies are detestable, but the worst of all is the monopoly of education.” — Frédéric Bastiat (1801–1850), 'What Is Money?'
Re: Request for assistance maintaining LibreWolf
> We should try to come up with a solution that alleviates the burden on > the maintainers. Given how often this issue arises, what if we try, as > a collective, to suggest new mechanisms that would improve the > situation? IMO one thing that would help is to somehow losen the current very tight coupling (the guix monorepo). a high level of centralization always translates to a bottleneck, and guix has reached a point where its centralized structure is seriously hindering it and frustrating away contributors. i.e. it's much less risk to delegate the management of a hypothethical python channel to someone trusted than to give commit access to every contributor who wants to help maintaining the python related packages. so, i think a lot of packages should be in channels. probably everything that is not essential for a minimally functional system that can bootstrap itself. part of python could be in the main guix repo, but whatever is not tightly needed could go into a channel with its own access control management. some channels would have loser standards (god forbid some wouldn't frustrate the contributors with the ChangeLog format in the commit messages), and the users could decide whom they trust with what. and the guix maintainers could decide which channels to list on the official web page, with what comments/warnings. but this is easier said than done, because formally expressing the dependencies between these channels is not a trivial task (think of e.g. keeping `guix time-machine` working in an environment where a patch in the official guix repo can break a package in some random channel, which is then fixed there in a new commit, etc). and even if doable, it would probably introduce extra code complexity. but i'm sure there were countless discussions about the pros and cons of the monorepo, and i wish such a topic was captured in a wiki where the main arguments, obstacles, and ideas were collected for people like me who are late to the party. but then guix doesn't really have an official wiki either. (and no, an afterthought in the libreplanet wiki doesn't count. and if you have an urge to argue, then just look at its current contents...) so we're pretty much stuck with the flat mailing list archive and IRC logs. hrm, a tangential: maybe digesting the guix mailing list and IRC archive will be my first actual use-case for an LLM... modern problems require modern solutions. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Millions of people never analyze themselves. Mentally they are mechanical products of the factory of their environment, preoccupied with breakfast, lunch, and dinner, working and sleeping, and going here and there to be entertained. They don't know what or why they are seeking, nor why they never realize complete happiness and lasting satisfaction. By evading self-analysis, people go on being robots, conditioned by their environment. True self-analysis is the greatest art of progress.” — Paramahansa Yogananda (1893–1952)
Re: Cookbook recipe from "The Repository as a Channel" section does not work for Guix with properly configured GUILE_LOAD_PATH
you might have stumbled on this issue: (current-filename) is #f when guix pull'ing https://issues.guix.gnu.org/55464 the discussion contains analysis and some things to try, and also a working solution that i'm currently using in my channel at: https://github.com/attila-lendvai/guix-crypto (see the `read-hashes-file` macro) the extra twist is that it works fine while working on the code from e.g. emacs + geiser, but when you package it into a commit and try to `guix pull` the same code from the channel then it fails, and IIRC in some non-obvious way. HTH, -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Him that I love, I wish to be free – even from me.” — Anne Morrow Lindbergh (1906–2001)
Re: Reducing "You found a bug" reports
> Just a quick side note that some members in our community (not I) are > offended by the word "bug" to describe software defects. welcome to the internet of billions of people, where the cost of expressing one's offence doesn't cost anything anymore, and there are dozens of offended people even for the most innocent of statements. and with the advent of LLMs, it can soon become a kind of a security hole for intellectual DoS attacks -- if not already. not sure how to keep the SNR high, but the communities will need to adapt. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Knowledge will forever govern ignorance: And a people who mean to be their own Governors, must arm themselves with the power which knowledge gives.” — James Madison (1751–1836), 'Epilogue: Securing the Republic' (1822)
Re: `guix system image` on a local, dirty checkout
> > ./pre-inst-env guix system image --system=armhf-linux [...] > > Why using armhf instead of aarch64 for the BPI-R4 ? because i'm not there yet. this is my very first such endeavor. i'm merely trying to reproduce something that i've found in a mail archive: https://lists.gnu.org/archive/html/guix-devel/2018-12/msg00514.html sorry about the confusion! -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “The rule is perfect: in all matters of opinion our adversaries are insane.” — Mark Twain (1835–1910)
`guix system image` on a local, dirty checkout
dear Guix, i'm trying to generate a guix image from my modified guix checkout. this is a baby step towards my ultimate goal, to add BPI-R4 support. ./pre-inst-env guix system image --system=armhf-linux -e '(begin (use-modules (gnu system) (gnu bootloader) (gnu bootloader u-boot) (gnu system install)) (operating-system (inherit installation-os) (bootloader (bootloader-configuration (bootloader u-boot-bootloader) (target #f)' [...] Updating channel 'guix' from Git repository at '/home/alendvai/workspace/guix/guix/', branch 'attila'... Authenticating channel 'guix', commits 9edb3f6 to becd9ff (205 new commits)... ▕▊ ▏guix system: error: commit 53074836dbe2066ef082e01e5f22d1295847a5b3 not signed by an authorized key: 69DA 8D74 F179 7AD6 7806 EE06 FEFA 9FE5 5CF6 E3CD the commit mentioned is the commit where i add my key to the .guix-authorizations (and this is the commit that i use as channel intro commit). is there a way to make `guix system image` work from a guix checkout that contains unauthorized commits? or is there an equivalent of channel intro commits in this context? or something else? -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- There certainly are laws, but human words cannot make one.
Re: Reproducer for failing shepherd startup
> (un?)fortunately, i can't reproduce it on a recent guix and shepherd. i forgot to add that it's fixed by a guix commit: services: shepherd: Failure to load a service does not prevent booting. https://git.savannah.gnu.org/cgit/guix.git/commit/?id=cca25a67693bb68a1884a081b415a43fad1e8641 -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Lisp […] made me aware that software could be close to executable mathematics.” — L. Peter Deutsch
Re: Reproducer for failing shepherd startup
> > I still don't know how to fix the issue properly, but at least I can > > reconfigure my system now <3 > > > I have a reproducer! In the code below, please change "sunday" to "0", > together with this line in your "services": (un?)fortunately, i can't reproduce it on a recent guix and shepherd. both the devel branch and my branch (which includes extended error handling) works as expected. they simply skip registering this service and log the error: Jul 2 14:59:05 localhost vmunix: [2.936007] shepherd[1]: Exception caught while loading '/gnu/store/h6c15frnkx7arkm2bna20js3v90880bh-shepherd-mdadm-resync.go': #<&message message: "calendar-event: 0: invalid day of week"> -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “For followers of most ideologies (openly religious or not), toleration is a concession of defeat. For libertarians, it is victory itself.” — François-René Rideau
[PATCH shepherd] service: Factor out SEND-TO-REGISTRY.
From: Attila Lendvai Also assert that (CURRENT-REGISTRY-CHANNEL) is valid. * modules/shepherd/service.scm (with-service-registry): New function. (stop-service), (fold-services), (service-name-count), (lookup-service), (respawn-service), (register-services), (unregister-services): Use it. --- modules/shepherd/service.scm | 22 +++--- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/modules/shepherd/service.scm b/modules/shepherd/service.scm index e1c602c..b5f3e23 100644 --- a/modules/shepherd/service.scm +++ b/modules/shepherd/service.scm @@ -1030,8 +1030,7 @@ in a list." (report-exception 'stop service key args)) (when (transient-service? service) - (put-message (current-registry-channel) - `(unregister ,(list service))) + (send-to-registry `(unregister ,(list service))) (local-output (l10n "Transient service ~a unregistered.") (service-canonical-name service))) @@ -1290,6 +1289,10 @@ task is one that should never fail)." (parameterize ((current-registry-channel (spawn-service-registry))) (thunk))) +(define (send-to-registry message) + (assert (current-registry-channel)) + (put-message (current-registry-channel) message)) + (define-syntax-rule (with-service-registry exp ...) "Spawn a new service monitor and evaluate @var{exp}... within that dynamic extent. This allows @var{exp}... and their callees to send requests to delegate @@ -2286,8 +2289,7 @@ This must be paired with @code{make-systemd-destructor}." (lambda (proc init) "Apply PROC to the registered services to build a result, and return that result. Works in a manner akin to `fold' from SRFI-1." - (put-message (current-registry-channel) - `(service-list ,reply)) + (send-to-registry `(service-list ,reply)) (fold (match-lambda* (((name . service) result) (if (eq? name (service-canonical-name service)) @@ -2311,15 +2313,14 @@ returned in unspecified." (define (service-name-count) "Return the number of currently-registered service names." (let ((reply (make-channel))) -(put-message (current-registry-channel) - `(service-name-count ,reply)) +(send-to-registry `(service-name-count ,reply)) (get-message reply))) (define lookup-service (let ((reply (make-channel))) (lambda (name) "Return the service that provides @var{name}, @code{#f} if there is none." - (put-message (current-registry-channel) `(lookup ,name ,reply)) + (send-to-registry `(lookup ,name ,reply)) (get-message reply (define (lookup-services name) @@ -2631,7 +2632,7 @@ then disable it." (disable-service serv) (when (transient-service? serv) - (put-message (current-registry-channel) `(unregister (,serv))) + (send-to-registry `(unregister (,serv))) (local-output (l10n "Transient service ~a terminated, now unregistered.") (service-canonical-name serv)) @@ -2657,7 +2658,7 @@ If it is currently stopped, replace it immediately." (assert (list-of-symbols? (service-requirement new))) (assert (boolean? (respawn-service? new))) -(put-message (current-registry-channel) `(register ,new))) +(send-to-registry `(register ,new))) (let ((services (if (service? services) (begin @@ -2681,8 +2682,7 @@ already stopped." services)) ;; Remove STOPPED from the registry. - (put-message (current-registry-channel) - `(unregister ,stopped)) + (send-to-registry `(unregister ,stopped)) #t) (define (deregister-service service-name) -- 2.45.2
[PATCH shepherd] service: Factor out SEND-TO-PROCESS-MONITOR.
From: Attila Lendvai Assert that (CURRENT-PROCESS-MONITOR) is valid. * modules/shepherd/service.scm (send-to-process-monitor): New function. (handle-SIGCHLD), (process-monitor), (monitor-service-process), (terminate-process): Use it. --- modules/shepherd/service.scm | 16 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/modules/shepherd/service.scm b/modules/shepherd/service.scm index d9dc0d5..e1c602c 100644 --- a/modules/shepherd/service.scm +++ b/modules/shepherd/service.scm @@ -2351,8 +2351,7 @@ otherwise by updating its state." #t) ((pid . status) ;; Let the process monitor handle it. - (put-message (current-process-monitor) -`(handle-process-termination ,pid ,status)) + (send-to-process-monitor `(handle-process-termination ,pid ,status)) ;; As noted in libc's manual (info "(libc) Process Completion"), ;; loop so we don't miss any terminated child process. @@ -2488,10 +2487,13 @@ which its completion status will be sent." (values (unboxed-errors (get-message reply)) reply))) +(define (send-to-process-monitor message) + (assert (current-process-monitor)) + (put-message (current-process-monitor) message)) + (define (spawn-via-monitor arguments) (let ((reply (make-channel))) -(put-message (current-process-monitor) - `(spawn ,arguments ,(current-service) ,reply)) +(send-to-process-monitor `(spawn ,arguments ,(current-service) ,reply)) (unboxed-errors (get-message reply)) (get-message reply))) @@ -2539,8 +2541,7 @@ while waiting for @var{program} to terminate." "Monitor process @var{pid} and notify @var{service} when it terminates." (assert (current-process-monitor)) (let ((reply (make-channel))) -(put-message (current-process-monitor) - `(await ,pid ,reply)) +(send-to-process-monitor `(await ,pid ,reply)) (spawn-fiber (lambda () (let ((status (get-message reply))) @@ -2583,7 +2584,7 @@ group; wait for @var{pid} to terminate and return its exit status. If @var{pid} is still running @var{grace-period} seconds after @var{signal} has been sent, send it @code{SIGKILL}." (let ((reply (make-channel))) -(put-message (current-process-monitor) `(await ,(abs pid) ,reply)) +(send-to-process-monitor `(await ,(abs pid) ,reply)) (catch-system-error (kill pid signal)) (match (get-message* reply grace-period #f) @@ -2904,4 +2905,3 @@ we want to receive these signals." "This does not work for the 'root' service." (lambda (running) (local-output (l10n "You must be kidding."))) - -- 2.45.2
[PATCH shepherd] service: Rename to QUERY-SERVICE-CONTROLLER and use CUT.
From: Attila Lendvai * modules/shepherd/service.scm (query-service-controller): New function based on the previous SERVICE-CONTROL-MESSAGE. Use CUT instead of the now deleted SERVICE-CONTROL-MESSAGE. (service-control-message): Deleted. --- modules/shepherd/service.scm | 31 +++ 1 file changed, 15 insertions(+), 16 deletions(-) diff --git a/modules/shepherd/service.scm b/modules/shepherd/service.scm index b018e39..d9dc0d5 100644 --- a/modules/shepherd/service.scm +++ b/modules/shepherd/service.scm @@ -776,55 +776,54 @@ wire." "Return the \"canonical\" name of @var{service}." (car (service-provision service))) -(define (service-control-message message) - "Return a procedure to send @var{message} to the given service's control -channel and wait for its reply." - (lambda (service) -(let ((reply (make-channel))) - (put-message (service-control service) (list message reply)) - (get-message reply +(define (query-service-controller service message) + "Send @var{message} to the service's control channel of @var{message} and +wait for its reply." + (let ((reply (make-channel))) +(put-message (service-control service) (list message reply)) +(get-message reply))) (define service-running-value ;; Return the "running value" of @var{service}. - (service-control-message 'running)) + (cut query-service-controller <> 'running)) (define service-status ;; Return the status of @var{service}, one of @code{stopped}, ;; @code{starting}, @code{running}, or @code{stopping}. - (service-control-message 'status)) + (cut query-service-controller <> 'status)) (define service-respawn-times ;; Return the list of respawn times of @var{service}. - (service-control-message 'respawn-times)) + (cut query-service-controller <> 'respawn-times)) (define service-startup-failures ;; Return the list of recent startup failure times for @var{service}. (compose ring-buffer->list - (service-control-message 'startup-failures))) + (cut query-service-controller <> 'startup-failures))) (define service-status-changes ;; Return the list of symbol/timestamp pairs representing recent state ;; changes for @var{service}. (compose ring-buffer->list - (service-control-message 'status-changes))) + (cut query-service-controller <> 'status-changes))) (define service-process-exit-statuses ;; Return the list of last exit statuses of @var{service}'s main process ;; (most recent first). (compose ring-buffer->list - (service-control-message 'exit-statuses))) + (cut query-service-controller <> 'exit-statuses))) (define service-enabled? ;; Return true if @var{service} is enabled, false otherwise. - (service-control-message 'enabled?)) + (cut query-service-controller <> 'enabled?)) (define service-replacement ;; Return the replacement of @var{service}, #f if there is none. - (service-control-message 'replacement)) + (cut query-service-controller <> 'replacement)) (define service-logger ;; Return the logger of @var{service}, #f if there is none. - (service-control-message 'logger)) + (cut query-service-controller <> 'logger)) (define (service-recent-messages service) "Return the list of recent messages logged for @var{service}." -- 2.45.2
[PATCH shepherd] service: Add custom printer for .
From: Attila Lendvai * modules/shepherd/service.scm (print-service): New function. --- modules/shepherd/service.scm | 6 ++ 1 file changed, 6 insertions(+) diff --git a/modules/shepherd/service.scm b/modules/shepherd/service.scm index 8c9d1c6..b018e39 100644 --- a/modules/shepherd/service.scm +++ b/modules/shepherd/service.scm @@ -34,6 +34,7 @@ #:use-module (fibers timers) #:use-module (srfi srfi-1) #:use-module (srfi srfi-9) + #:use-module (srfi srfi-9 gnu) #:use-module (srfi srfi-26) #:use-module (srfi srfi-34) #:use-module ((srfi srfi-35) #:hide (make-condition)) @@ -358,6 +359,11 @@ Log abnormal termination reported by @var{status}." ;; requests such as 'start' and 'stop' on this channel. (control %service-control set-service-control!)) +(define (print-service service port) + (format port "#" (service-canonical-name service))) + +(set-record-type-printer! print-service) + (define* (service provision #:key (requirement '()) base-commit: 9f2d5ea865a7a769fe2c7ef5cd13ff84cf277ec5 -- 2.45.2
[PATCH shepherd] service: Factor out SEND-TO-SERVICE-CONTROLLER.
From: Attila Lendvai * modules/shepherd/service.scm (service-running-value): New function. (query-service-controller), (enable-service), (disable-service), (record-service-respawn-time), (start-service), (stop-service), (service-registry), (handle-service-termination): Use it. --- modules/shepherd/service.scm | 24 +--- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/modules/shepherd/service.scm b/modules/shepherd/service.scm index b5f3e23..ae9fbed 100644 --- a/modules/shepherd/service.scm +++ b/modules/shepherd/service.scm @@ -776,11 +776,15 @@ wire." "Return the \"canonical\" name of @var{service}." (car (service-provision service))) +(define (send-to-service-controller service message) + "Send @var{message} to the service's control channel of @var{message}." + (put-message (service-control service) message)) + (define (query-service-controller service message) "Send @var{message} to the service's control channel of @var{message} and wait for its reply." (let ((reply (make-channel))) -(put-message (service-control service) (list message reply)) +(send-to-service-controller service (list message reply)) (get-message reply))) (define service-running-value @@ -836,11 +840,11 @@ wait for its reply." (define (enable-service service) "Enable @var{service}." - (put-message (service-control service) 'enable)) + (send-to-service-controller service 'enable)) (define (disable-service service) "Disable @var{service}." - (put-message (service-control service) 'disable)) + (send-to-service-controller service 'disable)) (define (register-service-logger service logger) "Register @var{logger}, a value as returned by @@ -850,7 +854,7 @@ wait for its reply." (define (record-service-respawn-time service) "Record the current time as the last respawn time for @var{service}." - (put-message (service-control service) 'record-respawn-time)) + (send-to-service-controller service 'record-respawn-time)) (define (service-running? service) "Return true if @var{service} is not stopped." @@ -949,7 +953,7 @@ found in the service registry." #f) ;; Start the service itself. (let ((reply (make-channel))) - (put-message (service-control service) `(start ,reply)) + (send-to-service-controller service `(start ,reply)) (match (get-message reply) (#f ;; We lost the race: SERVICE is already running. @@ -1008,7 +1012,7 @@ in a list." '( ;; Stop the service itself. (let ((reply (make-channel))) - (put-message (service-control service) `(stop ,reply)) + (send-to-service-controller service `(stop ,reply)) (match (get-message reply) (#f #f) @@ -1184,7 +1188,7 @@ requests arriving on @var{channel}." ;; Terminate the controller of each of SERVICES and return REGISTERED ;; minus SERVICES. (for-each (lambda (service) - (put-message (service-control service) 'terminate)) + (send-to-service-controller service 'terminate)) services) (vhash-fold (lambda (name service result) (if (memq service services) @@ -1211,8 +1215,7 @@ requests arriving on @var{channel}." (loop (register service))) ((_ . old) (let ((reply (make-channel))) -(put-message (service-control old) - `(replace-if-running ,service ,reply)) +(send-to-service-controller old `(replace-if-running ,service ,reply)) (match (get-message reply) (#t (loop registered)) (#f @@ -2603,8 +2606,7 @@ been sent, send it @code{SIGKILL}." @var{service}; @var{status} is the process's exit status as returned by @code{waitpid}. This procedure is called right after the process has terminated." - (put-message (service-control service) - `(handle-termination ,pid ,status))) + (send-to-service-controller service `(handle-termination ,pid ,status))) (define (respawn-service serv) "Respawn a service that has stopped running unexpectedly. If we have -- 2.45.2
shepherd patches, manual
pardon my previous mails with the shepherd patches! i've blindly followed the shepherd manual. i've sent a patch now -- to guix-patches this time --, to update its contribution chapter. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Go find yourself first, so you can find me.” — Rumi (1207–1273)
Re: Proposal for removing some serialization limitations of define-configuration
> authority letsencrypt { > api url "https://acme-v02.api.letsencrypt.org/directory"; > account key "/some/path.pem" rsa > contact "mailto:who@knows"; > } i have no time to verify this right now, but i think you should be able to write only the header and the wrapper in your custom serializer, and then iterate through the remaining fields of the config record, i.e. sans the `name` field. you can inspect the current serializer for how to iterate through the fields, or better yet, how to call it recursively with the remaining fields. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Government does not grow by seizing our freedoms, but by assuming our responsibilities.” — Michael Cloud
Re: Improving build-time checks for kernel module configuration + two other discussion points
> > My idea for how to mitigate this is adding some sort of extensible > > service where services can register necessary kernel configuration > > settings. When the config option is unset, a build-time error is raised. > > > That sounds good to me, although I am not sure why it should be a > runtime service; it seems like a part of the kernel package build > process, but maybe this is just a difference in choice of words? Or I > could be missing something entirely! the meaning of 'service' in the guix context is quite different than what that word means to most people today. you should think of it as a component of the declarative OS definition (as opposed to as a runtime agent that responds to requests). -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “You cannot legislate the poor into freedom by legislating the wealthy out of freedom. And what one person receives without working for, another person must work for without receiving. The government can’t give to anybody anything that the government does not first take from somebody. And when half of the people get the idea they don’t have to work because the other half’s going to take care of them, and when the other half get the idea it does no good to work because somebody’s going to get what I work for. That, dear friend, is about the end of any nation. You cannot multiply wealth by dividing it.” — Adrian Rogers (1931–2005), 1984
Re: watchdog triggered auto-rollback
> > i'm afraid that's not the case currently: > > > > %guile-static-stripped crashes with a sigsegv (i.e. the guile used in the > > initrd (?)) > > https://issues.guix.gnu.org/71211 > > > Interesting. Is this a recent bug? When I was trying to bring up > Guix on the VisionFive 2 I was being dropped into a Guile REPL when > the initrd failed to find the root partition. well, the reproducer "works" on a recent x86_64, but i originally noticed this long ago (maybe a year even). back then i investigated an early crash in the boot, and reached %GUILE-STATIC-STRIPPED, and made a TODO note to further investigate. then i forgot most of what happened, and recently i opened a bug report based on my note. since then EXPRESSION->INITRD may have changed, because it now uses %GUILE-STATIC-INITRD. but it's created with the same MAKE-GUILE-STATIC-STRIPPED that produces the faulty %GUILE-STATIC-STRIPPED, so... in short: the reproducer crashes both %GUILE-STATIC-STRIPPED and %GUILE-STATIC-INITRD on x86_64, and i believe that it crashes the same in the early phase of the boot when it tries to enter the debugger. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- Freedom cannot be given… it can only be taken away.
Re: watchdog triggered auto-rollback
> I believe that if the initrd fails during startup it will abort into an > interactive Guile REPL. This might hurt GRUB's ability to detect > something went wrong since the kernel would still be running. i'm afraid that's not the case currently: %guile-static-stripped crashes with a sigsegv (i.e. the guile used in the initrd (?)) https://issues.guix.gnu.org/71211 -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Self-education is, I firmly believe, the only kind of education there is.” — Isaac Asimov (1920–1992)
Re: [shepherd] several patches that i deem ready
i've just force-pushed it with more conformant commit messages: https://codeberg.org/attila-lendvai-patches/shepherd/commits/branch/various -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “If controlling another human being is the goal of parenting, then force is necessary. Fear, intimidation, threats, power-plays, and physical pain are the means of control. But if growing healthy humans is the goal, then building trust relationships, encouraging, guiding, leading, teaching, and communicating are the tools for success.” — L. R. Knost, 'The Problem with Punishment'
Re: Upgrading Shepherd services
hi Felix, > And Attila, as for your interaction with Ludo' I am not sure there is > great value in venting about Ludo' making changes that are difficult to > rebase upon. It is the privilege of a maintainer. > > You are not the only one to have felt that frustration. well, i have two hats on in this situation: when it's my developer hat on, then i agree with you. but when i have the enthusiastic guix user hat on... then i'm a bit concerned that shepherd seems to be a one-bus project (https://chaoss.community/kb/metric-bus-factor/). and it has issues that are stopping me from using guix in ways that i'd like to... which is why i sometimes put the developer hat on, and then send my contributions... which are then met with... well... a moderate level of enthusiasm. now, i, the dev, understand Ludo's perspective: i also prefer spending my free time hacking ahead on the joyous path of my own plans and inspirations, instead of reviewing contributions. but one of these contributions was a fix for a long-standing, and rather hard to find bug (that, BTW, also caused the recent, multi-day outage of several guix services). and the rest of the commits in my branch are mostly "just" the means to finding bugs like that, including the ones in my own services. and it's reasonable to expect that these commits will be useful for finding future bugs, too. and i, the user, am somewhat concerned about the way such contributions are greeted. now, the situation is tricky here, because i'm both guys... :) and the concerned voice of the enthusiastic user sure sounds like the whining of a self-righteous, misunderstood genius... so, yeah. but here we are nevertheless. > At the same time, your contributions to the Shepherd could be very > valuable. You are talented and committed to excellence. All you have > to do---if it's not an overreach for me to say so here---is to get > yourself on the same page with Ludo'. that sounds like a monarchy, but my preferred locales are meritocracies... ;) yet, i think i'm still going the extra mile for now, and i'm jumping even those hoops that i find arbitrary (even if i argue against them in the process). > Please forgive my professorial tone. no, it's welcome, i appreciate your feedback! it has helped me to understand my internal dev vs. user conflict. > For example, if Ludo' doesn't want debugging statements all over the > place there must be another plan to capture the output. (Ludo' has not > said how, or I read over it.) There is no point to litigate the details ultimately, you can't escape the fact that only the programmer knows what state is useful in a sequential log for understanding the dynamic behavior of a codebase. and "log statements scattered around the codebase" are exactly those annotations. and in addition they also serve as comments, only "smart" ones that are also observable at runtime when needed. > here, but I would be happy to offer my help to mediate so that your > contributions become more acceptable upstream. > > As a rule, I do not contribute to projects where my own direction > diverges too much, unless I offer features that are universally > attractive. Life is too short. sure, i get it. and with only my programmer hat on, i wouldn't even be here writing this mail... but with my enthusiastic user hat on, i'm all the more concerned about that sentiment! -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “The true test of intelligence is not how much we know how to do, but how we behave when we don't know what to do.” — John Holt (1923–1985), 'How Children Fail' (1964)
Re: Upgrading Shepherd services
> I see some services starting but no errors on the console. Also, there > is absolutely nothing in /var/log/messages. Would it help to diagnose > it using your Shepherd branch? yep, in two ways: my branch has extensive logging (and currently its default level is set to debug), and i also reworked and extended the error handling. my expectation is that your machine should both start up, and also emit some useful log why that specific service is failing. if that is not the case, then i'd really love to see a self-contained reproducer. if you want to dig deeper towards a reproducer, then one option is to try to write a guix system test that reproduces it (see gnu/tests/ for examples, and `make check-system`). to use my shepherd channel: (channel (name 'shepherd) (url "https://codeberg.org/attila-lendvai-patches/shepherd.git";) (branch "attila") (introduction (make-channel-introduction ;; note that this commit id changes whenever i rebase and force-push my commits "13557ba988f4976f6581149ecdc06fce031258c7" (openpgp-fingerprint "69DA 8D74 F179 7AD6 7806 EE06 FEFA 9FE5 5CF6 E3CD" and in your OS definition follow the instructions that are now in the shepherd README. HTH, -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Gradualism in theory is perpetuity in practice.” — Jared Howe
Re: [shepherd] several patches that i deem ready
> i've rebased my commits on top of the devel branch, and in the process i've > reordered them into a least controversial order for your cherry-picking > convenience: > > https://codeberg.org/attila-lendvai-patches/shepherd/commits/branch/various > > i just started a wave of deeper testing after the rebase, so the more complex > commits may change, but those need further work/negotiation anyway. Ludo, the first commit ('Replace stop with stop-service in power-off of the root service.') used to serve to avoid a warning, but on the 'devel' branch it is now essential: # halt halt: error: exception caught while executing 'power-off' on service 'root': Unbound variable: stop -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Tyranny is defined as that which is legal for the government but illegal for the citizenry.” — Thomas Jefferson (1743–1826)
Re: Upgrading Shepherd services
hi Felix, > Here is a small one for not booting, although the service activation > during 'guix deploy' succeeds. > > Please try the Guix timer below with the Shepherd development branch. > My equipment does not boot when the apparently erroneous (actions ...) > field in the shepherd-service record is present. i cannot reproduce this. maybe it fails for you due to some missing modules that are available in my test env? this below is with my shepherd branch, but later i double checked with vanilla 'devel', and it works the same. # herd trigger garbage-collector Triggering timer. # herd[210]: [debug] Got a reply, processing it shepherd[1]: [debug] fork+exec-command for (guix gc --free-space=1G), user #f, group #f, supplementary-groups (), log-file #f shepherd[1]: [debug] exec-command for (guix gc --free-space=1G), user #f, group #f, supplementary-groups (), log-file #f, log-port # shepherd[1]: Timer 'garbage-collector' spawned process 212. shepherd[1]: [debug] query-service-controller; message status, service #< provision: (garbage-collector) requirement: (guix shepherd[1]: [debug] query-service-controller; message running, service #< provision: (garbage-collector) requirement: (gui shepherd[1]: [guix] guix gc: already 30082.59 MiBs available on /gnu/store, nothing to do shepherd[1]: Process 212 of timer 'garbage-collector' terminated with status 0 after 1 seconds. HTH, -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “There are two ways to be fooled. One is to believe what isn't true; the other is to refuse to believe what is true.” — Søren Kierkegaard (1813–1855)
Re: [shepherd] several patches that i deem ready
hi Ludo, > nevertheless, i'll rebase my work on the devel branch eventually. it > will be a lot of pain in itself, but if i need to reimplement/rebase > stuff by hand anyway, then i'll try to further sort the commits in a > least-controversial order. i've rebased my commits on top of the devel branch, and in the process i've reordered them into a least controversial order for your cherry-picking convenience: https://codeberg.org/attila-lendvai-patches/shepherd/commits/branch/various i just started a wave of deeper testing after the rebase, so the more complex commits may change, but those need further work/negotiation anyway. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “We have been to the moon, we have charted the depths of the ocean and the heart of the atom, but we have a fear of looking inward to ourselves because we sense that is where all the contradictions flow together.” — Terence McKenna (1946–2000)
Re: A different way to build GCC to overcome issues, especially with C++ for embedded systems
hi Stefan, > > Does anyone know if this is available in a public repository? Or if it has > > been moved forward? > > > There is no public repository for it. this is a much more valuable piece of contribution than to allow it to hang in the void indefinitely! if the only reason that you have not made a channel for this yet is that you've never done it before, then i'd be happy to walk you through it off list. or i can help you setting up a guix fork where you can push your own signed commits, and guix pull from. or if you don't mind that the initial commit will be signed by me, then i can even set up a guix channel on codeberg, give you commit access, and then you can push any further changes there. just let me know. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Fear is not in the habit of speaking truth; when perfect sincerity is expected, perfect freedom must be allowed; nor has anyone, who is apt to be angry when he hears the truth, any cause to wonder that he does not hear it.” — Cornelius Tacitus (ca. 56–117)
Re: Upgrading Shepherd services
> I have a lot of custom Shepherd services. Every so often I make a > mistake that stalls the step in 'guix deploy' that upgrades Shepherd > services, but without any error messages. > > Unfortunately, I can also no longer run 'herd status', which likewise > hangs, or 'reboot'. How may I debug such issues in my operating-system > declaration, please? Ludo, this is the kind of issue for which extensive logging is needed. i.e. there's no self-contained reproducer (or is there, Felix?), and it requires a live environment to experience it. and i suspect that i may even have fixed this in one of the commits that cleans up shepherd's error handling. one of the issues i remember is that an exception from the start (or stop?) GEXP of a service sometimes brought shepherd into a non-responsive state (without any sign of it in its logs). Felix, i'm planning to rebase my branch on Ludo's devel branch. it's not trivial because Ludo continues hacking shepherd, but i'll hopefully do it in the next few days. after that you may give it a try and see if you experience this issue again, and if you do then you can have plenty of logs to give you a clue why/how it happens. if you do have a reproducer, then i'd be interested in adding it as a test in the shepherd codebase. https://codeberg.org/attila-lendvai-patches/shepherd/commits/branch/various -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “It is humiliating to realize that when you drive yourself underground, when you fake who you are, often you do so for people you do not even like or respect.” — Nathaniel Branden (1930–2014)
Re: branch master updated (2bea3f2562 -> 6745d692d4)
> I do not know what FIXUP commits are, but generally a Git history should a fixup is just a regular commit that was meant to be squashed on another commit before pushing. maybe the git hook could be extended to grep for 'fixup', 'squash me', 'KLUDGE', etc in the commit message? not sure whether to stick only to the formal annotations added by programs, or use a more heuristic set of words and then ask for a y/n (if feasible from the hook). -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Uniquely in us, nature opens her eyes and sees that she exists.” — Raymond Tallis (1946–)
Re: Changing the defaults for --localstatedir and --sysconfdir?
> What do others think? i don't really understand all the consequences of this choice... but as a newcomer it surely was strange that i have to use a special incantation that i need to remember, and is added to the documentation, and is explained repeatedly on IRC... instead of making it the default. good defaults are important. and it's better if the default value causes the least surprise to the ones who know the least. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “[…] the real self is dangerous: dangerous for the established church, dangerous for the state, dangerous for the crowd, dangerous for the tradition, because once a man knows his real self, he becomes an individual.” — Osho (1931–1990)
Re: Value in adding Shepherd requirements to file-systems entries?
> P.S. The code above should read (requirements ...) in the plural. inside shepherd there's a bit of anomaly, but it's called requirement in the public API, and also in the guix side of the config; i.e. it's not plural. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Moderation in temper is always a virtue; but moderation in principle is always a vice.” — Thomas Paine (1737–1809)
Re: "Error: Could not set character size"
seems like a `guix pull` and a `guix home reconfigure` resolved it. sorry for the noise. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “If you wind up with a boring, miserable life because you listened to your mom, your dad, your teacher, your priest, or some guy on TV telling you how to do your shit, then YOU DESERVE IT.” — Frank Zappa (1940–1993), 'The Real Frank Zappa Book' (1989)
"Error: Could not set character size"
...when trying to build a guix checkout. to reproduce: $ make doc/images/service-graph.png DOT doc/images/service-graph.png Error: Could not set character size [error message repeated 12 more times] this seems to be an error coming from graphviz. this is all i could find online, which is not very useful: https://gitlab.com/graphviz/graphviz/-/issues/863 maybe this commit a few days ago changed something? looks innocent, though: https://git.savannah.gnu.org/cgit/guix.git/commit/?id=4099b12f9f4561d0494c7765b484b53d9073b394 any hints? -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “You cannot transmit wisdom and insight to another person. The seed is already there. A good teacher touches the seed, allowing it to wake up, to sprout, and to grow.” — Thich Nhat Hanh (1926–)
Re: Guix bios installation: Grub error: unknown filesystem
> This should allow grub to recognise your filesystem during the > installation process. I think using a later version of grub would fix > this, but that hasn't happened yet. I think there's a patch to upgrade > it in `core-updates` somewhere, but I'm not sure. grub seems to be still v2.06 there: https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/bootloaders.scm?h=core-updates#n103 -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “In a democracy, mass opinion creates power. Power diverts funds to the manufacturers of opinion, who manufacture more, etc. […] This feedback loop generates a playing field on which the most competitive ideas are not those which best correspond to reality, but those which produce the strongest feedback.” — Mencius Moldbug
Re: [shepherd] several patches that i deem ready
hi Ludo, > > i have prepared the rest of my commits that were needed to hunt down the > > shepherd hanging bug. you can find them at: > > > > https://codeberg.org/attila-lendvai-patches/shepherd/commits/branch/attila > > > > there's some dependency among the commits, so sending them to debbugs would > > be either as one big series of commits, or a hopeless labirinth of patches > > otherwise. > > > Yes, but OTOH, piecemeal, focused changes sent to Debbugs are easier to > review for me. (There are 34 commits in this branch touching different > aspects.) i understand that, but cutting out some of the commits from this branch is a lot of work at best, and not possible at worst due to semantic dependencies. e.g. how shall i implement proper error handling without being able to inspect what's happening (i.e. proper logging)? nevertheless, i'll rebase my work on the devel branch eventually. it will be a lot of pain in itself, but if i need to reimplement/rebase stuff by hand anyway, then i'll try to further sort the commits in a least-controversial order. > I cherry-picked a couple of patches. > > Some notes: > > + 94c1143 shepherd: Add tests/startup-error.sh > > Redundant with ‘tests/startup-failure.sh’ I think? one of them just returns #f from its start lambda, while the new one throws an error. they exercise different code paths in shepherd. > + e802761 service: Add custom printer for records. > > > Good idea, but the goal is to remove GOOPS, so put aside for now. ok, i'll get rid of it (move it away into a local kludge branch). its main purpose is to be able to simply FORMAT some service objects into the log. > + af2ebec service: respawn-limit: make #f mean no limit. > > I’d rather not do that: one can use +inf.0 when needed. i found the respawn-limit API somewhat confusing (it requires a cons cell with two numbers). i thought #f could be a simple way to disable the respawn limit; simple both in implementation and as an API. FWIW, it's the first time i've ever met +inf.0 but as you wish, we can manage without this commit. > + 095e930 shepherd: Do not respawn disabled services. > > That’s already the case (see commit > 7c88d67076a0bb1d9014b3bc23ed9c68f1c702ab; maybe we hacked it > independently in parallel). err, hrm... i'm not sure anymore why i created that commit. "Respawning ~a." is printed before calling START-SERVICE (that then does honor the ENABLED? flag). maybe i recorded this commit without actually checking whether the service is respawned (as opposed to merely printing an inert log message). i'll get rid of this, but the incorrect respawning message will remain a source of confusion. > + dbc9150 shepherd: Increase the time range for the default respawn limit. > > This arbitrary and thus debatable, but I think the current setting > works well, doesn’t it? the current limit will not catch services whose start-to-fail time is not in the ballpark of 1 sec (5 times in 7 seconds). the startup-to-fail time of the service i'm working with is way above 1 sec. > + e03b958 support: Add logging operators. > + 39c2e14 shepherd: add call-with-error-handling > > I like the idea: we really need those backtraces to be logged! > There are mostly-stylistic issues that would need to be discussed > though. I’d like logging to be less baroque; I’m not convinced by: what do you mean by 'baroque' here? too verbose in the source code? > + 7183c9c shepherd: Populate the code with some log lines. > > This is exactly what I’d like to avoid—adding logging statements all > around the code base, possibly redundant with existing logging > statements that target users. > > What I do want though is to have “first-class logs”, pretty much like > what we see with ‘herd log’ etc. To me, that is much more useful than > writing the arguments passed each and every ‘fork+exec-command’ call. don't they serve two entirely different purposes? 1) logs meant for the users of shepherd (aka herd log) vs 2) logs that the shepherd and service developers need to understand shepherd's temporal behavior. i added every logging related code in the various pursuits of hunting down specific bugs. 1. bug gets triggered 2. stare at logs, have some questions 3. add some more log statements 4. goto 1. i'm not aware of any way to efficiently inspect the temporal behavior of a codebase other than adding explicit log statements. ideally using multiple, hierarchical log categories that can be turned on and off separately, both at runtime and at compile time. what i added to shepherd is a super simplified, local, mock version of that (short of porting/finding a proper logging library in scheme). > I’ll have to look
Re: No default OpenJDK version?
> Currently, most java packages use the implicit jdk from the build > system (ant- or maven-build-system), which is… icedtea@8. We still > have quite a lot of old packages that don't build with openjdk9, so > I'm not sure when we can update the default jdk… does that prevent the introduction of a newer default JDK and annotating the old java dependency on the (hopefully few) packages that fail to build with a newer default? -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Malthus was right. It's hard to see how the solar system could support much more than 10^28 people or the universe more than 10^50.” — John McCarthy (1927–2011), father of Lisp
Re: backdoor injection via release tarballs combined with binary artifacts (was Re: Backdoor in upstream xz-utils)
> > I think we should gradually move to building everything from > > source—i.e., fetching code from VCS and adding Autoconf & co. as inputs. > > > the big drawback of this approach is that we would lose maintainers' > signatures, right? it's possible to sign git commits and (annotated) tags, too. it's good practice to enable signing by default. admittedly though, few people sign all their commits, and even fewer sign their tags. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Never appeal to a man's "better nature". He may not have one. Invoking his self-interest gives you more leverage.” — Robert Heinlein (1907–1988), 'Time Enough For Love' (1973)
policy for packaging insecure apps
the context: there's an app currently packaged in guix, namely gnome-shell-extension-clipboard-indicator, that has a rather questionable practice: by default it saves the clipboard history (passwords included) in clear text, and the preferences for it is called something obscure. its author actively defends this situation for several years now, rejecting patches and bug reports. a detailed discussion is available at: https://github.com/Tudmotu/gnome-shell-extension-clipboard-indicator/issues/138 the fact that its name suggests that it is *the* standard gnome clipboard app makes the situation that much worse. my question: how shall we deal with a situation like this? 1) shall i create a guix patch that makes the necessary changes in this app, and submit it to guix? this would be a non-trivial, and a rather hidden divergence from upstream, potentially leading to confusion. 2) is there a way to attach a warning message to a package to explain such situations to anyone who installs them? should there be a feature like that? should there be a need for a --force switch, or an interactive y/n, to force installing such apps? 3) is there a point where packages refusing to address security issues should be unpackaged? and also added to a blacklist until the security issue is resolved? where is that point? would this one qualify? 4) is this the responsibility of a project like guix to address situations like this? 5) do you know another forum where this dispute should be brought up instead of guix-devel? i'm looking forward to your thoughts, and/or any pointers or patches to the documentation that i should read. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- The price of liberty is eternal vigilance.
Re: backdoor injection via release tarballs combined with binary artifacts (was Re: Backdoor in upstream xz-utils)
> Are there other issues (different from the "host cannot execute target > binary") that makes relesase tarballs indispensable for some upstream > projects? i didn't mean to say that tarballs are indispensible. i just wanted to point out that it's not as simple as going through each package definition and robotically changing the source origin from tarball to git repo. it costs some effort, but i don't mean to suggest that it's not worth doing. > So, while "almost all the world" is applying wrong solutions to the > source tarball reproducibility problem, what can Guix do? AFAIU the plan is straightforward: change all package definitions to point to the (git) repos of the upstream, and ignore any generated ./configure scripts if it happens to be checked into the repo. it involves quite some work, both in quantity, and also some thinking around surprises. i think a good first step would be to reword the packaging guidelines in the doc to strongly prefer VCS sources instead of tarballs. > Even if We™ (ehrm) find a solution to the source tarball reproducibility > problem (potentially allowing us to patch all the upstream makefiles > with specific phases in our packages definitions) are we really going to > start our own (or one managed by the reproducible build community) > "reproducible source tarballs" repository? Is this feaseable? but why would that be any better than simply building from git? which, i think, would even take less effort. > > but these generated man files are part of the release tarball, so > > cross compilation works fine using the tarball. > > > AFAIU in this case there is an easy alternative: distribute the > (generated) man files as code tracked in the DVCS (e.g. git) repo > itself. yes, that would work in this case (although, that man page is guaranteed to go stale). my proposal was to simply drop the generated man file. it adds very little value (although it's not zero; web search, etc). -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “It is easy to be conspicuously 'compassionate' if others are being forced to pay the cost.” — Murray N. Rothbard (1926–1995)
Re: backdoor injection via release tarballs combined with binary artifacts (was Re: Backdoor in upstream xz-utils)
> Are really "configure scripts containing hundreds of thousands of lines > of code not present in the upstream VCS" the norm? pretty much for all C and C++ projects that use autoconf... which is numerous, especially among the core GNU components. > If so, can we consider hundreds of thousand of lines of configure > scripts and other (auto)generated files bundled in release tarballs > "pragmatically impossible" to be peer reviewed? yes. > Can we consider that artifacts as sort-of-binary and "force" our > build-systems to regenerate all them? that would be a good practice. > ...or is it better to completely avoid release tarballs as our sources > uris? yes, and this^ would guarantee the previous point, but it's not always trivial. as an example see this: https://issues.guix.gnu.org/61750 in short: when building shepherd from git the man files need to be generated using the program help2man. this invokes the binary with --help and formats the output as a man page. the usefulness of this is questionable, but the point is that it breaks crosscompilation, because the host cannot execute the target binary. but these generated man files are part of the release tarball, so cross compilation works fine using the tarball. all in all, just by following my gut insctincts, i was advodating for building everything from git even before the exposure of this backdoor. in fact, i found it surprising as a guix newbie that not everything is built from git (or their VCS of choice). -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “For if you [the rulers] suffer your people to be ill-educated, and their manners to be corrupted from their infancy, and then punish them for those crimes to which their first education disposed them, what else is to be concluded from this, but that you first make thieves [and outlaws] and then punish them.” — Sir Thomas More (1478–1535), 'Utopia', Book 1
Re: backdoor injection via release tarballs combined with binary artifacts (was Re: Backdoor in upstream xz-utils)
> Also, in (info "(guix) origin Reference") I see that Guix packages can have a > list of uri(s) for the origin of source code, see xz as an example [7]: > are they intended to be multiple independent sources to be compared in > order to prevent possible tampering or are they "just" alternatives to > be used if the first listed uri is unavailable? a source origin is identified by its cryptographic hash (stored in its sha256 field); i.e. it doesn't matter *where* the source archive was acquired from. if the hash matches the one in the package definition, then it's the same archive that the guix packager has seen while packaging. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “We’ll know our disinformation program is complete when everything the American public believes is false.” — William Casey (1913–1987), the director of CIA 1981-1987
Re: [shepherd] several patches that i deem ready
> i have prepared the rest of my commits that > were needed to hunt down the shepherd hanging bug. > you can find them at: https://codeberg.org/attila-lendvai-patches/shepherd/commits/branch/various Ludo, i would appreciate if you could give some feedback on this. these commits are extensive (line-diff-wise) due to the added log statements, and the error handling wrapper forms. the more you work on shepherd, the more these commits rot away, and the more avoidable work/frustration it is to keep rebasing them. i believe that these are valuable additions to shepherd, as was the reconfigure hang fix that these were needed for. as a first phase, maybe you could cherry pick some of the commits that you find agreeable. i'm looking forward to your feedback on how i could/should improve these to get them merged. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “When training beats education, civilization dies.” — C. S. Lewis (1898–1963)
Re: xz backdoor
> There's actually suspicious code by the xz attacker in one of our > packages right now: > > https://issues.guix.gnu.org/issue/70113 > > Please help review that patch! as for gpaste (one of the dependees of libarchive): it doesn't build since the recent gnome merge. i've filed a patch for the necessary version bump: https://issues.guix.gnu.org/70133 which also gets rid of the libarchive dependency. it would be nice to get this fast tracked. although, judging from the (lack of) complaints, i might be the only user of it. PS: and meanwhile we're packaging an alternative, namely gnome-shell-extension-clipboard-indicator, with an enormous security flaw: by default it saves the clipboard history in clear text, and calls the feature "cache only favorites", so that even if you look for it, you still don't realize it: https://github.com/Tudmotu/gnome-shell-extension-clipboard-indicator/issues/138#issuecomment-904689439 ...and its author actively defends this situation. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “The noble-minded are calm and steady. Little people are forever fussing and fretting.” — Confucius (551–479 BC), 'Analects of Confucius'
Re: Emacs and Gnome branches are merged now
just a heads' up: after pulling the gnome changes i was unable to log in (empty screen with frozen pointer after the password). `herd restart elogind` can be used to get back to the login screen. i could fix it using the following steps: 1. move away my ~/.local/share/gnome-shell/extensions/ 2. login 3. start extension config and disable all extensions 4. re-login, move back that dir 5. re-login, update some extensions (still disabled) 6. reenable all extensions in the config HTH, -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “The most potent weapon of the oppressor is the mind of the oppressed.” — Steve Biko (1946–1977)
Re: xz backdoor
> The quick summary is that Guix currently shouldn't be affected > because a) Guix currently packages xz 5.2.8, which predates the > backdoor, and b) the backdoor includes checks based on absolute > paths e.g. under /usr and Guix executable paths generally don't > match the patterns checked for. and guix doesn't use systemd that patches sshd -- a critical piece of security -- in a way that made the backdoor possible... -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “War is a moral contest that is won in the temples before it is ever fought.” — Sun Tzu (c. 6th century BC), author of 'The Art of War' (as paraphrased by Jack Kennedy)
Re: Losing signing keys for custom Guix channel
> from reading about guix authentication I think the new signing key > must be first added to the .guix-authoriations file and that commit > must signed with the current signing keys before the new signing > key can be used. yep. otherwise anyone with access to the origin git repo could override the commit signature based authentication framework. if you think about it, if there were any options for you to sidestep this situation of a lost key, then any attacker could do the same. i'm afraid your only option is to re-record and re-sign every commit, force-push them, and publish a new channel intro snippet that all your users must copy into their config. alternatively, you *may* be able to simply publish a new channel intro snippet (and convince all your users that it's a genuine situation) that will point to the first new commit that is signed with the new key... but i doubt the contract (nor the implementation) of the authentication code would just silently accept the non-authenticated commits that precede your new channel intro commit. all the best in fixing the situation! -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “’Tis better it be a year later before he can read, than that he should this way get an aversion to learning.” — John Locke (1632–1704), 'Some Thoughts Concerning Education'
Re: the right to rewrite history to rectify the past (was Re: Concerns/questions around Software Heritage Archive)
> We are talking about social rules that we have here in the Guix > community not legal/state rules. ethics, i.e. the discussion of rights, is a branch of philosophy. ideally, it should inform the people who are writing and enforcing state laws, but these days -- sadly -- it has precious little to do with state laws. and i think you're the one here who conflates the two. > Specifically the social rules that we support trans people and we want > to include them. Any person really that want to change their name at > some point for some reason. > > To that end we listen to their concerns/wishes and we accommodate them. i've asked you this before, and i'll keep asking it: sure, accommodate, but to what extent? what is a reasonable cost i can incur on others? (see the discussion of negative vs. positive rights in this context) what if i declare that i only feel accommodated here if everyone attaches the local weather forcast to each mail they send to guix-devel? the limit of your demands begins where it starts to constrain the freedom of others. considering this is an essential part of respectful behavior towards others. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “I am not what happened to me, I am what I choose to become.” — Carl Jung (1875–1961)
Re: rewriting history; Was: Concerns/questions around Software Heritage Archive
> not an expert in guix internals) the only reason we care about > identity is that it's part of git commits. identities are deeply intertwined with trust (our best predictor of future behavior is past behavior). and how trust is facilitated by the tools and processes (including the social "technology") can make or break any group effort. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “The direct use of physical force is so poor a solution to the problem of limited resources that it is commonly employed only by small children and great nations.” — David D. Friedman (1945–), 'The Machinery of Freedom' (1973)
rewriting history; Was: Concerns/questions around Software Heritage Archive
> I was also distressed to see how poorly they treated a developer > who wished to update their name: > https://cohost.org/arborelia/post/4968198-the-software-heritag > https://cohost.org/arborelia/post/5052044-the-software-heritag let's put aside the trans aspect of this question for a moment, because this question has broad implications, much broader than the regrettable struggles of trans people. the question here is whether person A has the right to demand that others change their memory of A's past actions (i.e. rewrite history, or else become a felon... or maybe just unwelcome in polite society?). so, let's just assume that i have decided to prefer being called a new name (without disclosing my reasons). is it reasonable for me to demand from somebody else to change their memory of my past actions? e.g. to demand that they rewrite their memory/instances of my books that i have published under my previous name in the past? or that they forget my old name, and when the change happened? or that they do not link the two names to the same individual? if so, then where is the line? what's the principle here? and what are its implications? do i have the right to demand the replacement of a page in each copy that exists out there? i.e. should it be criminal (or just a sin?) to own old copies? do i have the right to demand that certain libraries must sell/burn their copies of my books and never own them again? what if i committed a fraud? e.g. i pushed a backdoor somewhere... do i have the right to memory-hole my old identity? and who will enforce such a right? the government? i.e. those people who already keep an (extralegal) record of whenever i farted in the past decade? where can i even file my GDPR request for that? would that really be a "right to be forgotten", or merely a tool of even tighter monopolization of The Central Database? what if i'm a joker and i demand a new change every week for the rest of my life? do i have the right to the resources of every library out there? to keep their staff and computers busy for the next couple of decades? but let's put the technical aspects aside; wherever we draw the line... what are the implications of that for borader society? because i sure see some actors out there who can hardly wait to start erasing certain records at the barrel of the law, including rewriting books of significance... (and while we are at it, i suggest to start preserving your offline/local copies, because we're up to a wild ride!) humanity has reached an enormous challenge with the complete marginalization of the costs of storing and transmitting information. it's a completely new/different playing field, and how we proceed from here has grave implications. this questions is nowhere near as obvious/trivial as presented in the cited blog post. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “It is only when compassion is present that people allow themselves to see the truth. […] Compassion is a kind of healing agent that helps us tolerate the hurt of seeing the truth.” — A.H. Almaas (1944–), 'Elements of the Real in Man (Diamond Heart, Book 1)'
Re: Concerns/questions around Software Heritage Archive
> only a 35 yrs old white cis boy you're judging a group of individuals, namely those who were handed the cis white male mix at the genetic lottery, as a uniform blob. and maybe even somewhat deplorable, if i'm reading your right. does it make sense to judge an individual based on some coincidental properties? or really, based on anything else than their actions? does it make sense to discuss the actions/morality of a group of individuals that is formed based on some coincidental properties? e.g. what can we say about the morality of all the blond people? and ultimately, is that an effective way of speaking up for human rights and welcoming environments -- of all things? maybe it's time to take a thorough look at the book that you're preaching from? if i may, let me attempt to inspire you: “The world is changed by your example, not by your opinion.” — Paulo Coelho (1947–) % “Yesterday I was clever, so I wanted to change the world. Today I am wise, so I am changing myself.” — Rumi (1207–1273) % “If there is to be peace in the world, There must be peace in the nations. If there is to be peace in the nations, There must be peace in the cities. If there is to be peace in the cities, There must be peace between neighbors. If there is to be peace between neighbors, There must be peace in the home. If there is to be peace in the home, There must be peace in the heart.” — Lao Tzu (sixth century BC) % “A man of humanity is one who, in seeking to establish himself, finds a foothold for others and who, in desiring attaining himself, helps others to attain.” — Confucius (551–479 BC) % “To put the world in order, we must first put the nation in order; to put the nation in order, we must first put the family in order; to put the family in order; we must first cultivate our personal life; we must first set our hearts right.” — Confucius (551–479 BC) % “Until we have met the monsters in ourselves, we keep trying to slay them in the outer world. And we find that we cannot. For all darkness in the world stems from darkness in the heart. And it is there that we must do our work.” — Marianne Williamson (1952–), 'Everyday Grace: Having Hope, Finding Forgiveness And Making Miracles' (2004) % “If things go wrong in the world, this is because something is wrong with the individual, because something is wrong with me. Therefore, if I am sensible, I shall put myself right first” — Carl Jung (1875–1961), 'The Meaning of Psychology for Modern Man' -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “If liberty means anything at all, it means the right to tell people what they do not want to hear.” — George Orwell (1903–1950)
Re: Contribute or create a channel?
> > channels are a step towards this, but they are not enough in their > > current form to successfully accommodate for such a setup. an obvious > > thing that is missing is a way to formally express inter-channel > > dependencies, including some form of versioning. > > > Do we not have this? The manual documents a mechanism for channel > dependencies in "(guix) Declaring Channel Dependencies". > > I haven't used it, but it looks like the dependencies are declared as > channels, which can have the usual branch/commit specifications to tie > them to specific versions. good point, thanks! i looked briefly at the code just now. it's not trivial, and it seems to treat the guix channel specially (because i don't need to specify it as a dependency in my channel's .guix-channel file), and i'm not sure how it behaves when e.g. two channels depend on the same channel, but pick two different commits... or all the other convoluted situations. the reason i assumed it doesn't exist is that i've never seen it used by any channels that i looked at. > What are we missing? i guess it's time to experiment to be able to answer your question. FTR, it's READ-CHANNEL-METADATA and friends in guix/channels.scm note that it's not the same thing as /etc/guix/channels.scm, even though they appear similar (https://issues.guix.gnu.org/53657). -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “People who have never gone to school have never developed negative attitudes toward exploring their world.” — Grace Llewellyn
Re: Contribute or create a channel?
> > the patch inflow to the guix repo is currently overwhelming the > > available capacity for review and pushing. > > > With an email like the one sent by Hartmut we can better arrange for > shepherding this large submission. (Nothing is to be gained from > repeatedly bemoaning well-known issues in the patch review processes > here and in other threads on the mailing list.) i was reflecting on why i wrote this, and what i wanted to express is that i think guix has reached a point where a monorepo is becoming a net negative, and i don't see this being discussed. my gut feeling is that new abstractions are needed that would enable splitting the monorepo/community into less tightly coupled subgroups where they can have their own coding standards, repos, channels, etc, and a more federated way to maintain/integrate all the software that exists out there into a guix system. in this hypothetical setup commit rights could be issued much more liberally to non-core sub-repos, and more rigorous code reviews would only need to be done when a new version of the split-out part is being incorporated back into a new revision of the core/bootstrap chain (if e.g. assuming python is needed for the bootstrap of the core, then the python subgroup's stuff would only need core review when a new version of that is pointed to by the core). or alternatively, simply try to split guix into a minimal core that is essential for the bootstrap, and everything else into multiple subchannels (gnome, gui stuff in general, random apps, etc). i have no impression how much that alone could shrink the monorepo part, though. channels are a step towards this, but they are not enough in their current form to successfully accommodate for such a setup. an obvious thing that is missing is a way to formally express inter-channel dependencies, including some form of versioning. sadly, i don't have any proposals beyond discussing the observable issue (i.e. the insufficient patch throughput). -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Values in a free society are accepted voluntarily, not through coercion, and certainly not by law… every time we write a law to control private behavior, we imply that somebody has to arrive with a gun [to enforce it].” — Ron Paul
Re: Mechanism for helping in multi-channels configuration
> Although I concur with this need, I do not see how it would be help for > detecting compatibility between channels. :-) maybe i'm overthinking this, and all we need is a way to point to git commit ranges that are compatible. more specifically, i'm maintaining the guix-crypto channel, and i often miss the ability to point to a guix commit, beyond which there is a change in guix that my channel is not yet compatible with. if my users issue a `guix pull`, then it would not pull the guix channel beyond that commit, and warn the users that it's being held back. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “In the electronics industry, patents are of no value whatsoever in spurring research and development.” — vice-president of Intel Corporation, Business Week, 11 May 1981.
Re: Guix Days: Patch flow discussion
> Somehow, the reader will judge if Message-ID is smoothly supported. :-) i regularly meet this most unfortunate attitude in the GNU circles, where oldtimers dismiss any discussion of friendlier defaults for newcomers with the "argument" that it's configurable, and therefore it's a non-issue. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “The wise prince must provide in such a way that the citizens will always be in need of him and of his government.” — Niccolo Machiavelli (1469–1527), 'The Prince' (1513)
Re: Contribute or create a channel?
> WDYT? I'm eager to learn about your thoughts. the patch inflow to the guix repo is currently overwhelming the available capacity for review and pushing. if you want an agile experience, i.e. where you can quickly fix/update this and that, then i suggest your own channel (unless you have the commit bit for the guix repo... or there's a committed maintainer who is a regular user, and as such will fast-track your patches). otherwise you'll end up using a channel anyway (i.e. your fork of the guix repo while your patches are waiting in the queue to be reviewed and pushed). PS: i don't mean to sound cynical here, just matter-of-factly. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Opportunity is missed by most people because it comes dressed in overalls and looks like work.” — Thomas A. Edison (1847–1931)
Re: A basic Shepherd bug?
hi Felix, > > you should follow the instructions in [1]; namely: > > > > https://lists.gnu.org/archive/html/guix-devel/2023-12/msg00018.html > > > > together with "Installing development snapshots with Guix" in > > shepherd's README to add shepherd's channel. > > > I did so on a production system which I do not reboot often. Two days > ago, I reconfigured and saw a new Shepherd version being deployed. It > used up a lot of CPU cycles until I rebooted. just to clarify: you `guix system reconfigure` into a new shepherd version, and after that the currently running shepherd init process went 100% CPU, i.e. it was busy looping in one thread? > It makes sense that upgrading the Shepherd requires a reboot, but maybe > a warning somewhere would be appropriate, if possible. Maybe an email to > root? unfortunately a lot of the infrastructure around guix is lacking explicit formal description of dependencies/requirements. e.g. there's nothing (that i know of) in the shepherd config files (which are generated by `guix system reconfigure`) about what shepherd version they require/assume. a quick and dirty solution here could be to manually emit an assert into the config file that there's an exact match between the shepherd that generated the config file, and the shepherd process trying to load it. a warning could be issued that the shepherd process is unable to load/process the generated config file until a reboot... which would probably be overkill in most cases. https://ngnghm.github.io/ this^ blog has interesting thoughts on migration and staged computation. it's a most interesting vision of how these abstractions could be formally captured, and what the resulting computing system would look like. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Love and do what you will.” — Augustine of Hippo (354–430), 'A sermon on love'
Re: Guile-Git 0.6.0 released; looking for maintainers!
> An idea might be to look into using nyacc’s ffi-helper to generate > struct definitions. over there in CL land i wrote an automatic FFI generator. it's now part of the main CL FFI lib: https://github.com/cffi/cffi/tree/master/src/c2ffi it is based on c2ffi: https://github.com/rpav/c2ffi which is a piece of C++ code that uses CLANG as a library to parse any C header file, and emit its content into a json file. a thin layer of lisp code can generate the actual sexp FFI definitions from the json files, that then can be hooked into the usual guile way of doing FFI. the json files can be checked into the repo, which then eliminates the dependency on c2ffi on the user side (i.e. the project is only as heavy as any other hand-written FFI wrapper). that way only the maintainer needs to regenerate the json files every once in a while. or short of a smarter build tool like ASDF, we can also check in the generated lisp files. if there's interest, then i can help porting this over to guile. below are some example projects that are using it. they are rather thin and simple, yet provide a full FFI: https://github.com/hu-dwim/hu.dwim.zlib https://github.com/hu-dwim/hu.dwim.sdl PS: clang now supports `-ast-dump=json`, which may or may not eliminate the need for c2ffi entirely: https://github.com/rpav/c2ffi/issues/112 -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Sometimes I wonder whether the world is being run by smart people who are putting us on or by imbeciles who really mean it.” — Mark Twain (1835-1910)
Re: Google Season of Docs 2024
> 3. Any for improving the documentation? just a general wish: much less prose and much more structure, please. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “There is no coming to consciousness without pain. People will do anything, no matter how absurd, in order to avoid facing their own Soul. One does not become enlightened by imagining figures of light, but by making the darkness conscious. The latter procedure, however, is disagreeable and therefore not popular.” — Carl Jung (1875–1961), 'Alchemical Studies' (1967)
Re: Mechanism for helping in multi-channels configuration
> The wishlist is: provide a machine-readable description on guix-science > channel side in order to help in finding the good overlap between > commits of different channels. i wrote about a missing abstraction here: https://lists.gnu.org/archive/html/guix-devel/2023-12/msg00104.html which is more or less related to this. the git commit log is a too fine-grained granularity here. there should be something like a 'guix log' above the git log that could be used, among other things, to encode inter-channel dependencies. maybe frequent semver releases for guix channels could work as reference points to be used to formally encode inter-channel dependencies? (and to guide the substitute chaching/building; mark "safe points" for the time-machine; etc) -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- Life is a tragedy to those who feel and a comedy to those who think.
Re: Mechanism for helping in multi-channels configuration
> Anything is better than an obscure failure/backtrace i disagree with this specific statement. in the long run, the (inconspicuous) cost of added complexity can easily move anything into net negative territory. IOW, feel encouraged to account for the cost of complexity. it's rarely done prior to setbacks. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Until we have met the monsters in ourselves, we keep trying to slay them in the outer world. And we find that we cannot. For all darkness in the world stems from darkness in the heart. And it is there that we must do our work.” — Marianne Williamson (1952–), 'Everyday Grace: Having Hope, Finding Forgiveness And Making Miracles' (2004)
Re: Introducing Guix "Features"! (Was: Syntactic Diabetes)
> In the systemd realm, there are different types of services, I think one > is called "one-shot" which is effectively quite similar to the types of > services guix has... they do something once, and there is no running > daemon. So, for better or worse, guix is not so far from one of the most > widespread and commonly used systems here... executed at each boot vs. executed when compiling the system (i.e. at different stages, as in 'staged computation'). it's a bit like using the same word to describe macros and functions in lisp: yes, deep down they are both just functions, but they are different enough to warrant a different name to facilitate a more efficient human comprehension. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “The nation that will insist on drawing a broad line of demarcation between the fighting man and the thinking man is liable to find its fighting done by fools and its thinking done by cowards.” — Sir William Francis Butler (1838–1910)
Re: Introducing Guix "Features"! (Was: Syntactic Diabetes)
> Am Donnerstag, dem 01.02.2024 um 20:30 + schrieb Attila Lendvai: > > > for an average unix user a service is a process that is running in > > the backgroud, doing stuff mostly without any user interaction. you > > can try to argue this away, but i'm afraid that this is the state of > > things. > > Which is exactly what etc-service-type does. It symlinks stuff to /etc > without user interaction. we can spend our life honing in on a satisfying definition, but let it be enough that what is commonly understood as a service has an active component (see 'run' in my definition); i.e. it has a temporal dimension. but honestly? it felt silly to even provide a definition in my mail. we either live in a different universe, or you're just focused on justify the status quo. whichever is the case, we have reached a dead end, because essentially, this is aesthetics. but anyway, i gave my feedback, and as i don't have the authority to lobby for renaming core guix abstractions, i'm out. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Until you figure out that [manipulation] is going on, we're all going to be run like rats through a maze. Culture is an intelligence test, and when you pass that test you don't give a shit about culture.” — Terence McKenna (1946–2000)
Re: Introducing Guix "Features"! (Was: Syntactic Diabetes)
> > for an average unix user a service is a process that is running in the > > backgroud, doing stuff mostly without any user interaction. you can > > try to argue this away, but i'm afraid that this is the state of > > things. > > > I don’t think it’s a good idea to aim to satisfy some presumed “average > unix user”, because such a user would not be familiar with many concepts > introduced by Guix (e.g. “guix shell” or “guix system”). the primary argument was that two, very different abstractions share the same name, and in shared contexts. it's just icing on the cake that one of the abstractions is nothing like what most users understand by the name 'service'. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “If you love somebody, let them go, for if they return, they were always yours. If they don’t, they never were.” — Khalil Gibran (1883–1931)
Re: Introducing Guix "Features"! (Was: Syntactic Diabetes)
> there for most of the time already. And if you think about it, > symlinking stuff to /etc is a service. i've arrived to guix after 3+ decades of programming, most of that in opensource environments, unix-like OS'es, and more than a decade using linux as my primary OS and lisp as my goto language. it could be me, of course, but it took me months of tinkering until i understood the guix service vs shepherd service nomenclature. and i still need to focus when i'm dealing with foo-service-type and shepherd services at the same time. this nomenclature was an obstacle to understanding, because the naming suggests something that was misleading me. for an average unix user a service is a process that is running in the backgroud, doing stuff mostly without any user interaction. you can try to argue this away, but i'm afraid that this is the state of things. and if you care whether your words (code) is communicating what you want to be understood by your audience, then you must consider their model of reality. which reminds me of: “Programs must be written for people to read, and only incidentally for machines to execute.” — Abelson & Sussman, SICP, preface to the first edition -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “As a rule, whatever we don’t deal with in our lives, we pass on to our children. Our unfinished emotional business becomes theirs. As a therapist said to me, "Children swim in their parents’ unconscious like fish swim in the sea."” — Gábor Máté (1944–), 'In the Realm of Hungry Ghosts' (2008)
Re: [OT: s/Joshua/Josiah/ in sig ; -] Re: [shepherd] several patches that i deem ready
> > “But if you wish to remain slaves of bankers and pay the cost of your own > > slavery, let them create money.” > > — Joshua Stamp > > ^^ > Josiah > https://en.wikipedia.org/wiki/Josiah_Stamp,_1st_Baron_Stamp > > Hi attila (and others who, like me, may enjoy the quotations > at the bottom of your posts :) your report is much appreciated, and thanks for your kind words, too! it's good to know that someone not only enjoys them, but that it has initiated further research. it reminds me of how it all started: years ago i found myself, that on a mailing list i was only reading the end of mail quotes from a great hacker (http://fare.tunes.org), from whom i have learned a lot on a wide range of topics. and then it struck me: i should have this too! (be the change you want to see in the world, and whatnot... :) in that spirit, my scripts and my collection is available below (it often has quotes and references in comments, and it's grouped by topics): https://codeberg.org/attila.lendvai/dotfiles > Should such misspellings be reported somewhere as a bug? an email like this is perfect. you may consider keeping it off-list though, to respect the topic of the list. thanks again and happy hacking, -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Many people believe that evil is the presence of something. I think it’s the absence of something.” — Lisa Unger (1970–), 'Sliver of Truth'
Re: [shepherd] several patches that i deem ready
> i have prepared the rest of my commits that were needed to hunt down the > shepherd hanging bug. you can find them at: > > https://codeberg.org/attila-lendvai-patches/shepherd/commits/branch/attila FWIW, here i have the guix side of the patches (they are not required for the shepherd changes): https://codeberg.org/attila-lendvai-patches/guix/commits/branch/shepherd-guix-side the first commit touches hurd, which i have not tested. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “But if you wish to remain slaves of bankers and pay the cost of your own slavery, let them create money.” — Joshua Stamp
Re: [shepherd] several patches that i deem ready
> About "cheaper code path when a log level is disabled at runtime", > perhaps it can be improved in guile-lib, but otherwise that's a nice > list. I just wish we had a good logging library in Guile and could stop > reinventing the wheel left and right. i've made my judgement that the logger in guile-lib was never applied seriously when i relized that it stores the enabled state in a hashtable (which must be looked up for every log statement). i made sure the log statements have a unique syntax, so the underlying machinery can be replaced easily later, and then i moved on. > OK. For levels greater than debug, they I see them as glorified > comments (executable comments as yo wrote), so I don't see a strong > reason to attempt to hide them or treat them specially. In Python > (which strives to be readable), we typically break logging lines (which > are concatenated for free inside the parens -- default Python behavior), > and that doesn't hurt readability in my opinion, and means we can just > follow the usual style rules, keeping things simple. my experience is different. i found myself only ever looking at log statements when i'm debugging something, regardless of the level, and including other people's code. and then i just toggle line wrap with the press of a button. must be related to my habit that i usually put more effort into making the code more self-documenting (readable) than i put into writing informal comments and documentation. and rethinking my "executable comment" metaphore: these log statements serve much less as comments than reporting the temporal state and program flow. but my primary aim is to color it all gray, and i don't immediately know how to do that in emacs for multiline sexp's (i.e. balanced parens). this is the primary reason our team just kept them on one line, but the flexibility of toggling word wrap as needed is also nice: the essetial part is always within a reasonable margin, and the rest can be read when word-wrap is enabled. if requested, then i'm willing to re-format the log statements if i can find a way to still color it all gray. it's important that logging stays out of sight while reading the code. > Thanks for working on this, I'm sure it'll help many, myself included, > following the execution of Shepherd more easily. my pleasure! in my experience when a project doesn't have proper logging, backtraces, error handling hygene, and warning-free compilation, then inefficient debugging quickly eats up more time than it would take to implement these features properly. unfortunately, guix and guile is not very good on this front, so i found myself working on these, too. such investment rarely pays off for the first bug, but it pays off very well in the long run. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “A political situation is the manifestation of a parallel psychological problem in millions of individuals. This problem is largely unconscious (which makes it a particularly dangerous one!)” — Carl Jung (1875–1961), Letters, vol.1 pg. 535
Re: [shepherd] several patches that i deem ready
hi Maxim, > > > - a lightweight logging infrastructure together with plenty of log > > > lines throughout the codebase, and some hints in the README on how > > > to turn log lines gray in emacs (i.e. easily ignorable). > > > Are you using guile-lib's logging library for it? I've used it in > guile-hall if you want to have an example. We should maximize its > users, refine it and aim to have it builtin Guile at some point. i looked at that lib first (IIRC by your recommendation), but i ended up rolling my own for the cost of two additional pages of code in shepherd. i think the main issue i had was the amount of unconditional computation that happens on the common code path, and its complexity in general, including its API. shepherd has some non-trivial machinery regarding logging output being captured and redirected through sockets to herd and whatnot; i.e. most of the handler machinery in guile-lib's logger would be just an impedance mismatch instead of being helpful. for those two pages it's: - one less external dependency - less resource use - more flexibility - cheaper code path when a log level is disabled at runtime - compile-time log level to drop entire log levels - and most importantly much less code complexity. you can find the relevant commit at: https://codeberg.org/attila-lendvai-patches/shepherd/commits/branch/attila FWIW, it's a partial bort of a CL lib of mine: https://github.com/hu-dwim/hu.dwim.logger > > a quick note on the log statements: they are essentially noise when it > > comes to reading the code, hence the gray coloring i suggest in > > emacs. (although they may often serve also as "executable" comments). > > > > i'd also like to propose to relax the 80 column limit for log lines > > for the same reason that i've explained above. > > > I don't think an exception is deserved here; the logging library from > guile-lib for example concatenates message strings by default, which > makes it easy to brake long messages on multiple lines. my ultimate goal here is to minimize the disruption of code readability. only some emacs (editor) magic and/or code formatting can help with that. log lines are only relevant when you're debugging something, or when you're trying to understand a codebase. all other times they are essentially noise. my proposal is what our CL team settled with: always one line per log statement, and setting the foreground color of the entire line gray in emacs. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “The pursuit of commerce reconciles nations, calms wars, strengthens peace, and commutes the private good of individuals into the common benefit of all.” — Hugh of Saint Victor (1096–1141)
Re: [shepherd] several patches that i deem ready
> - a lightweight logging infrastructure together with plenty of log > lines throughout the codebase, and some hints in the README on how > to turn log lines gray in emacs (i.e. easily ignorable). a quick note on the log statements: they are essentially noise when it comes to reading the code, hence the gray coloring i suggest in emacs. (although they may often serve also as "executable" comments). i'd also like to propose to relax the 80 column limit for log lines for the same reason that i've explained above. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Happiness, whether consisting in pleasure or virtue, or both, is more often found with those who are highly cultivated in their minds and in their character, and have only a moderate share of external goods.” — Aristotle (BC 384–322), 'Book VII, 1323.b1'
[shepherd] several patches that i deem ready
dear Guix, Ludo, i have prepared the rest of my commits that were needed to hunt down the shepherd hanging bug. you can find them at: https://codeberg.org/attila-lendvai-patches/shepherd/commits/branch/attila there's some dependency among the commits, so sending them to debbugs would be either as one big series of commits, or a hopeless labirinth of patches otherwise. therefore i recommend the following workflow instead (assuming that Ludo is pretty much the only one hacking on shepherd): Ludo, please take a look at my branch, and cherry-pick whatever you are happy with. then based on your feedback, and the new main branch, i'll rebase and refine my commits and give you a head's up when it's ready for another merge/review. the commits are more or less ordered in least controversial order, modulo dependencies. the main additions are: - a multi-layered error handler that got employed at various points in the codebase. this makes shepherd much more resilient, even in case of nested errors, and much more communicative in the log when errors end up happening. - a lightweight logging infrastructure together with plenty of log lines throughout the codebase, and some hints in the README on how to turn log lines gray in emacs (i.e. easily ignorable). looking forward to your feedback, -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “What you do speaks so loud I cannot hear what you say.” — Ralph Waldo Emerson (1803–1882)
Re: Guix deploy --keep-going equivalent for machine connection failures
> Another option worth considered is adding a `'can-connection-fail?' (default: > #f)` to either `machine` or `machine-ssh-configuration`. i'd call it `ignore-connection-failure?`, or if we want to ignore all problems for a machine, then `ignore-failure?`. --keep-going could set the default value for the session, and the machine specific variable would override it. as for the implementation, i'd use continuable exceptions of a specific type, and then from scheme code users could install handlers that ignore the situation. avoiding shell scripts these days is a good idea after all. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Democracy and liberty are not the same. Democracy is little more than mob rule, while liberty refers to the sovereignty of the individual.” — Walter E. Williams (1936–)
Re: Module unavailable on build side
this may help: https://github.com/attila-lendvai/guix-crypto/blob/main/src/guix-crypto/service-utils.scm#L56 and you should grep for its use in that repo. w-i-m ensures that the GEXP's that are instantiated in its dynamic extent will "capture" these modules as their dependencies, and i think it also inserts the appropriate use-modules forms at the head of the GEXP's. but take this all with a grain of salt, because what i understand i decoded on my own from the guix codebase, and the names of these abstractions are rather misleading. also, i'm using these in the service code that gets compiled for shepherd to be loaded. the environment surrounding the building of packages may behave differently. HTH, -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “There is no difference between living and learning […] it is impossible and misleading and harmful to think of them as being separate.” — John Holt (1923–1985)
Re: Guix wiki
> 1. People find the [data] service provides value (can someone restate what > that > value is exactly? Is it needed e.g. to power if you allow hijacking the above into the wiki discussion: this is a good example where a wiki page (central, easily editable, capturing the current state) would tremendously help this discussion. who, where, why, what, etc... such a wiki doesn't need to be completely open for self-registration, which is the source of most issues. kinda like the commit bit, but with more relaxed requirements. maybe invite-only, or a somewhat hidden static secret could be used for gatekeeping the registration. it's an illusion that everything is captured by the mailing list archive when finding stuff is inefficient in a discussion log. also, the wiki displays the current state, not the entire bumpy road getting there. i know about https://libreplanet.org/wiki/Group:Guix but the search engines don't really. and i don't know how others feel about it, but i subconsciously don't take it seriously because the search box is not focused on guix, and in this form it feels like just an aftertought, not a guix wiki. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “If you are ever tempted to look for outside approval, realize that you have compromised your integrity. If you need a witness, be your own.” — Epictetus (c. 55–135 AD)
Re: Are declarative app configs worth it?
> > - adding it to guix increases maintenance burden: new versions could > > add or remove config options > > > This is why there should be automated tests. There are too few of them. early detection of the breakage is just one part of the story. then it also needs to be fixed -- before dropping the hammer and abandoing the worksite. writing and maintaining the tests have a cost, too. > > - it requires documentation/translation, another hidden cost > > > We should only accept configuration procedures that have proper > documentation, yes. in this context i recommed: What is Seen and What is Not Seen by Bastiat https://oll.libertyfund.org/page/wswns or specifically: "In the sphere of economics an action, a habit, an institution or a law engenders not just one effect but a series of effects. Of these effects only the first is immediate; it is revealed simultaneously with its cause, it is seen. The others merely occur successively, they are not seen;3 we are lucky if we foresee them." if you demand that e.g. all services accepted into guix have a configuration entry for every possible config field, and that the documentation of these fields are duplicated into the guix codebase... then whatever is included into guix will have a 100% coverage. this is what is seen. but what about the lost potential? because i can guarantee you that while you'll get 100% coverage, you'll also only get a fraction of the total number services and fields. which one will yield a better guix experience? what i'm doing with my own services, and what i also recommend, is to always have an 'extra-arguments or 'extra-fields that allows defining any config value, and serializes it as-is. that way the user can rely on the documentation of the daemon, and blindly apply it while writing the guix config. and only reify those couple of config fields into scheme code that can provide something useful beyond merely serializing the value as-is. this way: - the guix codebase remains smaller (OAOO principle) - updating the app's package in guix is simpler - guaranteed not to get out of sync with the app - smaller threshold for new contributions - which translates to more supported services i find the free-form module type, as suggested by John Soo above, to be a good idea. so much so that i may even look into writing a prototype and try to use it to replace my two inline shepherd-service instances. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “War is a ritual, a deadly ritual, not the result of aggressive self-assertion, but of self-transcending identification. Without loyalty to tribe, church, flag or ideal, there would be no wars.” — Arthur Koestler (1905–1983)
Re: A basic Shepherd bug?
> I hope to test your hypothesis. Will the trick to enable logging [1] > also pull pull in the bug fix? yes, you should follow the instructions in [1]; namely: https://lists.gnu.org/archive/html/guix-devel/2023-12/msg00018.html together with "Installing development snapshots with Guix" in shepherd's README to add shepherd's channel. but it will not enable logging, but rather configure your operating system to compile and use the latest commit from the shepherd git repo, which now (probably) contains the fix for your problem. with the logging you're probably referring to my pending commits in: https://codeberg.org/attila-lendvai-patches/shepherd/commits/branch/attila but they are still being polished. i haven't even proposed them yet for inclusion, but i'm already running my servers with that branch. > Alternatively, may I fix the fcgiwrap-service-type to work with the > Shepherd version currently standard in Guix? if you can't make progress with the above, then send me your config, and i'll run it with my shepherd branch, and hopefully the extra logging can help easily fix any issues. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “If a nation values anything more than freedom, it will lose its freedom; and the irony of it is that if it is comfort or money that it values more, it will lose that, too.” — Somerset Maugham (1874–1965)
Re: A basic Shepherd bug?
> I added the service below to my operating-system, and 'herd status' > prints nothing but hangs. Is that a bug? > > Same on reboot. Thanks! this has probably been fixed in: https://git.savannah.gnu.org/cgit/shepherd.git/commit/?id=9be0b7e6fbe3c2e743b5626f4dff7c7bf9becc16 but it hasn't reached guix yet. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- An economist is a person who spends half his life telling us what will happen and the other half explaining why it didn't.
Re: A different way to build GCC to overcome issues, especially with C++ for embedded systems
TL;DR it's probably due to some cmake mess. and i gave up on compiling this; if i really want to, i'll do it in a debian chroot. > > i tried with your gcc … but it errors out with: > > > > $ make > > [ 1%] Linking ASM executable bs2_default.elf > > arm-none-eabi-gcc: error: nosys.specs: No such file or directory > > > That file is located here: > /gnu/store/…-newlib-arm-none-eabi-4.3.0/arm-none-eabi/lib/nosys.specs that didn't help: ~/workspace/guix/guix/pre-inst-env guix shell gcc-toolchain cmake make pkg-config -e '(@ (gnu packages embedded2) gcc12-cross-newlib-arm-none-eabi-toolchain)' cd ~/workspace/bios/pico-serprog cmake . COMPILER_PATH=/gnu/store/i9kpjzbagdlpm8bs10gxmm21b271s056-newlib-3.0.0-0.3ccfb40/arm-none-eabi/lib/ LDFLAGS=-B/gnu/store/i9kpjzbagdlpm8bs10gxmm21b271s056-newlib-3.0.0-0.3ccfb40/arm-none-eabi/lib/ make it leads to the same problem. but then i tried to compile the pico-sdk subproject as a standalone project, and then it succeeds (but fails with a different error later). therefore i think it's due to some cmake mess that i don't want to get deeper into. so, the rest is just FYI: git clone https://github.com/raspberrypi/pico-sdk ~/workspace/guix/guix/pre-inst-env guix shell gcc-toolchain cmake make pkg-config -e '(@ (gnu packages embedded2) gcc12-cross-newlib-arm-none-eabi-toolchain)' cmake . make $ make [ 0%] Building ASM object src/rp2_common/boot_stage2/CMakeFiles/bs2_default.dir/compile_time_choice.S.obj [ 0%] Linking ASM executable bs2_default.elf [ 0%] Built target bs2_default [ 0%] Creating directories for 'ELF2UF2Build' [ 0%] No download step for 'ELF2UF2Build' [ 0%] No update step for 'ELF2UF2Build' [ 0%] No patch step for 'ELF2UF2Build' [ 0%] Performing configure step for 'ELF2UF2Build' -- The C compiler identification is unknown -- The CXX compiler identification is unknown -- Detecting C compiler ABI info -- Detecting C compiler ABI info - failed -- Check for working C compiler: /gnu/store/3yqw7js9slid5q3d1f5bpbk92vann109-profile/bin/gcc -- Check for working C compiler: /gnu/store/3yqw7js9slid5q3d1f5bpbk92vann109-profile/bin/gcc - broken CMake Error at /gnu/store/4bn07jsqk6lxp0qdxv7kkc3krz3afnna-cmake-3.25.1/share/cmake-3.25/Modules/CMakeTestCCompiler.cmake:70 (message): The C compiler "/gnu/store/3yqw7js9slid5q3d1f5bpbk92vann109-profile/bin/gcc" is not able to compile a simple test program. It fails with the following output: Change Dir: /home/alendvai/workspace/bios/pico-sdk/elf2uf2/CMakeFiles/CMakeScratch/TryCompile-fg3QLY Run Build Command(s):/gnu/store/3yqw7js9slid5q3d1f5bpbk92vann109-profile/bin/make -f Makefile cmTC_12bac/fast && make[3]: Entering directory '/home/alendvai/workspace/bios/pico-sdk/elf2uf2/CMakeFiles/CMakeScratch/TryCompile-fg3QLY' /gnu/store/3yqw7js9slid5q3d1f5bpbk92vann109-profile/bin/make -f CMakeFiles/cmTC_12bac.dir/build.make CMakeFiles/cmTC_12bac.dir/build make[4]: Entering directory '/home/alendvai/workspace/bios/pico-sdk/elf2uf2/CMakeFiles/CMakeScratch/TryCompile-fg3QLY' Building C object CMakeFiles/cmTC_12bac.dir/testCCompiler.c.o /gnu/store/3yqw7js9slid5q3d1f5bpbk92vann109-profile/bin/gcc-o CMakeFiles/cmTC_12bac.dir/testCCompiler.c.o -c /home/alendvai/workspace/bios/pico-sdk/elf2uf2/CMakeFiles/CMakeScratch/TryCompile-fg3QLY/testCCompiler.c as: unrecognized option '--64' make[4]: *** [CMakeFiles/cmTC_12bac.dir/build.make:78: CMakeFiles/cmTC_12bac.dir/testCCompiler.c.o] Error 1 make[4]: Leaving directory '/home/alendvai/workspace/bios/pico-sdk/elf2uf2/CMakeFiles/CMakeScratch/TryCompile-fg3QLY' make[3]: *** [Makefile:127: cmTC_12bac/fast] Error 2 make[3]: Leaving directory '/home/alendvai/workspace/bios/pico-sdk/elf2uf2/CMakeFiles/CMakeScratch/TryCompile-fg3QLY' the root cause of the messy error message above is the following: as: unrecognized option '--64' this happens with the host gcc (or due to some misconfiguration the host gcc is used wrongly). $ file src/rp2_common/boot_stage2/bs2_default.elf src/rp2_common/boot_stage2/bs2_default.elf: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, not stripped -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.” — Antoine de Saint-Exupery (1900–1944)
Re: A different way to build GCC to overcome issues, especially with C++ for embedded systems
hi Stefan, > In the end it worked nicely for my embedded C++ stuff and I also managed > to compile a custom keyboard firmware based on ZMK using Zephyr, > although that is just C code. i've also encountered a problem with the guix cross compiling tools as i have described here: https://lists.gnu.org/archive/html/guix-devel/2023-12/msg00179.html i tried with your gcc (copied into guix as (gnu packages embedded2)): ~/workspace/guix/guix/pre-inst-env guix shell gcc-toolchain cmake make pkg-config -e '(@ (gnu packages embedded2) gcc12-cross-newlib-arm-none-eabi-toolchain)' cd ~/workspace/bios/pico-serprog cmake . make but it errors out with: $ make [ 1%] Linking ASM executable bs2_default.elf arm-none-eabi-gcc: error: nosys.specs: No such file or directory IIRC, this happens with the vanilla guix packages when i try to use a gcc package instead of a gcc-toolchain package. any thoughts on this? -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “There can be no causeless love or any sort of causeless emotion. An emotion is a response to a fact of reality, an estimate dictated by your standards.” — Ayn Rand (1905–1982)
arm-none-eabi toolchain and compiling C++ stuff
dear Guix, i'm trying to compile something to a raspberry rp2040 microcontroller (https://codeberg.org/Riku_V/pico-serprog). $ guix shell gcc-toolchain cmake make pkg-config -e "((@ (gnu packages embedded) make-arm-none-eabi-toolchain-7-2018-q2-update))" $ cd _deps $ git clone https://github.com/raspberrypi/pico-sdk.git pico_sdk-src $ cd .. $ cmake . $ make so far, so good. it compiles for a while, but then it fails with: [ 14%] Building CXX object CMakeFiles/pico_serprog.dir/_deps/pico_sdk-src/src/rp2_common/pico_standard_link/new_delete.cpp.obj In file included from pico-serprog/_deps/pico_sdk-src/src/rp2_common/pico_standard_link/new_delete.cpp:11:0: /gnu/store/7i9fw82x6hljy6sb4g10v2dl53l7pybl-profile/arm-none-eabi/include/c++/cstdlib:75:25: fatal error: stdlib.h: No such file or directory #include_next if i read this right, it tries to cross compile some C++ stuff, but a header file is missing. $ guix locate stdlib.h [...] gcc-toolchain@13.2.0 /gnu/store/dpfxpfyghkc19wz8jwaw31llhnvn8ngx-gcc-toolchain-13.2.0/include/stdlib.h gcc-toolchain@11.3.0 /gnu/store/5vn4pkf70ql7v1svrfknfkfsh4m3737h-gcc-toolchain-11.3.0/include/stdlib.h clang-toolchain@15.0.7 /gnu/store/6m5gi7l7bi93gnzm2j422q9wawq3p6al-clang-toolchain-15.0.7/include/stdlib.h [...] i.e. it's usually part of the gcc-toolchain package... but it's not part of the cross-compiling ones? is that a bug in (gnu packages embedded)? shall i look into fixing it? or am i the one who has invalid expectations? -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “If money is your hope for independence you will never have it. The only real security that a man will have in this world is a reserve of knowledge, experience, and ability.” — Henry Ford (1863–1947)
Re: shepherd, fibers, and signals (asyncs)
@emixa-d kindly proposed something that turned out to be a fix: https://github.com/wingo/fibers/issues/29#issuecomment-1858922276 i've sent it to: shepherd: sometimes hangs on `guix system reconfigure` https://issues.guix.gnu.org/67839#6 in essence: shepherd violates the fibers API by calling it from an async signal handler, and this is an issue indeed, but the bugs caused by this should manifest rarely and randomly; i.e. the frozen recofigure behavior must be caused by something else. the actual root cause here was that the with-process-monitor parameterize was not covering some code that it should have. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “I am not what happened to me, I am what I choose to become.” — Carl Jung (1875–1961)
shepherd, fibers, and signals (asyncs)
dear Guix, context: Shepherd stops responding during "guix system reconfigure" https://issues.guix.gnu.org/67538 https://issues.guix.gnu.org/65178 https://issues.guix.gnu.org/67230 i've added a ton of logging and asserts in my fork: https://codeberg.org/attila-lendvai-patches/shepherd which resulted in this report: https://github.com/wingo/fibers/issues/29#issuecomment-1858319291 to which @emixa-d kindly responded: https://github.com/wingo/fibers/issues/29#issuecomment-1858497720 which essentially identifies the following: -- posix signal handlers are async, and shepherd uses the fibers API from inside signal handlers, specifically in at least handle-SIGCHLD. this violates the fibers API, and most probably leads to the root cause of the reconfigure hang: a match-error flying out from service-controller due to losing the value of the parameter called (current-process-monitor), which then makes that fiber exit. i have little experience with posix signal handlers, so i probably won't come up with a fix for this, or at least not without someone's bird's eye view guidance. maybe the solution could be something like packaging up posix signals and delivering them to the fibers universe by some form of polling of an atomic variable? or is there some signal-safe semaphore facility in guile that could be used in accordance with the fibers API? -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Virtue is never left to stand alone. He who has it will have neighbors.” — Confucius (551–479 BC)
Re: shepherd: hardening error handling
while i found this bug: https://issues.guix.gnu.org/67839 i was reading the discussion under its probable root cause: https://github.com/wingo/fibers/issues/29 and it suggests that Guile before 3.0.5 had important bugs WRT fluids, which are relied upon in shepherd. maybe Guile 2.2 can not be used reliably even for current Shepherd? -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “It is only when compassion is present that people allow themselves to see the truth. […] Compassion is a kind of healing agent that helps us tolerate the hurt of seeing the truth.” — A.H. Almaas (1944–), 'Elements of the Real in Man (Diamond Heart, Book 1)'
Re: Should commits rather be buildable or small
> Preparing a large set of updates like this is already a great deal of > work. It does not seem to me like a good use of volunteers' time to ask > them to break such an update into hundreds of tiny pieces, especially > not if the result is hundreds of broken commits to Guix. fair enough. in that paragraph i did not consider the consts, only the benefits of the two approaches. i myself also had headaches multiple times when i fixed something that needed to touch several different packages, and they would only work when applied in one transaction: how many debbugs issues? multiple issues and record the dependencies? little gain for much more effort on both sides... but if one issue, then what should be the name of the debbugs issue? etc... the contribution process has quite some accidental complexity, and it most probably turns away valuable potential contributors... which is something that is both hard to notice, and has a strong impact. but this has already been discussed in a long thread recently. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “As long as habit and routine dictate the pattern of living, new dimensions of the soul will not emerge.” — Henry Van Dyke (1852–1933)
Re: Should commits rather be buildable or small
> > FWIW, this commit policy has always bothered me as a newcomer to > > Guix. pretty much everywhere else it's a major offence against your > > colleagues to commit something that breaks the build in any way. > > > In the last few months I’ve repeatedly seen assertions in a similar > style as this one. They always genuinely surprise me, and it’s probably > not just because I’m oblivious and out of touch. well, both point of views are reasonable. they just make different tradeoffs. i think an abstraction is missing here, let's call it guix log for this mail. it's something like the git log, but one that lists the buildable and substitutable states of the guix repo. it's probably the same thing that causes the discrepancy between git commits and substitutes: the build servers are not building every commit of the git repo. they pick an unpredictable (?) series of commits, skipping some inbetween. if i guix pull, or guix time-machine to the "wrong" commit, then i'll need to build some stuff locally. sometimes these can be heavy packages. this hypothetical 'guix log' is probably also what's missing between a hypothetical staging branch and master, whose role would be to make sure that commits don't reach the users prior to having substitutes for them. does this make sense? -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Sometimes I wonder whether the world is being run by smart people who are putting us on or by imbeciles who really mean it.” — Mark Twain (1835-1910)
Re: Should commits rather be buildable or small
> > Define "buildable" and "unbuildable". > > > I used these definitions: a buildable commit does not have build > failures (or at least no new ones). An unbuildable commit introduces > new build failures (in this case a lot of them). > > Buildable commits are safe spots to land on with time-machine in the > sense that the packages defined in them can be used. I expect it would > be very painful to try jumping to past commits with time-machine if a > large portion of the commits in Guix were unbuildable. [...] > I guess "required" here means that in some cases Guix's policy is to > prefer small commits over buildable commits (with the previous > definition). I at least don't see any technical reasons why it would be > required. The question then becomes whether that policy applies in this > case. FWIW, this commit policy has always bothered me as a newcomer to Guix. pretty much everywhere else it's a major offence against your colleagues to commit something that breaks the build in any way. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “I will probably be asked why I don't cite the author's name? Because my philosophy teacher taught me that it sometimes jeopardizes the effects of the quote.” — Author's name withheld.
Re: Heisenbug
> What's a good way to debug this, please? in Geiser i usually get the proper error message: M-x geiser ,m (gnu tests reconfigure) ,reload > Where is my error? good question! silently swallowing errors and warnings should be something that is frown upon, and only ever employed when deemed really necessery. and we thought about it... twice. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “One cannot redistribute wealth without first becoming master of all wealth; redistribution is first and foremost monopoly.” — Anselme Bellegarrigue (ca. 1820–1890)
shepherd: hardening error handling
dear Guix, i'm working on hardening shepherd's error handling and logging to debug an issue that i'm facing. these changes escalated quickly, so i'm writing to clarify a few things before i shape the codebase into a direction that the maintainers will not accept. the codebase seems to use catch/throw, and at some places with comments like "for Guile 2.2". what is the minimum guile version that the shepherd codebase wants to support? the README says "GNU Guile 3.0.x or 2.2.x". is this still intended? or can i assume guile 3? i.e. use with-exception-handler, raise-exception, guard, &co. instead of catch/throw with key and args? some WIP commits are available at: https://codeberg.org/attila-lendvai-patches/shepherd/commits/branch/attila -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “It is only when the people become ignorant and corrupt, when they degenerate into a populace, that they are incapable of exercising the sovereignty. Usurpation is then an easy attainment, and an usurper soon found. The people themselves become the willing instruments of their own debasement and ruin.” — James Monroe (1758–1831), 5th president of the USA
Re: shepherd GEXP module import mystery
> so, the only mystery left is that i still don't know where it is > imported into the unnamed package in which the GEXPs are > compiled/loaded, and whether that is intended. FTR, i've filed it as: https://issues.guix.gnu.org/67649 -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “What is history but the story of how politicians have squandered the blood and treasure of the human race?” — Thomas Sowell (1930–)
Re: Shepherd service logging
> Thanks for offering a logging facility! I run a custom Guix and would > like to test your changes. Is it enough to switch to your 'wip-logging' > branch in the package declaration? [1] Thanks! AFAIU that will lead to quite some local recompiling that are not necessary. you can just set the shepherd package of the shepherd-root-service-type to a custom package. e.g. this will use the latest shepherd from the shepherd channel: (operating-system ... (essential-services (modify-services (operating-system-default-essential-services this-operating-system) (shepherd-root-service-type config => (shepherd-configuration (inherit config) (shepherd (@ (shepherd-package) shepherd))) HTH, -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “[A] Computer [programming] language is inherently a pun — [it] needs to be interpreted by both men & machines.” — Henry Baker
Re: Syntactic Diabetes (was Re: A friendlier API for operating-system declarations)
> > the downside of generating countless macros is that they won't show up > > in backtraces, and they cannot be found when grep'ping the codebase, > > and as such make the codebase much less approachable. > > > Reading your words really helped me feel that I'm not alone. You more or > less summarized my feelings about the Guix codebase, which I have been > reading now for over a year. Guile's syntax features make the code more > symbolic and less approachable to newcomers. just FTR, i don't think that the guix codebase is too bad in this regard. here i just wanted to remind people to the not so obvious cost of syntactic abstractions that should be accounted for when making decisions. introducing macros that generate macros is rarely justified. in general, it's *very* valuable when stuff can be grep'ped -- and not only for newcomers. after enough time has passed i can feel like a newcomer to my own codebase... :) modulo the protocols that i keep while writing code. e.g. define.*whatever is a grep that i regularly employ. the pattern here is that although there are countless ways to define countless different stuff, there's a convention to stick to the define.*[name] pattern. intuitive, unique (i.e. grep'pable) names are also key to facilitate code approachability, especially for abstractions that are scattered around in many source files. in some situations i even copy-paste names of abstractions into comments for any future grep'ping to pick it up. a negative example for this in the guix codebase is the use of 'service' to describe two rather different abstractions: a component of an OS vs. a deamon process run by shepherd. for a while it caused me quite some confusion. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “The State is that organization in society which attempts to maintain a monopoly of the use of force and violence in a given territorial area; in particular, it is the only organization in society that obtains its revenue not by voluntary contribution or payment for services rendered but by coercion.” — Murray N. Rothbard (1926–1995), 'Anatomy of the State' (1974)
Re: Syntactic Diabetes (was Re: A friendlier API for operating-system declarations)
> lines of code. I think hyper-focusing on syntax to make services > "nicer" might be the wrong approach here: You could greatly reduce the > complexity by making them procedures instead of syntax and still keep > most of the configuration readable to a great extent. i agree. in my experience, it's good practice in general to try to make a functional API as convenient as possible, and if that is still too verbose or cumbersome, only then add a thin layer of syntactic abstractions that expand to code that uses this functional API. such code is much easier to work with, especially when debugging stuff interactively (i.e. it's possible to recompile a function that will then affect every place in the code where the macrology is used). the downside of generating countless macros is that they won't show up in backtraces, and they cannot be found when grep'ping the codebase, and as such make the codebase much less approachable. the size of their expansion can also become an issue if they are used often enough. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “They muddy the water, to make it seem deep.” — Friedrich Nietzsche (1844–1900)
Re: Failed to build in QA
> > The other issue with the v4 series is that Patchwork has got confused > > and only picked out the first of the v4 patches. The threading also > > looks weird to me in my email client, but I'm not quite sure why. How > > did you send the v4 patches? > > > I sent them with git send-mail but I also noticed that the order got > mixed up in issues.guix.gnu.org, no idea how this happened... i also see this every once in a while. i guess it's because the SMTP server-farm receives mutliple emails in close proximity, and they end up reaching debbugs in a different order. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “In matters of conscience, the law of majority has no place.” — Mahatma Gandhi (1869–1948)
Re: [maintenance] Compressed JSON files and served file extension?
> And if I do not want to use curl but instead another tool as wget? :-) then maybe complain to the authors that it doesn't comply with the standard? :) here's the bug report BTW: Wget not honouring Content-Encoding: gzip https://savannah.gnu.org/bugs/?61649 or use wget2 instead. i guess they didn't fix it in wget because they didn't want to break "backwards compatibility". (remember: if it's not backwards, it's not compatible! :) happy hacking, -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “We cannot train our babies not to need us. Whether it's the middle of the day or the middle of the night, their needs are real and valid, including the need for a simple human touch. A 'trained' baby may give up on his needs being met, but the need is still there, just not the trust.” — L. R. Knost
Re: shepherd GEXP module import mystery
hi Felix, Ludo, > > a start GEXP of my service sees bindings that come from the module > > called (shepherd support), but i have no idea who and where imports > > that module. > > > Without code it's hazardous to speculate, but could the Guix service > (gnu service mcron) cause that issue when it is being logged? unfortunately it's a complex web of stuff, but i managed to make a small reproducer that is attaced. it can be run with: $(guix system --no-graphic vm reproducer.scm) and in the VM (must use fold, because it's a dumb terminal): cat /var/log/messages | fold -150 to my surprise this one does list (shepherd support) in the module-use list. and i realized why: the logging infrastructure somewhere siently truncates the lines, and in my original case that module was chopped off. not that i understand everything here... e.g. why are there several (guix build utils) modules? *** reproducer gexp speaking, current module: #, module-uses: ( # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #), ringbuffer: # so, the only mystery left is that i still don't know where it is imported into the unnamed package in which the GEXPs are compiled/loaded, and whether that is intended. maybe it's part of the shepherd API that (shepherd support) is made available for the service GEXPs? looking at the public definitions in (shepherd support), it's not obvious that those are meant to be available for the users of shepherd, though. Ludo? > In my code tree, which is a month behind, (gnu services mcron) is the > only Guix service that imports (shepherd support). it's a good hint, but that could only cause this if all the service GEXPs were loaded into the same module, but that would have already broken things in countless other ways. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “We are products of our past, but we don't have to be prisoners of it.” — Rick Warren (1954–) ;; Run with something like this: ;; $(guix system --no-graphic vm reproducer.scm) (define-module (reproducer) #:use-module (gnu system) #:use-module (gnu system shadow) #:use-module (gnu system nss) #:use-module (gnu system vm) #:use-module (gnu tests) #:use-module (gnu services) #:use-module (gnu services base) #:use-module (gnu services dbus) #:use-module (gnu services shepherd) #:use-module (gnu packages admin) #:use-module (gnu packages base) #:use-module (gnu packages bash) #:use-module (gnu packages certs) #:use-module (gnu packages package-management) #:use-module (gnu packages linux) #:use-module (guix gexp) #:use-module (guix git) #:use-module (guix git-download) #:use-module (guix store) #:use-module (guix modules) #:use-module (guix packages) #:use-module (srfi srfi-1) #:use-module (ice-9 match)) (operating-system (inherit %simple-os) (services (cons* (simple-service 'reproducer shepherd-root-service-type (list (shepherd-service (requirement '(file-systems)) (provision '(reproducer)) (documentation "") (start #~(begin (lambda _ (format #t "*** reproducer gexp speaking, \ current module: ~A, \ module-uses: ~A, \ ringbuffer: ~A~%" (current-module) (module-uses (current-module)) (and=> (module-variable (current-module) 'ring-buffer) variable-ref)) 0)) %base-services)))
shepherd GEXP module import mystery
dear Guix, context: i'm adding logging to shepherd, and while i was testing it i encountered a conflict with my service code that shouldn't happen. i'm probably missing something from my mental model around guile modules, and/or how shepherd compiles and loads them. the symptom is that a start GEXP of my service sees bindings that come from the module called (shepherd support), but i have no idea who and where imports that module. i added the following two format lines to the beginning of my start GEXP: (format #t "*** start gexp speaking, current module is ~A, module-uses: ~A~% ringbuffer: ~A~%" (current-module) (module-uses (current-module)) (and=> (module-variable (current-module) 'ring-buffer) variable-ref)) (format #t "*** ringbuffer in packages: ~A~%" (map (lambda (module-name) (and=> (module-variable (resolve-interface module-name) 'ringbuffer) variable-ref)) '((guile) (oop goops) (shepherd service) (srfi srfi-34) (system repl error-handling) (guix build utils) (guix build syscalls) (gnu build file-systems and they print: *** start gexp speaking, current module is # module-uses: ( # # # # # # # # # # # # # # # # # # *** ringbuffer in packages: (#f #f #f #f #f #f #f #f) how else can a definition (i.e. 'ringbuffer) get into this module without it coming from one of the modules in its module-uses list? i'm pretty sure that it's coming from (shepherd support) because i'm neck deep in debugging the original issue: a macro from (shepherd support) overwrote my function that errored when it was called at runtime as a function. i'm also seeing this warning (i.e. my root issue): WARNING: (#{ g108}#): `log.debug' imported from both (guix-crypto utils) and (shepherd support) i've checked the module list of the gexp, and how guix compiles the .go files that are then given to shepherd, and i see nothing obvious. i even looked at the source file that gets generated and compiled for the service in question, and it doesn't contain any mention of this module. there are no direct mentions of (shepherd support) in the source, that's why i thought maybe something re-exports the entire module, so i checked the presence of 'ringbuffer in all the used modules... but it's not in any of them. could it be a bug in how different versions/instances of guile serialize/load a .go file? or could it be due to the module magic in (gnu services shepherd) that compiles the shepherd config into a .go file? i'm out of ideas, any hint is appreciated! -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “People always find partners [with] exactly the same level of unresolved traumas they are at, without any exception. Now the trauma may look differently, and how it manifests may look differently, but the degree of trauma is always equal, I mean with perfect accuracy. […] And that trauma then shows up in the relationship.” — Gábor Máté (1944–), 'On The Spot - Az ellenség gyermekei', 48m50s
Re: Syntactic Diabetes (was Re: A friendlier API for operating-system declarations)
> (service+ OS SERVICE [CONF]) > (service- OS SERVICE) > (modify-service OS SERVICE UPDATE) what would the benefit of generating multiple macros for each service compared to the above functional API (with 3-4 elements altogether)? i could be missing something here, but it seems to be precious little to me while it costs some extra complexity. if i were to add a syntactic abstraction for this, i'd generate a full DSL in the form of a (modify-operating-system OS [fictional DSL to describe desired changes]). but i don't think the extra complexity justifies any macrology here. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- By the time a man realises that his father was right, he has a son who thinks he’s wrong.