Re: Shepherd timers
Hi! Felix Lechner skribis: > While 'command' works and was in your initial example, I had trouble > tracking down the documentation for it. Is it a record entry destined > for 'shepherd-command'? >From the “Timers” section of the manual in the ‘devel’ branch: --8<---cut here---start->8--- -- Procedure: command LINE [#:user #f] [#:group #f] [#:environment-variables (default-environment-variables)] [#:directory (default-service-directory)] [#:resource-limits '()] Return a new command for LINE, a program name and argument list, to be executed as USER and GROUP, with the given ENVIRONMENT-VARIABLES, in DIRECTORY, and with the given RESOURCE-LIMITS. These arguments are the same as for ‘fork+exec-command’ and related procedures (*note ‘fork+exec-command’: exec-command.). --8<---cut here---end--->8--- > Spoiled by Guix's seamless Guile integration, I don't shell out much > anymore. Will 'command' accept a thunk? No, but ‘make-timer-constructor’ does: --8<---cut here---start->8--- -- Procedure: make-timer-constructor EVENT ACTION [#:wait-for-termination?] Return a procedure for use as the ‘start’ method of a service. The procedure will perform ACTION at every occurrence of ‘event’, a calendar event as returned by ‘calendar-event’. ACTION may be either a command (returned by ‘command’) or a thunk; in the latter case, the thunk must be suspendable or it could block the whole shepherd process. --8<---cut here---end--->8--- Ludo’.
Re: Shepherd timers
Hi Ludo' On Fri, Apr 19 2024, Ludovic Courtès wrote: > that’s already possible with the #:user argument of ‘command’. While 'command' works and was in your initial example, I had trouble tracking down the documentation for it. Is it a record entry destined for 'shepherd-command'? Spoiled by Guix's seamless Guile integration, I don't shell out much anymore. Will 'command' accept a thunk? Kind regards Felix
Re: Shepherd timers
Hi, Felix Lechner skribis: > On Wed, Apr 10 2024, Ludovic Courtès wrote: > >> Could you share this service or at least the (calendar-event …) bit? > > It was my error. The iota in > > (calendar-event #:minutes '(iota 12 3 5)) Oh, great that you found out. We should add checks to diagnose such issues earlier. >> Feedback & suggestions welcome! > > As a feature request, it would be nice to run timers as users other than > root. Good news: that’s already possible with the #:user argument of ‘command’. :-) > To encourage adoption of these timers among Guix user, I'll also share > some of my timers: > > Btrbk Snapshot [2] > Mdadm Resync service [3] > Guix Garbage Collector service [4] > Debbugs Config Sync service [5] > Debbugs Bug Data Sync service [6] > Borg Backup service [7] Nice, thanks for sharing! BTW, until 1.0 is released, the interface may still change. Just so you know! Ludo’.
Re: Shepherd timers
Hi Ludo' On Wed, Apr 10 2024, Ludovic Courtès wrote: > Could you share this service or at least the (calendar-event …) bit? It was my error. The iota in (calendar-event #:minutes '(iota 12 3 5)) should not have been quoted. The service definition was part of the message to which you responded. [1] My apologies. > Feedback & suggestions welcome! As a feature request, it would be nice to run timers as users other than root. The situation is somewhat aggravated because 'runuser' is not being built or shipped in 'util-linux' in order to avoid a dependency on linux-pam. The natural replacement su from the same package, which does nearly the same thing except it is setuid root, can only be found---as a working program---in the non-reproducible path /run/setuid-programs/su. The only other way may be to resort to the fork/wait functionality in GNU Guile. * * * To encourage adoption of these timers among Guix user, I'll also share some of my timers: Btrbk Snapshot [2] Mdadm Resync service [3] Guix Garbage Collector service [4] Debbugs Config Sync service [5] Debbugs Bug Data Sync service [6] Borg Backup service [7] A variation of the Mdadm Resync service should probably run on all Guix systems. It prevents bitrot on MD arrays from cosmic rays or heat by writing good copies of all sectors to all array elements once a month. The Borg Backup service currently requires a modified version of Guix to find SSH. Alternatives are the environment variable BORG_RSH or the command line variable --rsh, but I believe our version should be modified to find ssh for automated use. Kind regards Felix [1] https://lists.gnu.org/archive/html/guix-devel/2024-03/msg00256.html [2] https://codeberg.org/lechner/system-config/src/commit/db9edb46caf36fe15bc6f8abc5d1df184b6d5c5f/host/wallace-server/operating-system.scm#L664-L714 [3] https://codeberg.org/lechner/system-config/src/commit/db9edb46caf36fe15bc6f8abc5d1df184b6d5c5f/host/wallace-server/operating-system.scm#L716-L755 [4] https://codeberg.org/lechner/system-config/src/commit/db9edb46caf36fe15bc6f8abc5d1df184b6d5c5f/host/wallace-server/operating-system.scm#L757-L778 [5] https://codeberg.org/lechner/system-config/src/commit/db9edb46caf36fe15bc6f8abc5d1df184b6d5c5f/host/wallace-server/operating-system.scm#L780-L827 [6] https://codeberg.org/lechner/system-config/src/commit/db9edb46caf36fe15bc6f8abc5d1df184b6d5c5f/host/wallace-server/operating-system.scm#L829-L956 [7] https://codeberg.org/lechner/system-config/src/commit/db9edb46caf36fe15bc6f8abc5d1df184b6d5c5f/host/wallace-server/operating-system.scm#L556-L662
Re: Shepherd timers
Felix Lechner skribis: > The status seems a bit bungled, though. I now use a lambda, i.e. no > 'command', with a fresh pull from 'devel'. Thanks! [...] > Upcoming timer alarms: > Backtrace: >8 (primitive-load "/run/current-system/profile/bin/herd") > In shepherd/scripts/herd.scm: > 796:9 7 (main . _) > In ice-9/boot-9.scm: > 1747:15 6 (with-exception-handler # ice-9/boot-9.scm:1831:7 (exn)> _ #:unwind? _ # _) > In shepherd/scripts/herd.scm: >727:11 5 (_) > In srfi/srfi-1.scm: > 634:9 4 (for-each # args> ((service (version 0) (# (#)) # ?))) > In shepherd/scripts/herd.scm: >323:19 3 (display-timer-events #< seconds: (0) minutes: > (3 8 13 18 23 28 33 38 43 48 53 58) h?> ?) > In shepherd/service/timer.scm: >253:21 2 (next-calendar-event #< seconds: (0) minutes: (3 > 8 13 18 23 28 33 38 43 48 53 58) ho?> ?) >233:18 1 (week-days->month-days #f 4 _) > In unknown file: >0 (memv 1 #f) > > ERROR: In procedure memv: > In procedure memv: Wrong type argument in position 2 (expecting list): #f Could you share this service or at least the (calendar-event …) bit? Thanks, Ludo’.
Re: Shepherd timers
Hi! Felix Lechner skribis: > On Sun, Mar 24 2024, Ludovic Courtès wrote: > >> you can do anything you can do with a service: stop it, unload it, >> load a replacement, and so on. > > Wow, do I love those timers! I just converted my system Mcron jobs to > Shepherd timers. The user jobs are next. What a great new feature! :-) > One small suggestion: > > Could calendar-event please offer a symbolic interface for days-of-week? > I'm thinking of 'monday, 'tuesday, and so on instead of numbers. Sure, I can look into it. Maybe we’ll need to support both so that one can still more conveniently generate lists of weekdays. Thanks your feedback! Ludo’.
Re: Shepherd timers
Hi Ludo' On Sun, Mar 24 2024, Ludovic Courtès wrote: > you can do anything you can do with a service: stop it, unload it, > load a replacement, and so on. Wow, do I love those timers! I just converted my system Mcron jobs to Shepherd timers. The user jobs are next. What a great new feature! One small suggestion: Could calendar-event please offer a symbolic interface for days-of-week? I'm thinking of 'monday, 'tuesday, and so on instead of numbers. In Europe, the week starts on Mondays. In the US, it starts on Sundays. Cron and its descendants made an error in settling on a particular numbering. We can fix it now. With my tongue in cheek, at least one disputed but very old tradition we received recorded that the world was completed on a Friday, at some time before sunset. I could therefore see starting the count on Saturday or, if anyone would like to have a day of rest, on Sunday, but there is scant precedent for a start on Monday. That choice goes back less than a hundred years. It was only made official for some parts of the world in 1988. That was less than forty years ago. [1][2] A symbolic interface for the days of the week would, on the other hand, be more intuitive and equitable towards competiting traditions. Symbols would also avoid the ordering that elevates one of the weekdays into the first position and above its peers. I'd be happy to suggest a patch, if needed. Thanks for this great work! Kind regards Felix [1] https://www.timeanddate.com/calendar/days/first-day-of-the-week.html [2] https://www.calendarr.com/united-states/first-day-of-the-week/
Re: Shepherd timers
Hi Ludo' On Thu, Mar 28 2024, Ludovic Courtès wrote: > I start, restart, reload, restart, etc. and it seems fine. Okay, this works fine now. I had quoted the 'iota'. The status seems a bit bungled, though. I now use a lambda, i.e. no 'command', with a fresh pull from 'devel'. Thanks! Kind regards Felix * * * $ herd status rsync-debbugs Status of rsync-debbugs: It is running since 07:14:44 (4 minutes ago). Timed service. Periodically running Scheme code. It is enabled. Provides (rsync-debbugs). Requires (networking file-systems). Will be respawned. Recent messages: 2024-04-02 07:18:00 receiving incremental file list 2024-04-02 07:18:00 2024-04-02 07:18:00 sent 23 bytes received 1,244 bytes 844.67 bytes/sec 2024-04-02 07:18:00 total size is 144,672 speedup is 114.18 Upcoming timer alarms: Backtrace: 8 (primitive-load "/run/current-system/profile/bin/herd") In shepherd/scripts/herd.scm: 796:9 7 (main . _) In ice-9/boot-9.scm: 1747:15 6 (with-exception-handler # _ #:unwind? _ # _) In shepherd/scripts/herd.scm: 727:11 5 (_) In srfi/srfi-1.scm: 634:9 4 (for-each # ((service (version 0) (# (#)) # ?))) In shepherd/scripts/herd.scm: 323:19 3 (display-timer-events #< seconds: (0) minutes: (3 8 13 18 23 28 33 38 43 48 53 58) h?> ?) In shepherd/service/timer.scm: 253:21 2 (next-calendar-event #< seconds: (0) minutes: (3 8 13 18 23 28 33 38 43 48 53 58) ho?> ?) 233:18 1 (week-days->month-days #f 4 _) In unknown file: 0 (memv 1 #f) ERROR: In procedure memv: In procedure memv: Wrong type argument in position 2 (expecting list): #f
Re: Shepherd timers
Hi Dale, On Sun, Mar 31 2024, Adam Faiz wrote: > Could the Shepherd depend on mcron for the timer functionality? I know you your name isn't Dave. Sorry! Kind regards Felix
Re: Shepherd timers
Hi Adam, On Sun, Mar 31 2024, Adam Faiz wrote: > Could the Shepherd depend on mcron for the timer functionality? Since > mcron already handles scheduling commands to run at a certain time, > there would be no need to reimplement it again. I don't wish to speak for the Mcron maintainer Dave, who reads this list, or for Ludo', to whom your message was addressed, but the Shepherd's scheduler [1] strikes me as potentially superior as it may avoid this calendaring issue in Mcron. [2] Kind regards Felix [1] https://git.savannah.gnu.org/cgit/shepherd.git/tree/modules/shepherd/service/timer.scm?h=devel#n149 [2] https://lists.gnu.org/archive/html/bug-mcron/2023-12/msg0.html
Re: Shepherd timers
Hi Ludovic, I want to give my suggestion for the timer support in the Shepherd: Could the Shepherd depend on mcron for the timer functionality? Since mcron already handles scheduling commands to run at a certain time, there would be no need to reimplement it again. Mcron would just need to support handling user jobs dynamically, and the Shepherd implementing timers as a simple interface for it.
Re: Shepherd timers
Hello!
Felix Lechner skribis:
> The simple example worked great, but my more complex attempt to convert
> an mcron job seems to have failed. Do you know why?
>
> (define (rsync-debbugs-shepherd-service config)
> (shepherd-service
>(provision '(rsync-debbugs))
Nice use case. :-)
>> herd restart rsync-debbugs
> Service user-homes has been started.
> Starting service rsync-debbugs...
> Service rsync-debbugs started.
> Service rsync-debbugs running with value #< channel: #< getq:
> # getq-gc-counter: # 7fb467c1c980 value: 42> putq: #
> putq-gc-counter: #> event:
> #< seconds: (0) minutes: (iota 12 3 5) hours: (0 1 2 3 4 5 6
> 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23) days-of-month: (1 2 3 4 5 6
> 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31)
> months: (1 2 3 4 5 6 7 8 9 10 11 12) days-of-week: #f> action: #<
> arguments:
> ("/gnu/store/1g0gk9pcymgc0bb1dh115swakmh438p6-rsync-3.2.7/bin/rsync" "-4"
> "--delete" "-av" "--password-file=/root/secrets/var-lib-debbugs.rsync"
> "debbugs-...@debbugs.gnu.org::var-lib-debbugs" "/var/lib/debbugs/") user: #f
> group: #f environment-variables: ("HOME=/" "TERM=linux"
> "BOOT_IMAGE=/gnu/store/psn2jwyrvr8jxb1nrndxf3h7kg8vvfwa-linux-6.6.2/bzImage"
> "PATH=/gnu/store/2jbvnib5ar70q1vlhhngish1svqfry0j-e2fsck-static-1.46.4/sbin:/gnu/store/jlbzcgxxzhynv74ss2510b0x978srnf9-loadkeys-static-2.5.1/bin"
>
> "GUIX_LOCPATH=/gnu/store/5fmqijrs5f7vx8mc2q2pmq26yvhb74sm-glibc-utf8-locales-2.35/lib/locale")
> directory: "/" resource-limits: ()>>.
> Service rsync-debbugs has been started.
> #[STATUS] End time 2024-03-25 14:04:40, duration 1.081s
> (root@wallace-server.local)[/s/srv/patchwise]
>> herd status rsync-debbugs
> #[STATUS] End time 2024-03-25 14:07:41, duration 159.524s
So ‘herd status rsync-debbugs’ exited after 160s without printing
anything? What does /var/log/messages say?
I tried something similar on my laptop but it seems to work.
Namely, I have:
--8<---cut here---start->8---
(use-modules (shepherd service timer))
(define gc-timer
(service
'(gc)
#:start (make-timer-constructor
(calendar-event #:minutes '(0))
(command '("/run/current-system/profile/bin/guix" "gc" "-F2G")
#:user "ludo"))
#:stop (make-timer-destructor)
#:actions (list timer-trigger-action)))
(register-services (list gc-timer))
--8<---cut here---end--->8---
And then I do:
sudo herd load root that-file.scm
I start, restart, reload, restart, etc. and it seems fine.
Thanks for trying it out!
Ludo’.
Re: Shepherd timers
Hi, raingl...@riseup.net skribis: > Does it support RTC wakeup? No. I don’t have plans but it’d be nice to have. Ludo’.
Re: Shepherd timers
Hi Ludo',
On Sun, Mar 24 2024, Ludovic Courtès wrote:
> I pushed to the ‘devel’ branch of the Shepherd a new module that
> implements “timers”
Wow, that's super exciting! I promptly reconfigured my channels to pull
in the Shepherd branch. (Anyone who wants to do the same, please reply
here for info.) For me, Shepherd timers will allow the temporary
stoppage of individual cron jobs.
For a server that can make debugging much easier.
> (command '("sh" "-c" "echo Hi from $PWD.; sleep 20; echo done")))
The simple example worked great, but my more complex attempt to convert
an mcron job seems to have failed. Do you know why?
--8<---cut here---start->8---
(define (rsync-debbugs-shepherd-service config)
(shepherd-service
(provision '(rsync-debbugs))
(requirement '(networking file-systems))
(modules '((shepherd service timer)
(srfi srfi-1)))
(start #~(make-timer-constructor
(calendar-event #:minutes '(iota 12 3 5))
(command `(,#$(file-append rsync "/bin/rsync")
"-4"
"--delete"
"-av"
"--password-file=/root/secrets/var-lib-debbugs.rsync"
"debbugs-...@debbugs.gnu.org::var-lib-debbugs"
"/var/lib/debbugs/"
(stop #~(make-timer-destructor))
(documentation "Timer to rsync Debbugs bug details to this machine.")))
(define rsync-debbugs-service-type
(service-type
(name 'rsync-debbugs)
(description
"Rsync Debbugs bug details to this machine.")
(extensions
(list
(service-extension shepherd-root-service-type
(compose list rsync-debbugs-shepherd-service
(default-value #f)))
--8<---cut here---end--->8---
and then later
--8<---cut here---start->8---
(service rsync-debbugs-service-type)
--8<---cut here---end--->8---
At the bottom of this message is the log from my Eshell session. The
first funny thing was that the old timer info still showed in 'herd
status'. That's probably because some services are not reloaded
automatically. I can never figure out which are and which aren't.
More important, however, I was unable to get status information from the
more complex timer service. What did I do wrong, please?
Thanks for this excellent work, especially in calender-event. The
Shepherd is a super cute piece of software!
Kind regards
Felix
* * *
guix deploy: successfully deployed wallace-server
#[STATUS] End time 2024-03-25 14:02:35, duration 86.932s
[~]
> cd /ssh:root@wallace-server.local:/srv/patchwise
(root@wallace-server.local)[/s/srv/patchwise]
> herd status rsync-debbugs
Status of rsync-debbugs:
It is running since 13:44:40 (19 minutes ago).
Timed service.
Periodically running: sh -c "echo Hi from $PWD.; sleep 20; echo done".
It is enabled.
Provides (rsync-debbugs).
Requires (networking file-systems).
Will be respawned.
Recent runs:
2024-03-25 14:03:06 Process exited with code 1.
2024-03-25 14:03:14 Process exited with code 1.
2024-03-25 14:03:21 Process exited with code 1.
2024-03-25 14:03:29 Process exited with code 1.
2024-03-25 14:03:44 Process exited with code 1.
Recent messages:
2024-03-25 14:03:44 exec of "sh" failed: No such file or directory
2024-03-25 14:03:44
2024-03-25 14:03:44 Some deprecated features have been used. Set the
environment
2024-03-25 14:03:44 variable GUILE_WARN_DEPRECATED to "detailed" and rerun the
2024-03-25 14:03:44 program to get more information. Set it to "no" to
suppress
2024-03-25 14:03:44 this message.
Upcoming timer alarms:
14:04:00 (in 15 seconds)
14:04:07 (in 22 seconds)
14:04:15 (in 30 seconds)
14:04:22 (in 37 seconds)
14:04:30 (in 45 seconds)
#[STATUS] End time 2024-03-25 14:03:45, duration 1.430s
(root@wallace-server.local)[/s/srv/patchwise]
> herd restart rsync-debbugs
Service user-homes has been started.
Starting service rsync-debbugs...
Service rsync-debbugs started.
Service rsync-debbugs running with value #< channel: #< getq:
# getq-gc-counter: # putq: #
putq-gc-counter: #> event:
#< seconds: (0) minutes: (iota 12 3 5) hours: (0 1 2 3 4 5 6 7
8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23) days-of-month: (1 2 3 4 5 6 7 8
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31) months: (1
2 3 4 5 6 7 8 9 10 11 12) days-of-week: #f> action: #< arguments:
("/gnu/store/1g0gk9pcymgc0bb1dh115swakmh438p6-rsync-3.2.7/bin/rsync" "-4"
"--delete" "-av" "--password-file=/
Re: Shepherd timers
On 2024-03-24 22:20, Ludovic Courtès wrote:
> Hello Guix!
>
> I pushed to the ‘devel’ branch of the Shepherd a new module that
> implements “timers” along with ‘herd’ support to display information
> about them.
>
> It lets you provide configuration like this one:
>
> --8<---cut here---start->8---
> (use-modules (shepherd service timer))
>
> (define timer
> (service '(my-timer)
>#:start (make-timer-constructor
> (calendar-event #:seconds '(0 7 15 22 30 45))
> (command '("sh" "-c" "echo Hi from $PWD.; sleep 20; echo
> done")))
>#:stop (make-timer-destructor)))
>
> (register-services (list timer))
> (start-in-the-background '(my-timer))
> --8<---cut here---end--->8---
>
> And then ‘my-timer’ invokes the given command at the moments that match
> the constraints defined by ‘calendar-event’—in this case any time the
> number of seconds is equal to 0, 7, 15, 22, 30, or 45. You can also
> make it every Monday at 9AM etc., as you would expect.
>
> The ‘herd’ command provides details information about the timer:
>
> --8<---cut here---start->8---
> $ ./herd -s sock status my-timer
> Status of my-timer:
> It is running since 21:09:32 (68 seconds ago).
> Timed service.
> Periodically running: sh -c "echo Hi from $PWD.; sleep 20; echo done".
> Child process: 1814
> It is enabled.
> Provides (my-timer).
> Requires ().
> Will not be respawned.
>
> Recent runs:
> 2024-03-24 21:10:04 Process exited successfully.
> 2024-03-24 21:10:19 Process exited successfully.
> 2024-03-24 21:10:26 Process exited successfully.
> 2024-03-24 21:10:34 Process exited successfully.
> 2024-03-24 21:10:35 Process terminated with signal 15.
>
> Recent messages:
> 2024-03-24 21:10:29 Hi from /home/ludo.
>
> Upcoming timer alarms:
> 21:10:45 (in 5 seconds)
> 21:11:00 (in 20 seconds)
> 21:11:07 (in 27 seconds)
> 21:11:15 (in 35 seconds)
> 21:11:22 (in 42 seconds)
> --8<---cut here---end--->8---
>
> And of course you can do anything you can do with a service: stop it,
> unload it, load a replacement, and so on.
>
> Feedback & suggestions welcome!
>
> Ludo’.
Does it support RTC wakeup?
Shepherd timers
Hello Guix!
I pushed to the ‘devel’ branch of the Shepherd a new module that
implements “timers” along with ‘herd’ support to display information
about them.
It lets you provide configuration like this one:
--8<---cut here---start->8---
(use-modules (shepherd service timer))
(define timer
(service '(my-timer)
#:start (make-timer-constructor
(calendar-event #:seconds '(0 7 15 22 30 45))
(command '("sh" "-c" "echo Hi from $PWD.; sleep 20; echo
done")))
#:stop (make-timer-destructor)))
(register-services (list timer))
(start-in-the-background '(my-timer))
--8<---cut here---end--->8---
And then ‘my-timer’ invokes the given command at the moments that match
the constraints defined by ‘calendar-event’—in this case any time the
number of seconds is equal to 0, 7, 15, 22, 30, or 45. You can also
make it every Monday at 9AM etc., as you would expect.
The ‘herd’ command provides details information about the timer:
--8<---cut here---start->8---
$ ./herd -s sock status my-timer
Status of my-timer:
It is running since 21:09:32 (68 seconds ago).
Timed service.
Periodically running: sh -c "echo Hi from $PWD.; sleep 20; echo done".
Child process: 1814
It is enabled.
Provides (my-timer).
Requires ().
Will not be respawned.
Recent runs:
2024-03-24 21:10:04 Process exited successfully.
2024-03-24 21:10:19 Process exited successfully.
2024-03-24 21:10:26 Process exited successfully.
2024-03-24 21:10:34 Process exited successfully.
2024-03-24 21:10:35 Process terminated with signal 15.
Recent messages:
2024-03-24 21:10:29 Hi from /home/ludo.
Upcoming timer alarms:
21:10:45 (in 5 seconds)
21:11:00 (in 20 seconds)
21:11:07 (in 27 seconds)
21:11:15 (in 35 seconds)
21:11:22 (in 42 seconds)
--8<---cut here---end--->8---
And of course you can do anything you can do with a service: stop it,
unload it, load a replacement, and so on.
Feedback & suggestions welcome!
Ludo’.
