Your message dated Fri, 28 Aug 2026 06:18:58 +0000
with message-id <[email protected]>
and subject line Bug#1142756: fixed in chrony 4.9-1
has caused the Debian Bug report #1142756,
regarding chrony: NTP sources left permanently offline after link flap;
networkd-dispatcher hook runs before the interface is usable
to be marked as done.
This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.
(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact [email protected]
immediately.)
--
1142756: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1142756
Debian Bug Tracking System
Contact [email protected] with problems
--- Begin Message ---
Package: chrony
Version: 4.8-4
Severity: normal
chrony's networkd-dispatcher integration can leave NTP sources
permanently offline after a transient network outage. debian/links
symlinks 20-chrony-onoffline into networkd-dispatcher's routable.d and
off.d only. On a link flap the dispatched states are "no-carrier" and
then "routable" - "off" is never dispatched at all, so routable.d is
the only hook that ever runs, and it runs while the interface is still
being configured. "chronyc onoffline" consults the routing table,
correctly finds no route for the affected address family, and marks
those sources offline. Nothing dispatches afterwards to re-evaluate,
so they are never brought back and chronyd free-runs indefinitely
logging "Can't synchronise: no selectable sources". On my host that
persisted for 16.5 hours, until I ran "chronyc online" by hand.
I observed this on Ubuntu 26.04 with chrony 4.8-2ubuntu1, but I am
filing here because the packaging is Debian's: I checked debian/links
in the current Debian source package and it carries the same two
symlinks with no configured.d entry. Everything below was captured on
that host - an instrumented reproduction with per-state dispatcher
logging, and a verification that adding the missing configured.d
symlink resolves it. I have also raised this upstream on chrony-users,
reviving a June 2023 thread that reached the same diagnosis.
STEPS TO REPRODUCE
==================
On a networkd host running chrony, cause carrier loss followed by a
slow DHCP re-acquisition - e.g. power-cycle the upstream router. In my
case a router firmware update caused ~36 s of carrier flapping, and
DHCP did not complete for a further 2m15s.
OBSERVED
========
systemd-networkd (times UTC; LAN addresses redacted):
06:19:10 eth0: Lost carrier
06:19:10 eth0: DHCP lease lost
06:19:13 eth0: Gained carrier
06:19:19 eth0: Lost carrier
06:19:42 eth0: Gained carrier
06:19:44 eth0: Lost carrier
06:19:46 eth0: Gained carrier
06:22:01 eth0: DHCPv4 address 192.168.x.x/22, gateway 192.168.x.1
acquired from 192.168.x.1
chronyd:
06:21:36 Source 185.125.190.123 offline
06:21:36 Source 91.189.91.112 offline
06:21:36 Source 2620:2d:4000:1::1123 offline
06:21:36 Source 91.189.91.113 offline
06:21:36 Can't synchronise: no selectable sources (5 unreachable sources)
06:21:36 Source 2620:2d:4000:1::2123 offline
Note the ordering: chronyd marked all five sources offline at
06:21:36, twenty-five seconds BEFORE the interface acquired an address
and gateway at 06:22:01. Nothing ran the hook again afterwards.
State 16.5 hours later, before any manual intervention:
$ chronyc tracking
Reference ID : 45AE48D5 (ntp-nts-2.ps5.canonical.com)
Ref time (UTC) : Fri Jul 24 06:15:09 2026
System time : 0.000144547 seconds slow of NTP time
Leap status : Normal
$ timedatectl show -p NTPSynchronized --value
no
"chronyc -n sources" at that point showed all five sources as '^?'
(unusable), Reach 0, LastRx 16h. "sudo chronyc online" restored them
immediately, and NTPSynchronized returned to "yes" within one poll.
DELIBERATE REPRODUCTION
=======================
I reproduced this by physically disconnecting and reconnecting the
Ethernet cable, with networkd-dispatcher at increased verbosity (so
invoked hooks are named) and a logging script installed in every state
directory:
#!/bin/sh
logger -t dispatch-debug "STATE=$STATE IFACE=$IFACE
Admin=$AdministrativeState Oper=$OperationalState"
Full sequence (trimmed to the relevant units; addresses redacted):
04:06:46 kernel: eth0: Link is Down
04:06:46 systemd-networkd: eth0: Lost carrier
04:06:46 systemd-networkd: eth0: DHCP lease lost
04:06:46 networkd-dispatcher: Invoking '.../no-carrier.d/00-debug'
for interface eth0
04:06:46 dispatch-debug: STATE=no-carrier Admin=configured
Oper=no-carrier
04:07:41 kernel: eth0: Link is Up - 100Mbps/Full
04:07:41 systemd-networkd: eth0: Gained carrier
04:07:41 networkd-dispatcher: Invoking '.../degraded.d/00-debug' for
interface eth0
04:07:41 dispatch-debug: STATE=degraded Admin=configured
Oper=degraded
04:07:41 networkd-dispatcher: Invoking '.../configuring.d/00-debug'
for interface eth0
04:07:41 dispatch-debug: STATE=configuring Admin=configuring
Oper=degraded
04:07:42 networkd-dispatcher: Invoking '.../routable.d/00-debug' for
interface eth0
04:07:42 dispatch-debug: STATE=routable Admin=configuring
Oper=routable
04:07:42 networkd-dispatcher: Invoking
'.../routable.d/chrony-onoffline' for interface eth0
04:07:43 networkd-dispatcher: Invoking '.../configured.d/00-debug'
for interface eth0
04:07:43 dispatch-debug: STATE=configured Admin=configured
Oper=routable
04:07:45 systemd-networkd: eth0: DHCPv4 address 192.168.x.x/22,
gateway 192.168.x.1 acquired
04:07:45 networkd-dispatcher: Invoking '.../configuring.d/00-debug'
for interface eth0
04:07:45 dispatch-debug: STATE=configuring Admin=configuring
Oper=routable
04:07:45 networkd-dispatcher: Invoking '.../configured.d/00-debug'
for interface eth0
04:07:45 dispatch-debug: STATE=configured Admin=configured
Oper=routable
This establishes four things:
1. On carrier loss the dispatched state is "no-carrier", not "off".
chrony's off.d symlink is never invoked by a link flap. The only
chrony symlink that executes in this entire sequence is
routable.d/chrony-onoffline.
2. "routable" is dispatched with AdministrativeState=configuring, at
04:07:42 - three seconds before DHCPv4 supplies an address and gateway
at 04:07:45. Note that OperationalState reads "routable" from 04:07:42
onward, including at both "configured" dispatches; what changes across
the sequence is AdministrativeState (configuring -> configured ->
configuring -> configured). The claim is therefore specific: the
dispatch chrony's hook is attached to fires while networkd's
ADMINISTRATIVE state is still "configuring". The interface is
operationally routable at that point via IPv6 - as the IPv4-only
offline set confirms - but IPv4 is not yet configured.
3. chrony's hook runs at exactly that moment, and nothing runs it
again. Because the only hook is on "routable", and "routable" precedes
IPv4 configuration, a link flap does not merely risk leaving IPv4
sources offline - it does so deterministically.
4. "configured" is dispatched at 04:07:45, immediately after the
DHCPv4 lease. A configured.d symlink would therefore be invoked with a
working default route present.
Point 2 answers the question raised upstream in 2023 ("Doesn't the
routable state follow configured?" - msg03269): the "routable"
dispatch precedes the post-DHCP "configured" dispatch.
Note there are no chronyd log lines in this window: the IPv4 sources
were already offline from an identical earlier test, so "chronyc
onoffline" produced no state change and logged nothing. The
verification capture further below shows the same
routable.d/chrony-onoffline invocation immediately followed by "Source
... offline" for all three IPv4 sources, with the two IPv6 sources
untouched - onoffline correctly reading the routing table as
IPv6-routable and IPv4-not-yet.
The defect is not in onoffline's logic. It is that the only dispatch
wired to it fires before IPv4 configuration completes, and nothing
re-runs it afterwards.
Caveat on ordering: the sequence is not monotonic - "configuring" is
dispatched again at 04:07:45 after a "configured" at 04:07:43. On a
flapping link states can re-enter. In this capture "configured" is
dispatched last and after DHCP, so the proposed symlink would have
restored the sources; but it does not by itself guarantee the hook
always runs last.
ANALYSIS
========
/usr/lib/NetworkManager/dispatcher.d/20-chrony-onoffline guards its
action with "[ $# -ge 2 ]". networkd-dispatcher passes interface state
through environment variables (IFACE, STATE) and not positional
arguments, so that guard is always false and the script
unconditionally runs "chronyc onoffline". That command determines
online/offline from the routing table, so the outcome depends entirely
on WHEN it runs relative to address and route configuration.
debian/links, from the 4.8-2ubuntu1 source package, installs the
script into two networkd-dispatcher state directories only:
# Update sources in response to systemd-networkd events (LP: #1718227).
# This is reusing the NetworkManager dispatch script which has no hard
# dependency to NetworkManager (not using any of its arguments)
usr/lib/NetworkManager/dispatcher.d/20-chrony-onoffline
usr/lib/networkd-dispatcher/routable.d/chrony-onoffline
usr/lib/NetworkManager/dispatcher.d/20-chrony-onoffline
usr/lib/networkd-dispatcher/off.d/chrony-onoffline
(Identical in current Debian packaging. "dpkg -L chrony" on the
running system confirms these are the only two symlinks shipped.)
The comment makes clear the argument-less fall-through is deliberate
and understood. The gap is only that "configured" is not in the list,
so nothing runs the hook at the point where networkd reports the link
fully configured - i.e. after DHCP has supplied an address and
gateway.
VERIFICATION OF THE PROPOSED FIX
================================
I added the missing symlink locally:
/usr/lib/networkd-dispatcher/configured.d/50-chrony-onoffline-local
-> /usr/lib/NetworkManager/dispatcher.d/20-chrony-onoffline
and repeated the cable disconnect/reconnect:
04:25:59 systemd-networkd: eth0: Lost carrier
04:25:59 systemd-networkd: eth0: DHCP lease lost
04:25:59 networkd-dispatcher: Invoking '.../no-carrier.d/00-debug'
for interface eth0
04:25:59 dispatch-debug: STATE=no-carrier Admin=configured
Oper=no-carrier
04:26:39 systemd-networkd: eth0: Gained carrier
04:26:39 dispatch-debug: STATE=degraded Admin=configured
Oper=degraded
04:26:40 dispatch-debug: STATE=configuring Admin=configuring
Oper=degraded
04:26:42 dispatch-debug: STATE=routable Admin=configuring
Oper=routable
04:26:42 networkd-dispatcher: Invoking
'.../routable.d/chrony-onoffline' for interface eth0
04:26:42 chronyd: Source 185.125.190.123 offline
04:26:42 chronyd: Source 91.189.91.112 offline
04:26:42 chronyd: Source 91.189.91.113 offline
04:26:42 dispatch-debug: STATE=configured Admin=configured
Oper=routable
04:26:42 networkd-dispatcher: Invoking
'.../configured.d/50-chrony-onoffline-local' for interface eth0
04:26:44 systemd-networkd: eth0: DHCPv4 address 192.168.x.x/22,
gateway 192.168.x.1 acquired
04:26:44 dispatch-debug: STATE=configuring Admin=configuring
Oper=routable
04:26:44 dispatch-debug: STATE=configured Admin=configured
Oper=routable
04:26:44 networkd-dispatcher: Invoking
'.../configured.d/50-chrony-onoffline-local' for interface eth0
04:26:44 chronyd: Source 185.125.190.123 online
04:26:44 chronyd: Source 91.189.91.112 online
04:26:44 chronyd: Source 91.189.91.113 online
(The local symlink is named 50-chrony-onoffline-local only to avoid
colliding with any future package-shipped file; it is the same
script.)
The defect still occurs: routable.d/chrony-onoffline runs at 04:26:42
and parks the three IPv4 sources. The added hook then runs twice -
once at 04:26:42, still before DHCP, where it correctly makes no
change; and again at 04:26:44 immediately after the DHCPv4 lease,
where it restores all three sources.
Exposure drops from indefinite to two seconds, unattended. The
harmless pre-DHCP invocation also demonstrates that running onoffline
on an additional state introduces no churn - it is self-determining
and idempotent.
This run again confirms that carrier loss dispatches "no-carrier", not
"off": chrony's off.d symlink is not invoked at any point.
Separately, "systemctl restart networkd-dispatcher" silently repairs
the condition via its startup triggers (observed on this host at
04:18:12, where the restart's replayed "routable" dispatch brought the
sources back online with a route present). That is likely one reason
the defect is rarely reported - any routine restart of the dispatcher,
or of chrony, clears it.
PRIOR REPORTS, AND WHY THIS IS NOT A FEATURE REQUEST
====================================================
This defect has been independently reported before. LP #2043967 (Paul
Gear, November 2023) describes the same mechanism with a different
trigger: after suspend/resume the dispatcher ran "chronyc onoffline"
before IPv6 interfaces had completed Duplicate Address Detection,
leaving those sources ignored "until chronyc onoffline is run again
after the interfaces are usable". That report was marked Incomplete
pending the Ubuntu release, chrony version and reproduction steps, and
expired in January 2024 without them. All three are supplied above.
Taken together, the two reports generalise the defect: DAD and DHCP
are simply two different reasons an interface can reach "routable"
before it is actually usable. The problem is specific to neither - it
is that "routable" does not imply usable, and it is the only state
with a chrony hook attached.
It is also worth addressing the framing of the earlier report, which
was characterised as a feature request. The shipped integration exists
specifically to keep chrony's sources in step with network
availability. On this host it reliably takes sources offline on a link
flap and provides no path to bring them back, producing a 16.5 hour
loss of time synchronisation that required manual intervention to
clear. The one-line addition below restores the integration's stated
purpose rather than extending it.
SUGGESTED FIX
=============
Add a configured.d symlink alongside the existing two, so the hook
also runs once networkd reports the link fully configured:
usr/lib/NetworkManager/dispatcher.d/20-chrony-onoffline
usr/lib/networkd-dispatcher/configured.d/chrony-onoffline
Since "chronyc onoffline" is self-determining and idempotent, running
it on an additional state is harmless - demonstrated above, where the
pre-DHCP invocation correctly made no change.
A more targeted fix - guarding the script on $STATE /
$AdministrativeState - was proposed upstream in 2023 (see references).
It has not been applied: examples/chrony.nm-dispatcher.onoffline
carries only the positional-argument guard both in the 4.8 source
tarball and in upstream git master, and is byte-identical to the
script shipped on this system.
A caveat on the symlink as a general fix: the 2023 thread did not
stall for want of evidence - Jongkyu Seok supplied four event-sequence
logs (msg03289). Those logs showed that the order in which
AdministrativeState and OperationalState change VARIES BETWEEN
ENVIRONMENTS, and that linking into configured.d could misfire on a
host where AdministrativeState becomes "configured" while
OperationalState is not yet routable. That is a legitimate objection
to the bare symlink as a universal solution, and no maintainer
response to those logs appears in the archive.
The evidence in this report should therefore be read as a demonstrated
fix FOR THIS CONFIGURATION and an additional data point on ordering,
not as a claim that the symlink is universally correct. Note that
adding it ALONGSIDE the existing routable.d link rather than replacing
it means both orderings receive a dispatch - though the resulting
state is still determined by whichever fires last. If the variability
Seok documented makes a placement-only fix untenable, the
guarded-script approach looks like the more robust route, and that is
a decision for upstream.
REFERENCES
==========
- chrony-users, Jongkyu Seok, June 2023 - same diagnosis, proposes the
configured.d script:
https://www.mail-archive.com/[email protected]/msg03268.html
- Miroslav Lichvar's reply asking for event-sequence examples:
https://www.mail-archive.com/[email protected]/msg03269.html
- Jongkyu Seok's reply supplying four event-sequence logs, and noting
that the ordering of AdministrativeState and OperationalState varies
between environments - the thread ends here with no maintainer
response in the archive:
https://www.mail-archive.com/[email protected]/msg03289.html
- Upstream examples/chrony.nm-dispatcher.onoffline in git master still
has only the "[ $# -ge 2 ]" guard.
- Debian #1093763 - a different defect in the same integration
("chronyc onoffline" returning non-zero aborting if-up.d, killing the
DHCP client), fixed in chrony 4.8-4 by checking chronyd.sock rather
than the PID file and ensuring hook scripts always exit successfully.
Note 4.8-4 postdates the 4.8-2ubuntu1 in Ubuntu 26.04.
- LP #2043967 - same defect, different trigger (IPv6 sources not
restored after suspend/resume, onoffline running before DAD
completed). Filed Nov 2023, marked Incomplete pending
release/version/reproduction, expired Jan 2024.
- LP #2155244 - related but distinct: 20-chrony-dhcp has no
networkd-dispatcher equivalent at all.
- LP #1718227 - origin of the current networkd-dispatcher symlinks,
cited in debian/links.
- LP #2132159 - recent: chrony now declares "Recommends:
networkd-dispatcher" directly, since systemd no longer does. Note this
means networkd-dispatcher is only a Recommends, so on installs without
it the hook never runs in either direction.
--- End Message ---
--- Begin Message ---
Source: chrony
Source-Version: 4.9-1
Done: Vincent Blut <[email protected]>
We believe that the bug you reported is fixed in the latest version of
chrony, which is due to be installed in the Debian FTP archive.
A summary of the changes between this version and the previous one is
attached.
Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to [email protected],
and the maintainer will reopen the bug report if appropriate.
Debian distribution maintenance software
pp.
Vincent Blut <[email protected]> (supplier of updated chrony package)
(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing [email protected])
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
Format: 1.8
Date: Thu, 27 Aug 2026 17:34:10 +0200
Source: chrony
Architecture: source
Version: 4.9-1
Distribution: unstable
Urgency: medium
Maintainer: Vincent Blut <[email protected]>
Changed-By: Vincent Blut <[email protected]>
Closes: 1142756
Changes:
chrony (4.9-1) unstable; urgency=medium
.
* Import upstream version 4.9:
- Please see /usr/share/doc/chrony/NEWS.gz for the release notes.
.
* Upload to unstable.
.
* debian/chrony.examples:
- Install two more example files.
.
* debian/chrony.links:
- Create symlinks for the new networkd-dispatcher script. Based on the
work done by Guilherme Puida Moreira in MR: !20. (Closes: #1142756)
.
chrony (4.9~pre1-1) experimental; urgency=medium
.
* Import upstream version 4.9-pre1:
- Please see /usr/share/doc/chrony/NEWS.gz for the release notes.
.
* debian/:
- Rename .gitlab-ci.yml to salsa-ci.yml.
.
* debian/changelog:
- Fix non-conformant date.
.
* debian/patches/:
- Drop test_-make-007-cmdmon-test-even-more-reliable.patch, applied
upstream.
.
* debian/tests/upstream-simulation-test-suite:
- Update clknetsim version.
Checksums-Sha1:
b5356609a4529c57866e16f84a704c0112ea3d8b 2802 chrony_4.9-1.dsc
a4a92a2dfacfc05406433065db5bde07452a244b 663809 chrony_4.9.orig.tar.gz
b01773bfa9e9e8e8c8963d44e9ff7e445c44c643 833 chrony_4.9.orig.tar.gz.asc
e9051609b3520209df3872a70d725dd48c67affe 43152 chrony_4.9-1.debian.tar.xz
b738e0b3fdc53e4be22000900a2a446d6fd4abe4 1296276 chrony_4.9-1.git.tar.xz
a318b3fd6f051fb35a3b64797e572e4db0ce1e08 17656 chrony_4.9-1_source.buildinfo
Checksums-Sha256:
ce1106563016f1b691f75c51d924d9b7cbd91be228bc62a73ca35423e67f561e 2802
chrony_4.9-1.dsc
4924c6f530105bcd5b9e9e33c48a2ae1bfd889222c8480bc41601110efc864d0 663809
chrony_4.9.orig.tar.gz
380025ebf659589bedde8b1462b8ff7289289752cdd6143c8d028cc85212b327 833
chrony_4.9.orig.tar.gz.asc
f8f39619b276cc3284dc6d8975a73a897c5c0d16b6bdbf3e34236d7544cd0bad 43152
chrony_4.9-1.debian.tar.xz
c3ce58683b2adcd8975023d52aa2a3bcd6709a1533f4e1804251faca9bad00c4 1296276
chrony_4.9-1.git.tar.xz
738de7748289637a4beb9a9de32660d9a593c9a1cfb4505d446855446721022a 17656
chrony_4.9-1_source.buildinfo
Files:
211dd3fba8baff824938fcf4963b7c5d 2802 net optional chrony_4.9-1.dsc
ea6f098ec4d1b805dbce92b7cd38cb6e 663809 net optional chrony_4.9.orig.tar.gz
d0b2215a29518679873dc00ad3ba838a 833 net optional chrony_4.9.orig.tar.gz.asc
5bd04bf2cfecf11452f1ae921ef39361 43152 net optional chrony_4.9-1.debian.tar.xz
2b969ea03dc8710103ef0ecc2df16c98 1296276 net None chrony_4.9-1.git.tar.xz
cc1f1313b0beb13c367028446ea5f2be 17656 net optional
chrony_4.9-1_source.buildinfo
Git-Tag-Info: tag=dbbc5370e4f3fe5815eac0f67804fddd92b72e1c
fp=1b939355ac4c35cadfbd739c06a328e00c6ace15
Git-Tag-Tagger: Paul Gevers <[email protected]>
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCgAdFiEEN02M5NuW6cvUwJcqYG0ITkaDwHkFAmqRJC0ACgkQYG0ITkaD
wHnRkg//aeQe4A4d9nY6MSrPUWvH+bfllF8EaKN1Q38WMev0w6k/9pjOG36dB4YT
SZ+sA8jdqJ1FYND/wqNRzCJ2Jc6QCJBjc5A7FDyzVcPOp5cCPzOK+2wZCf8+Byto
RTh5dZVEskQxP8wsKJLD0w36pn34sioejXoXJlUAprzpQdqzS23AfCKJo9UBLFWx
SA/Prr49hFcawTTikPpFscDkpsXsPmmwLD+ev7vIgUJJLqZwRW2x3J79lcheeD2n
Y9bYlS05/CzTYNfExi6bSpLkWacrzz4iZENgL/NSQV/czmE98+z8eR4iWQUldL8n
N3jO6QzTk7dqew6qts49A4VDpJ6a/1719EjeRAa3YtZ7sZwohycTCAeqozUzVRHV
XiIQhqvqFgABkL05fyaexbiHnvedoKVGDIwJzgiH2QGTH+o0h+ReK6lM0ehGwWW1
0JrqIr78ZN4WChQ2EiR+s67Aup3a1gSNljSTPNfJ/Nu1zV/BcHanox9M9VlNht4e
6b01DuM11DdYvEcMIPRvM7Yw5qgvYTdEcN7mMifIuxU0paJeFqHJm6OPF4l4ktBh
X8Y8Ipo/SHNmsVNyhAkCOt4zIftqr3znA3BpUDwUUPDvYUcyqGJYeaqUmXTzgZDO
mXsNeP0r83h1IopfHCFqXYmlx78t9IrfeMTyPjK5MwndHjLPVbQ=
=MJEo
-----END PGP SIGNATURE-----
pgp7IwMegfRSw.pgp
Description: PGP signature
--- End Message ---
