Re: [systemd-devel] Antw: [EXT] [systemd‑devel] Query degraded state

[Barry]

> On 29 Apr 2022, at 07:06, Ulrich Windl  
> wrote:
> 
> 
>> 
 Barry Scott  schrieb am 28.04.2022 um 17:03 in
> Nachricht :
>> Is there a command I can use to test for the degraded state?
>> 
>> I could parsing the output of systemctl‑failed or
>> syetemctl status but was looking for something less fragile.
> 
> Do you mean "systemctl --state=failed list-units"?

I use systemctl —failed

Barry

> 
>> 
>> Barry
> 
> 
> 

[systemd-devel] Antw: [EXT] Re: Starting transient services securely from other service without root

[Ulrich Windl]

>>> Vašek Šraier  schrieb am 28.04.2022 um 19:47 in
Nachricht
:
> On Thu, 2022-04-28 at 19:53 +0300, Mantas Mikulėnas wrote:
...
> 
> Interesting. The issue also seems to be quite old meaning it's probably
> not a problem in practise.
...

That's a cool definition! So for any bug just sit and wait ;-)

...

[systemd-devel] Antw: [EXT] [systemd‑devel] Query degraded state

[Ulrich Windl]

>>> Barry Scott  schrieb am 28.04.2022 um 17:03 in
Nachricht :
> Is there a command I can use to test for the degraded state?
> 
> I could parsing the output of systemctl‑failed or
> syetemctl status but was looking for something less fragile.

Do you mean "systemctl --state=failed list-units"?

> 
> Barry

[systemd-devel] Antw: Re: Antw: [EXT] Re: [systemd‑devel] Splitting sd‑boot from systemd/bootctl for enabling sd‑boot in Fedora

[Ulrich Windl]

>>> Ian Pilcher  schrieb am 28.04.2022 um 16:40 in 
>>> Nachricht
:
> On 4/28/22 05:30, Ulrich Windl wrote:
>> So are there any distros that have /etc/fstab in initrd?
>> Having to start mount units manually is just terrible when a simple "mount
>> /var" would do.
> 
> Putting /etc/fstab in the initrd would mean that it would need to be
> rebuilt every time that file (or a plugin in /etc/fstab.d, or a mount
> unit) changed.

The environments I have here have fstabs that don't change for years, while 
kernel updates happen monthly at least.
Even more important, it's not necessary that the fstab in initrd is always up 
to date; it only needs the entries for the base OS (sometimes called the TCB).
Once you have the environment that allows a "mkinitrd" consistently you can fix 
all the other problems.

Regards,
Ulrich Windl

> 
> -- 
> 
> Google  Where SkyNet meets Idiocracy
> 

Re: [systemd-devel] Query degraded state

[Barry Scott]

On 28/04/2022 16:15, Alvin Šipraga wrote:

Hi,

On Thu, Apr 28, 2022 at 04:03:01PM +0100, Barry Scott wrote:

Is there a command I can use to test for the degraded state?

I could parsing the output of systemctl-failed or
syetemctl status but was looking for something less fragile.

I think the correct way to do this programatically is via the dbus API, but if
you want something moderately greppable, try:

# systemctl show | grep SystemState
SystemState=running


Thanks that will work well in a bash script.

I put off using dbus for a little longer :-)

Barry



Hope that helps.

Kind regards,
Alvin

Re: [systemd-devel] Starting transient services securely from other service without root

[Mantas Mikulėnas]

On Thu, Apr 28, 2022 at 6:56 PM Vašek Šraier  wrote:

> To update the current list of options:
>
> - PolicyKit
>   could technically help, but I've discovered that the documentation
>   explicitly prohibits our potential use-case:
>   "In particular, applications, [...] must never include any
>authorization rules."
>

That didn't stop many of them (including, apparently, systemd itself) from
doing so anyway.

$ pkgfile -vg '/usr/share/polkit-1/rules.d/*'
core/systemd 250.4-2
 /usr/share/polkit-1/rules.d/systemd-networkd.rules
extra/brltty 6.4-10
/usr/share/polkit-1/rules.d/org.a11y.brlapi.rules
extra/flatpak 1:1.12.7-1
 /usr/share/polkit-1/rules.d/org.freedesktop.Flatpak.rules
extra/geoclue 2.6.0-2
/usr/share/polkit-1/rules.d/org.freedesktop.GeoClue2.rules
extra/gnome-control-center 42.1-1
/usr/share/polkit-1/rules.d/gnome-control-center.rules
extra/gvfs 1.50.1-1
/usr/share/polkit-1/rules.d/org.gtk.vfs.file-operations.rules
extra/lightdm 1:1.30.0-4
 /usr/share/polkit-1/rules.d/lightdm.rules
extra/malcontent 0.10.3-2
/usr/share/polkit-1/rules.d/com.endlessm.ParentalControls.rules
extra/polkit 0.120-5
 /usr/share/polkit-1/rules.d/50-default.rules
community/bolt 0.9.2-1
 /usr/share/polkit-1/rules.d/org.freedesktop.bolt.rules
community/fwupd 1.7.7-1
/usr/share/polkit-1/rules.d/org.freedesktop.fwupd.rules
community/gnome-initial-setup 41.4-1
 /usr/share/polkit-1/rules.d/20-gnome-initial-setup.rules
community/libvirt 1:8.2.0-4
/usr/share/polkit-1/rules.d/50-libvirt.rules
community/libvirt-dbus 1.4.1-2
 /usr/share/polkit-1/rules.d/libvirt-dbus.rules
community/packagekit 1.2.5-1
 /usr/share/polkit-1/rules.d/org.freedesktop.packagekit.rules

I found a bugzilla about this:
https://bugs.freedesktop.org/show_bug.cgi?id=80921

-- 
Mantas Mikulėnas

Re: [systemd-devel] Query degraded state

[Alvin Šipraga]

Hi,

On Thu, Apr 28, 2022 at 04:03:01PM +0100, Barry Scott wrote:
> Is there a command I can use to test for the degraded state?
> 
> I could parsing the output of systemctl-failed or
> syetemctl status but was looking for something less fragile.

I think the correct way to do this programatically is via the dbus API, but if
you want something moderately greppable, try:

# systemctl show | grep SystemState
SystemState=running

Hope that helps.

Kind regards,
Alvin

[systemd-devel] Query degraded state

[Barry Scott]

Is there a command I can use to test for the degraded state?

I could parsing the output of systemctl-failed or
syetemctl status but was looking for something less fragile.

Barry

Re: [systemd-devel] Antw: [EXT] Re: [systemd‑devel] Splitting sd‑boot from systemd/bootctl for enabling sd‑boot in Fedora

[Ian Pilcher]

On 4/28/22 05:30, Ulrich Windl wrote:

So are there any distros that have /etc/fstab in initrd?
Having to start mount units manually is just terrible when a simple "mount
/var" would do.


Putting /etc/fstab in the initrd would mean that it would need to be
rebuilt every time that file (or a plugin in /etc/fstab.d, or a mount
unit) changed.

--

Google  Where SkyNet meets Idiocracy

Re: [systemd-devel] [systemd‑devel] Antw: [EXT] Re: Q: non‑ASCII in syslog

[Mantas Mikulėnas]

On Thu, Apr 28, 2022 at 1:26 PM Ulrich Windl <
ulrich.wi...@rz.uni-regensburg.de> wrote:

> >>> Lennart Poettering  schrieb am 28.04.2022 um
> 10:27
> in
> Nachricht :
> > On Do, 28.04.22 09:32, Ulrich Windl (ulrich.wi...@rz.uni‑regensburg.de)
> wrote:
> >
> >> Actually I wasn't quite sure about the default config in SLES12.
> >> It seems the flow is journald ‑> local rsyslogd ‑> remote syslogd
> >>
> >> > rsyslogd already knows if messages are UTF‑8 because the system's
> $LANG
> >> > (well, nl_langinfo) says so. And if rsyslog can't trust that for some
> >> > reason (e.g. because a user might have a different locale), then
> >> > systemd‑journald won't be able to trust it either, so it won't know
> whether
> >> > it could add the BOM.
> >>
> >> How could a remote syslog server know what the locale on the sending
> system
> >> is?
> >
> > Your local rsyslogd could add the BOM when it transforms journal
> > messages to syslog datagrams.
> >
> >> > RFC 3164 over the network to a remote server? Outside the scope for
> >> > systemd, since it doesn't generate the network packets; your local
> rsyslogd
> >> > forwarder does. (Also, why RFC 3164 and not 5425?)
> >>
> >> If you look outside the world of systemd, about 99% of systems create
> the
> > RFC
> >> 3164 type of messages.
> >
> > That's a wild claim, and simply wrong actually.
>
> Well actually as systemd cannot send syslog messages to remote, which
> systems
> do you know that send RFC 5424 messages?
> Actually I know none here.
>

syslog-ng does with destination{syslog()}, rsyslogd does with
RSYSLOG_SyslogProtocol23Format; the HP switches at $WORK (and I think the
Cisco ones) didn't even have BSD-format as an option, always producing
5424-format.


> >
> > systemd is focussed on reality: we generate and process the same
> > format glibc generates.
>
> I'm wondering which API all those programs use that create correct syslog
> entries.
>

It's not that they create correct syslog entries, it's that the syslogd
(well, the /dev/log listener, so including journald) *parses and rebuilds*
the entries that come from the API before storing them anywhere.

Whether you use rsyslog or syslog-ng, they don't just dump program-provided
data to /var/log – they both parse the input into date + hostname + pid +
message, then reformat according to whatever output format is specified.
(For example, we have syslog-ng configured to write RFC3339 timestamps.)
Journald also does the same by design.

-- 
Mantas Mikulėnas

[systemd-devel] Antw: [EXT] Re: [systemd‑devel] Waiting for all jobs to finish

[Ulrich Windl]

>>> Lennart Poettering  schrieb am 28.04.2022 um 10:47
in
Nachricht :
> On Mi, 27.04.22 18:29, Barry Scott (barry@barrys‑emacs.org) wrote:
> 
>> I have two target files that I use.
>>
>> prod.target is used to start up all the production services.
>> When I use systemctl start prod.target it blocks until all the
>> services are running.
>>
>> I also have prod‑upgrade.target that is used to stop service
>> via Conflicts=. When I use systemctl start prod‑upgrade.target
>> it returns immediately but there are stop jobs running.
>>
>> I believe that this is expected as all the services that need to
>> be started have been.
>>
>> Is there a systemctl command that will wait for all the stop jobs
>> without the need to poll for the systemctl list‑jobs to be empty?
> 
> There is not.
> 
> But the correct way to solve this is by combining Conflicts= with an
> order dep (After= or Before=).
> 
> In systemd the ordering deps After=/Before= control three things:
> 
> 1. The literally define the start‑up order if two units are started,
>i.e. this is the obvious case: if b.service has After=a.service
>this means a.service has to finish startup first, before b.service
>is started.
> 
> 2. If two units are stopped they define the order too, but in
>reverse. if b.service has After=a.service this hence means that if
>both are shutdown, then b.service has to stop *before* a.service is
>stopped, i.e. the opposite order of the start‑up order.
> 
> 3. If one unit is started and one is stopped then the existance of an
>ordering dep means the stopped unit must complete stopping first,
>before the starting of the other is initiated. Note that it doesn't
>mattre if you set After= or Before= here, that doesn#t matter. What
>matters is that you ordered the unit at all, regardless in which
>direction.
> 
> Now, this third rule is what matters here: if your target unit has
> Conflicst= on some service, then the target unit should not enter
> active state until the service fully shutdown. Thus you can place
> After= *or* Before= between the two (your choice) and get the desired
> behaviour.

Is there a tool to simulate actions? I mean: List the actions to be started
(not just "start", of course; "stop" is also "started") and the events to wait
for (events to trigger)?

> 
> Lennart
> 
> ‑‑
> Lennart Poettering, Berlin

Re: [systemd-devel] Antw: [EXT] Re: [systemd‑devel] Splitting sd‑boot from systemd/bootctl for enabling sd‑boot in Fedora

[Ulrich Windl]

>>> Lennart Poettering  schrieb am 28.04.2022 um 10:33
in
Nachricht :
> On Do, 28.04.22 10:25, Ulrich Windl (ulrich.wi...@rz.uni‑regensburg.de)
wrote:
> 
>> > Well, it sounds backwards to focus on the boot loader UI side of
>> > "recovery" so much if you don't even have any reasonably thing you
>> > could do in case of recovery better than a login prompt/shell...
>>
>> Well, not the shell, the tools are important:
>> Before systemd I could easily recover as system that failed booting (at
some
>> init stage), because I could easily mount the root filesystem and the
tools
>> were there.
>> With systemd I have a crippled minimum emergency environment where almost 
> all
>> required tools are absent (just es the real fstab is). That's one of the 
> first
>> and biggest frustrations with systemd.
> 
> That's a totally bogus claim. systemd has no control on the set of
> packages your distro installs or not. If you are missing some tool in
> your "emergency environment" (for whatever that is, systemd doesn't
> have a concept like that), then bring that up to your distro.
> 
> my educated guess is that your distro is providing some emergency
> kernel for you that comes with a minimized initrd? If that's the case
> it's purely the decision of your distro what to put in there and what
> not.

So are there any distros that have /etc/fstab in initrd?
Having to start mount units manually is just terrible when a simple "mount
/var" would do.

> 
> So you are barking up the very very wrong tree here. Go, complain to
> your distro instead, we have nothing to do with that.

OK.

> 
> Lennart
> 
> ‑‑
> Lennart Poettering, Berlin

[systemd-devel] Antw: Re: [systemd‑devel] Antw: [EXT] Re: Q: non‑ASCII in syslog

[Ulrich Windl]

>>> Lennart Poettering  schrieb am 28.04.2022 um 10:27
in
Nachricht :
> On Do, 28.04.22 09:32, Ulrich Windl (ulrich.wi...@rz.uni‑regensburg.de)
wrote:
> 
>> Actually I wasn't quite sure about the default config in SLES12.
>> It seems the flow is journald ‑> local rsyslogd ‑> remote syslogd
>>
>> > rsyslogd already knows if messages are UTF‑8 because the system's $LANG
>> > (well, nl_langinfo) says so. And if rsyslog can't trust that for some
>> > reason (e.g. because a user might have a different locale), then
>> > systemd‑journald won't be able to trust it either, so it won't know
whether
>> > it could add the BOM.
>>
>> How could a remote syslog server know what the locale on the sending
system
>> is?
> 
> Your local rsyslogd could add the BOM when it transforms journal
> messages to syslog datagrams.
> 
>> > RFC 3164 over the network to a remote server? Outside the scope for
>> > systemd, since it doesn't generate the network packets; your local
rsyslogd
>> > forwarder does. (Also, why RFC 3164 and not 5425?)
>>
>> If you look outside the world of systemd, about 99% of systems create the 
> RFC
>> 3164 type of messages.
> 
> That's a wild claim, and simply wrong actually.

Well actually as systemd cannot send syslog messages to remote, which systems
do you know that send RFC 5424 messages?
Actually I know none here.

> 
> I am pretty sure that more than 50% of syslog messages generated on
> this earth probably are synthesized by glibc's syslog() API. And that
> turns out to be neither conformant to RFC 3164 nor to RFC 5425.

No idea. Can you give an example?

> 
> What glibc sends is close to RFC 3164 but omits one key field that
> isn't really optionally according to RFC 3164: the 'HOSTNAME' field.

Maybe the API is not used correctly. The RFC 3164 says:
"A relay will add a TIMESTAMP and SHOULD add a HOSTNAME as follows (...)"
So when sending to any remote syslog a HOSTNAME should be there.
(It's like a MTA adding a Message-ID (and other fields) if none is present)

Most notable the RFC seems to allow a missing hostname initially.

> 
> systemd is focussed on reality: we generate and process the same
> format glibc generates.

I'm wondering which API all those programs use that create correct syslog
entries.
I tried with my own program:
It sends:
connect(1, {sa_family=AF_LOCAL, sun_path="/dev/log"}, 110) = 0
sendto(1, "<31>Apr 28 11:08:32 iotwatch[239"..., 56, MSG_NOSIGNAL, NULL, 0) =
56

What's logged is:
Apr 28 11:08:32 host-name iotwatch[239...

Also from the syntax being sent by the application, one cannot really say
whether the hostname is missing.
Maybe the trick is that /dev/log is specified as source for _local_ syslog
messages (so that there's no reason or sense in supplying the local hostname).
Also I'm not sure whether the messages in /dev/log are covered by the RFC.

Regards,
Ulrich Windl

> 
> Lennart
> 
> ‑‑
> Lennart Poettering, Berlin

Re: [systemd-devel] Waiting for all jobs to finish

[Barry Scott]

> On 28 Apr 2022, at 09:47, Lennart Poettering  wrote:
> 
> On Mi, 27.04.22 18:29, Barry Scott (ba...@barrys-emacs.org 
> ) wrote:
> 
>> I have two target files that I use.
>> 
>> prod.target is used to start up all the production services.
>> When I use systemctl start prod.target it blocks until all the
>> services are running.
>> 
>> I also have prod-upgrade.target that is used to stop service
>> via Conflicts=. When I use systemctl start prod-upgrade.target
>> it returns immediately but there are stop jobs running.
>> 
>> I believe that this is expected as all the services that need to
>> be started have been.
>> 
>> Is there a systemctl command that will wait for all the stop jobs
>> without the need to poll for the systemctl list-jobs to be empty?
> 
> There is not.
> 
> But the correct way to solve this is by combining Conflicts= with an
> order dep (After= or Before=).
> 
> In systemd the ordering deps After=/Before= control three things:
> 
> 1. The literally define the start-up order if two units are started,
> i.e. this is the obvious case: if b.service has After=a.service
> this means a.service has to finish startup first, before b.service
> is started.
> 
> 2. If two units are stopped they define the order too, but in
> reverse. if b.service has After=a.service this hence means that if
> both are shutdown, then b.service has to stop *before* a.service is
> stopped, i.e. the opposite order of the start-up order.
> 
> 3. If one unit is started and one is stopped then the existance of an
> ordering dep means the stopped unit must complete stopping first,
> before the starting of the other is initiated. Note that it doesn't
> mattre if you set After= or Before= here, that doesn#t matter. What
> matters is that you ordered the unit at all, regardless in which
> direction.
> 
> Now, this third rule is what matters here: if your target unit has
> Conflicst= on some service, then the target unit should not enter
> active state until the service fully shutdown. Thus you can place
> After= *or* Before= between the two (your choice) and get the desired
> behaviour.

I'd have never reasoned my way to this solution.

It work as you described.

But only if I use After= as Before= got a cyclic deps error.

Here is the pattern I now have:

[Unit]
Description=prod.target

Wants=serv1.service
After=serv1.service


[Unit]
Description=prod-upgrade.target

Conflicts=prod.target
Conflicts=serv1.service

# Need a After= so that systemctl start prod-upgrade.target
# will block until conflicting services have stopped
After=serv1.service

Barry


> 
> Lennart
> 
> --
> Lennart Poettering, Berlin

[systemd-devel] Antw: Re: [EXT] Re: Q: non-ASCII in syslog

[Ulrich Windl]

>>> Mantas Mikulenas  schrieb am 28.04.2022 um 09:39 in
Nachricht
:
> On Thu, Apr 28, 2022 at 10:32 AM Ulrich Windl <
> ulrich.wi...@rz.uni-regensburg.de> wrote:
> 
>> >>> Mantas Mikulenas  schrieb am 27.04.2022 um 12:03 in
>> Nachricht
>> :
>> > On Wed, Apr 27, 2022 at 10:09 AM Ulrich Windl <
>> > ulrich.wi...@rz.uni-regensburg.de> wrote:
>> >
>> >> Hi!
>> >>
>> >> Having written an RFC 3164 compatible syslog daemon, I noticed that
>> systemd
>> >> created syslog messages with non-ASCII characters.
>> >> The problem is that a remote syslogd can hardly guess the correct
>> character
>> >> set (I'm using rsyslog to forward local messages to a remote server).
>> >>
>> >> Example of such message:
>> >> systemd-tmpfiles[3311]: [/usr/lib/tmpfiles.d/svnserve.conf:1] Line
>> >> references
>> >> path below legacy directory /var/run/, updating /var/run/svnserve →
>> >> /run/svnserve; please update the tmpfiles.d/ drop-in file accordingly.
>> >>
>> >> (The arrow is encoded as three bytes (\xe2\x86\x92))
>> >>
>> >> RFC 5425 syslog messages require the use of a BOM (%xEF.BB.BF) at the
>> >> beginning of a message if the message used UTF-8:
>> >>
>> >>   MSG = MSG-ANY / MSG-UTF8
>> >>   MSG-ANY = *OCTET ; not starting with BOM
>> >>   MSG-UTF8= BOM UTF-8-STRING
>> >>   BOM = %xEF.BB.BF
>> >>
>> >> Wouldn't it make sense to add such a BOM for RFC 3164 syslog messages
>> also
>> >> if
>> >> non-ASCII (i.e.: UTF-8) encoded characters are used?
>> >>
>> >
>> > RFC 3164 over a local socket from journald to local rsyslogd? The local
>>
>> Actually I wasn't quite sure about the default config in SLES12.
>> It seems the flow is journald -> local rsyslogd -> remote syslogd
>>
>> > rsyslogd already knows if messages are UTF-8 because the system's $LANG
>> > (well, nl_langinfo) says so. And if rsyslog can't trust that for some
>> > reason (e.g. because a user might have a different locale), then
>> > systemd-journald won't be able to trust it either, so it won't know
>> whether
>> > it could add the BOM.
>>
>> How could a remote syslog server know what the locale on the sending
system
>> is?
>>
> 
> It's not remote, it's local. I'm talking about the one that's receiving
> messages from journald on the same machine.
> 
> 
>>
>> >
>> > RFC 3164 over the network to a remote server? Outside the scope for
>> > systemd, since it doesn't generate the network packets; your local
>> rsyslogd
>> > forwarder does. (Also, why RFC 3164 and not 5425?)
>>
>> If you look outside the world of systemd, about 99% of systems create the
>> RFC
>> 3164 type of messages.
>> Some may send non-ASCII too, however.
>>
> 
> Still outside the scope of systemd. Systemd doesn't send RFC 3164 messages
> over the network, either.

Correct: It does not send, because it's unable to do so. That's why I used
rsyslogd.

> 
> 
>>
>> >
>> > Generally, if a message successfully decodes as UTF-8 then it's most
>> likely
>> > actual UTF-8 (and if UTF-8 decode fails then you fall back to
ISO8859-1).
>> > Various old systems get away with this without needing a UTF-8 BOM.
>>
>> Yes, you can just output what you received, hoping the messages will be
>> presented correctly.
>> I't just like sending 8-bit E-Mmail without a coding system or charset in
>> the
>> past.

What I meant to say was: Guessing the encoding is a bad concept.

>>
> 
> Which is not what I was saying, but sure, whatever.
> 
> -- 
> Mantas Mikulėnas

Re: [systemd-devel] Waiting for all jobs to finish

[Lennart Poettering]

On Mi, 27.04.22 18:29, Barry Scott (ba...@barrys-emacs.org) wrote:

> I have two target files that I use.
>
> prod.target is used to start up all the production services.
> When I use systemctl start prod.target it blocks until all the
> services are running.
>
> I also have prod-upgrade.target that is used to stop service
> via Conflicts=. When I use systemctl start prod-upgrade.target
> it returns immediately but there are stop jobs running.
>
> I believe that this is expected as all the services that need to
> be started have been.
>
> Is there a systemctl command that will wait for all the stop jobs
> without the need to poll for the systemctl list-jobs to be empty?

There is not.

But the correct way to solve this is by combining Conflicts= with an
order dep (After= or Before=).

In systemd the ordering deps After=/Before= control three things:

1. The literally define the start-up order if two units are started,
   i.e. this is the obvious case: if b.service has After=a.service
   this means a.service has to finish startup first, before b.service
   is started.

2. If two units are stopped they define the order too, but in
   reverse. if b.service has After=a.service this hence means that if
   both are shutdown, then b.service has to stop *before* a.service is
   stopped, i.e. the opposite order of the start-up order.

3. If one unit is started and one is stopped then the existance of an
   ordering dep means the stopped unit must complete stopping first,
   before the starting of the other is initiated. Note that it doesn't
   mattre if you set After= or Before= here, that doesn#t matter. What
   matters is that you ordered the unit at all, regardless in which
   direction.

Now, this third rule is what matters here: if your target unit has
Conflicst= on some service, then the target unit should not enter
active state until the service fully shutdown. Thus you can place
After= *or* Before= between the two (your choice) and get the desired
behaviour.

Lennart

--
Lennart Poettering, Berlin

Re: [systemd-devel] Antw: [EXT] Re: [systemd‑devel] Splitting sd‑boot from systemd/bootctl for enabling sd‑boot in Fedora

[Lennart Poettering]

On Do, 28.04.22 10:25, Ulrich Windl (ulrich.wi...@rz.uni-regensburg.de) wrote:

> > Well, it sounds backwards to focus on the boot loader UI side of
> > "recovery" so much if you don't even have any reasonably thing you
> > could do in case of recovery better than a login prompt/shell...
>
> Well, not the shell, the tools are important:
> Before systemd I could easily recover as system that failed booting (at some
> init stage), because I could easily mount the root filesystem and the tools
> were there.
> With systemd I have a crippled minimum emergency environment where almost all
> required tools are absent (just es the real fstab is). That's one of the first
> and biggest frustrations with systemd.

That's a totally bogus claim. systemd has no control on the set of
packages your distro installs or not. If you are missing some tool in
your "emergency environment" (for whatever that is, systemd doesn't
have a concept like that), then bring that up to your distro.

my educated guess is that your distro is providing some emergency
kernel for you that comes with a minimized initrd? If that's the case
it's purely the decision of your distro what to put in there and what
not.

So you are barking up the very very wrong tree here. Go, complain to
your distro instead, we have nothing to do with that.

Lennart

--
Lennart Poettering, Berlin

Re: [systemd-devel] Antw: [EXT] Re: Q: non-ASCII in syslog

[Lennart Poettering]

On Do, 28.04.22 09:32, Ulrich Windl (ulrich.wi...@rz.uni-regensburg.de) wrote:

> Actually I wasn't quite sure about the default config in SLES12.
> It seems the flow is journald -> local rsyslogd -> remote syslogd
>
> > rsyslogd already knows if messages are UTF-8 because the system's $LANG
> > (well, nl_langinfo) says so. And if rsyslog can't trust that for some
> > reason (e.g. because a user might have a different locale), then
> > systemd-journald won't be able to trust it either, so it won't know whether
> > it could add the BOM.
>
> How could a remote syslog server know what the locale on the sending system
> is?

Your local rsyslogd could add the BOM when it transforms journal
messages to syslog datagrams.

> > RFC 3164 over the network to a remote server? Outside the scope for
> > systemd, since it doesn't generate the network packets; your local rsyslogd
> > forwarder does. (Also, why RFC 3164 and not 5425?)
>
> If you look outside the world of systemd, about 99% of systems create the RFC
> 3164 type of messages.

That's a wild claim, and simply wrong actually.

I am pretty sure that more than 50% of syslog messages generated on
this earth probably are synthesized by glibc's syslog() API. And that
turns out to be neither conformant to RFC 3164 nor to RFC 5425.

What glibc sends is close to RFC 3164 but omits one key field that
isn't really optionally according to RFC 3164: the 'HOSTNAME' field.

systemd is focussed on reality: we generate and process the same
format glibc generates.

Lennart

--
Lennart Poettering, Berlin

[systemd-devel] Antw: [EXT] Re: [systemd‑devel] Splitting sd‑boot from systemd/bootctl for enabling sd‑boot in Fedora

[Ulrich Windl]

>>> Lennart Poettering  schrieb am 27.04.2022 um 18:04
in
Nachricht :
> On Mi, 27.04.22 11:10, Neal Gompa (ngomp...@gmail.com) wrote:
> 
>> > Rebooting from the DE has advantages: nice UI without much work, l10n,
>> > accessibility, help, integration with normal auth mechanisms (e.g.
polkit
>> > auth for non‑default boot entries or firmware setup), no need to
>> > fiddle with pressing keys at the exactly right time.
>>
>> It also has a major downside that in the event the OS doesn't boot,
>> you don't have a friendly way to do recovery.
> 
> What does "recovery" precisely mean for you? I mean, on Linux this
> usually means you'll be dumped at a login prompt/shell in one way or
> another. How does it matter whether you first showed a graphical icon
> in that case?
> 
>> Nowadays both Windows and macOS provide graphical boot managers and
>> graphical tools/environments for recovery. These are both things I
>> want in Fedora as well.
> 
> Well, it sounds backwards to focus on the boot loader UI side of
> "recovery" so much if you don't even have any reasonably thing you
> could do in case of recovery better than a login prompt/shell...

Well, not the shell, the tools are important:
Before systemd I could easily recover as system that failed booting (at some
init stage), because I could easily mount the root filesystem and the tools
were there.
With systemd I have a crippled minimum emergency environment where almost all
required tools are absent (just es the real fstab is). That's one of the first
and biggest frustrations with systemd.

> 
> Quite frankly, I think we should actually focus on real improvements
> to recovery stuff, i.e. boot counting/automatic fallback on failed
> boots. which sd‑boot all implements btw, in conjunction with systemd

At the current state of AI, I'd prefer manual recovery over any "automatic".
Last time I had permitted Windows to try automatic recovery, it messed up the
system so severely that I had to restore from backup.
(Only the AHCI mode was lost after a drained BIOS battery).

> userspace. That kind of stuff makes whole sets of problems go away
> entirely, and is *actually* helpful. Whether we first show a graphical
> icon or just a text before we dump you in a shell prompt once all is
> lost anyway is kinda a pointless discussion if you ask me.

fsck, only tring to fix obvious non-controversial issues automatically, but
require manual mode otherwise proved to be a very successful approach over the
years.
Sill users could run with the "-y" option to get "something" that might work,
but still probably loosing some data that could be recovered otherwise.

> 
> For me recovery means something very different than graphical icons I
> must say.

Sadly, today many users judge from the look of the icons, not from the tools
behind.
(If you ever followed Android's syslog, you know what I mean... ;-)

Regards,
Ulrich

Re: [systemd-devel] Antw: [EXT] Re: Q: non-ASCII in syslog

[Lennart Poettering]

On Do, 28.04.22 09:37, Ulrich Windl (ulrich.wi...@rz.uni-regensburg.de) wrote:

> > There's plenty software that doesn't support RFC 5425, and putting a
> > BOM first is certainly not implemented in any of those. I think BOM is
> > hideous and defaulting to UTF-8 generally safe. If we'd put BOM first,
> > these messages would likely not be compatible with a large variety of
> > consumers anymore, because they can't handle BOM. This would be worse
>
> That's a non-argument:
> You say you don't adhere to any of the standards, and claim if you would do,
> things would break. ???

Yes, RFC 5425 is implemented by some logging infra, but it isn't by
all, and messages that are valid by rfc 5425 are not necessarily
compatible with messages generated/processed by software not knowing
rfc 5425. Adding the BOM is a sure way to guarantee software that
doesn't implement rfc 5425 won't be able to process your messages
anymore.

systemd's support for syslog (both on the generating and the consuming
side) is modelled after glibc's logging implementation, nothing
else. It also doesn't do BOM, hence we don't either.

> > than the status quo I am sure, since if we just send UTF-8 things
> > should generally just work fine for any software that either a) also
> > defaults to UTF-8 when encountering an 8bit char or b) is agonistic to
> > charsets and just passes data thorugh.
>
> Yes, put the head in the sand hoping problems are gone when you look up
> again... ;-)

I am pretty sure by inserting the BOM you create more
incompatibilities than you solve.

> > So, yeah, we might be stretching stdandards and tradition a bit, but
> > it actually works out quite well so far.
>
> A good argument for driving without a saftey-belt, BTW.

This comparison makes no sense. Please be civil.

Lennart

--
Lennart Poettering, Berlin

[systemd-devel] multiple systemd-nspawn container name resolution

[basti]

Hello,

i have setup multiple systemd-nspawn container with virtualEthernet=yes.
Inside the container systemd-networkd and systemd-resolved are enabled.
So far so good.

Is there a way to disable the 169.254... networking for systemd?
The problem is as followed:

ip r

169.254.0.0/16 dev ve-test3 proto kernel scope link src 169.254.144.44
169.254.0.0/16 dev ve-test2 proto kernel scope link src 169.254.38.46
169.254.0.0/16 dev ve-test proto kernel scope link src 169.254.80.104

So I only can ping test3.
The same problem is with name resolution.

best regards

[systemd-devel] Antw: [EXT] Re: Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora

[Ulrich Windl]

>>> Neal Gompa  schrieb am 27.04.2022 um 17:26 in Nachricht
:

...
>> E.g. the biggest development in how the boot looks in recent years
>> in Fedora has been hiding on the boot menus and boot messages by
>> default. I.e. the _design_ is that you start with the logo of the
>> manufacturer which is seamlessly replaced by the gdm login screen.
>> How the boot menu looks never factors into any of this.
>>
> 
> Hiding them by default doesn't mean making them scary and semi-useless
> when you access them. Most people don't access UEFI menus very often,
> if at all, and yet a huge amount of investment went into making that
> UX better than it was in the past. Is it spectacular? No. Is it less
> scary? Absolutely.

Actually without systemd it's good, but with systemd you just experience 
minutes of delay (while systemd waits for something that isn't working) while 
you don't see what's going on.
So I disable all those quiet and silent options. The boot looks really messy 
then (as nobody ever consideerd having boot messages on the console), but I'm 
less bored while waiting for the network cards to configure, disks being 
discovered and multipaths being collected...

...

Maybe those system should log boot messages on tty1, but switch to tty2 (and 
continue from there) for users that don't care about details.
Traditionally the "system console" was not intended for "users" (but for 
administrators).

Regards,
Ulrich

Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora

[Lennart Poettering]

On Mi, 27.04.22 18:57, Michael Biebl (mbi...@gmail.com) wrote:

> >From https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=996202#60
>
> """
> we have recently discussed the matter of systemd-boot in
> an upstream shim review gathering.

Ominous.

> We reject a signing of systemd-boot as
>
> * systemd-boot does not use current ways of communicating with
>   shim

It does not? What does that even mean, and why does that even matter?

Also, it does communicate with shim, see src/boot/efi/shim.c… And
there's SBAT support, which is a shim thing afaik.

> * There was some concern over general quality

Humpf. That's just 1st rate FUD. I mean, if systemd for PID 1 is OK'ed
by distros, then maybe the boot loader maintained by the same people
should be fine too. i'd be curious what precisely the "quality" issues
are supposed to be…

> * systemd-boot is an additional bootloader, rather than replacing
>   an existing one, thus increasing the attack surface.

Hmm, what? "additional bootloader"? Are they suggesting you use grub
to start sd-boot? I mean, you certainly could do that, but the only
people I know who do that do that to patch around the gatekeeping that
the shim people are doing. Technically the boot chain should either be
[firmware → sd-boot → kernel] or [firmware → shim → sd-boot → kernel]
(if you buy into the shim thing), and nothing else.

>   If people want to experiment with other bootloaders than the
>   default one, they can disable secure boot, or load their own
>   keys into the machine. We do not consider it valid to have
>   a choice of bootloaders.
>
> I want to note that the current shim has been signed with the
> understanding that it will trust grub, kernels, and fwupd. A
> signing of systemd-boot might be considered reasons for revoking
> the existing shim, and will certainly result in new shims not
> getting signed.

Christ! That's some gatekeeping.

Lennart

--
Lennart Poettering, Berlin

Re: [systemd-devel] [EXT] Re: Q: non-ASCII in syslog

[Mantas Mikulėnas]

On Thu, Apr 28, 2022 at 10:32 AM Ulrich Windl <
ulrich.wi...@rz.uni-regensburg.de> wrote:

> >>> Mantas Mikulenas  schrieb am 27.04.2022 um 12:03 in
> Nachricht
> :
> > On Wed, Apr 27, 2022 at 10:09 AM Ulrich Windl <
> > ulrich.wi...@rz.uni-regensburg.de> wrote:
> >
> >> Hi!
> >>
> >> Having written an RFC 3164 compatible syslog daemon, I noticed that
> systemd
> >> created syslog messages with non-ASCII characters.
> >> The problem is that a remote syslogd can hardly guess the correct
> character
> >> set (I'm using rsyslog to forward local messages to a remote server).
> >>
> >> Example of such message:
> >> systemd-tmpfiles[3311]: [/usr/lib/tmpfiles.d/svnserve.conf:1] Line
> >> references
> >> path below legacy directory /var/run/, updating /var/run/svnserve →
> >> /run/svnserve; please update the tmpfiles.d/ drop-in file accordingly.
> >>
> >> (The arrow is encoded as three bytes (\xe2\x86\x92))
> >>
> >> RFC 5425 syslog messages require the use of a BOM (%xEF.BB.BF) at the
> >> beginning of a message if the message used UTF-8:
> >>
> >>   MSG = MSG-ANY / MSG-UTF8
> >>   MSG-ANY = *OCTET ; not starting with BOM
> >>   MSG-UTF8= BOM UTF-8-STRING
> >>   BOM = %xEF.BB.BF
> >>
> >> Wouldn't it make sense to add such a BOM for RFC 3164 syslog messages
> also
> >> if
> >> non-ASCII (i.e.: UTF-8) encoded characters are used?
> >>
> >
> > RFC 3164 over a local socket from journald to local rsyslogd? The local
>
> Actually I wasn't quite sure about the default config in SLES12.
> It seems the flow is journald -> local rsyslogd -> remote syslogd
>
> > rsyslogd already knows if messages are UTF-8 because the system's $LANG
> > (well, nl_langinfo) says so. And if rsyslog can't trust that for some
> > reason (e.g. because a user might have a different locale), then
> > systemd-journald won't be able to trust it either, so it won't know
> whether
> > it could add the BOM.
>
> How could a remote syslog server know what the locale on the sending system
> is?
>

It's not remote, it's local. I'm talking about the one that's receiving
messages from journald on the same machine.


>
> >
> > RFC 3164 over the network to a remote server? Outside the scope for
> > systemd, since it doesn't generate the network packets; your local
> rsyslogd
> > forwarder does. (Also, why RFC 3164 and not 5425?)
>
> If you look outside the world of systemd, about 99% of systems create the
> RFC
> 3164 type of messages.
> Some may send non-ASCII too, however.
>

Still outside the scope of systemd. Systemd doesn't send RFC 3164 messages
over the network, either.


>
> >
> > Generally, if a message successfully decodes as UTF-8 then it's most
> likely
> > actual UTF-8 (and if UTF-8 decode fails then you fall back to ISO8859-1).
> > Various old systems get away with this without needing a UTF-8 BOM.
>
> Yes, you can just output what you received, hoping the messages will be
> presented correctly.
> I't just like sending 8-bit E-Mmail without a coding system or charset in
> the
> past.
>

Which is not what I was saying, but sure, whatever.

-- 
Mantas Mikulėnas

[systemd-devel] Antw: [EXT] Re: Q: non-ASCII in syslog

[Ulrich Windl]

>>> Lennart Poettering  schrieb am 27.04.2022 um 13:10
in
Nachricht :
> On Mi, 27.04.22 09:09, Ulrich Windl (ulrich.wi...@rz.uni-regensburg.de) 
> wrote:
> 
>> Hi!
>>
>> Having written an RFC 3164 compatible syslog daemon, I noticed that
systemd
>> created syslog messages with non-ASCII characters.
>> The problem is that a remote syslogd can hardly guess the correct
character
>> set (I'm using rsyslog to forward local messages to a remote
>> server).
> 
> It's 2022. I think at this point, software should always assume the
> charset is UTF-8 if it doesn't have an reason to believe otherwise.
> 
> It's kinda what we started to do all across our codebase really. We'll
> use UTF-8 for everything by default. For some things where people
> complain sufficeintly loudly we'll conditionalize them so that we have
> some fallback in place if we know for sure UTF-8 is not OK, but the
> default we do is always and everywhere UTF-8.
> 
>> Example of such message:
>> systemd-tmpfiles[3311]: [/usr/lib/tmpfiles.d/svnserve.conf:1] Line 
> references
>> path below legacy directory /var/run/, updating /var/run/svnserve →
>> /run/svnserve; please update the tmpfiles.d/ drop-in file accordingly.
>>
>> (The arrow is encoded as three bytes (\xe2\x86\x92))
>>
>> RFC 5425 syslog messages require the use of a BOM (%xEF.BB.BF) at the
>> beginning of a message if the message used UTF-8:
> 
> We do not implement RFC 5425, as glibc doesn't support that. In fact
> we don't even implement RFC 3164 in full, since glibc generates the
> messages in a very specific format only.
> 
>>
>>   MSG = MSG-ANY / MSG-UTF8
>>   MSG-ANY = *OCTET ; not starting with BOM
>>   MSG-UTF8= BOM UTF-8-STRING
>>   BOM = %xEF.BB.BF
>>
>> Wouldn't it make sense to add such a BOM for RFC 3164 syslog messages also

> if
>> non-ASCII (i.e.: UTF-8) encoded characters are used?
> 
> There's plenty software that doesn't support RFC 5425, and putting a
> BOM first is certainly not implemented in any of those. I think BOM is
> hideous and defaulting to UTF-8 generally safe. If we'd put BOM first,
> these messages would likely not be compatible with a large variety of
> consumers anymore, because they can't handle BOM. This would be worse

That's a non-argument:
You say you don't adhere to any of the standards, and claim if you would do,
things would break. ???

> than the status quo I am sure, since if we just send UTF-8 things
> should generally just work fine for any software that either a) also
> defaults to UTF-8 when encountering an 8bit char or b) is agonistic to
> charsets and just passes data thorugh.

Yes, put the head in the sand hoping problems are gone when you look up
again... ;-)

> 
> So, yeah, we might be stretching stdandards and tradition a bit, but
> it actually works out quite well so far.

A good argument for driving without a saftey-belt, BTW.

Regards,
Ulrich

> 
> Lennart
> 
> --
> Lennart Poettering, Berlin

[systemd-devel] Antw: [EXT] Re: Q: non-ASCII in syslog

[Ulrich Windl]

>>> Mantas Mikulenas  schrieb am 27.04.2022 um 12:03 in
Nachricht
:
> On Wed, Apr 27, 2022 at 10:09 AM Ulrich Windl <
> ulrich.wi...@rz.uni-regensburg.de> wrote:
> 
>> Hi!
>>
>> Having written an RFC 3164 compatible syslog daemon, I noticed that
systemd
>> created syslog messages with non-ASCII characters.
>> The problem is that a remote syslogd can hardly guess the correct
character
>> set (I'm using rsyslog to forward local messages to a remote server).
>>
>> Example of such message:
>> systemd-tmpfiles[3311]: [/usr/lib/tmpfiles.d/svnserve.conf:1] Line
>> references
>> path below legacy directory /var/run/, updating /var/run/svnserve →
>> /run/svnserve; please update the tmpfiles.d/ drop-in file accordingly.
>>
>> (The arrow is encoded as three bytes (\xe2\x86\x92))
>>
>> RFC 5425 syslog messages require the use of a BOM (%xEF.BB.BF) at the
>> beginning of a message if the message used UTF-8:
>>
>>   MSG = MSG-ANY / MSG-UTF8
>>   MSG-ANY = *OCTET ; not starting with BOM
>>   MSG-UTF8= BOM UTF-8-STRING
>>   BOM = %xEF.BB.BF
>>
>> Wouldn't it make sense to add such a BOM for RFC 3164 syslog messages also
>> if
>> non-ASCII (i.e.: UTF-8) encoded characters are used?
>>
> 
> RFC 3164 over a local socket from journald to local rsyslogd? The local

Actually I wasn't quite sure about the default config in SLES12.
It seems the flow is journald -> local rsyslogd -> remote syslogd

> rsyslogd already knows if messages are UTF-8 because the system's $LANG
> (well, nl_langinfo) says so. And if rsyslog can't trust that for some
> reason (e.g. because a user might have a different locale), then
> systemd-journald won't be able to trust it either, so it won't know whether
> it could add the BOM.

How could a remote syslog server know what the locale on the sending system
is?

> 
> RFC 3164 over the network to a remote server? Outside the scope for
> systemd, since it doesn't generate the network packets; your local rsyslogd
> forwarder does. (Also, why RFC 3164 and not 5425?)

If you look outside the world of systemd, about 99% of systems create the RFC
3164 type of messages.
Some may send non-ASCII too, however.

> 
> Generally, if a message successfully decodes as UTF-8 then it's most likely
> actual UTF-8 (and if UTF-8 decode fails then you fall back to ISO8859-1).
> Various old systems get away with this without needing a UTF-8 BOM.

Yes, you can just output what you received, hoping the messages will be
presented correctly.
I't just like sending 8-bit E-Mmail without a coding system or charset in the
past.

Regards,
Ulrich