Re: [systemd-devel] Trying to come up with udev rule for USB geographic naming

[Andrei Borzenkov]

11.07.2017 05:33, Paul D. DeRocco пишет:
> That is, I want a rule for creating a symlink that describes the USB port
> something is plugged into, rather than the specific device plugged into
> it. Linux USB has a way of representing the location of each physical
> jack, even when hubs are involved. For devices like USB serial and USB
> MIDI, I'd like apps to be able to refer to devices by where they're
> plugged in.
> 
> I used udevadm to generate the info for a USB MIDI converter. The first
> three levels look like this:
> 
> looking at device
> '/devices/platform/soc/3f98.usb/usb1/1-1/1-1.2/1-1.2:1.0/sound/card1/m
> idiC1D0':
>   KERNEL=="midiC1D0"
>   SUBSYSTEM=="sound"
>   DRIVER==""
> 
> looking at parent device
> '/devices/platform/soc/3f98.usb/usb1/1-1/1-1.2/1-1.2:1.0/sound/card1':
>   KERNELS=="card1"
>   SUBSYSTEMS=="sound"
>   DRIVERS==""
>   ATTRS{id}=="XMidi1X1"
>   ATTRS{number}=="1"
> 
> looking at parent device
> '/devices/platform/soc/3f98.usb/usb1/1-1/1-1.2/1-1.2:1.0':
>   KERNELS=="1-1.2:1.0"
>   SUBSYSTEMS=="usb"
>   DRIVERS=="snd-usb-audio"
>   ATTRS{authorized}=="1"
>   ATTRS{bAlternateSetting}==" 0"
>   ATTRS{bInterfaceClass}=="01"
>   ATTRS{bInterfaceNumber}=="00"
>   ATTRS{bInterfaceProtocol}=="00"
>   ATTRS{bInterfaceSubClass}=="01"
>   ATTRS{bNumEndpoints}=="00"
>   ATTRS{supports_autosuspend}=="1"
> 
> (There may be spurious line breaks in there, due to email.)
> 
> I'd like an alias for this device to be "/dev/midi1-1.2:1.0". I tried the
> following:
> 
> KERNEL=="midiC*", DRIVERS=="snd-usb-audio", SYMLINK+="midi%k"
> 

You probably want %b instead of %k here which should refer to device
name matched by DRIVERS.

> It did bupkis. The man page seems to imply that, while a single rule can't
> match items from different parents, it's okay to match something in the
> device and then something else in a parent. So I would expect the KERNEL
> key would match the device itself, DRIVERS would match a parent, but what
> then does %k refer to, the kernel name in the device or the kernel name in
> the parent?
> 
> But it not only didn't create a "midi1-1.2:1.0", it didn't create a
> "midimidiC1D0" either, so it doesn't look like the rule matched at all.
> 

It could be race condition between driver loading and follow-up
processing. Or there is some rule that stops further processing. Start
with "udevadm test" to verify.

> There is already a symlink
> "/dev/snd/by-path/platform-3f98.usb-usb-0:1.2:1.0", but it refers to
> the useless "controlC1" interface. It also doesn't appear to be generated
> by an explicit rule anywhere that might show me what to do.
> 

It is created by 60-persistent-alsa.rules that explicitly restricts
processing to controlC*.

> What am I doing wrong? Do I need a rule that just matches the MIDI device,
> and then runs a script that searches for the parent name?
> 

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

[systemd-devel] systemd-networkd not sending DHCP v6 requests

[Ian Pilcher]

I'm playing using systemd-networkd (rather than the legacy network
service) on my Banana Pi CentOS 7 firewall.  (See the "Bouncing
interface once chrony is synced" thread for background.)

I have "DHCP=yes" in the [Network] section of my WAN interface
(eth0.256.network):

 [Match]
 Name=eth0.256

 [Network]
 DHCP=yes
 IPForward=yes
 #IPv6AcceptRA=yes - Not supported in v219

 [DHCP]
 UseHostname=no

For some reason, though, only the IPv4 DHCP request is being sent.  I
have verified with tcpdump that no "dhcp6 solicit" packets are ever
sent.

I also have a 32-bit ARM VM (qemu) running CentOS 7.  It does not
exhibit this behavior; i.e. systemd-networkd does send dhcp6 solicit
packets from this VM.  The most significant difference between the VM
and the firewall is that the firewall has an extremely minimal set of
packages installed.

Is there some library or package that is required for DHCP v6 to work
(something that might have slipped by an RPM packager)?

Any other ideas?

--

Ian Pilcher arequip...@gmail.com
 "I grew up before Mark Zuckerberg invented friendship" 

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

[systemd-devel] Trying to come up with udev rule for USB geographic naming

[Paul D. DeRocco]

That is, I want a rule for creating a symlink that describes the USB port
something is plugged into, rather than the specific device plugged into
it. Linux USB has a way of representing the location of each physical
jack, even when hubs are involved. For devices like USB serial and USB
MIDI, I'd like apps to be able to refer to devices by where they're
plugged in.

I used udevadm to generate the info for a USB MIDI converter. The first
three levels look like this:

looking at device
'/devices/platform/soc/3f98.usb/usb1/1-1/1-1.2/1-1.2:1.0/sound/card1/m
idiC1D0':
  KERNEL=="midiC1D0"
  SUBSYSTEM=="sound"
  DRIVER==""

looking at parent device
'/devices/platform/soc/3f98.usb/usb1/1-1/1-1.2/1-1.2:1.0/sound/card1':
  KERNELS=="card1"
  SUBSYSTEMS=="sound"
  DRIVERS==""
  ATTRS{id}=="XMidi1X1"
  ATTRS{number}=="1"

looking at parent device
'/devices/platform/soc/3f98.usb/usb1/1-1/1-1.2/1-1.2:1.0':
  KERNELS=="1-1.2:1.0"
  SUBSYSTEMS=="usb"
  DRIVERS=="snd-usb-audio"
  ATTRS{authorized}=="1"
  ATTRS{bAlternateSetting}==" 0"
  ATTRS{bInterfaceClass}=="01"
  ATTRS{bInterfaceNumber}=="00"
  ATTRS{bInterfaceProtocol}=="00"
  ATTRS{bInterfaceSubClass}=="01"
  ATTRS{bNumEndpoints}=="00"
  ATTRS{supports_autosuspend}=="1"

(There may be spurious line breaks in there, due to email.)

I'd like an alias for this device to be "/dev/midi1-1.2:1.0". I tried the
following:

KERNEL=="midiC*", DRIVERS=="snd-usb-audio", SYMLINK+="midi%k"

It did bupkis. The man page seems to imply that, while a single rule can't
match items from different parents, it's okay to match something in the
device and then something else in a parent. So I would expect the KERNEL
key would match the device itself, DRIVERS would match a parent, but what
then does %k refer to, the kernel name in the device or the kernel name in
the parent?

But it not only didn't create a "midi1-1.2:1.0", it didn't create a
"midimidiC1D0" either, so it doesn't look like the rule matched at all.

There is already a symlink
"/dev/snd/by-path/platform-3f98.usb-usb-0:1.2:1.0", but it refers to
the useless "controlC1" interface. It also doesn't appear to be generated
by an explicit rule anywhere that might show me what to do.

What am I doing wrong? Do I need a rule that just matches the MIDI device,
and then runs a script that searches for the parent name?

-- 

Ciao,   Paul D. DeRocco
Paulmailto:pdero...@ix.netcom.com

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Unknown lvalue 'StartLimitIntervalSec' ?

[Neil MacLeod]

Aha! Many thanks Zbyszek

On 11 July 2017 at 02:26, Zbigniew Jędrzejewski-Szmek 
wrote:

> On Tue, Jul 11, 2017 at 01:55:21AM +0100, Neil MacLeod wrote:
> > I see that with the following commit, 'StartLimitInterval' is aliased to
> > 'StartLimitIntervalSec':
> >
> > https://github.com/systemd/systemd/commit/f0367da7d1a61ad698a55d17b5c28d
> dce0dc265a#diff-b3e16bf39e5d7ab233077b9b1dafb095
> >
> > This change should be in systemd-230.
> >
> > However, using 'StartLimitIntervalSec` in systemd-233 doesn't appear to
> be
> > working and the property is being logged as an "Unknown lvalue".
> >
> > I've built systemd-233 from source, and confirmed the source includes the
> > above alias change, but I cannot set a 'StartLimitIntervalSec' property.
> > The old 'StartLimitInterval' value does work.
> >
> > All the documentation suggests 'StartLimitIntervalSec' should now be
> > working/accepted.
>
> StartLimitInterval was moved from [Service] to [Unit] section in
> 6bf0f408e4833152197fb38fb10a9989c89f3a59, but the old location was
> still accepted for compatibility. The new StartLimitIntervalSec name
> is valid only in [Unit].
>
> Zbyszek
>
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Unknown lvalue 'StartLimitIntervalSec' ?

[Zbigniew Jędrzejewski-Szmek]

On Tue, Jul 11, 2017 at 01:55:21AM +0100, Neil MacLeod wrote:
> I see that with the following commit, 'StartLimitInterval' is aliased to
> 'StartLimitIntervalSec':
> 
> https://github.com/systemd/systemd/commit/f0367da7d1a61ad698a55d17b5c28ddce0dc265a#diff-b3e16bf39e5d7ab233077b9b1dafb095
> 
> This change should be in systemd-230.
> 
> However, using 'StartLimitIntervalSec` in systemd-233 doesn't appear to be
> working and the property is being logged as an "Unknown lvalue".
> 
> I've built systemd-233 from source, and confirmed the source includes the
> above alias change, but I cannot set a 'StartLimitIntervalSec' property.
> The old 'StartLimitInterval' value does work.
> 
> All the documentation suggests 'StartLimitIntervalSec' should now be
> working/accepted.

StartLimitInterval was moved from [Service] to [Unit] section in
6bf0f408e4833152197fb38fb10a9989c89f3a59, but the old location was
still accepted for compatibility. The new StartLimitIntervalSec name
is valid only in [Unit].

Zbyszek
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

[systemd-devel] Unknown lvalue 'StartLimitIntervalSec' ?

[Neil MacLeod]

I see that with the following commit, 'StartLimitInterval' is aliased to
'StartLimitIntervalSec':

https://github.com/systemd/systemd/commit/f0367da7d1a61ad698a55d17b5c28ddce0dc265a#diff-b3e16bf39e5d7ab233077b9b1dafb095

This change should be in systemd-230.

However, using 'StartLimitIntervalSec` in systemd-233 doesn't appear to be
working and the property is being logged as an "Unknown lvalue".

I've built systemd-233 from source, and confirmed the source includes the
above alias change, but I cannot set a 'StartLimitIntervalSec' property.
The old 'StartLimitInterval' value does work.

All the documentation suggests 'StartLimitIntervalSec' should now be
working/accepted.

What am I missing here?

Many thanks
Neil

This is the version, 233:
```
rpi22:~ # systemctl --version
systemd 233
-PAM -AUDIT -SELINUX -IMA -APPARMOR -SMACK -SYSVINIT -UTMP -LIBCRYPTSETUP
-GCRYPT -GNUTLS -ACL -XZ -LZ4 -SECCOMP +BLKID -ELFUTILS +KMOD -IDN
default-hierarchy=hybrid
```

And I've got a service where 'StartLimitInterval' has been renamed to
'StartLimitIntervalSec':
```
rpi22:~ # cat /usr/lib/systemd/system/smbd.service
[Unit]
Description=Samba SMB Daemon
After=network.target samba-config.service
ConditionPathExists=!/storage/.cache/services/samba.disabled
ConditionPathExists=/run/samba/smb.conf
Requires=samba-config.service

[Service]
Type=forking
PIDFile=/run/samba/smbd.pid
LimitNOFILE=16384
ExecStartPre=/usr/lib/samba/smbd-config
ExecStart=/usr/sbin/smbd
ExecReload=/bin/kill -HUP $MAINPID
TimeoutStopSec=1s
Restart=always
RestartSec=2
StartLimitIntervalSec=0

[Install]
WantedBy=multi-user.target
```

However 'StartLimitIntervalSec' isn't accepted as valid, whereas the "old"
'StartLimitInterval' continues to work as it should - this is the error
with 'StartLimitIntervalSec':
```
rpi22:~ # journalctl -a|grep StartLimitIntervalSec
Jul 11 01:29:17 rpi22 systemd[1]: /usr/lib/systemd/system/smbd.service:18:
Unknown lvalue 'StartLimitIntervalSec' in section 'Service'
```
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] [PATCH] nfs.man: document incompatibility between "bg" option and systemd.

[NeilBrown]

On Mon, Jul 10 2017, Steve Dickson wrote:

> Hey Neil,
>
> On 07/04/2017 06:20 PM, NeilBrown wrote:
>> On Tue, May 30 2017, NeilBrown wrote:
>> 
>>> Systemd does not, and will not, support "bg" correctly.
>>> It has other, better, ways to handle "background" mounting.
>> 
>> For those who aren't closely watching systemd development, a
>> patch was recently accepted which causes systemd to work correctly with
>> NFS bg mounts.  So the above "and will not" was, happily, not correct.
> Could you please post a pointer to the thread?

The main commmit is
 
https://github.com/systemd/systemd/commit/65e1dee7dcf1668c25c32f0238c935708dbffbcf

The link in the title leads to
  https://github.com/systemd/systemd/pull/6103
which is the discussion of the pull request.
The "Fixes" link at the bottom leads to
 https://github.com/systemd/systemd/issues/6046
which is an 'issue' that started out as "we should warn if 'bg' is used"
but became "here is a fix so bg works".

I've been wondering if we should add text to nfs.5 suggesting that
automount is often a good match for NFS, and can particularly be useful
where "bg" is currently used.  Not sure what to do about timeouts
though.
systemd currently waits for the mount command to fail, or kills it after
90 seconds.  It doesn't return an error to accesses of the mount point
(typically ENODEV) until mount exits.  I think I would prefer a shorter
timeout before the error, but a longer timeout before killing mount.
But I'm not really sure.  Does anyone have opinions?

Thanks,
NeilBrown




signature.asc
Description: PGP signature
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

[vcaputo]

On Sat, Jul 08, 2017 at 03:49:11AM +, Zbigniew Jędrzejewski-Szmek wrote:
> On Fri, Jul 07, 2017 at 03:54:09PM -0700, vcap...@pengaru.com wrote:
> > On Fri, Jul 07, 2017 at 10:34:22PM +, Zbigniew Jędrzejewski-Szmek wrote:
> > > On Fri, Jul 07, 2017 at 02:35:16PM -0700, vcap...@pengaru.com wrote:
> > > > On Fri, Jul 07, 2017 at 01:49:54PM -0700, vcap...@pengaru.com wrote:
> > > > > On Fri, Jul 07, 2017 at 08:37:08PM +, Mantas Mikulėnas wrote:
> > > > > > Back when that commit was made, didn't glibc cache the getpid() 
> > > > > > result in
> > > > > > userspace? That would explain why it was not noticed.
> > > > >
> > > > > Hmm, this crossed my mind, and come to think of it I did a 
> > > > > dist-upgrade
> > > > > from Debian jessie to stretch overnight machine and haven't rebooted.
> > > > > 
> > > > > Perhaps the vdso isn't working and the costly getpid() is a red 
> > > > > herring, will
> > > > > reboot and retest to confirm.
> > > > > 
> > > > 
> > > > It appears Debian has a glibc patch to disable the caching (I was 
> > > > unaware
> > > > such an elaborate dance was being performed to cache this!)
> > > > 
> > > > https://anonscm.debian.org/cgit/pkg-glibc/glibc.git/commit/debian/patches/any?id=5850253f509604dd46a6131acc057ea26e1588ba
> > > 
> > > Do we know the justification for this patch?
> > > 
> > 
> > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=857909
> > 
> > Which references this upstream glibc bug:
> > 
> > https://sourceware.org/bugzilla/show_bug.cgi?id=19957
> > https://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commit;h=0cb313f7cb0e418b3d56f3a2ac69790522ab825d
> > 
> > 
> > > > Unsure where I stand on core system software assuming certain syscalls 
> > > > are
> > > > always going to be exceptionally cheap though...
> > > 
> > > Optimization is never in a vacuum. If glibc does something cheaply, it
> > > seems reasonable to take advantage of it.
> > > 
> > 
> > Except there's always a risk of these things regressing to normal syscalls,
> > and one has to weigh the utility against that.  It's unclear to me what
> > significant utility having the sd-journal API police changing pids by
> > calling getpid() at every public entrypoint is bringing to the table.
> 
> So it seems the issue has been fixed in glibc upstream more than a year
> ago, and it doesn't seem to make sense to optimize current systemd git for
> that.
> 

Can you provide a commit id?  I took a glance at sourcewaire.org/git/gitweb.cgi
for getpid commits and didn't see anything relevant since the removal[1].


> I see the argument that the getpid() checks are a bit excessive. Is their
> overhead actually noticeable with current glibc?
> 

On my spare arch system I still see gratuitous getpid() calls from
journalctl, which is on glibc 2.5-2.

The pollution of strace output alone due to these checks is nuisance enough
for me to want the checks removed, considering their only value is to catch
programmer errors.  There's an abundance of potential programmer errors
we're not making any effort to prevent, why is this one so privileged that
it warrants policing?

I appreciate Lennart's point about the hazards of forking from threaded
programs.  It just doesn't seem like a valid rationalization for sprinkling
a system library's entrypoints with getpid() calls to catch this in
production.

Considering the associated potential costs, and the historic controversy
surrounding the caching of this particular syscall[2] I'm a bit confused by
the status quo.

Cheers,
Vito Caputo

1: 
https://sourceware.org/git/gitweb.cgi?p=glibc.git=search=HEAD=commit=getpid
2: http://yarchive.net/comp/linux/getpid_caching.html

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Bouncing interface once chrony is synced

[Mantas Mikulėnas]

On Mon, Jul 10, 2017 at 7:03 PM, Julian Andres Klode  wrote:

> On Wed, Jul 05, 2017 at 10:29:21AM -0500, Ian Pilcher wrote:
> > I am using CentOS 7 (systemd 219) on a Banana Pi as my residential
> > firewall/gateway.  The Banana Pi does not have a persistent clock, so
> > it has no idea what the time is until it is able to sync via NTP.  Thus,
> > the initial DHCP leases that the BPi receives have incorrect expiration/
> > renewal times (since the system can't sync via NTP before it has an IP
> > address - chicken and egg).
>
> Can't you just store the time on shutdown and restore it at boot (e.g.,
> take the time of the journal file) - that is usually close enough, at
> least it works well enough for use cases in openwrt and lede.
>

systemd-timesyncd already does this. I doubt it would help though, since
the clock would be late anyway, and dhclient seems to break if *any* clock
jumps happen at all (unlike e.g. TLS which only needs to know the
approximate time).

-- 
Mantas Mikulėnas 
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Github systemd issue 6237

[Lennart Poettering]

On Mon, 10.07.17 17:45, Zbigniew Jędrzejewski-Szmek (zbys...@in.waw.pl) wrote:

> On Mon, Jul 10, 2017 at 06:40:00PM +0200, Lennart Poettering wrote:
> > On Mon, 10.07.17 18:36, Lennart Poettering (lenn...@poettering.net) wrote:
> > 
> > > > After all (as other people said) systemd has no such requirements
> > > > itself. It is true that such user names are confusing and
> > > > non-portable, but if the local admin has or wants to have such an
> > > > account for whatever reason, we don't really care.
> > > 
> > > I don't think things are that simple. We do our user name validation
> > > in two places: for User=/Group= and for sysusers.d drop-ins. In both
> > > cases the setting may have the effect of registering users in the
> > > system user database (in the first case if DynamicUser= is used, in
> > > the latter case if the user doesn't exist yet), and I am pretty sure
> > > we shouldn't register users in the system user databases that aren't
> > > portable.
> > 
> > Or to say this differently: User=/Group=/sysusers.d shouldn't be
> > something you can create users with that for example ArchLinux'
> > useradd command wouldn't allow you to create.
> 
> I can see it both ways, but yeah, it never came up before and
> personally I never had the need (or even whim) to create a user that
> systemd would reject. So I'd like to #6300 to go in, and apart
> from that I'm happy with the status quo, and I merged #6321 now.

BTW, one more reference point to the discussion: shadow-utils upstream
enforces this regex apparently:

[a-z_][a-z0-9_-]*$?

The trailing $ thing appears to be a more recent addition, some
Windows thing. A minimum length of 1 is enforced, but apparently no
max length limit (neither _SC_LOGIN_NAME_MAX nor UT_NAMESIZE-1).

Fedora/RH deviate from that though and explicitly patch this out, replacing this
with the more relaxed regex mentioned earlier:

https://src.fedoraproject.org/cgit/rpms/shadow-utils.git/tree/shadow-4.1.5.1-goodname.patch

It appears our rules are hence pretty close to shadow-util's original
ones with the exeption of the max size limit and the Windows $ thing,
which really shouldn't apply to our system service users I figure.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Github systemd issue 6237

[Zbigniew Jędrzejewski-Szmek]

On Mon, Jul 10, 2017 at 06:40:00PM +0200, Lennart Poettering wrote:
> On Mon, 10.07.17 18:36, Lennart Poettering (lenn...@poettering.net) wrote:
> 
> > > After all (as other people said) systemd has no such requirements
> > > itself. It is true that such user names are confusing and
> > > non-portable, but if the local admin has or wants to have such an
> > > account for whatever reason, we don't really care.
> > 
> > I don't think things are that simple. We do our user name validation
> > in two places: for User=/Group= and for sysusers.d drop-ins. In both
> > cases the setting may have the effect of registering users in the
> > system user database (in the first case if DynamicUser= is used, in
> > the latter case if the user doesn't exist yet), and I am pretty sure
> > we shouldn't register users in the system user databases that aren't
> > portable.
> 
> Or to say this differently: User=/Group=/sysusers.d shouldn't be
> something you can create users with that for example ArchLinux'
> useradd command wouldn't allow you to create.

I can see it both ways, but yeah, it never came up before and
personally I never had the need (or even whim) to create a user that
systemd would reject. So I'd like to #6300 to go in, and apart
from that I'm happy with the status quo, and I merged #6321 now.

Zbyszek
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Github systemd issue 6237

[Lennart Poettering]

On Mon, 10.07.17 15:29, Zbigniew Jędrzejewski-Szmek (zbys...@in.waw.pl) wrote:

> > On current Fedora, the current regex useradd enforces appears to be
> > this:
> > 
> > [a-zA-Z0-9._][a-zA-Z0-9._-]{0,30}[a-zA-Z0-9._-$]?
> > 
> > If I read things correctly at least... (the trailing $ appears to be a
> > special exception they added to be nice to Windows, dunno). And this
> > regex appears not to be configurable.
> 
> Maybe the logic should be reversed: instead of trying to *enforce*
> the most-strict name that works everywhere, just check that the name
> is between 1 and 31 characters is length and not numeric, and relax
> the restrictions on the exact characters in the user name, and *warn*
> if the user name has "strange" characters:
>Warning: user name "foo.bar" contains a dot which conflicts with chown 
> user.group syntax
>Warning: user name "0day" is not portable
>Warning: user name "-user" looks like an option ;)
> etc.
> 
> After all (as other people said) systemd has no such requirements
> itself. It is true that such user names are confusing and
> non-portable, but if the local admin has or wants to have such an
> account for whatever reason, we don't really care.

I don't think things are that simple. We do our user name validation
in two places: for User=/Group= and for sysusers.d drop-ins. In both
cases the setting may have the effect of registering users in the
system user database (in the first case if DynamicUser= is used, in
the latter case if the user doesn't exist yet), and I am pretty sure
we shouldn't register users in the system user databases that aren't
portable.

I mean, again, systemd is the one defining both interfaces:
User=/Group= as well as sysusers.d, and I am pretty sure we should
make sure when they are used they are used in a reasonably safe and
sound and portable way. We want that people can write unit files and
run them everywhere and they work reasonably well. I think it's not
too much to ask from people to follow some simple basic rules when
making use of User=/Group=. We aren't really taking away anything
there, we aren#t really strict on something that was previously less
strict, simply because User=/Group= isn't a sysvinit concept, it's
genuinely a systemd concept.

Hence, it's not really a matter of consuming weird stuff local admins
created. It's a lot more than that: we *create* that stuff, possibly
leaving it around for good, and unit files are supposed to be
portable between systems.

Compare this with logind: we do not validate the user name passed to
us from PAM for new login sessions the same way as we do for
User=/Group=/sysusers.d. In the logind/PAM case the interface isn't
designed by us, PAM doesn't validate its input particularly carefully,
and we should probably just accept what PAM accepts as we don't create
anything here: we just consume what's listed in the user database.

Logging more verbosely about this is good, and maybe making this fatal
instead of just a warning might be good too, but just permitting it
sounds wrong to me.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Github systemd issue 6237

[Lennart Poettering]

On Mon, 10.07.17 18:36, Lennart Poettering (lenn...@poettering.net) wrote:

> > After all (as other people said) systemd has no such requirements
> > itself. It is true that such user names are confusing and
> > non-portable, but if the local admin has or wants to have such an
> > account for whatever reason, we don't really care.
> 
> I don't think things are that simple. We do our user name validation
> in two places: for User=/Group= and for sysusers.d drop-ins. In both
> cases the setting may have the effect of registering users in the
> system user database (in the first case if DynamicUser= is used, in
> the latter case if the user doesn't exist yet), and I am pretty sure
> we shouldn't register users in the system user databases that aren't
> portable.

Or to say this differently: User=/Group=/sysusers.d shouldn't be
something you can create users with that for example ArchLinux'
useradd command wouldn't allow you to create.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Github systemd issue 6237

[Carlos Silva]

On Mon, Jul 10, 2017 at 4:03 PM, Lennart Poettering
 wrote:
> On current Fedora, the current regex useradd enforces appears to be
> this:
>
> [a-zA-Z0-9._][a-zA-Z0-9._-]{0,30}[a-zA-Z0-9._-$]?

So, it *does* allow for usernames starting with numbers...
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Bouncing interface once chrony is synced

[Ian Pilcher]

On 07/10/2017 03:43 AM, Lennart Poettering wrote:

If your DHCP client implementation gets confused by the wallclock not
being steady then this appears to be a bug in the
implementation. Given that there are so many DHCP clients to choose
from, maybe use a different one?


Oh, it's definitely a bug in ISC dhclient:

  https://bugzilla.redhat.com/show_bug.cgi?id=1093803

Switching DHCP clients doesn't exactly sound like a trivial exercise,
however, since it would need to be integrated into the CentOS network
startup scripts; bouncing the interface (inelegant as it is) seems like
the easier option.

This unit file seems to be doing the job for IPv6 (since *something*
seems to be already doing this for IPv4):

  [Unit]
  Description=IPv6 DHCP client for %I
  Requires=chrony-wait.service
  After=chrony-wait.service

  [Service]
  Type=simple
  ExecStartPre=/usr/sbin/dhclient -6 -d -q -x -lf 
/var/lib/dhclient/dhclient6--%I.lease -pf /var/run/dhclient6-%I.pid %I
  ExecStart=/usr/sbin/dhclient -6 -d -q -1 -lf 
/var/lib/dhclient/dhclient6--%I.lease -pf /var/run/dhclient6-%I.pid %I


  [Install]
  WantedBy=multi-user.target

Maybe systemd-networkd ...

--

Ian Pilcher arequip...@gmail.com
 "I grew up before Mark Zuckerberg invented friendship" 

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Bouncing interface once chrony is synced

[Julian Andres Klode]

On Wed, Jul 05, 2017 at 10:29:21AM -0500, Ian Pilcher wrote:
> I am using CentOS 7 (systemd 219) on a Banana Pi as my residential
> firewall/gateway.  The Banana Pi does not have a persistent clock, so
> it has no idea what the time is until it is able to sync via NTP.  Thus,
> the initial DHCP leases that the BPi receives have incorrect expiration/
> renewal times (since the system can't sync via NTP before it has an IP
> address - chicken and egg).

Can't you just store the time on shutdown and restore it at boot (e.g.,
take the time of the journal file) - that is usually close enough, at
least it works well enough for use cases in openwrt and lede.

-- 
Debian Developer - deb.li/jak | jak-linux.org - free software dev
  |  Ubuntu Core Developer |
When replying, only quote what is necessary, and write each reply
directly below the part(s) it pertains to ('inline').  Thank you.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] [PATCH] nfs.man: document incompatibility between "bg" option and systemd.

[Steve Dickson]

Hey Neil,

On 07/04/2017 06:20 PM, NeilBrown wrote:
> On Tue, May 30 2017, NeilBrown wrote:
> 
>> Systemd does not, and will not, support "bg" correctly.
>> It has other, better, ways to handle "background" mounting.
> 
> For those who aren't closely watching systemd development, a
> patch was recently accepted which causes systemd to work correctly with
> NFS bg mounts.  So the above "and will not" was, happily, not correct.
Could you please post a pointer to the thread?

tia,

steved.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Github systemd issue 6237

[Zbigniew Jędrzejewski-Szmek]

On Mon, Jul 10, 2017 at 05:03:09PM +0200, Lennart Poettering wrote:
> On Mon, 10.07.17 22:23, Michael Chapman (m...@very.puzzling.org) wrote:
> 
> > > Well, it took 3 years or so, until someone noticed the strict rules we
> > > enforce. I seriously doubt that naming system users in such unsafe
> > > ways is really that wide-spread usage.
> > 
> > That _could_ be because people that have previously used such a username
> > hadn't looked in their logs and noticed that the User= directive wasn't
> > being applied. :-)
> 
> Well: if you write a unit file, it's probably a good idea to test it
> once before deploying it. I mean, it's not that we silently skip stuff
> that was OK on sysvinit or so — that's because sysvinit of course never
> supported dropping user IDs for you, you always had to do it yourself.
> 
> If your service is dropping privs on its own it's of course completely
> between you and that service to name the user any way you like. But if
> you tell systemd to do this for you and for the service, we do some
> basic checking and validation, that's all. Or in other words: if I
> write a unit file on my laptop here, and it works, then systemd is
> supposed to give you the guarantee that it also works correctly on
> my server system I deploy this on, too (to some degree at least), even
> if it runs a slightly different Linux distro.
> 
> > So be it. I'm fine with us remaining in disagreement... I just wish I
> > understood exactly what the security implications are in allowing such
> > usernames. I know my colleagues are going to ask me about this, and being
> > able to point at something and say "oh yeah, it breaks this specifically"
> > would be really handy.
> 
> I am pretty sure it would be very welcome if somebody would do a
> proper investigation and collect the precise restrictions different
> systems in different versions enforce on naming users. And not just
> across distro versions and their user name regexes, but also across
> application software packages running on top of those distros. I am
> not aware of any sufficiently comprehensive list of this.
> 
> On current Fedora, the current regex useradd enforces appears to be
> this:
> 
> [a-zA-Z0-9._][a-zA-Z0-9._-]{0,30}[a-zA-Z0-9._-$]?
> 
> If I read things correctly at least... (the trailing $ appears to be a
> special exception they added to be nice to Windows, dunno). And this
> regex appears not to be configurable.

Maybe the logic should be reversed: instead of trying to *enforce*
the most-strict name that works everywhere, just check that the name
is between 1 and 31 characters is length and not numeric, and relax
the restrictions on the exact characters in the user name, and *warn*
if the user name has "strange" characters:
   Warning: user name "foo.bar" contains a dot which conflicts with chown 
user.group syntax
   Warning: user name "0day" is not portable
   Warning: user name "-user" looks like an option ;)
etc.

After all (as other people said) systemd has no such requirements
itself. It is true that such user names are confusing and
non-portable, but if the local admin has or wants to have such an
account for whatever reason, we don't really care.

Zbyszek
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Github systemd issue 6237

[Lennart Poettering]

On Mon, 10.07.17 22:23, Michael Chapman (m...@very.puzzling.org) wrote:

> > Well, it took 3 years or so, until someone noticed the strict rules we
> > enforce. I seriously doubt that naming system users in such unsafe
> > ways is really that wide-spread usage.
> 
> That _could_ be because people that have previously used such a username
> hadn't looked in their logs and noticed that the User= directive wasn't
> being applied. :-)

Well: if you write a unit file, it's probably a good idea to test it
once before deploying it. I mean, it's not that we silently skip stuff
that was OK on sysvinit or so — that's because sysvinit of course never
supported dropping user IDs for you, you always had to do it yourself.

If your service is dropping privs on its own it's of course completely
between you and that service to name the user any way you like. But if
you tell systemd to do this for you and for the service, we do some
basic checking and validation, that's all. Or in other words: if I
write a unit file on my laptop here, and it works, then systemd is
supposed to give you the guarantee that it also works correctly on
my server system I deploy this on, too (to some degree at least), even
if it runs a slightly different Linux distro.

> So be it. I'm fine with us remaining in disagreement... I just wish I
> understood exactly what the security implications are in allowing such
> usernames. I know my colleagues are going to ask me about this, and being
> able to point at something and say "oh yeah, it breaks this specifically"
> would be really handy.

I am pretty sure it would be very welcome if somebody would do a
proper investigation and collect the precise restrictions different
systems in different versions enforce on naming users. And not just
across distro versions and their user name regexes, but also across
application software packages running on top of those distros. I am
not aware of any sufficiently comprehensive list of this.

On current Fedora, the current regex useradd enforces appears to be
this:

[a-zA-Z0-9._][a-zA-Z0-9._-]{0,30}[a-zA-Z0-9._-$]?

If I read things correctly at least... (the trailing $ appears to be a
special exception they added to be nice to Windows, dunno). And this
regex appears not to be configurable.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Github systemd issue 6237

[Lennart Poettering]

On Mon, 10.07.17 15:58, Lennart Poettering (lenn...@poettering.net) wrote:

> On Mon, 10.07.17 15:16, Jan Synacek (jsyna...@redhat.com) wrote:
> 
> > On Mon, Jul 10, 2017 at 12:42 PM, Lennart Poettering
> >  wrote:
> > > Now, because this is so weakly defined, we hence do not follow POSIX
> > > rules, but filter out more that might be dangerous. Specifically:
> > >
> > > 1. We do not permit empty usernames
> > > 2. We don't permit the first character to be numeric
> > >(This also filters out fully numeric user names)
> > > 3. We do not permit dots in usernames, neither at the beginning nor in
> > >the middle.
> > > 4. We do not permit "-" at the beginning of usernames (something which
> > >POSIX explicitly suggests, btw)
> > > 5. We require that the user name fits in the utmp user name field, so
> > >that we can always log properly about it.
> > 
> > Is this documented somewhere? If not, it would be great to have it
> > documented. I'm pretty sure that this exact paragraph would be ok.
> 
> There's a longer (and not entirely complete) comment about this in the
> sources, but other than that it's not explicitly documented.
> 
> If you prep a patch that adds this to the User=/Group= man page, this
> would certainly be welcome. However, it should be reworded, as we
> shouldn't say "We" there, and probably drop explicit references to
> POSIX and utmp there, and instead just dryly state the accepted
> character set + minimum and maximum string lengths.

I have posted a PR documenting this just now:

https://github.com/systemd/systemd/pull/6321

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Github systemd issue 6237

[Oliver Neukum]

Am Montag, den 10.07.2017, 12:57 +0200 schrieb Reindl Harald:
> 
> Am 10.07.2017 um 12:55 schrieb Lennart Poettering:
> > 
> > 
> > The "nobody" user has special semantics on Linux: it's where things
> > are mapped to that can't be mapped otherwise. It's used by user
> > namspacing, by NFS and others. It's really not a good idea, to permit
> > random services to create and access files under that ID
> 
> and run it as root is a better idea?
> seriously?

This is moot. If you specify a user that is considered wrong for whatever
reason the service must fail. Whether we disagree about who decide what is
a valid user name also does not matter. You cannot substitute users for
any reason.

Regards
Oliver

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Github systemd issue 6237

[Lennart Poettering]

On Mon, 10.07.17 15:16, Jan Synacek (jsyna...@redhat.com) wrote:

> On Mon, Jul 10, 2017 at 12:42 PM, Lennart Poettering
>  wrote:
> > Now, because this is so weakly defined, we hence do not follow POSIX
> > rules, but filter out more that might be dangerous. Specifically:
> >
> > 1. We do not permit empty usernames
> > 2. We don't permit the first character to be numeric
> >(This also filters out fully numeric user names)
> > 3. We do not permit dots in usernames, neither at the beginning nor in
> >the middle.
> > 4. We do not permit "-" at the beginning of usernames (something which
> >POSIX explicitly suggests, btw)
> > 5. We require that the user name fits in the utmp user name field, so
> >that we can always log properly about it.
> 
> Is this documented somewhere? If not, it would be great to have it
> documented. I'm pretty sure that this exact paragraph would be ok.

There's a longer (and not entirely complete) comment about this in the
sources, but other than that it's not explicitly documented.

If you prep a patch that adds this to the User=/Group= man page, this
would certainly be welcome. However, it should be reworded, as we
shouldn't say "We" there, and probably drop explicit references to
POSIX and utmp there, and instead just dryly state the accepted
character set + minimum and maximum string lengths.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Github systemd issue 6237

[Jan Synacek]

On Mon, Jul 10, 2017 at 12:42 PM, Lennart Poettering
 wrote:
> Now, because this is so weakly defined, we hence do not follow POSIX
> rules, but filter out more that might be dangerous. Specifically:
>
> 1. We do not permit empty usernames
> 2. We don't permit the first character to be numeric
>(This also filters out fully numeric user names)
> 3. We do not permit dots in usernames, neither at the beginning nor in
>the middle.
> 4. We do not permit "-" at the beginning of usernames (something which
>POSIX explicitly suggests, btw)
> 5. We require that the user name fits in the utmp user name field, so
>that we can always log properly about it.

Is this documented somewhere? If not, it would be great to have it
documented. I'm pretty sure that this exact paragraph would be ok.

Cheers,
-- 
Jan Synacek
Software Engineer, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Github systemd issue 6237

[Michael Chapman]

On Mon, 10 Jul 2017, Lennart Poettering wrote:

On Mon, 10.07.17 21:15, Michael Chapman (m...@very.puzzling.org) wrote:


Now, I do think that systemd has the duty to complain about any system
user names outside of the safe range. Not only for security reasons,
but also for portability and compatibility reasons: I think we should
ensure that unit files remain portable, and hence we should try to
filter out early stuff that's unlikely going to work outside of the
local scope.


I'm curious as to what you consider portability and compatibility
here.


I want that units written on a system A are likely to work on a system
B. And this means that making use of concepts that are valid on A but
knowingly invalid on B is something we should complain loudly about.

Sure, there are always limitations to make things portable. But this
specific issue is an easy one, and a widely understood one (again:
google for it).


But there are less obviously bad usernames, because -- as you point out --
they're _actually in use already_. I myself already have systems with
usernames that begin with a digit; I don't want those systems to suddenly
break just because I update the Linux release to something that runs
systemd. (In practice they probably won't break, since I'm unlikely to write
system units for these users. But the principle of the matter
stands.)


Well, it took 3 years or so, until someone noticed the strict rules we
enforce. I seriously doubt that naming system users in such unsafe
ways is really that wide-spread usage.


That _could_ be because people that have previously used such a username 
hadn't looked in their logs and noticed that the User= directive wasn't 
being applied. :-)



Sorry, but I really can't see how forbidding usernames like "joe.hacker" or
"0day" improves security. As you said, they're perfectly valid
usernames.


Did I say that? I really don't think they are "perfectly valid"! They
are questionable on all levels. And if people use them for regular
users that's fine for them, but for system users I think stricter
requirements need to apply.

But anyway, I doubt we have to continue this here, we have different
understandings of security. I think validation is a good thing, and
filtering out dangerous strings early is a good thing.

People can always shoot themselves in the foot, and you have every
right to, but I really doubt this easy, well understood superficial
check is the right place to insist that the right to shooting yourself
in the foot is more important than the intention to secure things
down.

Lennart


So be it. I'm fine with us remaining in disagreement... I just wish I 
understood exactly what the security implications are in allowing such 
usernames. I know my colleagues are going to ask me about this, and being 
able to point at something and say "oh yeah, it breaks this specifically" 
would be really handy.

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

[Michael Chapman]

On Mon, 10 Jul 2017, Lennart Poettering wrote:

On Mon, 10.07.17 21:51, Michael Chapman (m...@very.puzzling.org) wrote:


This all stems from my experiences with PulseAudio back in the day:
People do not grok the effect of fork(): it only duplicates the
invoking thread, not any other threads of the process, moreover all
data structures are copied as they are, and that's a time bomb really:
consider one of our context objects is being used by one thread at the
moment another thread invokes fork(): the thread using the object is
busy making changes to the object, rearranging some datastructure (for
example, rehashing a hash table, because it hit its fill limit) and
suchlike. Now the fork() happens while it is doing that: the data
structure will be copied in its half-written, half-updated status quo,
and in the child process there's no thread that could finish what has
been started, and there's neither a way to rollback the changes that
are in progress.

[...]

Thanks, that really does clear things up.

It's a pity glibc doesn't provide an equivalent for pthread_atfork() outside
of the pthread library. Having a notification that a fork has just occurred
would allow us to do the PID caching ourselves.


Well, pthread_atfork() is probably more a source of problems than a solution
for them.

Mutexes and fork() do not mix well: if you have a thread that acquired
a mutex right before a fork() then it will cease to exist but the
mutex remains locked. Now, you could use pthread_atfork() to unlock
it, but that really works only in trivial cases, with trivial data
structures, and otherwise creates ABBA problems and similar. I mean,
mutexes are supposed to make pieces of code atomic from the outside
view: but if you duplicate a process without the thread it will appear
aborted to the outside, and that's quite far from "atomic"...


I understand that... which is why I was only talking about PID caching. 
That is, it could be used to avoid the getpid() calls.


Anyway, it's all moot as I don't think we'd want to use pthread_atfork in 
any systemd APIs -- I'm not sure if they all link to libpthread yet 
anyway.



Of course, there's still a problem with people calling the clone syscall
directly... but I think once people start doing that we have to trust them
to know what they're doing.


Yes: if you invoke clone() directly, you should really invoke execve()
too soon, and in the time between these two syscalls you should not
invoke getpid() and limit yourself to known safe calls.

Lennart


___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

[Lennart Poettering]

On Mon, 10.07.17 21:51, Michael Chapman (m...@very.puzzling.org) wrote:

> > This all stems from my experiences with PulseAudio back in the day:
> > People do not grok the effect of fork(): it only duplicates the
> > invoking thread, not any other threads of the process, moreover all
> > data structures are copied as they are, and that's a time bomb really:
> > consider one of our context objects is being used by one thread at the
> > moment another thread invokes fork(): the thread using the object is
> > busy making changes to the object, rearranging some datastructure (for
> > example, rehashing a hash table, because it hit its fill limit) and
> > suchlike. Now the fork() happens while it is doing that: the data
> > structure will be copied in its half-written, half-updated status quo,
> > and in the child process there's no thread that could finish what has
> > been started, and there's neither a way to rollback the changes that
> > are in progress.
> [...]
> 
> Thanks, that really does clear things up.
> 
> It's a pity glibc doesn't provide an equivalent for pthread_atfork() outside
> of the pthread library. Having a notification that a fork has just occurred
> would allow us to do the PID caching ourselves.

Well, pthread_atfork() is probably more a source of problems than a solution
for them.

Mutexes and fork() do not mix well: if you have a thread that acquired
a mutex right before a fork() then it will cease to exist but the
mutex remains locked. Now, you could use pthread_atfork() to unlock
it, but that really works only in trivial cases, with trivial data
structures, and otherwise creates ABBA problems and similar. I mean,
mutexes are supposed to make pieces of code atomic from the outside
view: but if you duplicate a process without the thread it will appear
aborted to the outside, and that's quite far from "atomic"...

> Of course, there's still a problem with people calling the clone syscall
> directly... but I think once people start doing that we have to trust them
> to know what they're doing.

Yes: if you invoke clone() directly, you should really invoke execve()
too soon, and in the time between these two syscalls you should not
invoke getpid() and limit yourself to known safe calls.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

[Michael Chapman]

On Mon, 10 Jul 2017, Lennart Poettering wrote:

On Sat, 08.07.17 16:24, Michael Chapman (m...@very.puzzling.org) wrote:


On Sat, 8 Jul 2017, vcap...@pengaru.com wrote:

In doing some casual journalctl profiling and stracing, it became apparent
that `journalctl -b --no-pager` runs across a significant quantity of logs,
~10% of the time was thrown away on getpid() calls due to commmit a65f06b.

As-is:
# time ./journalctl -b --no-pager > /dev/null

real0m11.033s
user0m10.084s
sys 0m0.943s


After changing journal_pid_changed() to simply return 1:
# time ./journalctl -b --no-pager > /dev/null

 real0m9.641s
 user0m9.449s
 sys 0m0.191s


[...]


As this is public sd-journal API, it's somewhat set in stone.


So it's arguable whether making an API work in _more_ situations than it
previously did is a "breaking" change.

I've tried to go through the history for the various *_pid_changed()
functions in the APIs systemd presents, and I'm struggling to find a good
justification for them. It seems like it was originally added for sd-bus in:

  
https://github.com/systemd/systemd/commit/d5a2b9a6f455468a0f29483303657ab4fd7013d8

And then other APIs copied it to be consistent with sd-bus:

  
https://github.com/systemd/systemd/commit/a65f06bb27688a6738f2f94b7f055f4c66768d63
  
https://github.com/systemd/systemd/commit/eaa3cbef3b8c214cd5c2d75b04e70ad477187e17
  
https://github.com/systemd/systemd/commit/adf412b9ec7292e0c83aaf9ab93e08c2c8bd524a

Unfortunately none of these commits describe what will go wrong if one of
these APIs is used across fork. Does anybody know what specifically is the
problem being addressed here? Can we detect this problem in some
other way?


This all stems from my experiences with PulseAudio back in the day:
People do not grok the effect of fork(): it only duplicates the
invoking thread, not any other threads of the process, moreover all
data structures are copied as they are, and that's a time bomb really:
consider one of our context objects is being used by one thread at the
moment another thread invokes fork(): the thread using the object is
busy making changes to the object, rearranging some datastructure (for
example, rehashing a hash table, because it hit its fill limit) and
suchlike. Now the fork() happens while it is doing that: the data
structure will be copied in its half-written, half-updated status quo,
and in the child process there's no thread that could finish what has
been started, and there's neither a way to rollback the changes that
are in progress.

[...]

Thanks, that really does clear things up.

It's a pity glibc doesn't provide an equivalent for pthread_atfork() 
outside of the pthread library. Having a notification that a fork has just 
occurred would allow us to do the PID caching ourselves.


Of course, there's still a problem with people calling the clone syscall 
directly... but I think once people start doing that we have to trust them 
to know what they're doing.

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Github systemd issue 6237

[Lennart Poettering]

On Mon, 10.07.17 21:15, Michael Chapman (m...@very.puzzling.org) wrote:

> > Now, I do think that systemd has the duty to complain about any system
> > user names outside of the safe range. Not only for security reasons,
> > but also for portability and compatibility reasons: I think we should
> > ensure that unit files remain portable, and hence we should try to
> > filter out early stuff that's unlikely going to work outside of the
> > local scope.
> 
> I'm curious as to what you consider portability and compatibility
> here.

I want that units written on a system A are likely to work on a system
B. And this means that making use of concepts that are valid on A but
knowingly invalid on B is something we should complain loudly about.

Sure, there are always limitations to make things portable. But this
specific issue is an easy one, and a widely understood one (again:
google for it).

> But there are less obviously bad usernames, because -- as you point out --
> they're _actually in use already_. I myself already have systems with
> usernames that begin with a digit; I don't want those systems to suddenly
> break just because I update the Linux release to something that runs
> systemd. (In practice they probably won't break, since I'm unlikely to write
> system units for these users. But the principle of the matter
> stands.)

Well, it took 3 years or so, until someone noticed the strict rules we
enforce. I seriously doubt that naming system users in such unsafe
ways is really that wide-spread usage.

> Sorry, but I really can't see how forbidding usernames like "joe.hacker" or
> "0day" improves security. As you said, they're perfectly valid
> usernames.

Did I say that? I really don't think they are "perfectly valid"! They
are questionable on all levels. And if people use them for regular
users that's fine for them, but for system users I think stricter
requirements need to apply.

But anyway, I doubt we have to continue this here, we have different
understandings of security. I think validation is a good thing, and
filtering out dangerous strings early is a good thing.

People can always shoot themselves in the foot, and you have every
right to, but I really doubt this easy, well understood superficial
check is the right place to insist that the right to shooting yourself
in the foot is more important than the intention to secure things
down.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

[Lennart Poettering]

On Sat, 08.07.17 16:24, Michael Chapman (m...@very.puzzling.org) wrote:

> On Sat, 8 Jul 2017, vcap...@pengaru.com wrote:
> > In doing some casual journalctl profiling and stracing, it became apparent
> > that `journalctl -b --no-pager` runs across a significant quantity of logs,
> > ~10% of the time was thrown away on getpid() calls due to commmit a65f06b.
> > 
> > As-is:
> > # time ./journalctl -b --no-pager > /dev/null
> > 
> > real0m11.033s
> > user0m10.084s
> > sys 0m0.943s
> > 
> > 
> > After changing journal_pid_changed() to simply return 1:
> > # time ./journalctl -b --no-pager > /dev/null
> > 
> >  real0m9.641s
> >  user0m9.449s
> >  sys 0m0.191s
> 
> [...]
> 
> > As this is public sd-journal API, it's somewhat set in stone.
> 
> So it's arguable whether making an API work in _more_ situations than it
> previously did is a "breaking" change.
> 
> I've tried to go through the history for the various *_pid_changed()
> functions in the APIs systemd presents, and I'm struggling to find a good
> justification for them. It seems like it was originally added for sd-bus in:
> 
>   
> https://github.com/systemd/systemd/commit/d5a2b9a6f455468a0f29483303657ab4fd7013d8
> 
> And then other APIs copied it to be consistent with sd-bus:
> 
>   
> https://github.com/systemd/systemd/commit/a65f06bb27688a6738f2f94b7f055f4c66768d63
>   
> https://github.com/systemd/systemd/commit/eaa3cbef3b8c214cd5c2d75b04e70ad477187e17
>   
> https://github.com/systemd/systemd/commit/adf412b9ec7292e0c83aaf9ab93e08c2c8bd524a
> 
> Unfortunately none of these commits describe what will go wrong if one of
> these APIs is used across fork. Does anybody know what specifically is the
> problem being addressed here? Can we detect this problem in some
> other way?

This all stems from my experiences with PulseAudio back in the day:
People do not grok the effect of fork(): it only duplicates the
invoking thread, not any other threads of the process, moreover all
data structures are copied as they are, and that's a time bomb really:
consider one of our context objects is being used by one thread at the
moment another thread invokes fork(): the thread using the object is
busy making changes to the object, rearranging some datastructure (for
example, rehashing a hash table, because it hit its fill limit) and
suchlike. Now the fork() happens while it is doing that: the data
structure will be copied in its half-written, half-updated status quo,
and in the child process there's no thread that could finish what has
been started, and there's neither a way to rollback the changes that
are in progress.

In order to protect us against the time bomb we hence have these
checks that error our early and explicitly on all our calls, with
clean error codes as soon as a fork() is detected but people try to
reuse our objects.

Now, you could argue, that on purely single-threaded processes all
this is too restrictive, but it's really not like that anymore, as many
libraries (including our own, for example in sd-resolve) fork off
background threads, and invoke our code in them. Never forget that our
libraries such as sd-bus and so on or invoked by NSS, which is very
frequently invoked in threads (because NSS is synchronous, which is
very much incompatible with most graphical spps).

So yeah, we never know in which context our stuff is called, and hence
I think it's safe to refuse in a friendly way early on rather than
just wait for the time bomb to explode when our code touches those
objects again.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

[Lennart Poettering]

On Fri, 07.07.17 14:35, vcap...@pengaru.com (vcap...@pengaru.com) wrote:

> On Fri, Jul 07, 2017 at 01:49:54PM -0700, vcap...@pengaru.com wrote:
> > On Fri, Jul 07, 2017 at 08:37:08PM +, Mantas Mikulėnas wrote:
> > > Back when that commit was made, didn't glibc cache the getpid() result in
> > > userspace? That would explain why it was not noticed.
> >
> > Hmm, this crossed my mind, and come to think of it I did a dist-upgrade
> > from Debian jessie to stretch overnight machine and haven't rebooted.
> > 
> > Perhaps the vdso isn't working and the costly getpid() is a red herring, 
> > will
> > reboot and retest to confirm.
> > 
> 
> It appears Debian has a glibc patch to disable the caching (I was unaware
> such an elaborate dance was being performed to cache this!)
> 
> https://anonscm.debian.org/cgit/pkg-glibc/glibc.git/commit/debian/patches/any?id=5850253f509604dd46a6131acc057ea26e1588ba
> 
> Unsure where I stand on core system software assuming certain syscalls are
> always going to be exceptionally cheap though...

Hmm, we generally assume that a couple of syscalls are indeed very
cheap. That's getpid(), getuid() as well as the various ways to get
the system time.

Yes, glibc caching of the PID is very annoying in some cases (as it
complicates code that invokes clone() through a direct syscall
invocation), but still, I think it should be OK assume that it is
cheap to invoke it.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Github systemd issue 6237

[Michael Chapman]

On Mon, 10 Jul 2017, Lennart Poettering wrote:

On Thu, 06.07.17 13:21, Michael Chapman (m...@very.puzzling.org) wrote:


On Thu, 6 Jul 2017, Zbigniew Jędrzejewski-Szmek wrote:

On Thu, Jul 06, 2017 at 01:43:32AM +0200, Reindl Harald wrote:

well, it even don't look but pretend it can't while it does which is
the worst type of operations possible - as long as "adduser" of the
underlying OS accepts and create "0pointer" systemd has *no business
at all* to pretend it can't


Then it's good the that it doesn't ;)

# adduser 0pointer

adduser: Please enter a username matching the regular expression configured
via the NAME_REGEX configuration variable.  Use the `--force-badname'
option to relax this check or reconfigure NAME_REGEX.


I know you really only brought this up to counter Reindl's comment, but I
think it's important to point out that adduser's behaviour here is due to
its default configuration -- not due to any fundamental "problems" with
particular usernames. It's not clear why adduser's developers thought it was
a good default.

I guess what I'm saying is that saying "systemd should not support usernames
that start with a digit, since adduser doesn't" is problematic for at least
two reasons. First, adduser can be reconfigured by the sysadmin to allow
such usernames; and second, systemd places *fewer* restrictions on usernames
than adduser's default configuration. systemd allows usernames containing
uppercase letters and underscores, for instance.


Note one major difference between "adduser" and the unit file setting
"Unit=". The former is a tool you can create regular users with, while
the latter strictly applies to system users, as that's what system
services run as. And yes, different rules apply for system users than
for regular users.

And "0foobar" remains unportable and a bad idea, even if the user
bends his local system in the right way to make it accept it.


To summarize my thoughts on this matter, I think it's fine to restrict
usernames, but only for _very_ good reason. Specifically, we should not
justify such restrictions simply because they exist in one form or another
in other utilities. valid_user_group_name() currently disallows dots, for
instance, and while I recognize that using dots in a username can sometimes
be problematic, it is not in and of itself invalid. If other software can't
handle dots in usernames, that's their problem. libc can, and that's all
that's required to support it in order to use it in User= on most
systems.


I am sorry, but you and I have very different understanding of
computer security. I do believe it is essential to validate all input,
and stick to safe input wherever we can.


That is a misrepresentation of my viewpoint.

I _do_ think systemd should validate all input. I think my other posts in 
this thread make this clear: I want to see systemd complain noisily when 
unit validation fails.


However, I do not think systemd should validate input more than it needs 
to. Just because a particular value may (and _only_ may) cause problems 
downstream of systemd does not mean that systemd should outright forbid 
it. If it doesn't cause problems in systemd, it's not our business to 
prevent its use.



I understand that you'd like to remove input validation from the
systemd codebase, and I welcome you to patch your local systemd
version for it, but please understand that in systemd upstream this is
not how things can work. Sorry.

Lennart___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] [PATCH] rules: block - add dm devices to whitelist

[David Disseldorp]

On Mon, 10 Jul 2017 12:47:24 +0200, Peter Rajnoha wrote:

> On 07/10/2017 12:14 PM, Peter Rajnoha wrote:
...
> > Yes, please, any rules for symlinks which should be created under
> > /dev/disk for DM devices (including all its subsystems like LVM,
> > mpath...) should go into 13-dm-disk.rules that is part of LVM/DM source
> > tree:
> > 
> > https://sourceware.org/git/?p=lvm2.git;a=blob;f=udev/13-dm-disk.rules.in
> > 
> > Now, when we create a partition over a DM device, there's a new mapping
> > created on top for each partition (either by calling kpartx manually or
> > by having it created by partitioning tool directly if it supports that).
> > So in this case, it's not the kernel directly who creates the
> > partitions, but they're simply another DM devices created on top of the
> > underlying DM device to represent these partitions. But I think that
> > doesn't matter - we should still create those symlinks for people to
> > still have a possibility to reference the device by its part uuid - I'll
> > fix 13-dm-disk.rules to include this.
> >   
> 
> Fixed here:
> 
> https://sourceware.org/git/?p=lvm2.git;a=commit;h=c48149cf80c6582c2369bc7f8a33d794021d9dae

Looks good and works for me - thanks Peter.

Cheers, David
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Github systemd issue 6237

[Michael Chapman]

On Mon, 10 Jul 2017, Lennart Poettering wrote:

On Thu, 06.07.17 09:36, Michael Chapman (m...@very.puzzling.org) wrote:


User=0day fails a syntactic validation, not a semantic validation. systemd
never even checks to see whether the user exists when the unit is loaded.
And nor should it! The user must be allowed to not exist at unit-load time.

Contrary to some of the comments in this thread, there is no point in
systemd's operation where it goes "oh look, that user actually exists but
I'm going to pretend it doesn't".


Well, that's where we disagree: POSIX allows you to create a user
named "1000" that maps to a UID "1001". I find it very hard to swallow
that people actually defend that as a good concept, and claim that
systemd was negatively impacting security here by refusing this and
other ambiguous mappings.


I do think that User=1000 should always be treated as a UID, regardless of 
whether there's a "1000" username on the system or not.



We need to validate the input we get, that's security 101. And POSIX
is *not* a good standard to strictly follow here and use for
validating user names, because it's *very* underdefined:

- It permits fully numeric names
- It makes no size restrictions (strictly reading the spec even permits
 zero-length usernames!)
- It permits dots, which is conflicting with traditional chown syntax
- It permits dots at the beginning of usernames (which is dangerous
 security-wise as this permits users to hide home directories)
- It permits naming users "-", which is often used as special "does
 not apply" like value

Now, because this is so weakly defined, we hence do not follow POSIX
rules, but filter out more that might be dangerous. Specifically:

1. We do not permit empty usernames
2. We don't permit the first character to be numeric
  (This also filters out fully numeric user names)
3. We do not permit dots in usernames, neither at the beginning nor in
  the middle.
4. We do not permit "-" at the beginning of usernames (something which
  POSIX explicitly suggests, btw)
5. We require that the user name fits in the utmp user name field, so
  that we can always log properly about it.


I think 1, 4 and 5 are good things.

For 2 and 3, however, I'm not so sure.


Note that this isn't even as strict as other systems go. For example,
we do permit uppercase characters, and we do permit underscores.

Are these random rules we came up here? Nope. This actually matches
the various requirements enforced by the regex strings used by various
distros in one way or another. Moreover, if you type "valid linux user
name characters" into google, among the top links you'll find stuff
like this:

https://stackoverflow.com/questions/6949667/what-are-the-real-rules-for-linux-usernames-on-centos-6-and-rhel-6

or this:

https://unix.stackexchange.com/questions/157426/what-is-the-regex-to-validate-linux-users

Which generally suggest similar rules.

Now, I do think that systemd has the duty to complain about any system
user names outside of the safe range. Not only for security reasons,
but also for portability and compatibility reasons: I think we should
ensure that unit files remain portable, and hence we should try to
filter out early stuff that's unlikely going to work outside of the
local scope.


I'm curious as to what you consider portability and compatibility here.

There are some "obviously bad" usernames, like the empty string. We should 
definitely prevent these from being used, and I do hope we can get systemd 
to complain noisily should somebody try to use them.


But there are less obviously bad usernames, because -- as you point out -- 
they're _actually in use already_. I myself already have systems with 
usernames that begin with a digit; I don't want those systems to suddenly 
break just because I update the Linux release to something that runs 
systemd. (In practice they probably won't break, since I'm unlikely to 
write system units for these users. But the principle of the matter 
stands.)


For these "less obviously bad" usernames, I think we should just rely on 
getpwnam() doing its own validation -- that is, in systemd's child process 
when it's spawning commands. After all, getpwnam() may have any kinds of 
restrictions due to the system's NSS config... or it may not. We're not in 
any place to second-guess what its limitations are.


So long as _we_ never treat a User= value incorrectly (which we don't, we 
decide whether a it's a UID, a username, or an invalid value very early 
on) everything should just work fine.


Obviously we should be encouraging software vendors and distributors to 
use nice usernames. The portability and compatibility comes from them. But 
if a sysadmin wants to run one of their own services as User=0day I don't 
think we should be getting in the way of that -- they should know what the 
portability issues are with that, if any.



We should also not pretend all was good if other tools make less
careful restrictions and permit usernames that 

Re: [systemd-devel] Github systemd issue 6237

[Reindl Harald]

Am 10.07.2017 um 12:42 schrieb Lennart Poettering:

(I do accept though that it's a valid discussion whether systemd's
current behaviour of warning and skipping invalid User= rvalues is the
best choice, instead of erroring out completely.)


and *that* is the real point of the whole issue - if one runs "adduser" 
and the user got accepted and  place it into a unit file you can't 
expect that he notices that it get ignored by a random message in the syslog


and that is proven not to happen otherwise a ton of bugs and typos in 
systemd-units whould not have made it into stable updates when at least 
packagers would read their logs properly


well and with your sort of reactions to bugreports like 
https://bugzilla.redhat.com/show_bug.cgi?id=1072368 it's no wonder that 
some single warning get missed in the wood of messages

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Github systemd issue 6237

[Reindl Harald]

Am 10.07.2017 um 12:55 schrieb Lennart Poettering:

On Thu, 06.07.17 10:34, Reindl Harald (h.rei...@thelounge.net) wrote:




Am 06.07.2017 um 09:59 schrieb Jonathan de Boyne Pollard:

Reindl Harald:
  > at least fall back to “nobody”

Jonathan de Boyne Pollard:
  > That idea is wrong.
  >
  > https://news.ycombinator.com/item?id=14681377#14682059

Reindl Harald:
  > better than a stupid [...]

Not really, no.  It's the same category of error, in fact: substituting
an account other than the one that the system administrator explicitly
said to drop privileges to.


yes, it's both nonsense, but when i only have the option to choose between
two types of nonsense i take the one which don't give root permissions


The "nobody" user has special semantics on Linux: it's where things
are mapped to that can't be mapped otherwise. It's used by user
namspacing, by NFS and others. It's really not a good idea, to permit
random services to create and access files under that ID


and run it as root is a better idea?
seriously?
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Github systemd issue 6237

[Lennart Poettering]

On Thu, 06.07.17 10:34, Reindl Harald (h.rei...@thelounge.net) wrote:

> 
> 
> Am 06.07.2017 um 09:59 schrieb Jonathan de Boyne Pollard:
> > Reindl Harald:
> >  > at least fall back to “nobody”
> > 
> > Jonathan de Boyne Pollard:
> >  > That idea is wrong.
> >  >
> >  > https://news.ycombinator.com/item?id=14681377#14682059
> > 
> > Reindl Harald:
> >  > better than a stupid [...]
> > 
> > Not really, no.  It's the same category of error, in fact: substituting
> > an account other than the one that the system administrator explicitly
> > said to drop privileges to.
> 
> yes, it's both nonsense, but when i only have the option to choose between
> two types of nonsense i take the one which don't give root permissions

The "nobody" user has special semantics on Linux: it's where things
are mapped to that can't be mapped otherwise. It's used by user
namspacing, by NFS and others. It's really not a good idea, to permit
random services to create and access files under that ID.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] [PATCH] rules: block - add dm devices to whitelist

[Peter Rajnoha]

On 07/10/2017 12:14 PM, Peter Rajnoha wrote:
> On 07/10/2017 11:53 AM, Lennart Poettering wrote:
>> On Mon, 10.07.17 11:37, David Disseldorp (dd...@suse.de) wrote:
>>
>>> Thanks for the feedback, Lennart...
>>>
>>> On Mon, 10 Jul 2017 10:38:38 +0200, Lennart Poettering wrote:
>>>
 On Wed, 05.07.17 13:01, David Disseldorp (dd...@suse.de) wrote:

> Ceph relies on by-partuuid symlinks, in order to locate the journal
> partition from a given OSD partition. For details, see
> http://tracker.ceph.com/issues/19489.  

 This appears way too broad, as it would apply to all LVM and all other
 devices.

 It appears to me Ceph should do the same as LVM does for this, and
 ship its own set of rules, and be careful to only match against the
 actual devices it creates.
>>>
>>> We can certainly do this in a Ceph specific manner via the existing
>>> 95-ceph-osd.rules, but my impression was that the by-partuuid symlinks
>>> are "owned" by 60-persistent-storage.rules .
>>>
>>> If you don't think the existence of these symlinks for dm paths will be
>>> of use to others then I'll go ahead and propose it as a Ceph only
>>> change.
>>
>> Hmmm, so thinking about this again: "partuuid" is actually for GPT
>> partition UUIDs if I recall recorrectly, they wouldn't be generated
>> for DM devices anyway, since they are one layer removed from the GPT,
>> so are you even sure this will do what you are asking for?
>>
>> But even if this actually works: DM links so far are created by the
>> LVM/libdevicemapper/ packages, not by udev, and I don't think this
>> should change. Hence, please talk to the LVM/libdevicemapper folks
>> about this and ask them for including it.
>>
> 
> Yes, please, any rules for symlinks which should be created under
> /dev/disk for DM devices (including all its subsystems like LVM,
> mpath...) should go into 13-dm-disk.rules that is part of LVM/DM source
> tree:
> 
> https://sourceware.org/git/?p=lvm2.git;a=blob;f=udev/13-dm-disk.rules.in
> 
> Now, when we create a partition over a DM device, there's a new mapping
> created on top for each partition (either by calling kpartx manually or
> by having it created by partitioning tool directly if it supports that).
> So in this case, it's not the kernel directly who creates the
> partitions, but they're simply another DM devices created on top of the
> underlying DM device to represent these partitions. But I think that
> doesn't matter - we should still create those symlinks for people to
> still have a possibility to reference the device by its part uuid - I'll
> fix 13-dm-disk.rules to include this.
> 

Fixed here:

https://sourceware.org/git/?p=lvm2.git;a=commit;h=c48149cf80c6582c2369bc7f8a33d794021d9dae

-- 
Peter
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Github systemd issue 6237

[Lennart Poettering]

On Thu, 06.07.17 09:36, Michael Chapman (m...@very.puzzling.org) wrote:

> User=0day fails a syntactic validation, not a semantic validation. systemd
> never even checks to see whether the user exists when the unit is loaded.
> And nor should it! The user must be allowed to not exist at unit-load time.
> 
> Contrary to some of the comments in this thread, there is no point in
> systemd's operation where it goes "oh look, that user actually exists but
> I'm going to pretend it doesn't".

Well, that's where we disagree: POSIX allows you to create a user
named "1000" that maps to a UID "1001". I find it very hard to swallow
that people actually defend that as a good concept, and claim that
systemd was negatively impacting security here by refusing this and
other ambiguous mappings.

We need to validate the input we get, that's security 101. And POSIX
is *not* a good standard to strictly follow here and use for
validating user names, because it's *very* underdefined:

- It permits fully numeric names
- It makes no size restrictions (strictly reading the spec even permits
  zero-length usernames!)
- It permits dots, which is conflicting with traditional chown syntax
- It permits dots at the beginning of usernames (which is dangerous
  security-wise as this permits users to hide home directories)
- It permits naming users "-", which is often used as special "does
  not apply" like value

Now, because this is so weakly defined, we hence do not follow POSIX
rules, but filter out more that might be dangerous. Specifically:

1. We do not permit empty usernames
2. We don't permit the first character to be numeric
   (This also filters out fully numeric user names)
3. We do not permit dots in usernames, neither at the beginning nor in
   the middle.
4. We do not permit "-" at the beginning of usernames (something which
   POSIX explicitly suggests, btw)
5. We require that the user name fits in the utmp user name field, so
   that we can always log properly about it.

Note that this isn't even as strict as other systems go. For example,
we do permit uppercase characters, and we do permit underscores.

Are these random rules we came up here? Nope. This actually matches
the various requirements enforced by the regex strings used by various
distros in one way or another. Moreover, if you type "valid linux user
name characters" into google, among the top links you'll find stuff
like this:

https://stackoverflow.com/questions/6949667/what-are-the-real-rules-for-linux-usernames-on-centos-6-and-rhel-6

or this:

https://unix.stackexchange.com/questions/157426/what-is-the-regex-to-validate-linux-users

Which generally suggest similar rules.

Now, I do think that systemd has the duty to complain about any system
user names outside of the safe range. Not only for security reasons,
but also for portability and compatibility reasons: I think we should
ensure that unit files remain portable, and hence we should try to
filter out early stuff that's unlikely going to work outside of the
local scope.

We should also not pretend all was good if other tools make less
careful restrictions and permit usernames that cause ambiguities. Yes,
a lot of software is very sloppy with validating input, and yes, some
adduser/useradd implementations allow you to create a user called
"1000", but I am very sure that their sloppiness should not leak into
our codebase.

Also, do note that system users are different concepts than regular
users: system users are concepts required for system services which
are usually put together by developers, packagers and administrators
who hopefully understand these issues to some point and pick good
names instead.

And of course, note that systemd is open source. if you don't like the
restrictions we make, you can patch them out (or even use some other
project). But again, I think it's our duty to help build a more secure
ecosystem (and we do that by validating our input, by running systemd
services in sandboxes and so on), and hence these restrictions really
need to stay.

(I do accept though that it's a valid discussion whether systemd's
current behaviour of warning and skipping invalid User= rvalues is the
best choice, instead of erroring out completely.)

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Github systemd issue 6237

[Lennart Poettering]

On Thu, 06.07.17 13:21, Michael Chapman (m...@very.puzzling.org) wrote:

> On Thu, 6 Jul 2017, Zbigniew Jędrzejewski-Szmek wrote:
> > On Thu, Jul 06, 2017 at 01:43:32AM +0200, Reindl Harald wrote:
> > > well, it even don't look but pretend it can't while it does which is
> > > the worst type of operations possible - as long as "adduser" of the
> > > underlying OS accepts and create "0pointer" systemd has *no business
> > > at all* to pretend it can't
> > 
> > Then it's good the that it doesn't ;)
> > 
> > # adduser 0pointer
> > 
> > adduser: Please enter a username matching the regular expression configured
> > via the NAME_REGEX configuration variable.  Use the `--force-badname'
> > option to relax this check or reconfigure NAME_REGEX.
> 
> I know you really only brought this up to counter Reindl's comment, but I
> think it's important to point out that adduser's behaviour here is due to
> its default configuration -- not due to any fundamental "problems" with
> particular usernames. It's not clear why adduser's developers thought it was
> a good default.
> 
> I guess what I'm saying is that saying "systemd should not support usernames
> that start with a digit, since adduser doesn't" is problematic for at least
> two reasons. First, adduser can be reconfigured by the sysadmin to allow
> such usernames; and second, systemd places *fewer* restrictions on usernames
> than adduser's default configuration. systemd allows usernames containing
> uppercase letters and underscores, for instance.

Note one major difference between "adduser" and the unit file setting
"Unit=". The former is a tool you can create regular users with, while
the latter strictly applies to system users, as that's what system
services run as. And yes, different rules apply for system users than
for regular users.

And "0foobar" remains unportable and a bad idea, even if the user
bends his local system in the right way to make it accept it.

> To summarize my thoughts on this matter, I think it's fine to restrict
> usernames, but only for _very_ good reason. Specifically, we should not
> justify such restrictions simply because they exist in one form or another
> in other utilities. valid_user_group_name() currently disallows dots, for
> instance, and while I recognize that using dots in a username can sometimes
> be problematic, it is not in and of itself invalid. If other software can't
> handle dots in usernames, that's their problem. libc can, and that's all
> that's required to support it in order to use it in User= on most
> systems.

I am sorry, but you and I have very different understanding of
computer security. I do believe it is essential to validate all input,
and stick to safe input wherever we can.

I understand that you'd like to remove input validation from the
systemd codebase, and I welcome you to patch your local systemd
version for it, but please understand that in systemd upstream this is
not how things can work. Sorry.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] [PATCH] rules: block - add dm devices to whitelist

[Peter Rajnoha]

On 07/10/2017 11:53 AM, Lennart Poettering wrote:
> On Mon, 10.07.17 11:37, David Disseldorp (dd...@suse.de) wrote:
> 
>> Thanks for the feedback, Lennart...
>>
>> On Mon, 10 Jul 2017 10:38:38 +0200, Lennart Poettering wrote:
>>
>>> On Wed, 05.07.17 13:01, David Disseldorp (dd...@suse.de) wrote:
>>>
 Ceph relies on by-partuuid symlinks, in order to locate the journal
 partition from a given OSD partition. For details, see
 http://tracker.ceph.com/issues/19489.  
>>>
>>> This appears way too broad, as it would apply to all LVM and all other
>>> devices.
>>>
>>> It appears to me Ceph should do the same as LVM does for this, and
>>> ship its own set of rules, and be careful to only match against the
>>> actual devices it creates.
>>
>> We can certainly do this in a Ceph specific manner via the existing
>> 95-ceph-osd.rules, but my impression was that the by-partuuid symlinks
>> are "owned" by 60-persistent-storage.rules .
>>
>> If you don't think the existence of these symlinks for dm paths will be
>> of use to others then I'll go ahead and propose it as a Ceph only
>> change.
> 
> Hmmm, so thinking about this again: "partuuid" is actually for GPT
> partition UUIDs if I recall recorrectly, they wouldn't be generated
> for DM devices anyway, since they are one layer removed from the GPT,
> so are you even sure this will do what you are asking for?
> 
> But even if this actually works: DM links so far are created by the
> LVM/libdevicemapper/ packages, not by udev, and I don't think this
> should change. Hence, please talk to the LVM/libdevicemapper folks
> about this and ask them for including it.
> 

Yes, please, any rules for symlinks which should be created under
/dev/disk for DM devices (including all its subsystems like LVM,
mpath...) should go into 13-dm-disk.rules that is part of LVM/DM source
tree:

https://sourceware.org/git/?p=lvm2.git;a=blob;f=udev/13-dm-disk.rules.in

Now, when we create a partition over a DM device, there's a new mapping
created on top for each partition (either by calling kpartx manually or
by having it created by partitioning tool directly if it supports that).
So in this case, it's not the kernel directly who creates the
partitions, but they're simply another DM devices created on top of the
underlying DM device to represent these partitions. But I think that
doesn't matter - we should still create those symlinks for people to
still have a possibility to reference the device by its part uuid - I'll
fix 13-dm-disk.rules to include this.

-- 
Peter
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] [PATCH] rules: block - add dm devices to whitelist

[Lennart Poettering]

On Mon, 10.07.17 11:37, David Disseldorp (dd...@suse.de) wrote:

> Thanks for the feedback, Lennart...
> 
> On Mon, 10 Jul 2017 10:38:38 +0200, Lennart Poettering wrote:
> 
> > On Wed, 05.07.17 13:01, David Disseldorp (dd...@suse.de) wrote:
> > 
> > > Ceph relies on by-partuuid symlinks, in order to locate the journal
> > > partition from a given OSD partition. For details, see
> > > http://tracker.ceph.com/issues/19489.  
> > 
> > This appears way too broad, as it would apply to all LVM and all other
> > devices.
> >
> > It appears to me Ceph should do the same as LVM does for this, and
> > ship its own set of rules, and be careful to only match against the
> > actual devices it creates.
> 
> We can certainly do this in a Ceph specific manner via the existing
> 95-ceph-osd.rules, but my impression was that the by-partuuid symlinks
> are "owned" by 60-persistent-storage.rules .
> 
> If you don't think the existence of these symlinks for dm paths will be
> of use to others then I'll go ahead and propose it as a Ceph only
> change.

Hmmm, so thinking about this again: "partuuid" is actually for GPT
partition UUIDs if I recall recorrectly, they wouldn't be generated
for DM devices anyway, since they are one layer removed from the GPT,
so are you even sure this will do what you are asking for?

But even if this actually works: DM links so far are created by the
LVM/libdevicemapper/ packages, not by udev, and I don't think this
should change. Hence, please talk to the LVM/libdevicemapper folks
about this and ask them for including it.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] [PATCH] rules: block - add dm devices to whitelist

[David Disseldorp]

Thanks for the feedback, Lennart...

On Mon, 10 Jul 2017 10:38:38 +0200, Lennart Poettering wrote:

> On Wed, 05.07.17 13:01, David Disseldorp (dd...@suse.de) wrote:
> 
> > Ceph relies on by-partuuid symlinks, in order to locate the journal
> > partition from a given OSD partition. For details, see
> > http://tracker.ceph.com/issues/19489.  
> 
> This appears way too broad, as it would apply to all LVM and all other
> devices.
>
> It appears to me Ceph should do the same as LVM does for this, and
> ship its own set of rules, and be careful to only match against the
> actual devices it creates.

We can certainly do this in a Ceph specific manner via the existing
95-ceph-osd.rules, but my impression was that the by-partuuid symlinks
are "owned" by 60-persistent-storage.rules .

If you don't think the existence of these symlinks for dm paths will be
of use to others then I'll go ahead and propose it as a Ceph only
change.

Cheers, David
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] mkosi.postinst and user home permissions

[Federico Bruni]

Il giorno lun 10 lug 2017 alle 10:46, Lennart Poettering 
 ha scritto:

On Tue, 04.07.17 12:28, Federico Bruni (f...@inventati.org) wrote:


 I'm building fedora-26 in a directory.
 I want to add a normal user, so I've added to mkosi.postinst:

 useradd dev -p mypassword
 usermod dev -a -G wheel

 These two commands work fine, but the permissions are not correct:

 # ls -l /home/
 total 4
 drwxrwxr-x 2 root root 4096 Jul  3 18:13 dev

 Is this a bug? Or maybe what I'm trying to do should be 
accomplished in

 another way?
 As a workaround, I might try add a third line to mkosi.postinst:


Uh, this appears very strange to me, this feels like a bug in
"useradd" indeed. I have no idea why the directory isn#t created
properly...



I guess you missed my follow-ups in this thread.
The problem was caused by a file (.bashrc) I placed in 
mkosi.extra/home/dev.
My understanding is that mkosi first copied that file, then run the 
postinst script containing the useradd command, which failed 
(correctly) as /home/dev already existed.


I've moved the file to mkosi.extra/etc/skel and now everything works 
fine.




___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Is there a reason to run systemd Units with root access?

[Lennart Poettering]

On Tue, 04.07.17 20:33, Mariusz Wojcik (m6woj...@outlook.com) wrote:

> Hi,
> 
> I’m just asking because of the latest “not-a-bug” [1]. As far as I
> know, there aren’t many services that need full root access (maybe
> for getting a low port number). Except for that I don’t see many use
> cases. Therefore, I think it would be useful to make the decision
> for root access more explicit, e.g. User=root is needed to start
> units as root. Also I don’t think it is a sane default is to start
> any unit as root when there is no valid User property. Even the
> security of systemd would benefit because it would save people from
> accidentally running services as root.

Well, UNIX system services traditionally expect to be invoked as "root",
and then drop privs on their own, if they are well written, and
systemd was created to run UNIX system services, hence the default.

But yeah I think today it would be better for services to just let
systemd drop privs for them wherever possible. But it's hard to get
that into people's heads, and it needs to be done on a per-service
basis, so that the right user is used, and the right execution
parameters (for example ambient caps) are passed otherwise.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] [PATCH] rules: block - add dm devices to whitelist

[Tomasz Torcz]

On Mon, Jul 10, 2017 at 10:38:38AM +0200, Lennart Poettering wrote:
> On Wed, 05.07.17 13:01, David Disseldorp (dd...@suse.de) wrote:
> 
> > Ceph relies on by-partuuid symlinks, in order to locate the journal
> > partition from a given OSD partition. For details, see
> > http://tracker.ceph.com/issues/19489.
> 
> This appears way too broad, as it would apply to all LVM and all other
> devices.
> 
> It appears to me Ceph should do the same as LVM does for this, and
> ship its own set of rules, and be careful to only match against the
> actual devices it creates.

 Ceph does not create any devices (except /dev/rbd/* but it's not the case 
here).
Ceph block storage uses any kind of block device for its operation, be it plain
partition, LUKS encrypted storage, LVM, dm-multipath devices and so on.

  Ceph block storage contains few parts - actual storage, write-ahead log,
journal etc. Any of those parts can utilize block devices, so ceph
uses symlinks to point to proper block device:

# ls -l /var/lib/ceph/osd/ceph-11/block
lrwxrwxrwx. 1 root root 33 Jul 10 10:51 /var/lib/ceph/osd/ceph-11/block -> 
/dev/disk/by-partuuid/43bdcb85-06


 Nb. ceph already ships rules for autodetecting if given partition belongs
to ceph (discriminating by partition type, see
https://github.com/ceph/ceph/blob/master/udev/95-ceph-osd.rules ), but 
'by-partuuid'
links should be created by earlier udev rules - like they are for some 
whitelisted
set of device nodes names.

-- 
Tomasz Torcz   "Never underestimate the bandwidth of a station
xmpp: zdzich...@chrome.plwagon filled with backup tapes." -- Jim Gray

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Is there a reason to run systemd Units with root access?

[Reindl Harald]

Am 10.07.2017 um 10:49 schrieb Lennart Poettering:

On Tue, 04.07.17 20:33, Mariusz Wojcik (m6woj...@outlook.com) wrote:


Hi,

I’m just asking because of the latest “not-a-bug” [1]. As far as I
know, there aren’t many services that need full root access (maybe
for getting a low port number). Except for that I don’t see many use
cases. Therefore, I think it would be useful to make the decision
for root access more explicit, e.g. User=root is needed to start
units as root. Also I don’t think it is a sane default is to start
any unit as root when there is no valid User property. Even the
security of systemd would benefit because it would save people from
accidentally running services as root.


Well, UNIX system services traditionally expect to be invoked as "root",
and then drop privs on their own, if they are well written, and
systemd was created to run UNIX system services, hence the default.

But yeah I think today it would be better for services to just let
systemd drop privs for them wherever possible. But it's hard to get
that into people's heads, and it needs to be done on a per-service
basis, so that the right user is used, and the right execution
parameters (for example ambient caps) are passed otherwise.


it's not only about "get into people's heads"

it just don't make any sense for services like a webserver which needs 
to read certificate private keys but the user after drop privileges 
should not be able to read that files in case someone manages to execute 
code (which is not that hard when scripting langauges with commands like 
exec are part of the game)


the same when you read configurations containing sensible passwords, you 
do that before drop privileges and yes read memory with random executed 
coide is way harder than a file with a known path

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] mkosi.postinst and user home permissions

[Lennart Poettering]

On Tue, 04.07.17 12:28, Federico Bruni (f...@inventati.org) wrote:

> Hi all
> 
> I hope this is the right place to ask questions about mkosi.

Hmm, not really. We currently have no mailing list for mkosi
though. Maybe the github issue tracker of mkosi would be the best
place to discuss things like this for now.

But anyway, let's discuss this one issue here, given that it already
has been posted here.

> I'm building fedora-26 in a directory.
> I want to add a normal user, so I've added to mkosi.postinst:
> 
> useradd dev -p mypassword
> usermod dev -a -G wheel
> 
> These two commands work fine, but the permissions are not correct:
> 
> # ls -l /home/
> total 4
> drwxrwxr-x 2 root root 4096 Jul  3 18:13 dev
> 
> Is this a bug? Or maybe what I'm trying to do should be accomplished in
> another way?
> As a workaround, I might try add a third line to mkosi.postinst:

Uh, this appears very strange to me, this feels like a bug in
"useradd" indeed. I have no idea why the directory isn#t created
properly...

if you invoke "stat /home/dev" in that same script right after you
created the user, does it appear correct then?


Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] [PATCH] rules: block - add dm devices to whitelist

[Lennart Poettering]

On Wed, 05.07.17 13:01, David Disseldorp (dd...@suse.de) wrote:

> Ceph relies on by-partuuid symlinks, in order to locate the journal
> partition from a given OSD partition. For details, see
> http://tracker.ceph.com/issues/19489.

This appears way too broad, as it would apply to all LVM and all other
devices.

It appears to me Ceph should do the same as LVM does for this, and
ship its own set of rules, and be careful to only match against the
actual devices it creates.

I hope that makes sense,

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Bouncing interface once chrony is synced

[Lennart Poettering]

On Wed, 05.07.17 10:29, Ian Pilcher (arequip...@gmail.com) wrote:

> I am using CentOS 7 (systemd 219) on a Banana Pi as my residential
> firewall/gateway.  The Banana Pi does not have a persistent clock, so
> it has no idea what the time is until it is able to sync via NTP.  Thus,
> the initial DHCP leases that the BPi receives have incorrect expiration/
> renewal times (since the system can't sync via NTP before it has an IP
> address - chicken and egg).

DHCP only sends times as relative times, precisely to not require
synchronized clocks.

See RFC2131, section 3.3:

"As clients and servers may not have synchronized clocks, times
are represented in DHCP messages as relative times, to be
interpreted with respect to the client's local clock.
Representing relative times in units of seconds in an unsigned 32
bit word gives a range of relative times from 0 to approximately
100 years, which is sufficient for the relative times to be
measured using DHCP."

If your DHCP client implementation gets confused by the wallclock not
being steady then this appears to be a bug in the
implementation. Given that there are so many DHCP clients to choose
from, maybe use a different one?

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Realtime scheduling with CONFIG_RT_GROUP_SCHED=y

[Lennart Poettering]

On Thu, 06.07.17 10:56, Lars Kellogg-Stedman (l...@redhat.com) wrote:

> I'm running on a kernel with CONFIG_RT_GROUP_SCHED=y.  I understand that
> this is counter to the recommendation in the README ("We recommend to turn
> off Real-Time group scheduling in the kernel when using systemd"), but
> I don't have control over the kernel configuration.
> 
> On this system, it appears that starting "docker" (docker-ce-17.06.0.ce-1)
> results in the creation of new cpu cgroups that for some reason apply to
> systemd services.  That is, after starting docker,
> /sys/fs/cgroup/cpu/system.slice exists when previously it didn't.
> 
> Once this happens, a service that attempts to set realtime scheduling
> (SCHED_RR) via sched_setscheduler() will fail, presumably because the
> cgroup has no realtime budget in cpu.rt_runtime_us.
> 
> In older versions of systemd one could handle this using the directives
> described in
> https://www.freedesktop.org/wiki/Software/systemd/MyServiceCantGetRealtime/,
> but unfortunately that document, despite being the number 1 search result
> for pretty much anything involving "systemd" and "realtime", is obsolete
> and those directives no longer exist.
> 
> Is there a way to make this work correctly with modern versions of
> systemd?  I've hacked around it for now by creating
> /etc/systemd/system/myservice.service.d/realtime.conf that moves the
> service back to the root cgroup and then uses chrt to set the scheduling
> policy:
> 
>   [Service]
>   ExecStartPost=/bin/cgclassify -g cpu:/ $MAINPID
>   ExecStartPost=/bin/chrt -r -p 99 $MAINPID
> 
> ...and while that works, it seems really ugly.  I've attempted to set
> CPUSchedulingPolicy=rr in the unit, but that simply results in systemd
> failing to start the service and logging "Failed at step SETSCHEDULER
> spawning...".
> 
> Is there a better way of addressing this?

Hmm, by default, systemd should not be adding anything to the "cpu"
hierarchy, unless at least one service sets CPUShare=, CPUAccounting= or
related, or system-wide DefaultCPUAccounting= is set. There's
currently no nice tool unfortunately to track down why a cgroup was
created though...

Generally, RT group scheduling is not usable unless you explicitly
assign an RT budget to each cgroup that wants to have RT, and you
manually make sure you never hand out more RT budget than
possible. Because that's really nasty and no good defaults can be
picked for this mode we don't support it.

If you ignore this and try to make it work locally YMMV. What you
could do is drop in ExecStartPre= lines into the relevant services
that echo an RT budget into the relevant cgroup files in the "cpu"
hierarchy, possibly propagating these to the parent cgroups. To figure
out the right cgroup path to echo this into you'd have to query
/proc/self/cgroup...

Yeah, it's nasty, but at the moment a more automatic, and friendlier
exposure of the RT budget logic is not planned, as the kernel APIs are
just impossible to use with automatic management...

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel