Re: Guidance on solving the username namespacing problem

[Sean Whitton]

Hello,

On Sun 05 Jan 2020 at 11:33PM +01, Philipp Kern wrote:

> I'd also propose the following hunk as I was myself confused where this
> list was maintained - base-passwd is mentioned in 0-99 but not
> explicitly in the on demand part. As policy seems to defer to that
> package as the list, it would seem like policy should state this explicitly.
>
>> @@ -268,8 +271,10 @@ The UID and GID numbers are divided into classes as 
>> follows:
>>
>>  6-64999:
>>  Globally allocated by the Debian project, but only created on
>> -demand. The ids are allocated centrally and statically, but the
>> -actual accounts are only created on users' systems on demand.
>> +demand. The ids are allocated centrally, but the actual accounts are
>> +only created on users' systems on demand. Some of them are statically
>> +allocated. The authoritative allocation for this range is maintained
>> +in the ``base-passwd`` package.
>>
>>  These ids are for packages which are obscure or which require many
>>  statically-allocated ids. These packages should check for and create

I think it would be good to say what you mean by 'statically allocated'.
This could be done by combining your last two sentences to say that the
UIDs are statically allocated by means of the base-passwd package.

This is purely informative, not normative, so it doesn't need seconding.

> Now there's the question if we need explicit guidance in the UID bit
> about existing packages as well. How would the following sound instead
> of my prior proposal?
>
>> @@ -259,7 +262,9 @@ The UID and GID numbers are divided into classes as 
>> follows:
>>  and differently on each system, should use ``adduser --system`` to
>>  create the group and/or user. ``adduser`` will check for the
>>  existence of the user or group, and if necessary choose an unused id
>> -based on the ranges specified in ``adduser.conf``.
>> +based on the ranges specified in ``adduser.conf``. New packages
>> +should follow the guidance of using an underscore prefix for their
>> +username.
>>
>>  1000-5:
>>  Dynamically allocated user accounts. By default ``adduser`` will

I believe this should be a bit broader -- packages which are not new but
which are adding new users should also follow the underscore prefix
convention.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Re: Guidance on solving the username namespacing problem

[Lennart Poettering]

On Di, 14.01.20 15:55, Michael Biebl (bi...@debian.org) wrote:

> Lennart, Zbyszek,
>
> what's your take on this?
>
> For some more background, see
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=905817
> and the recent discussion at
> https://lists.debian.org/debian-policy/2020/01/msg00013.html

I glanced over these discussion only superficially. A few notes:

1. We document our numeric UID/GID expectations here:
   https://systemd.io/UIDS-GIDS which you already found.

2. I'd strongly advise you to stick to 16bit UIDs allocations for
   Debian's base OS (i. 0…65534). In userns container environments you
   can assume to get at least 16bit but only in some envs you'll get
   more. If you want to be compatible as payload with such envs
   without trouble you can't really use more hence.

3. Regarding naming: in systemd our own users are nowadays all
   prefixed with "systemd-". Besides that there are a couple of system
   users/groups beyond that defined ("audio", "video", and so on)
   which have not changed much in the past. They are not set in stone,
   but they certainly don't gain new names any time soon. The only
   addition we made in recent years was "render" as a group. I think
   primarily we should leave non-prefixed user/group names to the
   users of the systems, so I sympathize with the "prefix with
   underscore" idea for system users, but also: it's a bit too late
   for that I think, and not sure it's worth the trouble changing
   things towards that now (that said, in systemd-homed I opted to
   refuse allocating regular users with names starting with "_", to
   underline my sympathies — of course we also refuse regular users
   with names starting with "systemd-"). BTW, we changed Fedora (and
   ultimately RHEL) to now use the name "nobody" for the 65534 user,
   adjusting it more to match Debian. We don't use the same group name
   though (we generally follow the rule that user and group names
   match, and thus our group name for "nobody" is also "nobody". I
   think it would be great if Debian would agree ;-))

4. Regarding DynamicUser= UID allocation: it's mostly OK if on some
   installations people allocated from the same range, as long as they
   didn't take the whole range. systemd always checks with NSS before
   using a UID from the range, to avoid conflicts. It does not assume
   exclusive ownership of the UID range assigned for the purpose, but
   of course it good in the long run to keep the ranges isolated to
   max out their use for each purpose.

5. I fear using only DynamicUser= for all new services is not going to
   work, even though I wished it worked. As soon as daemons need to
   write data persisently that shall be available even when they
   aren't running the short-lived UIDs are a problem. It works
   perfectly for daemons that do not leave state however.

6. In general I'd recommend to use the same UID ranges if at all
   possible, i.e. to avoid needless incompatibilites as much as
   possible. I mean, there might be reason to change the range but it
   better be a really really really good reason. i.e. a few
   statically, historically allocated UIDs from that range are not
   enough a reason I think. But of course you have to figure that out
   yourself.

7. When I picked the default range for the DynamicUser= stuff I
   actually had a look at Debian's range allocation and opted for a
   range that was the least scary looking and also worked on other
   distros somewhat reasonably, in particular to allow distros to just
   share the UID range if at all possible. I wish picking the range
   like that was not an excercise in vain? :-(

8. Fedora doesn't do static UID allocations anymore for static system
   users. Maybe Debian can decide the same and thus allow
   qmail/netplan/… or what is currently allocating from that range to
   just pick a differnt UID from any range? that would mean
   DynamicUser= and qmail/netplan/… to coexist pretty nicely since
   both would implement a similar algorithm of "try & fall
   back". Static UID allocations in the year 2020 are a bit backwards
   anyway, isn't it?

9. If you decide to use a different range for dynamic users than our
   upstream defaults, then of course it would be great to make it at
   least a subset instead of picking a non-overlapping range.

10. The smaller the UID range for dynamic users the more likely the
chance of uid collisions during the allocation of a UID for a
username. systemd starts with a UID hashed from the desired user
name, but if that UID is already taken (which is likely given the
~4000 UIDs available is a small space) we pick random ones, but
give up eventually if we keep hitting already used UIDs. I picked
a high value of iterations before giving up (100 iterations before
we give up), because the range is so small. If you halve it again,
collisions are of course even more likely. A *lot* more likely
(birthday paradox…). The question you have to 

Re: Guidance on solving the username namespacing problem

[Michael Biebl]

Lennart, Zbyszek,

what's your take on this?

For some more background, see
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=905817
and the recent discussion at
https://lists.debian.org/debian-policy/2020/01/msg00013.html


Thanks,
Michael

Am 14.01.20 um 11:13 schrieb Philipp Kern:
> On 2020-01-05 23:33, Philipp Kern wrote:
>> And then the following (in spirit) to base-passwd to make the systemd
>> allocation explicit:
>>
>>> --- a/README
>>> +++ b/README
>>> @@ -32,6 +32,9 @@ registry of allocations.
>>>  Reserved uids:
>>>  uid   | name  | description
>>>  --+---+---
>>> +    61184 |   | reserved for systemd dynamic users
>>> +  -   |   |
>>> +    63433 |   |
>>>  63434 | netplan   | netplan
>>>  64000 | ftn   | fidogate
>>>  64001 | mysql | mysql-server
>>
>> I'd still like to hear from the systemd maintainers about their opinion
>> about the UID space shift and slight reduction, of course.
> 
> So it looks like this is effectively groundhog day for them as Michael
> pointed me to [1] where the same thing was discussed before.
> 
> Given the DynamicUser design[2] I'd still assume that where it is in the
> UID space effectively does not matter much, it's fungible. There will be
> effectively no files permanently owned by those UIDs because the
> filesystem locations where the services can write are restricted and
> tightly managed.
> 
> So dear systemd maintainers, how would you think about changing the UID
> space to the above? 2249 UIDs vs. 4335 UIDs means that the space is
> effectively halved, which might be concerning. It is unfortunate that
> this cannot be changed at runtime, but if we get bug reports about this
> I feel like it should be possible to make it take multiple ranges
> instead. Apart from where the space needs to be located it does not seem
> like there are strong reasons to prefer systemd's current range over any
> other. I don't know what happens if that range is changed across a
> package upgrade, though. Presumably the hashes would be different so
> actually making the change might be tricky.
> 
> Kind regards and thanks
> Philipp Kern
> 
> [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=905817
> [2] http://0pointer.net/blog/dynamic-users-with-systemd.html




signature.asc
Description: OpenPGP digital signature

Re: Guidance on solving the username namespacing problem

[Philipp Kern]

On 2020-01-05 23:33, Philipp Kern wrote:

And then the following (in spirit) to base-passwd to make the systemd
allocation explicit:


--- a/README
+++ b/README
@@ -32,6 +32,9 @@ registry of allocations.
 Reserved uids:
 uid   | name  | description
 --+---+---
+61184 |   | reserved for systemd dynamic users
+  -   |   |
+63433 |   |
 63434 | netplan   | netplan
 64000 | ftn   | fidogate
 64001 | mysql | mysql-server


I'd still like to hear from the systemd maintainers about their opinion
about the UID space shift and slight reduction, of course.


So it looks like this is effectively groundhog day for them as Michael 
pointed me to [1] where the same thing was discussed before.


Given the DynamicUser design[2] I'd still assume that where it is in the 
UID space effectively does not matter much, it's fungible. There will be 
effectively no files permanently owned by those UIDs because the 
filesystem locations where the services can write are restricted and 
tightly managed.


So dear systemd maintainers, how would you think about changing the UID 
space to the above? 2249 UIDs vs. 4335 UIDs means that the space is 
effectively halved, which might be concerning. It is unfortunate that 
this cannot be changed at runtime, but if we get bug reports about this 
I feel like it should be possible to make it take multiple ranges 
instead. Apart from where the space needs to be located it does not seem 
like there are strong reasons to prefer systemd's current range over any 
other. I don't know what happens if that range is changed across a 
package upgrade, though. Presumably the hashes would be different so 
actually making the change might be tricky.


Kind regards and thanks
Philipp Kern

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=905817
[2] http://0pointer.net/blog/dynamic-users-with-systemd.html

Re: Guidance on solving the username namespacing problem

[Philipp Kern]

Thanks again, Russ, Colin and Simon!

On 1/5/2020 7:25 PM, Russ Allbery wrote:
> 9.2.1 feels like the right spot to me.  I think that's close to 9.2.2.  We
> could also reiterate that guidance in 9.2.2.
> 
>>> --- a/policy/ch-opersys.rst
>>> +++ b/policy/ch-opersys.rst
>>> @@ -228,13 +228,16 @@ purpose for which they are allocated. This is a 
>>> serious restriction, and
>>>  we should avoid getting in the way of local administration policies. In
>>>  particular, many sites allocate users and/or local system groups
>>>  starting at 100.
>>>
>>>  Apart from this we should have dynamically allocated ids, which should
>>>  by default be arranged in some sensible order, but the behavior should
>>> -be configurable.
>>> +be configurable. When maintainers choose a new hardcoded or dynamically
>>> +generated username for packages to use, they should start this username
>>> +with an underscore. This minimizes collisions with locally created user
>>> +accounts.
>>>
>>>  Packages other than ``base-passwd`` must not modify ``/etc/passwd``,
>>>  ``/etc/shadow``, ``/etc/group`` or ``/etc/gshadow``.
>>>
>>>  .. _s9.2.2:
> 
> This part looks good to me.
> 
>>> @@ -256,13 +259,14 @@ The UID and GID numbers are divided into classes as 
>>> follows:
>>>  100-999:
>>>  Dynamically allocated system users and groups. Packages which need a
>>>  user or group, but can have this user or group allocated dynamically
>>>  and differently on each system, should use ``adduser --system`` to
>>>  create the group and/or user. ``adduser`` will check for the
>>>  existence of the user or group, and if necessary choose an unused id
>>> -based on the ranges specified in ``adduser.conf``.
>>> +based on the ranges specified in ``adduser.conf``. Usernames in this
>>> +range should be prefixed with an underscore.
> 
> I think this is too strong, since it implies that all packages that
> already create users should change, and I don't think we've thought
> through the implications of that yet.

Agreed. Re-reading it, it seems much too strong. Although I'm unsure how
to phrase it differently (see below). So the following has so far been
uncontroversial as a patch to policy:

> --- a/policy/ch-opersys.rst
> +++ b/policy/ch-opersys.rst
> @@ -231,7 +231,10 @@ starting at 100.
> 
>  Apart from this we should have dynamically allocated ids, which should
>  by default be arranged in some sensible order, but the behavior should
> -be configurable.
> +be configurable. When maintainers choose a new hardcoded or dynamically
> +generated username for packages to use, they should start this username
> +with an underscore. This minimizes collisions with locally created user
> +accounts.
> 
>  Packages other than ``base-passwd`` must not modify ``/etc/passwd``,
>  ``/etc/shadow``, ``/etc/group`` or ``/etc/gshadow``.

I'd also propose the following hunk as I was myself confused where this
list was maintained - base-passwd is mentioned in 0-99 but not
explicitly in the on demand part. As policy seems to defer to that
package as the list, it would seem like policy should state this explicitly.

> @@ -268,8 +271,10 @@ The UID and GID numbers are divided into classes as 
> follows:
> 
>  6-64999:
>  Globally allocated by the Debian project, but only created on
> -demand. The ids are allocated centrally and statically, but the
> -actual accounts are only created on users' systems on demand.
> +demand. The ids are allocated centrally, but the actual accounts are
> +only created on users' systems on demand. Some of them are statically
> +allocated. The authoritative allocation for this range is maintained
> +in the ``base-passwd`` package.
> 
>  These ids are for packages which are obscure or which require many
>  statically-allocated ids. These packages should check for and create

Now there's the question if we need explicit guidance in the UID bit
about existing packages as well. How would the following sound instead
of my prior proposal?

> @@ -259,7 +262,9 @@ The UID and GID numbers are divided into classes as 
> follows:
>  and differently on each system, should use ``adduser --system`` to
>  create the group and/or user. ``adduser`` will check for the
>  existence of the user or group, and if necessary choose an unused id
> -based on the ranges specified in ``adduser.conf``.
> +based on the ranges specified in ``adduser.conf``. New packages
> +should follow the guidance of using an underscore prefix for their
> +username.
> 
>  1000-5:
>  Dynamically allocated user accounts. By default ``adduser`` will

And then the following (in spirit) to base-passwd to make the systemd
allocation explicit:

> --- a/README
> +++ b/README
> @@ -32,6 +32,9 @@ registry of allocations.
>  Reserved uids:
>  uid   | name  | description
>  --+---+---
> +61184 |   | reserved for systemd dynamic users
> +   

Re: Guidance on solving the username namespacing problem

[Russ Allbery]

Colin Watson  writes:

> As Simon said, EF00-FFEF = 61184-65519 covers more than just netplan
> (https://salsa.debian.org/debian/base-passwd/blob/master/README), and
> several of the IDs allocated there in the vaguely recent past are hard
> to change (their rationales included "needs to be the same across
> multiple machines"), so I don't think we can use the existing systemd
> range - it needs to be adjusted for Debian at least to some extent.  I'm
> not prepared to cede all of 64000-64999 to systemd; perhaps it would
> have been better if base-passwd had started at 6 instead, but we're
> here now.

Oh, whoops, I misread and didn't double-check.  Yes, we definitely don't
want to stomp on the 64000 range that we've been using for forever.
Apologies for the confusion.

> The rate of static allocations in 6-64999 is low enough that I'm not
> concerned in principle about carving off a slice of it for dynamic
> allocations by systemd-sysusers, and in any case I wasn't expecting to
> ever need to allocate more static IDs under 64000 (netplan was before my
> time).  Perhaps we could start by reserving 61184-63433, given the
> netplan allocation?  Yes, it's a bit arbitrary, but also not really all
> that stingy, and base-passwd's allocations are meant to be permanent
> even if the package has been removed (since we can never guarantee that
> it's been removed from users' systems).

This sounds good to me.

-- 
Russ Allbery (r...@debian.org)  

Re: Guidance on solving the username namespacing problem

[Colin Watson]

[I haven't been following the rest of this discussion.  Thanks for the
CC - let me know if I'm egregiously missing anything.]

On Sun, Jan 05, 2020 at 10:25:37AM -0800, Russ Allbery wrote:
> Philipp Kern  writes:
> > It looks like the range must be contiguous, as it is compiled in[1].
> > What are the preexisting ones apart from netplan that you have in mind?
> > It feels like systemd's boundaries are pretty arbitrary (0xEF00 to
> > 0xFFEF) but at the same time we might want to reserve a range for it in
> > policy - given that right now it is effectively squatting in that range?
> 
> Yes.  We should also coordinate this with Colin as the base-passwd
> maintainer.  Let me cc him explicitly.
> 
> It's possible that we can just use the existing systemd range, provided
> that we can find some workable approach for netplan.

As Simon said, EF00-FFEF = 61184-65519 covers more than just netplan
(https://salsa.debian.org/debian/base-passwd/blob/master/README), and
several of the IDs allocated there in the vaguely recent past are hard
to change (their rationales included "needs to be the same across
multiple machines"), so I don't think we can use the existing systemd
range - it needs to be adjusted for Debian at least to some extent.  I'm
not prepared to cede all of 64000-64999 to systemd; perhaps it would
have been better if base-passwd had started at 6 instead, but we're
here now.

The rate of static allocations in 6-64999 is low enough that I'm not
concerned in principle about carving off a slice of it for dynamic
allocations by systemd-sysusers, and in any case I wasn't expecting to
ever need to allocate more static IDs under 64000 (netplan was before my
time).  Perhaps we could start by reserving 61184-63433, given the
netplan allocation?  Yes, it's a bit arbitrary, but also not really all
that stingy, and base-passwd's allocations are meant to be permanent
even if the package has been removed (since we can never guarantee that
it's been removed from users' systems).

An alternative would be to reserve 61184-63999, with a Debian patch to
exclude netplan's 63434.  That doesn't seem likely to be difficult; it
could go in the same place where systemd is already doing NSS checks.


I'm generally in favour of the underscore prefix recommendation in some
form, and would be happy to enforce that for new static allocations in
base-passwd.

-- 
Colin Watson   [cjwat...@debian.org]

Re: Guidance on solving the username namespacing problem

[Russ Allbery]

Philipp Kern  writes:

> I fear that we might need a local policy hook for migrations. If we end
> up renaming users that are actively referenced elsewhere, there might be
> cleanup tasks that need to be performed in lockstep.

> At the same time I'd strongly suggest that we do not go the way of
> distinguishing between the old user already being present on a machine
> and a new package install. That's a divergence that becomes actively
> harmful when you try to manage a set of machines.

Right, I think we should start by saying that all packages that already
create a user can keep using the user they are currently using for now,
and stop the bleeding by setting a requirement on newly-created users
going forward.

Then we can separately figure out what to do about the transition, if
anything.

> As I see it it isn't even guaranteed that services will have a dash,
> which is quite unfortunate. That would already make them way less likely
> to collide. Presumably we would not want to draw a distinction between
> "contains a dash and thus does not need an underscore" and "needs an
> underscore prefix to disambiguate", even though that would feel like an
> easier sell as it would be a behavior change for likely significantly
> less services. :/

I think starting everything with an underscore is best.  Talking systemd
upstream into using the underscore prefix seems like the best of all
worlds to me, if they'll go for it.

> Patching for Debian-specific behavior seems to be against the spirit of
> cross-distro reusability. So that question should be posed to systemd
> upstream and then we should decide based on the response?

That sounds right to me.

> It looks like the range must be contiguous, as it is compiled in[1].
> What are the preexisting ones apart from netplan that you have in mind?
> It feels like systemd's boundaries are pretty arbitrary (0xEF00 to
> 0xFFEF) but at the same time we might want to reserve a range for it in
> policy - given that right now it is effectively squatting in that range?

Yes.  We should also coordinate this with Colin as the base-passwd
maintainer.  Let me cc him explicitly.

It's possible that we can just use the existing systemd range, provided
that we can find some workable approach for netplan.

>> This should probably go somewhere near §9.2.2 "UID and GID classes":
>> it applies to future allocations in the 100-999 and 6-64999 ranges,
>> and maybe to future allocations in the 0-99 range (although I don't
>> think we should be migrating existing low-uid hard-coded names like
>> games and proxy to _games and _proxy).

> That sounds fine to me too, although I wonder about placement. You are
> right that it's technically about the system-assigned ranges. At the
> same time it's also somewhat of a principle that might go into §9.2.1,
> as that already talks about the importance of UID allocation wrt local
> administration policies.

9.2.1 feels like the right spot to me.  I think that's close to 9.2.2.  We
could also reiterate that guidance in 9.2.2.

>> --- a/policy/ch-opersys.rst
>> +++ b/policy/ch-opersys.rst
>> @@ -228,13 +228,16 @@ purpose for which they are allocated. This is a 
>> serious restriction, and
>>  we should avoid getting in the way of local administration policies. In
>>  particular, many sites allocate users and/or local system groups
>>  starting at 100.
>> 
>>  Apart from this we should have dynamically allocated ids, which should
>>  by default be arranged in some sensible order, but the behavior should
>> -be configurable.
>> +be configurable. When maintainers choose a new hardcoded or dynamically
>> +generated username for packages to use, they should start this username
>> +with an underscore. This minimizes collisions with locally created user
>> +accounts.
>> 
>>  Packages other than ``base-passwd`` must not modify ``/etc/passwd``,
>>  ``/etc/shadow``, ``/etc/group`` or ``/etc/gshadow``.
>> 
>>  .. _s9.2.2:

This part looks good to me.

>> @@ -256,13 +259,14 @@ The UID and GID numbers are divided into classes as 
>> follows:
>>  100-999:
>>  Dynamically allocated system users and groups. Packages which need a
>>  user or group, but can have this user or group allocated dynamically
>>  and differently on each system, should use ``adduser --system`` to
>>  create the group and/or user. ``adduser`` will check for the
>>  existence of the user or group, and if necessary choose an unused id
>> -based on the ranges specified in ``adduser.conf``.
>> +based on the ranges specified in ``adduser.conf``. Usernames in this
>> +range should be prefixed with an underscore.

I think this is too strong, since it implies that all packages that
already create users should change, and I don't think we've thought
through the implications of that yet.

-- 
Russ Allbery (r...@debian.org)  

Re: Guidance on solving the username namespacing problem

[Simon McVittie]

On Sun, 05 Jan 2020 at 17:16:58 +0100, Philipp Kern wrote:
> On 1/4/2020 5:08 PM, Simon McVittie wrote:
> > It's also worth noting that the 61184-65519 uid range used for DynamicUser
> > by default collides with the rarely-used 6-64999 uid range for system
> > users that are "globally allocated by the Debian project, but only created
> > on demand".
> > 
> > The uids in range 6-64999 that are really allocated in practice are
> > in the sub-range 64000-64062, except for 63434 'netplan' which appears
> > to be for an orphaned package that isn't in testing; so it might be
> > feasible to reconfigure the DynamicUser range to 6-63999 or similar
> > in Debian's systemd?
> 
> It looks like the range must be contiguous, as it is compiled in[1].

Yes.

> What are the preexisting ones apart from netplan that you have in mind?

https://sources.debian.org/src/base-passwd/3.5.47/README/#L28

As you can see, apart from netplan (63434, which I suspect may have been
imported from a previous ad-hoc allocation) in practice they are all in
the 640xx range. Most of the newer allocations seem to be cluster/cloud
services where having the same uid across a whole cluster is
desirable: the vast majority of system services are fine with the
dynamically-allocated 100-999 range.

> It feels like systemd's boundaries are pretty arbitrary (0xEF00 to
> 0xFFEF) but at the same time we might want to reserve a range for it in
> policy - given that right now it is effectively squatting in that range?

Yes, I think so.

The exact range used seems to be arbitrary, and having a slightly smaller
range in Debian would probably be OK, as long as it isn't orders of
magnitude smaller?

> The main argument presented by upstream is that it needs to be within
> 64bit for user namespacing to work properly.

I assume you meant within the 16-bit range (first 64k uids), but yes. The
big idea is that container number 0x1234 can have uids ranging from
0x1234 (root in the container) to 0x1234fffe ('nobody' in the
container, where 'nobody' is 65534 = 0xfffe), which avoids the problem
seen in e.g. Docker where uid 1000 in container A can sometimes
unintentionally get privileges over processes owned by uid 1000 in
container B, because they are both uid 1000 on the host system.

Managing uid and gid allocations for containers seems likely to be
much, much easier if each container gets a linear block of 65535 uids
within which it can use the entire legacy 16-bit uid space, rather than
having a discontinuous uid range so that some uids are unavailable to
the container.

smcv

Re: Guidance on solving the username namespacing problem

[Philipp Kern]

Hey,

thanks, Sam, Simon and Russ! That was all very helpful! Much appreciated!

[Adding the systemd maintainers to the Cc for Simon's question below.]

On 1/4/2020 5:08 PM, Simon McVittie wrote:
> On Sat, 04 Jan 2020 at 13:52:51 +0100, Philipp Kern wrote:
>> now that we are talking again about standardizing user creation using
>> sysusers, I wonder if you could give me any guidance on how to attack
>> the Debian system user namespacing problem.
> 
> It's a good reminder, but I think the naming convention for system users
> is mostly orthogonal to whether they're created by adduser, useradd,
> systemd-sysusers or something else. (But see below for DynamicUser.)

Correct. I think it mostly serves as a point of transition - when you
have to touch it anyhow, you might as well be able to rename the user.
(Although in many cases it is probably rather tricky.)

>> OpenBSD rather successfully standardized on the underscore prefix to
>> eliminate this conflict altogether. I would like that we recommend the
>> same thing.
> 
> I agree that's a good convention for new system users (better than
> debian- or Debian-, and a lot better than having no particular prefix),
> particularly now that apt creates _apt on basically every Debian-derived
> system.
> 
>> The main question that has been raised was how to manage the migration.
>> I think the priority should be on stopping the bleeding and new users
>> should follow a consistent scheme
> 
> I agree, and I don't think the absence of a migration path for existing
> system users like messagebus and systemd-coredump should prevent us
> from encouraging the OpenBSD underscore thing for new system users
> like _apt and _flatpak.
> 
> I maintain several game servers that use undesirable usernames like
> Debian-openarena, which might make good test subjects for automatic
> migration to names like _openarena in relatively unimportant packages
> (we shouldn't be using more important packages like dbus as our test
> subjects). After successfully prototyping it in a couple of packages,
> the right place for it would probably be debhelper or init-system-helpers.
> 
> The thing to do during migration would probably be to add a second user
> with the same uid and other details, but a different username (like
> useradd --non-unique); then optionally remove the original user record,
> leaving only the new name.

I fear that we might need a local policy hook for migrations. If we end
up renaming users that are actively referenced elsewhere, there might be
cleanup tasks that need to be performed in lockstep.

At the same time I'd strongly suggest that we do not go the way of
distinguishing between the old user already being present on a machine
and a new package install. That's a divergence that becomes actively
harmful when you try to manage a set of machines.

> On Sat, 04 Jan 2020 at 13:55:40 +0100, Philipp Kern wrote:
>> A more bold move would be to tell daemon packagers to use DynamicUser
>> where feasible and only allocate more permanent users if there's a need
>> for it.
> 
> I think this is *almost* orthogonal to how those users are named.
> 
> The only way in which DynamicUser affects the naming policy is that if
> foo-bar.service uses DynamicUser=yes and does not specify a User, the
> default is foo-bar. /lib/systemd/system/fwupd-refresh.service is currently
> the only example on my laptop.
> 
> For units with a long, namespaced name like fwupd-refresh.service,
> flatpak-system-helper.service and quake2-server.serice, this is probably
> good enough in practice, even though in principle these names might collide.
> 
> Possible routes:
> 
> - leave these units with behaviour that does not follow the recommendation,
>   reasoning that if they have fairly long names it won't actually be a
>   practical problem
> 
> - recommend that units with DynamicUser=yes should specify a User and Group
>   that fit the naming convention, for example DynamicUser=yes,
>   User=_fwupd-refresh
> 
> - ask the systemd Debian maintainers to patch systemd so that it defaults
>   to the equivalent of User=_fwupd-refresh instead (note that this
>   behaviour change is in principle an API break which could conceivably
>   make services that work in other distributions fail to work in Debian,
>   although probably it would work fine in practice)
> 
> - ask systemd upstream to make that change (they could probably be persuaded
>   that the OpenBSD underscore thing is a good convention; same notes about
>   this being a behaviour change)
> 
> - avoid DynamicUser=yes

As I see it it isn't even guaranteed that services will have a dash,
which is quite unfortunate. That would already make them way less likely
to collide. Presumably we would not want to draw a distinction between
"contains a dash and thus does not need an underscore" and "needs an
underscore prefix to disambiguate", even though that would feel like an
easier sell as it would be a behavior change for likely significantly
less services. :/

Re: Guidance on solving the username namespacing problem

[Russ Allbery]

Philipp Kern  writes:

> OpenBSD rather successfully standardized on the underscore prefix to
> eliminate this conflict altogether. I would like that we recommend the
> same thing.

I agree.

> The main question that has been raised was how to manage the migration.

I agree with this too.  I'm happy to have Policy standardize this
convention ASAP for newly-created users and then think at more leisure
about whether (and if so, how) to migrate existing users.

> I think the priority should be on stopping the bleeding and new users
> should follow a consistent scheme, but I understand how without a
> migration plan we just end up with "one more scheme" (even if it might
> be the most popular now except using none at all[1]).

In this particular case, I don't think standardizing one of the many
schemes in use would cause problems over the current situation even if we
don't go back and make everything consistent.

> I tried to raise this issue in [2] a year ago, but I think I don't know
> how to even start drafting a policy snippet about this. Would it be
> sufficient to just mandate "In order to avoid collisions with accounts
> created by the system administrator, usernames created by packages
> should start with an underscore." (assuming we could get a rough
> consensus for something like that) in 9.2.1 for now?

Yes.  I think we should say something about how packages that started
creating users before this recommendation was added don't need to change
the name of that user (until we figure out a migration strategy).

-- 
Russ Allbery (r...@debian.org)  

Re: Guidance on solving the username namespacing problem

[Simon McVittie]

On Sat, 04 Jan 2020 at 13:52:51 +0100, Philipp Kern wrote:
> now that we are talking again about standardizing user creation using
> sysusers, I wonder if you could give me any guidance on how to attack
> the Debian system user namespacing problem.

It's a good reminder, but I think the naming convention for system users
is mostly orthogonal to whether they're created by adduser, useradd,
systemd-sysusers or something else. (But see below for DynamicUser.)

> OpenBSD rather successfully standardized on the underscore prefix to
> eliminate this conflict altogether. I would like that we recommend the
> same thing.

I agree that's a good convention for new system users (better than
debian- or Debian-, and a lot better than having no particular prefix),
particularly now that apt creates _apt on basically every Debian-derived
system.

> The main question that has been raised was how to manage the migration.
> I think the priority should be on stopping the bleeding and new users
> should follow a consistent scheme

I agree, and I don't think the absence of a migration path for existing
system users like messagebus and systemd-coredump should prevent us
from encouraging the OpenBSD underscore thing for new system users
like _apt and _flatpak.

I maintain several game servers that use undesirable usernames like
Debian-openarena, which might make good test subjects for automatic
migration to names like _openarena in relatively unimportant packages
(we shouldn't be using more important packages like dbus as our test
subjects). After successfully prototyping it in a couple of packages,
the right place for it would probably be debhelper or init-system-helpers.

The thing to do during migration would probably be to add a second user
with the same uid and other details, but a different username (like
useradd --non-unique); then optionally remove the original user record,
leaving only the new name.

On Sat, 04 Jan 2020 at 13:55:40 +0100, Philipp Kern wrote:
> A more bold move would be to tell daemon packagers to use DynamicUser
> where feasible and only allocate more permanent users if there's a need
> for it.

I think this is *almost* orthogonal to how those users are named.

The only way in which DynamicUser affects the naming policy is that if
foo-bar.service uses DynamicUser=yes and does not specify a User, the
default is foo-bar. /lib/systemd/system/fwupd-refresh.service is currently
the only example on my laptop.

For units with a long, namespaced name like fwupd-refresh.service,
flatpak-system-helper.service and quake2-server.serice, this is probably
good enough in practice, even though in principle these names might collide.

Possible routes:

- leave these units with behaviour that does not follow the recommendation,
  reasoning that if they have fairly long names it won't actually be a
  practical problem

- recommend that units with DynamicUser=yes should specify a User and Group
  that fit the naming convention, for example DynamicUser=yes,
  User=_fwupd-refresh

- ask the systemd Debian maintainers to patch systemd so that it defaults
  to the equivalent of User=_fwupd-refresh instead (note that this
  behaviour change is in principle an API break which could conceivably
  make services that work in other distributions fail to work in Debian,
  although probably it would work fine in practice)

- ask systemd upstream to make that change (they could probably be persuaded
  that the OpenBSD underscore thing is a good convention; same notes about
  this being a behaviour change)

- avoid DynamicUser=yes

It's also worth noting that the 61184-65519 uid range used for DynamicUser
by default collides with the rarely-used 6-64999 uid range for system
users that are "globally allocated by the Debian project, but only created
on demand". systemd avoids uids that are already in use when allocating
a DynamicUser, so the only time this is a practical problem is if systemd
has already used a uid for DynamicUser, and then the package to which that
uid is allocated in base-passwd gets installed before the next reboot.

The uids in range 6-64999 that are really allocated in practice are
in the sub-range 64000-64062, except for 63434 'netplan' which appears
to be for an orphaned package that isn't in testing; so it might be
feasible to reconfigure the DynamicUser range to 6-63999 or similar
in Debian's systemd?

On Sat, 04 Jan 2020 at 09:00:37 -0500, Sam Hartman wrote:
> I think you could certainly do  usernames created by packages are
> encouraged to start with an underscore.
...
> You could also do something more complex like
>¯
> When maintainers choose a new hard-coded or dynamically generated username
> for packages to use, they should start this username with an underscore.

The latter sounds better to me, at least for now.

This should probably go somewhere near §9.2.2 "UID and GID classes":
it applies to future allocations in the 100-999 and 6-64999 ranges,
and maybe to future allocations in the 0-99 

Re: Guidance on solving the username namespacing problem

[Sam Hartman]

> "Philipp" == Philipp Kern  writes:

Philipp> I tried to raise this issue in [2] a year ago, but I think I don't 
know
Philipp> how to even start drafting a policy snippet about this. Would it be
Philipp> sufficient to just mandate "In order to avoid collisions with 
accounts
Philipp> created by the system administrator, usernames created by packages
Philipp> should start with an underscore." (assuming we could get a rough
Philipp> consensus for something like that) in 9.2.1 for now? Or is this
Philipp> effectively infeasible until we come up with a good migration 
story?

I think you could certainly do  usernames created by packages are
encouraged to start with an underscore.

Encouraged being the new normative word meaning that maintainers ought
to do x unless they have a reason (like an existing username) to do
something else.

You could also do something more complex like

When maintainers choose a new hard-coded or dynamically generated username
for packages to use, they should start this username with an underscore.

Intent there is to capture making it a bug to do something else for new
usernames without making requirements for existing names.

Note that in cases like debconf, I don't think we want an underscore
prepended to what the user chooses, although I think defaulting to
something with a leading underscore would be fine.

Re: Guidance on solving the username namespacing problem

[Philipp Kern]

(And then my broken keyboard driver caused this to be sent prematurely.
But alas, it's out now.)

On 1/4/2020 1:52 PM, Philipp Kern wrote:
> [Please cc me on replies as I am not currently subscribed to the list.]
> 
> now that we are talking again about standardizing user creation using
> sysusers, I wonder if you could give me any guidance on how to attack
> the Debian system user namespacing problem.
> 
> There are some well-known usernames like "root" that are a given for an
> organization to block. But there are many usernames dynamically created
> by applications. DynamicUser would solve part of the problem, but some
> services need to persist data and sometimes it is useful to reference a
> fixed identity even outside of a filesystem context (e.g. in iptables
> rules). At my organization we had collisions with regular usernames -
> e.g. a user legitimately called themselves "bind" because part of their
> name was "Bin". Debian does not maintain a complete list of such
> usernames and it is even hard to compute from the packages right now,
> given that the users are created from maintainer scripts and sometimes
> are even configured from Debconf (which is another arbitrary indirection).
> 
> OpenBSD rather successfully standardized on the underscore prefix to
> eliminate this conflict altogether. I would like that we recommend the
> same thing.
> 
> The main question that has been raised was how to manage the migration.
> I think the priority should be on stopping the bleeding and new users
> should follow a consistent scheme, but I understand how without a
> migration plan we just end up with "one more scheme" (even if it might
> be the most popular now except using none at all[1]).
> 
> I tried to raise this issue in [2] a year ago, but I think I don't know
> how to even start drafting a policy snippet about this. Would it be
> sufficient to just mandate "In order to avoid collisions with accounts
> created by the system administrator, usernames created by packages
> should start with an underscore." (assuming we could get a rough
> consensus for something like that) in 9.2.1 for now? Or is this
> effectively infeasible until we come up with a good migration story?

A more bold move would be to tell daemon packagers to use DynamicUser
where feasible and only allocate more permanent users if there's a need
for it.

In the end what I'm looking for is input from the policy editors on how
to possibly approach this. Especially as AIUI policy is supposed to
document current consensus rather than necessarily set the standards.

Kind regards
Philipp Kern

Guidance on solving the username namespacing problem

[Philipp Kern]

[Please cc me on replies as I am not currently subscribed to the list.]

Hi,

now that we are talking again about standardizing user creation using
sysusers, I wonder if you could give me any guidance on how to attack
the Debian system user namespacing problem.

There are some well-known usernames like "root" that are a given for an
organization to block. But there are many usernames dynamically created
by applications. DynamicUser would solve part of the problem, but some
services need to persist data and sometimes it is useful to reference a
fixed identity even outside of a filesystem context (e.g. in iptables
rules). At my organization we had collisions with regular usernames -
e.g. a user legitimately called themselves "bind" because part of their
name was "Bin". Debian does not maintain a complete list of such
usernames and it is even hard to compute from the packages right now,
given that the users are created from maintainer scripts and sometimes
are even configured from Debconf (which is another arbitrary indirection).

OpenBSD rather successfully standardized on the underscore prefix to
eliminate this conflict altogether. I would like that we recommend the
same thing.

The main question that has been raised was how to manage the migration.
I think the priority should be on stopping the bleeding and new users
should follow a consistent scheme, but I understand how without a
migration plan we just end up with "one more scheme" (even if it might
be the most popular now except using none at all[1]).

I tried to raise this issue in [2] a year ago, but I think I don't know
how to even start drafting a policy snippet about this. Would it be
sufficient to just mandate "In order to avoid collisions with accounts
created by the system administrator, usernames created by packages
should start with an underscore." (assuming we could get a rough
consensus for something like that) in 9.2.1 for now? Or is this
effectively infeasible until we come up with a good migration story?

Kind regards
Philipp Kern

[1] https://people.debian.org/~pkern/permanent/userlist.txt
[2] https://lists.debian.org/debian-devel/2019/02/msg00131.html and
following