Bug#848239: deluser on purge

[Dmitry Bogatov]

[2016-12-22 14:21] Antoine Beaupré 
>
> part   text/plain 981
> On 2016-12-21 22:56:06, Dmitry Bogatov wrote:
> > [2016-12-20 22:10] Russ Allbery 
> >> Hm, transient IDs is an interesting idea.  In a lot of cases, we create a
> >> system user just to isolate the running daemon, not to control file system
> >> access.  The drawback, though, is that one has to have a really clear idea
> >> of what resources the process would need in order to make sure this is
> >> safe.  (A much clearer idea than the understanding we need to know when
> >> it's safe to delete a system user, I think.)
> >
> > You just gave me good idea. What about not removing $HOME, but chowning
> > it to root? I mean, on install we create user and if its $HOME already
> > exists, just chown it.
>
> You would need to check for suid binaries, among other traps.

Good catch. Then chowning is no better then removing.

-- 
X-Web-Site: https://sinsekvu.github.io | Note that I process my email in batch,
Accept-Languages: eo,ru,en | at most once every 24 hours. If matter
Accept: text/plain, text/x-diff| is urgent, you have my phone number.


pgpTPosXCfeLQ.pgp
Description: PGP signature

Bug#848239: deluser on purge

[Antoine Beaupré]

On 2016-12-21 22:56:06, Dmitry Bogatov wrote:
> [2016-12-20 22:10] Russ Allbery 
>> Hm, transient IDs is an interesting idea.  In a lot of cases, we create a
>> system user just to isolate the running daemon, not to control file system
>> access.  The drawback, though, is that one has to have a really clear idea
>> of what resources the process would need in order to make sure this is
>> safe.  (A much clearer idea than the understanding we need to know when
>> it's safe to delete a system user, I think.)
>
> You just gave me good idea. What about not removing $HOME, but chowning
> it to root? I mean, on install we create user and if its $HOME already
> exists, just chown it.

You would need to check for suid binaries, among other traps.

a.
-- 
Arguing for surveillance because you have nothing to hide is no
different than making the claim, "I don't care about freedom of speech
because I have nothing to say."
- Edward Snowden

Bug#848239: deluser on purge

[Dmitry Bogatov]

[2016-12-20 22:10] Russ Allbery 
> Hm, transient IDs is an interesting idea.  In a lot of cases, we create a
> system user just to isolate the running daemon, not to control file system
> access.  The drawback, though, is that one has to have a really clear idea
> of what resources the process would need in order to make sure this is
> safe.  (A much clearer idea than the understanding we need to know when
> it's safe to delete a system user, I think.)

You just gave me good idea. What about not removing $HOME, but chowning
it to root? I mean, on install we create user and if its $HOME already
exists, just chown it. Unfortunately, it does not answer the question
what to do with files outside of $HOME.

> Using random high-numbered IDs, while appealing, probably isn't a great
> idea because we allow the local admin to use that space.  It's possible
> that they're doing something that's consuming millions of IDs for some
> reason, so although there's a lot of space there, we can't entirely rule
> out the possibility of a conflict.  Although we could probably carve out
> more space if we really needed to.

2 ^ 32 = 4 294 967 296. Probably we can reserve several millions
for such non-recycle usage.

So far my summary:

 - wasting system UIDs is bad. We have only 900 of them
 - so far we have no satisfactory solution to UID wasting
 - transient UIDs are no solution, but can reduce wasting
 - seems worth trying to implement transient UIDs and see, how
   well they are applicable.

-- 
X-Web-Site: https://sinsekvu.github.io | Note that I process my email in batch,
Accept-Languages: eo,ru,en | at most once every 24 hours. If matter
Accept: text/plain, text/x-diff| is urgent, you have my phone number.


pgpYVHxqoJ8io.pgp
Description: PGP signature

Bug#848239: deluser on purge

[Russ Allbery]

Dmitry Bogatov  writes:

> You are right. But what are we going to do anyway in case if user
> installs 901 different daemons? Seems that approach of system users does
> not scale at all.

Indeed.

That said, I don't think I've seen a system consume more than 50 users or
so.  In practice, there do seem to be enough users.  But that might not be
the case if we accumulate users forever and never recycle those system
UIDs.

> Just a thought: `setuid(2)' accepts uid_t, which is 32bit on my 64 bit
> system. So probably it possible to run process isolated without creating
> entry in /etc/passwd?

Hm, transient IDs is an interesting idea.  In a lot of cases, we create a
system user just to isolate the running daemon, not to control file system
access.  The drawback, though, is that one has to have a really clear idea
of what resources the process would need in order to make sure this is
safe.  (A much clearer idea than the understanding we need to know when
it's safe to delete a system user, I think.)

Using random high-numbered IDs, while appealing, probably isn't a great
idea because we allow the local admin to use that space.  It's possible
that they're doing something that's consuming millions of IDs for some
reason, so although there's a lot of space there, we can't entirely rule
out the possibility of a conflict.  Although we could probably carve out
more space if we really needed to.

-- 
Russ Allbery (r...@debian.org)   

Bug#848239: deluser on purge

[Dmitry Bogatov]

First of all, I am sorry, if my response sounded rough or impolite or
offensive or somehow demostrated negative attitude. It was not
intended. I beg your pardon.  English is not my native, and sometimes I
fail to notice differense between neutral wording and non-neutral. My
intention to convey purely technical idea.

Summary of out conversation, as I understood it:

  * You want system user and its $HOME to be removed at purge

  * You rationale it with piuparts and example of several other packages

  * I state that it would make dh-sysuser code complicated, and will
require handling unknown amount of corner cases. Failure to catch
one of them may lead to disaster,

  * I state that I do consider 'aganist' outweight 'for'

To make piuparts happy I can propose following algorithm on purge:

 * remove exactly those files in $HOME, that are same as in /etc/skel.
 * if $HOME now is empty, remove it and remove user.

[2016-12-19 08:53] Antoine Beaupré 
> I am not sure what "spirit of Debian" you are refering to. Does my
> proposal go against a specific policy item or the social contract? Is
> there a spiritual guide I have failed to read in my last decade of
> involvement?

Debian, as far as I observe, is fine with certain level of "overhead",
like linking programs with libraries like libsystemd or libaudit or
libdbus.  Maintainer gets simplicity of packaging, user gets more
dependencies and used memory. It is considered fine.

Back to our ocassion. You propose adding different features to
dh-sysuser. Some of them in my opinion has fair balance between
usefuness and complexity, like `chmod $HOME', I can't say so about
feature of removing $HOME and deleting user.

Here I am stating that I trade a little (IMHO) of user good for a lot of
my good, and that it is common practice.

> But I understand my argument is not well received so I will go back in
> my hole of mutual ignorance.

I am sorry to hear it. I hope I clarified my position well enough and I
hope that you will change your mind and we can return to technical
discussions.

PS. See next email about setsid() function.

PPS. When I use word 'state', I mean 'express my opinion', not 'state of
 world'. Arguments can change opinion.

--
X-Web-Site: https://sinsekvu.github.io | Note that I process my email in batch,
Accept-Languages: eo,ru,en | at most once every 24 hours. If matter
Accept: text/plain, text/x-diff| is urgent, you have my phone number.


pgpioEUYgl6Rn.pgp
Description: PGP signature

Bug#848239: deluser on purge

[Dmitry Bogatov]

[2016-12-19 09:04] Russ Allbery 
>
> part   text/plain 564
> Dmitry Bogatov  writes:
>
> > After all, leftover system user is just one extra line in /etc/passwd
> > and one more wasted UID, but we have ~32k of them.
>
> Er, no, we have 900 of them.  Maybe 1433 of them if we steal 65000-65533.
> Anywhere else will run into the UID space reserved for the local
> administrator, which would cause serious problems.
>
> Given that, I do think something like this would be a good idea if all the
> tricky implementation details can be worked out.

You are right. But what are we going to do anyway in case if user
installs 901 different daemons? Seems that approach of system users
does not scale at all.

Just a thought: `setuid(2)' accepts uid_t, which is 32bit on my 64 bit
system. So probably it possible to run process isolated without creating
entry in /etc/passwd? Here is little C program:

#include 
#include 
#include 

int
main(void)
{
printf("%d\n", (int)(sizeof(uid_t)));
setuid(10);
sleep(1000);
return 0;
}

Will it misbehave on different kernels or architectures, then x86_64 GNU/Linux?

If it is okay, then let's consider why system users are usually used? As
far as I understand, to run some process and

 - protect it from other processed
 - protect other processes from it
 - do not allow it to write files it do not need
 - do not allow it to read files it do not need

setuid(BIG_INT) is fine to solve first two tasks. About third we can do
following:

$ setuid $BIG_INT my-daemon 

Bug#848239: deluser on purge

[Russ Allbery]

Dmitry Bogatov  writes:

> After all, leftover system user is just one extra line in /etc/passwd
> and one more wasted UID, but we have ~32k of them.

Er, no, we have 900 of them.  Maybe 1433 of them if we steal 65000-65533.
Anywhere else will run into the UID space reserved for the local
administrator, which would cause serious problems.

Given that, I do think something like this would be a good idea if all the
tricky implementation details can be worked out.

-- 
Russ Allbery (r...@debian.org)   

Bug#848239: deluser on purge

[Antoine Beaupré]

On 2016-12-19 00:06:06, Dmitry Bogatov wrote:
> [2016-12-17 11:15] Antoine Beaupré 
>>
>> part   text/plain5082
>> Adding the base-passwd maintainers to the conversation. TL;DR: this is
>> about the new dh-sysuser wrapper to more easily create system users in
>> package maintainer scripts. I want to see how we can make piuparts happy
>> and cleanup properly after ourselves (by removing the user on purge) and
>> that may involve a registry of UIDs which seems to be maintained in
>> base-passwd now.
>
> Why go all this trouble to just purge system user? While I do appreciate
> this nit-picking spirit, it is aganist spirit of Debian.
>
> After all, leftover system user is just one extra line in /etc/passwd
> and one more wasted UID, but we have ~32k of them.
>
> Also, your proposals require significant complications in dh-sysuser
> code, which is aganinst my spirit. Practicality beats purity.

I find it rather unfortunate that you characterize my work as
"nitpicking". I thought i provided a well-reasoned rationale of why
managed user ids are useful beyond just "wasting UIDs" (which I haven't
made as an argument at all).

I am not sure what "spirit of Debian" you are refering to. Does my
proposal go against a specific policy item or the social contract? Is
there a spiritual guide I have failed to read in my last decade of
involvement?

But I understand my argument is not well received so I will go back in
my hole of mutual ignorance.

Have fun,

A.
-- 
La propriété est un piège: ce que nous croyons posséder nous possède.
- Alphonse Karr

Bug#848239: deluser on purge

[Dmitry Bogatov]

[2016-12-17 11:15] Antoine Beaupré 
>
> part   text/plain5082
> Adding the base-passwd maintainers to the conversation. TL;DR: this is
> about the new dh-sysuser wrapper to more easily create system users in
> package maintainer scripts. I want to see how we can make piuparts happy
> and cleanup properly after ourselves (by removing the user on purge) and
> that may involve a registry of UIDs which seems to be maintained in
> base-passwd now.

Why go all this trouble to just purge system user? While I do appreciate
this nit-picking spirit, it is aganist spirit of Debian.

After all, leftover system user is just one extra line in /etc/passwd
and one more wasted UID, but we have ~32k of them.

Also, your proposals require significant complications in dh-sysuser
code, which is aganinst my spirit. Practicality beats purity.

> On 2016-12-16 23:08:31, Dmitry Bogatov wrote:
> > control: tag -1 +moreinfo
> >
> > [2016-12-15 09:07] Antoine Beaupré 
> >>
> >> part   text/plain1199
> >> Package: dh-sysuser
> >> Version: 1.3
> >> Severity: wishlist
> >>
> >> I think that, under certain circumstances, users should be completely
> >> removed along with their $HOME directory when the package is purged.
> >
> > See #840469 for discussion, why deleting $HOME is troublesome.
>
> That would be a great reference to add in the documentation. Something
> like:
>
> .SH LIMITATIONS
>
> dh-sysuser does not remove the user on package purge, for security
> reasons. See bug #840469 for more details.
>
> > If you
> > can propose principled solution -- it would be great. In short:
> >
> >  - postrm has access only to essential tools
>
> I reviewed the thread and that *is* a tricky situation. How about prerm?
> Can we be wild and remove on purge there?
>
> >  - What if there is something valuable is in $HOME? What if sysadmin
> >made some obscure thing, link `ln -sf / $HOME'?
>
> In any case, don't follow symlinks.
>
> But the nastier attack scenario that was mentionned in #840469 is simply
> an admin mistakenly or mailiciously setting that user's home to /. The
> solution here could be to keep track of the *original* home directory so
> that changing the home directory shouldn't pose problems. I mentionned
> this in #848240 - having a way of knowing where the homedir will be
> created would be useful there.
>
> Shared UIDs are also a problem, but those can be checked before
> fairly easily.
>
> Could we simply make an exhaustive list of potential problems with
> rm -rf $HOME and address each one? Or is it an inherently impossible
> problem? What are admins supposed to do in this case? Review every file
> in $HOME before they rm -rf the directory? We should be able to automate
> whatever an admin is expected to do when the user is removed... and if
> we don't automate it, we should direct admins to do *something*...
>
> >  - What do to with files outside of of $HOME?
>
> Ignore, for now. One could seek and destroy all those files, but that
> would be too time-consuming. Disk quotas may be (ab)used to figure out
> if there are remaining files, when enabled, but that's another
> dependency we probably can't afford.
>
> So I would ignore that problem for now. The main issue would be stuff in
> /var, I believe, so one approach would be to restrict a possible search
> there. Another would be to have a set of known directories to cleanup on
> purge that would be specified in the sysuser configuration.
>
> >  - What about uid sharing? I install torrent client, purge it (files
> >must stay, you know). Now I install webserver, and it owns my
> >downloaded files. Wrong.
>
> Other operating systems have a static registry of UIDs. For example,
> FreeBSD ports, when they create users, have to ask for a UID allocation
> first, by patching two text files as part of their upload:
>
> https://www.freebsd.org/doc/en_US.ISO8859-1/books/porters-handbook/users-and-groups.html
>
> This is something we could consider implementing in dh-sysusr. We
> already have such a voluntary registry in base-passwd:
>
> https://sources.debian.net/src/base-passwd/3.5.42/README/
>
> ... on top of the builtin static set of users that are always present,
> obviously:
>
> https://sources.debian.net/src/base-passwd/3.5.42/passwd.master/
>
> Obviously, this would require cross-package coordination, something that
> is traditionnally more difficult in Debian than in more centralized
> distributions. But base-passwd seems to be making that effort already,
> so we could expand on that.
>
> Not purging the users only fixes part of this problem, by the way. Since
> UID allocation is currently order-dependent (e.g. depending on if the
> torrent client or webserver was installed first, in your example) you
> may still end up with different UIDs on different machines. While this
> may not seem to be a problem at first, consider ext-formatted USB drives
> that travel accross machines or 

Bug#848239: deluser on purge

[Antoine Beaupré]

Adding the base-passwd maintainers to the conversation. TL;DR: this is
about the new dh-sysuser wrapper to more easily create system users in
package maintainer scripts. I want to see how we can make piuparts happy
and cleanup properly after ourselves (by removing the user on purge) and
that may involve a registry of UIDs which seems to be maintained in
base-passwd now.

On 2016-12-16 23:08:31, Dmitry Bogatov wrote:
> control: tag -1 +moreinfo
>
> [2016-12-15 09:07] Antoine Beaupré 
>>
>> part   text/plain1199
>> Package: dh-sysuser
>> Version: 1.3
>> Severity: wishlist
>>
>> I think that, under certain circumstances, users should be completely
>> removed along with their $HOME directory when the package is purged.
>
> See #840469 for discussion, why deleting $HOME is troublesome.

That would be a great reference to add in the documentation. Something
like:

.SH LIMITATIONS

dh-sysuser does not remove the user on package purge, for security
reasons. See bug #840469 for more details.

> If you
> can propose principled solution -- it would be great. In short:
>
>  - postrm has access only to essential tools

I reviewed the thread and that *is* a tricky situation. How about prerm?
Can we be wild and remove on purge there?

>  - What if there is something valuable is in $HOME? What if sysadmin
>made some obscure thing, link `ln -sf / $HOME'?

In any case, don't follow symlinks.

But the nastier attack scenario that was mentionned in #840469 is simply
an admin mistakenly or mailiciously setting that user's home to /. The
solution here could be to keep track of the *original* home directory so
that changing the home directory shouldn't pose problems. I mentionned
this in #848240 - having a way of knowing where the homedir will be
created would be useful there.

Shared UIDs are also a problem, but those can be checked before
fairly easily.

Could we simply make an exhaustive list of potential problems with
rm -rf $HOME and address each one? Or is it an inherently impossible
problem? What are admins supposed to do in this case? Review every file
in $HOME before they rm -rf the directory? We should be able to automate
whatever an admin is expected to do when the user is removed... and if
we don't automate it, we should direct admins to do *something*...

>  - What do to with files outside of of $HOME?

Ignore, for now. One could seek and destroy all those files, but that
would be too time-consuming. Disk quotas may be (ab)used to figure out
if there are remaining files, when enabled, but that's another
dependency we probably can't afford.

So I would ignore that problem for now. The main issue would be stuff in
/var, I believe, so one approach would be to restrict a possible search
there. Another would be to have a set of known directories to cleanup on
purge that would be specified in the sysuser configuration.

>  - What about uid sharing? I install torrent client, purge it (files
>must stay, you know). Now I install webserver, and it owns my
>downloaded files. Wrong.

Other operating systems have a static registry of UIDs. For example,
FreeBSD ports, when they create users, have to ask for a UID allocation
first, by patching two text files as part of their upload:

https://www.freebsd.org/doc/en_US.ISO8859-1/books/porters-handbook/users-and-groups.html

This is something we could consider implementing in dh-sysusr. We
already have such a voluntary registry in base-passwd:

https://sources.debian.net/src/base-passwd/3.5.42/README/

... on top of the builtin static set of users that are always present,
obviously:

https://sources.debian.net/src/base-passwd/3.5.42/passwd.master/

Obviously, this would require cross-package coordination, something that
is traditionnally more difficult in Debian than in more centralized
distributions. But base-passwd seems to be making that effort already,
so we could expand on that.

Not purging the users only fixes part of this problem, by the way. Since
UID allocation is currently order-dependent (e.g. depending on if the
torrent client or webserver was installed first, in your example) you
may still end up with different UIDs on different machines. While this
may not seem to be a problem at first, consider ext-formatted USB drives
that travel accross machines or networked filesystems like NFS: in those
cases you still have the problem of conflicting UIDs.

So a more complete solution to this problem is necessary, and I believe
sysuser could be the answer for this.

> For now dh-sysuser does not pretend to do something big.  You just
> create user, optionally with home directory and run some server with
> this user. Shell, password, ... are irrelevant.

That's a great start! But I'm optimistic and hope we can fix more
problems through the use of a standardized tool. :)

> I am okay with making it one-stop solution, but I do not know how to
> achive it.

That's cool, let's talk! ;)

A.

-- 
Le matraquage publicitaire est une 

Bug#848239: deluser on purge

[Dmitry Bogatov]

control: tag -1 +moreinfo

[2016-12-15 09:07] Antoine Beaupré 
>
> part   text/plain1199
> Package: dh-sysuser
> Version: 1.3
> Severity: wishlist
>
> I think that, under certain circumstances, users should be completely
> removed along with their $HOME directory when the package is purged.

See #840469 for discussion, why deleting $HOME is troublesome. If you
can propose principled solution -- it would be great. In short:

 - postrm has access only to essential tools

 - What if there is something valuable is in $HOME? What if sysadmin
   made some obscure thing, link `ln -sf / $HOME'?

 - What do to with files outside of of $HOME?

 - What about uid sharing? I install torrent client, purge it (files
   must stay, you know). Now I install webserver, and it owns my
   downloaded files. Wrong.

For now dh-sysuser does not pretend to do something big.  You just
create user, optionally with home directory and run some server with
this user. Shell, password, ... are irrelevant.

I am okay with making it one-stop solution, but I do not know how to
achive it.

--
X-Web-Site: https://sinsekvu.github.io | Note that I process my email in batch,
Accept-Languages: eo,ru,en | at most once every 24 hours. If matter
Accept: text/plain, text/x-diff| is urgent, you have my phone number.


pgpQG4JYQxvrK.pgp
Description: PGP signature

Bug#848239: deluser on purge

[Antoine Beaupré]

Package: dh-sysuser
Version: 1.3
Severity: wishlist

I think that, under certain circumstances, users should be completely
removed along with their $HOME directory when the package is purged.

I think this is a reasonable expectation of Debian packages. Some
packages (e.g. mysql, iirc) explicitly prompt the user through debconf
to confirm file removal, which could be an option here as well.

I understand why the user is disabled when the package is removed
right now, but in my use case (IRC bot) there isn't much state left
behind and i believe it's fine to remove the files on purge.

Disabling the user could be done on "remove".

-- System Information:
Debian Release: 8.6
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'proposed-updates'), (500, 
'stable'), (1, 'testing')
Architecture: amd64 (x86_64)

Kernel: Linux 4.7.0-0.bpo.1-amd64 (SMP w/2 CPU cores)
Locale: LANG=fr_CA.UTF-8, LC_CTYPE=fr_CA.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages dh-sysuser depends on:
ii  perl  5.20.2-3+deb8u6

dh-sysuser recommends no packages.

dh-sysuser suggests no packages.

-- no debconf information