Re: tmpfiles.d and docker images (was Re: opentmpfiles & opensysusers, and its use in the Debian policy)

[Thomas Goirand]

On 2/19/20 1:55 AM, Michael Hudson-Doyle wrote:
> On Fri, 3 Jan 2020 at 06:29, Simon McVittie  > wrote:
> 
> On Fri, 03 Jan 2020 at 00:05:10 +0800, Shengjing Zhu wrote:
> > The advantages of sysusers.d and tmpfiles.d are that you don't need to
> > call some magic scripts.
> > You only need to write declarative configuration files.
> 
> Individual packages only need to install declarative configuration
> files,
> but the OS distribution infrastructure needs to do something to make
> the contents of those configuration files take effect (and, crucially,
> know when the implementation has finished running).
> 
> On systemd systems, that's approximately:
> 
> - run systemd-tmpfiles when a package installs a tmpfiles.d snippet
>   (this is added to the package's postinst by dh_installsystemd)
> 
> - run systemd-sysusers when a package installs a sysusers.d snippet
>   (I don't think we have tools to add this to the postinst yet, because
>   packages are currently meant to run adduser --system instead, but
>   more-systemd-centric distributions probably already do this in their
>   equivalent of the postinst)
> 
> - run systemd-tmpfiles during boot (this is
> systemd-tmpfiles-setup.service,
>   part of systemd)
> 
> - run systemd-sysusers during boot (this is
>   systemd-sysusers.service, part of systemd)
> 
> The opentmpfiles and opensysusers packaging will need to arrange to do
> something analogous, most likely in cooperation with dh_installsystemd
> or some other debhelper step for the first two points, and with LSB init
> scripts for the tasks where systemd uses one-shot services.
> 
> 
> So in Ubuntu we got this interesting
> bug https://bugs.launchpad.net/ubuntu/+source/haproxy/+bug/1855140 which
> can be summarized as saying that haproxy doesn't work out of the box in
> a docker container, because it installs a tmpfiles.d snippet but nothing
> processes it (I haven't tried, but I very much expect that the behaviour
> is the same between Debian and Ubuntu here).
> 
> On the face of it, this is a bug in the package: it depends for its
> operation on a package not listed in Depends but making the package
> depend on systemd wouldn't actually be very useful for getting it
> running in docker, for a few reasons. I think it would be better to
> change debhelper:
> 
>  * to add "systemd | opentmpfiles" to misc:Depends whenever a package
> installs a tmpfiles.d snippet
>  * stop checking for -d /run/systemd/system in
> autoscripts/postinst-init-tmpfiles and check for systemd-tmpfiles /
> opentmpfiles in $PATH instead (and using the one that is found, obviously)

Well, not right now, as we're still not sure on how this will happen,
and what the tech ctte will decide. I'm hoping for a resolution where
the TC will decide what we should do, both in terms of packaging
opemtmpfiles and systemd, and how packages should use them.

> Obviously there are a few variations on the above, like:
> 
>  * splitting a systemd-tmpfiles package out of the systemd package first

I very much hope this will happen. Though the bug has been opened for a
while already, and I see no communication from systemd maintainers about
it (or did I miss it?).

>  * change systemd(-tmpfiles) and opentmpfiles to Provides: tmpfiles and
> use alternatives to provide /usr/bin/tmpfiles, have debhelper reference
> these instead

What you describe above means that we will not have a cross-distro
/sbin/systemd-tmpfiles interface/path. That's annoying. Which is why
it'd be just preferred if opentmpfiles just use systemd's path. Though
that's still under discussion. I also hope that the TC will give
guidance for this.

The "Provides: tmpfiles" is really something I'd love we have.

> Does it make sense to people here? I can try to work on a patch
> (although my perl isn't the greatest).

Very much. Though IMO, it's not up to a single person to decide what
Debian should do.

Cheers,

Thomas Goirand (zigo)

Re: tmpfiles.d and docker images (was Re: opentmpfiles & opensysusers, and its use in the Debian policy)

[Vincent Bernat]

 ❦ 19 février 2020 13:55 +13, Michael Hudson-Doyle 
:

> So in Ubuntu we got this interesting bug
> https://bugs.launchpad.net/ubuntu/+source/haproxy/+bug/1855140 which can be
> summarized as saying that haproxy doesn't work out of the box in a docker
> container, because it installs a tmpfiles.d snippet but nothing processes
> it (I haven't tried, but I very much expect that the behaviour is the same
> between Debian and Ubuntu here).

The shipped init script contains the appropriate code to create
/run/haproxy. If the user is using the systemd unit file, the directory
was created by tmpfiles.d. If they are using the init script, the
directory is created by the init script. If the user uses neither of
them, there is not much we can do. We cannot create this directory at
install time, since /run can be wiped out on boot (the fact Docker
doesn't do that is likely an implementation detail).

> But the approach I outlined seems simplest and easiest to implement. Does
> it make sense to people here? I can try to work on a patch (although my
> perl isn't the greatest).

The simplest path would be to not do anything. Docker users can add "RUN
install -d -m 2775 -o haproxy -g haproxy /run/haproxy" in their
Dockerfile or work on a solution where the tmpfiles present in the image
are processed when spawning the image (like the fact that
`/etc/resolv.conf` is updated to match the current network
configuration). Adding more non-declarative stuff in postinst scripts
does not seem a good solution for me.
-- 
Don't stop at one bug.
- The Elements of Programming Style (Kernighan & Plauger)


signature.asc
Description: PGP signature

tmpfiles.d and docker images (was Re: opentmpfiles & opensysusers, and its use in the Debian policy)

[Michael Hudson-Doyle]

On Fri, 3 Jan 2020 at 06:29, Simon McVittie  wrote:

> On Fri, 03 Jan 2020 at 00:05:10 +0800, Shengjing Zhu wrote:
> > The advantages of sysusers.d and tmpfiles.d are that you don't need to
> > call some magic scripts.
> > You only need to write declarative configuration files.
>
> Individual packages only need to install declarative configuration files,
> but the OS distribution infrastructure needs to do something to make
> the contents of those configuration files take effect (and, crucially,
> know when the implementation has finished running).
>
> On systemd systems, that's approximately:
>
> - run systemd-tmpfiles when a package installs a tmpfiles.d snippet
>   (this is added to the package's postinst by dh_installsystemd)
>
> - run systemd-sysusers when a package installs a sysusers.d snippet
>   (I don't think we have tools to add this to the postinst yet, because
>   packages are currently meant to run adduser --system instead, but
>   more-systemd-centric distributions probably already do this in their
>   equivalent of the postinst)
>
> - run systemd-tmpfiles during boot (this is systemd-tmpfiles-setup.service,
>   part of systemd)
>
> - run systemd-sysusers during boot (this is
>   systemd-sysusers.service, part of systemd)
>
> The opentmpfiles and opensysusers packaging will need to arrange to do
> something analogous, most likely in cooperation with dh_installsystemd
> or some other debhelper step for the first two points, and with LSB init
> scripts for the tasks where systemd uses one-shot services.


So in Ubuntu we got this interesting bug
https://bugs.launchpad.net/ubuntu/+source/haproxy/+bug/1855140 which can be
summarized as saying that haproxy doesn't work out of the box in a docker
container, because it installs a tmpfiles.d snippet but nothing processes
it (I haven't tried, but I very much expect that the behaviour is the same
between Debian and Ubuntu here).

On the face of it, this is a bug in the package: it depends for its
operation on a package not listed in Depends but making the package depend
on systemd wouldn't actually be very useful for getting it running in
docker, for a few reasons. I think it would be better to change debhelper:

 * to add "systemd | opentmpfiles" to misc:Depends whenever a package
installs a tmpfiles.d snippet
 * stop checking for -d /run/systemd/system in
autoscripts/postinst-init-tmpfiles and check for systemd-tmpfiles /
opentmpfiles in $PATH instead (and using the one that is found, obviously)

Obviously there are a few variations on the above, like:

 * splitting a systemd-tmpfiles package out of the systemd package first
 * change systemd(-tmpfiles) and opentmpfiles to Provides: tmpfiles and use
alternatives to provide /usr/bin/tmpfiles, have debhelper reference these
instead

But the approach I outlined seems simplest and easiest to implement. Does
it make sense to people here? I can try to work on a patch (although my
perl isn't the greatest).

Cheers,
mwh

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Russ Allbery]

Zbigniew Jędrzejewski-Szmek  writes:

> we talked this over in systemd upstream, and decided to make the promise
> a bit more official [0]. Independent operation of a bunch of programs is
> now explicitly covered, and the stability promise has been extended to
> more interfaces. The old wiki page with the stability/portability charts
> [1,2] is now replaced by an in-repo page [3], which is the new official
> location for this data [4].

Thank you!  This is very helpful.

-- 
Russ Allbery (r...@debian.org)  

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Zbigniew Jędrzejewski-Szmek]

On Thu, Jan 02, 2020 at 10:04:28AM -0800, Russ Allbery wrote:
> Matthias Klumpp  writes:
> 
> > I find that statement quite encouraging. Of course they don't commit
> > to not having those depend on systemd-as-PID1, but there really isn't
> > a reason to create that dependency, and if for whatever reason there
> > will be one at some point, we can switch away on systems that don't
> > support that change rather easily.
> 
> I specifically asked that question in a follow-up to that message and
> Zbyszek said that this is very likely to continue to work indefinitely.

Hi,

we talked this over in systemd upstream, and decided to make the
promise a bit more official [0]. Independent operation of a bunch of
programs is now explicitly covered, and the stability promise
has been extended to more interfaces. The old wiki page with the
stability/portability charts [1,2] is now replaced by an in-repo
page [3], which is the new official location for this data [4].

[0] 
https://systemd.io/PORTABILITY_AND_STABILITY/#independent-operation-of-systemd-programs
[1] 
https://www.freedesktop.org/wiki/Software/systemd/InterfacePortabilityAndStabilityChart/
[2] https://www.freedesktop.org/wiki/Software/systemd/InterfaceStabilityPromise/
[3] https://systemd.io/PORTABILITY_AND_STABILITY/

Zbyszek

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Thomas Goirand]

On 1/3/20 1:50 PM, Ansgar wrote:
> (Probably Thomas forgot to push the branches to GitLab.)

Yes, that's the case. My apologies for this.

Cheers,

Thomas Goirand (zigo)

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Thomas Goirand]

On 1/3/20 10:39 PM, Marco d'Itri wrote:
> On Jan 03, Thomas Goirand  wrote:
> 
>> Could you please, therefore, tell me what feature is missing? If you
> If I am not mistaken then you started arguing that we should consider an 
> hypothetical alternative implementation with missing features, so maybe 
> you should explain what may be missing.

I haven't been the one bringing this topic. Sam did.

Sam wrote something along the line with "what if systemd implements a
new feature", to which I replied that systemd shouldn't dictate to
Debian all we should do.

There's multiple ways to fix this kind of situation. Obviously, one of
them is to implement that missing feature.

Another possible way, is an exit strategy where we declare the new
feature as not supported in Debian, just because we have the power to
decide.

All of this *may* happen, this is purely hypothetical, and this should
be addressed on a case by case basis. But we're not even talking about
something that we know exist, so this is all FUD and a very
counter-productive discussion.

Cheers,

Thomas Goirand (zigo)

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[The Wanderer]

On 2020-01-04 at 05:57, Simon McVittie wrote:

> On Fri, 03 Jan 2020 at 19:52:33 -0500, The Wanderer wrote:
> 
>> On 2020-01-03 at 19:33, Simon McVittie wrote:
>> 
>>> D-Bus activation is a D-Bus feature where instead of starting a 
>>> D-Bus service (another sort of daemon) "eagerly" in case it might
>>> be needed, the dbus-daemon starts that D-Bus service "lazily" the
>>> first time some other program sends a message to it.
>> 
>> This sounds like the feature I was thinking of. I think I
>> understood that the way this message got from the originating
>> program to D-Bus (and thence to the service, once running) was by
>> way of a socket which would / should be owned by that service, much
>> as you describe for systemd's socket-activation feature
> 
> No, part of the purpose of the "message bus" part of D-Bus is that it
> provides a hub-and-spoke topology so that connecting n clients and 
> services together only requires O(n) connections, not O(n**2).
> Clients connect to a socket owned by the message bus service, and
> send messages through it. Some messages are processed by the message
> bus itself. The rest have a header that tells the message bus which
> service is the intended destination, and it either: delivers the
> message to the destination service immediately; activates (starts)
> the destination service (via either traditional or systemd
> activation), and then delivers the message when it appears; or
> replies with an error message that means "no, I can't do that" (for
> example if the requested service isn't installed).

That actually sounds even more like how I originally thought this
feature worked, and the name "D-Bus socket activation" makes even more
intuitive sense in my mind for the "lazy" activation of services by this
method.

Still, if that's not what the terminology has been established to mean,
then there's no point in belaboring the issue. I certainly have enough
of an aversion for unnecessary and unintended ambiguity that I'm not
interested in introducing more without good cause.

Once again, thanks for the explanation!

-- 
   The Wanderer

The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man. -- George Bernard Shaw



signature.asc
Description: OpenPGP digital signature

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Simon McVittie]

On Fri, 03 Jan 2020 at 19:52:33 -0500, The Wanderer wrote:
> On 2020-01-03 at 19:33, Simon McVittie wrote:
> > D-Bus activation is a D-Bus feature where instead of starting a
> > D-Bus service (another sort of daemon) "eagerly" in case it might be
> > needed, the dbus-daemon starts that D-Bus service "lazily" the first
> > time some other program sends a message to it.
> 
> This sounds like the feature I was thinking of. I think I understood
> that the way this message got from the originating program to D-Bus (and
> thence to the service, once running) was by way of a socket which would
> / should be owned by that service, much as you describe for systemd's
> socket-activation feature

No, part of the purpose of the "message bus" part of D-Bus is that
it provides a hub-and-spoke topology so that connecting n clients and
services together only requires O(n) connections, not O(n**2). Clients
connect to a socket owned by the message bus service, and send messages
through it. Some messages are processed by the message bus itself. The
rest have a header that tells the message bus which service is the
intended destination, and it either: delivers the message to the
destination service immediately; activates (starts) the destination
service (via either traditional or systemd activation), and then
delivers the message when it appears; or replies with an error message
that means "no, I can't do that" (for example if the requested service
isn't installed).

Other IPC systems that are the same shape as D-Bus would often refer
to the message bus service as a broker - it's the same thing.
The reference implementation of the message bus is dbus-daemon, from the
dbus package. Alternatives like dbus-broker exist, but are not currently
available in Debian.

For the well-known system bus (dbus-daemon --system) the socket is
/var/run/dbus/system_bus_socket, and you can see connections to it in the
output of netstat, lsof or ss. It is conceptually owned by the message
bus service, but if socket activation is in use then it's really created
by systemd on behalf of the message bus service.

smcv

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Bernd Zeimetz]

On 1/3/20 10:05 AM, Thomas Goirand wrote:
> On 1/3/20 2:33 AM, Sam Hartman wrote:
>> Secondly, by using systemd-tmpfiles when we can, we gain support for any
>> additional features that are implemented.
> 
> That's where I don't agree. While it's nice to have such a declarative
> system, I don't think it's reasonable to impose the implementation of
> any change to systemd to all the other init systems. At some point, *we*
> must be able to decide.

We have decided. Its systemd. Please lets not start this discussion again.

-- 
 Bernd ZeimetzDebian GNU/Linux Developer
 http://bzed.dehttp://www.debian.org
 GPG Fingerprint: ECA1 E3F2 8E11 2432 D485  DD95 EB36 171A 6FF9 435F

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Sam Hartman]

> "Thomas" == Thomas Goirand  writes:

Thomas> On 1/3/20 7:31 PM, Russ Allbery wrote:
Thomas> explore and develop alternate init systems and alternatives to 
systemd
Thomas> features."

Thomas> Exploring alternatives is precisely what I'm doing. The same way 
I've
Thomas> felt harassed when I did the original porting of OpenRC to 1/ 
Debian,
Thomas> then 2/ kFreeBSD, then 3/ Hurd, I do feel like everyone is rushing 
into
Thomas> me for no reasons. Could you please stop doing that, both you and 
Sam,
Thomas> and instead, help me with deciding how both implementation can 
cohexist?

I tried to help break apart the exploring alternatives stuff earlier and
give concrete suggestions.

I see several cases:

1) You want to in limited circumstances test  opentmpfiles and
opensysusers as a replacement for systemd-tmpfiles on Linux--you want to
explore opentmpfiles or opensysusers in the terms of Proposal B.

My recommendation is use dpkg-divert..

You minimize impact to people not involved in your exploration.
You make it clear that it is exploratory.



2) You want an implementation of sysusers and tmpfiles on non-linux
platforms.

My recommendation is build using the systemd names and track the systemd
feature set as closely as you can


3) You want a platform on which to experiment with new features in
tmpfiles or sysusers that are different from systemd (or to experiment
with different triggering mechanisms or whatever)-- in the terms of
Proposal B, you're developing an alternative to a systemd feature.


You should use different names both for  /etc/tmpfiles.d and for the
binary to run them (and same for sysusers).
Your packages should be co-installable.
You might want your own dh targets etc.

Opt in should effectively be per-package not per-system.

4) You want an implementation of tmpfiles to  use that is compatible
with systemd for init systems other than systemd

Either work with maintainers to get tmpfiles split out from the systemd
package or work with I'm not really sure who to get systemd and elogind
co-installable.
Then use systemd-tmpfiles.

5) You want a way to use an implementation other than systemd's on linux
for non-experimental purposes.

That's not exploration or development.
At that point, I think Debian should evaluate whether what you're doing
makes sense.
I currently don't think it does.

I think it's fine for Debian to say we don't need that.
Right now, given  that you have option 1 and 3 above, I think that's
fine.

If you find some huge advantage in an alternate implementation--lets say
opentmpfiles is more secure, uses less power, and runs faster--then you
can come back and report the results of your exploration.  And at that
point I'll probably change my mind and either suggest we should move
entirely to opentmpfiles or pay the cost to support alternatives.

My reading of Proposal B is that the explorations and development you
want to do need to be possible.
They don't have to be hugely elegant, and we don't have to support them
in production.

We do need to support enough integration that other people can do their
explorations/development.
So, the fact that we don't have a solution for alternate inits with
tmpfiles or sysusers is a bug we need to fix.
My preference to fix that bug is to split out tmpfiles and sysusers from
systemd based on what I know today.
Other solutions like an implementation that conflicts with systemd are
possible; I think they are technically inferior choices to letting
people explore and develop alternate inits while providing the tmpfiles
and sysusers systemd features.

--Sam

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[The Wanderer]

On 2020-01-03 at 19:33, Simon McVittie wrote:

> On Fri, 03 Jan 2020 at 18:07:36 -0500, The Wanderer wrote:
> 
>> What I'm concerned about is dbus socket activation, or similar,
>> leading to e.g. logind getting activated by logging in at the text
>> console.
>> 
>> I thought I understood that socket activation via dbus was one of
>> the features which didn't require systemd as PID-1 to function.
> 
> I think you're confused. I don't think "socket activation via dbus" 
> has ever been a thing that existed.

I will certainly take your word for that. That it was a thing is an
understanding I acquired over the course of reading previous
mailing-list discussions (both contentious and otherwise), and in
examining the idea more closely over the course of this conversation
I've begun to notice some things that don't seem quite right about it
anyway.

> Socket activation is a systemd feature resembling inetd



Thanks for the explanation.

> D-Bus activation is a D-Bus feature where instead of starting a
> D-Bus service (another sort of daemon) "eagerly" in case it might be
> needed, the dbus-daemon starts that D-Bus service "lazily" the first
> time some other program sends a message to it.

This sounds like the feature I was thinking of. I think I understood
that the way this message got from the originating program to D-Bus (and
thence to the service, once running) was by way of a socket which would
/ should be owned by that service, much as you describe for systemd's
socket-activation feature, and that automatic "lazy" activation of the
service by D-Bus in this way was therefore called D-Bus socket activation.

This may itself be based on some confused understandings of some of the
terms involved. If so, please don't feel obliged to spend your time
explaining them; although I'd be glad to learn, I'm sure you have more
productive things to do.



> Some of systemd's smaller daemons, like logind, are activatable
> D-Bus services, so in principle they could be started by dbus-daemon
> via traditional activation. However, the "program to execute" in the
> D-Bus service definition is set to /bin/false, so they cannot
> actually be activated successfully unless dbus-daemon was told to
> carry out systemd activation, which only happens on systems where
> systemd is pid 1.

That clarifies much. Thank you for the explanation!

>> it also appears to leave systemd-the-package as much less useful to
>> install without systemd-sysv
> 
> The systemd binary package is useful for systemd-tmpfiles, 
> systemd-sysusers, controlling a container or 
> system-image-under-construction that *does* boot with systemd 
> (systemctl --root=/mnt/otherdevice), or booting with
> init=/bin/systemd as a non-default option. Other than that, you're
> correct to say that systemd-sysv is the package that actually
> provides services (pid 1, logind, journald, etc.).

All of these (except the system-image-under-construction) are part of
what I had in mind when saying "much less useful" rather than "useless".
Thanks for listing them, though!

>> (I wonder if maybe I had libpam-systemd installed at the time, and
>> that was what was triggering logind to run? It's possible that this
>> may have been back before that couldn't be installed without
>> systemd-sysv
> 
> I suspect you were using libpam-systemd in combination with
> systemd-shim. The entire point of systemd-shim was that it would run
> systemd-logind on systems that did not have systemd as pid 1. This
> turned out to be unsupportable and it was removed.

That sounds right, yes. I was surprised to find out just how much that
I'd thought would grow dependencies on systemd-logind, and hence become
unusable without the behavior changes which come along with logind, did
not actually do so.

-- 
   The Wanderer

The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man. -- George Bernard Shaw



signature.asc
Description: OpenPGP digital signature

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Simon McVittie]

On Fri, 03 Jan 2020 at 18:07:36 -0500, The Wanderer wrote:
> What I'm concerned about is dbus socket activation, or similar, leading
> to e.g. logind getting activated by logging in at the text console.
> 
> I thought I understood that socket activation via dbus was one of the
> features which didn't require systemd as PID-1 to function.

I think you're confused. I don't think "socket activation via dbus"
has ever been a thing that existed.

Socket activation is a systemd feature resembling inetd, where instead
of starting a daemon (let's say a web server) that listens on a port (let's
say TCP port 80), systemd starts listening on port 80 on behalf of the
web server, and doesn't actually run the web server itself yet. Later,
when a client connects to that socket, systemd starts the web server
and hands over the listening socket to it, the web server accepts the
client's connection and everything continues as usual.

Perhaps confusingly, on machines where systemd is pid 1, the dbus-daemon
is one of many services that makes use of socket activation: it
configures systemd to listen on the well-known AF_UNIX socket for the
system dbus-daemon, and hand over to the actual dbus-daemon the first
time there is a client. (The relevant files are dbus.socket and
dbus.service, if you're interested.)

D-Bus activation is a D-Bus feature where instead of starting a D-Bus
service (another sort of daemon) "eagerly" in case it might be needed,
the dbus-daemon starts that D-Bus service "lazily" the first time some
other program sends a message to it. dbus-daemon can either do this
directly as its own child process, which is portable to all reasonable
Unix platforms (this is called "traditional activation"); or on systemd
systems, if the D-Bus service definition includes the name of a systemd
unit, dbus-daemon can start it indirectly, by asking systemd to start
that unit (this is "systemd activation"). systemd activation is preferred
where supported, because systemd was designed to be a service manager,
whereas dbus-daemon is an IPC broker with some very basic service
management bolted onto the side.

Both of those lazy-loading mechanisms (socket activation and D-Bus
activation) let services start with greater parallelization and fewer
explicit dependencies, and they are particularly useful in situations
where service A can be configured in a way that makes it depend on
service B (but does not always), and service B can be configured in
a way that makes it depend on service A (but does not always). In a
traditional linearly-ordered boot process like sysv-rc, there would be no
correct order in which to start A and B, so Debian would have to choose
one configuration to allow, and deliberately break the other one (for
example if we chose to start A before B, then configuring A to depend on
B while configuring B to not depend on A can't work). With activation,
as long as you don't try to configure each of A and B to depend on the
other at the same time (which obviously can't work), either way round
works equally well.

Some of systemd's smaller daemons, like logind, are activatable D-Bus
services, so in principle they could be started by dbus-daemon via
traditional activation. However, the "program to execute" in the D-Bus
service definition is set to /bin/false, so they cannot actually be
activated successfully unless dbus-daemon was told to carry out systemd
activation, which only happens on systems where systemd is pid 1.

There has been a proposal to adjust the syntax of D-Bus service
definitions so that they can say "if systemd is pid 1, activate via
systemd unit foo.service, else behave as though this activatable
service didn't exist", rather than having to use Exec=/bin/false as a
workaround. I'll probably implement that at some point, and logind will
probably be one of the first services to use it.

> it also appears to leave systemd-the-package
> as much less useful to install without systemd-sysv

The systemd binary package is useful for systemd-tmpfiles,
systemd-sysusers, controlling a container or
system-image-under-construction that *does* boot with systemd
(systemctl --root=/mnt/otherdevice), or booting with init=/bin/systemd
as a non-default option. Other than that, you're correct to say that
systemd-sysv is the package that actually provides services (pid 1,
logind, journald, etc.).

> (I wonder if maybe I had libpam-systemd installed at the time, and that
> was what was triggering logind to run? It's possible that this may have
> been back before that couldn't be installed without systemd-sysv

I suspect you were using libpam-systemd in combination with systemd-shim.
The entire point of systemd-shim was that it would run systemd-logind
on systems that did not have systemd as pid 1. This turned out to be
unsupportable and it was removed.

smcv

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Matthias Klumpp]

Am Sa., 4. Jan. 2020 um 00:16 Uhr schrieb Russ Allbery :
>
> The Wanderer  writes:
>
> > What I'm concerned about is dbus socket activation, or similar, leading
> > to e.g. logind getting activated by logging in at the text console.
>
> > I thought I understood that socket activation via dbus was one of the
> > features which didn't require systemd as PID-1 to function.
>
> It may be that the user service manager in systemd supports D-Bus socket
> activation, but none of the user services provided in the systemd package
> seem to start logind, so I don't think you'll get that service started
> that way.  I'm only seeing systemd-exit and systemd-tmpfiles.
>
> Also, I'm not sure what would start the user service manager if you're not
> running systemd as PID 1.

Indeed. Looking through the large set of binaries and interfaces
provided by the systemd package, I do wonder though: One of the points
of the systemd project is to actually provide interfaces applications
and the OS can rely on being present. For that matter, it simply
doesn't make sense to split up the systemd package, as it makes
maintenance harder and may only introduce issues. At the same time
though, the usecase of containers and minimal chroots exists, where
you sometimes want to save as much space as possible and where you
will never ever run a full initsystem / service manager.
The systemd maintainers already split out machinectl and nspawn into
their own systemd-container package, as it's not needed in every
configuration.

Potentially, splitting out the bare minimum binaries to run a Debian
system in a container setting / minimal chroot makes sense for the
Debian systemd maintainers. This, coincidentally, would also put users
of other init systems at ease. I believe by approaching the problem
that way, there may actually be a possibility for mutually beneficial
synergies here.
Looking through the postinst scripts and contents of the systemd
package, I am very certain that it won't suddenly run anything if
you're not already on a systemd-as-PID1 system. So having it should be
safe for alternative initsystems (however, note that I haven't tested
that).

The systemd package currently (244-3) contains the following utilities[1]:

## (mostly) User-invoked CLI tools
Those are components usually invoked by a human, but may be used in
scripts as well.
 * journalctl: Read the system journal, if available (otherwise does nothing)
 * loginctl: Control login sessions, logind related stuff
 * systemctl: Control system services / system state
 * systemd-escape: Escape strings for usage in systemd unit names
 * systemd-inhibit: Execute a program with an inhibition lock taken
(e.g. prevent sleep/idle)
 * bootctl: Control the firmware and boot manager settings
 * busctl: Introspect the bus (D-Bus)
 * hostnamectl: Control system hostname and related stuff
 * kernel-install: Add and remove kernel and initramfs images to and from /boot
 * localectl: Change system locale
 * timedatectl: Modify system time/date
 * resolvectl: Introspect and reconfigure the DNS resolver
 * systemd-analyze: Analyze and debug system manager (only useful if
systemd is PID1)
 * systemd-cat: Connect a pipeline or program's output with the journal
 * systemd-cgls: Recursively show control group contents
 * systemd-cgtop: Show top control groups by their resource usage
 * systemd-delta: Find overridden configuration files
 * systemd-id128: Generate and print sd-128 identifiers
 * systemd-mount, systemd-umount: Establish and destroy transient
mount or auto-mount points
 * systemd-path: List and query system and user paths
 * systemd-run: Run programs in transient scope units, service units,
or path-, socket-, or timer-triggered service units (needs the service
manager to be PID1)
 * systemd-socket-activate: Test socket activation of daemons

## Systemd core components
These components are usually invoked by scripts, but may be used by
humans as well.
 * systemd: PID1
 * systemd-ask-password: Query for a root password, e.g. when a
bootsplash is active. Possibly useful w/o sd PID1
 * systemd-machine-id-setup - Initialize the machine ID in /etc/machine-id
 * systemd-notify: Notify service manager about start-up completion
and other daemon status changes
 * systemd-sysusers: Allocate system users and groups
 * systemd-tmpfiles: Creates, deletes and cleans up volatile and
temporary files and directories
 * systemd-tty-ask-password-agent: List or process pending systemd
password requests

 * systemd-detect-virt: Detect execution in a virtualized environment
 * systemd-stdio-bridge: STDIO or socket-activatable proxy to a given
DBus endpoint.

## Daemons
Some of them are enabled by default, some of them are optional and
have to be activated by the users, some of them (like journald) run in
modes which actually only forward data to other preexisting services
by default (rsyslog) unless instructed otherwise by the user.
None of those are run on systemd where systemd isn't PID1, because all
of the

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Simon McVittie]

On Fri, 03 Jan 2020 at 14:43:48 -0800, Russ Allbery wrote:
> The Wanderer  writes:
> > If I recall and understand correctly, installing systemd-the-package
> > will result in at least some of the daemons therein [being run]
> 
> This is the bit that I'm fairly sure is not the case.
> 
> All of the native systemd facilities are started via systemd units.  If
> systemd is not running as PID 1, nothing is going to load or act on the
> systemd units.  Therefore, nothing will start those services.

The three cases where this might not hold are:

* You have something that is specifically running one of systemd's
  various smaller daemons, like the way systemd-shim arranged for
  systemd-logind to run (and pretended to be the systemd "manager" - the
  actual init system - so that systemd-logind worked, at least partially).
  Now that systemd-shim has been removed, I don't think this case exists.

* One of systemd's smaller daemons is designed to run even on non-systemd
  systems, and so has an LSB init script as well as a native systemd unit.
  I think the only one in this situation is udev, which used to be a
  separate source package before becoming part of the systemd source
  package, and is already in its own separate binary package so it can
  be installed or removed independently.

* You install another init system that systematically runs systemd service
  units, either directly or by generating its own equivalent of service
  units: this would look a lot like the way systemd generates systemd
  services from LSB init scripts, but in reverse. I don't think this
  exists yet.

  If this happens, it will definitely need a mechanism to "mask" unsuitable
  units, such as those that are known to encode systemd assumptions, rely
  on systemd features that it doesn't implement, or have a better or more
  native way to work in the other init system - just like the way systemd
  "masks" LSB init scripts that have an equivalent systemd unit or are
  otherwise not suitable to be run under systemd, but in reverse.

  It will probably also need some more general rules or heuristics,
  like perhaps skipping early-boot systemd units (those with
  DefaultDependencies=no) because those are more likely to have
  systemd-specific behaviour and less likely to be compatible with
  other init systems. Again, this is the inverse of the way systemd's
  LSB init script compatibility layer in recent versions acts on
  /etc/rc[2-5].d (late boot) but ignores /etc/rcS.d (early boot).

  I suspect you'd find that all or nearly all of the units provided by
  the systemd binary package would be unsuitable to be run by the other
  init system, and would need to be either masked or excluded by a rule.

> (Splitting doesn't avoid the library dependency problem that currently
> causes problems with elogind, since I believe the programs in question
> also depend on that shared library.)

Actually...

systemd has a public shared library with a stable API, libsystemd0,
and a separate private shared library that is tightly coupled to the
contents of the systemd package, libsystemd-shared-${version}.so. If I
understand correctly, the systemd daemons and helper executables are
mostly (all?) linked to the private library, and are not linked to
the public library at all: the private library has its own copies of
some of the same functions that are in libsystemd0's ABI, for example
sd_uid_is_on_seat(). Certainly that seems to be true for -tmpfiles
and -sysusers.

(The practical effect is the same as if -tmpfiles and -sysusers were
statically linked to the shared systemd code that they need, but the
use of a private shared library makes them smaller than if that had
been done.)

If that's true in general, then the systemd binary package might not
actually need to depend on libsystemd0 at all; or failing that, it
might be feasible to split out -tmpfiles, -sysusers and possibly others
into a systemd-bin package that contains the private library and those
executables, and does not depend on libsystemd0. If that split took
place, systemd would depend on systemd-bin (= ${binary:Version}).

smcv

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Russ Allbery]

The Wanderer  writes:

> What I'm concerned about is dbus socket activation, or similar, leading
> to e.g. logind getting activated by logging in at the text console.

> I thought I understood that socket activation via dbus was one of the
> features which didn't require systemd as PID-1 to function.

It may be that the user service manager in systemd supports D-Bus socket
activation, but none of the user services provided in the systemd package
seem to start logind, so I don't think you'll get that service started
that way.  I'm only seeing systemd-exit and systemd-tmpfiles.

Also, I'm not sure what would start the user service manager if you're not
running systemd as PID 1.

-- 
Russ Allbery (r...@debian.org)  

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[The Wanderer]

On 2020-01-03 at 17:43, Russ Allbery wrote:

> The Wanderer  writes:
> 
>> If I recall and understand correctly, installing
>> systemd-the-package will result in at least some of the daemons
>> therein - including both systemd itself, and systemd-logind - being
>> set up to run automatically in the correct contexts (whether by
>> scripted invocation set up by a package, or by socket activation,
>> or by some other means), in such a way that they will generally
>> wind up running even on a computer where systemd-the-daemon is not
>> the init system. I'm currently digging through the result of
>> 'apt-get source systemd', and I haven't yet managed to either
>> confirm or refute this with certainty.
> 
> This is the bit that I'm fairly sure is not the case.
> 
> All of the native systemd facilities are started via systemd units.
> If systemd is not running as PID 1, nothing is going to load or act
> on the systemd units.  Therefore, nothing will start those services.

What I'm concerned about is dbus socket activation, or similar, leading
to e.g. logind getting activated by logging in at the text console.

I thought I understood that socket activation via dbus was one of the
features which didn't require systemd as PID-1 to function.

> This may be a concern in some future in which there are multiple
> init systems that interpret and act on systemd units, but this is not
> yet the case.  That will be an integration worry that we'll have to
> tackle should that be the case in the future, but I don't think it's
> a concern right now.

That does ease my mind about this somewhat, then.

It also leaves me confused about how these daemons were getting started
back when I was experimenting with this, to provide the side effects
which I (half-)remember seeing, since I definitely wasn't running
systemd as the init system; it also appears to leave systemd-the-package
as much less useful to install without systemd-sysv. But my confusion
about the past should not impede us acting in the present with regard to
the future.

(I wonder if maybe I had libpam-systemd installed at the time, and that
was what was triggering logind to run? It's possible that this may have
been back before that couldn't be installed without systemd-sysv, and
before I wound up giving up and removing it.)

>> Again if I recall correctly, there were some behaviors which arose
>> from having logind running in a non-systemd-as-init-system
>> environment which the maintainers did not consider something which
>> they would need to fix but which I found undesirable.
> 
> Possibly you're thinking of the problem that Sam pointed out, which
> is that the systemd package depends on libsystemd0 which currently 
> effectively conflicts with elogind, and therefore you can't install 
> elogind and the systemd package simultaneously?

No - at the point when I was experimenting with this, elogind did not
appear to have been packaged for Debian, and may (for all I know) not
even have been written yet.

>> I don't want to "risk" (in quotation marks because there may not
>> be much, if any, actual risk involved) my primary system on testing
>> this, and right now I don't have any spare systems or a working
>> virtualization environment (because I haven't been able to get
>> libvirt, as packaged for Debian, to work properly in a non-systemd
>> environment) to use for testing, so I'm not in a position to do
>> that test myself. I do have a functioning virtualization
>> environment at my workplace, so if downtime permits over the next
>> week or three, I may be able to do that there.
> 
> Totally understood, and obviously you're under no obligation to do
> the testing!

No, but if it would help resolve concerns (including my own) and
potentially help clear the way for things to move forward, I'd be
happier with it done - and if I can make myself happy, why shouldn't I? ^_^

>> (Personally, I'd argue that splitting the various daemons and
>> non-daemon tools out into packages according to which ones depend
>> on which others makes sense purely from a "granularity of
>> dependencies" perspective, but it's been clear for a long time that
>> that argument is a nonstarter with the systemd maintainers.)
> 
> The systemd maintainers do split out some binaries, but I don't
> think creating 20-odd packages for each individual small service
> (systemd has a general model of breaking the boot up into small,
> simple binaries that do a single thing) is necessarily an
> improvement.  My understanding of the preference of the systemd
> maintainers is to not split the package except where there's a clear
> benefit (in terms of dependency structure or some other problem) that
> outweighs the ongoing cost of maintaining more individual packages.
> Here, if the systemd package works the way that I believe that it
> does, I don't think splitting will change anything other than saving
> a small amount of disk space on non-systemd systems.

If you're right that installing systemd-the-package

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Russ Allbery]

The Wanderer  writes:

> If I recall and understand correctly, installing systemd-the-package
> will result in at least some of the daemons therein - including both
> systemd itself, and systemd-logind - being set up to run automatically
> in the correct contexts (whether by scripted invocation set up by a
> package, or by socket activation, or by some other means), in such a way
> that they will generally wind up running even on a computer where
> systemd-the-daemon is not the init system. I'm currently digging through
> the result of 'apt-get source systemd', and I haven't yet managed to
> either confirm or refute this with certainty.

This is the bit that I'm fairly sure is not the case.

All of the native systemd facilities are started via systemd units.  If
systemd is not running as PID 1, nothing is going to load or act on the
systemd units.  Therefore, nothing will start those services.

This may be a concern in some future in which there are multiple init
systems that interpret and act on systemd units, but this is not yet the
case.  That will be an integration worry that we'll have to tackle should
that be the case in the future, but I don't think it's a concern right
now.

> Again if I recall correctly, there were some behaviors which arose from
> having logind running in a non-systemd-as-init-system environment which
> the maintainers did not consider something which they would need to fix
> but which I found undesirable.

Possibly you're thinking of the problem that Sam pointed out, which is
that the systemd package depends on libsystemd0 which currently
effectively conflicts with elogind, and therefore you can't install
elogind and the systemd package simultaneously?

If so, that indeed is true but is a problem that probably has to be fixed
anyway, regardless of the current discussion, for elogind to be easily
usable in Debian.

This is a problem with package dependencies, though.  Installing the
systemd package should not cause systemd-logind to run.  It is started via
/lib/systemd/system/systemd-logind.service, which will be ignored unless
systemd is running as PID 1.

I would welcome corrections of I'm wrong, though!

> I don't want to "risk" (in quotation marks because there may not be
> much, if any, actual risk involved) my primary system on testing this,
> and right now I don't have any spare systems or a working virtualization
> environment (because I haven't been able to get libvirt, as packaged for
> Debian, to work properly in a non-systemd environment) to use for
> testing, so I'm not in a position to do that test myself. I do have a
> functioning virtualization environment at my workplace, so if downtime
> permits over the next week or three, I may be able to do that there.

Totally understood, and obviously you're under no obligation to do the
testing!

> (Personally, I'd argue that splitting the various daemons and non-daemon
> tools out into packages according to which ones depend on which others
> makes sense purely from a "granularity of dependencies" perspective, but
> it's been clear for a long time that that argument is a nonstarter with
> the systemd maintainers.)

The systemd maintainers do split out some binaries, but I don't think
creating 20-odd packages for each individual small service (systemd has a
general model of breaking the boot up into small, simple binaries that do
a single thing) is necessarily an improvement.  My understanding of the
preference of the systemd maintainers is to not split the package except
where there's a clear benefit (in terms of dependency structure or some
other problem) that outweighs the ongoing cost of maintaining more
individual packages.  Here, if the systemd package works the way that I
believe that it does, I don't think splitting will change anything other
than saving a small amount of disk space on non-systemd systems.

(Splitting doesn't avoid the library dependency problem that currently
causes problems with elogind, since I believe the programs in question
also depend on that shared library.)

-- 
Russ Allbery (r...@debian.org)  

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[The Wanderer]

(Here's another one I may later wind up regretting; I'm out on too many
if-I-remember and if-I-understand limbs, and am thus speaking based on
too many possibly-false premises, and I'm also now explaining personal
preferences which might readily be deemed too minor or too niche to be
reasonable to bother supporting. I don't want to just drop the
conversation midstream at this point, however.)

On 2020-01-03 at 14:36, Russ Allbery wrote:

> The Wanderer  writes:
> 
>> On 2020-01-03 at 13:35, Russ Allbery wrote:

>>> Why would that be necessary?
> 
>> Which part? The splitting out, or the accepting of the init
>> scripts?
> 
> Both.
> 
>> The splitting out I think I've already addressed, if not directly
>> or at length; I can go over it more specifically if desired,
>> although I sometimes have a hard time phrasing things in that
>> context in ways which aren't unintentionally provocative. The short
>> version would be: to avoid undesirable side effects from other
>> parts of the systemd package. (Again, it's been long enough that I
>> forget what those side effects were, although I think they had to
>> do with logind.)
> 
> So far as I can tell, installing the systemd package by itself
> shouldn't do anything.  It's possible that I'm wrong, but if so, it
> might be easier to just fix that problem rather than splitting out
> binaries.  It's also possible that you ran into some long-ago-fixed
> bug and there is no longer any problem with installing the systemd
> package on a system that isn't running systemd.

If I recall and understand correctly, installing systemd-the-package
will result in at least some of the daemons therein - including both
systemd itself, and systemd-logind - being set up to run automatically
in the correct contexts (whether by scripted invocation set up by a
package, or by socket activation, or by some other means), in such a way
that they will generally wind up running even on a computer where
systemd-the-daemon is not the init system. I'm currently digging through
the result of 'apt-get source systemd', and I haven't yet managed to
either confirm or refute this with certainty.

If that's not true - if having that package installed will (now) never
lead those daemons to be run without something external to the package
which triggers them to get run, and a non-systemd-as-init-system machine
will (now) not involve anything which triggers running them unless the
sysadmin specifically sets it up that way - then I'm on entirely the
wrong track here, and unless there's something else I'm not remembering,
my split-the-package objections almost certainly disappear. (Although it
would probably then become harder to ensure that I don't install
something which is going to trigger running any of them.)

Again if I recall correctly, there were some behaviors which arose from
having logind running in a non-systemd-as-init-system environment which
the maintainers did not consider something which they would need to fix
but which I found undesirable. Unfortunately, I have largely forgotten
what they were; I have a vague memory of something about
normal-operation status text stepping all over the text console at which
I log in and from which I startx, and that it may have been related to
the fact that I was using systemd-shim rather than
systemd-the-init-system and so the setup was considered unsupported by
the systemd maintainers, but I'm not even certain that that was the same
problem. (Of course, systemd-shim isn't even an option anymore.)

I thus want to avoid letting logind run, and as far as I can tell, doing
that without breaking other things means not letting systemd-the-package
get installed - because even if I disable logind post-install by some
mechanism, other things may have chosen to depend on systemd-the-package
because they need logind, and may quite reasonably assume that the
presence of the former means that the latter will be available for them
to use.

Given those experiences, I also have reservations about letting any
other systemd-related daemons run without specific reason, just because
I don't know what side effects they might cause - now or in the future.
It's not as nearly absolute an objection, but as indicated in my first
message, I don't trust systemd-the-project not to introduce behaviors
which I find undesirable.

> I don't think there's enough information on this thread to indicate
> that there's any need to split out the package.  Probably it would
> make sense for someone to just try installing that package on a
> non-systemd system and running systemd-tmpfiles and systemd-sysusers
> and see if anything breaks.

Given what Andrej Shadura wrote in reply to me this morning, I'd be
surprised if anything did break, at least anything related to tmpfiles
or sysusers.

I don't want to "risk" (in quotation marks because there may not be
much, if any, actual risk involved) my primary system on testing this,
and right now I don't have any spare systems or a working virtualizatio

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Marco d'Itri]

On Jan 03, Thomas Goirand  wrote:

> Could you please, therefore, tell me what feature is missing? If you
If I am not mistaken then you started arguing that we should consider an 
hypothetical alternative implementation with missing features, so maybe 
you should explain what may be missing.

> > Also, you have still not explained why we would need another 
> > implementation of these programs, except that "it's not systemd".
> Because with it, we allow running something else than systemd like for
> example OpenRC or sysv-rc, but not only: even maybe without an init
> system at all (think: minimal chroot, containers, etc.) we could
As many have already explained, there is no reason why the systemd 
programs cannot be used with other init systems.
The systemd maintainers themselves are interested in ways to make 
init-less containers lighter.

> > We systemd package maintainers have discussed using alternatives for 
> > these programs and I think that we have a strong consensus that this is 
> > not what alternatives are for: if another implementation is actually 
> > needed then it should conflict+provide something.
> Why is this the case with systemd's tmpfiles and sysusers, but it's not
> the same for gawk vs mawk? Since when in Debian users can't choose
Because these are two established implementations with different 
extensions and you can have multiple packages installed at the same time 
that use features of either of them.

> At this moment in time, I don't even know if open{tmpfiles,sysusers} are
> even useful. I'm politely asking you to either help me find out, or just
> do nothing (this includes: stop giving bad feedback).
Actually you are asking other maintainers to do things, so we share our 
opinions about them.

-- 
ciao,
Marco


signature.asc
Description: PGP signature

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Russ Allbery]

Thomas Goirand  writes:

> Yes, there's drawbacks in general. However, you *cannot* just say, we're
> going to use the systemd implementation "just because it's the
> refrence", without even giving me some space to at least *TRY* the
> alternative, to see if it's valuable or not.

Yes, I agree.

> As I wrote earlier, there's an easy path out: drop the systemd
> implementation entirely, and standardize on open{tmpfiles,sysusers}
> implementation.

This certainly doesn't seem easy to me.  It sounds like a lot more
integration work and pain, would involve replacing something that's
already working well, and would require proof that implementations written
in shell, which as a programming language is unsafe around file system
edge cases without extreme caution, handles the numerous security concerns
that the systemd implementations have been hardened against.

However, I completely agree that there's no reason not to try out the
packages and take a closer look, and even if all the things I'm concerned
about are true, they may still have a valuable role in our ecosystem for
support of non-Linux kernels.

> But maybe we should first *try* open{tmpfiles,sysusers} to see if it has
> any value.

Yes.  I agree with this.

-- 
Russ Allbery (r...@debian.org)  

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Thomas Goirand]

On 1/3/20 7:15 PM, Marco d'Itri wrote:
> On Jan 03, Thomas Goirand  wrote:
> 
>> That's where I don't agree. While it's nice to have such a declarative
>> system, I don't think it's reasonable to impose the implementation of
>> any change to systemd to all the other init systems.
> I do. Good luck persuading the consumers of this API that they should 
> not use some features because you did not implement them in your own 
> alternative package.

You seem to know very well the alternative implementations that we're
talking about. At least more than I do.

Could you please, therefore, tell me what feature is missing? If you
aren't able to point at it, then you're just doing FUD/trolling, instead
of letting me *try* the alternatives, and therefore, I'm politely asking
you to stop doing this, and give me some time for doing the work. If you
can tell what's wrong in these implementations, then that's super nice
of you to take the time to investigate with me and share, thanks for
doing it.

> Also, you have still not explained why we would need another 
> implementation of these programs, except that "it's not systemd".

Because with it, we allow running something else than systemd like for
example OpenRC or sysv-rc, but not only: even maybe without an init
system at all (think: minimal chroot, containers, etc.) we could
continue to use the sysusers and tmpfiles light implementations. But
that's not my main goal.

The main goal is: I also like declarative things like sysusers and
tmpfiles, as they are less error-prone. By having an implementation that
can work everywhere, with or without systemd, using sysusers and
tmpfiles becomes non-controversial, and we can standardize on it (and
even write it in the stones of the policy). I'm trying to facilitate
using these 2 systemd facilities because they are nice (especially the
sysusers one).

> We systemd package maintainers have discussed using alternatives for 
> these programs and I think that we have a strong consensus that this is 
> not what alternatives are for: if another implementation is actually 
> needed then it should conflict+provide something.
Why is this the case with systemd's tmpfiles and sysusers, but it's not
the same for gawk vs mawk? Since when in Debian users can't choose
alternative implementation? Oh, btw, "we support exploring alternatives"
is the winning option... That's precisely what I'm trying, though I'm
only getting bad feedback from the systemd fanboy team, instead of help
to do exactly what all the DDs have voted for (ie: exploring alternatives).

At this moment in time, I don't even know if open{tmpfiles,sysusers} are
even useful. I'm politely asking you to either help me find out, or just
do nothing (this includes: stop giving bad feedback).

>> You are being obviously biased toward systemd here. Just try to think a
> Indeed: obviously, most people actually do not mind using systemd...

As in, you think I don't? You are wrong. Clearly, you fail to see what
my motivations are. I am a systemd user, and I enjoy it. There's many
things I like about systemd, though that's probably off-topic.

Thomas Goirand (zigo)

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Marco d'Itri]

On Jan 03, Thomas Goirand  wrote:

> Yes, there's drawbacks in general. However, you *cannot* just say, we're
> going to use the systemd implementation "just because it's the
> refrence", without even giving me some space to at least *TRY* the
As usual this is about much more than you, and I am quite concerned that 
you are apparently oblivious to that.
You started by arguing that Debian should adopt/invent a custom API not 
used by any other distribution and proposed that the systemd maintainers 
work on adopting alternatives for something which the alternatives 
system was not designed for.
Since your proposal requires that other people do significant work then 
it should not be unexpected that you are being asked to justify its
usefulness.

> implementation. But maybe we should first *try* open{tmpfiles,sysusers}
> to see if it has any value. At the present moment, I just don't know yet
> if it can be a good or a bad replacement.
Maybe *you* should try that and then report back your findings.

-- 
ciao,
Marco


signature.asc
Description: PGP signature

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Thomas Goirand]

On 1/3/20 7:31 PM, Russ Allbery wrote:
> The guidance of option B is that we are committing to reviewing and
> working collaboratively with anyone who wants to support alternate init
> systems, but that implementation strategy is subject to technical review.

It reads:

"However, Debian remains an environment where developers and users can
explore and develop alternate init systems and alternatives to systemd
features."

Exploring alternatives is precisely what I'm doing. The same way I've
felt harassed when I did the original porting of OpenRC to 1/ Debian,
then 2/ kFreeBSD, then 3/ Hurd, I do feel like everyone is rushing into
me for no reasons. Could you please stop doing that, both you and Sam,
and instead, help me with deciding how both implementation can cohexist?

> I think Sam is providing that technical review by pointing out the
> drawbacks of using multiple competing implementations. That's a valid
> point of technical discussion; you may disagree with him, but I think that
> discussion is explicitly enabled by the GR result.

Yes, there's drawbacks in general. However, you *cannot* just say, we're
going to use the systemd implementation "just because it's the
refrence", without even giving me some space to at least *TRY* the
alternative, to see if it's valuable or not. At the moment, I only saw
FUD in this thread, nobody was able to point at missing feature or wrong
implementation. This is very annoying and frustrating.

I very much don't agree that the GR result tells that we should use
absolutely all implementation from systemd as the default, no mater
what, and that there's no room for alternative implementations. This was
proposal "Choice 1: F: Focus on systemd", and that's not the winning
option. The project has chosen "Systemd but we support exploring
alternatives", so please let me do just that, and give a chance to the
alternative to maybe win. Otherwise, I just give up as well, and do
something else (believe me, I do have a lot of things in my todo list to
improve Debian...:P ).

> That's a lot of work, and with my Policy hat on, I'm not committing to
> doing that work because I see the GR as asking me to spend my limited
> resources on other things, and also asking that we move a little bit more
> decisively in the direction of adopting systemd facilities than that
> (albeit not as decisively as option F).

Adopting systemd facilities is *NOT* the same as adopting systemd
implementation blindly, no mater what it is and does. That's not what
"Choice 2: B: Systemd but we support exploring alternatives" is about.

> Therefore, I think it's a reasonable question to ask whether we want, as a
> project, to commit to supporting the least common denominator between
> several competing implementations of these facilities, or instead ask that
> people arrange to run the systemd implementation so that we have the same
> features everywhere.
> 
> Support for kFreeBSD and Hurd is obviously a valid argument in favor of
> some level of support for non-systemd implementations.

As I wrote earlier, there's an easy path out: drop the systemd
implementation entirely, and standardize on open{tmpfiles,sysusers}
implementation. But maybe we should first *try* open{tmpfiles,sysusers}
to see if it has any value. At the present moment, I just don't know yet
if it can be a good or a bad replacement.

Again, *PLEASE*, let me at least try to have a working package, that is
somehow useful. We're not there yet at all, there's not even an init
script, or a way to replace systemd-X binaries.

Cheers,

Thomas Goirand (zigo)

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Bastian Blank]

On Fri, Jan 03, 2020 at 02:29:37PM -0500, The Wanderer wrote:
> The accepting of init scripts seemed to me like an essential piece of
> making sure those scripts would be present wherever they would be
> needed. Your suggestion above seems to provide a way to make it less
> essential, and thus would make moving forward easier and more practical.
> The only question is how to make sure that that other package would be
> present whenever a non-systemd init system is in use, and that seems
> like a simple matter of adding dependencies from the
> set-foo-as-active-init-system package for foo.

Those init-script would life in the package for openrc or sysvinit or
whatever you are just working on, because it is implementation to them.

Or you use the same systemd-service to own-format conversion, you need
anyway for the whole rest, and don't need to implement extra scripts at
all.

Bastian

-- 
There is an order of things in this universe.
-- Apollo, "Who Mourns for Adonais?" stardate 3468.1

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Sam Hartman]

> "Russ" == Russ Allbery  writes:

Russ> So far as I can tell, installing the systemd package by itself 
shouldn't
Russ> do anything.  It's possible that I'm wrong, but if so, it might be 
easier
Russ> to just fix that problem rather than splitting out binaries.  It's 
also
Russ> any problem with installing the systemd package on a system that isn't
Russ> running systemd.

Well, systemd does depend on libsystemd0 .
libelogind0 provides/replaces lybsystemd0 and conflicts with systemd.

So, it is not currently possible to get systemd-tmpfiles and elogind
installed on the same system.
Which clearly needs to get fixed somehow if we're going to use
systemd-tmpfiles everywhere we can.

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Russ Allbery]

The Wanderer  writes:
> On 2020-01-03 at 13:35, Russ Allbery wrote:
>> The Wanderer  writes:

>>> If the maintainers of systemd-the-package would be willing to not only
>>> split out these binaries into standalone package(s), but accept such
>>> init scripts for inclusion in those packages,

>> Why would that be necessary?

> Which part? The splitting out, or the accepting of the init scripts?

Both.

> The splitting out I think I've already addressed, if not directly or at
> length; I can go over it more specifically if desired, although I
> sometimes have a hard time phrasing things in that context in ways which
> aren't unintentionally provocative. The short version would be: to avoid
> undesirable side effects from other parts of the systemd package.
> (Again, it's been long enough that I forget what those side effects
> were, although I think they had to do with logind.)

So far as I can tell, installing the systemd package by itself shouldn't
do anything.  It's possible that I'm wrong, but if so, it might be easier
to just fix that problem rather than splitting out binaries.  It's also
possible that you ran into some long-ago-fixed bug and there is no longer
any problem with installing the systemd package on a system that isn't
running systemd.

I don't think there's enough information on this thread to indicate that
there's any need to split out the package.  Probably it would make sense
for someone to just try installing that package on a non-systemd system
and running systemd-tmpfiles and systemd-sysusers and see if anything
breaks.

> The accepting of init scripts seemed to me like an essential piece of
> making sure those scripts would be present wherever they would be
> needed. Your suggestion above seems to provide a way to make it less
> essential, and thus would make moving forward easier and more practical.
> The only question is how to make sure that that other package would be
> present whenever a non-systemd init system is in use, and that seems
> like a simple matter of adding dependencies from the
> set-foo-as-active-init-system package for foo.

Yeah, that was my thought.  It also keeps the infrastructure for
supporting other init systems in its own packages maintained by people who
use those init systems, which seems likely to improve the quality of
maintenance and shorten the loop on bug fixes.

-- 
Russ Allbery (r...@debian.org)  

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Bastian Blank]

On Fri, Jan 03, 2020 at 10:31:53AM -0800, Russ Allbery wrote:
> Support for kFreeBSD and Hurd is obviously a valid argument in favor of
> some level of support for non-systemd implementations.

But then there is the question on how much work it would be to port the
**systemd** implementations to FreeBSD or Hurd, if using and maintaining
those bash re-implementations is futile.

I tried to build systemd-sysusers against the FreeBSD libc, which
obviously is not going to be easy.  But all the Debian arches use glibc
and problems I found where in common, but unused, code and obviously
pretty glibc specific stuff.

Did someone already try to build just those binaries e.g. on Hurd?

Regards,
Bastian

PS: Why the hack is alloca in alloca.h on glibc, but stdlib.h on BSD?
PPS: Yes, systemd uses alloca.
-- 
Is truth not truth for all?
-- Natira, "For the World is Hollow and I have Touched
   the Sky", stardate 5476.4.

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[The Wanderer]

On 2020-01-03 at 13:35, Russ Allbery wrote:

> The Wanderer  writes:
> 
>> Unless my understanding of the architecture of
>> systemd-the-init-system is entirely incorrect, running these
>> .service'es is handled by /bin/systemd. If having these programs
>> run at boot time is considered essential to full functionality of
>> these facilities - and I'd be surprised if it wasn't - then
>> something is going to have to be done to permit that to happen
>> under other init systems.
> 
> So create a package with init scripts (or runit scripts, or OpenRC 
> scripts) that run those binaries, and make that package depend on
> systemd (*not* systemd-sysv).
> 
> This seems fairly simple to me.  Am I missing something (beyond the
> fact that this is problematic on the Hurd and kFreeBSD)?

Not as far as I can tell; the idea just hadn't occurred to me.

>> If the maintainers of systemd-the-package would be willing to not
>> only split out these binaries into standalone package(s), but
>> accept such init scripts for inclusion in those packages,
> 
> Why would that be necessary?

Which part? The splitting out, or the accepting of the init scripts?

The splitting out I think I've already addressed, if not directly or at
length; I can go over it more specifically if desired, although I
sometimes have a hard time phrasing things in that context in ways which
aren't unintentionally provocative. The short version would be: to avoid
undesirable side effects from other parts of the systemd package.
(Again, it's been long enough that I forget what those side effects
were, although I think they had to do with logind.)

The accepting of init scripts seemed to me like an essential piece of
making sure those scripts would be present wherever they would be
needed. Your suggestion above seems to provide a way to make it less
essential, and thus would make moving forward easier and more practical.
The only question is how to make sure that that other package would be
present whenever a non-systemd init system is in use, and that seems
like a simple matter of adding dependencies from the
set-foo-as-active-init-system package for foo.

-- 
   The Wanderer

The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man. -- George Bernard Shaw



signature.asc
Description: OpenPGP digital signature

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Sam Hartman]

> "Marco" == Marco d'Itri  writes:


>> You are being obviously biased toward systemd here. Just try to think a
Marco> Indeed: obviously, most people actually do not mind using systemd...

>> 2nd time: the same way, what if opentmpfiles implements a new feature
>> that is *not* in systemd implementation? There, we have an exit
Marco> What if a tree falls in a forest and no one is around to hear it?

Marco, I think that the two sentences I quoted from you, especially in
the context of other statements you've made are disrespectful of people
who disagree with you to the extent that they are not appropriate here.

The project just concluded a vote.  The winning option was specifically
about facilitating the sort of innovation Thomas asks about above.

You may think it is unlikely.
But there are perfectly good answers to talk about how to promote that
innovation without making things harder for systemd users or being rude.
I'd already given some; others also had already joined in.

The discussion was not advanced by you antagonizing Thomas.
Please stop.

--Sam

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Russ Allbery]

The Wanderer  writes:

> Unless my understanding of the architecture of systemd-the-init-system
> is entirely incorrect, running these .service'es is handled by
> /bin/systemd. If having these programs run at boot time is considered
> essential to full functionality of these facilities - and I'd be
> surprised if it wasn't - then something is going to have to be done to
> permit that to happen under other init systems.

So create a package with init scripts (or runit scripts, or OpenRC
scripts) that run those binaries, and make that package depend on systemd
(*not* systemd-sysv).

This seems fairly simple to me.  Am I missing something (beyond the fact
that this is problematic on the Hurd and kFreeBSD)?

> If the maintainers of systemd-the-package would be willing to not only
> split out these binaries into standalone package(s), but accept such
> init scripts for inclusion in those packages,

Why would that be necessary?

-- 
Russ Allbery (r...@debian.org)  

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Russ Allbery]

Thomas Goirand  writes:

> I think you and many others should be extremely careful when talking
> about proposal B just as if it was a clear winner of the poll. If you
> are then discarding the opinion of everyone else who didn't want it as
> the winning option, and not consider the GR result as a whole, then you
> are clearly dismissing the opinion of a large amount of DDs (even though
> they were not the majority).

I'm sorry to push a little bit on this, because I understand that many
people are upset and frustrated by the outcome of the vote, but I am also
concerned about the project falling into a different one of its very
long-standing problems: being unwilling to make a decision.

We have a constitutional mechanism to make a decision.  That mechanism
produced a decision.  We should move forward with that decision.  We don't
need to do so in a rush, and we should do so respectfully and carefully,
but as important as it is to the project to be welcoming and respectful of
the people who are unhappy with the outcome of a decision, it's also
important for the project to be able to *make decisions* and then follow
through on them.

Otherwise, we're going to still be having the same discussion in another
year, and by that point we'll be having it with fewer people because there
are very few things as demotivating and draining as being endlessly
trapped in the same cycle of argument.  For me at least, that's even worse
than losing an argument.  (Option B was my fourth pick in the vote, for
the record.)

> What I'm trying to do here, is to enable a middle ground where we have a
> common interface everywhere, with the possibility to implement things
> differently.  Just saying "systemd systemd systemd" many times, imposing
> it as the only reference and winner, where it should be enabled
> everywhere, and for absolutely all of its interfaces, will lead to
> nowhere. And that's *not* what the proposal B was about.

The guidance of option B is that we are committing to reviewing and
working collaboratively with anyone who wants to support alternate init
systems, but that implementation strategy is subject to technical review.
I think Sam is providing that technical review by pointing out the
drawbacks of using multiple competing implementations.  That's a valid
point of technical discussion; you may disagree with him, but I think that
discussion is explicitly enabled by the GR result.

There were two options on the ballot that called for standardizing
interfaces of systemd facilities that we were going to adopt so that other
implementations could implement only the Policy-defined functionality and
not other features.  For better or worse, those options did not win.
That's a lot of work, and with my Policy hat on, I'm not committing to
doing that work because I see the GR as asking me to spend my limited
resources on other things, and also asking that we move a little bit more
decisively in the direction of adopting systemd facilities than that
(albeit not as decisively as option F).

Therefore, I think it's a reasonable question to ask whether we want, as a
project, to commit to supporting the least common denominator between
several competing implementations of these facilities, or instead ask that
people arrange to run the systemd implementation so that we have the same
features everywhere.

Support for kFreeBSD and Hurd is obviously a valid argument in favor of
some level of support for non-systemd implementations.

-- 
Russ Allbery (r...@debian.org)  

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Marco d'Itri]

On Jan 03, Thomas Goirand  wrote:

> That's where I don't agree. While it's nice to have such a declarative
> system, I don't think it's reasonable to impose the implementation of
> any change to systemd to all the other init systems.
I do. Good luck persuading the consumers of this API that they should 
not use some features because you did not implement them in your own 
alternative package.

Also, you have still not explained why we would need another 
implementation of these programs, except that "it's not systemd".

We systemd package maintainers have discussed using alternatives for 
these programs and I think that we have a strong consensus that this is 
not what alternatives are for: if another implementation is actually 
needed then it should conflict+provide something.

> You are being obviously biased toward systemd here. Just try to think a
Indeed: obviously, most people actually do not mind using systemd...

> 2nd time: the same way, what if opentmpfiles implements a new feature
> that is *not* in systemd implementation? There, we have an exit
What if a tree falls in a forest and no one is around to hear it?

> Similarly, the way
> systemd is trying to take over DNS resolving and NTP in an "integrated"
I just want to clarify that it is not...

-- 
ciao,
Marco


signature.asc
Description: PGP signature

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Thomas Goirand]

On 1/3/20 2:23 PM, Sam Hartman wrote:
> Proposal B is about letting you innovate (for example  doing your own
> implementation of tmpfiles that you opt into on a per-package basis) and
> doing the integration work for alternatives like elogind where you
> cannot use the systemd interface.
> 
> --Sam

Sam,

I think you and many others should be extremely careful when talking
about proposal B just as if it was a clear winner of the poll. If you
are then discarding the opinion of everyone else who didn't want it as
the winning option, and not consider the GR result as a whole, then you
are clearly dismissing the opinion of a large amount of DDs (even though
they were not the majority).

What I'm trying to do here, is to enable a middle ground where we have a
common interface everywhere, with the possibility to implement things
differently. Just saying "systemd systemd systemd" many times, imposing
it as the only reference and winner, where it should be enabled
everywhere, and for absolutely all of its interfaces, will lead to
nowhere. And that's *not* what the proposal B was about.

Cheers,

Thomas Goirand (zigo)

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[The Wanderer]

On 2020-01-03 at 09:13, Ansgar wrote:

> The Wanderer writes:
> 
>> However, as it remains the case that they are shipped in the same 
>> package as /bin/systemd, and as I gather (mostly from this thread,
>> I think) that some of the ways they are expected to be invoked
>> probably rely on having systemd-the-daemon running
> 
> They don't rely on the `/bin/systemd` binary at all, and I'm fairly 
> certain that this was mentioned in the thread (and previous) and not
> the opposite.
> 
> There was something about dh_installsystemd currently only running 
> systemd-tmpfiles in maintainer scripts if systemd run, but that can
> be changed should Debian choose to use tmpfiles for more generic
> purposes.

It was Simon McVittie's message from half-past-noon (EST) yesterday,
citing the various ways in which these facilities are invoked
automatically on a systemd-as-PID1 system, that I was thinking of.
Specifically, this part:

>>> - run systemd-tmpfiles during boot (this is systemd-tmpfiles-setup.service,
>>>   part of systemd)
>>> 
>>> - run systemd-sysusers during boot (this is
>>>   systemd-sysusers.service, part of systemd)

Unless my understanding of the architecture of systemd-the-init-system
is entirely incorrect, running these .service'es is handled by
/bin/systemd. If having these programs run at boot time is considered
essential to full functionality of these facilities - and I'd be
surprised if it wasn't - then something is going to have to be done to
permit that to happen under other init systems.

That is, of course, equally true for any non-systemd implementations of
these same facilities; as Simon also indicated, LSB init scripts or
similar to invoke these facilities at boot time will be needed for
opentmpfiles and opensysusers.

If the maintainers of systemd-the-package would be willing to not only
split out these binaries into standalone package(s), but accept such
init scripts for inclusion in those packages, then I think that should
entirely eliminate this particular concern; it should then be possible
for other packages to depend on these facilities directly, without side
effects for non-systemd environments.

However, I don't see any particular reason to expect that to be the
case, and certainly if it is not raised as a thing to be requested it is
less likely to happen.

Using opentmpfiles and opensysusers would make it possible to include
those init scripts only in those packages, and avoid needing to have
them on systemd-as-PID1 computers where they will never be used. That's
not a major advantage, any more than avoiding .service files et cetera
on computers without systemd is, but may not be of negligible weight
either.

>> this does not entirely obviate my concerns related to needing to
>> have systemd-the-package's daemons present in order to gain access
>> to these facilities.
> 
> I'm happy to have helped overcome these concerns.

Unfortunately, due to the above, the concerns remain.

-- 
   The Wanderer

The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man. -- George Bernard Shaw



signature.asc
Description: OpenPGP digital signature

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[The Wanderer]

On 2020-01-03 at 07:50, Ansgar wrote:

> The Wanderer writes:
> 
>> On 2020-01-02 at 08:14, Thomas Goirand wrote:
>> 
>>> As I wrote, no need to complain, but act.
>>> 
>>> https://salsa.debian.org/debian/opentmpfiles
>> 
>> For me (with no salsa account, therefore not logged in; I don't
>> know if that makes any difference), this page states "The
>> repository for this project is empty".
> 
> The repository has only tags, but no branches. This seems to confuse 
> GitLab's web interface, but one can still clone the repository and 
> checkout tags. (Probably Thomas forgot to push the branches to
> GitLab.)
> 
> Alternatively the upstream repository without the Debian packaging
> bits can be found at [1] (might be a mirror, not sure).
> 
> Ansgar
> 
>   [1]: https://github.com/OpenRC/opentmpfiles

Thanks for the explanation!

-- 
   The Wanderer

The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man. -- George Bernard Shaw



signature.asc
Description: OpenPGP digital signature

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Ansgar]

The Wanderer writes:
> On 2020-01-02 at 08:14, Thomas Goirand wrote:
>> As I wrote, no need to complain, but act.
>> 
>> https://salsa.debian.org/debian/opentmpfiles
>
> For me (with no salsa account, therefore not logged in; I don't know if
> that makes any difference), this page states "The repository for this
> project is empty".

The repository has only tags, but no branches. This seems to confuse
GitLab's web interface, but one can still clone the repository and
checkout tags. (Probably Thomas forgot to push the branches to GitLab.)

Alternatively the upstream repository without the Debian packaging bits
can be found at [1] (might be a mirror, not sure).

Ansgar

  [1]: https://github.com/OpenRC/opentmpfiles

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Ansgar]

The Wanderer writes:
> However, as it remains the case that they are shipped in the same
> package as /bin/systemd, and as I gather (mostly from this thread, I
> think) that some of the ways they are expected to be invoked probably
> rely on having systemd-the-daemon running

They don't rely on the `/bin/systemd` binary at all, and I'm fairly
certain that this was mentioned in the thread (and previous) and not the
opposite.

There was something about dh_installsystemd currently only running
systemd-tmpfiles in maintainer scripts if systemd run, but that can be
changed should Debian choose to use tmpfiles for more generic purposes.

> this does not entirely
> obviate my concerns related to needing to have systemd-the-package's
> daemons present in order to gain access to these facilities.

I'm happy to have helped overcome these concerns.

Ansgar

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[The Wanderer]

On 2020-01-03 at 08:50, Andrej Shadura wrote:

> Hi,
> 
> On Fri, 3 Jan 2020 at 14:02, The Wanderer 
> wrote:
> 
>> On 2020-01-02 at 09:03, Sam Hartman wrote:
>> 
>>> My understanding is that systemd's implementation of tmpfiles
>>> and sysusers works even while systemd is not pid 1. Why do we
>>> need multiple implementations for Debian ports where systemd
>>> runs?
>> 
>> There are those who don't run systemd-the-daemon even as non-PID-1;
>> I'm one of them.
>> 
>> In my case, this is partly due to half-remembered
>> negative-impression behavior changes seen from even non-PID-1
>> systemd, back when I experimented with it around the time of the
>> default transition - but I can't remember those changes clearly
>> enough to specify, and it's possible that even if they did exist
>> back then they might no longer be present today. (For what it's
>> worth, my recollection is that they were related to logind.)
> 
> I believe there’s significant misunderstanding here.
> 
> Unless I’m mistaken, systemd-tmpfiles and systemd-sysusers not only
> don’t require systemd to run as PID 1, but they don’t require
> systemd to run at all. In fact, they don’t seem to require the
> /bin/systemd binary to be installed. They do use libsystemd-shared.so
> because they need e.g. bits of the string manipulation library, but
> that’s it.

I am not particularly surprised by this, and am pleased to learn that it
is the case.

However, as it remains the case that they are shipped in the same
package as /bin/systemd, and as I gather (mostly from this thread, I
think) that some of the ways they are expected to be invoked probably
rely on having systemd-the-daemon running, this does not entirely
obviate my concerns related to needing to have systemd-the-package's
daemons present in order to gain access to these facilities.

If these two programs were to be split out into one or more separate
packages (which IMO would probably also be a good idea for some of the
other binaries shipped in systemd-the-package, but that's another
discussion), and those packages did not depend on the systemd package,
that would greatly improve the situation from my perspective.

-- 
   The Wanderer

The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man. -- George Bernard Shaw



signature.asc
Description: OpenPGP digital signature

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Andrej Shadura]

Hi,

On Fri, 3 Jan 2020 at 14:02, The Wanderer  wrote:
> On 2020-01-02 at 09:03, Sam Hartman wrote:
> > My understanding is that systemd's implementation of tmpfiles and
> > sysusers works even while systemd is not pid 1. Why do we need
> > multiple implementations for Debian ports where systemd runs?
> There are those who don't run systemd-the-daemon even as non-PID-1; I'm
> one of them.
>
> In my case, this is partly due to half-remembered negative-impression
> behavior changes seen from even non-PID-1 systemd, back when I
> experimented with it around the time of the default transition - but I
> can't remember those changes clearly enough to specify, and it's
> possible that even if they did exist back then they might no longer be
> present today. (For what it's worth, my recollection is that they were
> related to logind.)

I believe there’s significant misunderstanding here.

Unless I’m mistaken, systemd-tmpfiles and systemd-sysusers not only
don’t require systemd to run as PID 1, but they don’t require systemd
to run at all. In fact, they don’t seem to require the /bin/systemd
binary to be installed. They do use libsystemd-shared.so because they
need e.g. bits of the string manipulation library, but that’s it.

-- 
Cheers,
  Andrej

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Sam Hartman]

> "Thomas" == Thomas Goirand  writes:

Thomas> On 1/3/20 2:33 AM, Sam Hartman wrote:
>> Secondly, by using systemd-tmpfiles when we can, we gain support for any
>> additional features that are implemented.

Thomas> That's where I don't agree. While it's nice to have such a 
declarative
Thomas> system, I don't think it's reasonable to impose the implementation 
of
Thomas> any change to systemd to all the other init systems. At some point, 
*we*
Thomas> must be able to decide.

This is in fact our disagreement.

The thing is that upstream software is going to start using the new
systemd facilities as they become available.
There's a cost to not having them available in Debian  when they are
available.

I think I would handle two cases differently:

1) Enabling the systemd tmpfiles facility for alternate inits:

* use systemd where possible

2) Experimenting with a better tmpfiles interface in hopes of some day
supplanting the systemd interface:

* I think you want to opt into that new interface on a per-package
  rather than a per-system level.

Opting in on a per-system level restricts both your ability to innovate
and systemd's ability to innovate.

Holding us back to the LCD of a systemd implementation and some
alternative implementation does not seem to be in the spirit of Proposal
B in my mind.  That seems much more consistent with Proposal D and
sticking a stable version of such an interface in policy.

Proposal B is about letting you innovate (for example  doing your own
implementation of tmpfiles that you opt into on a per-package basis) and
doing the integration work for alternatives like elogind where you
cannot use the systemd interface.

--Sam

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[The Wanderer]

On 2020-01-02 at 09:03, Sam Hartman wrote:

> My understanding is that systemd's implementation of tmpfiles and 
> sysusers works even while systemd is not pid 1. Why do we need
> multiple implementations for Debian ports where systemd runs?

There are those who don't run systemd-the-daemon even as non-PID-1; I'm
one of them.

In my case, this is partly due to half-remembered negative-impression
behavior changes seen from even non-PID-1 systemd, back when I
experimented with it around the time of the default transition - but I
can't remember those changes clearly enough to specify, and it's
possible that even if they did exist back then they might no longer be
present today. (For what it's worth, my recollection is that they were
related to logind.)

But it's also partly due to a philosophical objection to the way systemd
implements so many things all in more-or-less one place, and to distrust
that systemd-the-daemon (and/or the package) won't later grow behaviors
which I find undesirable, but which are enabled by virtue of having
systemd-the-daemon running at all.

I don't go so far as to prohibit even libsystemd0 (never mind udev) from
being installed, although I understand that some people do - but I
simply do not trust systemd-the-project not to bring undesirable
changes, and consequently I want to have as little running systemd code
on my system as I can reasonably get away with.

If it were possible to install and make use of systemd-the-suite's
implementations of tmpfiles and sysusers without needing to pull in
systemd-the-daemon - or, perhaps even more importantly, the *ten* other
apparent daemons which I see in a quick look at systemd-the-package - I
might well be OK with that. But from my understanding, it is not, and
certainly they are not packaged separately in Debian.

If there were no sufficient separate / independent / standalone
implementations of tmpfiles or sysusers, and packages which I can't do
without grew dependencies on those facilities, I'd probably (at least
temporarily) grit my teeth and bear with having to have
systemd-the-package and its daemons installed and active - and who
knows, maybe I'd even discover that the undesirable behaviors I
half-remember don't exist anymore. But as there apparently *are* such
implementations, I would greatly prefer to have the option to make use
of them instead.

A refusal to permit these alternate implementations to be available as
alternatives, for reasons other than technical limitations of one sort
or another, would seem to me like an attempt to shove systemd down my
throat - and would be that much more reason for me to finally give up on
Debian and try alternatives, whether Devuan or Gentoo or something
farther afield.

In that context, I find the entire premise of "why do we need multiple
implementations of this?" to be unfortunate; "need" is, IMO, the wrong
standard to use for this.

-- 
   The Wanderer (who will, as usual, probably regret posting this)

The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man. -- George Bernard Shaw



signature.asc
Description: OpenPGP digital signature

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[The Wanderer]

On 2020-01-02 at 08:14, Thomas Goirand wrote:

> As I wrote, no need to complain, but act.
> 
> https://salsa.debian.org/debian/opentmpfiles

For me (with no salsa account, therefore not logged in; I don't know if
that makes any difference), this page states "The repository for this
project is empty".

My default interpretation would be that maybe opentmpfiles doesn't exist
yet, and you just created the repo to be a place where people could get
started writing it. However, other people's comments seem to indicate
that opentmpfiles does exist (and is used in other distros), so that
doesn't fit.

Any idea what's going on here?

> https://salsa.debian.org/debian/opensysusers

This one shows the repository and its contents, however.

-- 
   The Wanderer

The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man. -- George Bernard Shaw



signature.asc
Description: OpenPGP digital signature

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Thomas Goirand]

On 1/3/20 2:33 AM, Sam Hartman wrote:
> Secondly, by using systemd-tmpfiles when we can, we gain support for any
> additional features that are implemented.

That's where I don't agree. While it's nice to have such a declarative
system, I don't think it's reasonable to impose the implementation of
any change to systemd to all the other init systems. At some point, *we*
must be able to decide.

> My recommendation would be to have one implementation of this interface
> per architecture; use the systemd one on architectures where it works.

My recommendation would be to be able to have some kind of CI to make
sure that both implementations are working the same way (obviously, I
will not have enough time available for this, unfortunately...). I don't
see why we would need to prefer one implementation over the other, if
they provide the same facility, there's no technical advantage to do so.
You are being obviously biased toward systemd here. Just try to think a
2nd time: the same way, what if opentmpfiles implements a new feature
that is *not* in systemd implementation? There, we have an exit
strategy: we can decide to stop using the systemd implementation
completely, since the alternative will work everywhere.

I'd like also to point to you that the alternatives are much easier to
maintain: they are written in shell script only, and are completely
independent from anything else. That's what a lot of people (including
myself) thought should have been done for systemd, and these 2
implementation really proves that it is possible. Similarly, the way
systemd is trying to take over DNS resolving and NTP in an "integrated"
way is the wrong way of doing things, and hopefully, Debian folks and
systemd lovers will be able to understand that fact (don't get me wrong:
understanding and recognizing that systemd should be more modular, and
broken better into individual *independent* pieces, is orthogonal to
deciding how to use the said systemd facility).

> It will not.
> I think another question is more interesting though.
> 
> Will having runtime selection of that interface harm Debian.
> 
> My answer is that I think it might:
> 
> * You might get the wrong version of tmpfiles implementation on a given
>   system.  The harm on a systemd system would be that you might not have
>   a feature available.

This would be a bug that would need to be addressed then. Just like any
other bugs.

> * You might be tempted to restrict yourself to the  subset of the
>   tmpfiles and sysusers interface that all the implementations of the
>   interface support.

Which would be perfectly fine. We did that for the bash -> dash
implementation of sh, because we found out dash was much faster. This
implied a loss of a lot of features. All we need to do is to balance
what we're doing, and see. If we think a feature is unimportant, maybe
it's best to forbid it (the right way, with a lintian warning/error) so
that we make sure all implementations are working. This is where the
Debian policy becomes important. Note that this works in both cases,
where the systemd or the alternative has a new thing.

> When there is a sufficient benefit, I
> think it would be fine to accept these costs.

That's another way to write what I wrote! :)

> Given that the systemd implementations will work for alternate inits on
> linux, I'd rather see us use them there.
> But I don't care that much.
> I'm just expressing an opinion about which policy Debian should adopt;
> if I'm in the rough that's okay with me.

If we have feature parity, or as if as I wrote above, we're ok with the
implemented features and think they are enough, then we could decide the
exact opposite way: choose the implementation that works everywhere, and
get rid of the systemd one (and maybe have systemd depend on the
alternative implementation), so that we know we'll have the same
behavior on all systems. Obviously, that's something to decide *later*,
once the alternatives have been tested long enough and we know they are
robust   enough (which I currently cannot tell).

> You talk later about wanting to have a runit facility where you could
> create tmpfiles before starting a service rather than at boot.
> That seems cool, but that's going to be an entirely different interface
> than what we're talking about here.
> If packages stick tmpfiles units in the right place and do the right
> thing from a maintainer script, they expect their directories to exist
> even if their service never gets started (or their package has no
> service).

If a service needs a tmpfile, then there's such facility in the .service
units, and this is where it should be written. So I don't think
systemd-tmpfiles is to be used for starting services, but for all other
cases.

Cheers,

Thomas Goirand (zigo)

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Andrey Rahmatullin]

On Fri, Jan 03, 2020 at 09:20:44AM +0100, Thomas Goirand wrote:
> On 1/2/20 6:01 PM, Matthias Plump wrote:
> > I don't have both of those. Since I am on an usrmerged system though,
> > /bin/systemd-sysusers and /usr/bin/systemd-sysusers are exactly the
> > same binary. Maybe that's the thing that caused a bit of confusion?
> 
> I'm not on a usrmerged system, and I have both on my laptop. So that's
> not what happened to me. I've checked with md5sum, and these really are
> the same file (and an ELF binary, not a bash implementation like on
> opensysusers).
There is no /usr/bin/systemd-sysusers in the apt-file DB.

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Simon McVittie]

On Fri, 03 Jan 2020 at 09:18:58 +0100, Thomas Goirand wrote:
> ... after this discussion, it looks like we would prefer:
> /bin/systemd-tmpfiles and /bin/systemd-sysusers
> 
> For this, we need systemd to use update-alternatives for them then, so
> that opentmpfiles & opensysusers do not need to use dpkg divertion.

Would it be simpler to have opentmpfiles (and opensysusers)
Conflicts/Replaces systemd, or even install the different implementations
under different names and make the postinst snippets try one and then
the other? I suspect we are not going to want a third implementation -
if a third implementation was added to the archive it would most likely
be because the OpenRC implementation had become problematic or been
superseded.

Using alternatives seems like an unexpectedly large number of moving
parts for something that, at most, I would expect to only be a sysadmin
choice at the level of "which packages do I install?" rather than "which
of several installed packages do I use?". We've seen with elogind and
its libsystemd0 implementation that offering choices at the bottom of
the dependency stack is hard to get right - I think that applies even
more so if those choices can't be represented in the dependency system.

I think the situations to be supported are:

* System booted with systemd. We should consistently use systemd's
  implementation of these tools, because otherwise, any missing features
  or behaviour differences in opentmpfiles could break systemd units that
  are (quite reasonably) relying on a versioned dependency on
  systemd (>= 321) being sufficient to provide all the interfaces of
  systemd 321.

* System booted with a different init system, or no init system at all
  (chroot or container). Either implementation could be OK; we should
  probably prefer the systemd implementation, because it's the reference
  implementation of these interfaces, but having it be possible to use
  opentmpfiles/opensysusers on Linux would make it easier to prototype
  those packages, even if we end up with only the non-Linux ports
  using them.

On -devel, Sam Hartman has recommended being consistent about which
implementation is used for each port (systemd-tmpfiles for Linux systems,
opentmpfiles for non-Linux), and there's certainly value in that.

> I'd like to understand how /usr/bin/systemd-sysusers got into my system,
> when others are saying they don't have it.

I have no idea. If systemd.deb contained both, then it would be
uninstallable on merged-/usr systems, which we would certainly have
noticed since they are the default for buster's d-i...

Do you have any dpkg diversions or statoverrides that look relevant?

You say they are the same file, which at least probably rules out dpkg
somehow failing to remove a /usr/bin/systemd-sysusers that might have
been shipped by an older systemd package.

smcv

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Thomas Goirand]

On 1/2/20 6:01 PM, Matthias Plump wrote:
> I don't have both of those. Since I am on an usrmerged system though,
> /bin/systemd-sysusers and /usr/bin/systemd-sysusers are exactly the
> same binary. Maybe that's the thing that caused a bit of confusion?

I'm not on a usrmerged system, and I have both on my laptop. So that's
not what happened to me. I've checked with md5sum, and these really are
the same file (and an ELF binary, not a bash implementation like on
opensysusers).

Cheers,

Thomas Goirand (zigo)

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Thomas Goirand]

On 1/2/20 5:13 PM, Ansgar wrote:
> Thomas Goirand writes:
>> My proposal is for Debian to standardize on:
>> /bin/tmpfiles
>> and:
>> /usr/bin/sysusers
> 
> Why rename things?

I don't mind either ways, but...

... after this discussion, it looks like we would prefer:
/bin/systemd-tmpfiles and /bin/systemd-sysusers

For this, we need systemd to use update-alternatives for them then, so
that opentmpfiles & opensysusers do not need to use dpkg divertion. See:
https://bugs.debian.org/947847

>> I'm not sure why
>> there's both /bin/systemd-sysusers and /usr/bin/systemd-sysusers, and
>> which one should be used.
> 
> For the same reason there is /bin/bash and /usr/bin/bash probably?
> 
> Ansgar

I'd like to understand how /usr/bin/systemd-sysusers got into my system,
when others are saying they don't have it.

Cheers,

Thomas Goirand (zigo)

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Thomas Goirand]

On 1/2/20 7:08 PM, Russ Allbery wrote:
> I really want to get rid of maintainer scripts as much as possible in
> favor of pure declarative syntax in the packages.  I think the fewer
> Debian packages that need maintainer scripts, the easier the distribution
> as a whole will be to maintain, analyze, and reason about.

+1

Which is why I have packaged those: so that there's no controversy if we
standardize on them. We could even make the decision that adduser
shouldn't be used, and that sysusers.d is the preferred way. Let's see
how it develops.

Again, I really would prefer if I'm not the only person working on the
packaging of these alternative implementations. Please anyone:
contribute the init script, dpkg triggers, add yourself as uploaders, etc.

Cheers,

Thomas Goirand (zigo)

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Thomas Goirand]

On 1/2/20 6:28 PM, Simon McVittie wrote:
> On systemd systems, that's approximately:
> 
> - run systemd-tmpfiles when a package installs a tmpfiles.d snippet
>   (this is added to the package's postinst by dh_installsystemd)
> 
> - run systemd-sysusers when a package installs a sysusers.d snippet
>   (I don't think we have tools to add this to the postinst yet, because
>   packages are currently meant to run adduser --system instead, but
>   more-systemd-centric distributions probably already do this in their
>   equivalent of the postinst)
> 
> - run systemd-tmpfiles during boot (this is systemd-tmpfiles-setup.service,
>   part of systemd)
> 
> - run systemd-sysusers during boot (this is
>   systemd-sysusers.service, part of systemd)
> 
> The opentmpfiles and opensysusers packaging will need to arrange to do
> something analogous, most likely in cooperation with dh_installsystemd
> or some other debhelper step for the first two points, and with LSB init
> scripts for the tasks where systemd uses one-shot services.

Right, and I haven't implemented this yet. Contributions welcome!

> At the moment, the policy is that system users are created by running
> adduser, and installing sysusers.d files is allowed but redundant. For
> example, dbus installs /usr/lib/sysusers.d/dbus.conf, but doesn't rely
> on it being processed - the postinst calls adduser, so the sysusers.d
> snippet will only have any effect if the messagebus user somehow gets
> deleted, in which case systemd-booted systems will recover by recreating
> it during the next boot.

I suppose we all agree that the goal is to have adduser manual call go
away and being replaced by the automated declarative way. Probably we
would need to have the system user be installed *before* the postinst
runs, so that the eventual daemon used in the package has the system
user available.

Cheers,

Thomas Goirand (zigo)

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Sam Hartman]

> "Lorenz" == Lorenz   writes:

Lorenz> Sam Hartman wrote:
>> My understanding is that systemd's implementation of tmpfiles and
>> sysusers works even while systemd is not pid 1.
>> Why do we need multiple implementations for Debian ports where systemd
>> runs?
>> I understand why we might want alternatives for kfreebsd and hurd.
>> But if my understanding that the systemd implementation does not require
>> systemd be running as pid 1, why do we need alternatives for the glibc
>> linux ports?
Lorenz> Why dump additional work on non-linux porters when we ( 
alternatives inits suppo
Lorenz> rters)
Lorenz> can have one implementation that works on both linux and non-linux?
Lorenz> Please consider that beeing able to work on non-linux port it's a 
feature that m
Lorenz> any
Lorenz> (if not all) alternative inits provide while systemd doesn't.
Lorenz> That feature it's clearly not important for a systemd advocate,
Lorenz> but may be important for a sysv/openrc/runit user/contributor.

I think it's easier to have a single implementation of an interface for
a given arch.

With elogind, policy-kit etc, what we've found is that runtime or
installed package time selection of facilities is much more complex than
build time selection.

We probably avoid bugs if we always used systemd-tmpfiles on
architectures where it can build and run.

Secondly, by using systemd-tmpfiles when we can, we gain support for any
additional features that are implemented.

My recommendation would be to have one implementation of this interface
per architecture; use the systemd one on architectures where it works.

Lorenz> In wich way having an alternative implementation of tmpfiles.d
Lorenz> and sysuser.d around will harm systemd?

It will not.
I think another question is more interesting though.

Will having runtime selection of that interface harm Debian.

My answer is that I think it might:

* You might get the wrong version of tmpfiles implementation on a given
  system.  The harm on a systemd system would be that you might not have
  a feature available.

* You might be tempted to restrict yourself to the  subset of the
  tmpfiles and sysusers interface that all the implementations of the
  interface support.

I think these are minor harms.  When there is a sufficient benefit, I
think it would be fine to accept these costs.
For example, being able to have tmpfiles and sysusers ond hurd and
kfreebsd seems like sufficient justification to accept the costs on
those platforms.

Given that the systemd implementations will work for alternate inits on
linux, I'd rather see us use them there.
But I don't care that much.
I'm just expressing an opinion about which policy Debian should adopt;
if I'm in the rough that's okay with me.

You talk later about wanting to have a runit facility where you could
create tmpfiles before starting a service rather than at boot.
That seems cool, but that's going to be an entirely different interface
than what we're talking about here.
If packages stick tmpfiles units in the right place and do the right
thing from a maintainer script, they expect their directories to exist
even if their service never gets started (or their package has no
service).

It would not be reasonable for runit to change the semantics of that
interface.

It would be fine for runit to provide its own tmpfiles interface that
works differently.  It would be fine for runit to provide a way to
signal that a package using the runit specific interface should disable
the systemd interface for that package when runit is in use.
I don't care what tmpfiles implementation you use in that case.

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Lorenz]

[ please keep me in CC, I'm not subscribed to this list]

Sam Hartman wrote:

> My understanding is that systemd's implementation of tmpfiles and
> sysusers works even while systemd is not pid 1.
> Why do we need multiple implementations for Debian ports where systemd
> runs?
> I understand why we might want alternatives for kfreebsd and hurd.
> But if my understanding that the systemd implementation does not require
> systemd be running as pid 1, why do we need alternatives for the glibc
> linux ports?

Why dump additional work on non-linux porters when we ( alternatives
inits supporters)

can have one implementation that works on both linux and non-linux?

Please consider that beeing able to work on non-linux port it's a
feature that many

(if not all) alternative inits provide while systemd doesn't.

That feature it's clearly not important for a systemd advocate,

but may be important for a sysv/openrc/runit user/contributor.


In wich way having an alternative implementation of tmpfiles.d

and sysuser.d around will harm systemd?


I read your proposal B several times:

'Debian remains an environment where developers and users can
explore and develop alternate init systems and alternatives to systemd
features. Those interested in exploring such alternatives need to
provide the necessary development and packaging resources to do that
work.'

What do you mean exactly with "alternatives to features"?

Please clarify, because here I see two alternatives to features and at
least one person

that is offering to do the job.

But there is more:

let's take tmpfiles.d: I don't like how systemd handles

tmpfiles creation and I do consider it totally unfitted

for my favourite init (runit).

* Why do I need to create all directories and files for all

  installed services at boot?

* Why do I have to create directories and files in maintainer

  scripts?

* Since I have a central utility that creates directory and files

   (which you guess what... i don't like it LOL, because it's a utility

that can fail all your services at once if something goes wrong)

   why not let such utility create all directories needed by a service?


I just need an utility like the following:

'mksvdir foo' --> create all (not just temporary) dir and files for foo service

'mksvdir delete foo' --> delete temporary dir and files for foo

then call such utility in invoke-run intepreter, and fail the service if

it exits non-zero. that's it.

Since invoke-run is executed only if runit is installed I can even

make 'mksvdir' a dependency of runit without bothering foo maintainer

with another dependency that almost nobody uses.


Is systemd upstream or debian maintainers available to accept patches that

adapt it's facilities to the needs of other inits?

IMHO imposing a systemd facility when there are alternatives and people

available to do the work it's a perfect setup for keep on arguing and fighting


Regards,

Lorenzo

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Niels Thykier]

Simon McVittie:
> [...]
> 
> The opentmpfiles and opensysusers packaging will need to arrange to do
> something analogous, most likely in cooperation with dh_installsystemd
> or some other debhelper step for the first two points, and with LSB init
> scripts for the tasks where systemd uses one-shot services.
> 
> [...]

Re: tmpfiles:

There is already a bug for extracting tmpfiles.d handling from
dh_installsystemd and put it in a separate helper for unrelated reasons
(see #944478).

Improvements/ideas welcome on #944478.

Re: sysusers:

I am happy to implement a helper for that as well assuming there is
consensus that this is a better approach over adduser.  Just file a
wishlist bug with instructions on what that helper is supposed to do and
how the maintscript should look.


Russ Allbery:
> [...]
> 
> I really want to get rid of maintainer scripts as much as possible in
> favor of pure declarative syntax in the packages.  [...]

Hear, Hear!

Bonus points for anyone with a solution *without* maintscripts!

(But in the absence of perfection, a debhelper tool with a autoscript
snippet can do).

Thanks,
~Niels

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Marco d'Itri]

On Jan 02, Thomas Goirand  wrote:

> My proposal is for Debian to standardize on:
> /bin/tmpfiles
> 
> and:
> /usr/bin/sysusers
We already have an interface used by many packages, hence I see no 
reason to change it.

Actually it is still not obvious to me why Debian would need a second 
implementation of these programs, so I think that this point should be 
addressed first.

-- 
ciao,
Marco


signature.asc
Description: PGP signature

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Russ Allbery]

Simon McVittie  writes:

> Using dpkg triggers to run the tmpfiles and sysusers tools automatically
> is probably not enough, because a package's postinst will often run
> system services (systemd unit, LSB init script, etc.), and those system
> services need to be run *after* creating at least this package's
> tmpfiles hierarchies and system users, because they probably rely on
> those directories and users to work correctly.

Oh, meh, that's a good point.  This is another good argument for getting
dpkg integration in the long run so that dpkg knows to run the right
things at the right points and can also track the ownership of files and
users.

I really want to get rid of maintainer scripts as much as possible in
favor of pure declarative syntax in the packages.  I think the fewer
Debian packages that need maintainer scripts, the easier the distribution
as a whole will be to maintain, analyze, and reason about.  When I was
doing lots of work on Lintian, there were so many times when we could have
done so much better if we didn't have to parse free-form shell scripts and
try to extract meaning from them.

-- 
Russ Allbery (r...@debian.org)  

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Russ Allbery]

Matthias Klumpp  writes:

> I find that statement quite encouraging. Of course they don't commit
> to not having those depend on systemd-as-PID1, but there really isn't
> a reason to create that dependency, and if for whatever reason there
> will be one at some point, we can switch away on systems that don't
> support that change rather easily.

I specifically asked that question in a follow-up to that message and
Zbyszek said that this is very likely to continue to work indefinitely.

That said, I personally don't mind if people want to explore alternate
implementations of the same concept as well.  The important part for me
(and this is with my Policy hat on) is that we can standardize on the
interface, and that interface should be declarative.  In other words,
packages should be able to drop a configuration file in a supported
location and have the right thing happen.

This also allows Guillem to support these facilities properly inside dpkg
with tracking of which users and tmp files belong to which package, using
the same interface, which I think would be ideal in the long run.

-- 
Russ Allbery (r...@debian.org)  

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Sam Hartman]

> "Simon" == Simon McVittie  writes:


Simon> I think this is partly down to whether the systemd maintainers (both
Simon> upstream and downstream) are willing to support this use, or whether
Simon> they would prefer non-systemd-booted systems to use a 
reimplementation
Simon> whose maintainers are responsible for supporting it. Do we know which
Simon> of those is true?

Simon> I seem to remember a systemd upstream developer being asked during
Simon> recent discussions whether they were willing to guarantee that
Simon> systemd-tmpfiles and systemd-sysusers will continue to work when 
used on
Simon> non-systemd-booted systems (not just sysvinit, but also chroots, 
Docker,
Simon> etc.), but I'm afraid I've lost track of what the answer
Simon> was.

I seem to recall it was surprisingly convincing that this will continue
to work.

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Matthias Klumpp]

Am Do., 2. Jan. 2020 um 18:41 Uhr schrieb Simon McVittie :
> [...]
> I seem to remember a systemd upstream developer being asked during
> recent discussions whether they were willing to guarantee that
> systemd-tmpfiles and systemd-sysusers will continue to work when used on
> non-systemd-booted systems (not just sysvinit, but also chroots, Docker,
> etc.), but I'm afraid I've lost track of what the answer was.
> [...]

No problem! => https://lists.debian.org/debian-devel/2019/12/msg00060.html

Am Mo., 9. Dez. 2019 um 00:23 Uhr schrieb Zbigniew Jędrzejewski-Szmek
:
> Hi,
>
> [disclaimer: on work on systemd upstream, I'm not an active Debian
> user anymore.]
>
> Using systemd-sysusers and systemd-tmpfiles more widely was mentioned
> a few times, along with a statement that an implementation for
> non-systemd systems would need to be provided. Both those programs
> work just fine without systemd not running as PID1. (systemd has unit
> files to start them automatically during boot and at regular
> intervals, so that part would need to be reimplemented appropriately
> for a given init system if desired. The programs themselves don't care
> at all how they are started.)
>
> For example, upstream distributes rpm scriptlets [1] to invoke them
> from an rpm transaction, i.e. possibly without any programs running in
> the install root.
>
> [1] 
> https://github.com/systemd/systemd/blob/master/src/core/macros.systemd.in#L123
>
> Zbyszek
>

I find that statement quite encouraging. Of course they don't commit
to not having those depend on systemd-as-PID1, but there really isn't
a reason to create that dependency, and if for whatever reason there
will be one at some point, we can switch away on systems that don't
support that change rather easily.
Since both features are covered by the interface stability promise (
https://www.freedesktop.org/wiki/Software/systemd/InterfacePortabilityAndStabilityChart/
) I think assuming that this will work for a really, really long time
is very reasonable.

Cheers,
Matthias

-- 
I welcome VSRE emails. See http://vsre.info/

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Simon McVittie]

On Thu, 02 Jan 2020 at 09:03:24 -0500, Sam Hartman wrote:
> My understanding is that systemd's implementation of tmpfiles and
> sysusers works even while systemd is not pid 1.
> Why do we need multiple implementations for Debian ports where systemd
> runs?

I think this is partly down to whether the systemd maintainers (both
upstream and downstream) are willing to support this use, or whether
they would prefer non-systemd-booted systems to use a reimplementation
whose maintainers are responsible for supporting it. Do we know which
of those is true?

I seem to remember a systemd upstream developer being asked during
recent discussions whether they were willing to guarantee that
systemd-tmpfiles and systemd-sysusers will continue to work when used on
non-systemd-booted systems (not just sysvinit, but also chroots, Docker,
etc.), but I'm afraid I've lost track of what the answer was. The current
dh_installsystemd postinst fragments for tmpfiles.d are guarded by
[ -d /run/systemd ] so that they will be a no-op on non-systemd-booted
machines.

(I don't see any particular reason why those tools would be init-dependent,
though - they share systemd core code for utility stuff like configuration
file parsing, and I suspect that's all.)

If we want opentmpfiles and opensysusers to be available on only the
non-Linux-kernel ports, prototyping the packages on Linux + sysvinit
systems might also be the most pragmatic short-term way to get them
working well.

smcv

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Simon McVittie]

On Fri, 03 Jan 2020 at 00:05:10 +0800, Shengjing Zhu wrote:
> On Thu, Jan 2, 2020 at 9:22 PM Thomas Goirand  wrote:
> [...]
> > My proposal is for Debian to standardize on:
> > /bin/tmpfiles
> >
> > and:
> > /usr/bin/sysusers

FWIW, the systemd versions of both tools are canonically in /bin (that's
their only location on the non-merged-/usr systems where it matters) so
it might be best to stay consistent with that. They're named systemd-foo
because that's systemd's convention for tools that are meant to be only
rarely run by hand, like a weaker version of the GNU Coding Standards'
${libexecdir} (tools that are meant to be run by hand more frequently,
like loginctl, don't get that prefix).

> I don't understand why we need these.
> 
> The advantages of sysusers.d and tmpfiles.d are that you don't need to
> call some magic scripts.
> You only need to write declarative configuration files.

Individual packages only need to install declarative configuration files,
but the OS distribution infrastructure needs to do something to make
the contents of those configuration files take effect (and, crucially,
know when the implementation has finished running).

On systemd systems, that's approximately:

- run systemd-tmpfiles when a package installs a tmpfiles.d snippet
  (this is added to the package's postinst by dh_installsystemd)

- run systemd-sysusers when a package installs a sysusers.d snippet
  (I don't think we have tools to add this to the postinst yet, because
  packages are currently meant to run adduser --system instead, but
  more-systemd-centric distributions probably already do this in their
  equivalent of the postinst)

- run systemd-tmpfiles during boot (this is systemd-tmpfiles-setup.service,
  part of systemd)

- run systemd-sysusers during boot (this is
  systemd-sysusers.service, part of systemd)

The opentmpfiles and opensysusers packaging will need to arrange to do
something analogous, most likely in cooperation with dh_installsystemd
or some other debhelper step for the first two points, and with LSB init
scripts for the tasks where systemd uses one-shot services.

Using dpkg triggers to run the tmpfiles and sysusers tools automatically
is probably not enough, because a package's postinst will often run system
services (systemd unit, LSB init script, etc.), and those system services
need to be run *after* creating at least this package's tmpfiles hierarchies
and system users, because they probably rely on those directories and users
to work correctly.

At the moment, the policy is that system users are created by running
adduser, and installing sysusers.d files is allowed but redundant. For
example, dbus installs /usr/lib/sysusers.d/dbus.conf, but doesn't rely
on it being processed - the postinst calls adduser, so the sysusers.d
snippet will only have any effect if the messagebus user somehow gets
deleted, in which case systemd-booted systems will recover by recreating
it during the next boot.

smcv

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Matthias Klumpp]

Am Do., 2. Jan. 2020 um 17:28 Uhr schrieb Ansgar :
>
> Thomas Goirand writes:
> > [...]
> > I'm not sure why
> > there's both /bin/systemd-sysusers and /usr/bin/systemd-sysusers, and
> > which one should be used.
>
> For the same reason there is /bin/bash and /usr/bin/bash probably?

I don't have both of those. Since I am on an usrmerged system though,
/bin/systemd-sysusers and /usr/bin/systemd-sysusers are exactly the
same binary. Maybe that's the thing that caused a bit of confusion?

Personally, I think it might make a lot more sense to have tools
depending on systemd-sysusers depend on the original systemd package,
given that those binaries can be used without systemd being PID1. That
of course needs people to write initscripts for them, which may need
to live in a separate package (or possibly even be part of the
alternative initsystems themselves, for easy maintenance) so people
who want them can pull them in. If pulling in the bigger systemd
package is a problem (as not everything in there works if systemd
isn't PID1), possibly splitting the sysusers binaries out to a
separate package may work as well.
By using the systemd-provided files, we can ensure that any possible
new features are immediately available everywhere and nothing has to
"catch up", and systemd systems won't get confused over which
implementation is the right one currently.

In any case, the existence of opensysusers/opentmpfiles is really
great already, that makes using these features viable faster and with
much less friction, since every system will be supported

Cheers,
Matthias

-- 
I welcome VSRE emails. See http://vsre.info/

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Ansgar]

Thomas Goirand writes:
> My proposal is for Debian to standardize on:
> /bin/tmpfiles
> and:
> /usr/bin/sysusers

Why rename things?

> I'm not sure why
> there's both /bin/systemd-sysusers and /usr/bin/systemd-sysusers, and
> which one should be used.

For the same reason there is /bin/bash and /usr/bin/bash probably?

Ansgar

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Shengjing Zhu]

On Thu, Jan 2, 2020 at 9:22 PM Thomas Goirand  wrote:
[...]
> My proposal is for Debian to standardize on:
> /bin/tmpfiles
>
> and:
> /usr/bin/sysusers
>

I don't understand why we need these.

The advantages of sysusers.d and tmpfiles.d are that you don't need to
call some magic scripts.
You only need to write declarative configuration files.

On systemd system, they are handled by systemd-sysusers.service and
systemd-tmpfiles-*.service.
You really don't need to care where the binaries are located.

On openrc system, I think they are handled by corresponding init services too.
https://github.com/OpenRC/opentmpfiles/tree/master/openrc

[...]
> opensysusers if a package is using /usr/bin/sysusers. I'm not sure why
> there's both /bin/systemd-sysusers and /usr/bin/systemd-sysusers, and
> which one should be used. Maybe Michael, you know?

Where is /usr/bin/systemd-sysusers? at least not in testing.


-- 
Shengjing Zhu

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Sam Hartman]

My understanding is that systemd's implementation of tmpfiles and
sysusers works even while systemd is not pid 1.
Why do we need multiple implementations for Debian ports where systemd
runs?
I understand why we might want alternatives for kfreebsd and hurd.
But if my understanding that the systemd implementation does not require
systemd be running as pid 1, why do we need alternatives for the glibc
linux ports?

Re: opentmpfiles & opensysusers, and its use in the Debian policy

[Mo Zhou]

Hi Thomas,

Thank you for working on these!

On Thu, Jan 02, 2020 at 02:14:34PM +0100, Thomas Goirand wrote:
> As I wrote, no need to complain, but act.
> 
> https://salsa.debian.org/debian/opentmpfiles
> https://salsa.debian.org/debian/opensysusers
> 
> Both are now in the NEW queue, and both need some kind of init.d sysv-rc
> script. Please anyone, contribute that init script.

I quickly scanned the etc files in my Gentoo chroot, and found some
openrc scripts for opentmpfiles. I volunteer to translate that one into
the form we can use, but currently I don't acquire the lock due to my
overflowing todo list. [1]

For opensysusers I suspect that Gentoo didn't package it as
 $ emerge --search opensysusers
shows nothing.

Maybe we can also borrow something from the non-systemd variant of
Archlinux.

[1] Does anyone know if systemd-nspawn can (theoretically) boot a
non-systemd stage3 image?  I tried to boot a gentoo+openrc image,
but it ended up stuck somewhere.
If this is doable, it will make my init script debugging and
testing process much much less painfull.