Re: s6 as a systemd alternative

[Casper Ti. Vector]

(Normally Jonathan would be replying to this point, but I still do not
see him in this thread, so I rashly take this job ;)

On Mon, Jun 26, 2017 at 05:05:15PM +0200, Istvan Szukacs wrote:
> I understand that service files are much better that shell scripts

Actually they are not.  This statement might appear logical when you see
[1] or some similar comparisons made by systemd proponents; however,
there are some problems with such comparisons, and here are the ones
that come to my mind at this time:
* The script on the left is a script for sysv-rc; sysv-rc is,
  admittedly, a terrible system.  rc scripts for some other traditional
  rc systems are much cleaner, eg. that for openbsd [2], or that for
  openrc [3].
* Traditional rc systems still deal with double-forking daemons, using
  pid-files, which is an awkward mechanism [4].  By abandoning double
  fork, process state handling (and daemon logging) would be much more
  straightforward [5].
* systemd unit files may seem a little more descriptive (and verbose ;)
  than the scripts in [5].  However, by chainloading, you can use
  noncommutative operator combinations (like passing some information in
  some environment variables and then cleaning up the environment after
  use of this information in some initialisation), which is difficult to
  achieve with systemd unit files.  Implementing the ordering of
  operators by ordering of lines in the unit file would probably result
  in reinventing execline / nosh, in an ugly way (actually they already
  partially reinvented the shell: note the `ExecStart*' fields in the
  unit file specification).
* systemd implements the mechanism for process state manipulation in
  pid-1, which leads to considerable safety/security implications.  By
  chainloading, all of these can be factored into the run script,
  without increasing complexity of the user interface.

[1] .
[2] .
[3] 
.
[4] .
[5] .

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: s6 as a systemd alternative

[Laurent Bercot]

 Would
this approach keep systemd installed just to provide for services 
directly

depending on it?

# rpm -q --requires openssh-server | grep systemd | sort -u
libsystemd.so.0()(64bit)
libsystemd.so.0(LIBSYSTEMD_209)(64bit)
systemd-units

I am not sure why any service would depend on these. Is there 
functionality

in libsystemd.so.0 that an ssh service actually needs?


 Not really, but that is exactly what I'm talking about when I speak
of vendor lock-in.
 libsystemd provides some library functions that applications can use
to interface themselves with systemd and allow it to provide advanced
functionality such as fd-holding, readiness notification, etc. But
all that functionality could be achieved in another way, without
needing a specific library: s6 provides tools to perform fd-holding
and an interface to perform readiness notification without needing
the application to link against a specific library.
 The fact that systemd encourages applications to link against
libsystemd is one of the way it ties itself to a system and makes
itself unremovable. That is the definition of vendor lock-in; that is
also, incidentally, the definition of cancer.

 You will not be able to entirely remove systemd from a distribution
that has chosen to embrace it - not until there is a will from the
higher technical management of that distribution to revert the
process and untie application software from systemd interfaces.
 Switching a distribution such as Fedora or Ubuntu to a different
init system is really difficult work. My approach is to first work
with other distributions, such as Alpine, Void or Devuan, which are
systemd-free, and get an integrated s6 init system working on those,
and get traction. Only when we have a fully integrated, easy to
use, widely spread platform will we be able to start turning the
tide and reclaim distributions from the systemd grasp. But this will
likely not happen for several years.

--
 Laurent

Re: s6 as a systemd alternative

[Jean Louis]

On Mon, Jun 26, 2017 at 04:02:47PM +0200, Istvan Szukacs wrote:
> Hi,
> 
> I have a crazy question about s6. Would it be possible to make systemd
> compatible? This question might sound stupid at first but here is the
> reasoning:
> 
> - we have services with systemd service files already
> (/etc/systemd/system/*.service, etc.)
> - the previous alternatives all failed to gain traction because there was
> too much effort to change systems around to use them (
> https://www.tecmint.com/best-linux-init-systems/) and the linux platform is
> very fragmented
> - there is already too much effort went into systemd that would be hard to
> duplicate
> 
> Questions:
> 
> - is there anybody interested in such project?
> - is this feasible to do?

I am user of s6 for reason of simplicity and to avoid trouble of
systemd.

While not being developer, maybe it could be possible to have s6 run
systemd as a service and systemd to run s6 as a service, with
modifications, but I think, it will be better to hit oneself with hard
rock to the head, it would be more pleasure.

Jean

Re: s6 as a systemd alternative

[Istvan Szukacs]

Hi Charles,

Yeah I can clarify. The only thing that I (or for that matter saner systems
engineers) want from systemd is to be a better sysv init. I do not want
logging, ntp and all the other crap that got sucked into it. I understand
that service files are much better that shell scripts and this is a good
argument but it does not justify the idiocracy that systemd became in the
recent years. Ideally we could have a service (like s6) that does only that
keeping the best parts of systemd as well. This would allow me to run
redhat/centos based systems and use service files without being worried
that a huge amount of CPU cycles going to a service that sole purpose is to
keep services up that actually provide the functionality that I need. Does
this clarify?

I see your point about the linux only things though. I am not sure what is
the right approach there. I should dig deeper into Amazon Linux and see how
they escaped systemd-hell.

I.


On Mon, Jun 26, 2017 at 4:47 PM, Charles Duffy  wrote:

> Could you be more clear about what you mean by "make systemd compatible"?
> Do you mean loading systemd configuration files into s6, or the reverse?
> The former strikes me as exceedingly difficult to implement in a complete
> and correct manner.
>
> One of the things that makes systemd so... controversial... is the amount
> of complexity it pulls into what (in many folks' view) ought to be designed
> as a simple subsystem (in the "simple enough that an implementation can be
> obviously correct" sense). Because of that amount of complexity -- one
> could rather easily implement a simple subset of its functionality, but
> reaching full parity (and *maintaining* that parity as it continues to
> grow, expand, and cross what would historically be distinct subsystem
> boundaries) strikes me as a very ambitious project.
>
> As a few examples of things that systemd provides that would need to be
> reimplemented:
>
> - A mechanism based on UNIX domain sockets for implementing a watchdog,
> and for processing file descriptors between subsequent invocations of the
> same service.
> - Sandboxing of allowed syscalls (using a Linux-only mechanism not
> applicable to other platforms s6 supports)
> - Management of process-local filesystem, PID, and user namespaces (again,
> using a Linux-only mechanism)
> - Integration with a (again, linux-only and glibc-only) nsswitch module to
> generate dynamic, transient user accounts local to an individual instance
> of a process
> - Integration with the linux-only cgroups mechanism for managing CPU,
> memory, and I/O throughput limits
>
> ...and so on, and so forth. Consequently, any effort would necessarily be
> a small subset, and plagued by compatibility issues whenever a distributed
> .service file attempts to use functionality that a different process
> supervision system couldn't implement without compromising portability (or
> changing its behavior between platforms).
>
> On Mon, Jun 26, 2017 at 9:02 AM Istvan Szukacs <
> ist...@streambrightdata.com> wrote:
>
>> Hi,
>>
>> I have a crazy question about s6. Would it be possible to make systemd
>> compatible? This question might sound stupid at first but here is the
>> reasoning:
>>
>> - we have services with systemd service files already
>> (/etc/systemd/system/*.service, etc.)
>> - the previous alternatives all failed to gain traction because there was
>> too much effort to change systems around to use them (
>> https://www.tecmint.com/best-linux-init-systems/) and the linux platform
>> is
>> very fragmented
>> - there is already too much effort went into systemd that would be hard to
>> duplicate
>>
>> Questions:
>>
>> - is there anybody interested in such project?
>> - is this feasible to do?
>>
>> Let me know what you think.
>>
>> Thanks in advance,
>> Istvan
>>
>> --
>> *Istvan Szukacs*
>> CTO
>>
>> ist...@streambrightdata.com
>>
>


-- 
*Istvan Szukacs*
CTO

+31647081521 
+36 70 229 20 59 
ist...@streambrightdata.com

Re: s6 as a systemd alternative

[Laurent Bercot]

a bunch of hackers behind Void Linux[2] made everything tick on top of
runit without all that much effort.


 The problem with the runit model is that it is pure supervision -
it does not provide service management. You have to run all your
oneshots _before_ you can start longruns. See
 https://skarnet.org/software/s6-rc/why.html and
 https://fosdem.org/2017/schedule/event/s6_supervision/ (15 mn video)
to understand what goes into an init system and why runit alone is
not quite enough.

 Of course Void Linux manages to do it, but it is, pardon my French,
a hack job: AFAICT, udevd is not supervised, and some oneshots are
implemented via longrun processes doing nothing. This is cramming
the service management peg into the process supervision hole, and
can never be perfect - Void would definitely benefit from switching
to real service management. But so far they haven't expressed any
desire to switch.

--
 Laurent

Re: s6 as a systemd alternative

[Charles Duffy]

Could you be more clear about what you mean by "make systemd compatible"?
Do you mean loading systemd configuration files into s6, or the reverse?
The former strikes me as exceedingly difficult to implement in a complete
and correct manner.

One of the things that makes systemd so... controversial... is the amount
of complexity it pulls into what (in many folks' view) ought to be designed
as a simple subsystem (in the "simple enough that an implementation can be
obviously correct" sense). Because of that amount of complexity -- one
could rather easily implement a simple subset of its functionality, but
reaching full parity (and *maintaining* that parity as it continues to
grow, expand, and cross what would historically be distinct subsystem
boundaries) strikes me as a very ambitious project.

As a few examples of things that systemd provides that would need to be
reimplemented:

- A mechanism based on UNIX domain sockets for implementing a watchdog, and
for processing file descriptors between subsequent invocations of the same
service.
- Sandboxing of allowed syscalls (using a Linux-only mechanism not
applicable to other platforms s6 supports)
- Management of process-local filesystem, PID, and user namespaces (again,
using a Linux-only mechanism)
- Integration with a (again, linux-only and glibc-only) nsswitch module to
generate dynamic, transient user accounts local to an individual instance
of a process
- Integration with the linux-only cgroups mechanism for managing CPU,
memory, and I/O throughput limits

...and so on, and so forth. Consequently, any effort would necessarily be a
small subset, and plagued by compatibility issues whenever a distributed
.service file attempts to use functionality that a different process
supervision system couldn't implement without compromising portability (or
changing its behavior between platforms).

On Mon, Jun 26, 2017 at 9:02 AM Istvan Szukacs 
wrote:

> Hi,
>
> I have a crazy question about s6. Would it be possible to make systemd
> compatible? This question might sound stupid at first but here is the
> reasoning:
>
> - we have services with systemd service files already
> (/etc/systemd/system/*.service, etc.)
> - the previous alternatives all failed to gain traction because there was
> too much effort to change systems around to use them (
> https://www.tecmint.com/best-linux-init-systems/) and the linux platform
> is
> very fragmented
> - there is already too much effort went into systemd that would be hard to
> duplicate
>
> Questions:
>
> - is there anybody interested in such project?
> - is this feasible to do?
>
> Let me know what you think.
>
> Thanks in advance,
> Istvan
>
> --
> *Istvan Szukacs*
> CTO
>
> ist...@streambrightdata.com
>

Re: s6 as a systemd alternative

[Kamil Cholewiński]

Hi Istvan,

check out nosh[1]. It explicitly states systemd compat as one of its
goals and goes to great lenghts to make it happen.

[1]: https://jdebp.eu/Softwares/nosh/

It preserves the daemontools style and spirit.

Meanwhile, while hordes of programmers are busy making everything fit
into the systemd-world (and fixing bugs they've created in the process),
a bunch of hackers behind Void Linux[2] made everything tick on top of
runit without all that much effort.

[2]: https://www.voidlinux.eu/

<3,K.

s6 as a systemd alternative

[Istvan Szukacs]

Hi,

I have a crazy question about s6. Would it be possible to make systemd
compatible? This question might sound stupid at first but here is the
reasoning:

- we have services with systemd service files already
(/etc/systemd/system/*.service, etc.)
- the previous alternatives all failed to gain traction because there was
too much effort to change systems around to use them (
https://www.tecmint.com/best-linux-init-systems/) and the linux platform is
very fragmented
- there is already too much effort went into systemd that would be hard to
duplicate

Questions:

- is there anybody interested in such project?
- is this feasible to do?

Let me know what you think.

Thanks in advance,
Istvan

-- 
*Istvan Szukacs*
CTO

ist...@streambrightdata.com