Re: s6 as a systemd alternative

[Jan Bramkamp]

The s6-svscan and s6-supervise are very simple (and elegant) and in a 
way do less than runsvdir and runsv: they don't go around allocating 
resources at runtime.



On 30.06.2017 22:38, Steve Litt wrote:

On Fri, 30 Jun 2017 19:50:17 +
"Laurent Bercot"  wrote:


The runsv executable is pretty robust, so it's unlikely to die.

   Yadda yadda yadda. Most daemons are also unlikely to die, so
following your reasoning, I wonder why we're doing supervision in the
first place. Hint: we're doing supervision because we are not content
with "unlikely". We want "impossible".

You want impossible. I'm quite happy with unlikely. With my use
case, rebooting my computer doesn't ruin my whole day. If it *did* ruin
my whole day, my priorities would be changed and I'd switch to s6.




  As far
as somebody killing it accidentally or on purpose with the kill
command, that's a marginal case. But if it were *really* important to
protect against, fine, have one link dir per early longrun, and run
an individual runsvdir on each of those link directories.

   And you just increased the length of the chain while adding no
guarantee at all, because now someone can just kill that runsvdir
first and then go down the chain, like an assassin starting with the
bodyguards of the bodyguards of the important people. Or the assassin
might just use a bomb and blow up the whole house in one go: kill -9
-1.

   The main point of supervision is to provide an absolute guarantee
that some process tree will always be up, no matter what gets killed
in what order, and even if everything is killed at the same time.

To me, the preceding isn't the main point of supervision. Supervision
benefits I value more are:

* Run my daemon in foreground, so homegrown daemons have no need to
   self-background.
* Consistent and easy handling of log files.
* Under almost all circumstances, dead daemons get restarted.
* Simple config and troubleshooting, lots of test points.
* POSIX methodologies ensure I can easily do special stuff with it.
* Ability to base process dependency on whether the dependee is
   *really* doing its job.


You
can only achieve that guarantee by rooting your supervision tree in
process 1.

Yes.


   With runit, only the main runsvdir is supervised - and even then it
isn't really, because when it dies runit switches to stage 3 and
reboots the machine. Which is probably acceptable behaviour, but
still not supervision.

If we're going to get into definitions, then let me start by saying
what I want is daemontools that comes up automatically when the machine
is booted. Whether or not that's supervision isn't something I care
about.


And everything running outside of that main
runsvdir is just hanging up in the air - they can be easily killed
and will not return.

Well, if they kill the runsv that's true, but if they kill the
daemon, no. Either way, I'm willing to live with it.



   By adding supervisors to supervisors, you are making probabilistic
statements, and hoping that nobody will kill all the processes in the
wrong order. But hope is not a strategy. There is, however, a strategy
that works 100% of the time, and that is also more lightweight because
it doesn't require long supervisor chains: rooting the supervision
tree in process 1. That is what an s6-based init does, and it
provides real, strong supervision; and unlike with runit, the machine
is only rebooted when the admin explicitly decides so.

I completely understand your point. I just don't need that level of
indestructibility.


   If you're not convinced: *even systemd* does better than your
solution. systemd obviously has numerous other problems, but it does
the "root the supervision tree in process 1" thing right.

LOL, my whole point is I don't necessarily think "root the supervision
tree in process 1" is right, at least for my use case. I *enjoy* having
a tiny, do-almost-nothing PID1.

Like I said before, if losing control of the system during special
circumstances would ruin my whole day, I'd change my priorities and use
s6.


   I appreciate your enthusiasm for supervision suites. I would
appreciate it more if you didn't stop halfway from understanding
everything they bring, and if you didn't paint your unwillingness to
learn more as a technical argument, which it definitely is not, while
flippantly dismissing contributions from people who know what they
are talking about.

But I didn't flippantly dismiss anybody or any contributions. I
pointed  out that one can, and I'll use different verbiage now, respawn
daemons early in the boot, before some of the one-shots had started.

I'm not an enemy of s6. I'm not an enemy of anything you apply the word
"supervision" to. I think I understand your reasons for doing what you
do. It's just that with my current use case, I've traded some of s6's
process and boot security (you know what I mean) for a simpler PID1 and
a standalone daemon respawner.

If and when I get a use case requiring more durability of processes and
what

Re: s6 as a systemd alternative

[Steve Litt]

On Fri, 30 Jun 2017 19:50:17 +
"Laurent Bercot"  wrote:

> >The runsv executable is pretty robust, so it's unlikely to die.  
>   Yadda yadda yadda. Most daemons are also unlikely to die, so
> following your reasoning, I wonder why we're doing supervision in the
> first place. Hint: we're doing supervision because we are not content
> with "unlikely". We want "impossible".

You want impossible. I'm quite happy with unlikely. With my use
case, rebooting my computer doesn't ruin my whole day. If it *did* ruin
my whole day, my priorities would be changed and I'd switch to s6.

> 
> 
> >  As far
> >as somebody killing it accidentally or on purpose with the kill
> >command, that's a marginal case. But if it were *really* important to
> >protect against, fine, have one link dir per early longrun, and run
> >an individual runsvdir on each of those link directories.  
>   And you just increased the length of the chain while adding no
> guarantee at all, because now someone can just kill that runsvdir
> first and then go down the chain, like an assassin starting with the
> bodyguards of the bodyguards of the important people. Or the assassin
> might just use a bomb and blow up the whole house in one go: kill -9
> -1.
> 
>   The main point of supervision is to provide an absolute guarantee
> that some process tree will always be up, no matter what gets killed
> in what order, and even if everything is killed at the same time. 

To me, the preceding isn't the main point of supervision. Supervision
benefits I value more are:

* Run my daemon in foreground, so homegrown daemons have no need to
  self-background.
* Consistent and easy handling of log files.
* Under almost all circumstances, dead daemons get restarted.
* Simple config and troubleshooting, lots of test points.
* POSIX methodologies ensure I can easily do special stuff with it.
* Ability to base process dependency on whether the dependee is
  *really* doing its job.

> You
> can only achieve that guarantee by rooting your supervision tree in
> process 1.

Yes.

> 
>   With runit, only the main runsvdir is supervised - and even then it
> isn't really, because when it dies runit switches to stage 3 and
> reboots the machine. Which is probably acceptable behaviour, but
> still not supervision. 

If we're going to get into definitions, then let me start by saying
what I want is daemontools that comes up automatically when the machine
is booted. Whether or not that's supervision isn't something I care
about.

> And everything running outside of that main
> runsvdir is just hanging up in the air - they can be easily killed
> and will not return.

Well, if they kill the runsv that's true, but if they kill the
daemon, no. Either way, I'm willing to live with it.


> 
>   By adding supervisors to supervisors, you are making probabilistic
> statements, and hoping that nobody will kill all the processes in the
> wrong order. But hope is not a strategy. There is, however, a strategy
> that works 100% of the time, and that is also more lightweight because
> it doesn't require long supervisor chains: rooting the supervision
> tree in process 1. That is what an s6-based init does, and it
> provides real, strong supervision; and unlike with runit, the machine
> is only rebooted when the admin explicitly decides so.

I completely understand your point. I just don't need that level of
indestructibility.

> 
>   If you're not convinced: *even systemd* does better than your
> solution. systemd obviously has numerous other problems, but it does
> the "root the supervision tree in process 1" thing right.

LOL, my whole point is I don't necessarily think "root the supervision
tree in process 1" is right, at least for my use case. I *enjoy* having
a tiny, do-almost-nothing PID1.

Like I said before, if losing control of the system during special
circumstances would ruin my whole day, I'd change my priorities and use
s6.

> 
>   I appreciate your enthusiasm for supervision suites. I would
> appreciate it more if you didn't stop halfway from understanding
> everything they bring, and if you didn't paint your unwillingness to
> learn more as a technical argument, which it definitely is not, while
> flippantly dismissing contributions from people who know what they
> are talking about.

But I didn't flippantly dismiss anybody or any contributions. I
pointed  out that one can, and I'll use different verbiage now, respawn
daemons early in the boot, before some of the one-shots had started.

I'm not an enemy of s6. I'm not an enemy of anything you apply the word
"supervision" to. I think I understand your reasons for doing what you
do. It's just that with my current use case, I've traded some of s6's
process and boot security (you know what I mean) for a simpler PID1 and
a standalone daemon respawner.

If and when I get a use case requiring more durability of processes and
what runs them, I'll for sure use s6 for that.
 
SteveT

Steve Litt 
June 2017 featured book: The Key to Ev

Re: s6 as a systemd alternative

[Laurent Bercot]

The runsv executable is pretty robust, so it's unlikely to die.

 Yadda yadda yadda. Most daemons are also unlikely to die, so following
your reasoning, I wonder why we're doing supervision in the first place.
Hint: we're doing supervision because we are not content with
"unlikely". We want "impossible".



 As far
as somebody killing it accidentally or on purpose with the kill
command, that's a marginal case. But if it were *really* important to
protect against, fine, have one link dir per early longrun, and run an
individual runsvdir on each of those link directories.

 And you just increased the length of the chain while adding no
guarantee at all, because now someone can just kill that runsvdir first
and then go down the chain, like an assassin starting with the
bodyguards of the bodyguards of the important people. Or the assassin
might just use a bomb and blow up the whole house in one go: kill -9 -1.

 The main point of supervision is to provide an absolute guarantee
that some process tree will always be up, no matter what gets killed
in what order, and even if everything is killed at the same time. You
can only achieve that guarantee by rooting your supervision tree in
process 1.

 With runit, only the main runsvdir is supervised - and even then it
isn't really, because when it dies runit switches to stage 3 and reboots
the machine. Which is probably acceptable behaviour, but still not
supervision. And everything running outside of that main runsvdir is
just hanging up in the air - they can be easily killed and will not
return.

 By adding supervisors to supervisors, you are making probabilistic
statements, and hoping that nobody will kill all the processes in the
wrong order. But hope is not a strategy. There is, however, a strategy
that works 100% of the time, and that is also more lightweight because
it doesn't require long supervisor chains: rooting the supervision tree
in process 1. That is what an s6-based init does, and it provides real,
strong supervision; and unlike with runit, the machine is only rebooted
when the admin explicitly decides so.

 If you're not convinced: *even systemd* does better than your solution.
systemd obviously has numerous other problems, but it does the
"root the supervision tree in process 1" thing right.

 I appreciate your enthusiasm for supervision suites. I would appreciate
it more if you didn't stop halfway from understanding everything they
bring, and if you didn't paint your unwillingness to learn more as a
technical argument, which it definitely is not, while flippantly
dismissing contributions from people who know what they are talking
about.

--
 Laurent

Re: s6 as a systemd alternative

[Steve Litt]

On Thu, 29 Jun 2017 19:57:55 -0300
Guillermo  wrote:

> 2017-06-29 1:43 GMT-03:00 Steve Litt:
> >
> > On Wed, 28 Jun 2017 22:31:12 -0300 Guillermo wrote:  
> >>
> >> But then you end up with an unsupervised runsv process,
> >> disconnected from the rest of the supervision tree...  
> >
> > I'm not sure about the unsupervised part of it, but I know if the
> > executable ends another one is run within a second, and I know that
> > you can turn the executable on and off with
> >
> > sv [up|down] /var/svlink/gnumeric  
> 
> The executable (gnumeric) is supervised, the supervisor (runsv) is
> not. Sure, all those things are true, but while the supervisor is
> alive. Kill runsv (if launched in this way) and see what happens.

The runsv executable is pretty robust, so it's unlikely to die. As far
as somebody killing it accidentally or on purpose with the kill
command, that's a marginal case. But if it were *really* important to
protect against, fine, have one link dir per early longrun, and run an
individual runsvdir on each of those link directories. So now you have
the same opportunity to kill each early longrun's supervision as you
have to kill the main supervision. I think that's huge overkill, but it
could be done.

Which is the real beauty of daemontools-inspired inits: They're just
POSIX, so a halfway savvy admin can mold them to his or her exact
situation.

SteveT

Steve Litt 
June 2017 featured book: The Key to Everyday Excellence
http://www.troubleshooters.com/key

Re: s6 as a systemd alternative

[Guillermo]

2017-06-29 1:43 GMT-03:00 Steve Litt:
>
> On Wed, 28 Jun 2017 22:31:12 -0300 Guillermo wrote:
>>
>> But then you end up with an unsupervised runsv process, disconnected
>> from the rest of the supervision tree...
>
> I'm not sure about the unsupervised part of it, but I know if the
> executable ends another one is run within a second, and I know that you
> can turn the executable on and off with
>
> sv [up|down] /var/svlink/gnumeric

The executable (gnumeric) is supervised, the supervisor (runsv) is
not. Sure, all those things are true, but while the supervisor is
alive. Kill runsv (if launched in this way) and see what happens.

There is a situation just like this with OpenRC's supervise-daemon.

G.

Re: s6 as a systemd alternative

[Jonathan de Boyne Pollard]

Charles Duffy
> - Integration with the linux-only cgroups mechanism for managing CPU,
> memory, and I/O throughput limits

As you say, this is just an exercise in chain loading tools and conversion to
scripts that employ them.

* https://news.ycombinator.com/item?id=11846083

* http://jdebp.eu./Softwares/nosh/guide/move-to-control-group.html

Re: s6 as a systemd alternative

[Jonathan de Boyne Pollard]

Casper Ti. Vector:

> 
> (Normally Jonathan would be replying to this point, but I still do not see
> him in this thread, so I rashly take this job ;)
> 

Goodonyer.  Thank you.

I have been distracted.

* https://github.com/neovim/neovim/pull/6816
https://github.com/neovim/neovim/pull/6816

I have got back to 1.34 and added some more bits and pieces this week.  Looking
at the changelog, it is probably time to release that and begin work on 1.35.

Re: s6 as a systemd alternative

[Steve Litt]

On Wed, 28 Jun 2017 22:31:12 -0300
Guillermo  wrote:

> 2017-06-28 14:40 GMT-03:00 Steve Litt:
> >
> > On Mon, 26 Jun 2017 14:53:50 + "Laurent Bercot" wrote:  
> >>
> >>   The problem with the runit model is that it is pure supervision -
> >> it does not provide service management. You have to run all your
> >> oneshots _before_ you can start longruns.  
> >
> > Not necessarily true. You can easily run longruns early by creating
> > a second service directory and a second link directory. You make the
> > service specific directory, create run script and the supervise
> > directory within it, and symlink like the following:
> >
> > ln -s /etc/sv2/gnumeric /var/svlink/gnumeric
> >
> > Then do the following:
> >
> > runsv /etc/svlink/gnumeric  
> 
> But then you end up with an unsupervised runsv process, disconnected
> from the rest of the supervision tree...

I'm not sure about the unsupervised part of it, but I know if the
executable ends another one is run within a second, and I know that you
can turn the executable on and off with 

sv [up|down] /var/svlink/gnumeric
 
SteveT

Steve Litt 
June 2017 featured book: The Key to Everyday Excellence
http://www.troubleshooters.com/key

Re: s6 as a systemd alternative

[Guillermo]

Hello,

2017-06-26 12:05 GMT-03:00 Istvan Szukacs:
>
> [...] I do not want
> logging, ntp and all the other crap that got sucked into it. I understand
> that service files are much better that shell scripts and this is a good
> argument but it does not justify the idiocracy that systemd became in the
> recent years. Ideally we could have a service (like s6) that does only that
> keeping the best parts of systemd as well. This would allow me to run
> redhat/centos based systems and use service files without being worried
> that a huge amount of CPU cycles going to a service that sole purpose is to
> keep services up that actually provide the functionality that I need. Does
> this clarify?

I'm partially repeating what others have said, but the way I see it:

You seem to be focused only on systemd unit files. An alternative init
system that understands unit files doesn't work as a drop-in
replacement.

1) systemd, the package, is not 'just' an init system. It is a whole
set of low-level userland components, that happens to include among
them a program intended to run as process 1 (also named 'systemd').
Some of the components can work without systemd (the program) being
process 1, like systemd-udevd, and maybe libsystemd (I don't know for
sure). Others, like systemd-logind, cannot. And software packages may
depend on any of those components. For example, GNOME desktop
components do not actually care about the init system, but do want to
be able to send messages over D-Bus to a server implementing the
logind API (as far as I understand).

2) Package dependencies and binary-based distributions. A lot of
software packages that have a real dependency on systemd (a package
that only provides a unit file does not qualify as having a
dependency), have an optional compile-time one (e.g. './configure
--with-systemd'). But if developers of a GNU/Linux distribution choose
to build the package with the option turned on, it becomes a mandatory
runtime dependency for the user. Maybe the dependency is on a
component that doesn't care who's process 1 and you are lucky, or
maybe not. And if that is done with every package shipped by the
distribution, the result is an entanglement with systemd you can't get
out of. So once a binary-based distribution decides to get married to
systemd, I think it is safe to assume that it is a one-way ticket.

So it might be possible to have GNU/Linux and not systemd, but highly
unlikely (IMO) if we are talking RHEL or CentOS.

> # rpm -q --requires openssh-server | grep systemd | sort -u
> libsystemd.so.0()(64bit)
> libsystemd.so.0(LIBSYSTEMD_209)(64bit)
> systemd-units
>
> I am not sure why any service would depend on these. Is there functionality
> in libsystemd.so.0 that an ssh service actually needs?

If that's the software from www.openssh.org, Gentoo's packaging of it
[1] does not list systemd as a dependency, not even conditionally on a
USE flag. I'm guessing that what is shown here is some indirect
dependency through PAM.

G.

[1] https://packages.gentoo.org/packages/net-misc/openssh

Re: s6 as a systemd alternative

[Guillermo]

2017-06-28 14:40 GMT-03:00 Steve Litt:
>
> On Mon, 26 Jun 2017 14:53:50 + "Laurent Bercot" wrote:
>>
>>   The problem with the runit model is that it is pure supervision -
>> it does not provide service management. You have to run all your
>> oneshots _before_ you can start longruns.
>
> Not necessarily true. You can easily run longruns early by creating a
> second service directory and a second link directory. You make the
> service specific directory, create run script and the supervise
> directory within it, and symlink like the following:
>
> ln -s /etc/sv2/gnumeric /var/svlink/gnumeric
>
> Then do the following:
>
> runsv /etc/svlink/gnumeric

But then you end up with an unsupervised runsv process, disconnected
from the rest of the supervision tree...

However, regarding Void Linux and udevd, as far as I can tell by
having a quick look at their GitHub repository, it is launched
unsupervised from /etc/runit/1 just to populate /dev, but then that
process is stopped using 'udevadm control --exit', and a supervised
one is launched. Probably the best possible solution if constrained by
the runit model...

G.

Re: s6 as a systemd alternative

[Alex Efros]

Hi!

On Wed, Jun 28, 2017 at 01:40:18PM -0400, Steve Litt wrote:
> The truth of the preceding statement depends entirely on your
> priorities. If you prioritize simplicity over software orthodoxy, built
> in process ordering, and a maximally recoverable boot instance, you'll
> prefer runit. That's why I prefer runit.
> 
> Runit sounds like it would have a lot more problems than it really
> does. I've used runit on Void for 2 years and have had no problems I
> could trace to any runit software.

+1

I believe there is a place for both runit and s6, and Steve is right, it's
just a question of personal preference.

I'm using runit with Gentoo with trivial /etc/runit/1 instead of default
Gentoo boot/rc scripts for… not sure how long, I believe ~13 years. I use
it this way both on workstations and servers.

Yeah, I run udevd not supervised. And know what? I never had any issues
because of this. And as Steve said this can be solved, but I'm too lazy to
invent /etv/sv2/ and supervise udevd while it works ok.

Moreover, there are a number of other non-supervised processes on my
workstation: mount.ntfs-3g, polkitd, dbus-launch, dbus-daemon, udisksd…
and also several user/GUI related (ssh-agent, xxkb, xscreensaver, dropbox…) -
up to 30 daemon-like processes in sum. Probably some of them will be
auto-restarted by their client software after crash, but not all of them.
So, udevd is "the only one left" only on servers, workstations have a lot
of other daemons, which are much harder or even impossible to supervise.

To be honest, from time to time I consider switching to s6, but this
require some amount of work while everything already works without issues.
And I do like simplicity of my ~250 lines /etc/runit/1.

-- 
WBR, Alex.

Re: s6 as a systemd alternative

[Steve Litt]

On Mon, 26 Jun 2017 14:53:50 +
"Laurent Bercot"  wrote:

> >a bunch of hackers behind Void Linux[2] made everything tick on top
> >of runit without all that much effort.  

[NOTE: Nothing I say in this reply should be interpreted as being
anti-s6, but instead be interpreted as pro-runit.]

> 
>   The problem with the runit model is that it is pure supervision -
> it does not provide service management. You have to run all your
> oneshots _before_ you can start longruns. 

Not necessarily true. You can easily run longruns early by creating a
second service directory and a second link directory. You make the
service specific directory, create run script and the supervise
directory within it, and symlink like the following:

ln -s /etc/sv2/gnumeric /var/svlink/gnumeric

Then do the following:

runsv /etc/svlink/gnumeric

Assuming that /etc and /var are on the root partition, the preceding
command can be done as early as you want, as long as the root partition
is mounted.

You can also run a one-shot very late, if willing to kludge. You simply
make the one-shot a long-run whose last command is a forever long
sleep. It's ugly, and it runs again if somebody does killall sleep, but
it works.

There's also a way to run two different instances of runsvdir, using
different link directories for each.


> See
>   https://skarnet.org/software/s6-rc/why.html and
>   https://fosdem.org/2017/schedule/event/s6_supervision/ (15 mn video)
> to understand what goes into an init system and why runit alone is
> not quite enough.
> 
>   Of course Void Linux manages to do it, but it is, pardon my French,
> a hack job: AFAICT, udevd is not supervised, 

As I mentioned above, making it an early long-run would be trivial.


> and some oneshots are
> implemented via longrun processes doing nothing. This is cramming
> the service management peg into the process supervision hole, 

True.

> and
> can never be perfect 

True.

> - Void would definitely benefit from switching
> to real service management. 

The truth of the preceding statement depends entirely on your
priorities. If you prioritize simplicity over software orthodoxy, built
in process ordering, and a maximally recoverable boot instance, you'll
prefer runit. That's why I prefer runit.

Runit sounds like it would have a lot more problems than it really
does. I've used runit on Void for 2 years and have had no problems I
could trace to any runit software.

> But so far they haven't expressed any
> desire to switch.

Different priorities. Runit sacrifices several features which seem
necessary but turn out not to be so necessary, for more simplicity than
s6, not that s6 is incredibly complicated. I'd be proud using either
one of them.

SteveT

Steve Litt 
June 2017 featured book: The Key to Everyday Excellence
http://www.troubleshooters.com/key

Re: s6 as a systemd alternative

[Luis Ressel]

On Wed, 28 Jun 2017 12:44:28 -0400
Steve Litt  wrote:

> 2) Install s6, but run it strictly as a process supervisor

You'd quickly run into dependency problems this way, though. There's a
reason why we have service managers such as s6-rc and anopa.

s6 is powerful on is own, so by all means, switch to it. But as soon as
you feel the need for oneshots or inter-service dependencies, keep in
mind that you'll probably be better of using s6-rc.

Cheers,
Luis Ressel

Re: s6 as a systemd alternative

[Steve Litt]

On Mon, 26 Jun 2017 18:17:29 +0300
Jean Louis  wrote:

> I am user of s6 for reason of simplicity and to avoid trouble of
> systemd.
> 
> While not being developer, maybe it could be possible to have s6 run
> systemd as a service 

Euuu!

> and systemd to run s6 as a service, 

This actually makes sense. One by one move daemons from the
unfathomable systemd to the completely understandable s6, and leave
your binary logs behind on the moved daemons :-)

SteveT

Steve Litt 
June 2017 featured book: The Key to Everyday Excellence
http://www.troubleshooters.com/key

Re: s6 as a systemd alternative

[Steve Litt]

On Mon, 26 Jun 2017 17:05:15 +0200
Istvan Szukacs  wrote:

> Hi Charles,
> 
> Yeah I can clarify. The only thing that I (or for that matter saner
> systems engineers) want from systemd is to be a better sysv init.

If what you want is a better sysv init, the world is your oyster:

* runit
* s6
* Epoch
* Suckless Init plus daemontools-encore (or any other supervisor)
* Sysvinit PID1 plus OpenRC plus supervisor for any longruns
* I've heard, but not personally confirmed, that busybox init is good

Making s6 "systemd compatible" would only lower s6's quality.

[snip]

> This would allow me to run redhat/centos based
> systems 

This is the crux of your problem. Redhat/centos are built from the
ground up, both technologically, economically, and monopolistically, to
use systemd. My advice would be that if there's a compelling reason for
you to use Redhat/centos, you stick with systemd. To do otherwise would
be to continually fight the system. If continuing to use Redhat/centos
isn't your top priority, I'd suggest you move to one of the many
sans-systemd distributions. With any distribution that uses any init
system other than systemd, it's pretty easy to replace that init system
with s6.

> and use service files without being worried

Sysvinit service files are responsible for untold heartache. Their
enigmatic complexity is what drove me to start using daemontools to
run processes a decade and a half ago. It's my personal belief that
sysvinit service files are what created the emotional opening for people
to unwisely switch to systemd. Many of these scripts are hundreds of
lines of cryptic bash code, and most of them would translate to run
scripts less than 10 lines long in runit, s6 or daemontools-encore.

My suggestion would be that for each daemon you run, you create a run
script, using the service files only to determine the args to use when
running the program (and don't forget to change the args to make the
program run in the foreground).

Here's my advice to you:

1) Switch to a sans-systemd distro

2) Install s6, but run it strictly as a process supervisor

3) 1 by 1, replace the sysvinit or openRC service file with an s6 run
   script, and then move that run script where it won't run in its
   original init.

4) Not necessary, but if you want to run a 100% s6 machine, switch PD1
   to the s6 PID1, and properly configure the s6 rc scripts. Install
   s6-rc to bring order to your boot process.


SteveT

Steve Litt 
June 2017 featured book: The Key to Everyday Excellence
http://www.troubleshooters.com/key

Re: s6 as a systemd alternative

[Casper Ti. Vector]

(Normally Jonathan would be replying to this point, but I still do not
see him in this thread, so I rashly take this job ;)

On Mon, Jun 26, 2017 at 05:05:15PM +0200, Istvan Szukacs wrote:
> I understand that service files are much better that shell scripts

Actually they are not.  This statement might appear logical when you see
[1] or some similar comparisons made by systemd proponents; however,
there are some problems with such comparisons, and here are the ones
that come to my mind at this time:
* The script on the left is a script for sysv-rc; sysv-rc is,
  admittedly, a terrible system.  rc scripts for some other traditional
  rc systems are much cleaner, eg. that for openbsd [2], or that for
  openrc [3].
* Traditional rc systems still deal with double-forking daemons, using
  pid-files, which is an awkward mechanism [4].  By abandoning double
  fork, process state handling (and daemon logging) would be much more
  straightforward [5].
* systemd unit files may seem a little more descriptive (and verbose ;)
  than the scripts in [5].  However, by chainloading, you can use
  noncommutative operator combinations (like passing some information in
  some environment variables and then cleaning up the environment after
  use of this information in some initialisation), which is difficult to
  achieve with systemd unit files.  Implementing the ordering of
  operators by ordering of lines in the unit file would probably result
  in reinventing execline / nosh, in an ugly way (actually they already
  partially reinvented the shell: note the `ExecStart*' fields in the
  unit file specification).
* systemd implements the mechanism for process state manipulation in
  pid-1, which leads to considerable safety/security implications.  By
  chainloading, all of these can be factored into the run script,
  without increasing complexity of the user interface.

[1] .
[2] .
[3] 
.
[4] .
[5] .

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: s6 as a systemd alternative

[Laurent Bercot]

 Would
this approach keep systemd installed just to provide for services 
directly

depending on it?

# rpm -q --requires openssh-server | grep systemd | sort -u
libsystemd.so.0()(64bit)
libsystemd.so.0(LIBSYSTEMD_209)(64bit)
systemd-units

I am not sure why any service would depend on these. Is there 
functionality

in libsystemd.so.0 that an ssh service actually needs?


 Not really, but that is exactly what I'm talking about when I speak
of vendor lock-in.
 libsystemd provides some library functions that applications can use
to interface themselves with systemd and allow it to provide advanced
functionality such as fd-holding, readiness notification, etc. But
all that functionality could be achieved in another way, without
needing a specific library: s6 provides tools to perform fd-holding
and an interface to perform readiness notification without needing
the application to link against a specific library.
 The fact that systemd encourages applications to link against
libsystemd is one of the way it ties itself to a system and makes
itself unremovable. That is the definition of vendor lock-in; that is
also, incidentally, the definition of cancer.

 You will not be able to entirely remove systemd from a distribution
that has chosen to embrace it - not until there is a will from the
higher technical management of that distribution to revert the
process and untie application software from systemd interfaces.
 Switching a distribution such as Fedora or Ubuntu to a different
init system is really difficult work. My approach is to first work
with other distributions, such as Alpine, Void or Devuan, which are
systemd-free, and get an integrated s6 init system working on those,
and get traction. Only when we have a fully integrated, easy to
use, widely spread platform will we be able to start turning the
tide and reclaim distributions from the systemd grasp. But this will
likely not happen for several years.

--
 Laurent

Re: s6 as a systemd alternative

[Jean Louis]

On Mon, Jun 26, 2017 at 04:02:47PM +0200, Istvan Szukacs wrote:
> Hi,
> 
> I have a crazy question about s6. Would it be possible to make systemd
> compatible? This question might sound stupid at first but here is the
> reasoning:
> 
> - we have services with systemd service files already
> (/etc/systemd/system/*.service, etc.)
> - the previous alternatives all failed to gain traction because there was
> too much effort to change systems around to use them (
> https://www.tecmint.com/best-linux-init-systems/) and the linux platform is
> very fragmented
> - there is already too much effort went into systemd that would be hard to
> duplicate
> 
> Questions:
> 
> - is there anybody interested in such project?
> - is this feasible to do?

I am user of s6 for reason of simplicity and to avoid trouble of
systemd.

While not being developer, maybe it could be possible to have s6 run
systemd as a service and systemd to run s6 as a service, with
modifications, but I think, it will be better to hit oneself with hard
rock to the head, it would be more pleasure.

Jean

Re: s6 as a systemd alternative

[Istvan Szukacs]

Hi Laurent,

First and foremost, thank you for working on s6! I see your point about the
drop-in replacement. I was not familiar with the efforts of uselessd. It
seems I am not the first one trying to have a usable system without
systemd. :) If I understand your suggestion correctly you are thinking
about having s6 as the init and using s6-rc for running the services. Would
this approach keep systemd installed just to provide for services directly
depending on it?

# rpm -q --requires openssh-server | grep systemd | sort -u
libsystemd.so.0()(64bit)
libsystemd.so.0(LIBSYSTEMD_209)(64bit)
systemd-units

I am not sure why any service would depend on these. Is there functionality
in libsystemd.so.0 that an ssh service actually needs?

I.



On Mon, Jun 26, 2017 at 4:48 PM, Laurent Bercot  wrote:

> I have a crazy question about s6. Would it be possible to make systemd
>> compatible?
>>
>
>  It all depends on what you mean by "compatible".
>
>  There will never be full drop-in compatibility, because only systemd
> can be drop-in compatible with systemd. It has been designed this way,
> its architecture is a way of ensuring vendor lock-in. The most
> successful "compatible systemd replacement" so far was uselessd, and
> its author decided to put an end to the project when they realized how
> infeasible, and pointless, the work was: when you follow the systemd
> architecture, you end up with systemd, no way around it.
>
>  What *would be* feasible, on the other hand, is a conversion tool
> from systemd unit files to a s6 + s6-rc database. It is still a lot
> of work, especially if one wants to implement all the systemd
> idiosyncrasies, but it is doable. I can't guarantee the output of
> such a conversion tool would be 100% compatible with the observed
> behaviour under systemd (in particular, bug-compatibility would not
> be ensured ;)) but it could probably be made to be close. nosh
> already has such a converter; I haven't tested it.
>
>  I am currently thinking of implementing a file format for
> s6 + s6-rc + s6-linux-init service definition, in order to make it
> easier for people used to OpenRC or systemd to implement services
> with s6. This format probably cannot be systemd unit files, because
> unit files follow the systemd architecture too closely; but it could
> be a first step to a systemd-to-s6 conversion tool.
>
>
> - the previous alternatives all failed to gain traction because there was
>> too much effort to change systems around to use them
>>
>
>  That's not an argument, especially given the following one:
>
> - there is already too much effort went into systemd
>>
>
>  If people actually did the effort to write systemd unit files, there
> is no reason why they can't do the effort to write service definitions
> under a better format in order to have a better init system. The format
> just needs to be friendlier than what the current s6 suite provides.
>
>  Working on conversion tools is currently my priority with s6. As usual,
> the limitation is my available free time.
>
> --
>  Laurent
>
>


-- 
*Istvan Szukacs*
CTO

+31647081521 
+36 70 229 20 59 
ist...@streambrightdata.com

Re: s6 as a systemd alternative

[Istvan Szukacs]

Hi Charles,

Yeah I can clarify. The only thing that I (or for that matter saner systems
engineers) want from systemd is to be a better sysv init. I do not want
logging, ntp and all the other crap that got sucked into it. I understand
that service files are much better that shell scripts and this is a good
argument but it does not justify the idiocracy that systemd became in the
recent years. Ideally we could have a service (like s6) that does only that
keeping the best parts of systemd as well. This would allow me to run
redhat/centos based systems and use service files without being worried
that a huge amount of CPU cycles going to a service that sole purpose is to
keep services up that actually provide the functionality that I need. Does
this clarify?

I see your point about the linux only things though. I am not sure what is
the right approach there. I should dig deeper into Amazon Linux and see how
they escaped systemd-hell.

I.


On Mon, Jun 26, 2017 at 4:47 PM, Charles Duffy  wrote:

> Could you be more clear about what you mean by "make systemd compatible"?
> Do you mean loading systemd configuration files into s6, or the reverse?
> The former strikes me as exceedingly difficult to implement in a complete
> and correct manner.
>
> One of the things that makes systemd so... controversial... is the amount
> of complexity it pulls into what (in many folks' view) ought to be designed
> as a simple subsystem (in the "simple enough that an implementation can be
> obviously correct" sense). Because of that amount of complexity -- one
> could rather easily implement a simple subset of its functionality, but
> reaching full parity (and *maintaining* that parity as it continues to
> grow, expand, and cross what would historically be distinct subsystem
> boundaries) strikes me as a very ambitious project.
>
> As a few examples of things that systemd provides that would need to be
> reimplemented:
>
> - A mechanism based on UNIX domain sockets for implementing a watchdog,
> and for processing file descriptors between subsequent invocations of the
> same service.
> - Sandboxing of allowed syscalls (using a Linux-only mechanism not
> applicable to other platforms s6 supports)
> - Management of process-local filesystem, PID, and user namespaces (again,
> using a Linux-only mechanism)
> - Integration with a (again, linux-only and glibc-only) nsswitch module to
> generate dynamic, transient user accounts local to an individual instance
> of a process
> - Integration with the linux-only cgroups mechanism for managing CPU,
> memory, and I/O throughput limits
>
> ...and so on, and so forth. Consequently, any effort would necessarily be
> a small subset, and plagued by compatibility issues whenever a distributed
> .service file attempts to use functionality that a different process
> supervision system couldn't implement without compromising portability (or
> changing its behavior between platforms).
>
> On Mon, Jun 26, 2017 at 9:02 AM Istvan Szukacs <
> ist...@streambrightdata.com> wrote:
>
>> Hi,
>>
>> I have a crazy question about s6. Would it be possible to make systemd
>> compatible? This question might sound stupid at first but here is the
>> reasoning:
>>
>> - we have services with systemd service files already
>> (/etc/systemd/system/*.service, etc.)
>> - the previous alternatives all failed to gain traction because there was
>> too much effort to change systems around to use them (
>> https://www.tecmint.com/best-linux-init-systems/) and the linux platform
>> is
>> very fragmented
>> - there is already too much effort went into systemd that would be hard to
>> duplicate
>>
>> Questions:
>>
>> - is there anybody interested in such project?
>> - is this feasible to do?
>>
>> Let me know what you think.
>>
>> Thanks in advance,
>> Istvan
>>
>> --
>> *Istvan Szukacs*
>> CTO
>>
>> ist...@streambrightdata.com
>>
>


-- 
*Istvan Szukacs*
CTO

+31647081521 
+36 70 229 20 59 
ist...@streambrightdata.com

Re: s6 as a systemd alternative

[Istvan Szukacs]

Thanks, I was not aware of this project.

I.

On Mon, Jun 26, 2017 at 4:39 PM, Kamil Cholewiński 
wrote:

> Hi Istvan,
>
> check out nosh[1]. It explicitly states systemd compat as one of its
> goals and goes to great lenghts to make it happen.
>
> [1]: https://jdebp.eu/Softwares/nosh/
>
> It preserves the daemontools style and spirit.
>
> Meanwhile, while hordes of programmers are busy making everything fit
> into the systemd-world (and fixing bugs they've created in the process),
> a bunch of hackers behind Void Linux[2] made everything tick on top of
> runit without all that much effort.
>
> [2]: https://www.voidlinux.eu/
>
> <3,K.
>



-- 
*Istvan Szukacs*
CTO

+31647081521 
+36 70 229 20 59 
ist...@streambrightdata.com

Re: s6 as a systemd alternative

[Laurent Bercot]

a bunch of hackers behind Void Linux[2] made everything tick on top of
runit without all that much effort.


 The problem with the runit model is that it is pure supervision -
it does not provide service management. You have to run all your
oneshots _before_ you can start longruns. See
 https://skarnet.org/software/s6-rc/why.html and
 https://fosdem.org/2017/schedule/event/s6_supervision/ (15 mn video)
to understand what goes into an init system and why runit alone is
not quite enough.

 Of course Void Linux manages to do it, but it is, pardon my French,
a hack job: AFAICT, udevd is not supervised, and some oneshots are
implemented via longrun processes doing nothing. This is cramming
the service management peg into the process supervision hole, and
can never be perfect - Void would definitely benefit from switching
to real service management. But so far they haven't expressed any
desire to switch.

--
 Laurent

Re: s6 as a systemd alternative

[Charles Duffy]

Could you be more clear about what you mean by "make systemd compatible"?
Do you mean loading systemd configuration files into s6, or the reverse?
The former strikes me as exceedingly difficult to implement in a complete
and correct manner.

One of the things that makes systemd so... controversial... is the amount
of complexity it pulls into what (in many folks' view) ought to be designed
as a simple subsystem (in the "simple enough that an implementation can be
obviously correct" sense). Because of that amount of complexity -- one
could rather easily implement a simple subset of its functionality, but
reaching full parity (and *maintaining* that parity as it continues to
grow, expand, and cross what would historically be distinct subsystem
boundaries) strikes me as a very ambitious project.

As a few examples of things that systemd provides that would need to be
reimplemented:

- A mechanism based on UNIX domain sockets for implementing a watchdog, and
for processing file descriptors between subsequent invocations of the same
service.
- Sandboxing of allowed syscalls (using a Linux-only mechanism not
applicable to other platforms s6 supports)
- Management of process-local filesystem, PID, and user namespaces (again,
using a Linux-only mechanism)
- Integration with a (again, linux-only and glibc-only) nsswitch module to
generate dynamic, transient user accounts local to an individual instance
of a process
- Integration with the linux-only cgroups mechanism for managing CPU,
memory, and I/O throughput limits

...and so on, and so forth. Consequently, any effort would necessarily be a
small subset, and plagued by compatibility issues whenever a distributed
.service file attempts to use functionality that a different process
supervision system couldn't implement without compromising portability (or
changing its behavior between platforms).

On Mon, Jun 26, 2017 at 9:02 AM Istvan Szukacs 
wrote:

> Hi,
>
> I have a crazy question about s6. Would it be possible to make systemd
> compatible? This question might sound stupid at first but here is the
> reasoning:
>
> - we have services with systemd service files already
> (/etc/systemd/system/*.service, etc.)
> - the previous alternatives all failed to gain traction because there was
> too much effort to change systems around to use them (
> https://www.tecmint.com/best-linux-init-systems/) and the linux platform
> is
> very fragmented
> - there is already too much effort went into systemd that would be hard to
> duplicate
>
> Questions:
>
> - is there anybody interested in such project?
> - is this feasible to do?
>
> Let me know what you think.
>
> Thanks in advance,
> Istvan
>
> --
> *Istvan Szukacs*
> CTO
>
> ist...@streambrightdata.com
>

Re: s6 as a systemd alternative

[Laurent Bercot]

I have a crazy question about s6. Would it be possible to make systemd
compatible?


 It all depends on what you mean by "compatible".

 There will never be full drop-in compatibility, because only systemd
can be drop-in compatible with systemd. It has been designed this way,
its architecture is a way of ensuring vendor lock-in. The most
successful "compatible systemd replacement" so far was uselessd, and
its author decided to put an end to the project when they realized how
infeasible, and pointless, the work was: when you follow the systemd
architecture, you end up with systemd, no way around it.

 What *would be* feasible, on the other hand, is a conversion tool
from systemd unit files to a s6 + s6-rc database. It is still a lot
of work, especially if one wants to implement all the systemd
idiosyncrasies, but it is doable. I can't guarantee the output of
such a conversion tool would be 100% compatible with the observed
behaviour under systemd (in particular, bug-compatibility would not
be ensured ;)) but it could probably be made to be close. nosh
already has such a converter; I haven't tested it.

 I am currently thinking of implementing a file format for
s6 + s6-rc + s6-linux-init service definition, in order to make it
easier for people used to OpenRC or systemd to implement services
with s6. This format probably cannot be systemd unit files, because
unit files follow the systemd architecture too closely; but it could
be a first step to a systemd-to-s6 conversion tool.


- the previous alternatives all failed to gain traction because there 
was

too much effort to change systems around to use them


 That's not an argument, especially given the following one:


- there is already too much effort went into systemd


 If people actually did the effort to write systemd unit files, there
is no reason why they can't do the effort to write service definitions
under a better format in order to have a better init system. The format
just needs to be friendlier than what the current s6 suite provides.

 Working on conversion tools is currently my priority with s6. As usual,
the limitation is my available free time.

--
 Laurent

Re: s6 as a systemd alternative

[Kamil Cholewiński]

Hi Istvan,

check out nosh[1]. It explicitly states systemd compat as one of its
goals and goes to great lenghts to make it happen.

[1]: https://jdebp.eu/Softwares/nosh/

It preserves the daemontools style and spirit.

Meanwhile, while hordes of programmers are busy making everything fit
into the systemd-world (and fixing bugs they've created in the process),
a bunch of hackers behind Void Linux[2] made everything tick on top of
runit without all that much effort.

[2]: https://www.voidlinux.eu/

<3,K.

s6 as a systemd alternative

[Istvan Szukacs]

Hi,

I have a crazy question about s6. Would it be possible to make systemd
compatible? This question might sound stupid at first but here is the
reasoning:

- we have services with systemd service files already
(/etc/systemd/system/*.service, etc.)
- the previous alternatives all failed to gain traction because there was
too much effort to change systems around to use them (
https://www.tecmint.com/best-linux-init-systems/) and the linux platform is
very fragmented
- there is already too much effort went into systemd that would be hard to
duplicate

Questions:

- is there anybody interested in such project?
- is this feasible to do?

Let me know what you think.

Thanks in advance,
Istvan

-- 
*Istvan Szukacs*
CTO

ist...@streambrightdata.com