Re: s6 usability (was: runit patches to fix compiler warnings on RHEL 7)

[Casper Ti. Vector]

On Sun, Dec 01, 2019 at 09:47:52PM -0500, Steve Litt wrote:
> Would it be acceptable to you and them to put the binaries in /bin/s6
> and then very early in the boot add /bin/s6 to the path? This isn't a
> lot different from what djb did with /command, except it's not off the
> root, which everyone seems to hate.

Or can we consider the Plan 9 style [1], which searches all relative
paths (in the broad sense, i.e. all which do not begin with `/', `./' or
`../') from $PATH, so we can install chainloaders into /bin/s6 and then
use `s6/cd' to run `/bin/s6/cd' in execline?

[1] 

I fully understand that this convention is not followed in most Unix
shells (except for rc(1) which is from Plan 9, and perhaps some others),
but execline is not a shell, and we can additionally symlink /bin/s6/*
into /bin for backward compatibility.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: s6 usability (was: runit patches to fix compiler warnings on RHEL 7)

[Laurent Bercot]

sure, that was just an idea for Jan, he could just create a dir somewhere,
populate it with symlinks he prefers to the original s6 tools and put this dir
in front of the PATH when running s6 since it seems the utilities do not bother 
under what name they run.


Right, but I've heard enough people complain about s6's UI that a 
one-stop

wrapper command sounds like a good idea anyway.



ok. i was more about insights into the design of the whole s6-rc toolset.
are the up/down scripts run by a dedicated service from within the supervision
tree? what exactly is the task of the "s6-rc-oneshot-run" and
"s6-rc-fdholder-filler" internal programs ? how is the startup organized,
how are "longruns" and "oneshots" intertwined ?

having to read the sources to get this information is somewhat inconvenient.


 This is exactly what I was saying: I'm not documenting those details
because I don't want to be bound by them. The s6-rc-fdholder-filler API
changed right before 0.2.0.0; making this necessary change would have 
been

a lot more difficult if people had relied on details of the interface.

 If you're doing something that requires knowledge of the internal 
programs,

you're definitely good enough to read the code. :P

--
 Laurent

Re: s6 usability (was: runit patches to fix compiler warnings on RHEL 7)

[Laurent Bercot]

Would it be acceptable to you and them to put the binaries in /bin/s6
and then very early in the boot add /bin/s6 to the path? This isn't a
lot different from what djb did with /command, except it's not off the
root, which everyone seems to hate.


s6 binaries aren't a problem for Debian; but apparently, execline
binaries are. They're doing exactly that: they're storing execline
binaries under /usr/lib/execline - except they're not putting that
directory in the default PATH, they're only adding it to PATH when
execlineb is invoked. Which breaks some s6 commands in older s6 
versions,

and breaks s6-rc commands (any version), because those assume that
execline binaries can be found in their PATH, and don't call execlineb.

It's completely acceptable to me to put binaries in a separate 
directory.

But said separate directory *needs* to be in the default PATH. If it is
not, it means that those binaries are considered second-class citizens,
and that is *not* acceptable - and even more importantly, it breaks
things.



As a guy who has both daemontools and s6 installed on the same box, I
thank you from the bottom of my heart for:

1) Prepending s6- to each command so they don't clash with djb's
2) Except for the s6-, naming them the same as djb's so I have less to
   remember.


Yes, there are a good number of people, me included, who prefer that
naming scheme. However, Jan's UX return is valid, and if I want to make
s6 adoption as easy as possible, it needs to be taken into account too.



A simple change change I think would do it is to change the
documentation to imply that, for the *user*, execlineb is a way to get
just a little extra whatever. Currently, when I read it, I thought I'd
be missing a lot by using /bin/sh.


I think it's already the case, but I'll make a pass on the documentation
to make it abundantly clear.



Does the *user* need to code execline scripts, or is it just
something the program does? If the former, then make a point that one
doesn't need to use execline for s6-rc to be a very powerful startup
system.


 1. execline is used *internally* by s6-rc, in autogenerated scripts. 
The

user doesn't have to know anything about it.

 2. source oneshot up/down scripts are parsed by execlineb, so this is a
place where users interact with it. However, the interaction can be made
pretty minimal if the up/down script just calls a shell script stored
somewhere else (typically /etc/init.d), which is common, and generally
good, practice.
 For instance, the "source/network-interfaces/up" script could simply
contain:

 /etc/init.d/network start

and that's it. Technically, this is an execline script, made of two 
words,
but nobody needs to know that; and the complex logic of the script can 
be

implemented in the /etc/init.d/network program, which can be written in
any language. Even in a compiled language, if you're a masochist.

Advanced users can write more complex logic in the
source/network-interfaces/up script itself, using execline binaries for
control flow, etc. But it's by no means essential. For all intents and
purposes, that script is just a small command line that execs into 
another

script, hosted wherever you want, written in whatever language you want.
So I'd say the necessary interaction between s6-rc users and execline is
really tiny.



If anybody would make an execline tutorial, that would help a lot. For
a guy like me who only does procedural programming (C, C++, Pascal,
Perl, Python, Ruby, Lua, etc), execline is difficult to understand.


Well, for one, the execline documentation is bad (it was my first real
attempt at writing software documentation, and it shows), so revamping 
it

would probably help a lot. And second, there *are* execline tutorials
online, you just haven't looked.

--
Laurent

Re: s6 usability (was: runit patches to fix compiler warnings on RHEL 7)

[Jeff]

30.11.2019, 19:58, "Laurent Bercot" :
>> the solution here could be a simple symlink to the original s6 tool without
>> the prefix if you prefer (maybe even located in an other dir than /bin).
>
> That would be a decision for users, not software authors - else it would
> defeat the point of not invading the namespace. Daemontools is still
> around with names such as "svc".

sure, that was just an idea for Jan, he could just create a dir somewhere,
populate it with symlinks he prefers to the original s6 tools and put this dir
in front of the PATH when running s6 since it seems the utilities do not bother
under what name they run.
 
>> using a single combined tool is more efficient since it avoids wasteful
>> further exec chaining steps, though.
>
> Sure, but if we're talking about UI, optimization at this level is a
> very
> moot point. A human choosing between "chpst" and "s6-applyuidgid" will
> *not* notice the extra millisecond taken by an execve() step. The
> primary focus should be usability.

i prefer short names like "chpst" (change process state ?) with multiple
command line options from a usability perspective. but the usage of single
tools with descriptive names is of course easier to read (not to write) and
hence understand when they occur in a script, that's true.

> I am reluctant to make the ABI details public because I want the freedom
> to change them. If people start relying on internals, their stuff may
> break when updating, which would be bad.
> There are *some* details that I could document as official and stable,
> but I'd need to go through all of it and decide with precision what can
> be guaranteed and what cannot - and that's extra work, so it will have
> to wait.

ok. i was more about insights into the design of the whole s6-rc toolset.
are the up/down scripts run by a dedicated service from within the supervision
tree? what exactly is the task of the "s6-rc-oneshot-run" and
"s6-rc-fdholder-filler" internal programs ? how is the startup organized,
how are "longruns" and "oneshots" intertwined ?

having to read the sources to get this information is somewhat inconvenient.
:-(

Re: s6 usability (was: runit patches to fix compiler warnings on RHEL 7)

[Steve Litt]

On Sat, 30 Nov 2019 10:15:27 +
"Laurent Bercot"  wrote:


> I hear you. Unfortunately, I have no control over what Debian does.
> Debian isn't even able to ship a not-broken execline package, so I'm
> at a loss on what to do with them. I'm working on a version of
> execline that
> they *might* accept to package correctly, but it's doubtful as long as
> the people in charge are prejudiced against the "lots of small
> binaries in /bin" approach. :(

Would it be acceptable to you and them to put the binaries in /bin/s6
and then very early in the boot add /bin/s6 to the path? This isn't a
lot different from what djb did with /command, except it's not off the
root, which everyone seems to hate.

[snip] 

> >3) s6 executables are somehow worse named than runit's. This may be
> >highly subjective, but I can recall and recognize the runit
> > commands far easier than the s6 ones. Possibly it's the "s6-"
> > prefix getting in the way of my brain pattern matching on visual
> > appearance of glyph sequences.
> >This point is exacerbated by #2 and the number of s6 executables.
> >Compare chpst with s6-envdir s6-envuidgid s6-fghack s6-setsid
> >s6-setuidgid s6-applyuidgid s6-softlimit. Yes, I know about the
> >historical reasons, but still.  
> 
> This is very interesting. I thought that having a s6- prefix was a 
> *good*
> thing, because I valued clarity above everything, and especially above
> terseness. 

As a guy who has both daemontools and s6 installed on the same box, I
thank you from the bottom of my heart for:

1) Prepending s6- to each command so they don't clash with djb's
2) Except for the s6-, naming them the same as djb's so I have less to
   remember.

[snip]
> 
> >4) s6 seems more complex (hello execline!), and I don't (yet?) see
> >any
> >benefit/feature I'd appreciate except minimizing wakeups.  
> 
> This, on the other hand, is a misconception that really needs to
> disappear. Understanding execline is *not needed* to run s6. s6
> depends on execline in two places (there were more, but I scrapped
> them because nobody used the commands that were involved):
> - at build-time, in s6-ftrig-listen
> - at run-time, in s6-log, for processor invocation
> 
> Would entirely removing s6's dependency on execline help clear that
> misunderstanding and help with s6 adoption? This could be made
> possible by:
> - duplicating el_semicolon() functionality in s6-ftrig-listen.c
> (it's not elegant, but clearing the dep may be worth it)
> - adding an alternative '?' processor directive to s6-log, that spawns
> the processor using /bin/sh instead of execlineb. (The '!' directive
> would still be there; processor invocations using '!' would just fail
> if execline is not installed.)
> 
> I don't like this, but if "execline" is a scarecrow that keeps people
> away from s6 for no other reason than perception, it's a possibility.
> Savvy users will still install execline for use in run scripts.

I don't think it's necessary to remove the dependency unless ordinary
users would be altering the code of s6-ftrig-listen or s6-log.

A simple change change I think would do it is to change the
documentation to imply that, for the *user*, execlineb is a way to get
just a little extra whatever. Currently, when I read it, I thought I'd
be missing a lot by using /bin/sh.


> 
> s6-rc, however, absolutely cannot do without execline, since it uses
> autogenerated execline scripts. But s6-rc is a different beast, that
> requires a lot more involvement than s6 anyway, and that isn't needed
> at all if we're just talking about runit-like functionality.

Does the *user* need to code execline scripts, or is it just
something the program does? If the former, then make a point that one
doesn't need to use execline for s6-rc to be a very powerful startup
system.

If anybody would make an execline tutorial, that would help a lot. For
a guy like me who only does procedural programming (C, C++, Pascal,
Perl, Python, Ruby, Lua, etc), execline is difficult to understand.

SteveT

Steve Litt
November 2019 featured book: Manager's Guide to Technical
Troubleshooting Second edition
http://www.troubleshooters.com/mgr

Re: s6 usability (was: runit patches to fix compiler warnings on RHEL 7)

[Jeff]

30.11.2019, 11:15, "Laurent Bercot" :
> This is very interesting. I thought that having a s6- prefix was a *good*
> thing, because I valued clarity above everything, and especially above
> terseness. I understand the advantages of having commands named "sv" and
> "chpst", but I believed, in my naïveté, that it wasn't a good idea for a
> specialized package to tread on a large namespace; and to me the s6-
> prefix would help users recognize exactly the domain of the command
> they're using, and then they could abstract it away and focus on the
> real command name.

totally agreed, Laurent.

using a dedicated namespace prefix like "s6-" is a very good idea.
this avoids nameclashes (i. e. overwriting on installation) with similar
utilities of other supervision suites and frees Laurent from the task
of coming up with proper AND unique command names. consider
nameclashes of several "init" program for example.

the solution here could be a simple symlink to the original s6 tool without
the prefix if you prefer (maybe even located in an other dir than /bin).

> The number of executables is a choice; I like to have more, smaller,
> executables than less, bigger ones. One function, one tool. It makes
> code easier to write; this is not really for historical reasons, it's a
> design choice. Personally, it's easier for me to remember several
> process state change command names than all the options to chpst.
> whenever I use chpst, I always need to check the doc; when I use
> something like softlimit or setuidgid, I may need to check the doc for
> specific options, but I always remember which command I want and its
> general syntax. So, I suppose it comes down to individual preference
> there.

using a single combined tool is more efficient since it avoids wasteful further
exec chaining steps, though.

> Would a generic "s6" command, that takes alternative syntax and rewrites
> itself into "internal" commands, help? It could emulate runit syntax,
> among other things.
>
> s6 runsv ... -> s6-supervise ...
> s6 sv ... -> s6-svc ...
> s6 chpst ... -> various s6-prefixed process state change commands
>
> My plan is for the future s6-frontend package to include such a
> one-stop-shop command that controls various aspects of an s6
> installation,
> but if this command can help with s6 adoption, I can work on it much
> earlier than the rest of the s6-frontend functionality.
>
> Or, if you have other ideas that could help with easier assimilation of
> the s6 commands, I'm very open to suggestions.

Busy/ToyBox style ?

> Would entirely removing s6's dependency on execline help clear that
> misunderstanding and help with s6 adoption? This could be made possible
> by:
> - duplicating el_semicolon() functionality in s6-ftrig-listen.c
> (it's not elegant, but clearing the dep may be worth it)
> - adding an alternative '?' processor directive to s6-log, that spawns
> the processor using /bin/sh instead of execlineb. (The '!' directive
> would still be there; processor invocations using '!' would just fail
> if execline is not installed.)

sounds not too bad, IMO. though i personally can live without it,
especially since the other suites also provide loggers (without any
execline deps of course), he can use dt encore's "multilog" utility.

> s6-rc, however, absolutely cannot do without execline, since it uses
> autogenerated execline scripts.

could you document the way s6-rc works (i. e. its architecture) somewhere ?
or are users requested to follow your C code to find out how it works
exactly ?

> But s6-rc is a different beast, that
> requires a lot more involvement than s6 anyway, and that isn't needed
> at all if we're just talking about runit-like functionality.

indeed.

s6 usability (was: runit patches to fix compiler warnings on RHEL 7)

[Laurent Bercot]

As a relatively new convert to supervision software, my reasons for
preferring runit over s6 are, in order of priority:


Hi Jan,

Thank you a lot for this feedback. This is very useful UX return.
Let me address the points one by one.



1) Debian ships with a working and maintained runit-init package. It
   provides pid 1 and hooks into /etc/rcS.d/* to integrate with other
   Debian packages. s6-linux-init and s6-rc are not packaged in Debian.


I hear you. Unfortunately, I have no control over what Debian does.
Debian isn't even able to ship a not-broken execline package, so I'm at
a loss on what to do with them. I'm working on a version of execline 
that

they *might* accept to package correctly, but it's doubtful as long as
the people in charge are prejudiced against the "lots of small binaries
in /bin" approach. :(



2) runit has manpages. s6 has HTML. :(


This is far from the first time I hear this, so it would be foolish to
ignore it, but I *really* have a hard time understanding how in 2019 
it's

so difficult for people to launch a browser to read a web page. I just
can't get it. Launching a browser and reading a web page is something 
that
we all do, every day, even the least computer-savvy ones among us, and 
for
the life of me I can't fathom how this is *not* a friendly user 
interface.
HTML even has the advantages of hyperlinks, so you can jump around in 
the

documentation as needed, which you can't do with man pages!

Do you work offline? don't you have access to a web browser? do you like
reading stuff in a terminal *better*? (Text-based web browsers still
exist, if you do.)

It really stumps me. I did learn "man" too, 25 years ago, before the
Web existed. It was nice when man pages were all we had. But a couple of
years later, we had something that seems unarguably better to me, and
I see exactly zero reason to go back, apart from the force of habit of
typing "man".

Nevertheless, if people like man pages, they should have man pages, so
it's been a few years that I have appealed to the community for this.
I'm not going to learn nroff, ever, but we have a relatively large user
community, so they could probably contribute man pages, right? We've had
a few people who seemed interested, some of them even started thinking
about it *really hard*. And one of them even wrote a tool to convert
text into nroff, à la markdown (but simpler). But when it was about 
doing

the actual work of writing the man pages (even if all the material is
already here, in the HTML doc)? You guessed it: crickets.

So, yeah, I want s6 to be accessible, but I figure that if people
really wanted man pages, they'd write man pages ¯\_(ツ)_/¯ Or maybe I was
wrong all along and the spirit of Open Source really is "If the one 
person
doing the work doesn't do exactly what I want, then I'm just gonna sit 
on

my ass and blame them".

If I'm sounding a bit jaded, it's because I am.



3) s6 executables are somehow worse named than runit's. This may be
   highly subjective, but I can recall and recognize the runit commands
   far easier than the s6 ones. Possibly it's the "s6-" prefix getting
   in the way of my brain pattern matching on visual appearance of glyph
   sequences.
   This point is exacerbated by #2 and the number of s6 executables.
   Compare chpst with s6-envdir s6-envuidgid s6-fghack s6-setsid
   s6-setuidgid s6-applyuidgid s6-softlimit. Yes, I know about the
   historical reasons, but still.


This is very interesting. I thought that having a s6- prefix was a 
*good*

thing, because I valued clarity above everything, and especially above
terseness. I understand the advantages of having commands named "sv" and
"chpst", but I believed, in my naïveté, that it wasn't a good idea for a
specialized package to tread on a large namespace; and to me the s6-
prefix would help users recognize exactly the domain of the command
they're using, and then they could abstract it away and focus on the
real command name.
Now you're saying that the s6- prefix *impedes* your pattern recognition
and detracts you from easily mapping the command name to its 
functionality;

that having it is worse UI than not having it. I had not heard this
before, but it quite makes sense.

The number of executables is a choice; I like to have more, smaller,
executables than less, bigger ones. One function, one tool. It makes
code easier to write; this is not really for historical reasons, it's a
design choice. Personally, it's easier for me to remember several
process state change command names than all the options to chpst.
whenever I use chpst, I always need to check the doc; when I use
something like softlimit or setuidgid, I may need to check the doc for
specific options, but I always remember which command I want and its
general syntax. So, I suppose it comes down to individual preference
there.

Would a generic "s6" command, that takes alternative syntax and rewrites
itself into "internal" commands, help? It could emulate runit syntax,