Re: Debianization

[Avery Payne]

I've been living with rc-shim for several months now, and while it isn't
perfect, it's certainly taken a lot of the pain out of migration while
using SysV rc scripts.  I'm planning on switching off of runit this summer
and moving to s6, so this is both a "proof in the pudding" moment for me
because the shim is supposedly supervisor-agnostic.  There are two basic
issues that I've yet to solve that aren't cosmetic in nature:

1. Debian's updates will overwrite /etc/init.d/{name-of-service-script}
with a new copy each time a service is updated.  It's annoying but easily
fixed, simply "head -n25 /etc/init.d/{script}" and copy the header comments
into a new shim, and then copy the modified shim over the init script.
2. The shim doesn't detect service flapping.  This nasty condition happens
when the service starts, runs for 1 second, the shim wakes from sleep and
detects the service is OK (because the supervisor task reports back that it
is OK), and the shim exits with a successful return code; but the service
crashes a second later, and the supervisor then restarts it over and over
again.  You're left thinking there is a good start, but the service is
flapping like a wounded moth.  Ugly all the way around, and I don't have a
good fix yet.

Both of these are temporary problems when viewed as a stepping stone to a
pure s6 system, so it's not too big of a concern.  The real problem remains
the same for Debian - writing the hundreds of run script definitions to
cover all packages.

On Sat, Apr 21, 2018 at 1:19 PM, Didier Kryn  wrote:

>
> Whether starting from Alpine or Devuan, at least you don't have to
> fight against Systemd which has put metastases pretty much everywhere in
> the system. You start from sysvinit and just replace the rc scripts even
> with keeping sysv's init proper, at the beginning. A big part of the work
> is packaging and transitionning flawlessly from one system to the other
> when installing/removing the packages.
>
> Didier
>
>

Re: Debianization

[Avery Payne]

On Sat, Apr 21, 2018 at 4:27 AM, SZÉPE Viktor  wrote:

>
> Actually my first step of setting up a production Debian-based server is
> reverting back to SysVinit
> https://github.com/szepeviktor/debian-server-tools/blob/
> master/debian-setup.sh#L88-L90
> Currently I haven't reached a single boot, so it is a task for me to
> migrate the above listed services to *native* s6-rc.
>
> Are you looking for run files?  There are several stashed away in various
repositories. Are you looking to have run files for every Debian service
possible?  That is a large task - on Debian 7 there were ~ 900 packages
that would leave rc scripts in /etc/init.d that were not one-shots, so you
have your work cut out.  I'm sure the list has grown since then.

I humbly suggest the following:

1. convert to SysV, and start out using stock rc scripts inside /etc/init.d
2. create Debian packages for s6, and install from the packages.
3. determine where you are locating the directories "service", "svc", and
"svcdef" (or whatever you are calling them) and try to comply with Debian's
rules.  For Debian, I suggest you create the directories /etc/svc and
/etc/svcdef, and then make a symlink from /service pointing to /etc/svc,
which I'm sure is not 100% "Debian acceptable" but is probably the right
thing to do.
4. decide on a logging layout.  You can do anything you want, but to keep
things similar, I suggest that all services that don't do their own logging
receive a s6-log instance, using the T option instead of t (since Debian
doesn't support TAI64N), and that your s6-log output be directed to a file
inside of a directory named /var/log/{service name}.
5. decide on a strategy to replace syslog.  Maybe something that provides
/dev/log.  Not 100% needed, but nice to have.
6. decide on a strategy for udev.  You will need it to run at startup,
prior to switching run levels; I suggest a one-shot that sets up all of the
device names at startup, and a separate longrun that supervises udevd in
normal use.  Not optimal, but it should suffice.
7. decide on a strategy for dbus.  You can easily make a service definition
for it, but you'll have the unpleasant discovery of finding consolekit,
policykit, rtkit, and others being unsupervised. (some day I will peer into
the bowels of dbus and see if there is a way to make it play nice...)
8. decide on a strategy for pulseaudio, which is a worse pain than udev and
dbus together.  You'll have the fun choices like "it runs system wide but
desktop audio is buggy or fails", or "I don't supervise it but I have
desktop audio", or my favorite, "just suffer without it and install ALSA".
9. start writing run scripts.  Look around github for little repos here and
there, and look at what people are doing.  You can contact people in
private if you get stuck.  Be sure to bundle the scripts into a package
that can be re-installed.  Perhaps name it s6-debian-runscripts or
something similar.
10. if you are willing to put up with beta-grade software while you
transition, use rc-shim to replace the scripts in /etc/init.d one by one,
so you can cleanly boot with SysV AND have s6 supervising your services.  I
will warn you that it still has some rough edges, but it should be "good
enough" for the time being.  If you do decide to use rc-shim, you will want
to make a small modification that copies /etc/svcdef/{servicename} to
/etc/svc/{servicename} when it starts a service for the first time...
11. once you have run scripts for everything, you can make the move to a
pure s6 based system, and you don't need the rc-shim anymore, or SysV.  Yay!
12. please share your s6-debian-runscripts package, so that all Debian and
Devuan users can move to s6 as well. :)

That's quite the list to do, and I wish you the best of luck.

Fwd: execline and $0-based stuff

[Avery Payne]

On Sat, Feb 18, 2017 at 4:46 AM,  wrote:

> Hello there,
>
> I would like to extract a part of the path of the executed script ($0) in
> a variable, e.g. if the script was run as /some/path/getty-tty1/run, I
> would like to extract "tty1" from the path, or similary, extract "dropbear"
> from /some/path/dropbear-log/run. This would allow to write templated
> services quite easily using symlinks.


The template concept does work, but with limitations.  If you don't care
about portability between installations, then it is entirely possible to
use symlinks and a template script.  I had tried a project very similar to
your idea, but had to halt progress on it because it did not properly
isolate settings, resulting in a portability problem; it worked for me, but
only in the environment I was running at home.  The project is dormant for
the moment, but I will eventually return and re-vamp it away from templates
for portability reasons.  Feel free to look around and/or borrow any ideas
from it at https://bitbucket.org/avery_payne/supervision-scripts.

Re: [announce] s6-rc: a s6-based service manager for Unix systems

[Avery Payne]

On 9/24/2015 2:47 AM, Laurent Bercot wrote:

On 24/09/2015 04:54, Avery Payne wrote:

1. Copy the entire contents of the svcdef/ directory into the s6
source definition directory, including dot file support directories.


 There is no room in the source definition directory for a support
directory as svcdef/ ; I mean, you can put it anywhere, but
s6-rc-compile won't know what to do with it, and won't include it
in the compiled database.
That's why I said to copy the contents, not the directory. ;)  The 
contents of svcdef is, in purpose, the same as the source directory, but 
it's not in the right format for s6-rc, which is why I need all of the 
steps in the prior message.  If the steps are followed by running the 
soon-to-be-written "use-s6-rc" script, then the result will be a 
complete conversion to the appropriate format that s6-rc requires to 
compile the definitions, including support for dependencies.


I am still thinking about if logging should be allowed to "break" so 
they can piecemeal their own solution, or if I should "automagically" 
create logging definitions in the source directory that are s6-rc compliant.




 If your created services need to reference things in a global
support directory, the best practice is to install this support
directory into an official location, and to use the absolute path
to this location in your scripts.
This is a problem, as the entire set of definitions are meant to use 
relative paths, so that they can be relocated just about anywhere 
without requiring a bunch of "de-referencing" during the install 
process.  More on this in a moment.



s6-rc-compile will only treat
local data: if you have global data, you're in charge of it.
The only data that may be an issue is the assignment of user accounts 
other than root.  This is Yet Another Problem(tm) that I've been mulling 
over and I haven't found a satisfactory answer that would be portable 
between various platforms.  This would eventually manifest itself as a 
small text database along with a single script to set user/group 
definitions.  The script would (in theory) reset the meaning of T_UID 
and T_GID in all of the env/ directories of each definition, so that 
your run-time accounts match the accounts used by your distribution.






2. Use the install command to copy .bin, .finish, .log, and .run from
the source definition directory into the compiled definition
directory.


 I would highly recommend NOT messing with the compiled database
after it has been created. The format isn't fixed, and I want to
have complete freedom to change it.
This is probably the only real problem in the arrangement I'm using 
then.  The entire premise is "the support scripts live where the 
definitions live and are called with relative pathing, allowing the 
definitions to be located anywhere".  This was done precisely to avoid 
the mess of "you MUST have your scripts/support scripts live at this 
exact path", which was originally inspired by a message exchange we had 
months ago about separating policy from mechanism. For my definitions to 
work, those four dot-directories have to live where the run-time 
definitions live.  In this case, your compiled directory is the 
equivalent of a run-time directory.


I am certainly not adverse to changing their names or locations - the 
entire purpose is to have a set of definitions that work out of the box, 
and I need to make it do whatever is required to achieve that - but I 
cringe at the concept that I would need to use an absolute path, because 
I'll end up making everything brittle as a result.  I'm open to 
suggestions on how to fix this.




The only official interface
is the source format; you should do whatever is required to
whip up a working source definition directory that doesn't need
extra magic after the compilation phase.
The only magic needed is that the support directories have to be located 
in the same runtime location, so that the relative path links will align 
and everything will be found.  The compiled definitions are the 
equivalent of a runtime location, therefore, they must live there.



Once you have run
s6-rc-compile, the database is opaque stuff you can't modify;
you can move the compiled directory around, but don't mess with it.
I don't.  None of the definitions write files or data into themselves at 
runtime.  The only thing that changes is the symlinks that are used, and 
if you look at the instructions I gave, that step is performed in the 
source, not the compiled.  (Ok, yes, I create some files in the *source* 
directory, but that's necessary to support s6-rc and currently acceptable)




 Also, when I said that s6-rc-compile does not dereference symlinks,
I partially lied. It is true for whatever is in data/ and env/
subdirectories.
The issue I am trying to work around with the dereference in step (7) is 
that the env/ in my definitions is not a real directory.  It is a

Re: [announce] s6-rc: a s6-based service manager for Unix systems

[Avery Payne]

On 9/24/2015 12:29 PM, Laurent Bercot wrote:

On 24/09/2015 18:45, Avery Payne wrote:

This is probably the only real problem in the arrangement I'm using
then.  The entire premise is "the support scripts live where the
definitions live and are called with relative pathing, allowing the
definitions to be located anywhere".


 My point of view is that service directories have their place on a
RAM filesystem, so they should not include heavy data. Stuff like
support scripts is read-only: it doesn't have to be in RAM.
 You can have local data with relative pathing... in your own
service directory. As soon as you exit your service directory,
it's global data.
Ah.  As it is designed right now, and based on your definition, my .run/ 
directory is global data.  This is where we slightly disagree; the ./run 
directory was meant to be movable, and not installed at a fixed 
location; and everything in it is relative to installation, so it is 
more "local" than "global".  That's ok, I think we can work this out.


A side note: there really isn't a lot of heavy data.  In fact, I feel 
ashamed to admit that there will be a lot of wasted block space because 
each file is either a symlink (which is a few characters), or a fixed 
file that has less than 80 bytes of data.  There's a lot of empty space 
in this arrangement, but that's ok.  For a laptop, desktop, or server, 
it's a few megabytes of install on a gargantuan multi-gigabyte hard 
drive.  For an embedded system, all of that empty block space will get 
compressed and squeezed into something much smaller and denser.  Either 
way, the empty space ends up not being so bad.






 This was done precisely to
avoid the mess of "you MUST have your scripts/support scripts live at
this exact path", which was originally inspired by a message exchange
we had months ago about separating policy from mechanism. For my
definitions to work, those four dot-directories have to live where
the run-time definitions live.  In this case, your compiled directory
is the equivalent of a run-time directory.

I am certainly not adverse to changing their names or locations - the
entire purpose is to have a set of definitions that work out of the
box, and I need to make it do whatever is required to achieve that -
but I cringe at the concept that I would need to use an absolute
path, because I'll end up making everything brittle as a result.  I'm
open to suggestions on how to fix this.


 I'd suggest something like a BASEDIR environment variable, defaulting
to "." or "../svcdef" or whatever the relative path for your support
scripts normally is. All your calls to the support scripts would then
start with ${BASEDIR}. Is it envisionable?
Yes and no.  I already have a default set of environment settings, 
including a PATH, that are included in a .env/ directory.  I could 
include BASEDIR as well and do some shennanigans with that to make this 
work.  As part of the use-s6-rc script, I can call upon the definition 
of BASEDIR and re-link all of the definitions in the source directory to 
point to the absolute path that contains the .run/ directory.  The 
problem remains, I think you are asking me to make all my symlinks 
non-relative, and while I can do that (and yes, everything will work) it 
"feels" brittle to me.


With just the relative links, supporting an end-user is easy - 
everything is at a known location, and if it isn't, well, "there's your 
problem right there", we just need to place these directories at the 
correct location and it's all fixed.  With fixed pathing, now I have to 
figure out where everything is hiding at, and if the install was munged 
badly, or worse the user did something completely off the wall, then who 
knows where it's at, etc. etc.  It muddies the waters when trying to 
figure out what is going wrong.


s6-rc gets around this because it's very explicit about where the source 
and compiled definitions are, no problem there.  But my support scripts 
work with relative pathing by design.  By the way, when I say support 
script, I really mean the main run script for every definition.  More 
info on that at the bottom of this message...






The only magic needed is that the support directories have to be
located in the same runtime location, so that the relative path links
will align and everything will be found.  The compiled definitions
are the equivalent of a runtime location, therefore, they must live
there.


 Nope, it doesn't work that way. s6-rc relocates the service directories
at run time, and it only knows how to handle service directories. If
you have extra data, put it either in a service directory or completely
outside s6-rc's reach.
There may be a (rather ugly) workaround I can do for this, pointing 
(definition)/run file to (definition)/data/run, which in turn points to 
a fixed location.  More in a moment...






(a) stay a symlink pointing to a directory in /data, 

Re: [announce] s6-rc: a s6-based service manager for Unix systems

[Avery Payne]

I've only skimmed the link in that sentence but it appears that 
supervision-scripts can be easily turned into source directories with a 
simple conversion script or program.  All of the links used in 
supervision-scripts are required to be relative-pathed, and the support 
scripts are in dot-directories which are not copied by default (which is 
fine).  There does not appear to be any namespace collisions between the 
projects.


With regard to http://skarnet.org/software/s6-rc/overview.html, 
specifically the section titled "The compilation phase", and this sentence:
"Users are expected to write their service definitions - be it oneshots, 
longruns or bundles - in one or more /source directories/, in the s6-rc 
source format 
."  I think 
only the following needs to be done (although I need to read that page 
completely and confirm this):


* Write a script to translate the ./needs directory of a definition into 
a dependencies file.  Which is more or less a one-liner that pipes the 
directory listing into a text file one line at a time. The script will 
be placed into svcdef/.bin (along with all of the other tool scripts).  
Should be easy.
* Write a script that, for a given definition, calls the ./needs 
translator (when required), then copies the definition + (optional) 
dependencies file into the source directory format, making any name 
changes as needed.  Not rocket science.
* Figure out how to place the .bin, .run, .log, and .finish directories 
into the live system.  This is probably done with another simple support 
script that simply does a "cp -Rav" of them to where the compiled 
definitions live.  Not exactly rocket science either.


Beyond that, the rest of the definitions should be plug and play.

With regard to converting rc scripts to source format...I'm all ears, 
this would of course accelerate my project by an order of magnitude, 
because I would be able to simply convert the giant blob of rc 
definitions that I extracted from Debian 7...


On 9/23/2015 1:59 PM, Laurent Bercot wrote:


 Reports and suggestions on usability are also welcome. I'd like
to get s6-rc adopted by Unix distributions, and that will only
happen if they don't see it as daunting work; ideally, I'd like to
spark a discussion about the best ways to convert existing rc
scripts to the s6-rc source format, and then submit the emerging
ideas to distributions.