Re: [DNG] Init scripts in packages

[Hendrik Boom]

On Sat, Aug 08, 2015 at 02:38:48AM -0500, T.J. Duchene wrote:
> You could always lift scripts from Wheezy and use them as a template.

Or even from jessie, now that Debian jessie in stable.

-- hendrik

> 
> On Sat, Aug 8, 2015 at 2:28 AM, Miles Fidelman 
> wrote:
> 
> > T.J. Duchene wrote:
> >
> >>
> >>
> >> On 08/07/2015 09:31 PM, Miles Fidelman wrote:
> >>
> >>>
> >>> Trivial as in, somebody has to do it.  The whole point of packaging is
> >>> to automate a lot of the routine things involved in installation.
> >>>
> >>> And, because Debian (and presumeably Devuan) don't put stuff in default
> >>> locations, packaging involves changing the default locations of things.
> >>>
> >>> Where this leads is that down the road, we either need a full set of
> >>> Devuan-specific package maintainers, or everybody is back to compiling and
> >>> installing from upstream source.
> >>>
> >>> Miles Fidelman
> >>>
> >>>
> >> Good evening, Miles!  =)
> >>
> >
> > Good morning T.J. !
> >
> >>
> >> If I might offer an opinion, I do not think that the situation is quite
> >> that dire.  The packages that require init scripts are a tiny fraction of
> >> the entire repository.  For the moment, the scripts Devuan needs are still
> >> in the Debian archives as Jesse has System 5 support.
> >>
> >> Devuan can just replicate them and support them moving forward.
> >>
> >>
> > Well, maybe.  The original poster started with the statement "Currently
> > Debian packages contains both systemd units and init scripts.  However,
> > Debian developers refused to support several init systems. So it's only a
> > matter of time when they remove init scripts from packages." If that's
> > true, then we have problem.
> >
> > My sense is that systemd is having close to zero effect on upstream code -
> > most stuff is shipping with traditional sysv init scripts, with some folks
> > adding systemd units, but most basically ignoring systemd.
> >
> > If the Debian packagers do what makes sense - i.e., simply tweak sysv init
> > scripts that come from upstream, and rely on systemd's support for init
> > scripts, then all is copacetic.
> >
> > If, instead, they start removing the sysv scripts, and including homebrew
> > systemd units - then we're in for a mess of rework.
> >
> > Miles
> >
> >
> >
> > --
> > In theory, there is no difference between theory and practice.
> > In practice, there is.    Yogi Berra
> >
> > ___
> > Dng mailing list
> > Dng@lists.dyne.org
> > https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
> >

> ___
> Dng mailing list
> Dng@lists.dyne.org
> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Steve Litt]

On Sat, 08 Aug 2015 09:46:07 +0200
Jaromil  wrote:

> 
> 
> On 8 August 2015 09:28:42 CEST, Miles Fidelman
>  wrote:
> 
> >If, instead, they start removing the sysv scripts, and including 
> >homebrew systemd units - then we're in for a mess of rework.
> 
> both me, Franco and other VUAs are literally aiming to a fork, either
> after Jessie or Ascii as infrastructure will grow and consolidate
> organically, as well the maintainer base will grow with usage.


   * *
\ o /
 \|/ 
  |   A W   R I I I G H T ! ! !
 / \  _  
/   \/
   /
  -

This is great news. To be honest with you, I was never a fan of "we'll
keep on forever being Debian but removing their cruft.

SteveT

Steve Litt 
July 2015 featured book: Rapid Learning for the 21st Century
http://www.troubleshooters.com/rl21
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Isaac Dunham]

On Sat, Aug 08, 2015 at 09:43:47AM +0200, Laurent Bercot wrote:
> On 08/08/2015 03:43, Isaac Dunham wrote:
> >Which, fortunately, is pretty easy to do: I wrote an environment
> >sanitizer yesterday, because I was curious how easily solved that is.
> >Usage is
> >cautenv [-c DIR] [-u] [-x] COMMAND [COMMAND_ARGS...]
> 
>  Would you mind linking it ? I'm interested in trying to break it. ;)

Not a problem, now that it's online.
Here's the command:
https://github.com/idunham/bits/raw/master/cautenv.c
Repository:
https://github.com/idunham/bits

CC0, so do what you wish with it.
The rest of that repository is also CC0, but almost all of it is only
useful for someone who wants an example of using some less-frequently
seen functions.

Thanks,
Isaac Dunham
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Miles Fidelman]

Jaromil wrote:

Jaromil wrote:

Its early to say, but this thread is just prospecting. I believe that
on a longer term we can hardly do worse tha Debian when untangling
dependencies that right now constantly drag in desktop oriented
stuff, like avahi and other similar nonsense that we almost got used
to swallow all these years.

on the mid - long term it won't be just systemd to make the
difference between Devuan and Debian.

On Sat, 08 Aug 2015, Miles Fidelman wrote:

But now we get into the question of can Devuan really attract a full
set of package maintainers?




IF what we do turns out to be useful for all those professionals
preferring GNU/Linux to *BSD (and realistically, the latter is today the
best pro- alternative to the amateurial mess Linux is becoming) then
there won't be need for an horde of mediocre package maintainers, but a
pack of few good ones.

There is much more to be said, for instance the emergence of new
packaging systems which will be surclassing old ones in 2-3 years
maximum, for instance see Guix and NixOS with the smart adoption of a
declarative language for the task.


Well yes - but that raises the question of why not just Guix on day 
one?  It strikes me that the primary value of Debian has always been 
dpkg and apt.


Cheers,

Miles

--
In theory, there is no difference between theory and practice.
In practice, there is.    Yogi Berra

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Rainer Weikusat]

Miles Fidelman  writes:
> Rainer Weikusat wrote:

[...]

>> Also, to re-iterate this: What an init script needs to do is really only
>> 'start a process'/ 'stop a process'. Most of the other code which crept
>> in there during the last 15 - 20 years will fall into one of two
>> categories
>>
>>  1) Never did anything useful
>>  2) Should never have been implemented in this way.
>
> It can be a bit more than that, for example, the sympa mailing list
> package consists of multiple programs that are started in order, and
> includes
> - start (all 4)
> - stop (all 4)
> - restart (stop, in order; start in order)
> - status
>
> Most server scripts do some setup and cleanup.  There's also typically
> a reload config files option.

I'm aware how existing init scripts look like but that's just another
example of 'coral reef' coding: Some code is needed (or believed to be
needed). Where's the most convenient place it can be added? The init
script, obviously! It's just a shell script and thus, easy to
modify. For simple modifications, this is even a good idea, because
after all, the intent is not to create a statue but to make something
work. This gets problematic when there are, say, 5 different people who
always work in this way whose small hacks keep piling up over the course
of a few years (I have some specific code in mind): The inevitable
result is a horrendous mess which doesn't work almost all of the time
and nobody can still tell which parts of it are doing what and how they
all interact.

For the example you're using, if these 4 programs really belong to the
same package, the obvious idea would be to write a script starting them
and a script stopping them and let the init script execute
these. Restart can be implemented as stop followed by start but that's
already a convenience compromise. The others have no business in the
init script as they're not about starting or stopping programs.

Dito for setup and cleanup: For as long as these are simple, limited
task, eg, something like this

CONFIG=/etc/config-file
. $CONFIG
: ${DEBUG:=3}

putting it into the init script makes sense. But not five such blocks in
a row, possibly even with some conditionals around them to execute or
not execute them in various combinations.
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Marlon Nunes]

On Sat, 8 Aug 2015 11:03:34 +0200
Jaromil  wrote:

> 
> > Jaromil wrote:
> 
> > >Its early to say, but this thread is just prospecting. I believe that
> > >on a longer term we can hardly do worse tha Debian when untangling
> > >dependencies that right now constantly drag in desktop oriented
> > >stuff, like avahi and other similar nonsense that we almost got used
> > >to swallow all these years.
> > >
> > >on the mid - long term it won't be just systemd to make the
> > >difference between Devuan and Debian.
> 
> On Sat, 08 Aug 2015, Miles Fidelman wrote:
> > But now we get into the question of can Devuan really attract a full
> > set of package maintainers?
> 
> it depends what you mean by "full". To us it certainly doesn't means the
> size of Debian, but a core system which can be reliably used as a base
> by both upstream and downstream: developers, devops, sysadmins and
> distributions who compile the key production packages from source and/or
> package themselves, as yourself pointed out in this thread.
> 
> what I call the hardest part we have already demonstrated we're able to
> do: putting together a continuous integration infrastructure for the
> core system, using software we wrote, hence we can scale organically and
> we can further develop ad-hoc to overcome initial difficulties (see for
> instance the caching approach taken with Amprolla, or our fixes to
> jenkins-debian-glue, or the upcoming fixes to qemu-arm builds).
> 
> IF what we do turns out to be useful for all those professionals
> preferring GNU/Linux to *BSD (and realistically, the latter is today the
> best pro- alternative to the amateurial mess Linux is becoming) then
> there won't be need for an horde of mediocre package maintainers, but a
> pack of few good ones.
> 
> There is much more to be said, for instance the emergence of new
> packaging systems which will be surclassing old ones in 2-3 years
> maximum, for instance see Guix and NixOS with the smart adoption of a
> declarative language for the task.
> 
> I'll also refrain to observe the sort of labour relationship Debian is
> instaurating with its volunteers, the majority being students and people
> who abandon once they got a job. I'll just say we are aiming at a
> different approach here and it shall be focused on quality, not
> quantity.
> 
> ciao

Nice!

> -- 
> Denis "Jaromil" Roio, Dyne.org Think (& Do) Tank
> We are free to share code and we code to share freedom
> Web: https://j.dyne.org Contact: https://j.dyne.org/c.vcf
> GPG: 6113 D89C A825 C5CE DD02  C872 73B3 5DA5 4ACB 7D10
> Confidential communications: https://keybase.io/jaromil
> 


-- 
Stop slacking you lazy bum!
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Rainer Weikusat]

Isaac Dunham  writes:

> On Thu, Aug 06, 2015 at 05:14:28PM +0200, Laurent Bercot wrote:
>> On 06/08/2015 16:31, Isaac Dunham wrote:
>> >If differences in environment can cause problems, it's a problem with
>> >design. A program that changes what it does just due to differences
>> >between the init environment and a login environment is going to be
>> >hard to debug.
>> 
>>  There are tons of those, and you can't fix them all. Stupid example:
>> less. Behaves differently when its stdout is a terminal and when it's
>> not.
>>  The only way to ensure reproducible behaviour for a program, no matter
>> what it is, is to start it in a reproducible environment.
>
> Which, fortunately, is pretty easy to do: I wrote an environment
> sanitizer yesterday, because I was curious how easily solved that is.
> Usage is
> cautenv [-c DIR] [-u] [-x] COMMAND [COMMAND_ARGS...]
>
> and it cleans the environment (saving some user variables if -u is
> specified and DISPLAY/XAUTH if -x is specified), closes all fds above 2,
> changes directory to DIR ("/" if that's not specified, and calls
> execvp(argv[optind], argv+optind).
>
> It comes out at 123 lines, and could probably be made shorter.

It could be split into three tools: One which changes the environment,
one which changes to a certain directory and one which 'sanitizes' the
set of inherited file descriptors.

Presently, I have a tool which combines the last task with creating a
properly backgrounded process because file descriptor leaks really only
matter if the descriptor is leaked to a long-running process and because
that's a somewhat dubious safeguard: File descriptors should be managed
by the programs creating them and closing them on the presumption that
this program surely didn't bother is a practical necessity but not
theoretically sound.

I don't have anything for changing the environment but in general, the
same concerns apply to that: An environment variable was created in
order to communicate certain information to other processes and it
shouldn't be thrown away blindly. As a practical example, one of the
things I'm dealing with is a tiny distributed system for creating
certificates for VPN servers based on a (a number of, actually) OpenSSL
based 'CA installations'. This uses ssh combined with keys as secure
transport and since there's a setuid-0 program involved which talks to
the network, it originally just did a

environ = NULL;

This caused (minor) problems later on because OpenSSL didn't know where
to put its .rnd file and in order to get around these, I had to create
the missing environment variables with sensible values.

And this is the really sensible solution to this problem: If a program
supposed to run from a non-interactive environment needs certain
environment variables with 'correct' values, whatever starts the program
has to create these (or overwrite them in case they're already set). The
only useful task of the environment sanitizer is that it force Joe
Someone to fix the startup code because relying on another program
having set that up correctly won't work anymore.
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Jaromil]

> Jaromil wrote:

> >Its early to say, but this thread is just prospecting. I believe that
> >on a longer term we can hardly do worse tha Debian when untangling
> >dependencies that right now constantly drag in desktop oriented
> >stuff, like avahi and other similar nonsense that we almost got used
> >to swallow all these years.
> >
> >on the mid - long term it won't be just systemd to make the
> >difference between Devuan and Debian.

On Sat, 08 Aug 2015, Miles Fidelman wrote:
> But now we get into the question of can Devuan really attract a full
> set of package maintainers?

it depends what you mean by "full". To us it certainly doesn't means the
size of Debian, but a core system which can be reliably used as a base
by both upstream and downstream: developers, devops, sysadmins and
distributions who compile the key production packages from source and/or
package themselves, as yourself pointed out in this thread.

what I call the hardest part we have already demonstrated we're able to
do: putting together a continuous integration infrastructure for the
core system, using software we wrote, hence we can scale organically and
we can further develop ad-hoc to overcome initial difficulties (see for
instance the caching approach taken with Amprolla, or our fixes to
jenkins-debian-glue, or the upcoming fixes to qemu-arm builds).

IF what we do turns out to be useful for all those professionals
preferring GNU/Linux to *BSD (and realistically, the latter is today the
best pro- alternative to the amateurial mess Linux is becoming) then
there won't be need for an horde of mediocre package maintainers, but a
pack of few good ones.

There is much more to be said, for instance the emergence of new
packaging systems which will be surclassing old ones in 2-3 years
maximum, for instance see Guix and NixOS with the smart adoption of a
declarative language for the task.

I'll also refrain to observe the sort of labour relationship Debian is
instaurating with its volunteers, the majority being students and people
who abandon once they got a job. I'll just say we are aiming at a
different approach here and it shall be focused on quality, not
quantity.

ciao

-- 
Denis "Jaromil" Roio, Dyne.org Think (& Do) Tank
We are free to share code and we code to share freedom
Web: https://j.dyne.org Contact: https://j.dyne.org/c.vcf
GPG: 6113 D89C A825 C5CE DD02  C872 73B3 5DA5 4ACB 7D10
Confidential communications: https://keybase.io/jaromil



signature.asc
Description: Digital signature
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Miles Fidelman]

Jaromil wrote:


On 8 August 2015 09:28:42 CEST, Miles Fidelman  
wrote:


If, instead, they start removing the sysv scripts, and including
homebrew systemd units - then we're in for a mess of rework.

both me, Franco and other VUAs are literally aiming to a fork, either after
Jessie or Ascii as infrastructure will grow and consolidate organically, as
well the maintainer base will grow with usage.

Its early to say, but this thread is just prospecting. I believe that on a 
longer
term we can hardly do worse tha Debian when untangling dependencies that
right now constantly drag in desktop oriented stuff, like avahi and other 
similar
nonsense that we almost got used to swallow all these years.

on the mid - long term it won't be just systemd to make the difference between
Devuan and Debian.



But now we get into the question of can Devuan really attract a full set 
of package maintainers?


Miles



--
In theory, there is no difference between theory and practice.
In practice, there is.    Yogi Berra

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Miles Fidelman]

But now we're back into having to have a completely separate package 
repository, along with a full set of package maintainers.  Sigh.


T.J. Duchene wrote:

You could always lift scripts from Wheezy and use them as a template.

On Sat, Aug 8, 2015 at 2:28 AM, Miles Fidelman 
mailto:mfidel...@meetinghouse.net>> wrote:


T.J. Duchene wrote:



On 08/07/2015 09:31 PM, Miles Fidelman wrote:


Trivial as in, somebody has to do it.  The whole point of
packaging is to automate a lot of the routine things
involved in installation.

And, because Debian (and presumeably Devuan) don't put
stuff in default locations, packaging involves changing
the default locations of things.

Where this leads is that down the road, we either need a
full set of Devuan-specific package maintainers, or
everybody is back to compiling and installing from
upstream source.

Miles Fidelman


Good evening, Miles!  =)


Good morning T.J. !


If I might offer an opinion, I do not think that the situation
is quite that dire.  The packages that require init scripts
are a tiny fraction of the entire repository.  For the moment,
the scripts Devuan needs are still in the Debian archives as
Jesse has System 5 support.

Devuan can just replicate them and support them moving forward.


Well, maybe.  The original poster started with the statement
"Currently Debian packages contains both systemd units and init
scripts.  However, Debian developers refused to support several
init systems. So it's only a matter of time when they remove init
scripts from packages." If that's true, then we have problem.

My sense is that systemd is having close to zero effect on
upstream code - most stuff is shipping with traditional sysv init
scripts, with some folks adding systemd units, but most basically
ignoring systemd.

If the Debian packagers do what makes sense - i.e., simply tweak
sysv init scripts that come from upstream, and rely on systemd's
support for init scripts, then all is copacetic.

If, instead, they start removing the sysv scripts, and including
homebrew systemd units - then we're in for a mess of rework.

Miles



-- 
In theory, there is no difference between theory and practice.

In practice, there is.    Yogi Berra

___
Dng mailing list
Dng@lists.dyne.org 
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng





--
In theory, there is no difference between theory and practice.
In practice, there is.    Yogi Berra

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Jaromil]

On 8 August 2015 09:28:42 CEST, Miles Fidelman  
wrote:

>If, instead, they start removing the sysv scripts, and including 
>homebrew systemd units - then we're in for a mess of rework.

both me, Franco and other VUAs are literally aiming to a fork, either after
Jessie or Ascii as infrastructure will grow and consolidate organically, as
well the maintainer base will grow with usage.

Its early to say, but this thread is just prospecting. I believe that on a 
longer
term we can hardly do worse tha Debian when untangling dependencies that
right now constantly drag in desktop oriented stuff, like avahi and other 
similar
nonsense that we almost got used to swallow all these years.

on the mid - long term it won't be just systemd to make the difference between
Devuan and Debian.

ciao

>In theory, there is no difference between theory and practice.
>In practice, there is.    Yogi Berra

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Laurent Bercot]

On 08/08/2015 03:43, Isaac Dunham wrote:

Which, fortunately, is pretty easy to do: I wrote an environment
sanitizer yesterday, because I was curious how easily solved that is.
Usage is
cautenv [-c DIR] [-u] [-x] COMMAND [COMMAND_ARGS...]


 Would you mind linking it ? I'm interested in trying to break it. ;)

--
 Laurent

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[T.J. Duchene]

You could always lift scripts from Wheezy and use them as a template.

On Sat, Aug 8, 2015 at 2:28 AM, Miles Fidelman 
wrote:

> T.J. Duchene wrote:
>
>>
>>
>> On 08/07/2015 09:31 PM, Miles Fidelman wrote:
>>
>>>
>>> Trivial as in, somebody has to do it.  The whole point of packaging is
>>> to automate a lot of the routine things involved in installation.
>>>
>>> And, because Debian (and presumeably Devuan) don't put stuff in default
>>> locations, packaging involves changing the default locations of things.
>>>
>>> Where this leads is that down the road, we either need a full set of
>>> Devuan-specific package maintainers, or everybody is back to compiling and
>>> installing from upstream source.
>>>
>>> Miles Fidelman
>>>
>>>
>> Good evening, Miles!  =)
>>
>
> Good morning T.J. !
>
>>
>> If I might offer an opinion, I do not think that the situation is quite
>> that dire.  The packages that require init scripts are a tiny fraction of
>> the entire repository.  For the moment, the scripts Devuan needs are still
>> in the Debian archives as Jesse has System 5 support.
>>
>> Devuan can just replicate them and support them moving forward.
>>
>>
> Well, maybe.  The original poster started with the statement "Currently
> Debian packages contains both systemd units and init scripts.  However,
> Debian developers refused to support several init systems. So it's only a
> matter of time when they remove init scripts from packages." If that's
> true, then we have problem.
>
> My sense is that systemd is having close to zero effect on upstream code -
> most stuff is shipping with traditional sysv init scripts, with some folks
> adding systemd units, but most basically ignoring systemd.
>
> If the Debian packagers do what makes sense - i.e., simply tweak sysv init
> scripts that come from upstream, and rely on systemd's support for init
> scripts, then all is copacetic.
>
> If, instead, they start removing the sysv scripts, and including homebrew
> systemd units - then we're in for a mess of rework.
>
> Miles
>
>
>
> --
> In theory, there is no difference between theory and practice.
> In practice, there is.    Yogi Berra
>
> ___
> Dng mailing list
> Dng@lists.dyne.org
> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
>
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Miles Fidelman]

T.J. Duchene wrote:



On 08/07/2015 09:31 PM, Miles Fidelman wrote:


Trivial as in, somebody has to do it.  The whole point of packaging 
is to automate a lot of the routine things involved in installation.


And, because Debian (and presumeably Devuan) don't put stuff in 
default locations, packaging involves changing the default locations 
of things.


Where this leads is that down the road, we either need a full set of 
Devuan-specific package maintainers, or everybody is back to 
compiling and installing from upstream source.


Miles Fidelman



Good evening, Miles!  =)


Good morning T.J. !


If I might offer an opinion, I do not think that the situation is 
quite that dire.  The packages that require init scripts are a tiny 
fraction of the entire repository.  For the moment, the scripts Devuan 
needs are still in the Debian archives as Jesse has System 5 support.


Devuan can just replicate them and support them moving forward.



Well, maybe.  The original poster started with the statement "Currently 
Debian packages contains both systemd units and init scripts.  However, 
Debian developers refused to support several init systems. So it's only 
a matter of time when they remove init scripts from packages." If that's 
true, then we have problem.


My sense is that systemd is having close to zero effect on upstream code 
- most stuff is shipping with traditional sysv init scripts, with some 
folks adding systemd units, but most basically ignoring systemd.


If the Debian packagers do what makes sense - i.e., simply tweak sysv 
init scripts that come from upstream, and rely on systemd's support for 
init scripts, then all is copacetic.


If, instead, they start removing the sysv scripts, and including 
homebrew systemd units - then we're in for a mess of rework.


Miles


--
In theory, there is no difference between theory and practice.
In practice, there is.    Yogi Berra

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[T.J. Duchene]

On 08/07/2015 09:31 PM, Miles Fidelman wrote:


Trivial as in, somebody has to do it.  The whole point of packaging is 
to automate a lot of the routine things involved in installation.


And, because Debian (and presumeably Devuan) don't put stuff in 
default locations, packaging involves changing the default locations 
of things.


Where this leads is that down the road, we either need a full set of 
Devuan-specific package maintainers, or everybody is back to compiling 
and installing from upstream source.


Miles Fidelman



Good evening, Miles!  =)

If I might offer an opinion, I do not think that the situation is quite 
that dire.  The packages that require init scripts are a tiny fraction 
of the entire repository.  For the moment, the scripts Devuan needs are 
still in the Debian archives as Jesse has System 5 support.


Devuan can just replicate them and support them moving forward.

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Miles Fidelman]

Rainer Weikusat wrote:

Miles Fidelman  writes:

Rainer Weikusat wrote:

Miles Fidelman  writes:


Alexey Rochev wrote

*Date: *2015-08-05 07:29 -400
*To: *dng
*Subject: *[DNG] Init scripts in packages
Currently Debian packages contains both systemd units and init scripts.
However, Debian developers refused to support several init
systems. So it's
only a matter of time when they remove init scripts from packages.
What will Devuan developers do when it happens? We can use sysvinit and
Devuan because these init scripts exist.

It occurs to me that nobody raised the obvious questions:

1. Are we seeing upstream developers shipping systemd scripts, or
systemd scripts w/o sysv init scripts?  I'm not sure I have.
2. What the heck are Debian developers (packagers, actually), doing
removing init scripts?

There's an answer to that and it's "it doesn't matter" (I tried to point
that out in an earlier reply). On the wheezy system I'm using to write
this, 'init scripts' make up 6789 LOC, nobody has the power to make them
disintegrate and I'd be very much surprised if there are more than 2000 LOC
in there which actually do something useful. Actually, I expect
yes. init scripts should be trivial and if they're not, something else
is amiss.

[...]


Right now, init script come from upstream, Debian "developers" (I
really can't bring myself to call a packager a developer) test & tweak
the upstream scripts to fit the Debian environment.  If they stop
doing that, someone is going to have to do that for Devuan.

Do they actually come 'from upstream'?


Absolutely.  At least for most server code,
tar xvf foo; ./configure; make install
leaves you with running server code, with default configuration (as 
tailored by configure), and init scripts


Packagers typically modify those scripts.


Also, to re-iterate this: What an init script needs to do is really only
'start a process'/ 'stop a process'. Most of the other code which crept
in there during the last 15 - 20 years will fall into one of two
categories

1) Never did anything useful
 2) Should never have been implemented in this way.


It can be a bit more than that, for example, the sympa mailing list 
package consists of multiple programs that are started in order, and 
includes

- start (all 4)
- stop (all 4)
- restart (stop, in order; start in order)
- status

Most server scripts do some setup and cleanup.  There's also typically a 
reload config files option.


This is a non-tempest in a teapot nobody ever really saw.


Worse, if "refuse to support multiple init systems" means that the
Debian packagers start stripping out the init scripts from Debian
packages, those, those packages become useless in Devuan.

Last time I looked, the point of Apache was "it's a web server" and not
"it comes with an init script" so this seems to have been blown somewhat
out of proportion. Even if 'Debian developers' should stop shipping
'init scripts' as part of their packages at some point in time in the
future, this won't cause them to disappear from packages people already
installed. And even in the extremely unlikely case to
$evil_debian_developper invades your computer in the middle of the
night and steals $mission_critical_init_script (this happening
simultaneously on all computers in the world), they'll be trivial to
replace.


Trivial as in, somebody has to do it.  The whole point of packaging is 
to automate a lot of the routine things involved in installation.


And, because Debian (and presumeably Devuan) don't put stuff in default 
locations, packaging involves changing the default locations of things.


Where this leads is that down the road, we either need a full set of 
Devuan-specific package maintainers, or everybody is back to compiling 
and installing from upstream source.


Miles Fidelman

--
In theory, there is no difference between theory and practice.
In practice, there is.    Yogi Berra

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Isaac Dunham]

On Thu, Aug 06, 2015 at 05:14:28PM +0200, Laurent Bercot wrote:
> On 06/08/2015 16:31, Isaac Dunham wrote:
> >If differences in environment can cause problems, it's a problem with
> >design. A program that changes what it does just due to differences
> >between the init environment and a login environment is going to be
> >hard to debug.
> 
>  There are tons of those, and you can't fix them all. Stupid example:
> less. Behaves differently when its stdout is a terminal and when it's
> not.
>  The only way to ensure reproducible behaviour for a program, no matter
> what it is, is to start it in a reproducible environment.

Which, fortunately, is pretty easy to do: I wrote an environment
sanitizer yesterday, because I was curious how easily solved that is.
Usage is
cautenv [-c DIR] [-u] [-x] COMMAND [COMMAND_ARGS...]

and it cleans the environment (saving some user variables if -u is
specified and DISPLAY/XAUTH if -x is specified), closes all fds above 2,
changes directory to DIR ("/" if that's not specified, and calls
execvp(argv[optind], argv+optind).

It comes out at 123 lines, and could probably be made shorter.

If one wants to use this in an init script, just change the shebang to
#!/path/to/cautenv /bin/sh

and you know that there are no extra variables or fds.


> >It took me ... less than a minute to find "pgrep -F".
> >That solves the problem of stale pidfiles.
> 
>  Do you think so? See for yourself:
>  https://gitlab.com/procps-ng/procps/blob/master/pgrep.c
> 
>  It just reads the pid in the pidfile, and does its matching with
> the read pid. Unless you also give other options to narrow the match,
> it will have the exact same problem.

I meant "the -F option that pgrep has solves the problem of stale
pidfiles".
I assumed that the reader would know to use that in addition to the
standard options, for which I apologize.

>  Now, of course, you can give the executable name, and add even more
> guards to make sure you don't find the wrong process. At the end of
> the day, you wrote a nice and complex pgrep command line, you're
> *almost* 100% sure that it will nail your process and only yours,
> and you just scanned /proc to send a goddamn signal to a daemon.

If someone finds "pgrep -F /var/run/$PIDFILE -x $NAME" complex,
I don't expect that they'd be competent to write any init script,
nor even a systemd unit.

And the only way that could be wrong is if:
-you have a stale pidfile
AND
-that stale pidfile happens to contain the same PID as a running process
AND
-that running process has the same executable name as the process that
created the pidfile, while being distinct.

I will acknowledge that whoever implemented -F did so in a lame way.
What I'd assumed that it did, because it's the right thing to do, is
make "-F $PIDFILE" check /proc//* to determine if there's a match
where  is strictly any pid specified in $PIDFILE.
But this could be solved, if one did a little refactoring.

>  I'd rather type "s6-svc -d /run/service/my-daemon" and kill my
> process with absolute certainty, using 10 or 20 times fewer system
> calls than pgrep and a small fraction of the CPU time.

Fair enough.

Thanks,
Isaac Dunham
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[T.J. Duchene]

>James Powell Thu, 06 Aug 2015 01:02:56 -0700

>Currently Debian packages contains both systemd units and init scripts. 
>However, Debian developers refused to support several init systems. So it's 
>only a matter of time when they remove init scripts from packages.What will 
>Devuan developers do when it happens? We can use sysvinit and Devuan because 
>these init scripts exist


Honestly?  I think that it is a "no brainer"  as my brothers would say.  I do 
agree, but with a proviso.  I think that System 5 init scripts will disappear 
from Debian packages while systemd becomes the Debian standard.

Please do not think badly of me when I say this, but the subject has been 
"beat to death" many times in the past on the DNG list.  

On many occasions I've commented that I think init scripts should be provided 
entirely separate from the other files that Debian bundles them with, so that 
the user might select whatever init they want to use. I see no technical 
reason why Devuan cannot detect whatever init is installed and then select the 
appropriate init scripts as a package or meta-package.

The majority of the repository is applications - which have nothing to do with 
the init process, so it would only be a limited number of upstream Debian 
packages that would have to be modified or replaced in this way.


___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Rob Owens]

- Original Message -
> From: "Rainer Weikusat" 
> Miles Fidelman  writes:

>> Worse, if "refuse to support multiple init systems" means that the
>> Debian packagers start stripping out the init scripts from Debian
>> packages, those, those packages become useless in Devuan.
> 
> This is actually such an absurd idea that I have some trouble
> considering it a serious concern (no disrespect intended --- I'm a
> developer and this seems 'a trifle' to me but maybe not to everybody
> else).

I get that an init script is very minor compared to the software it 
starts/stops.  The problem, though, is one of scale.  If the handful
of people who work on Devuan suddenly have to create init scripts
for hundreds or thousands of packages, that job will take a long 
time.  Even if it's just a matter of finding an old init script and 
verifying that it works.

That said, I am doubtful that this scenario will happen.  But even if 
it did, it would not be an insurmountable problem.  It would be a big 
pain in the neck, though.

-Rob
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Rainer Weikusat]

Miles Fidelman  writes:
> Rainer Weikusat wrote:
>> Miles Fidelman  writes:
>>
>>> Alexey Rochev wrote
 *Date: *2015-08-05 07:29 -400
 *To: *dng
 *Subject: *[DNG] Init scripts in packages
 Currently Debian packages contains both systemd units and init scripts.
 However, Debian developers refused to support several init
 systems. So it's
 only a matter of time when they remove init scripts from packages.
 What will Devuan developers do when it happens? We can use sysvinit and
 Devuan because these init scripts exist.
>>> It occurs to me that nobody raised the obvious questions:
>>>
>>> 1. Are we seeing upstream developers shipping systemd scripts, or
>>> systemd scripts w/o sysv init scripts?  I'm not sure I have.
>>> 2. What the heck are Debian developers (packagers, actually), doing
>>> removing init scripts?
>> There's an answer to that and it's "it doesn't matter" (I tried to point
>> that out in an earlier reply). On the wheezy system I'm using to write
>> this, 'init scripts' make up 6789 LOC, nobody has the power to make them
>> disintegrate and I'd be very much surprised if there are more than 2000 LOC
>> in there which actually do something useful. Actually, I expect
>> yes. init scripts should be trivial and if they're not, something else
>> is amiss.

[...]

> Right now, init script come from upstream, Debian "developers" (I
> really can't bring myself to call a packager a developer) test & tweak
> the upstream scripts to fit the Debian environment.  If they stop
> doing that, someone is going to have to do that for Devuan.

Do they actually come 'from upstream'? When debianizing a package via
dh_make, one of the debian/ template files generated by it is 'something
which should become your init script'. This suggests that 'the Debian
scripts' ultimatively come from the respective package maintainers (who
may have written them or may have gotten a working script from elsewhere
and adapted that). In any case, they'll be part of the Debian-specific
package files afterwards.

Also, to re-iterate this: What an init script needs to do is really only
'start a process'/ 'stop a process'. Most of the other code which crept
in there during the last 15 - 20 years will fall into one of two
categories

1) Never did anything useful
2) Should never have been implemented in this way.

This is a non-tempest in a teapot nobody ever really saw.

> Worse, if "refuse to support multiple init systems" means that the
> Debian packagers start stripping out the init scripts from Debian
> packages, those, those packages become useless in Devuan.

Last time I looked, the point of Apache was "it's a web server" and not
"it comes with an init script" so this seems to have been blown somewhat
out of proportion. Even if 'Debian developers' should stop shipping
'init scripts' as part of their packages at some point in time in the
future, this won't cause them to disappear from packages people already
installed. And even in the extremely unlikely case to
$evil_debian_developper invades your computer in the middle of the
night and steals $mission_critical_init_script (this happening
simultaneously on all computers in the world), they'll be trivial to
replace.

This is actually such an absurd idea that I have some trouble
considering it a serious concern (no disrespect intended --- I'm a
developer and this seems 'a trifle' to me but maybe not to everybody
else).
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Rainer Weikusat]

tilt!  writes:

[...]

> Laurent Bercot wrote on 06/08/2015 at 14:21 CEST:
>> [...]
>>   And "status". This is very service-dependent: since there is no
>> global API, no service manager, scripts will query the daemon's
>> status in a daemon-specific way. More vagueness again, because
>> "status" doesn't define exactly what you should say, and the lowest
>> common denominator is "is it alive?" but even for this basic check,
>> daemon-specific interfaces are used.
>> [...]
>
> From the discussion in this thread so far, I can determine at least
> the following two problems with "status":
>
> #1  There are not just plain, singular-per-service "daemons"
> involved (extreme, but valid examples include programs
> hosted inside application servers, even more extreme
> is a cumulative service called "networking" that might
> involve all sorts of stuff to be done), and
>
> #2  not all softwares that are providing "services" provide
> "specific interfaces" to query them even for a most
> basic information on them being "alive" or not.
>
> I personally am, however, a fan of simple semantics, and it is
> my understanding that UN*X has always done well when it provided
> simple semantics:

[systemd status]

>  * The status is "off" (the service is not alive, and this
>is not due to the service having failed at a previous
>attempt to run it),
>  * the status is "on" (the service is alive), or
>  * the status is "failed" (the service is alive, and this
>is due to it having failed to start on a previous attempt
>to do so).
>
> My question is, did I understand that correctly?

Short reply because I have a bunch of grotty Java-code I need to deal
with: off/ on/ failed is an ill-defined notion someone pulled
out of his posterior while thinking about the issue 'in abstract' (eg,
does dpkg --remove mean 'off').
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Miles Fidelman]

Rainer Weikusat wrote:

Miles Fidelman  writes:


Alexey Rochev wrote

*Date: *2015-08-05 07:29 -400
*To: *dng
*Subject: *[DNG] Init scripts in packages
Currently Debian packages contains both systemd units and init scripts.
However, Debian developers refused to support several init
systems. So it's
only a matter of time when they remove init scripts from packages.
What will Devuan developers do when it happens? We can use sysvinit and
Devuan because these init scripts exist.

It occurs to me that nobody raised the obvious questions:

1. Are we seeing upstream developers shipping systemd scripts, or
systemd scripts w/o sysv init scripts?  I'm not sure I have.
2. What the heck are Debian developers (packagers, actually), doing
removing init scripts?

There's an answer to that and it's "it doesn't matter" (I tried to point
that out in an earlier reply). On the wheezy system I'm using to write
this, 'init scripts' make up 6789 LOC, nobody has the power to make them
disintegrate and I'd be very much surprised if there are more than 2000 LOC
in there which actually do something useful. Actually, I expect
yes. init scripts should be trivial and if they're not, something else
is amiss.



Well... it kind of does, if the idea is to leverage Debian package repos 
as much as possible.


Right now, init script come from upstream, Debian "developers" (I really 
can't bring myself to call a packager a developer) test & tweak the 
upstream scripts to fit the Debian environment.  If they stop doing 
that, someone is going to have to do that for Devuan.


Worse, if "refuse to support multiple init systems" means that the 
Debian packagers start stripping out the init scripts from Debian 
packages, those, those packages become useless in Devuan.  (Note: I did 
a little checking re. packages re. code that I use - postfix doesn't 
seem to ship systemd files, nor does sympa; Apache puts its systemd 
support in a module; mysql has to compiled -WITHSYSTEMD --- judging from 
that small sample, it seems to me that we're going to see more and more 
Debian packages that won't work with other init systems).


Miles Fidelman






--
In theory, there is no difference between theory and practice.
In practice, there is.    Yogi Berra

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Laurent Bercot]

 I'm not sure how systemd does it, but in my vision, there should be
two different states for the service: the *wanted* state, and the
*current* state.

 The wanted state is what is set by the administrator when she runs
a command such as "rc thisrunlevel". The command should set all the
services in "thisrunlevel" in the "wanted up" state.
 The current state is exactly what it says: is the service actually
alive or not, ready or not?

 A service manager's job is to perform service state transitions
until the current state matches the wanted state - or an unrecoverable
error happens, in which case the admin should have easy access to the
current state in order to diagnose and fix the problems.

--
 Laurent

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[tilt!]

Apologies, a typo:

I wrote myself on 07/08/2015 at 15:21 CEST:

[...]
  * the status is "failed" (the service is *NOT* alive, and this
is due to it having failed to start on a previous attempt
to do so).

My question is, did I understand that correctly?

[...]


Kind regards.
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[tilt!]

Hi,

sorry for picking up on this edge while the thread's general
discussion has advanced further.

The "status" command matters to me; that is why I would like
to address its handling in a more detailed manner.

Laurent Bercot wrote on 06/08/2015 at 14:21 CEST:

[...]
  And "status". This is very service-dependent: since there is no
global API, no service manager, scripts will query the daemon's
status in a daemon-specific way. More vagueness again, because
"status" doesn't define exactly what you should say, and the lowest
common denominator is "is it alive?" but even for this basic check,
daemon-specific interfaces are used.
[...]


From the discussion in this thread so far, I can determine at least
the following two problems with "status":

#1  There are not just plain, singular-per-service "daemons"
involved (extreme, but valid examples include programs
hosted inside application servers, even more extreme
is a cumulative service called "networking" that might
involve all sorts of stuff to be done), and

#2  not all softwares that are providing "services" provide
"specific interfaces" to query them even for a most
basic information on them being "alive" or not.

I personally am, however, a fan of simple semantics, and it is
my understanding that UN*X has always done well when it provided
simple semantics:

* Simple semantics are good for implementing simple scripts.
* Simple scripts that have means to implement difficult setups
  make such environments favorable for engineers, auditors and
  economists alike.

I do understand SystemD's approach as one of trying to enforce an API
into all that is capable of providing "service". The goal is that such
software is required to, by design, provide a mechanism to report on
something called a "status", and that "status" is one of (I use an own
unofficial terminology here):

 * The status is "off" (the service is not alive, and this
   is not due to the service having failed at a previous
   attempt to run it),
 * the status is "on" (the service is alive), or
 * the status is "failed" (the service is alive, and this
   is due to it having failed to start on a previous attempt
   to do so).

My question is, did I understand that correctly?

Before engaging further into a discussion, I want to determine
if my assumptions are right or wrong.

Kind regards,
T.
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Rainer Weikusat]

Miles Fidelman  writes:

> Alexey Rochev wrote
>> *Date: *2015-08-05 07:29 -400
>> *To: *dng
>> *Subject: *[DNG] Init scripts in packages
>> Currently Debian packages contains both systemd units and init scripts.
>> However, Debian developers refused to support several init
>> systems. So it's
>> only a matter of time when they remove init scripts from packages.
>> What will Devuan developers do when it happens? We can use sysvinit and
>> Devuan because these init scripts exist. 
>
> It occurs to me that nobody raised the obvious questions:
>
> 1. Are we seeing upstream developers shipping systemd scripts, or
> systemd scripts w/o sysv init scripts?  I'm not sure I have.
> 2. What the heck are Debian developers (packagers, actually), doing
> removing init scripts?

There's an answer to that and it's "it doesn't matter" (I tried to point
that out in an earlier reply). On the wheezy system I'm using to write
this, 'init scripts' make up 6789 LOC, nobody has the power to make them
disintegrate and I'd be very much surprised if there are more than 2000 LOC
in there which actually do something useful. Actually, I expect
yes. init scripts should be trivial and if they're not, something else
is amiss.

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Edward Bartolo]

Simplification of init scripts can be done by replacing them with a
text file containing the following:
a) preliminary logic tests to verify whether daemon can be started
b) command to start daemon together with parameters
c) command to stop daemon with parameters if applicable

Only two lines will do as init scripts have essentially always the
same skeleton. Basically, they are some preliminary logic tests
followed by a case statement.

A generic executable can do execute these the above following the same
skeleton as init scripts. 'a' can be done away with if daemon
dependency management is somehow provided.

On 07/08/2015, Miles Fidelman  wrote:
> Alexey Rochev wrote
>> *Date: *2015-08-05 07:29 -400
>> *To: *dng
>> *Subject: *[DNG] Init scripts in packages
>> Currently Debian packages contains both systemd units and init scripts.
>> However, Debian developers refused to support several init systems. So
>> it's
>> only a matter of time when they remove init scripts from packages.
>> What will Devuan developers do when it happens? We can use sysvinit and
>> Devuan because these init scripts exist.
>
> It occurs to me that nobody raised the obvious questions:
>
> 1. Are we seeing upstream developers shipping systemd scripts, or
> systemd scripts w/o sysv init scripts?  I'm not sure I have.
> 2. What the heck are Debian developers (packagers, actually), doing
> removing init scripts?
>
> Me,  I've been installing key packages from upstream sources for years -
> avoids having to deal with out-of-date packages and such. (The basic
> environment is certainly easier to install and maintain via apt - but
> key production packages, hell no.)
>
> Miles Fidelman
>
>
> --
> In theory, there is no difference between theory and practice.
> In practice, there is.    Yogi Berra
>
> ___
> Dng mailing list
> Dng@lists.dyne.org
> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
>
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Miles Fidelman]

Alexey Rochev wrote

*Date: *2015-08-05 07:29 -400
*To: *dng
*Subject: *[DNG] Init scripts in packages
Currently Debian packages contains both systemd units and init scripts.
However, Debian developers refused to support several init systems. So 
it's

only a matter of time when they remove init scripts from packages.
What will Devuan developers do when it happens? We can use sysvinit and
Devuan because these init scripts exist. 


It occurs to me that nobody raised the obvious questions:

1. Are we seeing upstream developers shipping systemd scripts, or 
systemd scripts w/o sysv init scripts?  I'm not sure I have.
2. What the heck are Debian developers (packagers, actually), doing 
removing init scripts?


Me,  I've been installing key packages from upstream sources for years - 
avoids having to deal with out-of-date packages and such. (The basic 
environment is certainly easier to install and maintain via apt - but 
key production packages, hell no.)


Miles Fidelman


--
In theory, there is no difference between theory and practice.
In practice, there is.    Yogi Berra

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Laurent Bercot]

On 07/08/2015 00:09, Rainer Weikusat wrote:

Since this is maybe/ likely a bit harsh


 Not harsh, just unwilling to accept that I'm actually your ally and
not your enemy.

 I'm not trying to replace Unix, because Unix is not broken - at least,
not as far as system startup is concerned. There *are* broken things
in Unix, but that's a whole other enchilada that I won't have time to
address in the next 2 or 3 lifetimes.

 I'm trying to replace *systemd*, with something that embraces Unix
much, much more.

 sysvinit or sysvrc init scripts don't define Unix. Trying to do better
than them is not replacing Unix, it's trying to do things right. Now is
the time to do things right, because if we don't, systemd is going to
take over, for good. I don't want that, and since you're here, I don't
think you want that, either.



If you're convinced that "rip and replace" is the only
viable option then I won't claim that you're wrong because I really
don't know. However, until I hit a technical obstacle, I won't be
convinced that the existing system can't be fixed (I acknowledge almost
all defects you mentioned) and this is based on being familiar (as in 'I
wrote the code') with both approaches.


 Fine. I'm okay with the burden of proof. Let's take a simple example and
see how deep we have to dig to make it work.

 You have 3 longrun services, A, B and C.
 A and C can take a long time to start and be ready, because they access
some network server that can be slow, or even fail. But they don't depend
on anything else than this network server.
 B depends on A. It does not contact the network server, but it consumes
non-negligible resources on the local machine at startup time.

 Your objective is to reach the state where A, B and C are all up and
ready as quickly and reliably as possible. How do you proceed ?

 Serially ? "A start && B start ; C start"
 Not good: you add A's and C's latencies.

 Parallely ? "A start & B start & C start"
 Not good: B can be scheduled to start before A, and will crash.

 Using a supervision system to make sure all the services will eventually
be up ? "s6-svc -u /service/A ; s6-svc -u /service/B ; s6-svc -u /service/C"
 Better, but still not optimal: if B crashes because A is not up yet, it
will be restarted, and it's annoying because it's hogging important resources
every time it attempts to start.

 You need a dependency manager.

 "rc A+B+C"
 Much better. Except there's no such thing yet. The closest we have is
OpenRC, and for now it's serial: you'll eat the added latencies for A and C
just like in the naive serial approach. Ouch.
 ISTR there are plans to make OpenRC go parallel at some point. Let's
assume it is parallel already. What do you do if A crashes because the
network server is down ?

 You also need a supervision system, coupled with the dependency manager.
The OpenRC folks have realized this, and you can use a flag to have your
service auto-restarted. There's also some early support for s6 integration,
which I won't deny is flattering, but I still don't think the design is
right: for instance, there are early daemons you can't supervise.

 OK, now, how do you detect that A is ready and that you can start B ?
Well, you need readiness notification, assuming A supports it. You need
it because you don't want B to crash and restart. And now your rc system
must implement some support for it.

 And I haven't even mentioned logging yet.

 If you've written init systems, you must have reached that point. I'm
interested in knowing how you solved those issues.

 Now, if you try to implement process supervision, dependency management,
readiness notification and state tracking in pure init scripts, well,
gl&hf. That's what sysvrc, with tools like start-stop-daemon or other
horrors, has been trying to do for 10 years, without knowing exactly what
it was that it was trying to do. The result isn't pretty.

 Then systemd came and said "hey look! I can do just that, and more, and
you don't have to bother anymore with those horrible sysvrc scripts!"
And what did admins say? YES, PLEASE. And they gobbled up the crap because
it was coated with the sweet, sweet features. (And, yes, with an unhealthy
dose of propaganda and bullshit, but if you dig through the bullshit, you
can find a few things of value.)

 I'm saying the same causes will have the same results, and more tweaking
of the init scripts isn't going to accomplish anything without some serious
redesign. It's the easy path; I'm all for the easy path when it can be
taken, but in this case, it's not the right one. The right path is to
provide the sweet, sweet, *needed* features - but to do it the Unix way.



A social reason, eg, $someone
pays my bills ($someone =~ /hat/) and that's what I did and you will
have to eat it aka 'resistance is futile ...' won't do.


 You are fighting windmills. This is the Lennart way, it's not mine and
I don't think it is anyone's here. Even if I wanted to, which I don't
and never will, I don't have the power to

Re: [DNG] Init scripts in packages

[Philip Lacroix]

Am 06.08.2015 17:49 schrieb Steve Litt:

Laurent Bercot  wrote:

  I have never said, am not saying, and probably never will say that
systemd is any good. It's not, and Lennart and Kay should go back to
engineering school,


:s/engineering school/kindergarten/


Hell no, that wouldn't be good for the other kids.
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Rainer Weikusat]

Rainer Weikusat  writes:

[...]

> I'm going to ignore the remainder of this because - while system startup
> is a topic of some interest to me - people warring over the right way to
> replace UNIX(*) because it's broken isn't.

Since this is maybe/ likely a bit harsh (and I'm trying to rid myself of
this style): If you're convinced that "rip and replace" is the only
viable option then I won't claim that you're wrong because I really
don't know. However, until I hit a technical obstacle, I won't be
convinced that the existing system can't be fixed (I acknowledge almost
all defects you mentioned) and this is based on being familiar (as in 'I
wrote the code') with both approaches. A social reason, eg, $someone
pays my bills ($someone =~ /hat/) and that's what I did and you will
have to eat it aka 'resistance is futile ...' won't do.
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Rainer Weikusat]

Rainer Weikusat  writes:
> Laurent Bercot  writes:
>> On 06/08/2015 20:18, Rainer Weikusat wrote:
>>> UNIX(*) and therefore, Linux, provides two system calls named fork and
>>> exec which can be used to create a new process while inheriting certain
>>> parts of the existing environment and to execute a new program in an
>>> existing process, keeping most of the environment. This implies that it
>>> is possible to write a program which performs a particular 'environment
>>> setup task' and then executes another program in the change
>>> environment.
>
> [...]
>
>>> And that's finally the jboss start script. I have some more tools of
>>> this kind because whenever I need a new, generic process management task
>>> to be performed, I write a new program doing that which can be used in
>>> concert with the existing ones.
>>
>>  What you are saying is that your approach is exactly the same as the
>> one found here:
>>  http://skarnet.org/software/execline/
>
> No, it's not. This is an interpreter for another programming language
> sharing some concepts with the Thompson-shell.

For the sake of accuracy: While this calls itself an interpreter, it
actually isn't. It's a compiler generating threaded code for immediate
execution upon call. It's sort-of similar to the Thompson-shell because
it relies on separate programs for providing any 'real' functionality,
including 'control constructs'.
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Rainer Weikusat]

Laurent Bercot  writes:
> On 06/08/2015 20:18, Rainer Weikusat wrote:
>> UNIX(*) and therefore, Linux, provides two system calls named fork and
>> exec which can be used to create a new process while inheriting certain
>> parts of the existing environment and to execute a new program in an
>> existing process, keeping most of the environment. This implies that it
>> is possible to write a program which performs a particular 'environment
>> setup task' and then executes another program in the change
>> environment.

[...]

>> And that's finally the jboss start script. I have some more tools of
>> this kind because whenever I need a new, generic process management task
>> to be performed, I write a new program doing that which can be used in
>> concert with the existing ones.
>
>  What you are saying is that your approach is exactly the same as the
> one found here:
>  http://skarnet.org/software/execline/

No, it's not. This is an interpreter for another programming language
sharing some concepts with the Thompson-shell. What I was describing
were additions to an existing scripting environment in order to help
with 'process/ services management'.

I'm going to ignore the remainder of this because - while system startup
is a topic of some interest to me - people warring over the right way to
replace UNIX(*) because it's broken isn't.
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Laurent Bercot]

On 06/08/2015 20:18, Rainer Weikusat wrote:

UNIX(*) and therefore, Linux, provides two system calls named fork and
exec which can be used to create a new process while inheriting certain
parts of the existing environment and to execute a new program in an
existing process, keeping most of the environment. This implies that it
is possible to write a program which performs a particular 'environment
setup task' and then executes another program in the change
environment. 'nohup' is a well-known example. Because of this, instead
of "40 lines of boilerplate" in every init script, almost all of which
is identical, it's possible to identify common tasks and write programs
to handle them which can be chained with other programs handling other
tasks


 Yes. You are describing chain loading. (And the system call is named
execve, if you want to be pedantic.)



And that's finally the jboss start script. I have some more tools of
this kind because whenever I need a new, generic process management task
to be performed, I write a new program doing that which can be used in
concert with the existing ones.


 What you are saying is that your approach is exactly the same as the
one found here:
 http://skarnet.org/software/execline/
and here:
 http://skarnet.org/software/s6/

 It's free software, it works, it's maintained, and the author happens to
read the dng list, hoping to find technical interlocutors that share his
concerns for the future of GNU/whatever/Linux.

 Are you one of them ? Good. Let's talk.

 Now, chain loading is great, and all the necessary tools that perform
process state changes already exist, but that's not enough to make
init scripts safe. When you want to sanitize a process, or when you're
doing any kind of security really, you cannot have an "allow everything
by default and deny specific things" approach: fork your process from
the user's shell, then sanitize the environment, the fds, the cwd, etc.
You *will* forget something at some point; if you don't, the person who
writes the next init script will. Instead, you have to use a "deny
everything by default" approach: in the case of init scripts, that means
always starting daemons with the same, sanitized, environment. That can
only be done with a supervision system, as explained at
 http://skarnet.org/software/s6/overview.html

 And a supervision system itself is a great thing, but it's not enough,
because it does not handle dependencies between longrun services (i.e.
daemons that can be supervised) and oneshot services (i.e. stuff you run
once to change the machine state, but that do not leave a long-lived
process around). That is where a real service manager is needed.

 As loath as I am to admit it, systemd *is* both a supervision system
and a service manager. It does machine initialization, daemon supervision,
and state changes - badly, but it does them. And no amount of mucking with
init scripts, no matter how much chain loading you use, is going to
accomplish the same.

 The solution is not in criticizing, it's in doing. I have the supervision
system, and I'm working on the service manager. But this will be all for
naught if systemd opponents can't be convinced that it is necessary: the
admin who wants a service manager will hear "sure, systemd does that!" on
one side, and "nah, this is purely systemd propaganda, you don't really
need that shit" on the other side. Guess what they will choose.

 The other side should be able to answer "sure, you can use *this* piece of
software, or *this* other one, to do what you want, and you don't have
to put up with all the systemd crap for this". It's the only way we can
make it work.

--
 Laurent

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Rainer Weikusat]

Laurent Bercot  writes:

A leading remark: This is based on an existing system I have implemented
(originally for Debian 5) working in the described way. The code is
proprietary as I'm one of those evil guys who want to (and do) write code
for a living despite the 'free software community' traditionally hates
me (not entirely their fault because almost all people handling skills I
have at all were acquired by 'trial and error on the internet' and they
used to be pretty grueseome [and to a degree, still are]).

> On 06/08/2015 16:00, Rainer Weikusat wrote:
>> That's all nice and dandy but it all boils down to 'the code executed by
>> the init script was deficient in some way'.
>
>  Yes, just like root exploits boil down to "the code executed by the
> suid program was deficient in some way".
>  My point is that you shouldn't need to have 40 lines of boilerplate
> to sanitize your init script before you run it; if it's so fragile,
> then it's badly designed.

UNIX(*) and therefore, Linux, provides two system calls named fork and
exec which can be used to create a new process while inheriting certain
parts of the existing environment and to execute a new program in an
existing process, keeping most of the environment. This implies that it
is possible to write a program which performs a particular 'environment
setup task' and then executes another program in the change
environment. 'nohup' is a well-known example. Because of this, instead
of "40 lines of boilerplate" in every init script, almost all of which
is identical, it's possible to identify common tasks and write programs
to handle them which can be chained with other programs handling other
tasks, Eg, as a real world example, a command to start a certain Java
application I happen to be dealing with looks like this:

daemon chdir $DIR monitor -n jboss -g $RUNAS chids -u $RUNAS 
$JBOSS_HOME/bin/run.sh -b 127.0.0.1

'daemon' creates a daemon process and execvps its arguments in it.

'chdir' changes to a directory passed as first argument and execvps the
remaining arguments.

'monitor' forks, performs a few other things (creates a control socket
with a well-known name) and execvps its arguments in the new process. As
the name might suggests, it works as process monitor/ supervisor.

'chids' changes uid and gid as specified on the command and and execvps
its remaining arguments.

And that's finally the jboss start script. I have some more tools of
this kind because whenever I need a new, generic process management task
to be performed, I write a new program doing that which can be used in
concert with the existing ones.

[...]

>> [systemd]
>> Is it? Or is it an extremely incomplete, ad hoc designed programming
>> language with a service manager and a truckload of other potentially
>> useful stuff attached to it in order to encourage people to swallow the
>> language?

[...]

>  Know your enemy. It's easy and tempting to rant for days on systemd's
> weaknesses; but it's also vital to acknowledge its strengths.


Considering that SuSE almost adopted updstart, it came just at the right
to keep the traditional order of the universe where "Fedora guys"
'contribute' and moan about Canonical 'not giving back' intact.

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Steve Litt]

On Thu, 6 Aug 2015 16:41:38 +0200
Laurent Bercot  wrote:


>   I have never said, am not saying, and probably never will say that
> systemd is any good. It's not, and Lennart and Kay should go back to
> engineering school, 

:s/engineering school/kindergarten/


/* Litt ducks and runs */
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Steve Litt]

On Thu, 06 Aug 2015 10:28:47 +0100
Rainer Weikusat  wrote:

> But a bare-bones init script does really only three things:
> 
> 1. Execute a command to start something.
> 2. Execute a command which stops it again.
> 3. Execute 2) then 1) for a restart.

Those are easy. The tough part is process dependencies. Especially
since a lot of daemons aren't ready to do their job when they first get
run, and report their readiness only to the systemd communication
system.

I think the biggest hassle is writing tiny scripts that basically
answer the question: "Is daemon X now functional?"

SteveT

Steve Litt 
July 2015 featured book: Rapid Learning for the 21st Century
http://www.troubleshooters.com/rl21
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Steve Litt]

On Thu, 06 Aug 2015 15:00:20 +0100
Rainer Weikusat  wrote:

> Laurent Bercot  writes:

> >  There's a simple reason for that: "init scripts" aren't
> > "managing services". They can more or less start and stop them,
> > that's about it - and they're not even doing the starting and
> > the stopping right.
> >
> >  - Starting ? Sure, it's easy to start a service: run the stuff,
> > and if it's a daemon, just fork it. Right ? Wrong. When you just
> > spawn the stuff from your shell, it runs with your shell's
> > environment (open fds, variables, terminals, so many parameters
> > that are *difficult* to clean up and really harden). But when you
> > spawn the stuff at boot time, it runs with the very different
> > environment provided by init. I can't count how many problem
> > reports I've read in support mailing-lists that were really hard to
> > track down, but in the end it was just a config issue because the
> > daemon launching script did not properly sanitize all its
> > environment, so it didn't give the same results when run by the
> > admin or when auto-run at boot time.
> 
> That's all nice and dandy but it all boils down to 'the code executed
> by the init script was deficient in some way'.

Hi Rainer,

I think what Laurent is really saying is that if you choose to manage a
daemon via shellscript and PID file, it's very difficult to make your
shellscript in a way that *isn't* deficient in some way.

SteveT

Steve Litt 
July 2015 featured book: Rapid Learning for the 21st Century
http://www.troubleshooters.com/rl21
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Steve Litt]

On Thu, 06 Aug 2015 15:00:20 +0100
Rainer Weikusat  wrote:

 
> 'Winning' against systemd will require getting support of a
> commerically more potent company than RedHat and SuSE combined and one
> willing to sink a sizable amount of money into the task. 

The day I believe the preceding sentence will be the day I switch to
Mac (because presumably BSD will have been kidnapped by a moneybags
corporation too).

If having a sane init (and sane software architecture in general)
requires big money corporate sponsorship, and we get one and overthrow
systemd, how long before OUR daddy warbucks does something horrible in
order to FUD and EEE away the competition?

We're in much the same situation as 1995, when Microsoft was getting a
little obnoxious but wasn't quite bad enough to trigger a mass exodus
to Linux. By 1999, no-money, no corporation Linux was experiencing huge
inward migration on the server, and a noticible inward migration on the
desktop. Back then, Redhat was just another distro vendor, and indeed,
one of the goodguys.

History tells me it can be done without money.

SteveT

Steve Litt 
July 2015 featured book: Rapid Learning for the 21st Century
http://www.troubleshooters.com/rl21
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Laurent Bercot]

On 06/08/2015 16:31, Isaac Dunham wrote:

If differences in environment can cause problems, it's a problem with
design. A program that changes what it does just due to differences
between the init environment and a login environment is going to be
hard to debug.


 There are tons of those, and you can't fix them all. Stupid example:
less. Behaves differently when its stdout is a terminal and when it's
not.
 The only way to ensure reproducible behaviour for a program, no matter
what it is, is to start it in a reproducible environment.



It took me ... less than a minute to find "pgrep -F".
That solves the problem of stale pidfiles.


 Do you think so? See for yourself:
 https://gitlab.com/procps-ng/procps/blob/master/pgrep.c

 It just reads the pid in the pidfile, and does its matching with
the read pid. Unless you also give other options to narrow the match,
it will have the exact same problem.
 Now, of course, you can give the executable name, and add even more
guards to make sure you don't find the wrong process. At the end of
the day, you wrote a nice and complex pgrep command line, you're
*almost* 100% sure that it will nail your process and only yours,
and you just scanned /proc to send a goddamn signal to a daemon.

 I'd rather type "s6-svc -d /run/service/my-daemon" and kill my
process with absolute certainty, using 10 or 20 times fewer system
calls than pgrep and a small fraction of the CPU time.



restart means start and stop.
reload means reread config file.


 Ah, thanks.



Using a tool dedicated to the purpose is more helpful than any service
manager can be


 For "status" ? Yes, without a doubt. I believe a "status" command
should only give the up/down state and readiness state.

--
 Laurent

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Laurent Bercot]

On 06/08/2015 16:00, Rainer Weikusat wrote:

That's all nice and dandy but it all boils down to 'the code executed by
the init script was deficient in some way'.


 Yes, just like root exploits boil down to "the code executed by the
suid program was deficient in some way".
 My point is that you shouldn't need to have 40 lines of boilerplate
to sanitize your init script before you run it; if it's so fragile,
then it's badly designed.



But that's something different. Using inetd as simple, well-known
example, if I just changed /etc/inetd.conf, I want to daemon to reread
it and reconfigure itself accordingly to avoid interrupting unrelated
services but in case its on rampage in an endless loop, I want to get
rid of the current process and start a new instance. 'SIGHUP' is just a
random convention dating back to the times when signals where the only
kind of IPC supported by UNIX(*) and the signal was basically hijacked
because a 'daemon process' shouldn't ever receive it for some other
reason. It's not universally supported and not all daemons are so simple
that "reread the config file" is the only means of controlling them at
run time. Eg, someone might want to ask bind to reload a specific zone.


 All agreed. Service-specific configuration can only be achieved by
service-specific tools.



Service management is a more complex task than
[nohup] /usr/sbin/ochsenfrosch >>log 2>&1 &


 My point exactly.



[systemd]
Is it? Or is it an extremely incomplete, ad hoc designed programming
language with a service manager and a truckload of other potentially
useful stuff attached to it in order to encourage people to swallow the
language?


 I have never said, am not saying, and probably never will say that
systemd is any good. It's not, and Lennart and Kay should go back to
engineering school, and the people who hired them should go back to HR
school. Its Embrace and Extend strategy is as pernicious as Microsoft's
in the late 90s / early 2000s. It's technically inept and politically
evil.
 Nevertheless, those guys have understood a few things, and have done a
few things better than sysvinit or anything else really. You have to
analyze it, cut out the bad parts and keep the good parts. The concept
of service management is one of the good parts. They implemented it the
systemd way, but they did implement it.

 Know your enemy. It's easy and tempting to rant for days on systemd's
weaknesses; but it's also vital to acknowledge its strengths.



'Winning' against systemd will require getting support of a
commerically more potent company than RedHat and SuSE combined and one
willing to sink a sizable amount of money into the task.


 No, I don't think so. You don't fight Goliath with Goliath. You don't
fight Microsoft's proprietary-ness by investing into Apple. The last
thing we need against systemd is another systemd, as you say yourself
in the rest of your paragraph, which I fully agree with.



But 'booting the machine' is a much simpler task and it can be solved
within the existing framework by incrementally adding the missing
bits.


 Depending on what you mean by "adding the missing bits", I may or may
not agree. I'm not suggesting doing things the systemd way, but I do
believe that a quantum leap is needed. Which, of course, does not
preclude maintaining compatibility for some time to ease the transition.



Starting from the
presumption that this will turn out to be necessary is a guess.


 You either want to do things right or you don't. If you do, then it's
not a guess: starting and maintaining services reliably requires more
than the existing framework. There are countless web pages and
heartbreaking mails on support mailing-lists to prove it.

--
 Laurent

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Isaac Dunham]

On Thu, Aug 06, 2015 at 02:21:55PM +0200, Laurent Bercot wrote:
> On 06/08/2015 11:45, tilt! wrote:
> >Thing is, init scripts tend to have problems managing services
> >that do not offer sufficient commandline interfaces as described
> >above
> 
>  There's a simple reason for that: "init scripts" aren't
> "managing services". They can more or less start and stop them,
> that's about it - and they're not even doing the starting and
> the stopping right.
> 
>  - Starting ? Sure, it's easy to start a service: run the stuff, and if
> it's a daemon, just fork it. Right ? Wrong. When you just spawn the
> stuff from your shell, it runs with your shell's environment (open fds,
> variables, terminals, so many parameters that are *difficult* to clean
> up and really harden). But when you spawn the stuff at boot time, it
> runs with the very different environment provided by init. I can't
> count how many problem reports I've read in support mailing-lists that
> were really hard to track down, but in the end it was just a config
> issue because the daemon launching script did not properly sanitize
> all its environment, so it didn't give the same results when run by
> the admin or when auto-run at boot time.

If differences in environment can cause problems, it's a problem with
design. A program that changes what it does just due to differences
between the init environment and a login environment is going to be
hard to debug.

Also, if I'm reading this right, you're implying that execline does
clearenv(), sets a new environment, and also closes all fds above 2.

>  - Stopping ? Sure, just find a daemon's pid... oh wait, we need to
> have it stored somewhere. Hence, .pid files. If you don't know why
> .pid files are evil, look it up.

It took me ... less than a minute to find "pgrep -F".
That solves the problem of stale pidfiles.

>  And then you have the other functionality. Restarting, sometimes,
> can be lighter than stop + start: maybe there's a whole lot of
> config you don't have to do when you're just restarting a daemon -
> for instance, if "restart" means "read your config file again",
> some daemons support the convention that receiving a SIGHUP should
> make them do just that. So "restart" should simply send a SIGHUP
> to those, but do stop + start on the others. That's confusing.
> There's the "reload" keyword sometimes, but are there any precise
> semantics ?

restart means start and stop.
reload means reread config file.

>  And "status". This is very service-dependent: since there is no
> global API, no service manager, scripts will query the daemon's
> status in a daemon-specific way. More vagueness again, because
> "status" doesn't define exactly what you should say, and the lowest
> common denominator is "is it alive?" but even for this basic check,
> daemon-specific interfaces are used.

The status command was originally "is it running?",
but yes, that's barely useful.
If you use a pidfile and pgrep -F, that would work.

Using a tool dedicated to the purpose is more helpful than any service
manager can be (unless your service manager is a hundred-megabyte
hog).
Think it's hung?
strace/ltrace the pid.
Wondering what files it has open?
Use lsof or look in /proc/$PID/fd/.
...

Thanks,
Isaac Dunham
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Rainer Weikusat]

Laurent Bercot  writes:

> On 06/08/2015 11:45, tilt! wrote:
>> Thing is, init scripts tend to have problems managing services
>> that do not offer sufficient commandline interfaces as described
>> above
>
>  There's a simple reason for that: "init scripts" aren't
> "managing services". They can more or less start and stop them,
> that's about it - and they're not even doing the starting and
> the stopping right.
>
>  - Starting ? Sure, it's easy to start a service: run the stuff, and if
> it's a daemon, just fork it. Right ? Wrong. When you just spawn the
> stuff from your shell, it runs with your shell's environment (open fds,
> variables, terminals, so many parameters that are *difficult* to clean
> up and really harden). But when you spawn the stuff at boot time, it
> runs with the very different environment provided by init. I can't
> count how many problem reports I've read in support mailing-lists that
> were really hard to track down, but in the end it was just a config
> issue because the daemon launching script did not properly sanitize
> all its environment, so it didn't give the same results when run by
> the admin or when auto-run at boot time.

That's all nice and dandy but it all boils down to 'the code executed by
the init script was deficient in some way'.

[...]

>  And then you have the other functionality. Restarting, sometimes,
> can be lighter than stop + start: maybe there's a whole lot of
> config you don't have to do when you're just restarting a daemon -
> for instance, if "restart" means "read your config file again",
> some daemons support the convention that receiving a SIGHUP should
> make them do just that. So "restart" should simply send a SIGHUP
> to those, but do stop + start on the others.

But that's something different. Using inetd as simple, well-known
example, if I just changed /etc/inetd.conf, I want to daemon to reread
it and reconfigure itself accordingly to avoid interrupting unrelated
services but in case its on rampage in an endless loop, I want to get
rid of the current process and start a new instance. 'SIGHUP' is just a
random convention dating back to the times when signals where the only
kind of IPC supported by UNIX(*) and the signal was basically hijacked
because a 'daemon process' shouldn't ever receive it for some other
reason. It's not universally supported and not all daemons are so simple
that "reread the config file" is the only means of controlling them at
run time. Eg, someone might want to ask bind to reload a specific zone.

Runtime control of a long running process is a task dependent job. And
since init scripts are task-agnostic, they cannot (sensibly) provide
it. 

[...]

>  Init scripts just don't cut it, no matter where they come from, no
> matter how simple they are or how complex they are.

[...]

>  What is needed is a *service manager*, a real, full-fledged thing
> that doesn't only start and stop services, but does it properly,
> without hacks, with the same environment every time, and can tell
> you whether a service is alive or not, ready or not, and gives you
> a unified interface to interact with it in a simple way. (You still
> have to use daemon-specific interfaces for the more complex ways,
> because no service manager can unify that.)

Service management is a more complex task than

[nohup] /usr/sbin/ochsenfrosch >>log 2>&1 &

(although this already gets one rather far) and moreover, it's comprised
of both generic and task-specific jobs and it's sensible to decouple
these two: Let the applications process deal with its problems and
handle the generic issues with other code. Further, the runlevel/ init
script mechanisms and conventions address only a part of the generic
problems services management involves. But "X can't be used" is not
something which follows from "I need feature beyond those provided by
X".

>> what does SystemD do to address this problem?
>
>  It does exactly that. Among the myriad of useless things that systemd
> does, it does at least one useful thing: it's a proper *service
> manager*.

Is it? Or is it an extremely incomplete, ad hoc designed programming
language with a service manager and a truckload of other potentially
useful stuff attached to it in order to encourage people to swallow the
language?

[...]

> Winning against systemd involves providing better designed
> alternatives to the useful parts of what it does, and service
> management is one of those: there is a qualitative jump to do here,
> and messing around with init scripts is only a quantitative jump and
> unfortunately isn't going to accomplish anything if not paired with a
> serious redesign of how booting a machine should be done.

'Winning' against systemd will require getting support of a
commerically more potent company than RedHat and SuSE combined and one
willing to sink a sizable amount of money into the task. And that's only
a part of the solution because you'll also have to be on the lookout for
senior developers whose surnames 

Re: [DNG] Init scripts in packages

[Rainer Weikusat]

"tilt!"  writes:
> On 08/06/2015 11:28 AM, Rainer Weikusat wrote:
>> [...]
>> But a bare-bones init script does really only three things:
>>
>> 1. Execute a command to start something.
>> 2. Execute a command which stops it again.
>> 3. Execute 2) then 1) for a restart.
>
> There are additional actions required by [LSB]; out of these
> I appreciate a "status" action the most - the use of the others
> is debatable if not questionable, but "status" I find really
> important.

Even the restart is already an example of 'cat -v programming' (or
'coral reef design' -- $stuff growing on $other_stuff because $other_stuff
already exists) and 'status' is worse as the semantics are also muddy: What
ever the output happens to be, it contains information which happened to
be accurate at some random point in time in the past and it's neither
known if it's still accurate nor when it was accurate for the last
time. Further,  what precisely comprises a useful bit of "status
information" is very program dependent. Eg, the status of a Java
application running on top of JBoss might be 'the JVM got started and
deployment worked, however, because of a typo in some named query,
Hibernate couldn't make sense of it so it isn't really working at the
moment', the status of a ssh server might be "everything's fine but can
we disconnect China as the IPs could be put to a better use than for
scripts trying to break into the system", the status of a long runnning
computation might be "still running, n% completed" and some other task
might just wait for something to do.

Insofar a particular task involves a idempotent process with indefinite
lifetime ('idempotent' here supposed to mean that 'restart in case of
unexpected termination' make sense), "did it terminate despite it wasn't
asked to" would be a useful piece of generic status information but
that's primarily interesting to a program supposed to manage 'daemon
processes' and that's beyond the scope of an start/stop script which
knows nothing about the effects/ purpose of the tasks it's employed to
start or stop.
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Laurent Bercot]

On 06/08/2015 11:45, tilt! wrote:

Thing is, init scripts tend to have problems managing services
that do not offer sufficient commandline interfaces as described
above


 There's a simple reason for that: "init scripts" aren't
"managing services". They can more or less start and stop them,
that's about it - and they're not even doing the starting and
the stopping right.

 - Starting ? Sure, it's easy to start a service: run the stuff, and if
it's a daemon, just fork it. Right ? Wrong. When you just spawn the
stuff from your shell, it runs with your shell's environment (open fds,
variables, terminals, so many parameters that are *difficult* to clean
up and really harden). But when you spawn the stuff at boot time, it
runs with the very different environment provided by init. I can't
count how many problem reports I've read in support mailing-lists that
were really hard to track down, but in the end it was just a config
issue because the daemon launching script did not properly sanitize
all its environment, so it didn't give the same results when run by
the admin or when auto-run at boot time.

 - Stopping ? Sure, just find a daemon's pid... oh wait, we need to
have it stored somewhere. Hence, .pid files. If you don't know why
.pid files are evil, look it up.

 And then you have the other functionality. Restarting, sometimes,
can be lighter than stop + start: maybe there's a whole lot of
config you don't have to do when you're just restarting a daemon -
for instance, if "restart" means "read your config file again",
some daemons support the convention that receiving a SIGHUP should
make them do just that. So "restart" should simply send a SIGHUP
to those, but do stop + start on the others. That's confusing.
There's the "reload" keyword sometimes, but are there any precise
semantics ?
 And "status". This is very service-dependent: since there is no
global API, no service manager, scripts will query the daemon's
status in a daemon-specific way. More vagueness again, because
"status" doesn't define exactly what you should say, and the lowest
common denominator is "is it alive?" but even for this basic check,
daemon-specific interfaces are used.

 Init scripts just don't cut it, no matter where they come from, no
matter how simple they are or how complex they are. OpenRC is a step
in the right direction, because it provides real support for more
things than sysv-rc, but even it doesn't address all the issues of
init scripts.

 What is needed is a *service manager*, a real, full-fledged thing
that doesn't only start and stop services, but does it properly,
without hacks, with the same environment every time, and can tell
you whether a service is alive or not, ready or not, and gives you
a unified interface to interact with it in a simple way. (You still
have to use daemon-specific interfaces for the more complex ways,
because no service manager can unify that.)



what does SystemD do to address this problem?


 It does exactly that. Among the myriad of useless things that systemd
does, it does at least one useful thing: it's a proper *service manager*.
I'm not saying it's correctly designed - it's not: see for instance
http://ewontfix.com/15/ - but it's definitely one of the real, serious
advantages that systemd has over its current alternatives, and it is
important to acknowledge it.

 Winning against systemd involves providing better designed alternatives
to the useful parts of what it does, and service management is one of
those: there is a qualitative jump to do here, and messing around with
init scripts is only a quantitative jump and unfortunately isn't going
to accomplish anything if not paired with a serious redesign of how
booting a machine should be done.

 That's why I'm currently writing a service manager, and will keep you
guys informed when it's out.

--
 Laurent

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[tilt!]

On 08/06/2015 11:28 AM, Rainer Weikusat wrote:

[...]
But a bare-bones init script does really only three things:

1. Execute a command to start something.
2. Execute a command which stops it again.
3. Execute 2) then 1) for a restart.


There are additional actions required by [LSB]; out of these
I appreciate a "status" action the most - the use of the others
is debatable if not questionable, but "status" I find really
important.

This is usually where init scripts get ugly and try to work
around missing commandline interfaces of the services they
attempt to manage. As soon a /bin/sleep command appears, it's
FUBAR.

Thing is, init scripts tend to have problems managing services
that do not offer sufficient commandline interfaces as described
above; what does SystemD do to address this problem?

Regards,
T.

Links:
[LSB] Linux Standard Base Core Specification 3.1.
20.2. Init Script Actions.
URL: 
http://refspecs.linuxbase.org/LSB_3.1.0/LSB-Core-generic/LSB-Core-generic/iniscrptact.html

___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Rainer Weikusat]

Alexey Rochev  writes:
> Currently Debian packages contains both systemd units and init scripts.
> However, Debian developers refused to support several init systems. So it's
> only a matter of time when they remove init scripts from packages.

This would rid the world of a lot of code of very dubious usefulness most
of which should never have been written. One of the justifications for
systemd is actually that they idea (orchestrate/ implement system
startup based on the filesystem and with code written in the Bourne
shell language) must be bad because "Can't you see that a horrendous
mess they made of it?!" (only partially based on the visual effect of
seeing a lot of complicated looking code in a language one happens to be
almost totally unfamiliar with and all written by different people, ie,
in different styles --- even a lot of people who consider themselves
highly competent developers can't really cope with that).

But a bare-bones init script does really only three things:

1. Execute a command to start something.
2. Execute a command which stops it again.
3. Execute 2) then 1) for a restart.

Implementing this needs only marginally more text than this description
and the world won't end anytime soon for want of a fifteen (estimate)
line program. If anything else fails, I'd be willing to write it for
you.

Pet peeve: As part of my present job, I've developed a set of relatively
small command-line tools supposed to add more 'verbs' suitable for
managing services to the shell language (the largest is about 1300 LOC
but contains some features which really ought to become independent
programs). After experiences with a (simple) process manageing init I
wrote for an earlier embedded system, this was an intentional experiment
in "let's try it with small, independent programs written in C while
using the shell as control/ skeleton language for connecting them and
see how far I can get with that" (I can always try it with a big,
complicated program should this fail). After five years, the conclusion
is that this worked out beautifully. But this is - of course - all code
my employer claims copyright in, hence, it will ultimatively end up
getting systemd (or bernsteined or ... insert whatever the name of your
favorite init-with-process-manager-ipc-system-and-some-kitchen-sinks
written in a "real" programming language happens to be).


life is futile
we're all gonna to die
in the time between
have a coffee
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[Daniel Reurich]

On 05/08/15 23:29, Alexey Rochev wrote:

Currently Debian packages contains both systemd units and init
scripts. However, Debian developers refused to support several init
systems. So it's only a matter of time when they remove init scripts
from packages. What will Devuan developers do when it happens? We can
use sysvinit and Devuan because these init scripts exist.


Debian will likely either cease to exist or just have become a rebranded
Fedora by the time that happens, so Devuan will likely have forked those 
packages anyway.  Besides, once init-scripts have been written and 
proven they require very little maintenance so it' unlikely packages 
will drop the init-scripts soon.





--
Daniel Reurich
Centurion Computer Technology (2005) Ltd.
021 797 722
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng

Re: [DNG] Init scripts in packages

[James Powell]

Currently Debian packages contains both systemd units and init scripts. 
However, Debian developers refused to support several init systems. So it's 
only a matter of time when they remove init scripts from packages.What will 
Devuan developers do when it happens? We can use sysvinit and Devuan because 
these init scripts exist.
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng