Re: nosh version 1.33
Talking of Gentoo, Guillermo: The Gentoo people have been inspired by Archnosh, but it looks like they could benefit from your experience. * https://wiki.gentoo.org/wiki/Nosh
Re: nosh version 1.33
Guillermo:
> Is this so that package managers can alert users when they try to install
> packages with these conflicts?
No. It was to make it so that M. Caravia could build packages for Arch Linux.
Arch Linux has a system like /command, except that it doesn't have the
alternatives-like symbolic link idea and it has an oddball name from the 1970s.
(-: I wasn't planning to cross the bridge of having things install outwith
/usr/local yet, but M. Caravia wanted that. However, when just taking the
existing setup and substituting /usr for /usr/local, xe had a list of conflicts
on Arch Linux. Since I had the shims package concept anyway, I extended it.
Guillermo:
> It seems that every program and program alias created by package/compile
> in ${src}/command/ is installed to some ${dest}/nosh-*/ directory, except the
> aliases disable, enable, preset, reset, show and show-json, so no binary
> packages will contain them. I'm not sure if this is intentional, and it is a
> minor
> thing, but I though I should mention it as well.
Only some of these sub-commands of system-control are available like that, and
in general only where something such as upstart compatibility (hence the start,
stop, and status aliases) requires it. These names are fairly central. Note
for starters that there is a completely different command named "reset" that is
in common use. And "enable" and "disable" are System 5 printer spool management
commands. (-:
Re: Vim `.swp' files in nosh source tarballs
I purge the directories of these from time to time, as I have just done again. But they tend to very slowly accrue, for one reason or another. It's not a major worry. Anyway, better that the archive have everything including some harmless text editor temporary files than that it have one vital file missing because of a wildcard exclusion/deletion, which I have experienced in the past. (-:
nosh version 1.33
The nosh package is now up to version 1.33 .
* http://jdebp.eu./Softwares/nosh/
*
https://www.freebsd.org/news/status/report-2015-07-2015-09.html#The-nosh-Project
* http://jdebp.info./Softwares/nosh/
This has been held back because of work being done by someone else. I don't
want to steal xyr thunder, so I'll leave the announcement of that work to xem.
Suffice it to say that it will interest a new group of people.
There are several major improvements in 1.33 .
Packaging
-
In the version 1.29 announcement I said that the Debian packaging system was
going to be brought into line with the system used for FreeBSD/TrueOS and
OpenBSD. This is now done. Debian and the BSDs all now use a similar system
for generating each package manager's package maintenance instructions from an
abstract package description.
==
=== IMPORTANT UPGRADE NOTE FOR Debian: ===
==
An important consequence of the aforementioned is that the semantics of the
nosh-bundles package have changed. In earlier versions, the various nosh-run-*
packages were how one set services running, except for a small rump set of
services that were set up by the nosh-bundles package.
This is now no longer the case. The nosh-bundles package now presets and starts
no services at all. *All* running of services must be achieved with the
nosh-run-* packages or some other sets of scripts and presets.
To this end, there are now two new packages, nosh-run-debian-desktop-base and
nosh-run-debian-server-base. These parallel the
nosh-run-{freebsd,trueos}-{desktop,server}-base packages already available since
1.29 for FreeBSD/TrueOS. You must install, for a working fully-nosh-managed
system, exactly one of the nosh-run-debian-{desktop,server}-base packages.
If you are running nosh service management under systemd, you can of course run
as many or as few services under the nosh service manager as you care to switch
over from systemd. But if you are running a fully-nosh-managed system these
packages will arrange to run the various fundamentals that one pretty much
cannot do without, such as mounting/unmounting volumes, running
udev/eudev/vdev/mdev, binfmt loading, and initializing the PRNG.
Log service account names
-
The naming scheme used for the user accounts for dedicated log service users has
changed. Installing the new nosh-bundles package should automatically rename
all existing log service accounts to use the new scheme.
The new naming scheme is slightly more compact, and copes better with services
that have things like underscores and plus characters (e.g. powerd++) in their
names.
As an ancillary to this, system-control now has an "escape" subcommand which can
be (and indeed is) used in scripts to perform the escaping transformations.
More packages
-
There are now four more -shims packages, for commands whose names conflict with
commands from other packages: nosh-kbd-shims, nosh-bsd-shims, nosh-core-shims,
and nosh-execline-shims.
nosh-kbd-shims, for example, contains a chvt shim that is an alias for the (also
new) console-multiplexor-control command; with it, and suitable privileges to
access the virtual terminal's input queue, one can switch between multiplexed
user-space virtual terminals in much the same way as the old chvt command does
with kernel virtual terminals.
The Z Shell command-line completion for the various commands in the toolset
(system-control, svcadm, shutdown, svstat, and so forth), which has been
available to the people building from source for a while, is now also available
as a binary package.
Configuration import
ldconfig on TrueOS is now properly handled. In particular, the external
configuration import subsystem now correctly pulls in and converts all of the
ldconfig directories. (TrueOS has a lot more things that require ldconfig
support than stock FreeBSD does.)
The configuration import subsystem also now handles instances of Percona server,
alongside MySQL and MariaDB. Moreover, these are now handled by the same set of
service bundles, which always produce service bundles named mysql@*. MySQL
version 5.7 or later is now assumed.
The configuration import subsystem now automatically generates OpenVPN service
bundles based upon the current OpenVPN configuration.
===
CAVE: OpenVPN
===
The upgrade process attempts to remove the old hardwired openvpn@server and
openvpn@client service bundles. However, you might encounter remnants of these
service bundles lying around in /var/sv that you will find that you need to
clean up by hand.
GOPHER
--
To accompany the new gopherd server in djbwares 5, there is a gopher6d service
bundle that runs it, serving up the same static files area as http6d, https6d,
and ftp4d do.
The FreeBSD, OpenBSD, and Debian package re
Immortal: "based on daemontools & runit"
"Immortal", a tool that is a "*nix cross-platform (OS agnostic) supervisor based on daemontools & runit" according to its blurb, has just come up on Hacker News. I mentioned M. Bercot. * https://news.ycombinator.com/item?id=13994687
djbwares version 5
djbwares is now at version 5. * http://jdebp.eu./Softwares/djbwares/ * http://jdebp.info./Softwares/djbwares/ This contains some long-overdue changes: ip6.int has been replaced by ip6.arpa in tinydns-data and dnscache, and rblsmtpd no longer falls back to using an RBL that has been defunct for many years. It also contains some additions: some UCSPI-SSL capability, a new gopherd UCSPI server to go alongside httpd and ftpd in publicfile, and most of the previously missing manual pages (including a few for commands which had no manuals in the original toolsets). There are no longer any placeholder manual pages for the "man" command. There are still a few manual pages that are only present in roff form, though. You can see gopherd in action: * gopher://jdebp.info./1/Repository/
Re: register runsvdir as subreaper
Laurent Bercot: if runsvdir uses PATH resolution to find runsv prog[0] ="runsv"; prog[1] =name; prog[2] =0; ... pathexec_run(*prog, prog, (const char* const*)environ);
Re: register runsvdir as subreaper
Laurent Bercot: You want runsvdir to be your reaper, so you'd just run "local-reaper runsvdir scandir" instead of "runsvdir scandir". Actually you'd run > local-reaper true runsvdir scandir
Re: register runsvdir as subreaper
Roger Pate: Simple, but you do point out in local-reaper's docs, "One cannot just apply this willy-nilly." :P One can apply it to runsvdir. I checked. It waits for arbitrary children, and does not get confused by children that it did not spawn turning up. Roger Pate: (And how should we apply chain-loading local-reaper to runsv?) How to wrap a program inside something else that does additional stuff then exec()s the original program *is* a solved problem on Unices and Linux, you know. (-:
Re: register runsvdir as subreaper
Mitar: I would like to ask if runsvdir could by default be defined as a subreaper on Linux. You are talking to people well versed in the idea of chain-loading programs for affecting process state. The answer here is to simply run runsvdir through a chain-loading program that sets the process as a subreaper. You could write your own, or use the one that I wrote, packaged up, and published. I called it "local-reaper". * http://jdebp.eu./Softwares/nosh/guide/local-reaper.html
Re: register runsvdir as subreaper
Kamil Cholewiński: Reaping orphaned children should be the duty of PID 1. * http://unix.stackexchange.com/a/197472/5132 * http://unix.stackexchange.com/a/177361/5132 There is no objective basis for such a claim, this not actually being a minimal requirement of process #1. Welcome to the future. Your service manager does not have to be process #1. Your interactive logins are ordinary services controlled by your service manager. Orphaned child processes are reparented to your service manager or to some other process that is even closer to them.
Re: nosh version 1.31
Guillermo: One could instead attach controllers to the hierarchy rooted in /sys/fs/cgroup/systemd, for example by mounting it with 'mount -t cgroup -o cpu,name=systemd cgroup /sys/fs/cgroup/systemd' (not with system-manager as process 1), and things would appear to be OK: It's tempting to do that, and make the adjustments to the current control group location code. But the harsh reality I suspect is that version 1 control groups are on the way out. Let's put it this way: I am targetting Debian Linux, notorious for being behind the times, and even that has a kernel (in Debian 8 backports, admittedly) with version 2 control groups now. I'll save the idea up. But there's other, higher priority stuff to do. I've just finally turned unload-when-stopped into a simple service-control --exit wrapper, for example. (-:
nosh version 1.32
The nosh package is now up to version 1.32 . * http://jdebp.eu./Softwares/nosh/ * https://www.freebsd.org/news/status/report-2015-07-2015-09.html#The-nosh-Project * http://jdebp.info./Softwares/nosh/ This release fixes two problems with Gentoo Linux (control group version detection and a problem with mounting API filesystems) that we hashed out on the Supervision mailing list. It furthermore contains a change to the way that convert-systemd-units generates service bundles that fixes problems with control group setup when the service unit defines a "slice" for the service or when the service unit is a template. In furtherance of that there's a new create-control-group command. Other things in this release include improvements to the (unpackaged) Z Shell command-line completions, which now display option completion menus properly; some improvements to the Terminals chapter in the Guide; fixes to various service bundles that were using shell reserved words and operators such as "for" and "&&" without explicitly invoking the shell; additions to userenv for setting DBus and XDG Runtime variables; and a fix that prevents "system-control reset" from looping indefinitely when run by an unprivileged user such as "messagebus" that lacks access to the control/status API. The major improvement in this release, though, is to console-fb-realizer on TrueOS. FreeBSD gives console-fb-realizer uhid device files to use for input devices, which speak the USB HID report protocol and which console-fb-realizer has been happy with for a long time. TrueOS provides either ums/ukbd devices, which lack various features because they speak the old sysmouse and atkbd protocols, or ugen devices. There are no uhid devices available. console-fb-realizer can now use the ugen devices. Moreover, it will detach the ums/ukbd drivers from the ugen devices using the new detach-kernel-usb-driver command, so that there aren't two things both attempting to read HID reports. console-fb-realizer also now correctly sets the keyboard LEDs on both FreeBSD and TrueOS. There have been several minor adjustments to the kernel VT sharing parts of console-fb-realizer, preparatory to splitting the program up into separate parts for input and output devices, permitting things such as multiple keyboards each with its own keyboard map and numlock semantics, in a future release.
nosh with Debian's sysstat package
Someone:
I haven't installed much else yet on the system but I tried the
sysstat package which gives me the following error:
preset: ERROR: sysstat: No such file or directory
I haven't yet investigated this problem. Sysstat seems to be part of
the Debconf enable/disable system, I'm not quite sure how that
interacts with nosh.
This is a good example for general consumption.
The maintenance script for the package is trying to enable the "sysstat"
service using the "update-rc.d" command. You've installed the shim for
this command from the nosh-debian-shims package, so the maintenance
script is actually ending up trying to preset the "sysstat" service
bundle using "system-control preset". You don't have a "sysstat"
service bundle. Yet.
Ironically, if the systemd support in Debian's "sysstat" were as good as
the author's own, you could just make one. The origin package comes
with a systemd service unit, built by passing this through a macro
processor to turn things like @SA_LIB_DIR@ into "/usr/lib/sysstat":
* https://github.com/sysstat/sysstat/blob/master/sysstat.service.in
Debian, however, only builds and packages up a Debian-supplied van
Smoorenburg rc file. It doesn't actually package up the systemd support
that comes from the author. It does provide the /usr/lib/sysstat/sa1
script that is referenced by the systemd service unit, however. So you
could take the sysstat.service.in, manually make a sysstat.service out
of it, and pass that through convert-systemd-units to make a service
bundle that would invoke /usr/lib/sysstat/sa1 .
However, we are heading into systemd House of Horror territory here, as
Debian also provides a "Debianized" version of the sa1 script as
/usr/lib/sysstat/debian-sa1 that does the stuff that Debian's van
Smoorenburg rc script does. So using what's in the box we would have
sysstat.service which sets up settings the systemd way, running the
debian-sa1 script that sets up things the Debian way (reading
/etc/defaults/sysstat), running the sa1 script that sets things up the
RedHat/SuSE way (/etc/sysconfig/sysstat). It's a mess of nested shell
scripts and overlapping configuration mechanisms. And that's
overlooking the surprise secret second service disable mechanism.
The systemd people don't like surprise secret second service disable
mechanisms, and the modern Debian practice is to not have them. The
Debian sysstat package has more than one thing to improve. Moreover
there's no real need for all of these configuration mechanisms,
especially since the underlying command has only two knobs to twiddle in
the first place. So start with a more ideal-world version of what
sysstat should have for systemd: a simple service unit that has 1
configuration mechanism, and cuts out all of the daft middle-men layers
of shell scripting entirely.
1. Take this service unit instead. Call it sysstat.service .
2. Use convert-systemd-units to make a service bundle from it. chown
everything to root if you didn't do this as root.
3. Place that in /var/local/sv/sysstat .
4. install -d -m 0755 /var/local/sv/sysstat/service/env
5. system-control set-service-env sysstat OPTIONS -D
See what happens when you install the package then.
[Unit]
Description=Insert a dummy record in sysstat's current daily data
file to indicate that the counters have restarted from 0.
[Service]
# The service is "ready" after it has run to completion.
Type=oneshot
# This enables controlling service options with rcctl set and rcctl get .
EnvironmentDirectory=env
# Two - characters, note. Also, this is specifically targetting
being converted into a nosh service bundle.
ExecStart=/usr/lib/sysstat/sadc -F -L ${OPTIONS} "${DIR:--}"
[Install]
WantedBy=workstation.target
nosh packaged for Arch Linux
Someone: I am trying to create one or multiple packages for Archlinux to install nosh This has come up several times with multiple people, now. I have no Arch Linux machines, and I am entirely dependent from Arch Linux people for the packaging part. As I already told one person back in August 2016, this is your opportunity to shine. (-: My second redo is already packaged for Arch, I am told: * https://aur.archlinux.org/packages/redo-jdebp/ * https://git.parabola.nu/abslibre.git/plain/pcr/jdebp-redo/ Here's what I know. The redo packaging runs package/compile followed by package/export. This is not the way to approach packaging the nosh toolset. It's not one giant lump, as you can see from the FreeBSD/TrueOS, Debian, and OpenBSD packages that I have done. What you Arch Linux people with the packaging experience have to provide is a way, on Arch, for one source package to build multiple binaries packages. This should not be hard. It's a very common thing to do, after all. It is, indeed, pretty much the norm in the worlds of Debian and Ubuntu. As you can see from the WWW pages about the FreeBSD/TrueOS, Debian, and OpenBSD packages, there are doco packages, service bundle packages, multiple toolset packages, and multiple -run packages. The way that this works on Debian and the BSDs is largely driven by package/debian/rules and package/bsd/rules, which are in turn determined by Debian's package build system, in particular its dpkg-buildpackage tool. Notice what the rules system does. It runs package/export to populate the ./debian/tmp/ tree, and then package/stage to populate the individual binary package trees (with links) from that in the layout that is actually used within the packages. The Debian Maintainers' Guide and the FreeBSD Porters' Handbook have more on the concept of "staging" when building binary packages. * https://www.debian.org/doc/manuals/maint-guide/build.en.html#completebuild * https://www.freebsd.org/doc/en_US.ISO8859-1/books/porters-handbook/special.html#staging Coming up with how something like this works on Arch is another part of your opportunity to shine, Arch packaging people. Note that I currently plan to take package/export out of the build process and have package/stage work directly from (a copy of) the original slashpackage tree, i.e. command/ manual/ config/ guide/ et al.. Rearranging everything from slashpackage into another layout under ./debian/tmp/ only in order to rearrange it again in the staging areas seems like an unnecessary complexity. pax -r -w of the original slashpackage layout into ./debian/tmp/ or ./bsd/tmp/ seems a better idea. There has been for a fair while now an important note at http://jdebp.eu./Softwares/nosh/source-package.html that the package/export command may change. It does both too much and too little as a tool for installing from source. It exports one giant lump, and it doesn't do all of the things that the package "maintenance scripts" do. In pursuit of that latter, back in 1.29 the BSD side of package creation was modified to make it possible to auto-generate installation/upgrade/deinstallation scripts for the old and new BSD package tools. This is, as you can see, done with package/bsd/gencontrol, which makes (the very different) +MANIFEST files for both the old and new BSD package tools. This is on the cards for Debian as well. For Arch, the build system will need to process a similar set of files to the various package/bsd/nosh-*.p files, in some way that is appropriate to how such maintenance scripts are handled on Arch, as well as do more specific things for likes of nosh-run-system-manager. Yes, this is vague. I've refactored this into a way of making different maintenance scripts from effectively non-imperative lists of service bundles and user accounts. I'll press on with getting you a set of package/debian/nosh-*.p files, which at least will list the Linux service bundle suite not the BSD one, which can then be patched into arch/*.p files. You have to work out how to then join these lists up with however Arch does installation/upgrade/deinstallation scripts in packages, however. This is another part of your opportunity to shine, Arch packaging people.
Re: nosh version 1.31
Guillermo: 11363 login 1:name=systemd:/service-manager.slice/agetty@.service/agetty@tty1.service 115 113 bash 1:name=systemd:/service-manager.slice/agetty@.service/agetty@tty1.service 131 115 ps 1:name=systemd:/service-manager.slice/agetty@.service/agetty@tty1.service agetty is overkill for a virtual terminal. (-:
Re: nosh version 1.31
Guillermo: It's just a matter of ordering constraints, [...] You are forgetting CONFIG_PROC_FS. (-:
Re: nosh version 1.31
Guillermo: The problem seems to be those whose corresponding array of iovecs doesn't have a 'from' element, I hit this with cgroupfs and tmpfs in version 1.28 and thought that it was just specific to those filesystem types. The reason that I never hit it with procfs, devtmpfs, and sysfs is actually tied in to your other problem. Guillermo: And the catch-all logger reports this: Reading the logs is always a good thing, I have found. In this case, it meant that I knew what was going on in just a couple of minutes. Otherwise I'd be wondering why this worked in my system when I bootstrapped into the older Debian 8 kernel to test what happens when only v1 control groups are available, but it didn't work for you. Debian's initramfs mounts /proc, /sys, /run, and a whole lot of other stuff before /sbin/init is run. Yours does not. I see far more messages in my log than you do. So: 1. There's a chicken-and-egg situation with having to read /proc/filesystems in order to determine what API filesystems can be mounted. Good old Linux, eh? 2. The procfs, devtmpfs, and sysfs problems did exist, but I never hit them because on Debian they had already been mounted with a "from" name. I've already fixed the "from" problem ready for 1.32. I'll have a go at fixing the other one, too. There are simply going to have to be two phases of API filesystem mounting. Three on Debian. (-:
[trueos/trueos-core] PCDM-session does not wait for the right process ID (#260)
For those joining us, TrueOS (formerly PC-BSD) has a display manager.
It involves a set of rather confusingly named programs. An rc shell
script named pcdm invokes a shell script named PCDMd, which invokes
xinit, which invokes a C++ program named PCDM-session, which invokes a C
program named pcdm-session, which invokes another shell script, which
invokes more shell scripts and runs the GUI login session. There's a
whole lot of mess in there and several outright bugs. I gave the TrueOS
people a couple of suggestions and fixes.
* https://github.com/trueos/trueos-core/issues/260
* https://github.com/trueos/trueos-core/issues/261
Ken Moore:
I just wanted to point out that the code block you mention is there,
but I am fairly certain I left disabled/disused via a compile flag in
the main() function. That was just some random code I left in there
while testing out an alternative to the PCDMd startup script. I may
resurrect it later on with version 2 of PCDM, but my available time is
still too limited to tackle this right now.
You should let the world help you. Laurent Bercot et al. can help you.
I can help you.
You've switched TrueOS from Mewburn rc to OpenRC. Laurent Bercot et al.
can help you because OpenRC can integrate with M. Bercot's s6 service
manager. You can get PCDM running under proper service management, done
by s6. Then you do not need *either* of the Poor Man's Dæmon
Supervisors that you have, neither the one written in shell script in
PCDMd nor the (as the bug report explains, broken) one written in C++ in
PCDM-session.
* https://github.com/OpenRC/openrc/blob/master/s6-guide.md
* http://www.mail-archive.com/supervision@list.skarnet.org/msg00848.html
* https://github.com/trueos/pcbsd/blob/releng/10.3/src-qt5/PCDM/PCDMd#L52
* https://github.com/trueos/pcbsd/blob/master/src-qt5/PCDM/src/main.cpp#L240
We can help you with more than that. We know how to do this stuff.
We've done it. The likes of M. Bercot, Gerrit Pape, Wayne Marshall, me,
and others all look at your pcdm-session program and the first thought
(if my experience is anything to go by) is that it is setuidgid done ...
well ... very oddly. Again, you can just use tools like s6-setuidgid
rather than reinventing this particular wheel.
* http://www.skarnet.org/software/s6/s6-setuidgid.html
*
https://github.com/trueos/pcbsd/blob/releng/10.3/src-qt5/PCDM/src/pcdm-session.c#L35
In fact, I have successfully replaced your entire pcdm-session program
with my two tools setuidgid and userenv, as follows:
#!/bin/sh -
chown -v -h -- "$1" "${XAUTHORITY}"
exec >>"$5" 2>&1 \
setuidgid --supplementary "$1" \
userenv --set-path --set-other --set-tools --set-timezone
--set-locale \
sh "$4"
This illustrates the benefits of your not reinventing this particular
wheel. Not only have we already written toolsets to handle this sort of
stuff, we've written far more capable tools than what you are
reinventing. My userenv, for example, is designed to set up the
environment variables relevant to a user account. It actually
understands the FreeBSD login.conf system, and so with a couple of
general-purpose tools, rather than a custom PC-BSD program, I actually
get *better* functionality. All of my PCDM login sessions now get all
of the environment variables set up that I have in ~/.login_conf and
/etc/login.conf . And I don't get an extra process, running as me,
sitting around in the process list, doing nothing more than waiting for
a child process to terminate.
Which brings us to other obvious things that we have some expertise in.
All of these programs that start background desktop applications leave
unnecessary shell processes lying around. The lesson from the
daemontools world that can be usefully applied here is to ensure that
they overlay the shell processes with the main program, by using "exec".
*
https://github.com/trueos/pcbsd/tree/releng/10.3/src-qt5/xtrafiles/local/share/pcbsd/xdg-auto
These leave processes around parented to process #1, not all of which
properly exit when the X server goes away. Martin Misuth has been
working on cleaning up all the dross left around by services, which
covers desktop sessions, using subreapers.
We even know how to run display managers under proper service
management. I've been running TrueOS/PC-BSD under nosh service
management for years, now. I've been running PCDM under proper service
management since before nosh version 1.25, back in January 2016. I've
done all of the work to turn PCDMd and its collection of subordinate
shell scripts into a more coherent system that runs as a service under a
service manager, including even the "firstboot" session and the display
wizard fallback. I've even done part of the work of stopping it using
global flag files in /tmp, which is both a security problem and
something that stands in the way of the goal of having multiple PCDM GUI
login sessions running
nosh version 1.31
The nosh package is now up to version 1.31 . * http://jdebp.eu./Softwares/nosh/ * https://www.freebsd.org/news/status/report-2015-07-2015-09.html#The-nosh-Project * http://jdebp.info./Softwares/nosh/ This release fixes a problem with emergency mode that was introduced by accident in 1.29 . The emergency-login@console service was not properly enabled by package installation. Now it once again is. There are a number of bug fixes in this release, such as rare corner cases in how convert-systemd-units generates arguments to pass to sh, what port the nginx server part of Appcafe binds to when not the default, the use of setuidgid-fromenv to set more than 1 supplementary group ID, and making the Makefile in tinydns@* services work with both BSD and GNU make. Various service bundles that perform clean-up-directories actions at bootstrap have been made more difficult to accidentally re-trigger after bootstrap. There is also a fair amount of new features: * The automatically-generated data for tinydns@* services now encompasses all of the reverse lookup domain names for private/local IP addresses, so none of the DNS traffic involving such lookups will leak out of your machine/organization to the rest of Internet. * The userenv command has gained the ability to (optionally) set a whole lot more environment variables from the capabilities in /etc/login.conf and ~/.login_conf . It now can be used as the setup-the-user-environment part of a command chain that is designed to perform the setup of an interactive login session. This is particularly useful for fixing PCDM, the display manager in TrueOS. * The pipe command can now arrange to clean up the child process in one of two ways. This is made use of in the dnscache service bundles, and dnscache services no longer contain the perpetual zombie process that they had in version 1.30 . * Presets now support wildmat-style character set wildcards. e.g. one can now write "ttylogin@vc[0-9]-tty" as a service name pattern. * If you have been using the --verbose option to the start/stop/reset subcommands of system-control, you'll notice that it now colourizes its output. Its output has also been adjusted to more clearly indicate blocked services and what they are blocked by. The big item is that there is now a complete set of simple control groups manipulation commands, the pre-supplied service bundles all make use of it, and all service bundles created by convert-systemd-units make use of it. (All of this is a no-op on FreeBSD/TrueOS and OpenBSD, of course.) If you've read the Linux doco, you'll know that control groups do not require any sort of centralized gatekeeper process, and are a decentralized system that can be driven with just the echo command. In practice, using echo is non-trivial. The move-to-control-group, delegate-control-group-to, and set-control-group-knob commands take the hassle out of working out exactly what to echo where. They do all of the hard work of determining what the directory name of the current control group under /sys/fs/cgroup is, and present a simple system allowing one to create and navigate to another control group, delegate control over the current control group (and its subgroups) to an unprivileged user, and set control group knobs. The set-control-group-knob utility further illustrates the convenience functionality over and above a simple echo command. It can calculate a knob setting as a percentage of another number, handle SI and IEEE/IEC multiplier suffixes, and translate the device file names that are (comparatively) convenient for humans into the literal major and minor device numbers that the Linux control groups API actually operates in terms of. There are new chapters in the Guide covering the automatic import of FreeBSD 9 and PC-BSD Warden jails, how jailing services on FreeBSD/TrueOS works, and limiting services. The limiting services chapter covers both the original Unix resource limits system and Linux control groups.
Re: Adding capability control into the `run' script comparison page
Guillermo: Nope, it's a 4.4-series kernel. I've wangled a later kernel out of Debian 8 backports. (-: I'm going to have to write v1 detection for you, then. Alright. 1.31 is going to be slightly delayed.
Re: Adding capability control into the `run' script comparison page
Jonathan de Boyne Pollard: To anyone running the service manager and bundles from nosh version 1.28 or later on Linux: You are encouraged to look at your control group hierarchy, with a tool like "systemd-cgls /", with the "cgroup" field of the ps command, or by simply listing your /sys/fs/cgroup/ hierarchy. You are in for an interesting surprise. There are more interesting surprises in the same vein in 1.31. I've put a sneak peak of the 1.31 Guide up for you.
Re: Adding capability control into the `run' script comparison page
Guillermo: (/sys/fs/cgroup itself is a tmpfs on my machine) Does your kernel support version 2 cgroupfs?
Re: s6 talk at FOSDEM 2017
Casper Ti. Vector: resource control and [...] cgroup support can be easily implemented with chainloading I pointed at the nosh Guide back in December. The sharp-eyed will notice the advent of a new command in the command list chapter. This is a sneak preview of 1.31.
Re: s6 talk at FOSDEM 2017
Your goo.gl hyperlink on page 34 is to the old WWW site. (-:
nosh version 1.30
The nosh package is now up to version 1.30 . * http://jdebp.eu./Softwares/nosh/ * https://www.freebsd.org/news/status/report-2015-07-2015-09.html#The-nosh-Project * http://jdebp.info./Softwares/nosh/ service bundles --- As usual, there are more service bundles, including for the UWSGI "Emperor" and the new services in FreeBSD/TrueOS 11 such as ypldap. There are now services to run Sendmail in the same manner as the services that run exim. Note that this is slightly different to the old FreeBSD division of labour. There are individually controllable services for SMTP Submission, SMTP Relay, the Submission queue runner, and the Relay queue runner. doco The Guide has been extended with several new chapters, including a gazetteer of interesting directories, a chapter on log file post-processing, a chapter on logging security, a chapter on per-user service management, and some notes for individual services. The commands list has moved from the blurb into the Guide, too, as it seems like something that an administrator might find handy to have available when there's no Internet connection. * http://jdebp.eu./Softwares/nosh/guide.html service management -- There's now a hardlimit chain-loading command, analogous to softlimit. The convert-systemd-services utility now makes use of this and permits setting separate hard and soft limits, or only one or the other, with settings like LimitOFILE=32:128 and LimitNPROC=:infinity . There's now a local-reaper chain-loading command, that can turn "local reaper" status for the current process on or off. Have a care when using this, per the note on the manual page. There is a LocalReaper=true extension to systemd service units for this. netlink-datagram-socket-listen is now available on the BSDs for script compatibility. It always aborts with an address family error. There's a new hangup subcommand of system-control, equivalent to the existing -H option to svc . enhancements to system-control stop/start/reset and single-shot services This is the first big item for 1.30 : The start and stop subcommands of system-control now operate more quickly. Instead of polling once per second, they monitor the supervise/status files of each service that is in the process of being started and stopped, with kevent(). In addition, system-control now supports the notion of services that become ready when their main process has exited, marked with a new flag file in the service directory. convert-systemd-units has been modified to convert "oneshot" services to this, instead of to services that put all of the run code into the start program. Thus "oneshot" services that are running their actual main programs are reported as "running" by svstat, rather than as "starting". This takes advantage of the extended status information that service-manager has been writing to the status file since version 1.28. The sharp-eyed may have noticed that in version 1.28 the output of "svstat"/"system-control status" gained information about the exit statuses of the start, run, restart, and stop programs. This is what system-control now uses to detect whether ready-after-run services ran before they stopped. (Detection of ready-after-run services that are running with no processes, because they are "remain" services, can be and is done with just the daemontools-encore-compatible status information.) Old-style "oneshot"s will continue to work as before, as of course they become ready as soon as the run process is spawned, which is after they have run their programs as part of start. The benefit of this new style, apart from reporting a running service as actually "running", which should help with nagios monitoring and the like, is that "oneshot" services converted from systemd no longer have to be marked as RemainAfterExit=true in order to avoid a dummy "pause" process hanging around. This is the case for old-style "oneshot" services. They have to run something in run, after all, and that something has to keep running in order for the service to be considered ready and services ordered after it to be unblocked. A ready-after-run service, however, unblocks ordered-after services if it has reached the stopped state via a run, thus puts its programs in run, thus doesn't have to have a dummy pause process, and can be RemainAfterExit=false without adding to the process list. log file management --- export-to-rsyslog had a bug that caused it to skip old log files (the @.s ones) in catch-up mode. This has been corrected. There is now a follow-log-directories command that can substitute for tail -F . It knows the actual structure of log directories, operates using one or more cursors like export-to-rsyslog does, and copes correctly with cyclog/multilog log rotation (which GN
Re: nosh: service-control --exit
Guillermo: for some reason 'service-control --exit' doesn't unload a service that is already in the 'stopped' state. I'll look into it for 1.31. You won't see a fix in 1.30 because that is coming out ... well ... now.
Re: NOTE_TRACK, EVFILT_PROC, kqueue, and subreapers
Jilles Tjoelker: This should probably be fixed. There's another more insidious bug hiding inside kevent() somewhere that causes a kernel abend complaining about a sleeping thread holding a non-sleepable lock. One needs to make fairly heavy use of kevent() in order to trigger it, I believe, as I have only seen it so far on systems that do.
Re: subreapers
Martin "eto" Misuth: Subreaper won't even wake when it acquires new child. You're following the other discussion about this, ne?
Re: nosh per-user service management
Jean Louis:
So, placing user daemons into system supervision may not be the best
option, due to so many customization that have to be done for the
user, especially for GNU Emacs -- as one cannot know which programming
languages and their variables are required.
I just explained that these are *not* system-wide services, but per-user
ones. A user who is setting up environment variables for xyr own needs
to run emacs as a service simply sets up environment variables for xyr
own needs against the per-user service. Indeed, I already showed how
that is done. Once again:
Adjust its environment, if desired, in the conventional way
$ system-control --user set-service-env emacs DISPLAY :15.2
or (if /usr/local/sbin is on one's path)
$ rcctl set --user emacs DISPLAY :15.2
The idea that this is somehow difficult because one might have to set an
environment variable named GUILE_LOAD_PATH in this way, is just plain
wrong. This is just an envdir in a conventional place in a service
directory. It's actually easier to manipulate than a $HOME/.{z,}profile
or a $HOME/.login_conf for setting such an environment variable so that
one could spawn the daemon in an interactive login session.
subreapers
Martin "eto" Misuth: Point of tool is simple: it always runs marking itself as "subreaper", thus any descendants who lose parent, that was running under it, will get reparented (and their attached process subtrees as well) under it's process. Once "main", the important and original, child exits, tool tries to term and then kill all remaining descendants. By the way, you should talk to this person: * http://unix.stackexchange.com/questions/329631/
Re: subreapers
What M. Misuth is doing is the most imaginative use of local reapers that I have come across. What I wrote in the nosh doco back in version 1.0 was: > This yields a slightly more informative process tree. This was presented as a mere side-effect by Poettering and Sievers in 2012. The main idea was a rather vague, and in the end not implemented, notion that user instances of systemd could somehow make use of the fact that they were made the parents of orphaned grandchildren processes in order to Do Stuff. In fact, it is the mere side-effect that turns out in practice to be the major benefit. You should not underestimate how useful the effects on the process tree are. All service manager instances in nosh are local reapers, and one sees the effects with the system-wide service manager as well as with per-user service managers. One doesn't see such a pronounced effect with the system-wide service manager because, as I have noted elsewhere in this thread, as a result of the pressure from the daemontools world for the past two decades the world already makes the process tree of system-wide service management fairly well organized. Lots of daemons do not fork-and-exit-parent any more, and orphaned grandchildren simply do not arise as much in this area as they used to at the turn of the century. But the effect for per-user stuff is marked. This is because the world of per-user stuff includes "desktop services", like the roughly ten servers that have to be started up in order to run the "small and lightweight" GNOME Editor. This world is still replete with things that fork-and-exit-parent. To give another example: The PCDM startup of X desktop environments on TrueOS (formerly PC-BSD) starts up a whole bunch of user processes via fork-and-exit-parent. These all end up in a different part of the process tree to the desktop processes sitting under the top-of-desktop-session process, under process #1 or the nearest local reaper. In the output of "ps -dax", all of these processes are scattered all over the shop. One can improve this subtree with broken branches all over the forest floor with a local reaper. The benefit of local reapers is not a programmatic one for the likes of you and me. It is a usability one for administrators and end users trying to follow their process trees.
Re: GNU Emacs now runs in foreground
Martin "eto" Misuth: exec mpd --no-daemon /usr/home/eto/.config/mpd/mpd.conf Jonathan de Boyne Pollard: Tip: In the daemontools world all services have ther own service directory, at minimum; and this directory is the working directory of the service. You can put an mpd.conf that is particular to the service in the service directory itself. Martin "eto" Misuth: I use with my own scripts, but for things having local /etc, /usr/local/etc alternatives I prefer those. That's not /usr/local/etc/ . It's /usr/home/eto/.config/ . $ system-control --user find mpd /usr/home/JdeBP/.config/service-bundles/services/mpd $ system-control --user cat mpd start:#!/bin/nosh start:#Start file generated from ./mpd.socket start:foreground mkdir -- /run/user/JdeBP/mpd ; start:true stop:#!/bin/nosh stop:#Stop file generated from ./mpd.service stop:foreground rm -r -f -- /run/user/JdeBP/mpd/ ; stop:true run:#!/bin/nosh run:#Run file generated from ./mpd.socket run:#Music Player socket used by JdeBP run:local-stream-socket-listen --systemd-compatibility --backlog 5 --pass-credentials /run/user/JdeBP//mpd/socket run:./service service:#!/bin/nosh service:#Service file generated from ./mpd.service service:#Music Player daemon run by JdeBP service:musicpd --no-daemon --stderr --stdout --verbose ./mpd.conf restart:#!/bin/sh restart:#Restart file generated from ./mpd.service restart:exec true # ignore script arguments $
Re: GNU Emacs now runs in foreground
Jonathan de Boyne Pollard: One of the interesting developments over the past couple of decades is how much the world has been influenced to come around to the daemontools way of doing this. I've observed before, elsewhere, the number of daemon programs, especially in the BSD worlds, that have in that time gained -F/--foreground or similar options that tell them not to do a whole bunch of this unnecessary stuff. Martin "eto" Misuth: I think this is bit related to daemon(8) program which acts as poor man's runsv, with all the "advandages" of pid files. It isn't, though. Several of the manual pages that I've seen explicitly mention daemontools. Here's winbindd's manual page, for example: -F If specified, this parameter causes the main winbindd process to not daemonize, i.e. double-fork and disassociate with the terminal. Child processes are still created as normal to service each connection request, but the main process does not exit. This operation mode is suitable for running winbindd under process supervisors such as supervise and svscan from Daniel J. Bernstein's daemontools package, or the AIX process monitor.
NOTE_TRACK, EVFILT_PROC, kqueue, and subreapers
Martin "eto" Misuth: I think that might be the reason why my PID1 s6-svscan on FreeBSD is accumulating zombies sometimes (seems like it is affected by dead descendants of ssh and my experiments). [...] Anyway as you are probably much closer to FreeBSD team than I am, [...] I'm not. You have the same access as I and the rest of the world have. For what it's worth, I've seen similar behaviour with zombies lying around. If we can nail it down you can file a kernel bug report. Have you checked that you aren't getting a NOTE_EXIT?
nosh per-user service management
Jean Louis: If I understand it well, in your system, you define services, and then the service may be marked for start by user? And then it runs on each boot by user? In this system, there is a per-user service manager, that manages services run by the user. All of the processes live outwith any of the user's login sessions. Each user has a place in xyr home directory where xe can define service bundles for services and targets. The per-user service manager works from those service bundles. Per-user service management is itself of course just another *system-level* service. The services are named user-services@jlouis and so forth, and are managed by the system-wide service manager. One starts them, stops them, enables them, and disables them exactly as one would do any other system-level service. So if one wants the per-user service manager for user jlouis auto-started at bootstrap, one enables the user-services@jlouis service. ... or one (more usually) enables the user@jlouis target. This target encompasses the per-user service manager and a per-user D-BUS broker daemon. One doesn't have to enable user@jlouis, and if it isn't enabled it does not auto-start at bootstrap, just like any other. And the configuration import subsystem tries to set up an initial set of per-user service bundles for each "real" user. This setup now includes emacs in that new mode, albeit that I have no way to test it in operation, not having the bleeding edge version of emacs.
client-server desktop application design
Martin "eto" Misuth: It was experiment to see whether it kills all spawned "shell trees" when X goes away. Which it of course does. Maybe Mr Schmorp would be willing to implement feature to freeze shell instances in such cases? Who knows. These so-called client-server designs for these things are fairly pathetic and hardly worthy of the appellation. The clients do little more than marshal up their arguments and environment, send those in a message to the server, and immediately exit. This is not really a client-server architecture. It's the architecture of WIN16, where programs would use DDE or whatever to forward secondary invocations to their hPrevInstance instances and then exit. This is, of course, exactly what the userbase wants. The desire is not to have a client-server terminal emulator, or a client-server text editor, per se. The userbase wants a text editor or a terminal emulator where they can add tabs to an existing instance by just running a simple command. Nonetheless, a design far more worthy of the name "client-server" would split the monolithic application in twain at some point. One obvious point, that relates to what you just said, is to have the display rendering in the client and the "being a terminal" part in the server. The display rendering could still be a multi-tabbed GUI window, with its own API for adding tabs via external program. But the actual terminal emulation part would not be tightly coupled inside that. Designed that way, the idea of keeping the terminal sessions around when the display is not becomes a viable one. Here's an off-the-cuff design using the console-terminal-emulator tool from the nosh toolset. One would still have the main "terminal-server" process. When told to start up an emulation instance, instead of running that internally it spins up an on-the-fly "terminal-emulator" service bundle complete with a directory containing the input FIFO and display file, spins up an on-the-fly "terminal-login" or "terminal-shell" service bundle for the attached pseudo-TTY, and hands them both off to the per-user service manager to run and to manage. (For an idea of what such service bundles would look like, see the "terminal-emulator@vc1" service bundle for a system-wide terminal emulator and the "ttylogin@vc1-tty" service bundle for a login service on that terminal, both supplied with the nosh toolset. One would have the device directory in some per-user place like /run/user/$USER/dev/vcs/ rather than as it is done there, of course.) It can also attach to existing terminal-emulator services that are already running. On its front end, it would connect to one or more X displays and render the terminals as windows and tabs. I'm not proposing to implement this myself, as I have more than enough to do, but I hope that it piques someone's interest. Here are some interesting facets of the design: * console-terminal-emulator implements the terminal characteristics of the terminal emulator programs that are built into the Linux, FreeBSD/TrueOS, and NetBSD (and indeed SCO Xenix!) kernels. It doesn't have to be the only game in town. If one wants something that is closer to some other terminal type, then one just writes a different terminal emulator with the same input FIFO and display file protocol. The "terminal-server"'s knowledge of this is entirely limited to spinning up an on-the-fly service that invokes some other program than console-terminal-emulator. * The relationship between the "terminal-server" and the underlying terminal emulators is many-to-one. There's nothing stopping this design from allowing multiple tabs or multiple windows to render the display of a single shared terminal instance. * Further to the preceding point: Multiple servers, even. I already do this myself. The system-wide terminal emulator services such as terminal-emulator@vc1 can be "realized" on the console, using the framebuffer and HIDs, as well as simultaneously over (say) an SSH session using console-ncurses-realizer. The "terminal-server" in this design is just a fancy, GUI, multi-tabbed, multi-terminal, realizer and more of the same. One can connect, disconnect, and reconnect emulator services from multiple such "realizers" on the fly. I have. * Further to the preceding point: With suitable permissions, a "terminal-server" could attach itself to one of the system-wide terminal emulators. So one could do things such as: Start up without X; log on on the system virtual terminals; do stuff; shut down console-fb-realizer and start up the X server; reconnect to the very same terminals with a GUI realizer program; continue doing stuff. * The terminal sessions know nothing about the X display(s) and are unaffected by their absence. Of course the converse of this is that the terminal sessions know nothing about the X displays and in order to run GUI programs one has to explicitly set a
services that need DISPLAY
Daniel Kahn Gillmor: Yet surely there are some user-wide services that don't need DISPLAY at all, and would be happy to run per-user? GNOME Terminal isn't one of them. Witness the behaviour of gnome-terminal-server run as a service if it doesn't have a DISPLAY environment variable: @4000584830f10ae09b9b Unable to init server: Could not connect: Connection refused @4000584830f10ae0dbb8 Failed to parse arguments: Cannot open display:
passing the listening socket as an open file descriptor
Martin "eto" Misuth: I personally am not so "hot" about this listening socket passing stuffs. When you've had to deal with tens if not hundreds of different ways of saying "listen on this IP address and port", you'll come around to the idea of having a single tool that does this one job universally. (-:
subreapers
Martin "eto" Misuth:
I think Mr Jonathan de Boyne Pollard might be cooking, or even already
has, something similar in nosh.
Long since. (-: It was in version 1.0 .
Martin "eto" Misuth:
at some point I was interested in digging out whether systemd had
"subreapers" at it's disposal, and why it didn't use them
It has. The systemd people were responsible for getting this mechanism
into Linux in the first place.
* http://unix.stackexchange.com/a/177361/5132
Martin "eto" Misuth:
- observing behaviour of reparenting under init for thousands of times
in htop
Now observe it under not init and not htop. (-:
Here's the nosh service manager's subreaper mechanism in action. Process
31427 is a per-user service manager, which is a subreaper. Process 9408
is GNOME Terminal and process 9412 is a Z Shell running connected to
that terminal. In the shell, I ran (sleep 6000&) which resulted in an
orphaned sleep process, process 9451. As you can see, that process has
been reparented to the nearest subreaper.
per-user-manage(31423)-+-cyclog(31426)
`-service-manager(31427)-+-3(6088)
|-4(31592)-+-{decoder}(31598)
| |-{io}(31596)
| |-{output:default
}(31599)
| `-{player}(31597)
|-at-spi-bus-laun(9276)-+-dbus-daemon(9290)
|
|-{dconf worker}(9277)
|
|-{gdbus}(9279)
|
`-{gmain}(9291)
|-at-spi2-registr(9293)---{gdbus}(9296)
|-cyclog(6070)
|-cyclog(18028)
|-cyclog(20138)
|-cyclog(30089)
|-emacs(6094)---{gmain}(6095)
|-gnome-terminal-(9408)-+-gnome-pty-helpe(9411)
|
|-zsh(9412)---pstree(9482)
|
|-{dconf worker}(9409)
|
|-{gdbus}(9410)
|
`-{gmain}(9413)
|-gvfsd(9305)---{gdbus}(9306)
|-pulseaudio(3625)---{null-sink}(3626)
`-sleep(9451)
Re: GNU Emacs now runs in foreground
Martin "eto" Misuth: On my presonal box "user level" s6 /services subtrees are in `.config/s6/host` For comparison: ~/.config/service-bundles/services/ and ~/.config/service-bundles/targets/ Martin "eto" Misuth: #!/bin/sh exec mpd --no-daemon /usr/home/eto/.config/mpd/mpd.conf Tip: In the daemontools world all services have ther own service directory, at minimum; and this directory is the working directory of the service. You can put an mpd.conf that is particular to the service in the service directory itself. Martin "eto" Misuth: Getting tmux "supervised" was bit tricky as it double forks the daemon if not running, and is thus is "escaping under init" under normal conditions. Note: This can be easily handled on FreeBSD and TrueOS. See NOTE_TRACK in EVFILT_PROC for kevent(). Martin "eto" Misuth: It would be great if tmux project allowed to spawn raw user level daemon by explicit command, One of the interesting developments over the past couple of decades is how much the world has been influenced to come around to the daemontools way of doing this. I've observed before, elsewhere, the number of daemon programs, especially in the BSD worlds, that have in that time gained -F/--foreground or similar options that tell them not to do a whole bunch of this unnecessary stuff. It's in a way somewhat saddening to see the new option to emacs that started this thread. It's welcome; but the sad part is that it's a decade and a half behind quite a lot of others. For some systems like emacs the process has been glacially slow. MySQL is another. daemontools users were asking about letting daemontools do the service management and doing away with mysqld_safe in 2002. It took thirteen years for it to happen ... in MariaDB. Oracle, where software goes to die, may or may not have done the same for MySQL. PostgreSQL lags further behind. * http://jdebp.eu./Softwares/nosh/mariadb-and-mysql.html#Prompt
Re: Adding capability control into the `run' script comparison page
Guillermo: I suppose the interesting suprise is that as consequence, when a service definition gets 'imported' to nosh from a unit file (and this covers pretty much everything in the nosh-bundles* binary packages),the corresponding service gets placed in a cgroup of its own when launched by nosh's service manager: This also happens in per-user service management. /service-manager.slice/user-services@.service: └─user-services@jdebp.service ├─31423 per-user-manager ├─per-user-manager-log.slice │ └─31426 cyclog --max-file-size 32768 --max-total-size 1048576 . └─service-manager.slice ├─31427 service-manager ├─gvfs-daemon.service │ └─9305 /usr/lib/gvfs/gvfsd ├─at-spi-dbus-bus.service │ ├─9276 /usr/lib/at-spi2-core/at-spi-bus-launcher │ ├─9290 /usr/bin/dbus-daemon --config-file=/etc/at-spi2/accessibility.co... │ └─9293 /usr/lib/at-spi2-core/at-spi2-registryd --use-gnome-session ├─dbus-servers-log.service │ └─30089 cyclog jdebp/dbus-servers/ ├─pulseaudio.service │ └─3625 pulseaudio --exit-idle-time=-1 ├─mpd.service │ └─31592 mpd --no-daemon --stderr --stdout --verbose ./mpd.conf ├─emacs.service │ ├─6088 strace -f emacs --daemon │ └─6094 emacs --daemon ├─cyclog@.service │ ├─cyclog@pulseaudio.service │ │ └─20138 cyclog jdebp/pulseaudio/ │ └─cyclog@mpd.service │ └─18028 cyclog jdebp/mpd/ ├─simple-servers-log.service │ └─6070 cyclog jdebp/simple-servers/ └─gnome-terminal-server.service ├─9408 /usr/lib/gnome-terminal/gnome-terminal-server ├─9411 gnome-pty-helper ├─9412 zsh └─9451 sleep 6000
Re: GNU Emacs now runs in foreground
Daniel Kahn Gillmor: dbus-user-session supports at most one graphical session concurrently ... in order to avoid people encountering the very problem of half-hearted and flawed implementations that I described. Non-half-hearted implementations are the goal, however. Read https://lists.debian.org/debian-devel/2016/08/msg00554.html and http://www.mail-archive.com/supervision@list.skarnet.org/msg01393.html .
Re: Adding capability control into the `run' script comparison page
Casper Ti. Vector: But I do think the capability argument has its validity: chainloading is, at this time, not well known to normal users, which is why many systemd supporters compulsorily identify cgroup support with systemd with few people opposing. Therefore I suggest to add some examples of capacility control (eg. one example for ulimit, plus one example for cgroup) into the comparison page, or an independent page. Such "systemd supporters" don't actually know systemd. * http://jdebp.eu./FGA/linux-control-groups-are-not-jobs.html To anyone running the service manager and bundles from nosh version 1.28 or later on Linux: You are encouraged to look at your control group hierarchy, with a tool like "systemd-cgls /", with the "cgroup" field of the ps command, or by simply listing your /sys/fs/cgroup/ hierarchy. You are in for an interesting surprise.
GNU Emacs now runs in foreground
Martin "eto" Misuth: - rxvt-unicode - uberterminal - this thing can operate [...] daemon, when single process hosts all your terminals - benefits are [...] Drawbacks are that it doesn't understand receiving the listening socket as an already-open file descriptor, and by default it places the socket in $HOME/.urxvt/ rather than in /run/user/$USER/urxvt/ . Martin "eto" Misuth: - mpd - the music player daemon This can accept listening sockets passed to it. I've added to the nosh bundles package a service bundle for it as a system-wide instance, and generators in the configuration import subsystem for per-user instances. You'll see these in version 1.30.
Re: djbwares version 4
Jonathan de Boyne Pollard: In celebration of the forthcoming leap second, djbwares is now at version 4. * http://jdebp.eu./Softwares/djbwares/ * http://jdebp.info./Softwares/djbwares/ Jean Louis: http://jdebp.info./Softwares/djbwares is not working: "access denied" and I instinctively tried that one first, as to avoid .eu (even it makes no sense). You should have just tried the URL that I gave to you, without your changing it to something different. Ironically, Bernstein publicfile is part of the package at hand, and this is the documented behaviour of publicfile, in its original Bernstein manual: > A request for http://v/f refers to the file named ./v/f inside the root directory hierarchy, if f does not end with a slash. > httpd will refuse to read a file if the file [...] is anything other than a regular file: a directory, socket, device, etc. publicfile isn't going to let you read the WWW server's directories directly with URL tricks. You attempt that in vain. (-: For *not* trying to trick the WWW server, and simply reading the blurb and the download instructions, just use the actual URL that I gave.
GNU Emacs now runs in foreground
Martin "eto" Misuth: First, there are two major caveats, There are actually three. They break scripting. For example: People cannot use the GNOME Editor as $VISUAL or $EDITOR because one of the things implicit in the $EDITOR/$VISUAL mechanism is that when the program that has been invoked exits, the editing is over and the file being edited has been saved in the desired form. That is not the behaviour of the "small and lightweight" GNOME Editor, however. * http://unix.stackexchange.com/questions/201900/ * http://unix.stackexchange.com/a/323700/5132 * https://news.ycombinator.com/item?id=13056252 Other "interesting" problems result from the move that the Desktop Bus and the Desktop Environment people are making away from per-session instances of the D-BUS daemon to per-user instances of the same. This causes fun with deciding what the daemon's $DISPLAY should be set to. A per-session Desktop Bus obviously has one $DISPLAY by and large. But a per-user Desktop Bus not only has to handle multiple logins from a single user, it has to handle that the per-user session management can be running when there's no X server at all. (systemd starts its per-user instances via PAM hooks that act upon every login, including logins over SSH and on terminals.) Even though some daemons try to take the approach that the daemon supports multiple $DISPLAYs, sent in from multiple clients as part of the client session, one unfortunately finds that the daemons themselves still have to have an arbitrary $DISPLAY in order to start up in their initial, not connected to any clients and their displays yet, mode. In practice, thus, the implementation of the user-wide client-server idea is half-hearted and flawed in this respect.
Re: Adding capability control into the `run' script comparison page
Casper Ti. Vector: the docs are in tarballs on jdebp.eu * http://jdebp.eu./Softwares/nosh/guide.html
djbwares version 4
In celebration of the forthcoming leap second, djbwares is now at version 4. * http://jdebp.eu./Softwares/djbwares/ * http://jdebp.info./Softwares/djbwares/ I've added in the rest of M. Bernstein's public domain libtai library, parts of which were already included by some of the tools. This has added the easter, nowutc, and yearcal commands, which are packaged up alongside libtai.a, the libtai C language headers, and the libtai manual pages in a new libtai package. More importantly, it has added the leapsecs command, and the /usr/local/etc/leapsecs.dat file is now generated from leapsecs.txt rather than included as a binary in the source as it was before. The sharp-eyed will also note that support for /usr/local/etc/leapsecs.dat (as an alternative to /etc/leapsecs.dat for systems that like non-operating system files in /usr/local/etc) has also been added. The leapsecs.txt is the Bernstein 2015-06-30 version (which is still the latest published by M. Bernstein) patched with the forthcoming leap second. The libtai package does not include /usr/local/etc/leapsecs.dat . Rather, that is packaged in a separate leapsecs package, to allow updated versions to be substituted with ease when they come along, as well as to permit installing only that without the rest of libtai.
Re: Adding capability control into the `run' script comparison page
Casper Ti. Vector: one example for ulimit An irony here is that the page *already contains* two entire sets of examples that set memory resource limits, using daemontools, daemontools-encore, freedt, perp, s6, and nosh tools.
Re: nosh version 1.29
Bloody Thunderbird! Here's that again, I hope without the surprise reformatting after pressing "send" this time: The nosh package is now up to version 1.29. * http://jdebp.eu./Softwares/nosh/ * https://www.freebsd.org/news/status/report-2015-07-2015-09.html#The-nosh-Project * http://jdebp.info./Softwares/nosh/ There's been a lot going on since version 1.28 . 2016 leap second The TAI to UTC conversions know about the forthcoming leap second. service bundles --- As usual, there are several new service bundles, from powerd++ through zfsd to fwknopd. The new fs-servers target allows one to order the initialization of NFS servers before loop-to-self NFS mounts. The new multi-user-pre target is another ordering target that allows services such as the motd file updater to be ordered before TTY login services. The instantiated kdm@tty7 and kdm@ttyv6 services have been replaced with a single kdm service, with a view to dealing with display managers better in the future. I have some plans in this area. The Samba service names have been fixed. Debian calls them nmb, smb, and winbind; but the Samba doco and most places on the WWW call them nmbd, smbd, and winbindd. The latter names are used in the service bundles package, with aliases pointing to them from the Debian names. doco The doco has been improved and kept up-to-date in various places, including correct descriptions of set-service-env and print-service-env after one confused user got in touch. PC-BSD is now named as TrueOS where the reference is not historical. code review --- As a result of some code review that was offered, std::auto_ptr is now gone and a rare memory corruption bug in safe_execvp() has been fixed. Building from scratch when one doesn't have a prior daemontools or freedt toolset installed also no longer hits a bug. configuration import improvements - In an effort to clear those last few remaining items on the nosh roadmap, a whole load of configuration import (pppd, sppp, rfcomm_ppp, dhclient, wpa_supplicant, natd, and hostapd) has been consolidated under the umbrella of static-networking. I plan to expand this further in 1.31, given how much is already in 1.30. Linux kernel VTs Management of Linux kernel virtual terminals has some improvements, including setting UTF-8 canonical mode editing and keyboard composition modes, and emitting the control sequences that set up the screen saver. tai64nlocal changes --- tai64nlocal has adopted a minor but important change from the BSD and GNU C libraries: before reading the start of a line it flushes its output. This came from trying to use it as a co-process in GNU awk. To prevent deadlocks, GNU awk co-processes need to be in what is effectively line buffered output mode even though their standard inputs and outputs are not terminal devices. This is now the case for tai64nlocal and it can be used to convert TAI64N timestamps as a GNU awk co-process. FreeBSD and TrueOS packaging The largest change, however, is in the FreeBSD/TrueOS and OpenBSD packaging. This is a change that is going to happen in the Debian packaging in a later version. It's partly to simplify the package maintenance, and partly a step towards having OpenBSD packages that work. A single package description is fed to both the new pkg tool that exists on FreeBSD/TrueOS and the old pkg tool that exists on OpenBSD. It's not perfect, as there are things that are easy with the new pkg tool that are hard with the old one; and the OpenBSD packages are still not fully functional. But things are better than they were. The OpenBSD service bundles package now almost properly sets up per-service user accounts and log directories, for example. === === IMPORTANT UPGRADE NOTE FOR FreeBSD/TrueOS: === === An important consequence of the aforementioned is that the semantics of the nosh-bundles package have changed. In earlier versions, the various nosh-run-* packages were how one set services running, except for a small rump set of services that were set up by the nosh-bundles package. This is now no longer the case. The nosh-bundles package now presets and starts no services at all. *All* running of services must be achieved with the nosh-run-* packages or some other sets of scripts and presets. To this end, there are now two new packages, nosh-run-freebsd-desktop-base and nosh-run-freebsd-server-base. These parallel the already existing nosh-run-trueos-desktop-base and nosh-run-trueos-server-base packages; except that they do not start any of the services that exist in TrueOS but do not exist in FreeBSD, such as the various pc-* services. You must install,
Re: GNU Emacs now runs in foreground
Jean Louis: emacs --new-daemon=NAME I have added a new per-user service for this to nosh, ready for version 1.30 . So one just has to start the per-user service manager # system-control start user@jlouis.target then start the emacs server $ system-control --user start emacs Adjust its environment, if desired, in the conventional way $ system-control --user set-service-env emacs DISPLAY :15.2 or (if /usr/local/sbin is on one's path) $ rcctl set --user emacs DISPLAY :15.2
nosh version 1.29
The nosh package is now up to version 1.29. * http://jdebp.eu./Softwares/nosh/ * https://www.freebsd.org/news/status/report-2015-07-2015-09.html#The-nosh-Project * http://jdebp.info./Softwares/nosh/ There's been a lot going on since version 1.28 . 2016 leap second The TAI to UTC conversions know about the forthcoming leap second. service bundles --- As usual, there are several new service bundles, from powerd++ through zfsd to fwknopd. The new fs-servers target allows one to order the initialization of NFS servers before loop-to-self NFS mounts. The new multi-user-pre target is another ordering target that allows services such as the motd file updater to be ordered before TTY login services. The instantiated kdm@tty7 and kdm@ttyv6 services have been replaced with a single kdm service, with a view to dealing with display managers better in the future. I have some plans in this area. The Samba service names have been fixed. Debian calls them nmb, smb, and winbind; but the Samba doco and most places on the WWW call them nmbd, smbd, and winbindd. The latter names are used in the service bundles package, with aliases pointing to them from the Debian names. doco The doco has been improved and kept up-to-date in various places, including correct descriptions of set-service-env and print-service-env after one confused user got in touch. PC-BSD is now named as TrueOS where the reference is not historical. code review --- As a result of some code review that was offered, std::auto_ptr is now gone and a rare memory corruption bug in safe_execvp() has been fixed. Building from scratch when one doesn't have a prior daemontools or freedt toolset installed also no longer hits a bug. configuration import improvements - In an effort to clear those last few remaining items on the nosh roadmap, a whole load of configuration import (pppd, sppp, rfcomm_ppp, dhclient, wpa_supplicant, natd, and hostapd) has been consolidated under the umbrella of static-networking. I plan to expand this further in 1.31, given how much is already in 1.30. Linux kernel VTs Management of Linux kernel virtual terminals has some improvements, including setting UTF-8 canonical mode editing and keyboard composition modes, and emitting the control sequences that set up the screen saver. tai64nlocal changes --- tai64nlocal has adopted a minor but important change from the BSD and GNU C libraries: before reading the start of a line it flushes its output. This came from trying to use it as a co-process in GNU awk. To prevent deadlocks, GNU awk co-processes need to be in what is effectively line buffered output mode even though their standard inputs and outputs are not terminal devices. This is now the case for tai64nlocal and it can be used to convert TAI64N timestamps as a GNU awk co-process. FreeBSD and TrueOS packaging The largest change, however, is in the FreeBSD/TrueOS and OpenBSD packaging. This is a change that is going to happen in the Debian packaging in a later version. It's partly to simplify the package maintenance, and partly a step towards having OpenBSD packages that work. A single package description is fed to both the new pkg tool that exists on FreeBSD/TrueOS and the old pkg tool that exists on OpenBSD. It's not perfect, as there are things that are easy with the new pkg tool that are hard with the old one; and the OpenBSD packages are still not fully functional. But things are better than they were. The OpenBSD service bundles package now almost properly sets up per-service user accounts and log directories, for example. === === IMPORTANT UPGRADE NOTE FOR FreeBSD/TrueOS: === === An important consequence of the aforementioned is that the semantics of the nosh-bundles package have changed. In earlier versions, the various nosh-run-* packages were how one set services running, except for a small rump set of services that were set up by the nosh-bundles package. This is now no longer the case. The nosh-bundles package now presets and starts no services at all. *All* running of services must be achieved with the nosh-run-* packages or some other sets of scripts and presets. To this end, there are now two new packages, nosh-run-freebsd-desktop-base and nosh-run-freebsd-server-base. These parallel the already existing nosh-run-trueos-desktop-base and nosh-run-trueos-server-base packages; except that they do not start any of the services that exist in TrueOS but do not exist in FreeBSD, such as the various pc-* services. You must install, for a working fully-nosh-managed system, exactly one of these four packages. If you are running nosh service management under Mewburn rc, you can of cou
perp WWW site problems
The perp WWW site seems to have endured some damage. For example: * http://b0llix.net/perp/site.cgi?page=tinylog.8 is truncated partway through. * http://b0llix.net/perp/site.cgi?page=perpd.8 is similarly truncated. * http://b0llix.net/perp/site.cgi?page=links has some malformed markup.
Re: Problem with s6-softlimit -c
Jan Olszak:
Command that fails:
s6-softlimit -c 1 pwd
# strace s6-softlimit -c 204800 pwd
...
prlimit64(0, RLIMIT_CORE, NULL, {rlim_cur=RLIM64_INFINITY,
rlim_max=RLIM64_INFINITY}) = 0
prlimit64(0, RLIMIT_CORE, {rlim_cur=200*1024, rlim_max=RLIM64_INFINITY}, NULL)
= 0
You seem to have mis-spelled "works". (-: The program has made the
system calls that you told it to, and they have succeeded, setting the
soft limit that you instructed.
Re: ruinit-rpm systemd service file
Otheus: From systemd's viewpoint, the *service *is runsvdir, and not runsvdir-start. The latter is simply a wrapper script for the former and needed because of initttab's limitations. With systemd, a service file can contain all the information in that script and more. Similarly: http://jdebp.eu./FGA/inittab-is-history.html#svscanboot
Re: Runit questions
Colin Booth: --system loads the system defaults. Actually, it tells it that it is a system-wide broker rather than a per-session or per-user broker.
Re: s6, listen(8), etc.
Laurent Bercot: fds 6 and 7 are only used for UCSPI clients, which are a very minor subset of the programs you'd want to use that mechanism with. Laurent Bercot: I don't want the caller to tell me "here's a bunch of fds, you sort them out": that's just laziness. Just so that everyone is not operating under any more misapprehensions: One dictum is that if you're using the "LISTEN" protocol for any UCSPI use-case, you are doing it wrong. The name "LISTEN" in "LISTEN_FDS" is a big clue. It's not for accepted socket file descriptors. It's for listening ones. UCSPI deals in accepted connections, conversely. The systemd people reinvented their own, pretty poor, protocol for accepted connections. It is not the "LISTEN" protocol. In 2015 I tried to point them at UCSPI where there's years to decades of existing practice, doco, and implementation (including GNU inetd!) to gain from. See https://lists.freedesktop.org/archives/systemd-devel/2015-June/033299.html (The UCSPI FGA is now http://jdebp.eu./FGA/UCSPI.html , by the way.) If you think that "here's a bunch of file descriptors" is the protocol, then you've missed a subtlety. The protocol is that the list of file descriptors is *ordered*, by the system administrator/package writer. The systemd manual pages explain that it's the order that the various ListenXXX directives occur in the .INI file. (systemd treats .INI files as ordered in various ways, of which this is one.) In the nosh toolset, it's simply the order in which you chain together the chain-loading *-listen tools. Each tool appends a further listening file descriptor to the end of the list that it begins execution with. So whilst in the wild generally programs just scan the whole list looking for the first/last file descriptor whose type (FIFO, socket address family, and so on) they like, on the basis that they are generally only looking for one file descriptor of any type, this is an interpretation that's defined by the particular server programs concerned. They could equally well define something like "You specify my control listening FIFO first, my TCP4 service listening socket second, and my TCP6 service listening socket third in the list.". How the ordered list of file descriptors is treated is defined by the service programs themselves.
Re: s6, listen(8), etc.
Laurent Bercot: how does the daemon know what fd corresponds to what use? In the wild, it's generally a for() loop over the passed-in descriptors that checks each socket type. In the wild, only one of any type is often the case. "If AF_INET4 and SOCK_DGRAM, this must be my UDP4 socket." That's how I patched dnscache and tinydns to work, too. A simple set of socket_is() functions in the Bernstein socket library, and it is off and away.
Re: s6, listen(8), etc.
Daniel Kahn Gillmor: #!/bin/sh mkdir -p /run/kresd/workdir && \ setfacl -m u:kresd:rwx /run/kresd/workdir && \ cd /run/kresd/workdir && \ exec listen -udp::53 \ -tcp::53 \ -tcp:label=tls:853 \ -unix:label=control,mode=0600:/run/kresd/control \ chpst -u kresd -p 1 \ /usr/sbin/kresd start: #!/bin/sh -e install -d -m 0755 -o kresd /run/kresd/workdir stop: #!/bin/sh -e rm -r /run/kresd/ run: #!/bin/nosh udp-socket-listen --systemd-compatibility --combine4and6 :: domain tcp-socket-listen --systemd-compatibility --combine4and6 --backlog 2 :: domain local-datagram-socket-listen --systemd-compatibility --mode 0666 /run/kresd/query.socket local-stream-socket-listen --systemd-compatibility --mode 0600 /run/kresd/control ./service service: #!/bin/nosh chdir /run/kresd/workdir softlimit -p 1 setuidgid kresd kresd
Re: s6, listen(8), etc.
Daniel Kahn Gillmor wrote: So i'm hoping that it'll find a taker in one of these more toolkit-style supervisor suites. http://jdebp.eu./Softwares/nosh/#Features socket services section
Re: nosh: G++ warnings
Just so everyone knows. This didn't get ignored. Guillermo has been quietly playing with 1.29 behind the scenes. This was a mixture of things. 1.29 will have (already has) some fixes for the genuine bugs and for some of the things which are the subject of much debate. (-: The fprintf()s I simply haven't touched yet; not because LP64/LLP64 correctness isn't a good thing, but because some of them say "DEBUG:" and I really should first review whether they should remain in there at all. (-:
Mass bug filing: use and misuse of dbus-launch (dbus-x11)
Simon McVittie: This can already work. If you put XDG_RUNTIME_DIR in user programs' environment, and arrange for your favourite service manager to make a dbus-daemon (or something else that speaks the same protocol) listen on $XDG_RUNTIME_DIR/bus before any user process would try to connect to it, then modern versions of at least libdbus, GDBus and sd-bus will connect to it by default with no additional effort on your part. This client-side code path does not depend on systemd, does not depend on libsystemd (except obviously sd-bus which is part of libsystemd), and is compiled in for all supported Unix platforms. That's the problem. No the whole unix:runtime=yes mechanism is not. As I said, this is something that you and Joe Marcus Clarke and whomever else have to sort out with each other. I'm unfortunately stuck in the middle, here. Please do whatever it is that you need to do with each other to make your program understand address=systemd: and address=unix:runtime=yes on FreeBSD/TrueOS/OpenBSD. It does not do so. Simon McVittie: Meanwhile, if you want the relevant integration files (your favourite service manager's equivalent of systemd units) to be part of dbus (the reference implementation of D-Bus), please propose tested patches; if they follow the "user session" model[1], they could eventually go in dbus-user-session.deb, with its dependencies changed from the current systemd-sysv to "systemd-sysv | your-service-manager". Kudos for being the first project to offer integration, ever. (-: Yes, down the road it would be marvellous if people included service bundles in their own projects. Yes, I'd like to see the day when the number of service bundles in the nosh-bundles package actually starts going down, because people are taking on shipping their own service bundles for their own services, instead of going up. So yes, eventually you'll be taken up on that offer I hope. But one step at a time. Simon McVittie: As for what I would like, I'd like you (where that's plural, including Joe Marcus Clarke or whomever else) to please make either address=systemd: or address=unix:runtime=yes work in your program on FreeBSD/PC-BSD/OpenBSD. To the best of my knowledge, the listenable address "unix:runtime=yes" (as in "dbus-daemon --address=unix:runtime=yes") does work on generic Unix, and should interoperate fine with the XDG_RUNTIME_DIR/bus fallback used by clients with no DBUS_SESSION_BUS_ADDRESS. It is compiled and tested whenever DBUS_UNIX is defined (i.e. everything except Windows), and I haven't seen bug reports about that test failing. There you go, then. New knowledge. (-: It doesn't work with your program as ported to FreeBSD/TrueOS/OpenBSD. Joe Marcus Clarke is the porter for FreeBSD, according to the port information, and Antoine Jacoutot the porter for OpenBSD. This is why I am saying that it's something that you (plural, remember) need to sort out amongst yourselves. We users stuck in the middle cannot use address=systemd: and address=unix:runtime=yes with your program on these systems. As I said, it's something that I had to fix in November 2015, to stop trying to use address=systemd: on FreeBSD/TrueOS because it turned out that it didn't actually work. I thought that address=unix:runtime=yes might, but that did not either. Simon McVittie: I am *not* going to go looking for patches on display at the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying "beware of the leopard"; Luckily, you didn't need to. The page that I hyperlinked before pointed directly to Pierre-Yves Ritschard's and Cameron T. Norman's actual code, even down to positioning the window around the first lines of the functions. Now if one *did* want to follow the Debian way of having mention of these things buried down in the depths, in a bug report from years ago, then this is a truly excellent example of the genre that one could enjoy. One should begin with Cameron T. Norman's post here, roughly one thousand eight hundred messages down, in a bug report from 3 years ago: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=727708#1859 (-: Simon McVittie: To be brutally honest, there is a fairly low limit to how much benefit I can create by giving new things to PC-BSD users, [...] That's not the right way to look at it. You yourself have just said several times that this is stuff that is supposed to be on "supported Unix platforms". This isn't giving new things to anyone. This is making existing things, that you yourself think are existing, work. I shouldn't dismiss PC-BSD so readily, if I were you, either. PC-BSD (now rebranded as TrueOS Desktop a few days ago -- I just got through changing a whole load of preset file and -run package names.) is the BSD that comes in the box with the desktop environments and with all of the desktop programs that use Desktop Bus. Yes, people can and do run
Is missing SysV-init support a bug?
Gerrit Pape: To me too this readiness IPC ideas and implementations look over-engineered. A good convention for service programs would be to functionally test for services it needs very early on startup, and fail if dependencies are not available. The service supervisor (any modern init scheme seems to now support this) relaunches eventually, until all dependencies are fulfilled. The problem with the thundering herd approach is twofold. Firstly, it really does matter in practice when the machine has tens if not hundreds of client processes all continually restarting whilst they wait for (say) the RabbitMQ server to come up. Secondly, these explanations never seem to take system shutdown into account. In the ordered services world, shutdown order is the reverse of startup order, and things generally work. In the thundering herd world, often the theory is just to send terminate and kill signals willy-nilly to every service on the system. This almost never works cleanly in any but the most trivial systems. (People will no doubt be thinking the classic example of NFS mounts, here. But there are all sorts of possibilities, from /var/ being unmounted before logging services are turned off to the proxy DNS server being turned off whilst other services are still doing DNS lookups.) We discussed this on the Supervision mailing list last year: http://www.mail-archive.com/supervision%40list.skarnet.org/msg00673.html
Re: Linuxisms in s6
Adrian Chadd:
[...] the uptime stuff really threw us.
It's unfair to lay such system time problems at s6's door. Systems whose
system clock jumps 46 years during system bootstrap don't get to blame
s6 for mad time gaps that appear in logs and service start time
records. There is a *lot* of the Unix and Linux worlds that depends
from time being right. It's not just s6 that is affected by such
things. You note crypto. There are a lot of other things as well that
have unstated, sometimes undocumented, and sometimes surprising
dependencies upon system time being current.
Here's one such.
For quite a while, Linux distributions had rather an odd problem at
bootstrap. They'd repeatedly fsck volumes at every bootstrap when they
need not have. And this didn't affect U.S. or U.K. people, which is in
part why it persisted for so long.
* https://bugs.launchpad.net/ubuntu/+source/e2fsprogs/+bug/63175
* https://bugs.archlinux.org/task/17438
* http://lwn.net/Articles/264498/
The problem was that people were erroneously running their real-time
clocks in local time rather than UTC, and this triggered an odd hidden
dependency upon having the right time in the system clock in countries
where local time was in advance of UTC. The Linux method for handling
RTCs erroneously running in local time is for the system bootstrap to
make a special settimeofday() call that effectively tells the kernel
what the UTC offset is for the RTC hardware. This could happen *after*
the fsck of the root volume, however. So whilst that fsck was
happening, the kernel was assuming that UTC was the local time that it
had taken from the RTC and initialized its system clock with. In
effect, as soon as the special settimeofday() call was executed, the
system clock would jump backwards by one or more hours, to what UTC
actually was.
But the ext2/3/4 filesystem format has last checked/mounted/written
timestamps in its superblock. Part of the checking to see whether a
full fsck is needed at bootstrap is comparing them to the current time.
If they are in the future by hours or more, something is clearly wrong,
thinks fsck, and it runs the full check. At bootstrap, when the initial
fsck (of at least the root volume and sometimes other volumes as well)
is run, the system clock is not UTC yet. Comedy results.
Both systemd and the nosh system manager have to ensure that they do the
special settimeofday() system call before they start off service
management and thus run mount/fsck services, or indeed anything else
that might have a closet dependency from not stepping the system time by
hours partway through bootstrap. The nosh system-manager's manual page
has a whole section on this subject.
FreeBSD/PC-BSD has a mechanism for correctly reading a RTC that is
erroneously in local time. One sets up the RTC's offset from UTC in the
machdep.adjkerntz variable in /boot/loader.conf{,.local} and the system
clock never has to jump by hours during bootstrap. I've yet to
experience a FreeBSD/PC-BSD system where the installer actually
configures this, though.
Interestingly, FreeBSD/PC-BSD also has a fallback mechanism that uses
the latest volume mount timestamp that it can find as the initial system
time when no hardware clock device registers at bootstrap. Presumably
you have a clock device that registers but it is not battery-backed,
your volumes don't preserve (or reset) their mount timestamps, or you
are encountering the comedy situation where FreeBSD/PC-BSD is setting
the system clock to 1970-01-01 because the last time around it mounted
the filesystems before the clock was corrected.
Re: Linuxisms in s6
Adrian Chadd: Sure, but I'm looking for something more generic than just devd. Like, notifications about events like "default route is up" can be done by sniffing the rtsock, but notifications like "ntpdate has updated the date, we can now do crypto services" doesn't happen there right now. You're reinventing upstart. The lesson of upstart is that whilst the event-driven paradigm looks like the bright shiny future, once one gets down to the details it is a lot harder than it at first appears. I strongly recommended learning about upstart, and especially learning the problems that people hit with it, to anyone going down the same route. The Debian systemd Hoo-Hah had some lengthy discussion of upstart. (I regret not having bookmarked the discussion that I once came across, where someone opined that xe preferred systemd to upstart because at a Linux conference the systemd presentation had been exciting and had been put forward as the wave of the future, where upstart had been presented as old-school, traditional, and boring. Ironically, this person wasn't aware that the designs are exactly the opposite of that. upstart has the novel event-driven design where the system is configured with the information that event A triggers programs P, Q, and R, and the system starts by raising a "first event", that runs programs, that raise further events, that run further programs. Whereas it is systemd that has the conventional design, shared by Mewburn rc and others, of starting from a goal, working through a dependency tree, and doing topological sorts.) The Debian people chose to improve a non-event-driven architecture instead. It's a lesson to be learned from SMF, in fact. One can have a lot more additional abstract targets, such as "/milestone/name-services" and "/milestone/system-clock", and dependencies to and from them. The world is not 2 to 4 run levels plus "DAEMON", "NETWORKING", and "$local-fs". That said, something like this hypothetical "/milestone/system-clock" is a milestone that would need to be reached *very* early on in the bootstrap process. Fixing up the clock is something that both the nosh system manager and systemd handle themselves directly, outwith of service management. More on this in a moment.
Re: Linuxisms in s6
Jonathan de Boyne Pollard: What are these Linuxisms in s6? s6-linux-utils and s6-linux-init have Linuxisms, obviously. But what Linuxisms does s6 have? Adrian Chadd: We just had a bunch of fun trying to get it to build right, [...] Such as what, specifically?
Linuxisms in s6
http://adrianchadd.blogspot.co.uk/2016/08/freebsd-on-tiny-system-whats-missing.html?showComment=1471236502051#c1305086913155850955 , Adrian Chadd: We're using s6 at work, and it works out mostly ok. Mostly once you get around the linuxisms, and the lack of sensible time code in it (its calculations for daemon run duration is based on system time, not wall clock, so if your box boots jan 1, 1970 then gets NTP, things are.. hilarious), and some of the arcane bits to get logging working right. What are these Linuxisms in s6? s6-linux-utils and s6-linux-init have Linuxisms, obviously. But what Linuxisms does s6 have?
Re: s6-linux-init: SIGUSR1 and SIGUSR2
Guillermo: > a) SIGTERM for reboot > b) SIGUSR1 for halt > c) SIGUSR2 for poweroff > d) SIGINT for a programmable CTRL + ALT + Del action > e) SIGWINCH for a programmable 'keyboard request' action > > nosh's system-manager supports e) via the 'kbrequest' target. But for > compatibility reasons, supports d) on [GNU/]Linux via the > 'secure-attention-key' target, but not on the BSDs, where SIGINT means > 'halt'. Supports a) and b) on the BSDs, but not a), because SIGTERM > means 'enter rescue mode'. And doesn't support a) to c) on > [GNU/]Linux, because it follows the systemd signal convention (signals > above SIGRTMIN) there instead. Or, to put it another way: On FreeBSD, PC-BSD, and OpenBSD, the nosh system-control and system-manager tools use (b) and (c) for compatibility with existing BSD tools. But for further compatibility with existing BSD tools, SIGINT means (orderly shutdown and) reboot and SIGTERM means bring up rescue mode. To the current FreeBSD/PC-BSD init and OpenBSD init and accompanying toolset, SIGINT means reboot (indivisibly from Ctrl+Alt+Del) and SIGTERM means single-user mode. The FreeBSD/PC-BSD and OpenBSD kernels send SIGINT but don't send SIGWINCH in the first place. If they did, though, it would be regarded as (e). On Linux, the nosh system-control and system-manager are designed to use the Linux kernel and systemd signalling conventions. (d) and (e) are Linux kernel conventions. (a), (b), and (c) are BSDisms that as Guillermo noted have never really been the case on Linux at all. (b) actually outright conflicts with a van Smoorenburg init convention (where it means "restart the initctl server"). It also conflicts with an upstart convention (where it means "re-connect to Desktop Bus"). Moreover, upstart and systemd have made the semantics of SIGTERM into a mess, where for a system-wide systemd/upstart init it means one thing and for a per-user systemd/upstart init it means something quite different. Neither meaning is reboot. Orderly shutdown and halt/reboot/poweroff are various real-time signals, instead. The systemd real-time signals for orderly shutdown and halt/reboot/poweroff are also available on FreeBSD and PC-BSD, in fact. Apart from this last, all of the aforegiven is documented in the system-control and system-manager user manuals. See https://jdebp.eu./FGA/emergency-and-rescue-mode-bootstrap.html for the evolution of single-user mode into rescue mode. The nosh system-manager can in fact do both emergency and rescue modes on FreeBSD/PC-BSD and OpenBSD as well as on Linux. The BSD bootstraps themselves don't have two flags, though.
Re: nosh version 1.28
I don't know why you asked about FreeBSD rc.d just on the Debian mailing
list; but I'm going to deal in both of those and others besides, here,
and things that apply across both, so I've re-included the FreeBSD
mailing list. (-:
2016-08-14 15:10, Julian Elischer:
I don't know if I just missed it, or it isn't there but I have a
question..
You give examples of importing systemd service files. What about
importing rc.d files with all their ability to run arbitrary shell
commands.
And once you have the services defined, what is the logical equivalent
of rc.conf, which can supply parameters for each service and turn them
on and off? can you import from rc.conf?
You did miss it. (-:
What you missed has grown to be a significant subsystem. It was actually
mentioned a couple of times in the 1.28 announcement. It's the external
configuration import subsystem. You can read about it in the nosh Guide:
xdg-open /usr/local/share/doc/nosh/external-formats.html
As you can see, there's a whole section on importing from rc.conf into
native service management mechanisms. ("rc.conf" covers several
sources, note, including a FreeNAS configuration database and
/etc/defaults/rc.conf .)
The native service mangement mechanisms are the "enable" and "disable"
subcommands to the system-control command, and using the envdir command
in the normal daemontools-family style way. The enable/disable
mechanism in "rc.conf" is treated as if it were a preset (in systemd
nomenclature). You tell service management to "preset" a service, and
it will look at /etc/rc.conf and /etc/rc.conf.local (as well as some
other preset mechanisms) to determine what to set the native
enable/disable state to. The user manual page for the preset subcommand
(of system-control) explains what the preset mechanisms are in detail.
You can set up environment directories how and where you like, but
there's a convention that is shared by the "convert-systemd-units" tool,
the "rcctl" shim, and the external configuration import subsystem as a
whole. This convention is an environment directory named "env" that is
in the service directory. The "rcctl" shim gets and sets variables
there; and the import subsystem places converted "rc.conf", /etc/fstab,
/etc/ttys, /etc/my.cnf, and other stuff there.
One example of this in action, out of many in the import subsystem, is
jails that have been set up the version 9 way in "rc.conf". Those are
turned into service bundles, with "env" environment directories that
contain environment settings such as "hostname", "mount_devfs", and
"interface". The "run" script for the jail service very simply turns
the environment variables into arguments to the "jail" comand. In a
system with an original OpenBSD "rcctl" command, one would expect to be
able to set (version 9) jail control variables by manipulating
/etc/rc.conf with commands like "rcctl set wibble hostname wobble". The
"rcctl" shim and this shared convention mean that one need not stray
that far from this if "rcctl" is one's habit: "rcctl set v9-jail@wibble
hostname wobble" does the "native" thing of setting the "hostname"
variable in the (conventional) environment variable directory for the
"v9-jail@wibble" service.
Bonus feature for those with other habits: With nosh service management
in place, one can actually import from /etc/rc.conf settings *on Debian*
(as long as one sets up a FreeBSD/PC-BSD-style /etc/defaults/rc.conf
pointing to it with rc_conf_files). One can use /etc/ttys, too.
As for importing scripts that run "arbitrary shell commands", there are
several points.
First, you may not need to. Note that most of what you get out of the
box in /etc/rc.d/ and /usr/local/etc/rc.d/ on FreeBSD and PC-BSD has
already been converted. Remember that project that I had to convert 157
services? Take a look at the nosh roadmap page. It's almost done.
Second, you may not need to. Take a look at what actually comes in the
nosh-bundles package nowadays. Discounting the 'cyclog@' service
bundles there are just over 540 service bundles in there, from samba to
ntp, from saned to ossec@agentd. (Including the 'cyclog@' service
bundles, it is over a thousand service bundles.) The Debian world
doesn't get left out, either. Although it's a lot more difficult than in
the BSD worlds to come up with a list of "core" Debian services, a lot
of the basics of Debian are also covered by this, from kernel-vt-setfont
through irqbalance to update-binfmts. And those more-than-540 service
bundles cover lots of "non-core" stuff, from (as aforementioned)
OSSEC-HIDS, Salt, and RabbitMQ to publicfile httpd over IPV6.
Third, you may not need to. This was mentioned in the 1.28
announcement, in fact. The external configuration import subsystem
makes *further* service bundles, beyond the pre-made ones that come in a
binary package. It creates service bundles to run (optional) per-user
service management,
Re: nosh version 1.28
Guillermo: OpenBSD === [...] There are an awful lot of limitations to OpenBSD, [...] How funny it is that this summary and the WWW page echo the sentiments in skarnet.org packages' source files comments and commit messages :D We didn't collaborate. (-: I don't actually know what M. Bercot has said on the matter. It's not unexpected that two projects sharing several design principles will hit the same problems with OpenBSD, though. The more interesting things to consider are other operating systems. For starters: Ubuntu on Windows NT would possibly be a less problematic port than OpenBSD. Whilst it, too, has obstacles with pseudo-terminals, framebuffers, and the system manager; what it doesn't have, that OpenBSD has, is the difficulty with the package management. Ubuntu on Windows NT has APT like Debian, of course. I've said before, elsewhere, that one could probably successfully get nosh service management, UCSPI support, and log management working on Ubuntu on Windows NT; although obtaining an actual daemon context is still problematic. (https://news.ycombinator.com/item?id=11416376) Moreover: UbuntuBSD and Debian FreeBSD shouldn't have the obstacles with the pseudo-terminals, framebuffers, and the system manager; these, after all, being things that the FreeBSD operating system kernel provides in largely suitable form. (https://news.ycombinator.com/item?id=11326457) Incidentally: I wrote a while ago that UbuntuBSD probably wouldn't use Mewburn rc. It doesn't. UbuntuBSD 16.04, released this month, uses BusyBox init and (the Debian port of) OpenRC rc.
Re: nosh version 1.28
> > I'm going to be very naughty and patch 1.28 post-release. It's a minor > change. > I've changed my mind. I'm going to point the two of you at a potential version 1.29 and see how you get on. This is because I have ended up doing slightly more than a 2-line script patch.
Re: nosh version 1.28
Guillermo: Could the plain 'setlock' invocation in script source/default.html.do be changed to './exec setlock', as in scripts source/default.1.do and source/default.8.do, please? Otherwise we have the chicken-and-egg situation we had with nosh-1.19: It was supposed to be that way already. (-: You've been beaten by the person who is kindly doing a PKGBUILD for Arch Linux, who spotted this a couple of days ago. I'm going to be very naughty and patch 1.28 post-release. It's a minor change. It turns out that even when I was building from scratch on OpenBSD, I was building on machines that always had *someone's* setlock available. (-: I had a similar problem building something else quite recently. It had several Bernstein libraries in it. I didn't see several C2011 build problems because I had libowfat installed and that was being used instead.
Re: nosh and redo have moved
Guillermo: Clicking on the links to the *.bz2 source packages results in an HTTP 404 error (but not with the links to the *.gz ones, so the packages can still be downloaded). The same thing happens with the links about their slashpackage-style, which point to a nonexistent FGA (I don't remember if it existed in the old site), and with the links to any of the OpenBSD binary packages. It's a complication in the mirroring arrangements amongst the WWW servers. I'm hoping to do something about it next week. The OpenBSD binary packages and the *.tar.bz2 source archives are on http://jdebp.info./Softwares/nosh/ but haven't made it to https://jdebp.eu./Softwares/nosh/ . The hyperlinked Frequently Given Answer did not exist before.
nosh version 1.28
The nosh package is now up to version 1.28 . * https://jdebp.eu./Softwares/nosh/ * https://www.freebsd.org/news/status/report-2015-07-2015-09.html#The-nosh-Project * http://jdebp.info./Softwares/nosh/ There's a lot in this one: MySQL and MariaDB changes; more prophylaxis for Desktop Bus bus activation; improvements to systemd unit conversion; support for the old svc -x; machineenv; improvements to service management; fixes for the per-user manager; improvements to the console terminal emulator; BSD boot mode changes; the ability to pass more open sockets to connection-accepting programs; cron; and OpenBSD. Italics and colour == * https://jdebp.eu./Softwares/nosh/italics-in-manuals.html This isn't a toolset change, per se. But the WWW site now has a guide to seeing actual italic text in manual pages. The nosh toolset's user-space virtual terminals support true italics (if one has the fonts) or obliquing, and this works with them. MySQL and MariaDB changes = * https://jdebp.eu./Softwares/nosh/mariadb-and-mysql.html New in version 1.28 is a different and up-to-date way of managing MySQL and MariaDB server services — where "new" translates to finally getting rid of that unnecessary mysql_safe wrapper and doing things the way that daemontools-family toolset users have wanted to do them since the turn of the century. There's a lengthy exposition on the WWW site, q.v.. The major visible effect is that your "mysql" or "mariadb" service is now an alias, for something like a "mysql@" or "mysql@01" (if you have [mysql01] in your my.cnf) service. The configuration file import mechanism tries to construct/update mariadb@NN and mysql@NN service bundles for you, based upon your MariaDB and MySQL configuration files. Further prophylaxis for Desktop Bus bus activation == * https://jdebp.eu./Softwares/nosh/avoid-dbus-bus-activation.html The nosh toolset now comes with a dbus-daemon-launch-helper replacement. The purpose of this is to sit in your /usr/local/etc/dbus-1/system.conf (or equivalent) and redirect to service management attempts, by the Desktop Bus broker daemon, to demand-start services. It is slightly fiddly to install, requiring manual setup by the system administrator, there being no simple way to add overrides to /usr/local/etc/dbus-1/system.conf and it requiring that you allow the "messagebus" user the necessary access for starting and stopping services (but not necessarily *superuser* access — rembember ACLs). To assist with this, several popular Desktop Bus "services" now exist as alias names for service management services. These are just symbolic links to the service bundle directories, of course. So, for example: With the helper in place, Desktop Bus bus activation will try to demand-start a service named "org.freedesktop.PackageKit" using service management. This is just an alias for the "packagekit" service. Improvements to systemd unit conversion === Ideal mode is now closer to the daemontools-family mainstream, defaulting to the daemontools-family norm of always restarting services. Quirks mode, conversely, now implements more of the non-daemontools redirection semantics for standard I/O, in particular with regard to listening socket units. Some more Linuxisms have been added. Limits (where applicable) can now take SI and IEC suffixes (so you can, say, express limits in kiloseconds). This latter is actually an augmentation to the underlying softlimit command. Passing more open sockets to connection-accepting programs == The improvements to systemd unit conversion also allow passing more than one listen()ing socket to connection-accepting programs. You can use, say, ListenStream and ListenDatagram and the conversion utility will translate this into an appropriate chain of multiple invocations of udp-socket-listen and tcp-socket-listen. It will do local-stream-socket-listen, local-datagram-socket-listen, netlink-datagram-socket-listen, and fifo-listen too. The motivator for this was Daniel J. Bernstein's dnscache. I have modified versions of tinydns, dnscache, and taiclockd that understand the LISTEN_FDS protocol for their being told about listening sockets that have been opened for them, and don't open their own sockets in that case. dnscache, in particular, takes a UDP socket and a listening TCP socket. The UCSPI tools in this version of the toolset can now provide these two to a dnscache process. One simply chains through udp-socket-listen and tcp-socket-listen to dnscache, using the --systemd-compatibility flag. The sharp-eyed will notice that the tinydns and dnscache services are following in the footsteps of the mariadb and mysql services, being instantiated for relevant IP addresses by the configuration import subsyst
nosh and redo have moved
The whole sorry tale of why is on the new WWW site. The upshot of it is that nosh and redo are in a new place. * https://jdebp.eu./Softwares/nosh/ ** https://jdebp.eu./Softwares/nosh/source-package.html ** https://jdebp.eu./Softwares/nosh/freebsd-binary-packages.html ** https://jdebp.eu./Softwares/nosh/debian-binary-packages.html * https://jdebp.eu./Softwares/redo/
Re: Runit debian package fix for Ubuntu 16.04
Pierre Jacomet: I did a couple of fixes to the debian package installer such that the debian/ubuntu installer deals properly with the dichotomy between systemd and upstart and the fact that they both may be installed in the system at the same time. This was causing the installer to fail in ubuntu 16.04. I have a fix, which involves strictly working with the debian files and wanted to submit it. Is the right place to start here or in some debian or possibly an ubuntu maintainers list? See these, which I have already tried to draw M. Pape's attention to once already (without a response so far): * http://www.mail-archive.com/supervision@list.skarnet.org/msg01224.html * http://unix.stackexchange.com/a/284453/5132
Re: logging and mutualise outputs
Vincent de RIBOU: Is it possible to perform the same mechanism but having concatenated file along all the others at all times ? There's always tail with the -F option on a list of "current" files. But you're then limited to just the "current" log data, it's tricky to track which output comes from which log, and it's tricky to track over missed log rotations when tail wasn't running. One way to track things better when one knows that one is specifically looking at a TAI64N log directory (as produced by cyclog/multilog/s6-log) is exemplified by Russ Allbery's multilog-watch (https://www.eyrie.org/~eagle/software/multilog-watch/) and my export-to-rsyslog (http://homepage.ntlworld.com./jonathan.deboynepollard/Softwares/nosh/). It is the idea of log cursors. multilog-watch maintains a single cursor because it only watches one log directory. export-to-rsyslog has a suite of maintained log cursors which it holds in a cursor directory, because it can watch multiple log directories. A cursor is roughly the TAI64N stamp of the last log entry to be processed. (Read the export-to-rsyslog manual page for details.) Watching a TAI64N log directory with cursors has the advantage over some other mechanisms that it entirely de-couples the log watching from the logging daemon. There's no back pressure on the running service (back down a pipeline) if the log watcher is very behind. One can have multiple quite independent log watchers with their own sets of cursors. One can run the log watcher when the logging daemon isn't running, and vice versa. The caveats of (say) making sure that the reading end of a FIFO never goes away never arise. multilog-watch mails logs, which isn't really what you want. export-to-rsyslog writes log entries in RFC 5424 form to a (datagram) socket. Its major usage is providing near real time export of multiple TAI64N log directories to something like logstash, with the individual log cursor names encoded in the RFC 5424 data. You are not required to point it at logstash, though, and can point it at something simpler. However, RFC 5424 is still a lossy protocol, not least because it doesn't have full TAI64N resolution. So pointing this at an RFC 5424 receiver is to introduce codec loss. Nonetheless, it's one way to obtain a centralized combined log, in a properly size limited and rotated TAI64N log directory no less, that you can manipulate with all of the log directory tools that you are used to. For those interested despite the aforementioned, the nosh service bundles package has pre-made service bundles for an RFC 5424 server over either a local datagram socket or a RFC 5426 UDP socket. The local-syslog-read service bundle is an instance of local-datagram-socket-listen opening /run/log (/dev/log on Linux operating systems) and chaining to syslog-read. The udp-syslog-read service bundle is an instance of udp-socket-listen binding to the well-known "syslog" UDP port and chaining to syslog-read. syslog-read itself is a UCSPI-style tool that reads RFC 5424 from the already-opened socket and writes it to standard log suitable for piping into cyclog/multilog/s6-log. There are accompanying cyclog@local-syslog-read and cyclog@udp-syslog-read service bundles for that very thing. There's no reason that someone cannot take the idea of log cursors and make a tool like export-to-rsyslog but with a non RFC 5424 back-end, though.
Re: logging and mutualise outputs
Laurent Bercot: for logdir in `cat logdir_list` ; do (cd $logdir && cat *.s *.u current) ; done | sort > logfile find `cat logdir_list` -maxdepth 1 -type f \( -name '*.[su]' -o -name current \) -print0|xargs -0 sort -m -- > logfile sort can do a merge sort directly from the original files, which are of course already sorted. It's better when one's logs are *big*.
Re: system-control unload-when-stopped (was: nosh version 1.23)
Jonathan de Boyne Pollard: 1.28 is pretty much frozen now, and its release is awaiting my writing a WWW page about the MySQL/MariaDB changes. Enjoy a peek at one of the new pages already written: * http://homepage.ntlworld.com./jonathan.deboynepollard/Softwares/nosh/italics-in-manuals.html Now enjoy the page about MariaDB and MySQL changes: * http://homepage.ntlworld.com./jonathan.deboynepollard/Softwares/nosh/mariadb-and-mysql.html
Re: Entering a passphrase interactively in a runit script
Christophe-Marie Duquesne: Any idea how to proceed? You're running a daemon. It really shouldn't have an interactive user interface. Remember the lessons that resulted in Session 0 Isolation in Windows NT. There are several poor approaches. Here are two more: * Make your password into configuration information, that is saved by something else in a secure place that only your daemon can access. Yes, this has the gross disadvantage that you are writing the password to storage somewhere, in plaintext. I mention this not because it is good, but because it is the next thing that people think of. It is as rife with security problems as having an interactive user interface is. Really, this should not be a password at all. Ciphertext encrypted with a secret private key that the daemon can decrypt using a public key that it holds, is a step in the right direction. Remember that attackers will know the decryption key, the encryption/decryption algorithm, and the plaintext. * Provide other services that mediate between your daemon and users. This can be subdivided into the Windows NT approach and the systemd approach. The Windows NT approach depends from the architecture of Windows NT, in particular secured desktops. The systemd approach is to copy the ideas of lpr and mailx, and turn password prompts into batch jobs. The daemon drops a password request file into an input hopper, and waits for something else to process and deal with it. The something else is another service, possibly more than one, running interactively on a user's logged in desktop or connected to a virtual terminal. See https://freedesktop.org/wiki/Software/systemd/PasswordAgents/ . It of course lacks a Secure Attention Key and provides no mechanism for users to distinguish spoofed password requests. The resultant systemd design choice is to prevent non-superuser processes from being able to submit requests into the input hopper. A daemon that drops privileges cannot use this, for example.
Re: system-control unload-when-stopped (was: nosh version 1.23)
Guillermo: And as a consequence, the Guide ends up never actually explaining what the unload-when-stopped subcommand does (AFAICS, make service-manager unload the named service after its 'stop' file finishes executing, whatever the reason for running it was, so that a subsequent service-status on the service's directory says that no supervisor is running). I really don't want to have to change the manual back, so I might have to get on and make the change. (-: I'll see what I can do for 1.29. 1.28 is pretty much frozen now, and its release is awaiting my writing a WWW page about the MySQL/MariaDB changes. Enjoy a peek at one of the new pages already written: * http://homepage.ntlworld.com./jonathan.deboynepollard/Softwares/nosh/italics-in-manuals.html
Re: nosh: convert-systemd-units' treatment of Standardxxx=tty directives
Guillermo: So, the convert-systemd-units tool recognizes the systemd 'StandardInput=tty' unit file directive, and translates it to a vc-get-tty + open-controlling-tty sequence in the 'run' file of the generated bundle directory. Which does more than just redirecting stdin to the appropriate character device file. Out of curiosity, why is the controlling terminal allocation done there? Is it just to respect systemd's behaviour, or is there a more fundamental reason? It is to provide largely equivalent behaviour, yes. In 1.28, forthcoming soon, you'll see that I spent a little time on this. Quirks mode remains much the same. But in ideal mode there are some tweaks with respect to the defaults, as well as a StandardError=log override. Guillermo: I also note that, unlike the StandardInput case, 'StandardOutput=tty' and 'StandardError=tty' directives are not supported by the conversion tool and ignored with a warning, when it appears that in the absence of a 'StandardInput=tty' directive they could just be translated to an fdredir (or fdredir + fdmove --copy sequence) naming the device file. No controlling terminal required in this case according to the systemd documentation. And when 'StandardInput=tty' is present, 'StandardOutput=tty' (or 'inherit') and 'StandardError=tty' (or 'inherit') look already covered to me by the open-controlling-tty invocation, since that utility redirects stdout and stderr to the terminal, along with stdin. So, again out of curiosity, is there a reason for ignoring these uses of StandardOutput and StandardError? It's an unusual case, that I haven't seen in the wild. The 1.28 changes were partly to make ideal mode more normal but also partly in response to things that I have seen. That, though, I have not. In practice, service unit writers either want the like-a-ttylogin-service scenario -- setting the controlling terminal, allowing input, and all -- or they push logs out to a log service. See http://askubuntu.com/questions/770673/ for just one of many examples of the former use case.
Re: Setting open files for a process started by runit (using chpst)
it looks like chpst cannot currently (or never could, not sure) change hard limits. The limit stanza in an upstart job file sets both hard and soft limits, explicitly. * http://upstart.ubuntu.com./cookbook/#limit * http://smarden.org/runit/useinit.html#upstart Bernstein's original softlimit did as the command name says, and changed only soft limits. The same is true of the softlimit in daemontools-encore, s6-softlimit, chpst in runit, runlimit in perp, and the softlimit in nosh. To change hard limits instead or as well, I wrote a chain-loading ulimit for nosh. If one is writing shell scripts, then of course shells have built-in ulimit commands. The nosh toolset's ulimit is for when one isn't writing shell scripts, e.g. nosh or execlineb scripts, or command chains embedded within a script. * http://homepage.ntlworld.com./jonathan.deboynepollard/Softwares/nosh/guide/ulimit.html
Re: HAProxy Hot Reconfiguration with s6
IMHO more "proper" or rather interesting approach, could perhaps be some kind of helper program - "injecting" "alternator" sub-process - which would be running under s6-supervise. The haproxy people have already done this work for you. They call the program haproxy-systemd-wrapper. It's undocumented, naturally. There's also a haproxy-upstart-wrapper. * http://www.serverphorums.com/read.php?10,649689 * https://unix.stackexchange.com/questions/239805/ * https://github.com/wankdanker/haproxy-upstart-wrapper
Re: svlogd sub-directory log rotation?
I have an application for which I need to keep all logs indefinitely. Rotating them into sub-directories would help keep the number of files from expanding into an unmanageable number. So if someone has any other suggestions on how to manage log rotation without pruning, I'm all ears. So what you really need is a logging tool without the maximum total size cap, or with it disabled somehow. * http://homepage.ntlworld.com/jonathan.deboynepollard/FGA/daemontools-family.html#Logging I think that you've identified a niche, in between dumblog and tinylog, for someone to fill. (-:
runit and daemontools on Ubuntu 16 and Ubuntu 15
Ubuntu 15 and 16 have both systemd and ubuntu installed. This causes problems, and has been causing for over a year. Yes, all of these are the same core problem: * http://unix.stackexchange.com/a/284453/5132 * https://bugs.launchpad.net/ubuntu/+source/runit/+bug/1448164 * https://bugs.launchpad.net/ubuntu/+source/runit/+bug/1528396 * https://bugs.launchpad.net/ubuntu/+source/runit/+bug/1563844 * https://bugs.launchpad.net/ubuntu/+source/runit/+bug/1522416 * https://bugs.launchpad.net/ubuntu/+source/runit/+bug/1515090 * https://bugs.launchpad.net/ubuntu/+source/runit/+bug/1543447 * https://bugs.launchpad.net/ubuntu/+source/runit/+bug/1545415 * https://bugs.launchpad.net/ubuntu/+source/runit/+bug/1573541 * https://bugs.launchpad.net/ubuntu/+source/runit/+bug/1576919 * https://bugs.launchpad.net/ubuntu/+source/runit/+bug/1577826 * https://bugs.launchpad.net/ubuntu/+source/runit/+bug/1579310 * https://bugs.launchpad.net/ubuntu/+source/runit/+bug/1579397 * https://bugs.launchpad.net/ubuntu/+source/runit/+bug/1580208 * https://bugs.launchpad.net/ubuntu/+source/runit/+bug/1581473 * https://bugs.launchpad.net/ubuntu/+source/runit/+bug/1582000 * https://bugs.launchpad.net/ubuntu/+source/runit/+bug/1584214
nosh version 1.27
The nosh package is now up to version 1.27 . * http://homepage.ntlworld.com./jonathan.deboynepollard/Softwares/nosh.html * https://www.freebsd.org/news/status/report-2015-07-2015-09.html#The-nosh-Project In fact, it is soon to be version 1.28. This is a somewhat delayed notice for 1.27, because I forgot to send out the notices for versions 1.27 and 1.26 after updating the WWW site. As can be seen from the roadmap, we are at the point in rc.d conversion for FreeBSD/PC-BSD where it's actually easier to count the things that remain unconverted. Discounting the PC-BSD Active Directory services and a handful of suspect FreeBSD services (such as growfs, which doesn't apply to ZFS in the first place) the remaining things to be converted can be counted on the fingers of one hand. The external configuration import mechanism has gained the ability to handle stf, atabridge, mdconfig, and a few others. There are also a whole bunch more service bundles: cross-platform, for Linux, and for BSD. * http://homepage.ntlworld.com./jonathan.deboynepollard/Softwares/nosh/roadmap.html#FreeBSDrc.d The OOM Killer avoidance measures from version 1.25 are now employed in the PostgreSQL service bundle as well. The systemd service unit conversion tool has gained a whole load of NUMA-related extensions: NUMAInterleave, NUMAMemBind, NUMACPUNodeBind, NUMAPhysCPUBind, NUMALocalAlloc, and NUMAPreferred. These it translates into the equivalent invocations of the numactl chain-loading utility. It has also gained a couple of minor fixes and tweaks. The %m substitution now works, and service bundles comprising FIFOs or AF_LOCAL sockets are now created so that they are ordered after any relevant filesystem mount services. By request, the nosh Guide has gained a whole chapter of cheatsheets, giving quick one-liner pointers to some common tasks. The chapter is divided into three sections: chain loading, logging, and service management. The service management division is subdivided into daemontools-style commands, systemd-style commands, OpenBSD-style commands, SMF-style commands, and common commands. The chain loading division gives a number of the more common commands used in chain-loading run scripts (and whereever else one might want to use them). There have been improvements in static network setup, including fixes for some bugs in static_arp and static_ndp and a more cross-platform replacement for the static-networking service. The nosh-bundles package now supplies several aliases for services, which are just plain old symbolic links. So (for example) one can address the CUPS service as either org.cups.cupsd or just plain cupsd. Things to look forward to in version 1.28 already include: more service bundles; another chain-loading utility; a major revision to MySQL and MariaDB service bundling, to reflect the pushes by their own developers to obviate their rc scripts and the mysql-safe command and just run mysqld directly under service management using the tools provided by the service management system; and a change relevant to the all-important linux_logo command. (-:
Re: Announcing cgid v0.1.0
fREW Schmidt: To read the docs check out https://github.com/frioux/cgid/releases/download/v0.1.0/cgid-v0.1.0, where you can see examples of how to run under `nosh` or `s6`. That URL returned an XML error document from Amazon to me. https://github.com/frioux/cgid/releases/tag/v0.1.0 was a more productive URL. https://github.com/frioux/cgid took me right to the aforesaid doco, though. (-:
nosh version 1.25
The nosh package is now up to version 1.25 . * http://homepage.ntlworld.com./jonathan.deboynepollard/Softwares/nosh.html * https://www.freebsd.org/news/status/report-2015-07-2015-09.html#The-nosh-Project As you may have noticed from discussions elsewhere, a new oom-kill-protect utility has snuck in at the last moment. This takes Linux-style OOM Killer score adjustments (an integer between -1000 and 1000), BSD-style binary YES/NO settings, or a special setting for querying the "oomprotect" environment variable; and tries to do the closest matching thing for each platform. Details are in the manual, of course. With this, the OOMScoreAdjust setting is now converted by the convert-systemd-units utility. The local-syslogd, udp-syslogd, and syslogd service bundles make use of oom-kill-protect with the special environment variable setting in their run programs. So FreeBSD bug #204741 is addressed in a more general fashion that can be easily used in other service bundles. "rcctl set syslogd oomprotect YES" and "rcctl set syslogd oomprotect NO" can be used to turn OOM Killer protection on and off. Other things in this version include: * More configuration import utilities, covering ip6addrctl, webcamd, and NFS settings. * A fix for a problem with configuration import on Linux in version 1.24. * Two minor utilities for querying the fstab database, get-mount-what and get-mount-where, needed by the configuration import for mdconfig (but generally usable). * New binary "run-" packages for OpenSSH server, syslog on a local socket, and klog. * The new syslog and klog packages provide the Debian package manager's virtual package names "linux-kernel-log-daemon" and "system-log-daemon" (per Debian Bug #67604). As can be seen from the roadmap, we are nearing the end of the rc.d conversion for FreeBSD. Additions in this release include nfsserver, gptboot, rtadvd, virecover, and pcdm. Almost all of mdconfig is actually done, bar some after/before orderings. * http://homepage.ntlworld.com./jonathan.deboynepollard/Softwares/nosh/roadmap.html#FreeBSDrc.d
Re: UCSPI httpd?
It might be worth asking what * http://lua-users.org/lists/lua-l/2006-10/msg00770.html was all about. See also: * https://dev.openwrt.org/ticket/8453 * http://people.igalia.com/aperez/bill/lib/www/http.html * http://www.mail-archive.com/fossil-users@lists.fossil-scm.org/msg02065.html
Re: How to compile nosh?
I was going to warn you that you're about to kick yourself, but I see that Guillermo has already replied, so you're probably already kicking yourself. (-: fREW Schmidt: First off, there's no README or anything, so I sorta have to guess. [...] Again there's no README for redo. As Guillermo said, the instructions for building from source are right there on the WWW pages whence one downloads the source package. * http://homepage.ntlworld.com./jonathan.deboynepollard/Softwares/nosh/source-package.html * http://homepage.ntlworld.com./jonathan.deboynepollard/Softwares/redo.html Both can be built either in the Debian way into installable packages, with the Debian dpkg-buildpackage tool (with directories prepared with package/debian/prepare); or as a self-contained package, slashpackage style, with the conventional slashpackage commands. FreeBSD/PC-BSD doesn't have Debian toolsets, but there's a bsd/rules mechanism (prepared with package/bsd/prepare) available for building BSD package files with a "Debian-like" system. Or one can build as a self-contained package on BSD, too. As Guillermo said, what you were fiddling with were actually internals of the build system, that it sets up for you. All that you have to remember, whether you choose self-contained package or Debian-esque build, is one command line. It's *the same* command across nosh and redo, moreover. (It is indeed the same command in daemontools, UCSPI-SSL, ftpcopy, and others. I've also employed it in my own source packages that I use for djbdns and publicfile.) * http://www.superscript.com/ucspi-ssl/install.html * https://ohse.de/uwe/ftpcopy/install.html * http://www.fehcom.de/ipnet/ucspi-ssl.html As the (nosh and redo) source package download and build instructions pages say, slashpackage is a whole subject in its own right. One day I might put up a FGA about it. But if one looks to other systems and spots hierarchical package naming, self-contained build-trees, "current version" symbolic links, atomic upgrades, and "index"/"alternatives" directories one can see those all in slashpackage. * https://news.ycombinator.com/item?id=10505897 It is not mine, but the invention of Some Bloke On Another Continent: * http://cr.yp.to/slashpackage.html He didn't really document the build process too well. It's the build process that mainly concerns us, here. Some Other Bloke On A Different Continent To The First Bloke wrote about that part in 2002, though: * https://web.archive.org/web/20021011214224/http://www.skarnet.org/software/compile.html A Third Bloke On A Third Continent (at the time) also wrote about it in 2002: * http://thedjbway.b0llix.net/builddjb.html#slashpackage A Fourth Bloke even got in on the act: * http://code.dogmap.org/slashpackage/
Re: Supervising a pipeline?
Laurent Bercot: You can't supervise a pipeline per se; you need to supervise both processes in the pipeline independently, and make sure the pipe isn't broken when one of them dies. So, have "exec inotifywait /dev/disk" as foobar/run, and have "exec automounter.py" as foobar/log/run. This will work with daemontools, runit and s6. (You can accomplish the same goal with perp and nosh too; the syntax will just be different.) Actually, the syntax for nosh can be exactly as described: something/run and something/log/run . It's not ideal, because of course then there's no properly separated logging of the automounter. Laurent Bercot: Alternatively, you could use s6-rc and create the "inotifywait" and "automounter" longrun services in a pipeline; your compiled database would then include instructions to set up the supervised pipeline for you. This is more complex to set up than just using the integrated pipe management in svscan and runsvdir, but it's also more powerful, because you can pipeline an arbitrary number of processes that way (this is also what nosh does). Yes: The nosh toolkit makes this extensible. The aforegiven configuration is enacted by the daemontools service scanner. The underlying service management supports arbitrarily long pipelines of services, each one's standard output and standard error feeding into the standard input of the next one along. The system-control utility looks at the "log" symbolic links of whatever collection of services it is operating upon and sends a "plumb services together" order to the service manager. So one could construct a 3-service long pipeline: inotifywait into Python program into cyclog instance.
Re: Holidays Brainstorming: instanced supervision
Olivier Brunel: With instanced services, it means when you enable such a service you add/specify an instance name. So e.g. the servicedir is "getty@" but you enable "getty@tty2" -- which just means the servicedir getty@ is copied under a different name in the scandir. The intent being that to enable another getty, you just enable getty@tty3 and that's it. [...] nosh has much the same. Look at the nosh-bundles package and you'll find a set of ttylogin@ttyv* service bundles, for starters. There are plenty of others, from mdmfs@-tmp to populate@var, with a whole bunch of kmod@* service bundles along the way. The convert-systemd-units tool can convert templatized systemd units. The configuration import subsystem instantiates multiple instances of a generic service for per-user dbus daemons, Warden jails, webcamd services, rfcomm_pppd services, static_arp, static_ndp, pefs, geli, geom, fsck, ZFS mounts, and so forth. If you want to know where the idea of instances of a generic can be used, look in the nosh-bundles package. There's no instantiation mechanism in the service bundles themselves. Everything is pre-instantiated at conversion/creation time. That's when all of the template parameter substitution happens, not at runtime. This is not to say that it's impossible to share service/ directories amongst service bundles. It's quite possible, and part of the design. It's just not used for instantiation; because in that case the service/ directories differ in their run/start/stop/restart programs, because of the instance names.
nosh version 1.24
The nosh package is now up to version 1.24 . * http://homepage.ntlworld.com./jonathan.deboynepollard/Softwares/nosh.html * https://www.freebsd.org/news/status/report-2015-07-2015-09.html#The-nosh-Project Minor items in this release include: * A fix for BSD keyboard layout import, that makes both "duml" and "ddia" be U+0308 for now. Technically, diaeresis and umlaut are distinguishable in Unicode decomposed forms (using U+034F). But for now, everything is simple unadorned combining diaeresis. * A few more service bundles, for DBMail and for sudo (which in its vanilla form puts its timestamp files in /var/lib instead of /var/run and needs a cleanup service -- see Debian Bug #786555). * Use of rtprio and idprio when converting system service units on FreeBSD/PC-BSD. * Improvements to the framebuffer video mode selection in user-space virtual terminals for FreeBSD/PC-BSD. It now comes up in the same display size as on Debian Linux on my test machines. * Doco and other fixes from user feedback on version 1.23. (I've already begun some further VirtualBox host adjustments, as we discussed, for 1.25.) There is one major item in this release. PC-BSD 10.2 === Until now, I'd been testing on a PC-BSD system that had been upgraded, with various contortions, from version 9. This was still using UFS filesystems, listed in /etc/fstab; which the external configuration import subsystem had been happily importing to native service bundles. Over Christmastide I installed a PC-BSD 10.2 system from scratch, discovering some interesting oddities. These included installation failing if you tell it that you are in the United Kingdom using a U.K. keyboard (PC-BSD Bug #12986); and the GRUB menu editor, as configured by the installer, operating in a truly eye-stretching 46 column by 28 row mode (by my count), and not displaying the underscore character correctly. The important thing to know is that PC-BSD has for some time (at least since 2013) been ZFS-only, as far as installation goes. (One can of course mount other filesystem types after installation.) As Henry Ford might have said "Any customer can install to any filesystem type that xe likes, as long as it is ZFS.". The result is that if installing from scratch one gets a whole load of ZFS datasets, and an empty (save for /proc and swap) /etc/fstab file. So the major push for version 1.24 has been to get the configuration import system to deal with this, which it now does. It will create mount services for all ZFS mounts, enable the ones that are "on", give them an inter-service ordering, and deal with the special-casing for the root (which the installer, oddly, marks as not automatically mounted, even though it of course is). Alongside this, a large chunk of the remaining NetBSD rc.d services, from the on-going project to entirely replace them, have been crossed off the list. These include mfs for /tmp, static networking and static ARP, pefs, serial port BPS and framing setup, ppp, rfcomm_pppd, persistent "entropy" for the randomness subsystem, and ipfw. The progress of this work has been open from the start, and you can follow along on the roadmap WWW page. Indeed, you can even join in, if you can convert any of the remaining few items. There's more work to be done. But I now have ZFS-only PC-BSD 10.2 running nosh system-managed and service-managed. Some notes for those eager to follow: * Yes; I'm working on a pcdm service. No; it doesn't help that it's undocumented. Yes; that hoopla and palaver with forked subshells and multiple while loops calling sleep is exactly the sort of thing that proper service management is intended to obviate. * If you have problems with devd, stale nologin from previous boots, and other things that use /var/run, it's because the convert_varrun service isn't enabled and your system has not been thus or otherwise migrated to /run. This will be properly enabled by a preset in the next version. Enable it and reboot. Or just start it and reboot. Or just boot into rescue mode and turn /var/run into a symbolic link to /run yourself. * No; the nosh-run-system-manager package doesn't work properly on PC-BSD, as it does on vanilla FreeBSD. PC-BSD 10.2 doesn't use the FreeBSD boot loader, like my old upgraded installation of PC-BSD 9 did. It uses GRUB. The PC-BSD people apparently plan to get rid of GRUB in the future, and use the FreeBSD loader once more. So this problem goes away when GRUB does. (-: In the meantime, use 'set kFreeBSD.init_path="/sbin/service-manager"' in the GRUB configuration. * The root-resizing subsystem that was new to FreeBSD version 10 still needs conversion. But ironically it doesn't work on PC-BSD 10.2 in the first place. It can only grow UFS volumes, and PC-BSD's root is not a UFS volume.
Re: nosh version 1.23
Guillermo: So I looked at the source/targets/virtualbox-host.target file from which the bundle is created, and, IMHO, there are some issues with it. That's not surprising. Running as a virtualbox host is not something that is well tested. As you can see from the roadmap WWW page, there are a few things that are hard to test because I simply don't have them in use anywhere, like GELI and AppleTalk. Another of those is a machine hosting virtualbox VMs. Guillermo: But on a more conceptual level, I don't really understand why these 'Wants=' lines are even there. They might well be entirely redundant, as you say. I'll add looking at them afresh to the to-do list. It may well be simply the case that they can be deleted outright from the target, and enabled by presets. That's certainly the design on the running-as-guest side. (There seems to be a slight bug in 80-virtualbox-guest.preset, considering this. Hmmm.)
Re: nosh version 1.23
Guillermo: It seems that the 'unload-when-stopped' and 'version' subcommands of system-control are not mentioned in the documentation. They both should be. I've added the latter to the doco ready for the next version. The former I'm wavering about. I've had a to-do item for some time to turn that into "service-control --exit" (a.k.a. "svc -x"), changing it to use the supervise/control API instead of the service manager's control socket. The service manager would do the same thing. Just the API and the control utilities would change. The problems are (a) I haven't got around to that part of the to-do list yet and (b) the upgrade path is a little tricky and I think would involve more than one release.
nosh version 1.23
The nosh package is now up to version 1.23 . * http://homepage.ntlworld.com./jonathan.deboynepollard/Softwares/nosh.html * https://www.freebsd.org/news/status/report-2015-07-2015-09.html#The-nosh-Project There is one major item in this release. * I've adjusted console-fb-realizer's keyboard handling on BSD to use USB directly. There are a few more minor changes. * I've upgraded the version of clang++ that is used to build the binary packages to 3.8.0. This should have no visible effect whatsoever. (-: * The handling of the DECDA2 control sequence by console-terminal-emulator now copes with what vim sends. (What vim sends isn't what my DEC VT tests had been checking.) * convert-systemd-units now inserts uses of the ionice and chrt chain-loading commands on Linux into the generated service bundles. Mostly this is a clearing the decks release in the hope that I will be able to do some more work on the remaining few FreeBSD conversions before the new year. USB keyboard support The keyboard handling is a change to using the USB HID devices (/dev/uhid*) on FreeBSD in preference to (but not forcibly instead of) the ATA keyboard protocol. In part this is in order to handle the "consumer" keys that USB has. In part this is in order to handle the extra keys that one finds on 106-key, 107-key, and 109-key keyboards and on some numeric keypads (such as the ABNT2 thousands-separator key). In part it's to remove an extra layer of the user-space virtual terminal system that can be outwith the kernel. In part it's to match the USB mouse capability from version 1.22 of the toolset. Please note that the structure of kbdmap files has changed slightly, to accomodate mappings for "consumer" keys, to reposition the entries for some of the 106/107/109-key keyboards' extra keys, and to cover all of the function key gymnastics that vim can accept. The /etc/system-control/convert/ system should automatically re-convert your VT kbd files into the new format.As part of this, I've moved the mapping for the Euro symbol in the fallback U.K. layout (as generated on Linux in the absence of VT kbd files). It used to be level 3 shift on the [eE] key in prior versions of the toolset. Almost all real U.K. keyboards nowadays have it engraved as level 3 shift on the [4$] key, and that's where it now is. Also note that I'm still working on this. There might be further changes in 1.24. I've found a U.K. keyboard with two [#~] keys (at A00 and C12), and I need to check out whether this actually employs what I had thought to be an error in the USB HID usage tables (distinct usages for "\|" and "Europe1") and had corrected, or whether this is a quite mad keyboard that simply has two "Europe1" keys (or two "\|" keys). Also, I've ordered an ABNT2 and a Japanese USB keyboard, and hope to do some testing with them, which may prompt further tweaks. (I really wanted to buy a Leadership 4530 keyboard. They seem to be out of stock in a lot of places.)
Re: nosh: service-dt-scanner gets repeatedly killed by SIGABRT
On 2015-11-05 01:29, Guillermo wrote: So problem solved. Thank you, this is a major improvement. Brill. Thank you for the confirmation.
nosh version 1.22
The nosh package is now up to version 1.22 .
* http://homepage.ntlworld.com./jonathan.deboynepollard/Softwares/nosh.html
*
https://www.freebsd.org/news/status/report-2015-07-2015-09.html#The-nosh-Project
There are several things in this release:
* a new binary package for FreeBSD
* improvements to the user-mode virtual terminal subsystem
* changes and additions to UCSPI tools
* log export to remote servers
* kqueue on Linux
* miscellany
a new binary package for FreeBSD
There's now a debian-shims binary package for FreeBSD. This contains the
heretofore not packaged invoke-rc.d and update-rc.d shim programs. I
decided to make this separate from the systemv-shims package because
these areless general-purpose than those shims are.
The haltsys, fasthalt, fastpoweroff, and fastboot shims are now
packaged, also.
improvements to the user-mode virtual terminal subsystem
The console-fb-realizer now displays a mouse pointer sprite on the
display, to reflect the position of the mouse, when an application has
turned it on with the relevant control sequences. Mouse support via the
evdev input subsystem on Linux is thus now fully implemented, including
support for tablets that use absolute rather than relative positioning.
On the FreeBSD side, you can use sysmouse devices. But this only
permits relative positioning. This is a limitation of sysmouse itself,
as far as I can see. A lot has to change, including the kernel, the
protocol, and moused, to enable absolute positioned devices via
sysmouse. Absolute positioning devices will therefore be supported using
uhid devices. Some of that is done already, but it's not complete yet.
Keyboard maps are now generated by the external configuration import
subsystem from whatever one has in /usr/share/vt/keymaps , rather than
being hardwired to a fixed set of countries. In the absence of this
directory (as will usually be the case on Linux operating systems),
fallback U.S.A. and U.K. keyboard maps are generated.
This generation is worthy of note, as it exemplifies the mechanism that
allows multiple BSD keyboard maps to be overlaid to make one generated
map. The fallback U.K. keyboard map is generated by taking the built-in
U.S.A. keyboard map and applying a "us_to_uk" overlay map on top of it
that only has the few differences between the U.S.A. International
layout and the U.K. one. (This currently produces the basic U.K.
layout. "U.K. Extended" should be a simple matter of another overlay
that does the various Option+A -> a-acute mappings, but that's
somethingfor the future.) Similarly, versions of existing maps that
swap Caps Lock and Control are generated by adding a simple overlay that
does solely that. Likewise, generated maps have an overlay applied that
sets the Backspace key to the application-programmable DEC VT behaviour
that console-terminal-emulator supports, that out-of-the-box BSD keymaps
don't know anything about.
changes and additions to UCSPI tools
For consistency, the UCSPI tools that supported a single --numeric
option now support --numeric-host and --numeric-port options, for
separately determining whether hosts and ports are taken to be names or
just numbers.
There are now client-side tcp-socket-connect and udp-socket-connect
tools, that open client sockets, connect them to servers, and then
chain. These adhere to the UCSPI conventions for inherited open file
descriptors in client-side tools.
log export to remote servers
The new UCSPI clients were motivated by the new export-to-rsyslog
command. This is a daemon that expects to be invoked as a UCSPI client,
connected to a remote RFC 5424/RFC 5426 ("rsyslog") server. It
maintains a set of "log cursors" that point to daemontools-stylelog
directories. Tracking its position in the logs using those cursors, it
sends new log information to the connected server. In the usual nosh
fashion, the filesystem is the database, and the "cursors" are just
files and symbolic links. The details are on the manual page.
In conjunction with the UCSPI clients, export-to-rsyslog thus makes a
log remote export service.
This isn't intended to be the last word in such things. RFC 5426is
unreliable, and RFC 5424 loses the microsecond and nanosecond
information of TAI64N. But it demonstrates the idea and shows that this
can be done in the daemontools world. One can indeed export
daemontools-stylelogsif one has (say) a suite of servers whose log data
should be copied over, on the fly, to a centralized rsyslog server.
There's room here for someone to take this idea and run further with it
using something like RELP.
miscellany
--
The several miscellaneous items include OpenLDAP services in the
autoconfiguration subsystem and some tweaks to the /etc/fstab conversion
on Linux to de
