Hi Russ, On Thu, Dec 19, 2013 at 11:19:37AM -0800, Russ Allbery wrote: > Could someone (Steve, most likely) provide a bit of background for how > upstart upstream maintenance works and relates to its packaging?
> This question is prompted by a few different things, set off by looking at > the SELinux support since this is something I expect to start looking at > for my day job. From there, I found that the SELinux context support in > upstart is somewhat limited, but more interestingly is being maintained as > a patch in the Debian package. (But maybe that's because the Debian > packaging is one version behind?) > I then looked at the Ubuntu patch for upstart and was rather surprised to > find that it's quite large (although a lot of that is regeneration of the > Autotools files -- can I recommend dh-autoreconf?). There appear to be > substantive code changes in Ubuntu's packaging of upstart that aren't > upstream, which surprised me. > How does this all work? How do changes flow from Debian and Ubuntu > packaging into upstream, and why would the packaging be carrying > substantial local patches for software that's maintained upstream by (at > least as I understand it) the same project? Is there a separate policy > about what goes into upstream that precludes things that aren't considered > fully baked? I'm attaching the full delta against upstream currently in the Ubuntu packaging VCS branch, for reference. FWIW, I'm not sure which version you were looking at, but the current version of the Ubuntu package (1.11-0ubuntu1) does not include any delta against autogenerated autotools files (and does use dh-autoreconf). Also FWIW, the attached patch includes changes not yet released to Ubuntu (actually, just an update to sync the packaging with the version in Debian). The attached delta is generated from: bzr diff -pa/:b/ -r tag:upstream-1.11 lp:ubuntu/upstart | filterdiff -x '**/debian/**' plus a bit of postprocessing. I've dissected the current delta and provided an explanation below for each bit, but the short answer to your question is yes: there are different policies for upstream vs. the Ubuntu package. Although efforts are made to keep the distro delta as small as possible, changes are sometimes applied to the package before they're in a state that they can be included in upstream in the interest of expediency. As for Upstart and Ubuntu being maintained "by the same project": if Upstart were intended to be "Ubuntu's init system", it would be reasonable to do all of the maintenance on a single upstream branch. But it was never intended to be Ubuntu's init system, but rather "the init system", and as there are other downstreams besides Ubuntu, care is taken to not embed Ubuntu-specific policy upstream. So while there is some delta between Ubuntu and upstream right now, the delta between Debian and Ubuntu packages has been the greater focus of attention and has posed the greater maintenance challenge while trying to converge Ubuntu's packaging (which made reasonable assumptions of Upstart-everywhere) with Debian's (which did not). Now for the current delta, the changes here consist of a few different things: - Changes to core jobs for integration with Debian/Ubuntu. E.g., conf/rc-sysinit.conf is changed to refer to 'filesystem' and 'static-network-up' events that are provided by components external to upstart (mountall and ifupdown respectively). They are deemed inappropriate for upstream, because upstream upstart shouldn't have dependencies on distro-specific implementations. Likewise, conf/rc.conf has 'emits' lines added for documentation, which are only true when using a version of initscripts that includes upstart integration. - References to the Debian/Ubuntu-specific upstart-events(7) manpage, which describes event policy as it exists here. Not upstreamable for the same reason as the job changes; none of this applies to other users of upstart (e.g.: RHEL6, Chrome OS). - SELinux support. This was accepted as a distro patch in Debian, but has not been submitted under Canonical's contributor licensing agreement, so at present is not suitable for inclusion upstream per the upstream contribution policy. We are happy to carry such patches in the Debian/Ubuntu packaging delta where appropriate, though of course would prefer that such changes be mergeable upstream. - A minor build fix for configure.ac, to make the test suite function correctly under make -j. This is needed on systems with newer automake, and incompatible with systems with older automake, so carried as a distro patch for now. - A patch to disable apparmor in containers and liveCDs. This patch is from a member of the Ubuntu security team. I'm not sure why it's not upstreamed, but it's a minor patch and not at the top of my todo list to resolve. - A patch to work around <https://bugs.launchpad.net/ubuntu/+source/upstart/+bug/1058356>, which I believe will be dropped after the next Ubuntu LTS release (14.04). - Changes to the default console handling, which were validated for Ubuntu but no one has had a chance to work through yet upstream to confirm whether they should be applied as is. - A hack to import pids of processes started in the initramfs (used for plymouth). Scott apparently never felt this was good enough for inclusion upstream because it encoded too much policy, and we haven't really revisited because it hasn't been a maintenance burden. - Various fixes to test cases that were failing in Debian builds. These fixes have been pushed to trunk and will be included in the next upstream release. Aside: the full upstream delta is 441 lines. So this message is 25% of the size of the delta that it describes. ;) -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developer http://www.debian.org/ slanga...@ubuntu.com vor...@debian.org
=== modified file 'conf/rc-sysinit.conf'
--- a/conf/rc-sysinit.conf 2013-04-09 06:45:29 +0000
+++ b/conf/rc-sysinit.conf 2013-09-13 00:06:35 +0000
@@ -6,7 +6,7 @@
description "System V initialisation compatibility"
author "Scott James Remnant <sc...@netsplit.com>"
-start on startup
+start on (filesystem and static-network-up) or failsafe-boot
stop on runlevel
# Default runlevel, this may be overriden on the kernel command-line
@@ -23,9 +23,11 @@
env RUNLEVEL=
env PREVLEVEL=
+console output
+env INIT_VERBOSE
+
task
-console owner
script
# Check for default runlevel in /etc/inittab
if [ -r /etc/inittab ]
=== modified file 'conf/rc.conf'
--- a/conf/rc.conf 2013-04-09 06:45:29 +0000
+++ b/conf/rc.conf 2013-09-13 00:06:35 +0000
@@ -6,13 +6,18 @@
description "System V runlevel compatibility"
author "Scott James Remnant <sc...@netsplit.com>"
+emits deconfiguring-networking
+emits unmounted-remote-filesystems
+
start on runlevel [0123456]
stop on runlevel [!$RUNLEVEL]
export RUNLEVEL
export PREVLEVEL
+console output
+env INIT_VERBOSE
+
task
-console output
exec /etc/init.d/rc $RUNLEVEL
=== modified file 'configure.ac'
--- a/configure.ac 2013-11-14 16:48:22 +0000
+++ b/configure.ac 2013-11-30 07:02:17 +0000
@@ -8,7 +8,7 @@
AC_GNU_SOURCE
-AM_INIT_AUTOMAKE([1.10 gnu nostdinc check-news color-tests silent-rules])
+AM_INIT_AUTOMAKE([1.10 gnu nostdinc check-news color-tests silent-rules serial-tests])
AM_SILENT_RULES([yes])
AM_MAINTAINER_MODE([enable])
@@ -109,6 +109,15 @@
AM_CONDITIONAL([HAVE_ABI_CHECKER], [test ! -z "$ABI_COMPLIANCE_CHECKER" && test -e "$srcdir"/lib/abi/"$host_cpu"-"$host_os"/*.abi])
+AC_ARG_ENABLE(selinux,
+ AS_HELP_STRING([--enable-selinux], [enable SELinux support]),
+ [], [enable_selinux=no])
+
+if test "x$enable_selinux" = "xyes" ; then
+ PKG_CHECK_MODULES(SELINUX, [libselinux])
+ AC_DEFINE(HAVE_SELINUX, 1, [Define if we have SELinux])
+fi
+
# Checks for header files.
AC_CHECK_HEADERS([valgrind/valgrind.h, sys/prctl.h])
=== modified file 'init/Makefile.am'
--- a/init/Makefile.am 2013-11-14 16:48:22 +0000
+++ b/init/Makefile.am 2013-11-30 06:23:08 +0000
@@ -8,6 +8,7 @@
$(NIH_CFLAGS) \
$(NIH_DBUS_CFLAGS) \
$(DBUS_CFLAGS) \
+ $(SELINUX_CFLAGS) \
$(JSON_CFLAGS)
AM_CPPFLAGS = \
@@ -70,6 +71,7 @@
$(NIH_LIBS) \
$(NIH_DBUS_LIBS) \
$(DBUS_LIBS) \
+ $(SELINUX_LIBS) \
$(JSON_LIBS) \
-lrt
=== modified file 'init/apparmor.c'
--- a/init/apparmor.c 2013-05-24 17:21:47 +0000
+++ b/init/apparmor.c 2013-07-18 14:55:33 +0000
@@ -111,6 +111,18 @@
return FALSE;
}
+ /* Do not load if running in a container.
+ * This is a distro-specific patch.
+ */
+ if (stat ("/run/container_type", &statbuf) == 0)
+ return FALSE;
+
+ /* Do not load if running in a Ubuntu live CD.
+ * This is a distro-specific patch.
+ */
+ if (stat ("/rofs/etc/apparmor.d", &statbuf) == 0)
+ return FALSE;
+
return TRUE;
}
=== modified file 'init/job_process.c'
--- a/init/job_process.c 2013-11-14 16:48:22 +0000
+++ b/init/job_process.c 2013-11-30 06:23:08 +0000
@@ -2,7 +2,7 @@
*
* job_process.c - job process handling
*
- * Copyright © 2011 Canonical Ltd.
+ * Copyright 2011 Canonical Ltd.
* Author: Scott James Remnant <sc...@netsplit.com>.
*
* This program is free software; you can redistribute it and/or modify
@@ -1677,10 +1677,18 @@
/* We should always fail the job if the security profile
* failed to load
*/
+
+ /* Disabled for now to emulate current Ubuntu behaviour
+ * in /lib/init/apparmor-profile-load to work around bug
+ * LP: #1058356
+
if (status) {
failed = TRUE;
stop = TRUE;
}
+
+ */
+
break;
case PROCESS_PRE_START:
nih_assert (job->state == JOB_PRE_START);
=== modified file 'init/main.c'
--- a/init/main.c 2013-11-14 16:48:22 +0000
+++ b/init/main.c 2013-11-30 06:23:08 +0000
@@ -21,7 +21,6 @@
# include <config.h>
#endif /* HAVE_CONFIG_H */
-
#include <sys/types.h>
#include <sys/time.h>
#include <sys/wait.h>
@@ -39,6 +38,7 @@
#include <errno.h>
#include <stdio.h>
+#include <dirent.h>
#include <limits.h>
#include <signal.h>
#include <stdlib.h>
@@ -46,6 +46,10 @@
#include <syslog.h>
#include <unistd.h>
+#ifdef HAVE_SELINUX
+#include <selinux/selinux.h>
+#endif
+
#include <linux/kd.h>
#include <nih/macros.h>
@@ -202,6 +206,25 @@
{
char **args = NULL;
int ret;
+#ifdef HAVE_SELINUX
+ int enforce = 0;
+
+ if (getenv ("SELINUX_INIT") == NULL) {
+ putenv ("SELINUX_INIT=YES");
+ if (selinux_init_load_policy (&enforce) == 0 ) {
+ execv (argv[0], argv);
+ } else {
+ if (enforce > 0) {
+ /* SELinux in enforcing mode but load_policy
+ * failed. At this point, we probably can't
+ * open /dev/console, so log() won't work.
+ */
+ fprintf (stderr, "Unable to load SELinux Policy. Machine is in enforcing mode. Halting now.\n");
+ exit (1);
+ }
+ }
+ }
+#endif /* HAVE_SELINUX */
conf_dirs = NIH_MUST (nih_str_array_new (NULL));
@@ -336,23 +359,16 @@
* resetting it to sane defaults unless we're inheriting from another
* init process which we know left it in a sane state.
*/
- if (system_setup_console (CONSOLE_OUTPUT, (! restart)) < 0) {
+ if (system_setup_console (CONSOLE_NONE, (! restart)) < 0) {
NihError *err;
err = nih_error_get ();
- nih_warn ("%s: %s", _("Unable to initialize console, will try /dev/null"),
- err->message);
+ nih_fatal ("%s: %s", _("Unable to initialize console as /dev/null"),
+ err->message);
nih_free (err);
- if (system_setup_console (CONSOLE_NONE, FALSE) < 0) {
- err = nih_error_get ();
- nih_fatal ("%s: %s", _("Unable to initialize console as /dev/null"),
- err->message);
- nih_free (err);
-
- exit (1);
- }
+ exit (1);
}
/* Set the PATH environment variable */
@@ -661,6 +677,16 @@
* init daemon that exec'd us
*/
if (! restart) {
+ DIR *piddir;
+
+ /* Look in well-known locations for pid files.
+ *
+ * Try /run (the newer) location first, but fall back to
+ * the original location for older systems.
+ */
+ const char * const pid_paths[] = { "/run/initramfs/", "/dev/.initramfs/", NULL };
+ const char * const *pid_path;
+
if (disable_startup_event) {
nih_debug ("Startup event disabled");
} else {
@@ -669,6 +695,68 @@
? initial_event
: STARTUP_EVENT,
NULL));
+ }
+
+ for (pid_path = pid_paths; pid_path && *pid_path; pid_path++) {
+ struct dirent *ent;
+
+ /* Total hack, look for .pid files in known
+ * locations - if there's a job config for them pretend
+ * that we started it and it has that pid.
+ */
+ piddir = opendir (*pid_path);
+ if (! piddir)
+ continue;
+
+ while ((ent = readdir (piddir)) != NULL) {
+ char path[PATH_MAX];
+ char * ptr;
+ FILE * pidfile;
+ pid_t pid;
+ JobClass *class;
+ Job * job;
+
+ if (ent->d_name[0] == '.')
+ continue;
+
+ strcpy (path, *pid_path);
+ strcat (path, ent->d_name);
+
+ ptr = strrchr (ent->d_name, '.');
+ if ((! ptr) || strcmp (ptr, ".pid"))
+ continue;
+
+ *ptr = '\0';
+ pidfile = fopen (path, "r");
+ if (! pidfile)
+ continue;
+
+ pid = -1;
+ if (fscanf (pidfile, "%d", &pid))
+ ;
+ fclose (pidfile);
+
+ if ((pid < 0) || (kill (pid, 0) < 0))
+ continue;
+
+ class = (JobClass *)nih_hash_lookup (job_classes, ent->d_name);
+ if (! class)
+ continue;
+ if (! class->process[PROCESS_MAIN])
+ continue;
+ if (strlen (class->instance))
+ continue;
+
+ job = NIH_MUST (job_new (class, ""));
+ job->goal = JOB_START;
+ job->state = JOB_RUNNING;
+ job->pid[PROCESS_MAIN] = pid;
+
+ nih_debug ("%s inherited from initramfs with pid %d", class->name, pid);
+ }
+
+ closedir (piddir);
+ break;
}
} else {
=== modified file 'init/man/init.5'
--- a/init/man/init.5 2013-08-27 08:08:20 +0000
+++ b/init/man/init.5 2013-09-11 16:35:08 +0000
@@ -1116,4 +1116,5 @@
.BR prctl (2)
.BR pty (7)
.BR sh (1)
+.BR upstart-events (7)
.BR apparmor (7)
=== modified file 'init/man/init.8'
--- a/init/man/init.8 2013-11-14 16:48:22 +0000
+++ b/init/man/init.8 2013-11-30 06:23:08 +0000
@@ -48,6 +48,10 @@
and
.BR stopped (7)
events emitted as jobs change state.
+
+See
+.BR upstart-events (7)
+for a summary of well-known events.
.\"
.SS System V compatibility
The Upstart
@@ -231,3 +235,4 @@
.BR stopping (7)
.BR stopped (7)
.BR telinit (8)
+.BR upstart-events (7)
=== modified file 'init/tests/test_job_process.c'
--- a/init/tests/test_job_process.c 2013-11-14 16:48:22 +0000
+++ b/init/tests/test_job_process.c 2013-11-30 06:23:08 +0000
@@ -8161,6 +8161,7 @@
class->expect = EXPECT_STOP;
TEST_ALLOC_FAIL {
+ pid_t tmp_pid;
TEST_ALLOC_SAFE {
job = job_new (class, "");
@@ -8174,7 +8175,10 @@
exit (0);
}
- assert0 (waitid (P_PID, pid, &info, WSTOPPED | WNOWAIT));
+ tmp_pid = waitid (P_PID, pid, &info, WSTOPPED | WNOWAIT);
+ if (tmp_pid < 0)
+ printf("waitid failed: %m");
+ assert0 (tmp_pid);
job->goal = JOB_START;
job->state = JOB_SPAWNED;
=== modified file 'init/tests/test_log.c'
--- a/init/tests/test_log.c 2013-06-28 15:36:50 +0000
+++ b/init/tests/test_log.c 2013-11-30 06:23:08 +0000
@@ -1136,6 +1136,8 @@
int pty_master;
int pty_slave;
int found_fd;
+ char filename[1024];
+ int fd;
TEST_FUNCTION ("log_destroy");
@@ -1179,7 +1181,16 @@
TEST_EQ (openpty (&pty_master, &pty_slave, NULL, NULL, NULL), 0);
- log = log_new (NULL, "/bar", pty_master, 0);
+ TEST_FILENAME (filename);
+
+ /* Make file inaccessible to ensure data cannot be written
+ * and will thus be added to the unflushed buffer.
+ */
+ fd = open (filename, O_CREAT | O_EXCL, 0);
+ TEST_NE (fd, -1);
+ close (fd);
+
+ log = log_new (NULL, filename, pty_master, 0);
TEST_NE_P (log, NULL);
ret = write (pty_slave, str, strlen (str));
@@ -1200,7 +1211,7 @@
TEST_EQ (openpty (&pty_master, &pty_slave, NULL, NULL, NULL), 0);
- log = log_new (NULL, "/bar", pty_master, 0);
+ log = log_new (NULL, filename, pty_master, 0);
TEST_NE_P (log, NULL);
found_fd = 0;
@@ -1238,6 +1249,8 @@
}
}
+ TEST_EQ (unlink (filename), 0);
+
/* Freeing the log object should have removed the watch */
TEST_EQ (found_fd, 0);
=== modified file 'init/tests/test_state.c'
--- a/init/tests/test_state.c 2013-11-13 14:25:38 +0000
+++ b/init/tests/test_state.c 2013-11-30 06:23:08 +0000
@@ -4503,7 +4503,7 @@
NIH_HASH_FOREACH (job_classes, iter) {
JobClass *class = (JobClass *)iter;
if (strcmp (class->name, "whoopsie") == 0) {
- TEST_EQ (class->reload_signal, SIGUSR1);
+ TEST_EQ (class->reload_signal, 10);
} else
TEST_EQ (class->reload_signal, SIGHUP);
}
signature.asc
Description: Digital signature
