Bug#911806: Please provide a way to opt out of AppArmor confinement when running tests
> 1. Ask the src:lxc maintainers to apply these 3 upstream patches >until they upgrade the package to 3.1.0+. #916639 > 2. Ask the debci maintainers to use the config described above >for LXC containers used to run autopkgtests, once they upgrade >to Buster. #916644
Bug#911806: Please provide a way to opt out of AppArmor confinement when running tests
Hi,
Michael Biebl:
> $ ./make-dsc
>
It took me a while to get back to this thread; obviously upstream
master has moved on since you wrote this, so:
I had to update Use-Debian-specific-config-files.patch to make it
apply and use helper functions in a way that's compatible with their
current API. Updated patch attached, in case it may save you
some time.
I've also added two patches to BLACKLIST (refreshing them seemed
non-trivial as I'm not a C developer):
Revert-udev-network-device-renaming-immediately-give.patch
fsckd-daemon-for-inter-fsckd-communication.patch
> lxc.aa_profile = unconfined
FTR with LXC 3.x that's now:
lxc.apparmor.profile = unconfined
> There are currently two, known failures with AA turned off:
> - dnsmasq 2.80 introduced a regression in networkd-test.py
> - test-bpf failing
Confirmed with lxc 1:3.0.3-1 from sid.
> With AA turned on, the list of failing tests is too long to list here.
Confirmed with lxc 1:3.0.3-1 from sid: the exact same tests pass/fail
as in your logs. I see lots of mount operations denied by AppArmor.
That's expected because the AppArmor vs. systemd fixes are in LXC
3.1.0, that was released a few days ago, but not in LXC 3.0.3.
Then I've cherry-picked on top of lxc 3.0.3-1 the 3 upstream commits
that implement the nested containers vs. AppArmor fixes; and in the
config of the LXC container I use for autopkgtests I've set:
lxc.apparmor.profile = generated
lxc.apparmor.allow_nesting = 1
With this configuration, the systemd autopkgtests work just as well
as unconfined \o/
My current plan is thus:
1. Ask the src:lxc maintainers to apply these 3 upstream patches
until they upgrade the package to 3.1.0+.
2. Ask the debci maintainers to use the config described above
for LXC containers used to run autopkgtests, once they upgrade
to Buster.
3. Let you decide what to do with the request this bug report was
originally about.
> Hope this helpful.
This was *very* helpful and saved me lots of time :)
Thanks for your patience,
cheers,
--
intrigeri
>From 035ef2dcad71f449835c41387672ed680fc54dac Mon Sep 17 00:00:00 2001
From: Michael Biebl
Date: Thu, 18 Jul 2013 20:11:02 +0200
Subject: [PATCH 01/19] Use Debian specific config files
Use /etc/default/locale instead of /etc/locale.conf for locale settings.
Use /etc/default/keyboard instead of /etc/X11/xorg.conf.d/00-keyboard.conf for
keyboard configuration.
Read/write /etc/timezone if /etc/localtime does not exist.
Gbp-Pq: Topic debian
Gbp-Pq: Name Use-Debian-specific-config-files.patch
---
src/basic/time-util.c| 21 +++-
src/core/locale-setup.c | 22
src/locale/keymap-util.c | 210 +++
src/timedate/timedated.c | 10 ++
4 files changed, 156 insertions(+), 107 deletions(-)
diff --git a/src/basic/time-util.c b/src/basic/time-util.c
index 557c75debc..ea5f6a4491 100644
--- a/src/basic/time-util.c
+++ b/src/basic/time-util.c
@@ -1382,8 +1382,25 @@ int get_timezone(char **tz) {
int r;
r = readlink_malloc("/etc/localtime", &t);
-if (r < 0)
-return r; /* returns EINVAL if not a symlink */
+if (r < 0) {
+if (r != -EINVAL)
+return r; /* returns EINVAL if not a symlink */
+
+r = read_one_line_file("/etc/timezone", &t);
+if (r < 0) {
+if (r != -ENOENT)
+log_warning_errno(r, "Failed to read /etc/timezone: %m");
+return -EINVAL;
+}
+
+if (!timezone_is_valid(t, LOG_DEBUG))
+return -EINVAL;
+z = strdup(t);
+if (!z)
+return -ENOMEM;
+*tz = z;
+return 0;
+}
e = PATH_STARTSWITH_SET(t, "/usr/share/zoneinfo/", "../usr/share/zoneinfo/");
if (!e)
diff --git a/src/core/locale-setup.c b/src/core/locale-setup.c
index 584fb220a1..3d3cb641de 100644
--- a/src/core/locale-setup.c
+++ b/src/core/locale-setup.c
@@ -59,6 +59,28 @@ int locale_setup(char ***environment) {
log_warning_errno(r, "Failed to read /etc/locale.conf: %m");
}
+ if (r <= 0) {
+ r = parse_env_file(NULL, "/etc/default/locale",
+ "LANG", &variables[VARIABLE_LANG],
+ "LANGUAGE", &variables[VARIABLE_LANGUAGE],
+ "LC_CTYPE", &variables[VARIABLE_LC_CTYPE],
+ "LC_NUMERIC",&variables[VARIABLE_LC_NUMERIC],
+ "LC_TIME", &variables[VARIABLE_LC_TIME],
+ "LC_COLLATE",&variables[VARIABLE_LC_COLLATE],
+ "LC_MONETARY", &variables[VARIABLE_LC_MONETARY],
+ "LC_MESSAGES",
Bug#911806: Please provide a way to opt out of AppArmor confinement when running tests
Hi! Am 27.10.18 um 10:20 schrieb intrigeri: > So on the LXC + AppArmor vs. systemd v240+ front, I think the next > steps are: > > 1. Try running current systemd master branch and its autopkgtests >inside a container managed by LXC 2.x on current testing/sid. >Goal: confirm the issues Michael discovered and have a baseline >to evaluate LXC 3.x against. > > 2. Try running current systemd master branch and its autopkgtests >inside a container managed by LXC 3.x on current testing/sid. >Report any issue so they're fixed before 3.x becomes stable >and is hopefully included in Buster. In case you want to reproduce the failures, these are the rough steps I used on Debian sid 1/ Install lxc # apt install lxc 2/ Setup /etc/default/lxc-net: # cat /etc/default/lxc-net USE_LXC_BRIDGE="true" LXC_BRIDGE="lxcbr0" LXC_ADDR="10.0.3.1" LXC_NETMASK="255.255.255.0" LXC_NETWORK="10.0.3.0/24" LXC_DHCP_RANGE="10.0.3.2,10.0.3.254" LXC_DHCP_MAX="253" # cat /etc/lxc/default.conf lxc.network.type = veth lxc.network.link = lxcbr0 lxc.network.flags = up # systemctl restart lxc-net 3/ Install autopkgtest and create an LXC container for autopkgtest: # apt install autopkgtest # autopkgtest-build-lxc debian sid 4/ Clone the systemd trunk-ci scripts: $ git clone https://salsa.debian.org/systemd-team/trunk-ci.git 5/ Create a dsc $ ./make-dsc 6/ Build the package using dpkg-buildpackage, pbuilder, sbuild, your favourite tool. You want to specify the "noudeb" profile, to speed up the build and avoid building udebs. 7/ Run autopkgtest via LXC # autopkgtest -o logs *.dsc *.deb -- lxc -s autopkgtest-sid In case you want to run the LXC container with AA turned off, add the following to /var/lib/lxc/autopkgtest-sid/config lxc.aa_profile = unconfined There are currently two, known failures with AA turned off: - dnsmasq 2.80 introduced a regression in networkd-test.py - test-bpf failing With AA turned on, the list of failing tests is too long to list here. I've made two runs of current systemd git master with AA turned on and off. See https://people.debian.org/~biebl/lxc/log-confined.txt https://people.debian.org/~biebl/lxc/log-unconfined.txt Hope this helpful. Regards, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? signature.asc Description: OpenPGP digital signature
Bug#911806: Please provide a way to opt out of AppArmor confinement when running tests
Hi LXC maintainers, intrigeri: >Ideally, picking the best strategy and implementing it would be >a matter of coordination between LXC and systemd (ideally upstream, >but quite possibly distro maintainers will need to be involved >here). I'll raise this issue to the Ubuntu LXC and AppArmor folks. Done. To sum up: - LX*D* apparently has the support needed to set up AppArmor policy in a way that should not be affected by the problem this thread is about (running systemd v240+ in a container). autopkgtest has a LXD backend but LXD is not in Debian yet (WIP, see #768073; I'm not counting on this being completed in time for the Buster freeze). - Similar support was added to LXC 3.x branch. It won't be backported to 2.x (that we currently have in testing/sid). I've asked about the timeline to release 3.x as stable and Stéphane Graber replied: "I think we were aiming towards February-ish originally but there's no real reason to wait that long either, so if Christian isn't waiting for some big changes to land before doing a non-LTS feature release, we should be able to tag one next month. It'd be worth someone make sure that current master with the apparmor work that was done by Wolfgang will do the right thing out of the box though, otherwise that wouldn't really achieve a whole lot." So on the LXC + AppArmor vs. systemd v240+ front, I think the next steps are: 1. Try running current systemd master branch and its autopkgtests inside a container managed by LXC 2.x on current testing/sid. Goal: confirm the issues Michael discovered and have a baseline to evaluate LXC 3.x against. 2. Try running current systemd master branch and its autopkgtests inside a container managed by LXC 3.x on current testing/sid. Report any issue so they're fixed before 3.x becomes stable and is hopefully included in Buster. For details, see: https://lists.ubuntu.com/archives/apparmor/2018-October/011830.html … except Stéphane Graber's messages are apparently held for moderation so they don't appear in the list archives yet. Cheers, -- intrigeri
Bug#911806: Please provide a way to opt out of AppArmor confinement when running tests
Hi, Michael Biebl: > E.g. if I try to run autopkgtest for systemd on sid in a AA-confined LXC > container, it explodes left and right, whereas it passes with > AA-confinement turned off. Thanks for reporting this! Disclaimer: I don't use LXC. I've spent some time reading the various related systemd upstream issues and commits and my understanding is that we have two problems: 1. Recent systemd sets up namespacing with calls to mount that are not allowed under the default AppArmor policy shipped in the lxc package (or rather: it has always done so but previously, failures to set up said namespacing were non-fatal). If I got it right, this means that lots of services will be broken when run in LXC once systemd v240 is released and uploaded to Debian, i.e. the problem is broader than our CI: most typical LXC use cases will be affected as well. It's an instance of a classical problem when nesting security systems: - Either the outside one (in this case, AppArmor) allows the inside one (in this case, systemd) whatever it takes so the latter can do its job. Ideally this cooperation gives us the best of both worlds. But in practice, this sometimes requires the outside security system to give permissions that are so relaxed that it becomes essentially useless (which might be the case here, I don't know); when this happens, we basically can't rely on the outside security system anymore, and instead we rely mostly on the inside one. When the inside security system is controlled by untrusted parties, such as an untrusted container's root user, that's a risky approach, to say the least. In other words, there's a risk to break isolation between containers in order to improve isolation between services inside a given container. - Or the outside security system is restrictive to the point that it breaks the inside one, and then it's the opposite: weaker service isolation inside the container but stronger isolation between containers. Ideally, picking the best strategy and implementing it would be a matter of coordination between LXC and systemd (ideally upstream, but quite possibly distro maintainers will need to be involved here). I'll raise this issue to the Ubuntu LXC and AppArmor folks. Worst case, if this does not get resolved in time for Buster, we may have to disable AppArmor confinement for LXC containers in Buster; that would be sad but it may be the best we can do for Buster, and it would not be a regression vs. Stretch for most users (the Stretch kernel lacks the AppArmor features that LXC needs for meaningful confinement so even if one opts in for AppArmor on Stretch, they get pretty weak isolation). And then whoever particularly cares about hardening LXC on Debian has a couple more years to sort this out for Bullseye :) 2. Regardless of #1, some software is broken when run inside a container, be it due to the way it the container technology uses Linux namespaces or due to additional LSM-based sandboxing. In this case the best approach is probably to run these tests inside a KVM runner instead. During the autopkgtest at last DebConf, with another similar issue in mind, I've asked about KVM runners for ci.debian.net and I was told this was WIP and would have it soonish, which will provide a tool to solve this class of problems: any affected autopkgtest can declare the isolation-machine restriction. A special sub-case is when the software would run just fine in a container but its test suite exercises it in a way that requires different privileges than normal usage of said software inside a container, e.g. by setting up stuff in super custom way, which might break under the default sandboxing rules. In this case, either I would argue that the test suite should instead exercise something closer to what actual users will run under that confinement: that's a guiding design principle of autopkgtests. Whenever fixing such a test suite is not possible for some reason, then running the affected tests in KVM could be an acceptable workaround. > It is my understanding, that Ubuntu itself turns off AA-confinement in > their LXC based CI or at least ship a AA policy which is much less > strict [1]. If I got it right, that's correct for armhf but for every other architecture, they run tests in standard Ubuntu KVM cloud instances and not in LXC. Advantage: apart of armhf, no need to deal with the peculiarities of LXC+AppArmor, and AppArmor-wise they're testing exactly what users will experience outside of containers. Drawback: to exercise software that would be broken under the default LXC+AppArmor sandboxing, they need to relax the AppArmor sandboxing for _all_ tests on armhf, meaning they don't test software in exactly the same sandbox that will be used by actual users of Ubuntu LXC co
Bug#911806: Please provide a way to opt out of AppArmor confinement when running tests
Source: debci Severity: wishlist Hi, atm, debci runs on stable, i.e. AppArmor is not active by default. Once buster is released, this is likely going to change. autopktest with lxc as backend and AppArmor confinement enabled is problematic atm, at least for certain packages. E.g. if I try to run autopkgtest for systemd on sid in a AA-confined LXC container, it explodes left and right, whereas it passes with AA-confinement turned off. It is my understanding, that Ubuntu itself turns off AA-confinement in their LXC based CI or at least ship a AA policy which is much less strict [1]. Looking ahead at the buster release, we either need to turn off AA confinement in our Debian CI as well, ship a custom AA policy or provide a way for individual packages to opt out of AA confinement. In the latter case, this will probably need some changes to autopkgtest as well. Would welcome your input on this. Regards, Michael [1] https://github.com/systemd/systemd/issues/10166#issuecomment-428523883 -- System Information: Debian Release: buster/sid APT prefers unstable APT policy: (500, 'unstable'), (500, 'testing'), (200, 'experimental') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 4.18.0-2-amd64 (SMP w/4 CPU cores) Locale: LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8), LANGUAGE=de_DE.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled
