[PATCH] capabilities: add capability cgroup controller
There are many basic ways to control processes, including capabilities,
cgroups and resource limits. However, there are far fewer ways to find
out useful values for the limits, except blind trial and error.
Currently, there is no way to know which capabilities are actually used.
Even the source code is only implicit, in-depth knowledge of each
capability must be used when analyzing a program to judge which
capabilities the program will exercise.
Add a new cgroup controller for monitoring of capabilities
in the cgroup.
Test case demonstrating basic capability monitoring and how the
capabilities are combined at next level (boot to rdshell):
(initramfs) cd /sys/fs
(initramfs) mount -t cgroup2 cgroup cgroup
(initramfs) cd cgroup
(initramfs) echo +capability > cgroup.subtree_control
(initramfs) mkdir test; cd test
(initramfs) echo +capability > cgroup.subtree_control
(initramfs) ls
capability.used cgroup.events cgroup.subtree_control
cgroup.controllers cgroup.procs
(initramfs) mkdir first second
(initramfs) sh
BusyBox v1.22.1 (Debian 1:1.22.0-19) built-in shell (ash)
Enter 'help' for a list of built-in commands.
(initramfs) cd first
(initramfs) echo $$ >cgroup.procs
(initramfs) cat capability.used
# nothing so far
(initramfs) mknod /dev/z_$$ c 1 2
(initramfs) cat capability.used
0800 # CAP_MKNOD
(initramfs) cat ../capability.used
0800 # also seen at next higher level
(initramfs) exit
(initramfs) sh
BusyBox v1.22.1 (Debian 1:1.22.0-19) built-in shell (ash)
Enter 'help' for a list of built-in commands.
(initramfs) cd second
(initramfs) echo $$ >cgroup.procs
(initramfs) cat capability.used
# nothing so far
(initramfs) chown 1234 /dev/z_*
(initramfs) cat capability.used
0001 # CAP_CHROOT
(initramfs) cat ../capability.used
0801 # combined at next higher level
(initramfs) exit
Signed-off-by: Topi Miettinen
---
Documentation/cgroup-v2.txt | 17 +++
include/linux/capability_cgroup.h | 7 +++
include/linux/cgroup_subsys.h | 4 ++
init/Kconfig | 6 +++
kernel/capability.c | 2 +
security/Makefile | 1 +
security/capability_cgroup.c | 99 +++
7 files changed, 136 insertions(+)
create mode 100644 include/linux/capability_cgroup.h
create mode 100644 security/capability_cgroup.c
diff --git a/Documentation/cgroup-v2.txt b/Documentation/cgroup-v2.txt
index 4cc07ce..2b3d277 100644
--- a/Documentation/cgroup-v2.txt
+++ b/Documentation/cgroup-v2.txt
@@ -1118,6 +1118,23 @@ writeback as follows.
total available memory and applied the same way as
vm.dirty[_background]_ratio.
+5-4. Capabilities
+
+The "capability" controller is used to monitor capability use in the
+cgroup. This can be used to discover a starting point for capability
+bounding sets, even when running a shell script under ambient
+capabilities, with only short-lived helper processes exercising the
+capabilities.
+
+
+5-4-1. Capability Interface Files
+
+ capability.used
+
+ A read-only file which exists on all cgroups.
+
+ This reports the combined value of capability use in the
+ current cgroup and all its children.
6. Namespace
diff --git a/include/linux/capability_cgroup.h
b/include/linux/capability_cgroup.h
new file mode 100644
index 000..c03b58d
--- /dev/null
+++ b/include/linux/capability_cgroup.h
@@ -0,0 +1,7 @@
+#ifdef CONFIG_CGROUP_CAPABILITY
+void capability_cgroup_update_used(int cap);
+#else
+static inline void capability_cgroup_update_used(int cap)
+{
+}
+#endif
diff --git a/include/linux/cgroup_subsys.h b/include/linux/cgroup_subsys.h
index 0df0336a..a5161d0 100644
--- a/include/linux/cgroup_subsys.h
+++ b/include/linux/cgroup_subsys.h
@@ -56,6 +56,10 @@ SUBSYS(hugetlb)
SUBSYS(pids)
#endif
+#if IS_ENABLED(CONFIG_CGROUP_CAPABILITY)
+SUBSYS(capability)
+#endif
+
/*
* The following subsystems are not supported on the default hierarchy.
*/
diff --git a/init/Kconfig b/init/Kconfig
index f755a60..25d17ef 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -1141,6 +1141,12 @@ config CGROUP_PERF
Say N if unsure.
+config CGROUP_CAPABILITY
+ bool "Capability controller"
+ help
+ Provides a simple controller for monitoring of capabilities in the
+ cgroup.
+
config CGROUP_DEBUG
bool "Example controller"
default n
diff --git a/kernel/capability.c b/kernel/capability.c
index 45432b5..b57d7f9 100644
--- a/kernel/capability.c
+++ b/kernel/capability.c
@@ -17,6 +17,7 @@
#include
#include
#include
+#include
#include
/*
@@ -380,6 +381,7 @@ bool ns_capable(struct user_namespace *ns, int cap)
}
if (security_capable(current_cred(), ns, cap) == 0) {
+ capability_cgroup_update_used(cap);
current->flags |= PF_SUPERPRIV;
return true;
}
diff --git a/security/Makef
Re: [PATCH] capabilities: add capability cgroup controller
On 07/07/16 09:16, Petr Mladek wrote: > On Sun 2016-07-03 15:08:07, Topi Miettinen wrote: >> The attached patch would make any uses of capabilities generate audit >> messages. It works for simple tests as you can see from the commit >> message, but unfortunately the call to audit_cgroup_list() deadlocks the >> system when booting a full blown OS. There's no deadlock when the call >> is removed. >> >> I guess that in some cases, cgroup_mutex and/or css_set_lock could be >> already held earlier before entering audit_cgroup_list(). Holding the >> locks is however required by task_cgroup_from_root(). Is there any way >> to avoid this? For example, only print some kind of cgroup ID numbers >> (are there unique and stable IDs, available without locks?) for those >> cgroups where the task is registered in the audit message? > > I am not sure if anyone know what really happens here. I suggest to > enable lockdep. It might detect possible deadlock even before it > really happens, see Documentation/locking/lockdep-design.txt > > It can be enabled by > >CONFIG_PROVE_LOCKING=y > > It depends on > > CONFIG_DEBUG_KERNEL=y > > and maybe some more options, see lib/Kconfig.debug Thanks a lot! I caught this stack dump: starting version 230 [3.416647] [ cut here ] [3.417310] WARNING: CPU: 0 PID: 95 at /home/topi/d/linux.git/kernel/locking/lockdep.c:2871 lockdep_trace_alloc+0xb4/0xc0 [3.417605] DEBUG_LOCKS_WARN_ON(irqs_disabled_flags(flags)) [3.417923] Modules linked in: [3.418288] CPU: 0 PID: 95 Comm: systemd-udevd Not tainted 4.7.0-rc5+ #97 [3.418444] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Debian-1.8.2-1 04/01/2014 [3.418726] 0086 7970f3b0 8816fb00 813c9c45 [3.418993] 8816fb50 8816fb40 81091e9b [3.419176] 0b3705e2c798 0046 0410 [3.419374] Call Trace: [3.419511] [] dump_stack+0x67/0x92 [3.419644] [] __warn+0xcb/0xf0 [3.419745] [] warn_slowpath_fmt+0x5f/0x80 [3.419868] [] lockdep_trace_alloc+0xb4/0xc0 [3.419988] [] kmem_cache_alloc_node+0x42/0x600 [3.420156] [] ? debug_lockdep_rcu_enabled+0x1d/0x20 [3.420170] [] __alloc_skb+0x5b/0x1d0 [3.420170] [] audit_log_start+0x29b/0x480 [3.420170] [] ? __lock_task_sighand+0x95/0x270 [3.420170] [] audit_log_cap_use+0x39/0xf0 [3.420170] [] ns_capable+0x45/0x70 [3.420170] [] capable+0x17/0x20 [3.420170] [] oom_score_adj_write+0x150/0x2f0 [3.420170] [] __vfs_write+0x37/0x160 [3.420170] [] ? update_fast_ctr+0x17/0x30 [3.420170] [] ? percpu_down_read+0x49/0x90 [3.420170] [] ? __sb_start_write+0xb7/0xf0 [3.420170] [] ? __sb_start_write+0xb7/0xf0 [3.420170] [] vfs_write+0xb8/0x1b0 [3.420170] [] ? __fget_light+0x66/0x90 [3.420170] [] SyS_write+0x58/0xc0 [3.420170] [] do_syscall_64+0x5c/0x300 [3.420170] [] entry_SYSCALL64_slow_path+0x25/0x25 [3.420170] ---[ end trace fb586899fb556a5e ]--- [3.447922] random: systemd-udevd urandom read with 3 bits of entropy available [4.014078] clocksource: Switched to clocksource tsc Begin: Loading essential drivers ... done. This is with qemu and the boot continues normally. With real computer, there's no such output and system just seems to freeze. Could it be possible that the deadlock happens because there's some IO towards /sys/fs/cgroup, which causes a capability check and that in turn causes locking problems when we try to print cgroup list? -Topi
Re: [PATCH] capabilities: add capability cgroup controller
On Thu 2016-07-07 20:27:13, Topi Miettinen wrote:
> On 07/07/16 09:16, Petr Mladek wrote:
> > On Sun 2016-07-03 15:08:07, Topi Miettinen wrote:
> >> The attached patch would make any uses of capabilities generate audit
> >> messages. It works for simple tests as you can see from the commit
> >> message, but unfortunately the call to audit_cgroup_list() deadlocks the
> >> system when booting a full blown OS. There's no deadlock when the call
> >> is removed.
> >>
> >> I guess that in some cases, cgroup_mutex and/or css_set_lock could be
> >> already held earlier before entering audit_cgroup_list(). Holding the
> >> locks is however required by task_cgroup_from_root(). Is there any way
> >> to avoid this? For example, only print some kind of cgroup ID numbers
> >> (are there unique and stable IDs, available without locks?) for those
> >> cgroups where the task is registered in the audit message?
> >
> > I am not sure if anyone know what really happens here. I suggest to
> > enable lockdep. It might detect possible deadlock even before it
> > really happens, see Documentation/locking/lockdep-design.txt
> >
> > It can be enabled by
> >
> >CONFIG_PROVE_LOCKING=y
> >
> > It depends on
> >
> > CONFIG_DEBUG_KERNEL=y
> >
> > and maybe some more options, see lib/Kconfig.debug
>
> Thanks a lot! I caught this stack dump:
>
> starting version 230
> [3.416647] [ cut here ]
> [3.417310] WARNING: CPU: 0 PID: 95 at
> /home/topi/d/linux.git/kernel/locking/lockdep.c:2871
> lockdep_trace_alloc+0xb4/0xc0
> [3.417605] DEBUG_LOCKS_WARN_ON(irqs_disabled_flags(flags))
> [3.417923] Modules linked in:
> [3.418288] CPU: 0 PID: 95 Comm: systemd-udevd Not tainted 4.7.0-rc5+ #97
> [3.418444] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
> BIOS Debian-1.8.2-1 04/01/2014
> [3.418726] 0086 7970f3b0 8816fb00
> 813c9c45
> [3.418993] 8816fb50 8816fb40
> 81091e9b
> [3.419176] 0b3705e2c798 0046 0410
>
> [3.419374] Call Trace:
> [3.419511] [] dump_stack+0x67/0x92
> [3.419644] [] __warn+0xcb/0xf0
> [3.419745] [] warn_slowpath_fmt+0x5f/0x80
> [3.419868] [] lockdep_trace_alloc+0xb4/0xc0
> [3.419988] [] kmem_cache_alloc_node+0x42/0x600
> [3.420156] [] ? debug_lockdep_rcu_enabled+0x1d/0x20
> [3.420170] [] __alloc_skb+0x5b/0x1d0
> [3.420170] [] audit_log_start+0x29b/0x480
> [3.420170] [] ? __lock_task_sighand+0x95/0x270
> [3.420170] [] audit_log_cap_use+0x39/0xf0
> [3.420170] [] ns_capable+0x45/0x70
> [3.420170] [] capable+0x17/0x20
> [3.420170] [] oom_score_adj_write+0x150/0x2f0
> [3.420170] [] __vfs_write+0x37/0x160
> [3.420170] [] ? update_fast_ctr+0x17/0x30
> [3.420170] [] ? percpu_down_read+0x49/0x90
> [3.420170] [] ? __sb_start_write+0xb7/0xf0
> [3.420170] [] ? __sb_start_write+0xb7/0xf0
> [3.420170] [] vfs_write+0xb8/0x1b0
> [3.420170] [] ? __fget_light+0x66/0x90
> [3.420170] [] SyS_write+0x58/0xc0
> [3.420170] [] do_syscall_64+0x5c/0x300
> [3.420170] [] entry_SYSCALL64_slow_path+0x25/0x25
> [3.420170] ---[ end trace fb586899fb556a5e ]---
> [3.447922] random: systemd-udevd urandom read with 3 bits of entropy
> available
> [4.014078] clocksource: Switched to clocksource tsc
> Begin: Loading essential drivers ... done.
>
> This is with qemu and the boot continues normally. With real computer,
> there's no such output and system just seems to freeze.
>
> Could it be possible that the deadlock happens because there's some IO
> towards /sys/fs/cgroup, which causes a capability check and that in turn
> causes locking problems when we try to print cgroup list?
The above warning is printed by the code from
kernel/locking/lockdep.c:2871
static void __lockdep_trace_alloc(gfp_t gfp_mask, unsigned long flags)
{
[...]
/* We're only interested __GFP_FS allocations for now */
if (!(gfp_mask & __GFP_FS))
return;
/*
* Oi! Can't be having __GFP_FS allocations with IRQs disabled.
*/
if (DEBUG_LOCKS_WARN_ON(irqs_disabled_flags(flags)))
return;
The backtrace shows that your new audit_log_cap_use() is called
from vfs_write(). You might try to use audit_log_start() with
GFP_NOFS instead of GFP_KERNEL.
Note that this is rather intuitive advice. I still need to learn a lot
about memory management and kernel in general to be more sure about
a correct solution.
Best Regards,
Petr
Re: [PATCH] capabilities: add capability cgroup controller
On 07/08/16 09:13, Petr Mladek wrote:
> On Thu 2016-07-07 20:27:13, Topi Miettinen wrote:
>> On 07/07/16 09:16, Petr Mladek wrote:
>>> On Sun 2016-07-03 15:08:07, Topi Miettinen wrote:
The attached patch would make any uses of capabilities generate audit
messages. It works for simple tests as you can see from the commit
message, but unfortunately the call to audit_cgroup_list() deadlocks the
system when booting a full blown OS. There's no deadlock when the call
is removed.
I guess that in some cases, cgroup_mutex and/or css_set_lock could be
already held earlier before entering audit_cgroup_list(). Holding the
locks is however required by task_cgroup_from_root(). Is there any way
to avoid this? For example, only print some kind of cgroup ID numbers
(are there unique and stable IDs, available without locks?) for those
cgroups where the task is registered in the audit message?
>>>
>>> I am not sure if anyone know what really happens here. I suggest to
>>> enable lockdep. It might detect possible deadlock even before it
>>> really happens, see Documentation/locking/lockdep-design.txt
>>>
>>> It can be enabled by
>>>
>>>CONFIG_PROVE_LOCKING=y
>>>
>>> It depends on
>>>
>>> CONFIG_DEBUG_KERNEL=y
>>>
>>> and maybe some more options, see lib/Kconfig.debug
>>
>> Thanks a lot! I caught this stack dump:
>>
>> starting version 230
>> [3.416647] [ cut here ]
>> [3.417310] WARNING: CPU: 0 PID: 95 at
>> /home/topi/d/linux.git/kernel/locking/lockdep.c:2871
>> lockdep_trace_alloc+0xb4/0xc0
>> [3.417605] DEBUG_LOCKS_WARN_ON(irqs_disabled_flags(flags))
>> [3.417923] Modules linked in:
>> [3.418288] CPU: 0 PID: 95 Comm: systemd-udevd Not tainted 4.7.0-rc5+ #97
>> [3.418444] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
>> BIOS Debian-1.8.2-1 04/01/2014
>> [3.418726] 0086 7970f3b0 8816fb00
>> 813c9c45
>> [3.418993] 8816fb50 8816fb40
>> 81091e9b
>> [3.419176] 0b3705e2c798 0046 0410
>>
>> [3.419374] Call Trace:
>> [3.419511] [] dump_stack+0x67/0x92
>> [3.419644] [] __warn+0xcb/0xf0
>> [3.419745] [] warn_slowpath_fmt+0x5f/0x80
>> [3.419868] [] lockdep_trace_alloc+0xb4/0xc0
>> [3.419988] [] kmem_cache_alloc_node+0x42/0x600
>> [3.420156] [] ? debug_lockdep_rcu_enabled+0x1d/0x20
>> [3.420170] [] __alloc_skb+0x5b/0x1d0
>> [3.420170] [] audit_log_start+0x29b/0x480
>> [3.420170] [] ? __lock_task_sighand+0x95/0x270
>> [3.420170] [] audit_log_cap_use+0x39/0xf0
>> [3.420170] [] ns_capable+0x45/0x70
>> [3.420170] [] capable+0x17/0x20
>> [3.420170] [] oom_score_adj_write+0x150/0x2f0
>> [3.420170] [] __vfs_write+0x37/0x160
>> [3.420170] [] ? update_fast_ctr+0x17/0x30
>> [3.420170] [] ? percpu_down_read+0x49/0x90
>> [3.420170] [] ? __sb_start_write+0xb7/0xf0
>> [3.420170] [] ? __sb_start_write+0xb7/0xf0
>> [3.420170] [] vfs_write+0xb8/0x1b0
>> [3.420170] [] ? __fget_light+0x66/0x90
>> [3.420170] [] SyS_write+0x58/0xc0
>> [3.420170] [] do_syscall_64+0x5c/0x300
>> [3.420170] [] entry_SYSCALL64_slow_path+0x25/0x25
>> [3.420170] ---[ end trace fb586899fb556a5e ]---
>> [3.447922] random: systemd-udevd urandom read with 3 bits of entropy
>> available
>> [4.014078] clocksource: Switched to clocksource tsc
>> Begin: Loading essential drivers ... done.
>>
>> This is with qemu and the boot continues normally. With real computer,
>> there's no such output and system just seems to freeze.
>>
>> Could it be possible that the deadlock happens because there's some IO
>> towards /sys/fs/cgroup, which causes a capability check and that in turn
>> causes locking problems when we try to print cgroup list?
>
> The above warning is printed by the code from
> kernel/locking/lockdep.c:2871
>
> static void __lockdep_trace_alloc(gfp_t gfp_mask, unsigned long flags)
> {
> [...]
> /* We're only interested __GFP_FS allocations for now */
> if (!(gfp_mask & __GFP_FS))
> return;
>
> /*
>* Oi! Can't be having __GFP_FS allocations with IRQs disabled.
>*/
> if (DEBUG_LOCKS_WARN_ON(irqs_disabled_flags(flags)))
> return;
>
>
> The backtrace shows that your new audit_log_cap_use() is called
> from vfs_write(). You might try to use audit_log_start() with
> GFP_NOFS instead of GFP_KERNEL.
>
> Note that this is rather intuitive advice. I still need to learn a lot
> about memory management and kernel in general to be more sure about
> a correct solution.
Here's what I got now:
[ 18.043181]
[ 18.044123] ==
[ 18.044123] [ INFO: possible circular locking dependency detected ]
[ 18.044123] 4.7.0-rc5+ #99 Not tainted
[ 18.044123]
Re: [PATCH] capabilities: add capability cgroup controller
On 07/08/16 09:13, Petr Mladek wrote:
> On Thu 2016-07-07 20:27:13, Topi Miettinen wrote:
>> On 07/07/16 09:16, Petr Mladek wrote:
>>> On Sun 2016-07-03 15:08:07, Topi Miettinen wrote:
The attached patch would make any uses of capabilities generate audit
messages. It works for simple tests as you can see from the commit
message, but unfortunately the call to audit_cgroup_list() deadlocks the
system when booting a full blown OS. There's no deadlock when the call
is removed.
I guess that in some cases, cgroup_mutex and/or css_set_lock could be
already held earlier before entering audit_cgroup_list(). Holding the
locks is however required by task_cgroup_from_root(). Is there any way
to avoid this? For example, only print some kind of cgroup ID numbers
(are there unique and stable IDs, available without locks?) for those
cgroups where the task is registered in the audit message?
>>>
>>> I am not sure if anyone know what really happens here. I suggest to
>>> enable lockdep. It might detect possible deadlock even before it
>>> really happens, see Documentation/locking/lockdep-design.txt
>>>
>>> It can be enabled by
>>>
>>>CONFIG_PROVE_LOCKING=y
>>>
>>> It depends on
>>>
>>> CONFIG_DEBUG_KERNEL=y
>>>
>>> and maybe some more options, see lib/Kconfig.debug
>>
>> Thanks a lot! I caught this stack dump:
>>
>> starting version 230
>> [3.416647] [ cut here ]
>> [3.417310] WARNING: CPU: 0 PID: 95 at
>> /home/topi/d/linux.git/kernel/locking/lockdep.c:2871
>> lockdep_trace_alloc+0xb4/0xc0
>> [3.417605] DEBUG_LOCKS_WARN_ON(irqs_disabled_flags(flags))
>> [3.417923] Modules linked in:
>> [3.418288] CPU: 0 PID: 95 Comm: systemd-udevd Not tainted 4.7.0-rc5+ #97
>> [3.418444] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
>> BIOS Debian-1.8.2-1 04/01/2014
>> [3.418726] 0086 7970f3b0 8816fb00
>> 813c9c45
>> [3.418993] 8816fb50 8816fb40
>> 81091e9b
>> [3.419176] 0b3705e2c798 0046 0410
>>
>> [3.419374] Call Trace:
>> [3.419511] [] dump_stack+0x67/0x92
>> [3.419644] [] __warn+0xcb/0xf0
>> [3.419745] [] warn_slowpath_fmt+0x5f/0x80
>> [3.419868] [] lockdep_trace_alloc+0xb4/0xc0
>> [3.419988] [] kmem_cache_alloc_node+0x42/0x600
>> [3.420156] [] ? debug_lockdep_rcu_enabled+0x1d/0x20
>> [3.420170] [] __alloc_skb+0x5b/0x1d0
>> [3.420170] [] audit_log_start+0x29b/0x480
>> [3.420170] [] ? __lock_task_sighand+0x95/0x270
>> [3.420170] [] audit_log_cap_use+0x39/0xf0
>> [3.420170] [] ns_capable+0x45/0x70
>> [3.420170] [] capable+0x17/0x20
>> [3.420170] [] oom_score_adj_write+0x150/0x2f0
>> [3.420170] [] __vfs_write+0x37/0x160
>> [3.420170] [] ? update_fast_ctr+0x17/0x30
>> [3.420170] [] ? percpu_down_read+0x49/0x90
>> [3.420170] [] ? __sb_start_write+0xb7/0xf0
>> [3.420170] [] ? __sb_start_write+0xb7/0xf0
>> [3.420170] [] vfs_write+0xb8/0x1b0
>> [3.420170] [] ? __fget_light+0x66/0x90
>> [3.420170] [] SyS_write+0x58/0xc0
>> [3.420170] [] do_syscall_64+0x5c/0x300
>> [3.420170] [] entry_SYSCALL64_slow_path+0x25/0x25
>> [3.420170] ---[ end trace fb586899fb556a5e ]---
>> [3.447922] random: systemd-udevd urandom read with 3 bits of entropy
>> available
>> [4.014078] clocksource: Switched to clocksource tsc
>> Begin: Loading essential drivers ... done.
>>
>> This is with qemu and the boot continues normally. With real computer,
>> there's no such output and system just seems to freeze.
>>
>> Could it be possible that the deadlock happens because there's some IO
>> towards /sys/fs/cgroup, which causes a capability check and that in turn
>> causes locking problems when we try to print cgroup list?
>
> The above warning is printed by the code from
> kernel/locking/lockdep.c:2871
>
> static void __lockdep_trace_alloc(gfp_t gfp_mask, unsigned long flags)
> {
> [...]
> /* We're only interested __GFP_FS allocations for now */
> if (!(gfp_mask & __GFP_FS))
> return;
>
> /*
>* Oi! Can't be having __GFP_FS allocations with IRQs disabled.
>*/
> if (DEBUG_LOCKS_WARN_ON(irqs_disabled_flags(flags)))
> return;
>
>
> The backtrace shows that your new audit_log_cap_use() is called
> from vfs_write(). You might try to use audit_log_start() with
> GFP_NOFS instead of GFP_KERNEL.
>
> Note that this is rather intuitive advice. I still need to learn a lot
> about memory management and kernel in general to be more sure about
> a correct solution.
>
> Best Regards,
> Petr
>
With the attached patch, the system boots without deadlock. Locking
problems still remain:
[3.652221] ==
[3.652221] [ INFO: possible circular locking dependency detected ]
[3.6522
Re: [PATCH] capabilities: add capability cgroup controller
On Sun 2016-07-03 15:08:07, Topi Miettinen wrote: > The attached patch would make any uses of capabilities generate audit > messages. It works for simple tests as you can see from the commit > message, but unfortunately the call to audit_cgroup_list() deadlocks the > system when booting a full blown OS. There's no deadlock when the call > is removed. > > I guess that in some cases, cgroup_mutex and/or css_set_lock could be > already held earlier before entering audit_cgroup_list(). Holding the > locks is however required by task_cgroup_from_root(). Is there any way > to avoid this? For example, only print some kind of cgroup ID numbers > (are there unique and stable IDs, available without locks?) for those > cgroups where the task is registered in the audit message? I am not sure if anyone know what really happens here. I suggest to enable lockdep. It might detect possible deadlock even before it really happens, see Documentation/locking/lockdep-design.txt It can be enabled by CONFIG_PROVE_LOCKING=y It depends on CONFIG_DEBUG_KERNEL=y and maybe some more options, see lib/Kconfig.debug Best Regards, Petr
Re: [PATCH] capabilities: add capability cgroup controller
On Thu, Jun 23, 2016 at 8:07 AM, Topi Miettinen wrote: > There are many basic ways to control processes, including capabilities, > cgroups and resource limits. However, there are far fewer ways to find > out useful values for the limits, except blind trial and error. > > Currently, there is no way to know which capabilities are actually used. > Even the source code is only implicit, in-depth knowledge of each > capability must be used when analyzing a program to judge which > capabilities the program will exercise. > > Add a new cgroup controller for monitoring of capabilities > in the cgroup. > > Test case demonstrating basic capability monitoring and how the > capabilities are combined at next level (boot to rdshell): > > (initramfs) cd /sys/fs > (initramfs) mount -t cgroup2 cgroup cgroup > (initramfs) cd cgroup > (initramfs) echo +capability > cgroup.subtree_control > (initramfs) mkdir test; cd test > (initramfs) echo +capability > cgroup.subtree_control > (initramfs) ls > capability.used cgroup.events cgroup.subtree_control > cgroup.controllers cgroup.procs > (initramfs) mkdir first second > (initramfs) sh > > BusyBox v1.22.1 (Debian 1:1.22.0-19) built-in shell (ash) > Enter 'help' for a list of built-in commands. > > (initramfs) cd first > (initramfs) echo $$ >cgroup.procs > (initramfs) cat capability.used > # nothing so far > (initramfs) mknod /dev/z_$$ c 1 2 > (initramfs) cat capability.used > 0800 # CAP_MKNOD > (initramfs) cat ../capability.used > 0800 # also seen at next higher level > (initramfs) exit > (initramfs) sh > > BusyBox v1.22.1 (Debian 1:1.22.0-19) built-in shell (ash) > Enter 'help' for a list of built-in commands. > > (initramfs) cd second > (initramfs) echo $$ >cgroup.procs > (initramfs) cat capability.used > # nothing so far > (initramfs) chown 1234 /dev/z_* > (initramfs) cat capability.used > 0001 # CAP_CHROOT nitpick: this is CAP_CHOWN, not CAP_CHROOT -Kees -- Kees Cook Chrome OS & Brillo Security
Re: [PATCH] capabilities: add capability cgroup controller
Hello, On Thu, Jun 23, 2016 at 06:07:10PM +0300, Topi Miettinen wrote: > There are many basic ways to control processes, including capabilities, > cgroups and resource limits. However, there are far fewer ways to find > out useful values for the limits, except blind trial and error. > > Currently, there is no way to know which capabilities are actually used. > Even the source code is only implicit, in-depth knowledge of each > capability must be used when analyzing a program to judge which > capabilities the program will exercise. > > Add a new cgroup controller for monitoring of capabilities > in the cgroup. > > Test case demonstrating basic capability monitoring and how the > capabilities are combined at next level (boot to rdshell): This doesn't have anything to do with resource control and I don't think it's a good idea to add arbitrary monitoring mechanisms to cgroup just because it's easy to add interface there. Given that capabilities are inherited and modified through the process hierarchy, shouldn't this be part of that? Thanks. -- tejun
Re: [PATCH] capabilities: add capability cgroup controller
On Thu, 23 Jun 2016 18:07:10 +0300 Topi Miettinen wrote: > There are many basic ways to control processes, including capabilities, > cgroups and resource limits. However, there are far fewer ways to find > out useful values for the limits, except blind trial and error. > > Currently, there is no way to know which capabilities are actually used. > Even the source code is only implicit, in-depth knowledge of each > capability must be used when analyzing a program to judge which > capabilities the program will exercise. > > Add a new cgroup controller for monitoring of capabilities > in the cgroup. I'm having trouble understanding how valuable this feature is to our users, and that's a rather important thing! Perhaps it would help if you were to explain your motivation: particular use cases which benefited from this, for example.
Re: [PATCH] capabilities: add capability cgroup controller
On 06/23/16 21:38, Tejun Heo wrote: > Hello, > > On Thu, Jun 23, 2016 at 06:07:10PM +0300, Topi Miettinen wrote: >> There are many basic ways to control processes, including capabilities, >> cgroups and resource limits. However, there are far fewer ways to find >> out useful values for the limits, except blind trial and error. >> >> Currently, there is no way to know which capabilities are actually used. >> Even the source code is only implicit, in-depth knowledge of each >> capability must be used when analyzing a program to judge which >> capabilities the program will exercise. >> >> Add a new cgroup controller for monitoring of capabilities >> in the cgroup. >> >> Test case demonstrating basic capability monitoring and how the >> capabilities are combined at next level (boot to rdshell): > > This doesn't have anything to do with resource control and I don't > think it's a good idea to add arbitrary monitoring mechanisms to > cgroup just because it's easy to add interface there. Given that > capabilities are inherited and modified through the process hierarchy, > shouldn't this be part of that? With per process tracking, it's easy to miss if a short-lived process exercised capabilities. Especially with ambient capabilities, the parent process could be a shell script which might not use capabilities at all, but its children do the heavy lifting. Per process tracking (like in the version I sent earlier) could still be added on top of this to complement cgroup level tracking, but I think cgroup approach is more flexible as it can cover anything from a single task to a collection of processes. -Topi > > Thanks. >
Re: [PATCH] capabilities: add capability cgroup controller
On 06/23/16 23:46, Andrew Morton wrote: > On Thu, 23 Jun 2016 18:07:10 +0300 Topi Miettinen wrote: > >> There are many basic ways to control processes, including capabilities, >> cgroups and resource limits. However, there are far fewer ways to find >> out useful values for the limits, except blind trial and error. >> >> Currently, there is no way to know which capabilities are actually used. >> Even the source code is only implicit, in-depth knowledge of each >> capability must be used when analyzing a program to judge which >> capabilities the program will exercise. >> >> Add a new cgroup controller for monitoring of capabilities >> in the cgroup. > > I'm having trouble understanding how valuable this feature is to our > users, and that's a rather important thing! > > Perhaps it would help if you were to explain your motivation: > particular use cases which benefited from this, for example. > It's easy to control with for example systemd or many other tools, which capabilities a service should have at the start. But how should a system administrator, application developer or distro maintaner ever determine a suitable value for this? Currently the only way seems to be to become an expert on capabilities, make an educated guess how the set of programs in question happen to work in this context and especially how they could exercise the capabilites in all possible use cases. Even then, the outcome is to just try something to see if that happens to work. Reading the source code (if available) does not help very much, because the use of capabilities is anything but explicit there. This is way too difficult, there must be some easier way. The information which capabilities actually were used in a trial run gives a much better starting point. The users can just use the list of used capabilities with configuring the service or when developing or maintaining the application. Of course, even that could still fail eventually, but then you simply copy the new value of used capabilities to the configuration, whereas currently you have to reconsider your understanding of the capabilities and the programs in light of the failure, which by itself might give no new useful information. One way to solve this for good would be to make the use of capabilities explicit in the ABI. For example, there could be a system call dac_override() which would be the only possible way ever to use the capability CAP_DAC_OVERRIDE and so forth. Then reading source code, tracing and many other approaches would be useful. But the OS with that kind of ABI (not Linux) would not be Unix-like at all for any (potentially) capability using programs, like find(1) or cat(1). -Topi
Re: [PATCH] capabilities: add capability cgroup controller
On Thu, Jun 23, 2016 at 6:14 PM, Topi Miettinen wrote: > On 06/23/16 23:46, Andrew Morton wrote: >> On Thu, 23 Jun 2016 18:07:10 +0300 Topi Miettinen wrote: >> >>> There are many basic ways to control processes, including capabilities, >>> cgroups and resource limits. However, there are far fewer ways to find >>> out useful values for the limits, except blind trial and error. >>> >>> Currently, there is no way to know which capabilities are actually used. >>> Even the source code is only implicit, in-depth knowledge of each >>> capability must be used when analyzing a program to judge which >>> capabilities the program will exercise. >>> >>> Add a new cgroup controller for monitoring of capabilities >>> in the cgroup. >> >> I'm having trouble understanding how valuable this feature is to our >> users, and that's a rather important thing! >> >> Perhaps it would help if you were to explain your motivation: >> particular use cases which benefited from this, for example. >> > > It's easy to control with for example systemd or many other tools, which > capabilities a service should have at the start. But how should a system > administrator, application developer or distro maintaner ever determine > a suitable value for this? Currently the only way seems to be to become > an expert on capabilities, make an educated guess how the set of > programs in question happen to work in this context and especially how > they could exercise the capabilites in all possible use cases. Even > then, the outcome is to just try something to see if that happens to > work. Reading the source code (if available) does not help very much, > because the use of capabilities is anything but explicit there. > > This is way too difficult, there must be some easier way. The > information which capabilities actually were used in a trial run gives a > much better starting point. The users can just use the list of used > capabilities with configuring the service or when developing or > maintaining the application. Of course, even that could still fail > eventually, but then you simply copy the new value of used capabilities > to the configuration, whereas currently you have to reconsider your > understanding of the capabilities and the programs in light of the > failure, which by itself might give no new useful information. > > One way to solve this for good would be to make the use of capabilities > explicit in the ABI. For example, there could be a system call > dac_override() which would be the only possible way ever to use the > capability CAP_DAC_OVERRIDE and so forth. Then reading source code, > tracing and many other approaches would be useful. But the OS with that > kind of ABI (not Linux) would not be Unix-like at all for any > (potentially) capability using programs, like find(1) or cat(1). The problem is that most of the capabilities are so powerful on their own that limiting services to just a few may be all but useless. --Andy
Re: [PATCH] capabilities: add capability cgroup controller
Hello, On Fri, Jun 24, 2016 at 12:22:54AM +, Topi Miettinen wrote: > > This doesn't have anything to do with resource control and I don't > > think it's a good idea to add arbitrary monitoring mechanisms to > > cgroup just because it's easy to add interface there. Given that > > capabilities are inherited and modified through the process hierarchy, > > shouldn't this be part of that? > > With per process tracking, it's easy to miss if a short-lived process > exercised capabilities. Especially with ambient capabilities, the parent > process could be a shell script which might not use capabilities at all, > but its children do the heavy lifting. But isn't being recursive orthogonal to using cgroup? Why not account usages recursively along the process hierarchy? Capabilities don't have much to do with cgroup but everything with process hierarchy. That's how they're distributed and modified. If monitoring their usages is necessary, it makes sense to do it in the same structure. Thanks. -- tejun
Re: [PATCH] capabilities: add capability cgroup controller
Quoting Tejun Heo (t...@kernel.org): > Hello, > > On Fri, Jun 24, 2016 at 12:22:54AM +, Topi Miettinen wrote: > > > This doesn't have anything to do with resource control and I don't > > > think it's a good idea to add arbitrary monitoring mechanisms to > > > cgroup just because it's easy to add interface there. Given that > > > capabilities are inherited and modified through the process hierarchy, > > > shouldn't this be part of that? > > > > With per process tracking, it's easy to miss if a short-lived process > > exercised capabilities. Especially with ambient capabilities, the parent > > process could be a shell script which might not use capabilities at all, > > but its children do the heavy lifting. > > But isn't being recursive orthogonal to using cgroup? Why not account > usages recursively along the process hierarchy? Capabilities don't > have much to do with cgroup but everything with process hierarchy. > That's how they're distributed and modified. If monitoring their > usages is necessary, it makes sense to do it in the same structure. That was my argument against using cgroups to enforce a new bounding set. For tracking though, the cgroup process tracking seems as applicable to this as it does to systemd tracking of services. It tracks a task and the children it forks.
Re: [PATCH] capabilities: add capability cgroup controller
Hello, On Fri, Jun 24, 2016 at 10:59:16AM -0500, Serge E. Hallyn wrote: > Quoting Tejun Heo (t...@kernel.org): > > But isn't being recursive orthogonal to using cgroup? Why not account > > usages recursively along the process hierarchy? Capabilities don't > > have much to do with cgroup but everything with process hierarchy. > > That's how they're distributed and modified. If monitoring their > > usages is necessary, it makes sense to do it in the same structure. > > That was my argument against using cgroups to enforce a new bounding > set. For tracking though, the cgroup process tracking seems as applicable > to this as it does to systemd tracking of services. It tracks a task and > the children it forks. Just monitoring is less jarring than implementing security enforcement via cgroup, but it is still jarring. What's wrong with recursive process hierarchy monitoring which is in line with the whole facility is implemented anyway? Thanks. -- tejun
Re: [PATCH] capabilities: add capability cgroup controller
Quoting Tejun Heo (t...@kernel.org): > Hello, > > On Fri, Jun 24, 2016 at 10:59:16AM -0500, Serge E. Hallyn wrote: > > Quoting Tejun Heo (t...@kernel.org): > > > But isn't being recursive orthogonal to using cgroup? Why not account > > > usages recursively along the process hierarchy? Capabilities don't > > > have much to do with cgroup but everything with process hierarchy. > > > That's how they're distributed and modified. If monitoring their > > > usages is necessary, it makes sense to do it in the same structure. > > > > That was my argument against using cgroups to enforce a new bounding > > set. For tracking though, the cgroup process tracking seems as applicable > > to this as it does to systemd tracking of services. It tracks a task and > > the children it forks. > > Just monitoring is less jarring than implementing security enforcement > via cgroup, but it is still jarring. What's wrong with recursive > process hierarchy monitoring which is in line with the whole facility > is implemented anyway? As I think Topi pointed out, one shortcoming is that if there is a short-lived child task, using its /proc/self/status is racy. You might just miss that it ever even existed, let alone that the "application" needed it. Another alternative we've both mentioned is to use systemtap. That's not as nice a solution as a cgroup, but then again this isn't really a common case, so maybe it is precisely what a tracing infrastructure is meant for. -serge
Re: [PATCH] capabilities: add capability cgroup controller
Hello, Serge. On Fri, Jun 24, 2016 at 11:59:10AM -0500, Serge E. Hallyn wrote: > > Just monitoring is less jarring than implementing security enforcement > > via cgroup, but it is still jarring. What's wrong with recursive > > process hierarchy monitoring which is in line with the whole facility > > is implemented anyway? > > As I think Topi pointed out, one shortcoming is that if there is a short-lived > child task, using its /proc/self/status is racy. You might just miss that it > ever even existed, let alone that the "application" needed it. But the parent can collect whatever its children used. We already do that with other stats. Thanks. -- tejun
Re: [PATCH] capabilities: add capability cgroup controller
"Serge E. Hallyn" writes: > Quoting Tejun Heo (t...@kernel.org): >> Hello, >> >> On Fri, Jun 24, 2016 at 10:59:16AM -0500, Serge E. Hallyn wrote: >> > Quoting Tejun Heo (t...@kernel.org): >> > > But isn't being recursive orthogonal to using cgroup? Why not account >> > > usages recursively along the process hierarchy? Capabilities don't >> > > have much to do with cgroup but everything with process hierarchy. >> > > That's how they're distributed and modified. If monitoring their >> > > usages is necessary, it makes sense to do it in the same structure. >> > >> > That was my argument against using cgroups to enforce a new bounding >> > set. For tracking though, the cgroup process tracking seems as applicable >> > to this as it does to systemd tracking of services. It tracks a task and >> > the children it forks. >> >> Just monitoring is less jarring than implementing security enforcement >> via cgroup, but it is still jarring. What's wrong with recursive >> process hierarchy monitoring which is in line with the whole facility >> is implemented anyway? > > As I think Topi pointed out, one shortcoming is that if there is a short-lived > child task, using its /proc/self/status is racy. You might just miss that it > ever even existed, let alone that the "application" needed it. > > Another alternative we've both mentioned is to use systemtap. That's not > as nice a solution as a cgroup, but then again this isn't really a common > case, so maybe it is precisely what a tracing infrastructure is meant for. Hmm. We have capability use wired up into auditing. So we might be able to get away with just adding an appropriate audit message in commoncap.c:cap_capable that honors the audit flag and logs an audit message. The hook in selinux already appears to do that. Certainly audit sounds like the subsystem for this kind of work, as it's whole point in life is logging things, then something in userspace can just run over the audit longs and build a nice summary. Eric
Re: [PATCH] capabilities: add capability cgroup controller
Quoting Eric W. Biederman (ebied...@xmission.com): > "Serge E. Hallyn" writes: > > > Quoting Tejun Heo (t...@kernel.org): > >> Hello, > >> > >> On Fri, Jun 24, 2016 at 10:59:16AM -0500, Serge E. Hallyn wrote: > >> > Quoting Tejun Heo (t...@kernel.org): > >> > > But isn't being recursive orthogonal to using cgroup? Why not account > >> > > usages recursively along the process hierarchy? Capabilities don't > >> > > have much to do with cgroup but everything with process hierarchy. > >> > > That's how they're distributed and modified. If monitoring their > >> > > usages is necessary, it makes sense to do it in the same structure. > >> > > >> > That was my argument against using cgroups to enforce a new bounding > >> > set. For tracking though, the cgroup process tracking seems as > >> > applicable > >> > to this as it does to systemd tracking of services. It tracks a task and > >> > the children it forks. > >> > >> Just monitoring is less jarring than implementing security enforcement > >> via cgroup, but it is still jarring. What's wrong with recursive > >> process hierarchy monitoring which is in line with the whole facility > >> is implemented anyway? > > > > As I think Topi pointed out, one shortcoming is that if there is a > > short-lived > > child task, using its /proc/self/status is racy. You might just miss that > > it > > ever even existed, let alone that the "application" needed it. > > > > Another alternative we've both mentioned is to use systemtap. That's not > > as nice a solution as a cgroup, but then again this isn't really a common > > case, so maybe it is precisely what a tracing infrastructure is meant for. > > Hmm. > > We have capability use wired up into auditing. So we might be able to > get away with just adding an appropriate audit message in > commoncap.c:cap_capable that honors the audit flag and logs an audit > message. The hook in selinux already appears to do that. > > Certainly audit sounds like the subsystem for this kind of work, as it's > whole point in life is logging things, then something in userspace can > just run over the audit longs and build a nice summary. Good point, so long as we can also track ppid or fork info (using taskstats?) that would seem the best way.
Re: [PATCH] capabilities: add capability cgroup controller
On Fri, Jun 24, 2016 at 6:15 AM, Andy Lutomirski wrote: > On Thu, Jun 23, 2016 at 6:14 PM, Topi Miettinen wrote: >> On 06/23/16 23:46, Andrew Morton wrote: >>> On Thu, 23 Jun 2016 18:07:10 +0300 Topi Miettinen >>> wrote: >>> There are many basic ways to control processes, including capabilities, cgroups and resource limits. However, there are far fewer ways to find out useful values for the limits, except blind trial and error. Currently, there is no way to know which capabilities are actually used. Even the source code is only implicit, in-depth knowledge of each capability must be used when analyzing a program to judge which capabilities the program will exercise. Add a new cgroup controller for monitoring of capabilities in the cgroup. >>> >>> I'm having trouble understanding how valuable this feature is to our >>> users, and that's a rather important thing! >>> >>> Perhaps it would help if you were to explain your motivation: >>> particular use cases which benefited from this, for example. >>> >> >> It's easy to control with for example systemd or many other tools, which >> capabilities a service should have at the start. But how should a system >> administrator, application developer or distro maintaner ever determine >> a suitable value for this? Currently the only way seems to be to become >> an expert on capabilities, make an educated guess how the set of >> programs in question happen to work in this context and especially how >> they could exercise the capabilites in all possible use cases. Even >> then, the outcome is to just try something to see if that happens to >> work. Reading the source code (if available) does not help very much, >> because the use of capabilities is anything but explicit there. >> >> This is way too difficult, there must be some easier way. The >> information which capabilities actually were used in a trial run gives a >> much better starting point. The users can just use the list of used >> capabilities with configuring the service or when developing or >> maintaining the application. Of course, even that could still fail >> eventually, but then you simply copy the new value of used capabilities >> to the configuration, whereas currently you have to reconsider your >> understanding of the capabilities and the programs in light of the >> failure, which by itself might give no new useful information. >> >> One way to solve this for good would be to make the use of capabilities >> explicit in the ABI. For example, there could be a system call >> dac_override() which would be the only possible way ever to use the >> capability CAP_DAC_OVERRIDE and so forth. Then reading source code, >> tracing and many other approaches would be useful. But the OS with that >> kind of ABI (not Linux) would not be Unix-like at all for any >> (potentially) capability using programs, like find(1) or cat(1). > > The problem is that most of the capabilities are so powerful on their > own that limiting services to just a few may be all but useless. May be there is some gain _if_ the resources that a process interact with _can_ also be made invisible with namespaces. > --Andy > -- > To unsubscribe from this list: send the line "unsubscribe > linux-security-module" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- tixxdz http://opendz.org
Re: [PATCH] capabilities: add capability cgroup controller
On 06/28/16 04:57, Eric W. Biederman wrote: > Topi Miettinen writes: > >> On 06/24/16 17:21, Eric W. Biederman wrote: >>> "Serge E. Hallyn" writes: >>> Quoting Tejun Heo (t...@kernel.org): > Hello, > > On Fri, Jun 24, 2016 at 10:59:16AM -0500, Serge E. Hallyn wrote: >> Quoting Tejun Heo (t...@kernel.org): >>> But isn't being recursive orthogonal to using cgroup? Why not account >>> usages recursively along the process hierarchy? Capabilities don't >>> have much to do with cgroup but everything with process hierarchy. >>> That's how they're distributed and modified. If monitoring their >>> usages is necessary, it makes sense to do it in the same structure. >> >> That was my argument against using cgroups to enforce a new bounding >> set. For tracking though, the cgroup process tracking seems as >> applicable >> to this as it does to systemd tracking of services. It tracks a task and >> the children it forks. > > Just monitoring is less jarring than implementing security enforcement > via cgroup, but it is still jarring. What's wrong with recursive > process hierarchy monitoring which is in line with the whole facility > is implemented anyway? As I think Topi pointed out, one shortcoming is that if there is a short-lived child task, using its /proc/self/status is racy. You might just miss that it ever even existed, let alone that the "application" needed it. Another alternative we've both mentioned is to use systemtap. That's not as nice a solution as a cgroup, but then again this isn't really a common case, so maybe it is precisely what a tracing infrastructure is meant for. >>> >>> Hmm. >>> >>> We have capability use wired up into auditing. So we might be able to >>> get away with just adding an appropriate audit message in >>> commoncap.c:cap_capable that honors the audit flag and logs an audit >>> message. The hook in selinux already appears to do that. >>> >>> Certainly audit sounds like the subsystem for this kind of work, as it's >>> whole point in life is logging things, then something in userspace can >>> just run over the audit longs and build a nice summary. >> >> Even simpler would be to avoid the complexity of audit subsystem and >> just printk() when a task starts using a capability first time (not on >> further uses by same task). There are not that many capability bits nor >> privileged processes, meaning not too many log entries. I know as this >> was actually my first approach. But it's also far less user friendly >> than just reading a summarized value which could be directly fed back to >> configuration. > > Your loss. > >> Logging/auditing approach also doesn't work well for other things I'd >> like to present meaningful values for the user. For example, consider >> RLIMIT_AS, where my goal is also to enable the users to be able to >> configure this limit for a service. Should there be an audit message >> whenever the address space limit grows (i.e. each mmap())? What about >> when it shrinks? For RLIMIT_NOFILE we'd have to report each >> open()/close()/dup()/socket()/etc. and track how many are opened at the >> same time. I think it's better to store the fully cooked (meaningful to >> user) value in kernel and present it only when asked. > > That doesn't have anything to do with anything. > > My suggestion was very much to do with capabilities which are already > logged with the audit subsystem with selinux. The idea was to move > those audit calls into commoncap where they arguably belong allow anyone > to use them for anything. > > That is a non-controversial code cleanup that happens to cover your > special case. That is enough to build a tool in userspace that will > tell you which capabilities you need without penalizing the kernel, or > the vast majority of everyone who does not use your feature. > > From what I have seen of this conversation there is not and will not be > one interface to rule them all. Now that I know taskstats better, it looks like a good choice for most of the highwater marks, complemented with audit logging. The taskstats interface is only available to privileged processes but that's OK. I'll make new patches based on this approach. -Topi > > Eric >
Re: [PATCH] capabilities: add capability cgroup controller
On 06/27/16 19:49, Serge E. Hallyn wrote: > Quoting Tejun Heo (t...@kernel.org): >> Hello, >> >> On Mon, Jun 27, 2016 at 3:10 PM, Topi Miettinen wrote: >>> I'll have to study these more. But from what I saw so far, it looks to >>> me that a separate tool would be needed to read taskstats and if that >>> tool is not taken by distros, the users would not be any wiser, right? >>> With cgroup (or /proc), no new tools would be needed. >> >> That is a factor but shouldn't be a deciding factor in designing our >> user-facing interfaces. Please also note that kernel source tree >> already has tools/ subdirectory which contains userland tools which >> are distributed along with the kernel. > > And, if you take audit+cgroup approach then no tools are needed. So long > as you can have audit print out the cgroups for a task as part of the > capability audit record. > The attached patch would make any uses of capabilities generate audit messages. It works for simple tests as you can see from the commit message, but unfortunately the call to audit_cgroup_list() deadlocks the system when booting a full blown OS. There's no deadlock when the call is removed. I guess that in some cases, cgroup_mutex and/or css_set_lock could be already held earlier before entering audit_cgroup_list(). Holding the locks is however required by task_cgroup_from_root(). Is there any way to avoid this? For example, only print some kind of cgroup ID numbers (are there unique and stable IDs, available without locks?) for those cgroups where the task is registered in the audit message? I could remove the cgroup part from the audit message entirely, but then knowing which capabilities were used in what cgroup gets much more difficult. The rest of the patch would be useful without it and of course simpler. In my earlier versions a per-task cap_used variable summarized all uses of capabilities, but it was not clear when to reset the variable (fork? exec? capset?), so it's gone for now. This was also used to rate limit printing audit messages by only acting when each capability was first used by the task, but now all uses of capabilities trigger audit logging. Could that become a problem? I think it only makes sense to summarize capability use per cgroup (via taskstats). -Topi >From 2d5248f91998873174dbcbcafe87e5b30c3858aa Mon Sep 17 00:00:00 2001 From: Topi Miettinen Date: Sat, 2 Jul 2016 16:25:20 +0300 Subject: [PATCH] capabilities: audit capability use There are many basic ways to control processes, including capabilities, cgroups and resource limits. However, there are far fewer ways to find out useful values for the limits, except blind trial and error. Currently, there is no way to know which capabilities are actually used. Even the source code is only implicit, in-depth knowledge of each capability must be used when analyzing a program to judge which capabilities the program will exercise. Generate an audit message when capabilities are used. This can then be used to configure capability sets for services by a software developer, maintainer or system administrator. Test case demonstrating basic capability monitoring with the new message type 1330 and how the cgroups are displayed (boot to rdshell): BusyBox v1.22.1 (Debian 1:1.22.0-19) built-in shell (ash) Enter 'help' for a list of built-in commands. (initramfs) cd /sys/fs (initramfs) mount -t cgroup2 cgroup cgroup [ 16.503902] audit_printk_skb: 4026 callbacks suppressed [ 16.505059] audit: type=1330 audit(1467543885.733:469): cap_used=21 pid=214 auid=4294967295 uid=0 gid=0 ses=4294967295 cgroups= [ 16.506845] audit: type=1330 audit(1467543885.733:469): cap_used=21 pid=214 auid=4294967295 uid=0 gid=0 ses=4294967295 cgroups= [ 16.509234] audit: type=1300 audit(1467543885.733:469): arch=c03e syscall=165 success=yes exit=0 a0=7ffc2f394e2d a1=7ffc2f394e34 a2=7ffc2f394e25 a3=8000 items=0 ppid=213 pid=214 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=ttyS0 ses=4294967295 comm="mount" exe="/bin/mount" key=(null) [ 16.510134] audit: type=1327 audit(1467543885.733:469): proctitle=6D6F756E74002D74006367726F757032006367726F7570006367726F7570 (initramfs) cd cgroup (initramfs) mkdir test; cd test [ 16.533829] audit: type=1330 audit(1467543885.765:470): cap_used=1 pid=215 auid=4294967295 uid=0 gid=0 ses=4294967295 cgroups=:/; [ 16.536587] audit: type=1300 audit(1467543885.765:470): arch=c03e syscall=83 success=yes exit=0 a0=7ffe4f0bfe29 a1=1ff a2=0 a3=1e2 items=0 ppid=213 pid=215 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=ttyS0 ses=4294967295 comm="mkdir" exe="/bin/mkdir" key=(null) [ 16.537263] audit: type=1327 audit(1467543885.765:470): proctitle=6D6B6469720074657374 (initramfs) echo $$ >cgroup.procs (initramfs) mknod /dev/z_$$ c 1 2 [ 16.571516] audit: type=1330 audit(1467543885.801:471): cap_used=27 pid=216 auid=4294967295 uid=0 gid=0 ses=4294967295 cgroups=:/test; [ 16.572812] audit: type=1300 aud
Re: [PATCH] capabilities: add capability cgroup controller
On 06/24/16 17:21, Eric W. Biederman wrote: > "Serge E. Hallyn" writes: > >> Quoting Tejun Heo (t...@kernel.org): >>> Hello, >>> >>> On Fri, Jun 24, 2016 at 10:59:16AM -0500, Serge E. Hallyn wrote: Quoting Tejun Heo (t...@kernel.org): > But isn't being recursive orthogonal to using cgroup? Why not account > usages recursively along the process hierarchy? Capabilities don't > have much to do with cgroup but everything with process hierarchy. > That's how they're distributed and modified. If monitoring their > usages is necessary, it makes sense to do it in the same structure. That was my argument against using cgroups to enforce a new bounding set. For tracking though, the cgroup process tracking seems as applicable to this as it does to systemd tracking of services. It tracks a task and the children it forks. >>> >>> Just monitoring is less jarring than implementing security enforcement >>> via cgroup, but it is still jarring. What's wrong with recursive >>> process hierarchy monitoring which is in line with the whole facility >>> is implemented anyway? >> >> As I think Topi pointed out, one shortcoming is that if there is a >> short-lived >> child task, using its /proc/self/status is racy. You might just miss that it >> ever even existed, let alone that the "application" needed it. >> >> Another alternative we've both mentioned is to use systemtap. That's not >> as nice a solution as a cgroup, but then again this isn't really a common >> case, so maybe it is precisely what a tracing infrastructure is meant for. > > Hmm. > > We have capability use wired up into auditing. So we might be able to > get away with just adding an appropriate audit message in > commoncap.c:cap_capable that honors the audit flag and logs an audit > message. The hook in selinux already appears to do that. > > Certainly audit sounds like the subsystem for this kind of work, as it's > whole point in life is logging things, then something in userspace can > just run over the audit longs and build a nice summary. Even simpler would be to avoid the complexity of audit subsystem and just printk() when a task starts using a capability first time (not on further uses by same task). There are not that many capability bits nor privileged processes, meaning not too many log entries. I know as this was actually my first approach. But it's also far less user friendly than just reading a summarized value which could be directly fed back to configuration. Logging/auditing approach also doesn't work well for other things I'd like to present meaningful values for the user. For example, consider RLIMIT_AS, where my goal is also to enable the users to be able to configure this limit for a service. Should there be an audit message whenever the address space limit grows (i.e. each mmap())? What about when it shrinks? For RLIMIT_NOFILE we'd have to report each open()/close()/dup()/socket()/etc. and track how many are opened at the same time. I think it's better to store the fully cooked (meaningful to user) value in kernel and present it only when asked. -Topi > > Eric >
Re: [PATCH] capabilities: add capability cgroup controller
On 06/24/16 17:24, Tejun Heo wrote: > Hello, Serge. > > On Fri, Jun 24, 2016 at 11:59:10AM -0500, Serge E. Hallyn wrote: >>> Just monitoring is less jarring than implementing security enforcement >>> via cgroup, but it is still jarring. What's wrong with recursive >>> process hierarchy monitoring which is in line with the whole facility >>> is implemented anyway? >> >> As I think Topi pointed out, one shortcoming is that if there is a >> short-lived >> child task, using its /proc/self/status is racy. You might just miss that it >> ever even existed, let alone that the "application" needed it. > > But the parent can collect whatever its children used. We already do > that with other stats. The parent might be able do it if proc/pid/xyz files are still accessible after child exit but before its exit status is collected. But if the parent doesn't do it (and you are not able to change it to do it) and it collects the exit status without collecting other info, can you suggest a different way how another process could collect it 100% reliably? -Topi > > Thanks. >
Re: [PATCH] capabilities: add capability cgroup controller
Hello, Topi. On Sun, Jun 26, 2016 at 3:14 PM, Topi Miettinen wrote: > The parent might be able do it if proc/pid/xyz files are still > accessible after child exit but before its exit status is collected. But > if the parent doesn't do it (and you are not able to change it to do it) > and it collects the exit status without collecting other info, can you > suggest a different way how another process could collect it 100% reliably? I'm not saying that there's such mechanism now. I'm suggesting that that'd be a more fitting way of implementing a new mechanism to track capability usages. Thanks. -- tejun
Re: [PATCH] capabilities: add capability cgroup controller
Quoting Tejun Heo (t...@kernel.org): > Hello, Topi. > > On Sun, Jun 26, 2016 at 3:14 PM, Topi Miettinen wrote: > > The parent might be able do it if proc/pid/xyz files are still > > accessible after child exit but before its exit status is collected. But > > if the parent doesn't do it (and you are not able to change it to do it) > > and it collects the exit status without collecting other info, can you > > suggest a different way how another process could collect it 100% reliably? > > I'm not saying that there's such mechanism now. I'm suggesting that > that'd be a more fitting way of implementing a new mechanism to track > capability usages. Hi Topi, I think Eric was right a few emails earlier that the audit subsystem is really the most appropriate answer to this. (Perhaps sysctl-controllered?) Combined with taskstats it would give you what you need. Or you could even use an empty new named cgroup controller, say 'none,name=caps', and then look only at audit results for cgroup '/myapp' in the caps hierarchy.
Re: [PATCH] capabilities: add capability cgroup controller
On 06/27/16 14:54, Serge E. Hallyn wrote: > Quoting Tejun Heo (t...@kernel.org): >> Hello, Topi. >> >> On Sun, Jun 26, 2016 at 3:14 PM, Topi Miettinen wrote: >>> The parent might be able do it if proc/pid/xyz files are still >>> accessible after child exit but before its exit status is collected. But >>> if the parent doesn't do it (and you are not able to change it to do it) >>> and it collects the exit status without collecting other info, can you >>> suggest a different way how another process could collect it 100% reliably? >> >> I'm not saying that there's such mechanism now. I'm suggesting that >> that'd be a more fitting way of implementing a new mechanism to track >> capability usages. > > Hi Topi, > > I think Eric was right a few emails earlier that the audit subsystem is > really the most appropriate answer to this. (Perhaps sysctl-controllered?) > Combined with taskstats it would give you what you need. Or you could even > use an empty new named cgroup controller, say 'none,name=caps', and then > look only at audit results for cgroup '/myapp' in the caps hierarchy. > I'll have to study these more. But from what I saw so far, it looks to me that a separate tool would be needed to read taskstats and if that tool is not taken by distros, the users would not be any wiser, right? With cgroup (or /proc), no new tools would be needed. -Topi
Re: [PATCH] capabilities: add capability cgroup controller
Hello, On Mon, Jun 27, 2016 at 3:10 PM, Topi Miettinen wrote: > I'll have to study these more. But from what I saw so far, it looks to > me that a separate tool would be needed to read taskstats and if that > tool is not taken by distros, the users would not be any wiser, right? > With cgroup (or /proc), no new tools would be needed. That is a factor but shouldn't be a deciding factor in designing our user-facing interfaces. Please also note that kernel source tree already has tools/ subdirectory which contains userland tools which are distributed along with the kernel. Thanks. -- tejun
Re: [PATCH] capabilities: add capability cgroup controller
Quoting Tejun Heo (t...@kernel.org): > Hello, > > On Mon, Jun 27, 2016 at 3:10 PM, Topi Miettinen wrote: > > I'll have to study these more. But from what I saw so far, it looks to > > me that a separate tool would be needed to read taskstats and if that > > tool is not taken by distros, the users would not be any wiser, right? > > With cgroup (or /proc), no new tools would be needed. > > That is a factor but shouldn't be a deciding factor in designing our > user-facing interfaces. Please also note that kernel source tree > already has tools/ subdirectory which contains userland tools which > are distributed along with the kernel. And, if you take audit+cgroup approach then no tools are needed. So long as you can have audit print out the cgroups for a task as part of the capability audit record.
Re: [PATCH] capabilities: add capability cgroup controller
Topi Miettinen writes: > On 06/24/16 17:21, Eric W. Biederman wrote: >> "Serge E. Hallyn" writes: >> >>> Quoting Tejun Heo (t...@kernel.org): Hello, On Fri, Jun 24, 2016 at 10:59:16AM -0500, Serge E. Hallyn wrote: > Quoting Tejun Heo (t...@kernel.org): >> But isn't being recursive orthogonal to using cgroup? Why not account >> usages recursively along the process hierarchy? Capabilities don't >> have much to do with cgroup but everything with process hierarchy. >> That's how they're distributed and modified. If monitoring their >> usages is necessary, it makes sense to do it in the same structure. > > That was my argument against using cgroups to enforce a new bounding > set. For tracking though, the cgroup process tracking seems as applicable > to this as it does to systemd tracking of services. It tracks a task and > the children it forks. Just monitoring is less jarring than implementing security enforcement via cgroup, but it is still jarring. What's wrong with recursive process hierarchy monitoring which is in line with the whole facility is implemented anyway? >>> >>> As I think Topi pointed out, one shortcoming is that if there is a >>> short-lived >>> child task, using its /proc/self/status is racy. You might just miss that >>> it >>> ever even existed, let alone that the "application" needed it. >>> >>> Another alternative we've both mentioned is to use systemtap. That's not >>> as nice a solution as a cgroup, but then again this isn't really a common >>> case, so maybe it is precisely what a tracing infrastructure is meant for. >> >> Hmm. >> >> We have capability use wired up into auditing. So we might be able to >> get away with just adding an appropriate audit message in >> commoncap.c:cap_capable that honors the audit flag and logs an audit >> message. The hook in selinux already appears to do that. >> >> Certainly audit sounds like the subsystem for this kind of work, as it's >> whole point in life is logging things, then something in userspace can >> just run over the audit longs and build a nice summary. > > Even simpler would be to avoid the complexity of audit subsystem and > just printk() when a task starts using a capability first time (not on > further uses by same task). There are not that many capability bits nor > privileged processes, meaning not too many log entries. I know as this > was actually my first approach. But it's also far less user friendly > than just reading a summarized value which could be directly fed back to > configuration. Your loss. > Logging/auditing approach also doesn't work well for other things I'd > like to present meaningful values for the user. For example, consider > RLIMIT_AS, where my goal is also to enable the users to be able to > configure this limit for a service. Should there be an audit message > whenever the address space limit grows (i.e. each mmap())? What about > when it shrinks? For RLIMIT_NOFILE we'd have to report each > open()/close()/dup()/socket()/etc. and track how many are opened at the > same time. I think it's better to store the fully cooked (meaningful to > user) value in kernel and present it only when asked. That doesn't have anything to do with anything. My suggestion was very much to do with capabilities which are already logged with the audit subsystem with selinux. The idea was to move those audit calls into commoncap where they arguably belong allow anyone to use them for anything. That is a non-controversial code cleanup that happens to cover your special case. That is enough to build a tool in userspace that will tell you which capabilities you need without penalizing the kernel, or the vast majority of everyone who does not use your feature. >From what I have seen of this conversation there is not and will not be one interface to rule them all. Eric
