I think I may have replicated, in that I got log entries with task blocked for more than 120 seconds, very similar to the above logs. And the apparmor_parser could running ps on the system did show several apparmor_parsers waiting. However it did not crash nor did the apparmor_parser instances hang for ever, it all eventually cleared up.
To replicate I overloaded the system spawning 1000 apparmor_parsers loading/replacing profiles and 1000 apparmor_parsers removing profiles. This resulted in each parser competing for the policy load mutex lock, that causes all loads and replaces to be serialized. With the system under very high load several processes even after obtaining the policy mutex would be slept waiting on the memory subsystem and oom killer. This isn't an exact parallel as I didn't cause it to create namespaces etc, I am now planning to do that as another round of testing. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1645037 Title: apparmor_parser hangs indefinitely when called by multiple threads Status in apparmor package in Ubuntu: Triaged Status in linux package in Ubuntu: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Status in linux source package in Zesty: In Progress Bug description: This bug surfaced when starting ~50 LXC container with LXD in parallel multiple times: # Create the containers for c in c foo{1..50}; do lxc launch images:ubuntu/xenial $c; done # Exectute this loop multiple times until you observe errors. for c in c foo{1..50}; do lxc restart $c & done After this you can ps aux | grep apparmor and you should see output similar to: root 19774 0.0 0.0 12524 1116 pts/1 S+ 20:14 0:00 apparmor_parser -RWL /var/lib/lxd/security/apparmor/cache /var/lib/lxd/security/apparmor/profiles/lxd-foo30 root 19775 0.0 0.0 12524 1208 pts/1 S+ 20:14 0:00 apparmor_parser -RWL /var/lib/lxd/security/apparmor/cache /var/lib/lxd/security/apparmor/profiles/lxd-foo26 root 19776 0.0 0.0 13592 3224 pts/1 D+ 20:14 0:00 apparmor_parser -RWL /var/lib/lxd/security/apparmor/cache /var/lib/lxd/security/apparmor/profiles/lxd-foo30 root 19778 0.0 0.0 13592 3384 pts/1 D+ 20:14 0:00 apparmor_parser -RWL /var/lib/lxd/security/apparmor/cache /var/lib/lxd/security/apparmor/profiles/lxd-foo26 root 19780 0.0 0.0 12524 1208 pts/1 S+ 20:14 0:00 apparmor_parser -RWL /var/lib/lxd/security/apparmor/cache /var/lib/lxd/security/apparmor/profiles/lxd-foo43 root 19782 0.0 0.0 12524 1208 pts/1 S+ 20:14 0:00 apparmor_parser -RWL /var/lib/lxd/security/apparmor/cache /var/lib/lxd/security/apparmor/profiles/lxd-foo34 root 19783 0.0 0.0 13592 3388 pts/1 D+ 20:14 0:00 apparmor_parser -RWL /var/lib/lxd/security/apparmor/cache /var/lib/lxd/security/apparmor/profiles/lxd-foo43 root 19784 0.0 0.0 13592 3252 pts/1 D+ 20:14 0:00 apparmor_parser -RWL /var/lib/lxd/security/apparmor/cache /var/lib/lxd/security/apparmor/profiles/lxd-foo34 root 19794 0.0 0.0 12524 1208 pts/1 S+ 20:14 0:00 apparmor_parser -RWL /var/lib/lxd/security/apparmor/cache /var/lib/lxd/security/apparmor/profiles/lxd-foo25 root 19795 0.0 0.0 13592 3256 pts/1 D+ 20:14 0:00 apparmor_parser -RWL /var/lib/lxd/security/apparmor/cache /var/lib/lxd/security/apparmor/profiles/lxd-foo25 apparmor_parser remains stuck even after all LXC/LXD commands have exited. dmesg output yields lines like: [41902.815174] audit: type=1400 audit(1480191089.678:43): apparmor="STATUS" operation="profile_load" profile="unconfined" name ="lxd-foo30_</var/lib/lxd>" pid=12545 comm="apparmor_parser" and cat /proc/12545/stack shows: [<ffffffff8c9b9378>] aa_remove_profiles+0x88/0x270 21:19 brauner [<ffffffff8c9ac3e4>] profile_remove+0x144/0x2e0 21:19 brauner [<ffffffff8c8319b8>] __vfs_write+0x18/0x40 21:19 brauner [<ffffffff8c832108>] vfs_write+0xb8/0x1b0 21:19 brauner [<ffffffff8c833565>] SyS_write+0x55/0xc0 21:19 brauner [<ffffffff8ce952f6>] entry_SYSCALL_64_fastpath+0x1e/0xa8 21:19 brauner [<ffffffffffffffff>] 0xffffffffffffffff This looks like a potential kernel bug. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/apparmor/+bug/1645037/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
