Re: [Touch-packages] [Bug 1873627] Re: auditd fails after moving /var it a new filesystem and turning /var/run into a symlink to /run
Alright, close the bug, I have built two different machines and tried to reproduce the problem several different ways, but have been unable to do so. Sorry to bother you with this. Trey Schisser Waveland Technologies - https://wavelandrcm.com Director of Security Operations and IT Infrastructure tschis...@wavelandrcm.com *mobile* 512-496-6660 Never send passwords, regulated data, or confidential information via unencrypted email. Please use Signal (https://signal.org) to send short secure messages (512-496-6660) Or PGP to send encrypted messages via email (See attached public key) PGP Public key for tschisser@wavelandrcm.comLongid: F056 40E6 AEE2 EB92 -BEGIN PGP PUBLIC KEY BLOCK-Version: FlowCrypt 7.6.0 Gmail Encryption Comment: Seamlessly send and receive encrypted emailxjMEXmbFHhYJKwYBBAHaRw8BAQdAaaneOd78NEVLKtQROusVcda2zUSTeF2o NCWvClyafb3NKVRyZXkgU2NoaXNzZXIgPHRzY2hpc3NlckB3YXZlbGFuZHJj bS5jb20+wncEEBYKAB8FAl5mxR4GCwkHCAMCBBUICgIDFgIBAhkBAhsDAh4B AAoJEPBWQOau4uuSIcYBAImzI7Dc+63PVDI3OTyL6VoDZOegP1dnC1Jug3wu 1uYBAP4/bZz5Pa1qNgsAuB+HfptfdJvq+EQkbFQv4t7oDwfVAs44BF5mxR4S CisGAQQBl1UBBQEBB0A5TvjO/keeyllJllXC1fvwKNvADE/T+gNXZJ8EzYVC GAMBCAfCYQQYFggACQUCXmbFHgIbDAAKCRDwVkDmruLrkkO7AP9CUat2JSbw nk5fIpKG23eLrZOZ1JGhHQMDYMus/kOXowD/bNVOZ/yoWZ4cWq7gV9/3k3dD 4pxkHA1GaPmQwKr2/gI= =fqR7 -END PGP PUBLIC KEY BLOCK- On Mon, Apr 20, 2020 at 4:40 PM Seth Arnold <1873...@bugs.launchpad.net> wrote: > Running under strace may change the execution environment enough that > it's not reflective of the actual error, but it's still worth a shot -- > can you pastebin the whole auditd strace logs? That openat() line is > actually a success -- the error we're looking for will come from the > audit_set_pid(3) function, which uses netlink, which is an incredibly > complicated protocol. The error may not look like an error in strace > output. > > Is there any chance the kernel has logged whatever the failure was in > dmesg output? > > Thanks > > -- > You received this bug notification because you are subscribed to the bug > report. > https://bugs.launchpad.net/bugs/1873627 > > Title: > auditd fails after moving /var it a new filesystem and turning > /var/run into a symlink to /run > > Status in audit package in Ubuntu: > New > > Bug description: > Auditd was working on my system (Ubuntu 18.04LTS, kernel > 4.15.0-1065-aws) until recently. But after splitting off /var into a > new filesystem it fails to launch. > > running '/sbin/auditd -f' as root indicates a problem writing the pid > file (no file exists even when it says one does) Post config load command > output: > Started dispatcher: /sbin/audispd pid: 16927 > type=DAEMON_START msg=audit(1587280022.692:2019): op=start ver=2.8.2 > format=raw kernel=4.15.0-1065-aws auid=878601141 pid=16925 uid=0 ses=24 > subj=unconfined res=success > config_manager init complete > Error setting audit daemon pid (File exists) > type=DAEMON_ABORT msg=audit(1587280022.692:2020): op=set-pid > auid=878601141 pid=16925 uid=0 ses=24 subj=unconfined res=failed > Unable to set audit pid, exiting > The audit daemon is exiting. > Error setting audit daemon pid (Permission denied) > > /var/run is a symlink to /run > /var/run permissions are 777 root:root > /run permissions are 755f root:root > no /run/auditd.pid and subsiquently no /var/run/auditd.pid exists (even > though the error incorrectly reports otherwise. > > /var/log/audit/audit.log output > type=DAEMON_START msg=audit(1587278222.942:5617): op=start ver=2.8.2 > format=raw kernel=4.15.0-1065-aws auid=4294967295 pid=7529 uid=0 > ses=4294967295 subj=unconf > ined res=success > type=DAEMON_ABORT msg=audit(1587278222.943:5618): op=set-pid > auid=4294967295 pid=7529 uid=0 ses=4294967295 subj=unconfined res=failed > > I have been pulling my hair out over this one. So I ran 'strace > /sbin/auditd -f' and found the following line in the output. > "openat(AT_FDCWD, "/var/run/auditd.pid", > O_WRONLY|O_CREAT|O_TRUNC|O_NOFOLLOW, 0644) = 4" > I am grasping at straws, but suspect that the O_NOFOLLOW option is > causing a failure in creating the pid file since /var/run is a symlink. I > could be wrong but I can't find anything else to suspect. > > Since it is best practice to split/var into a separate file system to > prevent filling the root filesystem in case of an unexpected increase > in log collection I suspect this is a bug. So either the system needs > to be able to follow symlinks or an option such as pid_file=[filepath] > needs to be available in /etc/audit/auditd.conf. > > To manage notifications about this bug go to: > https://bugs.launchpad.net/ubuntu/+source/audit/+bug/1873627/+subs
Re: [Touch-packages] [Bug 1873627] Re: auditd fails after moving /var it a new filesystem and turning /var/run into a symlink to /run
Unfortunately I can't, because I fixed the problem with a workaround and can't recreate the problem on _this_ server. My workaround was to mount the new filesystem as /var/log (since the goal was to keep logs from filling up the root file system), leaving the /var/run symlink on the same filesystem as /run and now everything works. If you give me a couple of days I can throw up a new server and see if I can reproduce the behavior. Trey Schisser Waveland Technologies - https://wavelandrcm.com Director of Security Operations and IT Infrastructure tschis...@wavelandrcm.com *mobile* 512-496-6660 Never send passwords, regulated data, or confidential information via unencrypted email. Please use Signal (https://signal.org) to send short secure messages (512-496-6660) Or PGP to send encrypted messages via email (See attached public key) PGP Public key for tschisser@wavelandrcm.comLongid: F056 40E6 AEE2 EB92 -BEGIN PGP PUBLIC KEY BLOCK-Version: FlowCrypt 7.6.0 Gmail Encryption Comment: Seamlessly send and receive encrypted emailxjMEXmbFHhYJKwYBBAHaRw8BAQdAaaneOd78NEVLKtQROusVcda2zUSTeF2o NCWvClyafb3NKVRyZXkgU2NoaXNzZXIgPHRzY2hpc3NlckB3YXZlbGFuZHJj bS5jb20+wncEEBYKAB8FAl5mxR4GCwkHCAMCBBUICgIDFgIBAhkBAhsDAh4B AAoJEPBWQOau4uuSIcYBAImzI7Dc+63PVDI3OTyL6VoDZOegP1dnC1Jug3wu 1uYBAP4/bZz5Pa1qNgsAuB+HfptfdJvq+EQkbFQv4t7oDwfVAs44BF5mxR4S CisGAQQBl1UBBQEBB0A5TvjO/keeyllJllXC1fvwKNvADE/T+gNXZJ8EzYVC GAMBCAfCYQQYFggACQUCXmbFHgIbDAAKCRDwVkDmruLrkkO7AP9CUat2JSbw nk5fIpKG23eLrZOZ1JGhHQMDYMus/kOXowD/bNVOZ/yoWZ4cWq7gV9/3k3dD 4pxkHA1GaPmQwKr2/gI= =fqR7 -END PGP PUBLIC KEY BLOCK- On Mon, Apr 20, 2020 at 4:40 PM Seth Arnold <1873...@bugs.launchpad.net> wrote: > Running under strace may change the execution environment enough that > it's not reflective of the actual error, but it's still worth a shot -- > can you pastebin the whole auditd strace logs? That openat() line is > actually a success -- the error we're looking for will come from the > audit_set_pid(3) function, which uses netlink, which is an incredibly > complicated protocol. The error may not look like an error in strace > output. > > Is there any chance the kernel has logged whatever the failure was in > dmesg output? > > Thanks > > -- > You received this bug notification because you are subscribed to the bug > report. > https://bugs.launchpad.net/bugs/1873627 > > Title: > auditd fails after moving /var it a new filesystem and turning > /var/run into a symlink to /run > > Status in audit package in Ubuntu: > New > > Bug description: > Auditd was working on my system (Ubuntu 18.04LTS, kernel > 4.15.0-1065-aws) until recently. But after splitting off /var into a > new filesystem it fails to launch. > > running '/sbin/auditd -f' as root indicates a problem writing the pid > file (no file exists even when it says one does) Post config load command > output: > Started dispatcher: /sbin/audispd pid: 16927 > type=DAEMON_START msg=audit(1587280022.692:2019): op=start ver=2.8.2 > format=raw kernel=4.15.0-1065-aws auid=878601141 pid=16925 uid=0 ses=24 > subj=unconfined res=success > config_manager init complete > Error setting audit daemon pid (File exists) > type=DAEMON_ABORT msg=audit(1587280022.692:2020): op=set-pid > auid=878601141 pid=16925 uid=0 ses=24 subj=unconfined res=failed > Unable to set audit pid, exiting > The audit daemon is exiting. > Error setting audit daemon pid (Permission denied) > > /var/run is a symlink to /run > /var/run permissions are 777 root:root > /run permissions are 755f root:root > no /run/auditd.pid and subsiquently no /var/run/auditd.pid exists (even > though the error incorrectly reports otherwise. > > /var/log/audit/audit.log output > type=DAEMON_START msg=audit(1587278222.942:5617): op=start ver=2.8.2 > format=raw kernel=4.15.0-1065-aws auid=4294967295 pid=7529 uid=0 > ses=4294967295 subj=unconf > ined res=success > type=DAEMON_ABORT msg=audit(1587278222.943:5618): op=set-pid > auid=4294967295 pid=7529 uid=0 ses=4294967295 subj=unconfined res=failed > > I have been pulling my hair out over this one. So I ran 'strace > /sbin/auditd -f' and found the following line in the output. > "openat(AT_FDCWD, "/var/run/auditd.pid", > O_WRONLY|O_CREAT|O_TRUNC|O_NOFOLLOW, 0644) = 4" > I am grasping at straws, but suspect that the O_NOFOLLOW option is > causing a failure in creating the pid file since /var/run is a symlink. I > could be wrong but I can't find anything else to suspect. > > Since it is best practice to split/var into a separate file system to > prevent filling the root filesystem in case of an unexpected increase > in log collection I suspect this is a bug. So either the system
[Touch-packages] [Bug 1873627] [NEW] auditd fails after moving /var it a new filesystem and turning /var/run into a symlink to /run
Public bug reported: Auditd was working on my system (Ubuntu 18.04LTS, kernel 4.15.0-1065-aws) until recently. But after splitting off /var into a new filesystem it fails to launch. running '/sbin/auditd -f' as root indicates a problem writing the pid file (no file exists even when it says one does) Post config load command output: Started dispatcher: /sbin/audispd pid: 16927 type=DAEMON_START msg=audit(1587280022.692:2019): op=start ver=2.8.2 format=raw kernel=4.15.0-1065-aws auid=878601141 pid=16925 uid=0 ses=24 subj=unconfined res=success config_manager init complete Error setting audit daemon pid (File exists) type=DAEMON_ABORT msg=audit(1587280022.692:2020): op=set-pid auid=878601141 pid=16925 uid=0 ses=24 subj=unconfined res=failed Unable to set audit pid, exiting The audit daemon is exiting. Error setting audit daemon pid (Permission denied) /var/run is a symlink to /run /var/run permissions are 777 root:root /run permissions are 755f root:root no /run/auditd.pid and subsiquently no /var/run/auditd.pid exists (even though the error incorrectly reports otherwise. /var/log/audit/audit.log output type=DAEMON_START msg=audit(1587278222.942:5617): op=start ver=2.8.2 format=raw kernel=4.15.0-1065-aws auid=4294967295 pid=7529 uid=0 ses=4294967295 subj=unconf ined res=success type=DAEMON_ABORT msg=audit(1587278222.943:5618): op=set-pid auid=4294967295 pid=7529 uid=0 ses=4294967295 subj=unconfined res=failed I have been pulling my hair out over this one. So I ran 'strace /sbin/auditd -f' and found the following line in the output. "openat(AT_FDCWD, "/var/run/auditd.pid", O_WRONLY|O_CREAT|O_TRUNC|O_NOFOLLOW, 0644) = 4" I am grasping at straws, but suspect that the O_NOFOLLOW option is causing a failure in creating the pid file since /var/run is a symlink. I could be wrong but I can't find anything else to suspect. Since it is best practice to split/var into a separate file system to prevent filling the root filesystem in case of an unexpected increase in log collection I suspect this is a bug. So either the system needs to be able to follow symlinks or an option such as pid_file=[filepath] needs to be available in /etc/audit/auditd.conf. ** Affects: audit (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to audit in Ubuntu. https://bugs.launchpad.net/bugs/1873627 Title: auditd fails after moving /var it a new filesystem and turning /var/run into a symlink to /run Status in audit package in Ubuntu: New Bug description: Auditd was working on my system (Ubuntu 18.04LTS, kernel 4.15.0-1065-aws) until recently. But after splitting off /var into a new filesystem it fails to launch. running '/sbin/auditd -f' as root indicates a problem writing the pid file (no file exists even when it says one does) Post config load command output: Started dispatcher: /sbin/audispd pid: 16927 type=DAEMON_START msg=audit(1587280022.692:2019): op=start ver=2.8.2 format=raw kernel=4.15.0-1065-aws auid=878601141 pid=16925 uid=0 ses=24 subj=unconfined res=success config_manager init complete Error setting audit daemon pid (File exists) type=DAEMON_ABORT msg=audit(1587280022.692:2020): op=set-pid auid=878601141 pid=16925 uid=0 ses=24 subj=unconfined res=failed Unable to set audit pid, exiting The audit daemon is exiting. Error setting audit daemon pid (Permission denied) /var/run is a symlink to /run /var/run permissions are 777 root:root /run permissions are 755f root:root no /run/auditd.pid and subsiquently no /var/run/auditd.pid exists (even though the error incorrectly reports otherwise. /var/log/audit/audit.log output type=DAEMON_START msg=audit(1587278222.942:5617): op=start ver=2.8.2 format=raw kernel=4.15.0-1065-aws auid=4294967295 pid=7529 uid=0 ses=4294967295 subj=unconf ined res=success type=DAEMON_ABORT msg=audit(1587278222.943:5618): op=set-pid auid=4294967295 pid=7529 uid=0 ses=4294967295 subj=unconfined res=failed I have been pulling my hair out over this one. So I ran 'strace /sbin/auditd -f' and found the following line in the output. "openat(AT_FDCWD, "/var/run/auditd.pid", O_WRONLY|O_CREAT|O_TRUNC|O_NOFOLLOW, 0644) = 4" I am grasping at straws, but suspect that the O_NOFOLLOW option is causing a failure in creating the pid file since /var/run is a symlink. I could be wrong but I can't find anything else to suspect. Since it is best practice to split/var into a separate file system to prevent filling the root filesystem in case of an unexpected increase in log collection I suspect this is a bug. So either the system needs to be able to follow symlinks or an option such as pid_file=[filepath] needs to be available in /etc/audit/auditd.conf. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/audit/+bug/1873627/+subscriptions -- Mailing list: https://l
