Re: Auditing nftables changes
These clarifications had really helped. I will definitely check out your presentations, Richard. Thank you again to both of you for your patient explanations. They are much appreciated. Have a good weekend! Bruce On Fri, Mar 10, 2023 at 1:38 PM Richard Guy Briggs wrote: > > On 2023-03-10 11:04, Paul Moore wrote: > > On Fri, Mar 10, 2023 at 9:36 AM Steve Grubb wrote: > > > On Thursday, March 9, 2023 5:52:28 PM EST Bruce Elrick wrote: > > > > Anyway, I think I need to spend some time playing until that "aha!" > > > > moment comes. It's feels a lot closer thanks to both of your responses > > > > and I really apprecaite the time you've taken to read my emails and > > > > respond to them. > > > > > > There are simple events which are one line and compound events which are > > > multiple lines - called records. The simple events tend to be hardwired > > > and > > > not optional. For example, logins are hardwired; kernel config changes are > > > hardwired; authentication is hardwired. > > > > Reading Steve's response I'm not sure we use the same terminology, or > > perhaps we explain it a bit differently. Regardless, here is a quick > > definition that I stick to when discussing audit: > > > > "audit record": An audit record is a single line in the audit log that > > consists of a timestamp, record type (type=XXX), and a series of > > fields which are dependent on the record type. Here is an example of > > a SYSCALL record: > > > > type=SYSCALL msg=audit(03/10/2023 10:59:00.797:563) : > > arch=x86_64 syscall=bpf success=yes exit=12 a0=BPF_PROG_LOAD > > a1=0x7ffde0efc650 a2=0x80 a3=0x13 items=0 ppid=1 pid=2683 > > auid=root uid=root gid=root euid=root suid=root fsuid=root > > egid=root sgid=root fsgid=root tty=(none) ses=10 comm=systemd > > exe=/usr/lib/systemd/systemd > > subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=(null) > > > > "audit event": An audit event consists of multiple audit records > > grouped together by a single timestamp. Single record audit events > > are allowed and do exist. There is no upper bound on the number of > > records allowed in an audit event. Here is an example of an audit > > event consisting of PROCTITLE, SYSCALL, and BPF audit records: > > > > type=PROCTITLE msg=audit(03/10/2023 10:59:00.797:563) : > > proctitle=(systemd) > > type=SYSCALL msg=audit(03/10/2023 10:59:00.797:563) : > > arch=x86_64 syscall=bpf success=yes exit=12 a0=BPF_PROG_LOAD > > a1=0x7ffde0efc650 a2=0x80 a3=0x13 items=0 ppid=1 pid=2683 > > auid=root uid=root gid=root euid=root suid=root fsuid=root > > egid=root sgid=root fsgid=root tty=(none) ses=10 comm=systemd > > exe=/usr/lib/systemd/systemd > > subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=(null) > > type=BPF msg=audit(03/10/2023 10:59:00.797:563) : > > prog-id=172 op=LOAD > > An "audit event" which is a collection of audit records with the same > timestamp and serial number correspond to *one* event of interest to the > audit subsystem either due to internal rules or added audit rules that > when triggered record audit information into a set of records that are > all related to give a larger picture of the circumstances of that event. > Configuration changes, being audit rules added, or firewall rules > changes, are hardwired. > > > I hope that helps. > > > > -- > > paul-moore.com > > > > - RGB > > -- > Richard Guy Briggs > Sr. S/W Engineer, Kernel Security, Base Operating Systems > Remote, Ottawa, Red Hat Canada > IRC: rgb, SunRaycer > Voice: +1.647.777.2635, Internal: (81) 32635 > -- Linux-audit mailing list Linux-audit@redhat.com https://listman.redhat.com/mailman/listinfo/linux-audit
Re: Auditing nftables changes
On 2023-03-10 11:04, Paul Moore wrote: > On Fri, Mar 10, 2023 at 9:36 AM Steve Grubb wrote: > > On Thursday, March 9, 2023 5:52:28 PM EST Bruce Elrick wrote: > > > Anyway, I think I need to spend some time playing until that "aha!" > > > moment comes. It's feels a lot closer thanks to both of your responses > > > and I really apprecaite the time you've taken to read my emails and > > > respond to them. > > > > There are simple events which are one line and compound events which are > > multiple lines - called records. The simple events tend to be hardwired and > > not optional. For example, logins are hardwired; kernel config changes are > > hardwired; authentication is hardwired. > > Reading Steve's response I'm not sure we use the same terminology, or > perhaps we explain it a bit differently. Regardless, here is a quick > definition that I stick to when discussing audit: > > "audit record": An audit record is a single line in the audit log that > consists of a timestamp, record type (type=XXX), and a series of > fields which are dependent on the record type. Here is an example of > a SYSCALL record: > > type=SYSCALL msg=audit(03/10/2023 10:59:00.797:563) : > arch=x86_64 syscall=bpf success=yes exit=12 a0=BPF_PROG_LOAD > a1=0x7ffde0efc650 a2=0x80 a3=0x13 items=0 ppid=1 pid=2683 > auid=root uid=root gid=root euid=root suid=root fsuid=root > egid=root sgid=root fsgid=root tty=(none) ses=10 comm=systemd > exe=/usr/lib/systemd/systemd > subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=(null) > > "audit event": An audit event consists of multiple audit records > grouped together by a single timestamp. Single record audit events > are allowed and do exist. There is no upper bound on the number of > records allowed in an audit event. Here is an example of an audit > event consisting of PROCTITLE, SYSCALL, and BPF audit records: > > type=PROCTITLE msg=audit(03/10/2023 10:59:00.797:563) : > proctitle=(systemd) > type=SYSCALL msg=audit(03/10/2023 10:59:00.797:563) : > arch=x86_64 syscall=bpf success=yes exit=12 a0=BPF_PROG_LOAD > a1=0x7ffde0efc650 a2=0x80 a3=0x13 items=0 ppid=1 pid=2683 > auid=root uid=root gid=root euid=root suid=root fsuid=root > egid=root sgid=root fsgid=root tty=(none) ses=10 comm=systemd > exe=/usr/lib/systemd/systemd > subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=(null) > type=BPF msg=audit(03/10/2023 10:59:00.797:563) : > prog-id=172 op=LOAD An "audit event" which is a collection of audit records with the same timestamp and serial number correspond to *one* event of interest to the audit subsystem either due to internal rules or added audit rules that when triggered record audit information into a set of records that are all related to give a larger picture of the circumstances of that event. Configuration changes, being audit rules added, or firewall rules changes, are hardwired. > I hope that helps. > > -- > paul-moore.com > - RGB -- Richard Guy Briggs Sr. S/W Engineer, Kernel Security, Base Operating Systems Remote, Ottawa, Red Hat Canada IRC: rgb, SunRaycer Voice: +1.647.777.2635, Internal: (81) 32635 -- Linux-audit mailing list Linux-audit@redhat.com https://listman.redhat.com/mailman/listinfo/linux-audit
Re: Auditing nftables changes
On Fri, Mar 10, 2023 at 9:36 AM Steve Grubb wrote: > > On Thursday, March 9, 2023 5:52:28 PM EST Bruce Elrick wrote: > > Anyway, I think I need to spend some time playing until that "aha!" > > moment comes. It's feels a lot closer thanks to both of your responses > > and I really apprecaite the time you've taken to read my emails and > > respond to them. > > There are simple events which are one line and compound events which are > multiple lines - called records. The simple events tend to be hardwired and > not optional. For example, logins are hardwired; kernel config changes are > hardwired; authentication is hardwired. Reading Steve's response I'm not sure we use the same terminology, or perhaps we explain it a bit differently. Regardless, here is a quick definition that I stick to when discussing audit: "audit record": An audit record is a single line in the audit log that consists of a timestamp, record type (type=XXX), and a series of fields which are dependent on the record type. Here is an example of a SYSCALL record: type=SYSCALL msg=audit(03/10/2023 10:59:00.797:563) : arch=x86_64 syscall=bpf success=yes exit=12 a0=BPF_PROG_LOAD a1=0x7ffde0efc650 a2=0x80 a3=0x13 items=0 ppid=1 pid=2683 auid=root uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=(none) ses=10 comm=systemd exe=/usr/lib/systemd/systemd subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=(null) "audit event": An audit event consists of multiple audit records grouped together by a single timestamp. Single record audit events are allowed and do exist. There is no upper bound on the number of records allowed in an audit event. Here is an example of an audit event consisting of PROCTITLE, SYSCALL, and BPF audit records: type=PROCTITLE msg=audit(03/10/2023 10:59:00.797:563) : proctitle=(systemd) type=SYSCALL msg=audit(03/10/2023 10:59:00.797:563) : arch=x86_64 syscall=bpf success=yes exit=12 a0=BPF_PROG_LOAD a1=0x7ffde0efc650 a2=0x80 a3=0x13 items=0 ppid=1 pid=2683 auid=root uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=(none) ses=10 comm=systemd exe=/usr/lib/systemd/systemd subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=(null) type=BPF msg=audit(03/10/2023 10:59:00.797:563) : prog-id=172 op=LOAD I hope that helps. -- paul-moore.com -- Linux-audit mailing list Linux-audit@redhat.com https://listman.redhat.com/mailman/listinfo/linux-audit
Re: Auditing nftables changes
On Thursday, March 9, 2023 5:52:28 PM EST Bruce Elrick wrote: > Anyway, I think I need to spend some time playing until that "aha!" > moment comes. It's feels a lot closer thanks to both of your responses > and I really apprecaite the time you've taken to read my emails and > respond to them. There are simple events which are one line and compound events which are multiple lines - called records. The simple events tend to be hardwired and not optional. For example, logins are hardwired; kernel config changes are hardwired; authentication is hardwired. The compound events tend to be related to audit rules (but not always). When the rule triggers, the syscall triggering the recording travels around different parts of the kernel. As it does so, there is code that observes and records different attributes of what it's doing. It may record the path, the socket, the command line, arguments of the syscall, etc. Then when the syscall finishes, the different observations are lumped together with the same serial number and output to the audit daemon. The events originating from a rule can optionally have a key. This is to allow grouping of multiple rules that meet the same requirement. Simple events never have a key. There are a couple presentations here that may help understand the audit system: https://people.redhat.com/sgrubb/audit/ -Steve -- Linux-audit mailing list Linux-audit@redhat.com https://listman.redhat.com/mailman/listinfo/linux-audit
Re: Auditing nftables changes
Thanks Richard and also thanks Paul. Yes, that was me asking in IRC. I didn't see a reply for a while (at least an hour I thought) plus no other conversations so I wasn't sure how often people looked at it. I think I got disconnected and reconnected and thus probably completely missed your reply. I wondered if there were IRC logs but wasn't sure how to look for them; it sounds like Libera doesn't log and I didn't see any mention about logging of #audit so I assume it is not. I think I'm starting to understand but I have to admit my brain tends to get hung up on trivial things until it gets to an "aha!' moment then I have a working model in my head. I haven't quite got there. For example in some contexts it seems that an "event" is equivalent to one line in the audit logs or in the output of ausearch but in other contexts it seems that event refers to a series of lines that are related as a sequence of related events with the same event id. What happens is that in some contexts in the docs when the word event is used it isn't clear to me whether it is referring one line of the audit output or the group of lines. For me it would be much clearer if the sequence of events with the same id were called, say, an event cluster and then the id would be an event cluster id. I think what confuses me is much the same confusion I would have had decades ago when learning awk. If someone had drilled into me the conceptual mechanic of surrounding the entire awk script in a "foreach line of input" loops and then the pattern drilled in as an "if this pattern matches" and the action as "then do this sequence of commands" then I would have saved some time. I see current awk books drill that in better than I recall. So for audit rules I haven't quite internalized the equivalent conceptual mechanic. I've got pieces of it but it is not 100% there. One thing I haven't got is the concept of the flow of time (say from fork/clone+exec when a new process is created to when that process might call setuid to later when that process makes a system call (like sendmsg) to later when the process is in the kernel and decodes the message structure and determines it is a NETFILTER_CFG change message. In my head I picture that flow as time-based but then I get confused about "when" and "where" the rules apply. I know the "first rule that matches" applies but I haven't grokked at what point(s) that happens (i.e. what is the implicit loop a la awk's implicit loop that I'm missing). Up until Paul clarified about the exclusion list I don't think I had fully understood the "list,action" and how that interacted with the other options. Specifically I see that a rule added to the syscall exit list then "requires" a '-S' option (requires in the sense that it is implicitly there as a default "-S all". Up until then I'd been blindly copying examples. I need to understand "when/where" each list applies in system time and kernel/user space. Anyway, I think I need to spend some time playing until that "aha!" moment comes. It's feels a lot closer thanks to both of your responses and I really apprecaite the time you've taken to read my emails and respond to them. Thanks, Bruce On Thu, Mar 9, 2023 at 2:17 PM Richard Guy Briggs wrote: > > On 2023-03-09 15:14, Paul Moore wrote: > > On Thu, Mar 9, 2023 at 11:33 AM Bruce Elrick > > wrote: > > > > > > I think I need to clarify where I'm confused ;-) > > > > > > With iptables you could write a rule that would only catch system > > > calls that were for iptables changes. That is, you didn't need to > > > capture *all* setsockopt calls (not that there would be lots of > > > *those*) but rather you could add the a2=64 to only get the > > > op=IPT_SO_SET_REPLACE ones. > > > > > > With netfilter, however, since the control interface is netlink and > > > netlink requires a message to a socket and messages are structs, there > > > is no way to have a similarly narrow audit rule as in the case of > > > iptables. > > > > > > That's the first thing I want to confirm: whether my understanding > > > above is correct? > > > > Yes, you are correct. > > > > > I'm confused because your answer implies I'm correct > > > but you didn't explicitly confirm that my interpretation of how it > > > works was correct. > > > > > > You talk about having an exclude filter on NETFILTER_CFG (or rather > > > exclude everything except NETFILTER_CFG??) but my understanding is > > > that you can only do that filtering after the fact using ausearch or > > > writing some sort of correlation code using the auparse library. > > > > The kernel implements an exclude filter which is described in the > > auditctl(8) manpage: > > > > "Add a rule to the event type exclusion filter list. > > This list is used to filter events that you do not > > want to see. For example, if you do not want to see > > any avc messages, you would using this list to > > record that. Events can be excluded by process ID, > > user ID, group ID, login user ID,
Re: Auditing nftables changes
On 2023-03-09 15:14, Paul Moore wrote:
> On Thu, Mar 9, 2023 at 11:33 AM Bruce Elrick
> wrote:
> >
> > I think I need to clarify where I'm confused ;-)
> >
> > With iptables you could write a rule that would only catch system
> > calls that were for iptables changes. That is, you didn't need to
> > capture *all* setsockopt calls (not that there would be lots of
> > *those*) but rather you could add the a2=64 to only get the
> > op=IPT_SO_SET_REPLACE ones.
> >
> > With netfilter, however, since the control interface is netlink and
> > netlink requires a message to a socket and messages are structs, there
> > is no way to have a similarly narrow audit rule as in the case of
> > iptables.
> >
> > That's the first thing I want to confirm: whether my understanding
> > above is correct?
>
> Yes, you are correct.
>
> > I'm confused because your answer implies I'm correct
> > but you didn't explicitly confirm that my interpretation of how it
> > works was correct.
> >
> > You talk about having an exclude filter on NETFILTER_CFG (or rather
> > exclude everything except NETFILTER_CFG??) but my understanding is
> > that you can only do that filtering after the fact using ausearch or
> > writing some sort of correlation code using the auparse library.
>
> The kernel implements an exclude filter which is described in the
> auditctl(8) manpage:
>
> "Add a rule to the event type exclusion filter list.
> This list is used to filter events that you do not
> want to see. For example, if you do not want to see
> any avc messages, you would using this list to
> record that. Events can be excluded by process ID,
> user ID, group ID, login user ID, message type,
> subject context, or executable name. The action is
> ignored and uses its default of "never".
>
> Taken from https://man7.org/linux/man-pages/man8/auditctl.8.html
>
> However, in my last reply I wasn't advocating for this use of the
> exclude filter, I was simply trying to explain that unless you are
> explicitly excluding the creation of NETFILTER_CFG records via the
> exclude filter you should be seeing NETFILTER_CFG in your audit stream
> with basic auditing enabled.
Bruce, it appears you asked this question on IRC Libera #audit as
"virtuous-sloth". I replied there as Paul has clarified here that all
those configuration changes for iptables and nft should be in the audit
log by default if audit is enabled, because they are considered system
configuration changes which are required by certifications to be
audited by default. This wasn't always the behaviour, and as you asked
about "GHAK124" (https://github.com/linux-audit/audit-kernel/issues/124)
has been fixed and updated. At first, they were dependant on audit
configuration and were missing some iptables events (ghak 25, 35, 43,
44), and nft wasn't monitored at all, but as of upstream v5.8 the
iptables issues are resolved, as of v5.9 nft support was added and as of
upstream v5.13 nft support was tamed a bit due to the unnecessarily
large volume of records produced in the initial nft support. If these
events are not showing up in your logs you may have an older kernel.
Have I understood your problem and does this help clarify things?
> > It just seemed surprising that there is a non-trivial loss of audit
> > functionality but that I could not find any obvious discussion about
> > that. By obvious discussion I mean as explicitly as what I'm trying to
> > say here.
>
> Unfortunately it is a fairly common practice for kernel features to be
> added, and removed, without consulting with the various Linux Kernel
> security developers, e.g. audit, SELinux, LSM, etc. Sometimes we are
> successful in retrofitting the necessary security and/or auditing
> hooks, sometimes we are limited due to design choices.
>
> > The other thing I'm trying to understand is how heavy an audit load
> > would it be to have an audit rule that captures *all* sendmsg calls
> > (well, all except where auid=-1 or auid=${serviceuser_uid}). I don't
> > have a good enough understanding of systems programming to know where
> > and how often the sendmsg is called. Of course I know this is highly
> > dependent on workload, but my knowledge is limited enough that I I can
> > convince myself both that the audit load would be not trivial but
> > still manageable in most cases but also I can convince myself that no
> > same sysadmin would consider running such an audit rule. With file IO
> > it's easy to distinguish that file opens are worth auditing but file
> > reads and writes would be insane to audit. It's not so clear for me
> > for sockets.
>
> This is going to be dependent on both the workloads and applications
> used on the system, there is no one "right" answer here.
>
> --
> paul-moore.com
>
> --
> Linux-audit mailing list
> Linux-audit@redhat.com
> https://listman.redhat.com/mailman/listinfo/linux-audit
- RGB
--
Richard Guy Briggs
Sr. S/W Engineer, Kernel Security, Base Operating Systems
Remote, Ottawa, Red
Re: Auditing nftables changes
On Thu, Mar 9, 2023 at 11:33 AM Bruce Elrick wrote:
>
> I think I need to clarify where I'm confused ;-)
>
> With iptables you could write a rule that would only catch system
> calls that were for iptables changes. That is, you didn't need to
> capture *all* setsockopt calls (not that there would be lots of
> *those*) but rather you could add the a2=64 to only get the
> op=IPT_SO_SET_REPLACE ones.
>
> With netfilter, however, since the control interface is netlink and
> netlink requires a message to a socket and messages are structs, there
> is no way to have a similarly narrow audit rule as in the case of
> iptables.
>
> That's the first thing I want to confirm: whether my understanding
> above is correct?
Yes, you are correct.
> I'm confused because your answer implies I'm correct
> but you didn't explicitly confirm that my interpretation of how it
> works was correct.
>
> You talk about having an exclude filter on NETFILTER_CFG (or rather
> exclude everything except NETFILTER_CFG??) but my understanding is
> that you can only do that filtering after the fact using ausearch or
> writing some sort of correlation code using the auparse library.
The kernel implements an exclude filter which is described in the
auditctl(8) manpage:
"Add a rule to the event type exclusion filter list.
This list is used to filter events that you do not
want to see. For example, if you do not want to see
any avc messages, you would using this list to
record that. Events can be excluded by process ID,
user ID, group ID, login user ID, message type,
subject context, or executable name. The action is
ignored and uses its default of "never".
Taken from https://man7.org/linux/man-pages/man8/auditctl.8.html
However, in my last reply I wasn't advocating for this use of the
exclude filter, I was simply trying to explain that unless you are
explicitly excluding the creation of NETFILTER_CFG records via the
exclude filter you should be seeing NETFILTER_CFG in your audit stream
with basic auditing enabled.
> It just seemed surprising that there is a non-trivial loss of audit
> functionality but that I could not find any obvious discussion about
> that. By obvious discussion I mean as explicitly as what I'm trying to
> say here.
Unfortunately it is a fairly common practice for kernel features to be
added, and removed, without consulting with the various Linux Kernel
security developers, e.g. audit, SELinux, LSM, etc. Sometimes we are
successful in retrofitting the necessary security and/or auditing
hooks, sometimes we are limited due to design choices.
> The other thing I'm trying to understand is how heavy an audit load
> would it be to have an audit rule that captures *all* sendmsg calls
> (well, all except where auid=-1 or auid=${serviceuser_uid}). I don't
> have a good enough understanding of systems programming to know where
> and how often the sendmsg is called. Of course I know this is highly
> dependent on workload, but my knowledge is limited enough that I I can
> convince myself both that the audit load would be not trivial but
> still manageable in most cases but also I can convince myself that no
> same sysadmin would consider running such an audit rule. With file IO
> it's easy to distinguish that file opens are worth auditing but file
> reads and writes would be insane to audit. It's not so clear for me
> for sockets.
This is going to be dependent on both the workloads and applications
used on the system, there is no one "right" answer here.
--
paul-moore.com
--
Linux-audit mailing list
Linux-audit@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-audit
Re: Auditing nftables changes
I think I need to clarify where I'm confused ;-)
With iptables you could write a rule that would only catch system
calls that were for iptables changes. That is, you didn't need to
capture *all* setsockopt calls (not that there would be lots of
*those*) but rather you could add the a2=64 to only get the
op=IPT_SO_SET_REPLACE ones.
With netfilter, however, since the control interface is netlink and
netlink requires a message to a socket and messages are structs, there
is no way to have a similarly narrow audit rule as in the case of
iptables.
That's the first thing I want to confirm: whether my understanding
above is correct? I'm confused because your answer implies I'm correct
but you didn't explicitly confirm that my interpretation of how it
works was correct.
You talk about having an exclude filter on NETFILTER_CFG (or rather
exclude everything except NETFILTER_CFG??) but my understanding is
that you can only do that filtering after the fact using ausearch or
writing some sort of correlation code using the auparse library. But
you are then, in this case, still capturing a haystack and, after the
fact, searching for the needle afterwards. Actually, that's a bad
analogy because ausearch easily finds the events of type=NETFILTER_CFG
very easily and then backtracks and gives you the proctitle, sockaddr
call, and sendmsg syscall associated with the type=netfilter_cfg at
which point you can look at the auid and decide what to do then.
But this is very different from what was possible with iptables where
the rule itself can filter just the iptables-related setsockopt
syscalls.
It just seemed surprising that there is a non-trivial loss of audit
functionality but that I could not find any obvious discussion about
that. By obvious discussion I mean as explicitly as what I'm trying to
say here.
The other thing I'm trying to understand is how heavy an audit load
would it be to have an audit rule that captures *all* sendmsg calls
(well, all except where auid=-1 or auid=${serviceuser_uid}). I don't
have a good enough understanding of systems programming to know where
and how often the sendmsg is called. Of course I know this is highly
dependent on workload, but my knowledge is limited enough that I I can
convince myself both that the audit load would be not trivial but
still manageable in most cases but also I can convince myself that no
same sysadmin would consider running such an audit rule. With file IO
it's easy to distinguish that file opens are worth auditing but file
reads and writes would be insane to audit. It's not so clear for me
for sockets.
Cheers...
Bruce
On Wed, Mar 8, 2023 at 8:34 PM Paul Moore wrote:
>
> On Wed, Mar 8, 2023 at 7:13 PM Bruce Elrick
> wrote:
> > Hello all,
> >
> > I'm not sure if this list is appropriate for questions so please let
> > me know and otherwise ignore if this message is not appropriate.
> >
> > I'm trying to help someone who is finally migrating from iptables to
> > nftables on the back-end and needs to therefore migrate their audit
> > capability.
> >
> > Currently they have a single simple audit rule to detect when there is
> > a iptable change from any audit user apart from their service user
> > using a rule like the accepted answer given in this[0] StackExchange
> > question, although with added filters on the auid (I have to admit I
> > don't know the origin of auid=-1 events):
> >
> > auditctl -a exit,always -F arch=b64 -F a2=64 -F auid!=-1 -F
> > auid!=${serviceuser_uid} -S setsockopt -k iptablesChange
> >
> > They are migrating from Ubuntu bionic to jammy and still using the
> > iptables front-end but since the back-end changes from default
> > iptables to default nftables they need to change their audit rules
> >
> > They did strace testing and noted the syscall changing from
> >
> > setsockopt(4, SOL_IP, IPT_SO_SET_REPLACE,
> > "filter\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
> > 80952) = 0
> >
> > to
> >
> > sendto(3, [{nlmsg_len=20,
> > nlmsg_type=NFNL_SUBSYS_NFTABLES<<8|NFT_MSG_GETGEN,
> > nlmsg_flags=NLM_F_REQUEST, nlmsg_seq=0, nlmsg_pid=0},
> > {nfgen_family=AF_UNSPEC, version=NFNETLINK_V0, res_id=htons(0)}], 20,
> > 0, {sa_family=AF_NETLINK, nl_pid=0, nl_groups=}, 12) = 20
> >
> > between the two versions.
> >
> > In my own testing, I decided to approach from the audit tools
> > perspective so I created a broad rule to capture all system call
> > related to a test user:
> >
> > auditctl -a always,exit -S all -F auid=1001 # 1001 is uid of testuser
> >
> > Then I tried various operations using my testuser such as
> > iptables-restore of either a default-accept rule set with no rules or
> > with one or two simple drop rules. I also tested adding just a single
> > iptables rule. I then used ausearch to discover what the audit system
> > captured:
> >
> > # ausearch -i -m NETFILTER_CFG
> > ...
> >
> > type=PROCTITLE msg=audit(03/07/2023 17:18:55.152:143044) :
> >
Re: Auditing nftables changes
On Wed, Mar 8, 2023 at 7:13 PM Bruce Elrick wrote:
> Hello all,
>
> I'm not sure if this list is appropriate for questions so please let
> me know and otherwise ignore if this message is not appropriate.
>
> I'm trying to help someone who is finally migrating from iptables to
> nftables on the back-end and needs to therefore migrate their audit
> capability.
>
> Currently they have a single simple audit rule to detect when there is
> a iptable change from any audit user apart from their service user
> using a rule like the accepted answer given in this[0] StackExchange
> question, although with added filters on the auid (I have to admit I
> don't know the origin of auid=-1 events):
>
> auditctl -a exit,always -F arch=b64 -F a2=64 -F auid!=-1 -F
> auid!=${serviceuser_uid} -S setsockopt -k iptablesChange
>
> They are migrating from Ubuntu bionic to jammy and still using the
> iptables front-end but since the back-end changes from default
> iptables to default nftables they need to change their audit rules
>
> They did strace testing and noted the syscall changing from
>
> setsockopt(4, SOL_IP, IPT_SO_SET_REPLACE,
> "filter\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
> 80952) = 0
>
> to
>
> sendto(3, [{nlmsg_len=20,
> nlmsg_type=NFNL_SUBSYS_NFTABLES<<8|NFT_MSG_GETGEN,
> nlmsg_flags=NLM_F_REQUEST, nlmsg_seq=0, nlmsg_pid=0},
> {nfgen_family=AF_UNSPEC, version=NFNETLINK_V0, res_id=htons(0)}], 20,
> 0, {sa_family=AF_NETLINK, nl_pid=0, nl_groups=}, 12) = 20
>
> between the two versions.
>
> In my own testing, I decided to approach from the audit tools
> perspective so I created a broad rule to capture all system call
> related to a test user:
>
> auditctl -a always,exit -S all -F auid=1001 # 1001 is uid of testuser
>
> Then I tried various operations using my testuser such as
> iptables-restore of either a default-accept rule set with no rules or
> with one or two simple drop rules. I also tested adding just a single
> iptables rule. I then used ausearch to discover what the audit system
> captured:
>
> # ausearch -i -m NETFILTER_CFG
> ...
>
> type=PROCTITLE msg=audit(03/07/2023 17:18:55.152:143044) :
> proctitle=iptables-restore
> type=SYSCALL msg=audit(03/07/2023 17:18:55.152:143044) :
> arch=x86_64 syscall=sendmsg success=yes exit=764 a0=0x3
> a1=0x7ffdb0e98db0 a2=0x0 a3=0x7ffdb0e98d9c items=0 ppid=5673 pid=5676
> auid=testuser uid=root gid=root euid=root suid=root fsuid=root
> egid=root sgid=root fsgid=root tty=pts2 ses=108 comm=iptables-restor
> exe=/usr/sbin/xtables-nft-multi subj=unconfined key=(null)
> type=NETFILTER_CFG msg=audit(03/07/2023 17:18:55.152:143044) :
> table=filter:30 family=ipv4 entries=12 op=nft_unregister_table
> pid=5676 subj=unconfined comm=iptables-restor
> type=NETFILTER_CFG msg=audit(03/07/2023 17:18:55.152:143044) :
> table=filter:30 family=ipv4 entries=7 op=nft_register_chain pid=5676
> subj=unconfined comm=iptables-restor
>
> type=PROCTITLE msg=audit(03/07/2023 17:23:04.390:144459) :
> proctitle=sudo /usr/sbin/iptables -A OUTPUT -d 10.100.249.64 -j DROP
> type=SOCKADDR msg=audit(03/07/2023 17:23:04.390:144459) : saddr={
> saddr_fam=netlink nlnk-fam=16 nlnk-pid=0 }
> type=SYSCALL msg=audit(03/07/2023 17:23:04.390:144459) :
> arch=x86_64 syscall=sendmsg success=yes exit=304 a0=0x3
> a1=0x7ffc80659110 a2=0x0 a3=0x7ffc806590fc items=0 ppid=5703 pid=5704
> auid=testuser uid=root gid=root euid=root suid=root fsuid=root
> egid=root sgid=root fsgid=root tty=pts2 ses=108 comm=iptables
> exe=/usr/sbin/xtables-nft-multi subj=unconfined key=(null)
> type=NETFILTER_CFG msg=audit(03/07/2023 17:23:04.390:144459) :
> table=filter:31 family=ipv4 entries=1 op=nft_register_rule pid=5704
> subj=unconfined comm=iptables
>
> The event sequences seem to make sense with the sockaddr function
> selecting the netlink family which agrees with the strace output.
>
> With the change in the back-end to nftables, I can see in either case
> that the setsockopt system call with a nice, crisp, single argument
> (a2=64/IPT_SO_SET_REPLACE) option with either a sendto or sendmsg
> system call but with a pointer to a message structure. I read that
> audit rules cannot filter using data inside struct arguments.
>
> My naive interpretation of this is that I'd need to have a rule that
> captures all sendmsg syscalls with (auid!=-1 and
> auid!=${serviceuser_uid} but I don't know enough about socket syscall
> usage to know whether this is too much. I see that write(2) to a
> socket is the same as send(2) without the flags so I might assume that
> most socket syscalls that are sending data use write(2) and not
> send/sendto/sendmsg(2) but I worry this would be too much audit data.
>
> Anyone care to comment or point me in the correct direction?
The problem I think you're going to have, and I believe you've already
suspected it, is that auditing socket writes is going to result in a
firehose of records. However, unless you have
Auditing nftables changes
Hello all,
I'm not sure if this list is appropriate for questions so please let
me know and otherwise ignore if this message is not appropriate.
I'm trying to help someone who is finally migrating from iptables to
nftables on the back-end and needs to therefore migrate their audit
capability.
Currently they have a single simple audit rule to detect when there is
a iptable change from any audit user apart from their service user
using a rule like the accepted answer given in this[0] StackExchange
question, although with added filters on the auid (I have to admit I
don't know the origin of auid=-1 events):
auditctl -a exit,always -F arch=b64 -F a2=64 -F auid!=-1 -F
auid!=${serviceuser_uid} -S setsockopt -k iptablesChange
They are migrating from Ubuntu bionic to jammy and still using the
iptables front-end but since the back-end changes from default
iptables to default nftables they need to change their audit rules
They did strace testing and noted the syscall changing from
setsockopt(4, SOL_IP, IPT_SO_SET_REPLACE,
"filter\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
80952) = 0
to
sendto(3, [{nlmsg_len=20,
nlmsg_type=NFNL_SUBSYS_NFTABLES<<8|NFT_MSG_GETGEN,
nlmsg_flags=NLM_F_REQUEST, nlmsg_seq=0, nlmsg_pid=0},
{nfgen_family=AF_UNSPEC, version=NFNETLINK_V0, res_id=htons(0)}], 20,
0, {sa_family=AF_NETLINK, nl_pid=0, nl_groups=}, 12) = 20
between the two versions.
In my own testing, I decided to approach from the audit tools
perspective so I created a broad rule to capture all system call
related to a test user:
auditctl -a always,exit -S all -F auid=1001 # 1001 is uid of testuser
Then I tried various operations using my testuser such as
iptables-restore of either a default-accept rule set with no rules or
with one or two simple drop rules. I also tested adding just a single
iptables rule. I then used ausearch to discover what the audit system
captured:
# ausearch -i -m NETFILTER_CFG
...
type=PROCTITLE msg=audit(03/07/2023 17:18:55.152:143044) :
proctitle=iptables-restore
type=SYSCALL msg=audit(03/07/2023 17:18:55.152:143044) :
arch=x86_64 syscall=sendmsg success=yes exit=764 a0=0x3
a1=0x7ffdb0e98db0 a2=0x0 a3=0x7ffdb0e98d9c items=0 ppid=5673 pid=5676
auid=testuser uid=root gid=root euid=root suid=root fsuid=root
egid=root sgid=root fsgid=root tty=pts2 ses=108 comm=iptables-restor
exe=/usr/sbin/xtables-nft-multi subj=unconfined key=(null)
type=NETFILTER_CFG msg=audit(03/07/2023 17:18:55.152:143044) :
table=filter:30 family=ipv4 entries=12 op=nft_unregister_table
pid=5676 subj=unconfined comm=iptables-restor
type=NETFILTER_CFG msg=audit(03/07/2023 17:18:55.152:143044) :
table=filter:30 family=ipv4 entries=7 op=nft_register_chain pid=5676
subj=unconfined comm=iptables-restor
type=PROCTITLE msg=audit(03/07/2023 17:23:04.390:144459) :
proctitle=sudo /usr/sbin/iptables -A OUTPUT -d 10.100.249.64 -j DROP
type=SOCKADDR msg=audit(03/07/2023 17:23:04.390:144459) : saddr={
saddr_fam=netlink nlnk-fam=16 nlnk-pid=0 }
type=SYSCALL msg=audit(03/07/2023 17:23:04.390:144459) :
arch=x86_64 syscall=sendmsg success=yes exit=304 a0=0x3
a1=0x7ffc80659110 a2=0x0 a3=0x7ffc806590fc items=0 ppid=5703 pid=5704
auid=testuser uid=root gid=root euid=root suid=root fsuid=root
egid=root sgid=root fsgid=root tty=pts2 ses=108 comm=iptables
exe=/usr/sbin/xtables-nft-multi subj=unconfined key=(null)
type=NETFILTER_CFG msg=audit(03/07/2023 17:23:04.390:144459) :
table=filter:31 family=ipv4 entries=1 op=nft_register_rule pid=5704
subj=unconfined comm=iptables
The event sequences seem to make sense with the sockaddr function
selecting the netlink family which agrees with the strace output.
With the change in the back-end to nftables, I can see in either case
that the setsockopt system call with a nice, crisp, single argument
(a2=64/IPT_SO_SET_REPLACE) option with either a sendto or sendmsg
system call but with a pointer to a message structure. I read that
audit rules cannot filter using data inside struct arguments.
My naive interpretation of this is that I'd need to have a rule that
captures all sendmsg syscalls with (auid!=-1 and
auid!=${serviceuser_uid} but I don't know enough about socket syscall
usage to know whether this is too much. I see that write(2) to a
socket is the same as send(2) without the flags so I might assume that
most socket syscalls that are sending data use write(2) and not
send/sendto/sendmsg(2) but I worry this would be too much audit data.
Anyone care to comment or point me in the correct direction?
Cheers...
Bruce
[0]
https://unix.stackexchange.com/questions/206891/audit-on-changes-to-the-running-iptables-configuration
--
Bruce Elrick, Ph.D.
Dedicated Support Engineer
Canonical
--
Linux-audit mailing list
Linux-audit@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-audit
