Re: pf - SCTP ports are not allowed in filter rules.
On 25 Apr 2021, at 7:56, Özkan KIRIK wrote: SCTP protocol header has src port and dst port fields. But pf doesn't supports. # echo "pass log (to pflog0) quick proto SCTP from any to any port 13873" | pfctl -f - stdin:1: port only applies to tcp/udp stdin:1: skipping rule due to errors stdin:1: rule expands to no valid combination pfctl: Syntax error in config file: pf rules not loaded # I tried to write same rule with ipfw. It works. # ipfw add 200 allow sctp from any to any 13873 00200 allow sctp from any to any 13873 Do I have a mistake or filtering for SCTP ports are not supported by pf ? Is it possible to fix ? Pf does not support SCTP in any meaningful way. I have no plans to add SCTP support either. Note that doing so involves a lot more than just teaching it to look at SCTP port numbers. Pf is a /stateful/ firewall, so we’d have to teach it the entire SCTP protocol lifecycle. Best regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: [RFC] pf ioctl changes
On 27 Mar 2021, at 12:54, Kristof Provost wrote: > I hope to post preliminary patches in the coming week. > - https://reviews.freebsd.org/D29556 - https://reviews.freebsd.org/D29557 - https://reviews.freebsd.org/D29558 - https://reviews.freebsd.org/D29559 - https://reviews.freebsd.org/D29560 - https://reviews.freebsd.org/D29561 - https://reviews.freebsd.org/D29562 Best regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: [RFC] pf ioctl changes
On 29 Mar 2021, at 17:16, Cy Schubert wrote: In message <18dc1ea9-abfc-4a06-8710-a3068370e...@freebsd.org>, "Kristof Provost " writes: On 29 Mar 2021, at 16:03, Cy Schubert wrote: In message <24e09373-ebcd-4ed1-8b59-a44e687f2...@freebsd.org>, "Kristof Provost " writes: Hi, There are several patches in the pipeline that require changes in pfâÂÂs interface between kernel and userspace. In the past these have been handled in multiple ways. Either by simply making the change, breaking binary compatibility, or by introducing a v2 ioctl (e.g. DIOCADDALTQV1). While one is better than the other neither is wholly satisfying. New versions of calls constitute a maintenance burden after all. IâÂÂd like to change the ioctl interface to use nvlists, which would make such extensions much easier, because fields can be optional. That is, if userspace doesnâÂÂt supply the âÂÂshinynewfeatureâ field the kernel can assume the default value and things just work. Similarly, if the kernel supplies a âÂÂshinynewfeatureâ which userspace doesnâÂÂt know about itâÂÂs simply ignored. The rough plan is to introduce nvlist versions of the get/add rules calls for now. Others will follow as the need presents itself. As these are new ioctls it is safe to MFC them to stable/12 and stable/13. The old interface will remain supported in those branches, but IâÂÂd like to remove it from main (and thus FreeBSD 14). As part of this effort I may end up splitting off the ioctl interface code from pfctl into libpfctl, which should make reuse of that code easier. I hope to post preliminary patches in the coming week. Thoughts? Objections? Kernel and userland should be, I'd say must be, kept in sync. We have many examples of userland and kernel not being in sync over the years. For ipfilter, I've made incompatible changes to data structures requiring userand and kernel be in sync. These are few and far between. I've gotten away with this because there is no third party software that relies on the ipfilter kernel interfaces. I could be wrong but I doubt there may be third party software requiring pf ABI compatibility. But if there is then verstioned library interfaces are required. Given that the advice is to keep kernel and userland in sync there probably is no requirement for an UPDATING entry but that would be your call. There are out-of-tree users of the pf ioctl interface. security/expiretable[1] for example. security/snort2pfcd appears to as well. sysutils/pfstat and sysutils/pftop use the ioctl interface as well, although not the three specific calls of immediate interest. This complicates things. IMO you'll probably need versioned function calls for at least 13-STABLE EOL. Or, versioning the data structures passed into the kernel such that the new fields are at the tail of the existing structures. That’s essentially the plan. I plan to keep the existing definitions (of both structure and ioctl numbers) in stable/12 and stable/13. They’ll disappear in main (i.e. 14). Alongside we’ll introduce new nvlist variants for those calls, which will have the new features. Iâm trying to work out how many examples there are, because one way or the other theyâre going to have to cope with changes. So, Iâd prefer to not just change the definitions of structs, even if weâve done that in the past. struct pf_rule contains a few peculiarities from historical mistakes that I hope to correct now. Technical debt is difficult to eliminate. We either fix it, paying it off in one lump sum or we pay it off through aggravation and design limitations, with interest, over time. Indeed. To take struct pf_rule as an example: it contains counter_u64’s, which don’t really work for userspace, so we’ve added uint64_t versions of those variables. Now the struct has two version of the same field. That can be cleaned up once the ioctls which use the struct have been removed (so on main only). My plan is to remove the struct definition from the kernel’s headers (again, once there are alternative ioctls and only in main), moving it into libpfctl. Then there will be nothing to stop us from removing the counter_u64 versions of those fields, cleaning up the struct. Given that pf uses ioctl, versioned function calls won't help. A new ioctl may be the only answer. If you do choose this, add an identifier and version number to the head of each new struct to future proof pf. The nvlist versions will be much more flexible, so embedding a version number seem redundant. Best regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: [RFC] pf ioctl changes
On 29 Mar 2021, at 16:03, Cy Schubert wrote: In message <24e09373-ebcd-4ed1-8b59-a44e687f2...@freebsd.org>, "Kristof Provost " writes: Hi, There are several patches in the pipeline that require changes in pfâs interface between kernel and userspace. In the past these have been handled in multiple ways. Either by simply making the change, breaking binary compatibility, or by introducing a v2 ioctl (e.g. DIOCADDALTQV1). While one is better than the other neither is wholly satisfying. New versions of calls constitute a maintenance burden after all. Iâd like to change the ioctl interface to use nvlists, which would make such extensions much easier, because fields can be optional. That is, if userspace doesnât supply the âshinynewfeatureâ field the kernel can assume the default value and things just work. Similarly, if the kernel supplies a âshinynewfeatureâ which userspace doesnât know about itâs simply ignored. The rough plan is to introduce nvlist versions of the get/add rules calls for now. Others will follow as the need presents itself. As these are new ioctls it is safe to MFC them to stable/12 and stable/13. The old interface will remain supported in those branches, but Iâd like to remove it from main (and thus FreeBSD 14). As part of this effort I may end up splitting off the ioctl interface code from pfctl into libpfctl, which should make reuse of that code easier. I hope to post preliminary patches in the coming week. Thoughts? Objections? Kernel and userland should be, I'd say must be, kept in sync. We have many examples of userland and kernel not being in sync over the years. For ipfilter, I've made incompatible changes to data structures requiring userand and kernel be in sync. These are few and far between. I've gotten away with this because there is no third party software that relies on the ipfilter kernel interfaces. I could be wrong but I doubt there may be third party software requiring pf ABI compatibility. But if there is then verstioned library interfaces are required. Given that the advice is to keep kernel and userland in sync there probably is no requirement for an UPDATING entry but that would be your call. There are out-of-tree users of the pf ioctl interface. security/expiretable[1] for example. security/snort2pfcd appears to as well. sysutils/pfstat and sysutils/pftop use the ioctl interface as well, although not the three specific calls of immediate interest. I’m trying to work out how many examples there are, because one way or the other they’re going to have to cope with changes. So, I’d prefer to not just change the definitions of structs, even if we’ve done that in the past. struct pf_rule contains a few peculiarities from historical mistakes that I hope to correct now. Best regards, Kristof [1] Perhaps not the greatest example, because its use of struct pf_state was incorrect, so it couldn’t actually have worked correctly before it stopped building. See PR #253547 for details. ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
[RFC] pf ioctl changes
Hi, There are several patches in the pipeline that require changes in pf’s interface between kernel and userspace. In the past these have been handled in multiple ways. Either by simply making the change, breaking binary compatibility, or by introducing a v2 ioctl (e.g. DIOCADDALTQV1). While one is better than the other neither is wholly satisfying. New versions of calls constitute a maintenance burden after all. I’d like to change the ioctl interface to use nvlists, which would make such extensions much easier, because fields can be optional. That is, if userspace doesn’t supply the ‘shinynewfeature’ field the kernel can assume the default value and things just work. Similarly, if the kernel supplies a ’shinynewfeature’ which userspace doesn’t know about it’s simply ignored. The rough plan is to introduce nvlist versions of the get/add rules calls for now. Others will follow as the need presents itself. As these are new ioctls it is safe to MFC them to stable/12 and stable/13. The old interface will remain supported in those branches, but I’d like to remove it from main (and thus FreeBSD 14). As part of this effort I may end up splitting off the ioctl interface code from pfctl into libpfctl, which should make reuse of that code easier. I hope to post preliminary patches in the coming week. Thoughts? Objections? Best regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: pfctl segmentation fault in pfctl_optimize.c
On 12 Mar 2021, at 14:00, Patrick Lamaiziere wrote: I've read the code of pfctl a bit. If pfctl crashes in pfctl_optimize_ruleset, is there a risk to leave pf in a bad state ? Looks like the rules are sent to pf via ioctl after the optimization so a crash before should be harmless (?). That should be the case, yes. I’ve not checked the pfctl code to see if it actually starts the operation to change the rules or not, but either way, pf rule changes are atomic. They either succeed completely or not at all. Pf accomplishes this by keeping an active and inactive ruleset, and when you load new rules pfctl will start a transaction (DIOCXBEGIN), add the complete new ruleset (DIOCADDRULE) and only then commit to swapping the active and inactive rulesets (DIOCXCOMMIT). Best regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: pfctl segmentation fault in pfctl_optimize.c
On 9 Mar 2021, at 11:05, Patrick Lamaiziere wrote: Hello, FreeBSD 11.4-RELEASE-p3 / amd64 Yesterday while loading a ruleset, pfctl core dumped with a segmentation fault (see gdb below) We are recently using some big tables so may be this is what triggered the problem (?), i can't reproduce this. I've found something on t...@openbsd.org that looks closely related: https://www.mail-archive.com/tech@openbsd.org/msg42870.html At first glance that looks like a sane change, but I can’t reproduce the crash described there. Can you reproduce your crash? I try to avoid making changes I can’t write a test for. Best regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: "set skip on lo" on 12.x and 13.0
On 9 Feb 2021, at 15:50, Marek Zarychta wrote: Dear list, I am observing changed behaviour of the rule "set skip on lo". This rule previously allowed for communication between the host and the jail no only on loopback interfaces, but also on shared network interfaces, for example, if a host had address x.x.x.x/24 and jail had address x.x.x.y/32 on the same NIC, the rule above allowed for communication between the host and jail using x.x.x.x and x.x.x.y addresses. I am considering jails without VNET enabled and using the same fib number. Now to allow this kind of communication I had to add "pass quick on lo", but I went out of free states rather quickly, so instead of increasing the state limit, I have changed the method of communication between the host and the jails to utilize only loopback addresses. It's rather not a regression but a change, some people might consider it POLA violation, but probably won't if it gets widely announced. I’m not aware of the behaviour change you describe. However, there have been subtle issues around set skip on that may be confusing you. See #250994 / 0c156a3c32cd0d9168570da5686ddc96abcbbc5a for some of the details. Best regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: pflog0 showing up in my vnet jails
On 3 Feb 2021, at 7:11, R. Tyler Croy wrote: I noticed this evening that pflog0 is propagated into my vnet-based jails (12.2-RELEASE) and I'm somewhat surprised to see it there. My host's /etc/rc.conf simply has `pflog_enable="YES"`, so nothing too esoteric. My /etc/jail.conf doesn't do anything with pflog0 for the jails, so the fact that it shows up _feels_ like a bug, from within a jail: # ifconfig lo0: flags=8049 metric 0 mtu 16384 options=680003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 inet 127.0.0.1 netmask 0xff00 groups: lo nd6 options=21 pflog0: flags=0<> metric 0 mtu 33160 groups: pflog epair2b: flags=8843 metric 0 mtu 1500 options=8 ether 02:c4:52:c8:47:0b inet 10.0.1.4 netmask 0xff00 broadcast 10.0.1.255 groups: epair media: Ethernet 10Gbase-T (10Gbase-T ) status: active nd6 options=29 # Fortunately, when I tcpdump that device from within the jail, it has none of the host pflog0's entries being reported. Regardless, should I file this as a bug? I wouldn’t consider this to be a bug, no. Or if it is one, one that won’t be fixed anyway. As soon as the pflog module is loaded pf creates a pflog0 interface. That interface is per-vnet, so it’s perfectly safe to have. Arguably pf shouldn’t create a log interface automatically, but that ship has sailed. If we change it we’re going to break expectations for at least some users, so we’re not going to change that. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: PF not keeping counters in a counters-defined table
On 5 Jan 2021, at 20:35, Dobri Dobrev wrote: You are correct, Kristof. If I place the table in the rdr rule - it starts keeping counters, however, what is the point of having the ability to place a table in a rdr-anchor rule in the first place, if it won't be able to keep counters? Tables are not just about counters. They’re about making a rule filter on a whole selection of addresses (or ranges). In this case you’re choosing to filter what traffic may go into the anchor. Maybe consider not filtering on the rdr-anchor rule, but on the rdr rule in the anchor itself? I'm doing the followi ng scenario: table counters table persist rdr-anchor "ASDFGH" on igb0 proto tcp from to any port 123 no-rdr on igb0 from any to port 123 rdr-anchor "ASDFGH" on igb0 proto tcp from any to any port 123 load anchor ASDFGH from "/etc/ASDFGH-anchor" # contents of /etc/ASDFGH-anchor: # (tested separately) # rdr on igb0 proto tcp from any to 192.168.0.1 port 123 -> 192.168.0.1 port 124 # no counters # rdr on igb0 proto tcp from to 192.168.0.1 port 123 -> 192.168.0.1 port 124 # counters working So, in this case - how do I keep counters in the without breaking the current "workflow"? If IP 192.168.0.1 is not in and I have on all rdr rules @ the anchor - I won't ever be able to reach 123->192.168.0.1:124 Is there a way? I have no idea, and I’m not the best person to talk to about how to configure your firewall. Best regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: PF not keeping counters in a counters-defined table
On 5 Jan 2021, at 14:42, Dobri Dobrev wrote: # # /etc/pf.conf: set timeout tcp.first 45 set timeout tcp.opening 45 set timeout tcp.closing 15 set timeout tcp.finwait 15 set timeout tcp.closed 10 set timeout interval 10 set timeout tcp.established 3600 set timeout src.track 10 set limit table-entries 50 set limit states 200 set limit src-nodes 200 set require-order no set block-policy drop set ruleset-optimization basic set skip on lo0 table counters rdr-anchor "ASDFGH" on igb0 proto tcp from to any port 123 load anchor ASDFGH from "/etc/ASDFGH-anchor" # contents of /etc/ASDFGH-anchor: # rdr on igb0 proto tcp from any to 192.168.0.1 port 123 -> 192.168.0.1 port 124 # Use pflog to confirm, but I’m pretty sure your issue is that you’re hitting the rdr rule in the anchor, which doesn’t contain the table with the counters rather than the anchor rule. Counts are only done on the final matching rule, not on all of the rules looked at along the way. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: net.pf.request_maxcount not working after upgrading from 12.1-RELEASE to 12.2 RELEASE.
On 12 Dec 2020, at 15:07, Franco Fichtner wrote: There's no reason for this to be a tunable. It's perfectly safe to change this at runtime. Well, RWTUN would have enabled both boot and runtime which is also "perfectly safe". :) Good idea. Done in 368588. I expect to be bothering people about an EN for the vnet/epair issue next week, I’ll see if I can include this in the bothering. Best regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Packets passed by pf don't make it out?
On 14 Oct 2020, at 21:35, J David wrote: On Wed, Oct 14, 2020 at 3:20 PM Kristof Provost wrote: I’ve not dug very deep yet, but I wonder if we shouldn’t have to teach pf to change the source port to avoid conflicting states in the first place. That was my first thought as well, framed mentally as some sort of port-only Frankenstein's binat because my level of understanding is clearly more cartoonish than yours. ;-) My second thought was to wonder if my approach is architecturally wrong. Would it make sense for the many-to-many case to use route-to instead of rdr, leave the packet unmodified, and expect every machine in the server pool to catch all the public IPs? That might still be tricky. Using rdr would presumably hit the same problem. Maybe something gross like ifconfig'ing the public pool addresses as /32's on lo0, then binding on those, maybe? I honestly don’t know. The pf NAT/RDR/… code is complex, and I certainly don’t understand all edge cases. It may be worth experimenting with such options though, because this is unlikely to be fixed short-term. Best regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Packets passed by pf don't make it out?
On 14 Oct 2020, at 21:16, J David wrote: On Wed, Oct 14, 2020 at 1:59 PM Kristof Provost wrote: There’s good reason to do this, as we have to be able to match state on both the pre-translation side (when processing LAN -> WAN traffic) and post-translation (WAN -> LAN). So, basically, pf would need separate states for each pre-redirect destination address in order to have the information needed to map the reply packet back to the original destination address. But even if pf did that, the problem does not go away. It just moves to the reply packet coming back with only the post-redirect info. That info matches multiple states, leaving pf no way to pick the right one. Is that about right? Pretty much, I think. I’ve not dug very deep yet, but I wonder if we shouldn’t have to teach pf to change the source port to avoid conflicting states in the first place. It’s a non-trivial problem in any case. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Packets passed by pf don't make it out?
On 14 Oct 2020, at 18:52, J David wrote: On 12 Oct 2020, at 23:48, Andreas Longwitz wrote: pf gives this messages in debug mode (pfctl -x loud). Yes, with that setting I'm also seeing those messages. On Tue, Oct 13, 2020 at 5:35 PM Kristof Provost wrote: I see the same ‘stack key attach failed’ error message. My current thinking is that we’re hitting a state collision, because post-RDR our connection information is the same (192.168.14.10:23456 192.168.14.100:12345). That means we can’t create a new state, and the packet gets dropped. This is probably a dumb question because I know less than nothing about pf internals, but why wouldn't it match the existing state? “It’s complicated”. In essence, pf tracks both the pre- and post-translation tuple, so what we’re seeing here is one of those conflicting with an existing session and that’s causing the failure. There’s good reason to do this, as we have to be able to match state on both the pre-translation side (when processing LAN -> WAN traffic) and post-translation (WAN -> LAN). Best regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Packets passed by pf don't make it out?
On 12 Oct 2020, at 23:48, Andreas Longwitz wrote: Hello, now I can confirm (on FreeBSD 10 Stable) what you see on fb2 when your program udp_client is running on fb1. pf creates a state for the first packet only, for the other packets pf failes to create a state with messages like pf: stack key attach failed on re0: UDP in wire: 192.168.14.10:23456 172.16.0.2:12345 stack: 192.168.14.10:23456 192.168.14.100:12345 1:0, existing: UDP in wire: 192.168.14.10:23456 172.16.0.1:12345 stack: 192.168.14.10:23456 192.168.14.100:12345 1:0 pf gives this messages in debug mode (pfctl -x loud). I do not know if we see a bug in pf or if your program udp_client does something illegal, I think Kristof can tell us. Your confidence is both flattering and misplaced :) I think I can reproduce the problem on CURRENT and with VNET jails, which is convenient. I see the same ‘stack key attach failed’ error message. My current thinking is that we’re hitting a state collision, because post-RDR our connection information is the same (192.168.14.10:23456 192.168.14.100:12345). That means we can’t create a new state, and the packet gets dropped. It’s a little unusual for a client to keep re-using src ports like that, but it’s not actually wrong. I’m not sure how we can fix this. Best, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: The best of both worlds “using mac filtering in pf”
On 10 Jul 2020, at 22:37, Ultima wrote: Hey Kristof, (It’s already possible to use pf on top of a bridge in bump-in-the-wire mode. Given the gotchas in that code I **strongly** recommend people don’t use that functionality.) Do you mind going into details on the gotchas or providing links? I am reluctant to, because people will delude themselves into believing they can avoid the landmines. The entire way this feature is implemented is wrong, and you cannot reliably avoid the landmines. If you use it at some point you will find yourself spread out over the landscape. That said, very briefly, (and understand that it **will** blow up in your face when it’s most annoying): the way this feature works is by stripping off the ethernet header, passing the IP packet to pf, and then re-adding the ethernet header once pf is done with it. This explodes spectacularly if you do something that causes the packet to not be returned by pf, such as a route-to/reply-to rule, or anytime IPv6 fragmentation is involved. Best regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: The best of both worlds “using mac filtering in pf”
On 10 Jul 2020, at 19:57, l.m.v.br...@xs4all.nl wrote: Hello, I am using pfSense, build on top of pf. And of course pfSense/pf is a terrific firewall, however the world is changing in the direction of IPV6 and that leads to new issues and related new requirements. One of the major issues is that IPV6 does not provide a stable source address you can use to filter in your firewall. Many firewalls “out there” are *using the level-2 mac as a way around this issue*. � However ….. pfSense cannot provide that functionality, since it is built on top of …… pf. Tja, and then there is a “striking” issue ….. suppose that pfSense would have been built on top of OpenBSD, still using pf ………. That had been possible ……. So as user I would be very pleased if there could be a joined “pf-release” having *best of both worlds* Assume we were running OpenBSD …… things like � � step-1: ifconfig bridge0 rule pass in on fxp0 src tag step-2: And then in pf.conf: pass in on fxp0 tagged (policy based rule) would have been an option, …. not saying it is the best option ….. �better option would be if pf could set the tag itself Whatever please consider adding this functionality to pf preferable on short term, since IPV6 is fast becoming very important! Sincerely, � Louis PS … should I raise an feature request for this? You can, but adding L2 filtering functionality to pf isn’t even on my long-term todo list. It is essentially out of the question that it’d be added in the short term (or even in the next year or two, unless someone decides it’s worth contracting me for several months to do it). I don’t personally see the use case for it either, but perhaps I’m missing something. Can you explain what exactly you’d like to accomplish with L2 filtering? (It’s already possible to use pf on top of a bridge in bump-in-the-wire mode. Given the gotchas in that code I **strongly** recommend people don’t use that functionality.) Best regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: pf - state counter tracking like pfsync
On 26 Jun 2020, at 13:56, Özkan KIRIK wrote: My goal is save pkt/byte counters of each expired/killed/closed states into a txt file. What is the right way to do this in userspace ? There’s no real right way to do this using pf. There are a couple of things that’ll get close, but no 100% solution. Is it possible to do something with ioctl & poll ? No. You could poll the states, but you’d heavily affect throughput and you’re going to miss data. Alternatively is it possible to create multiple pfsync interfaces, first one for real purpose to send state changes to slave host, the second one for sending this log collect process lo1? No, it’s not possible to create more than one pfsync interface. Pfsync can send its data to a multicast group, so you could have multiple subscribers. Note that pfsync optimises updates, so it’s likely that short-lived connections (i.e. where the connection is set up, used and closed before the next sync) will not result in sync messages. Following lines prevents cloning second pfsync interface: /usr/src/sys/netpfil/pf/if_pfsync.c on line 331 (pfsync_clone_create function) if (unit != 0) return (EINVAL); If I remove these lines, do I hit any error ? Yes, that will break. Pfsync is not designed to have multiple interfaces. Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Need a PF consultant
On 22 Jun 2020, at 2:06, David Mehler wrote: Thanks for all your replies. Donald, the IPv6 dns is working fine in this situation. Kristof, here's what I originally had in my pf.conf file for ICMP: pass out quick on $ext_if proto { icmp, icmp6 } modulate state pass in quick on $ext_if proto { icmp, icmp6 } That’s a somewhat bigger hammer than what I proposed, but that should work as well. I commented that out, added in your rules, disabled and reenabled PF, and did a ping6. Good news is the first time I tried ping6 it worked, bad news is the second time I tried it about two minutes later it sent out the ping6 but didn't return anything, zero packets received. A few minutes later doing the UDP connect no route to host thing again. While the original focus of my question was IPv6 would you be willing to assist me with my general configuration? As I said I can go in to much more detail on this. Please do send me the information you have, yes. Best regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Need a PF consultant
On 21 Jun 2020, at 23:11, David Mehler wrote: Anyone a pf expert wanting to make some extra money? I'm in need of consulting, I'm having an issue with my PF configuration, I've got a much longer message with output and netstat and all that, if anyone is interested email me privately with rates and I'll send the details, but in brief I've got FreeBSD 12.1 going, my provider gives me an IPv6 address, on boot if I ping6 out I get a UDP connect no route to host message, disable and reenable pf and I can ping6 out as root, and as a user, for about five minutes, then I start getting packets are able to be sent out, but nothing comes back. About five minutes later again as root and as a user I'm getting the UDP connect no route to host message. That sounds a lot like you’re dropping router and/or neighbour advertisements. Make sure you’ve got at least the following pass rules: # IPv6 link-local traffic pass quick inet6 proto icmp6 from :: to ff02::/16 pass quick inet6 proto icmp6 from fe80::/10 to fe80::/10 pass quick inet6 proto icmp6 from fe80::/10 to ff02::/16 # IPv6 Traffic That Must Not Be Dropped (RFC4890 4.3.1) pass quick inet6 proto icmp6 from any to any icmp6-type { unreach, toobig } pass quick inet6 proto icmp6 from any to any icmp6-type timex code 0 pass quick inet6 proto icmp6 from any to any icmp6-type { paramprob code 1, paramprob code 2 } pass quick inet6 proto icmp6 from any to any icmp6-type { echoreq, echorep } # IPv6 Traffic That Normally Should Not Be Dropped (RFC4890 4.3.2) pass quick inet6 proto icmp6 from any to any icmp6-type timex code 1 pass quick inet6 proto icmp6 from any to any icmp6-type paramprob code 0 # IPv6 local configuration (ND, DAD, RS, etc...) pass quick inet6 proto icmp6 from any to any icmp6-type { routersol, routeradv } pass quick inet6 proto icmp6 from any to any icmp6-type { neighbrsol, neighbradv } pass quick inet6 proto icmp6 from any to any icmp6-type { 141, 142 } pass quick inet6 proto icmp6 from any to any icmp6-type { listqry, listenrep, listendone, 143 } pass quick inet6 proto icmp6 from any to any icmp6-type { 148, 149 } pass quick inet6 proto icmp6 from any to any icmp6-type { 151, 152, 153 } At a guess the routersol/routeradv and neighbrsol/neigbradv are the ones you’re running into, but you likely want to allow all of these. Best regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: ALTQ feature of PF in FreeBSD
On 29 Feb 2020, at 0:35, Sean Yeh wrote: Hi FreeBSD-pf members, I hope you guys are enjoying your weekend! I was wondering if any of you happened to know if the code for the ALTQ feature of pf could be separated and used for NetBSD's pf function. I'm currently investigating methods to improve NetBSD's ALTQ feature, which hasn't been updated in 15+ years: https://wiki.netbsd.org/projects/project/altq/ According to the man pages of freeBSD's pf function, FreeBSD uses a modified pf of openBSD 4.5 pf function. Are there any complications that you foresee trying to port FreeBSD's current ALTQ code into NetBSD? Thank you for all your help, It might be more interesting to look at dummynet. Last year there was a GSoC proposal (in FreeBSD) to port dummynet to pf. Sadly the project wasn’t selected, but I believe the student did spend some time on investigating it. I’ll ask them to get in touch with you. Best regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Updating our translation functionality
On 27 Feb 2020, at 10:08, J.R. Oldroyd wrote: I read back and found the thread last August "Update to PF from OpenBSD 6.5". I was going to ask the same thing but, given the complexities discussed in the responses there, perhaps the question should be asked a different way round. How much work would it be to add in OpenBSD's latest translation functionality to our implementation? OpenBSD's pf has new translation functionality, specifically nat64 support using the "af-to" syntax. At the same time, existing translation syntax was changed with the nat, binat and rdr rule syntax changing to "pass ... nat-to ..." etc. I think it is good that we are still called "pf" here and that we do try to maintain compatibility with other pf implementations. So, we should consider adding the new translation functionality to our implementation. Understood that this means requiring changes to existing pf.conf configurations but these can be documented with examples and announced in advance. How big of a project would this be? I don’t know. I’ve not specifically investigated the nat64 bits, and they’re (to me) the least interesting bits as well. It’s possible that they can be imported without too much trouble, but someone would have to sit down and spend the time on it. Right now this isn’t even on my todo list and I’m not planning to add it either. Given that this change would break compatibility with existing configurations (unless significant extra work is done to cope with this) I’m not keen on it. I’d need to see very good arguments for introducing an intermediate painful step between the current situation and a state where we have the same syntax as OpenBSD. If you’re looking for nat64, IPFW has an implementation. Best regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Flow of broadcast/multicast packets in pf when a bridge is present
> On 28 Dec 2019, at 12:52, Andreas Longwitz wrote: > > In the meantime I have understand I was wrong about the code snippet > >> mc2 = m_dup(m, M_NOWAIT); >> if (mc2 != NULL) { >> /* Keep the layer3 header aligned */ >> int i = min(mc2->m_pkthdr.len, max_protohdr); >> mc2 = m_copyup(mc2, i, ETHER_ALIGN); >> } >> if (mc2 != NULL) { >> mc2->m_pkthdr.rcvif = bifp; >> (*bifp->if_input)(bifp, mc2); >> } > > My mistake concerned the function call m_copyup(): The mbuf chain is > copied correct and not shortened, I was confused because of the field > m_len in mc2. So reinjecting the packet in the bridge is ok. > > Another aspect is what is done next with the broadcast/multicast packet > handled by this code: > >> /* Return the original packet for local processing. */ >> return (m); > > Therefore local processing on the member interface is done for > broadcast/multicast packets without checking the pfil_local_phys > variable. That was confusing me because these packets are counting twice > in the pf rules. I think this is needless and pfil_local_phys should > respect all packets not only unicast. > > After introducing the patch > > --- if_bridge.c.iorig2019-05-14 09:43:33.0 +0200 > +++ if_bridge.c 2019-12-28 11:54:52.0 +0100 > @@ -2386,6 +2386,10 @@ > (*bifp->if_input)(bifp, mc2); > } > > + if (!pfil_local_phys ) { > + m_freem(m); > + return (NULL); > + } > /* Return the original packet for local processing. */ > return (m); > } > > everything works fine and all the counters in pf have values as expected > Can you put that in Phabricator and cc me and kevans@? (I seem to remember him touching related code a few months ago). Thanks, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Rule last match timestamp
On 27 Dec 2019, at 21:49, Franco Fichtner wrote: Hi, On 27. Dec 2019, at 6:45 PM, Kristof Provost wrote: What are you trying to accomplish? Some people believe that "last match" is a great metric to audit rules for intrusion detection and all sorts ruleset optimisation and refinement. In OPNsense the question has popped up a few times to support it, but without doing it in pf(4) directly it makes little sense as you'd have to crawl pflog output and even then you can't crawl non-log rules this way... Would SDT probe points be useful for this? I have a background todo item to add those where they’d be meaningful. They have the advantage of not really having a cost when they’re not active, of being really easy to add, and of not imposing ABI changes. Best regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Rule last match timestamp
On 26 Dec 2019, at 1:20, Özkan KIRIK wrote: > Hi, > > I need last match timestamps for each rule. ipfw has an option for this. > But pfctl -v -sr command doesnt show last match timestamp. > Is there way to gather this information in pf? > Pf does not track this. What are you trying to accomplish? Best regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Blocking SYN with data
On 26 Dec 2019, at 1:13, Özkan KIRIK wrote: Hi, I want to block SYN with data packets. I read the pf.conf manual, but couldn't find a clear way to do this. Is it possible to match packets greater then N bytes using pf on FreeBSD 12.1 stable? There isn’t a way to express this in pf right now. Does synproxy state or modulate state perform this operation? I’ve had a quick look at the code, and I’m somewhat surprised to find that pf doesn’t stop this by default. There may be good reasons for this, or perhaps it’s not considered to be a problem (i.e. it doesn’t happen often, and host stacks discard it anyway). I’ve not gone through the sync-proxy code flow, but I’d expect that to prevent this from happening. Why are you concerned about it? Best regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: PF frag entries limit reached on a server with hw.ncpu: 24
On 23 Dec 2019, at 12:00, Andreas Longwitz wrote: On one of my servers a saw some messages dssinet kernel: [zone: pf frag entries] PF frag entries limit reached The output of the command vmstat -z | grep "pf frag entries" was pf frag entries:40, 5000,0, 5000, 18760, 0, 0 So there are 5000 free items, none is in use and the limit of 5000 is reached. This situation is possible, when all the free 5000 items are hold in the caches of e.g cpu 0,1..20 and now a thread running on cpu 21 wants to allocate an item of the zone pf_frent_z. The alloc fails because of the limit and the zone allocater from time to time logs the error message "limit reached". After putting the line set limit frags 1 in pf.conf the problem was gone, now I have 5700 free items and this value did not change for some weeks. For a server with 24 cpu the default PFFRAG_FRENT_HIWAT = 5000 was too small. Maybe this default value should be adjusted. I’m happy to tweak defaults, but I’m not sure there’s a good way to figure out what a sane value is. Possibly we’d want a heuristic based on the RAM size, but that too is not ideal: a firewall device will reasonably want to dedicate more of its memory to pf than a file server. I’d prefer to have a holistic view of what defaults make sense for these limits before we start changing them. Best regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: pf tracker
On 6 Sep 2019, at 2:51, ravi new wrote: In packet filter rule ,there is a text called "tracker" what is use of this? I’ve not seen ‘tracker’ before in pf rules. Do you have an example? Can i find syntax of packet filter rules of freebsd. man 5 pf.conf Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Update to PF from OpenBSD 6.5
On 20 Aug 2019, at 12:32, Goran Mekić wrote: > On Tue, Aug 20, 2019 at 11:49:18AM +0200, Kristof Provost wrote: >> One thing I’ve thought of trying, and that might be an interesting stepping >> stone, is to create a port (/usr/ports/net/opf or whatever) of OpenBSD’s pf. >> In that version it’d be acceptable to not fix any of the above issues. It’d >> still give users to option of getting the new syntax. I’d expect this to be >> a relatively straightforward exercise. > That would be cool, but only if FreeBSD PF can not be "fixed" to support > OpenBSD PF syntax. > The main issue there is one of compatibility. How happy will our users be if their rulesets suddenly stop working after an upgrade? Anyway, none if this is on my active todo list. Don’t expect to see it any time soon. >> In principle there’s nothing to stop us from doing that same work in base, >> but we’re **NOT** going to import a fourth firewall. We’re just not. > Are you sure? https://2019.eurobsdcon.org/talk-speakers/#NPF. At least I > hope the import is pfil based. > I don’t know what George’s plans are exactly, but it’s likely that he’s doing the porting work to get an apples-to-apples comparison of firewall performance, not because he wants to maintain another firewall. Either way, I’m not pushing for another firewall. George gets to own one if he wants to. Regards, Kristof signature.asc Description: OpenPGP digital signature
Re: Update to PF from OpenBSD 6.5
On 20 Aug 2019, at 11:36, Tom Marcoen wrote: Hey all, I'm quite new to FreeBSD so appologies if this is a stupid question. Is there a good reason for not upgrading PF to the version from OpenBSD 6.5? There are several reasons why updating pf is a non-trivial problem. From an e-mail I sent on this subject in April: It’s a good goal, but there are three major issues along the way to importing the latest OpenBSD version. (And I’m sure a whole bunch of smaller ones.) Those are: - scalability - syntax - vimage The scalability issue is the obvious difference: OpenBSD’s pf is still very much single-core oriented, whereas the FreeBSD pf version can cope with multiple cores (somewhat) and is significantly faster on multicore hardware. Our version is by no means perfect, but it’s much faster than OpenBSD’s version. Much of the imperfections we have now is there because pf was designed in a giant locked kernel in the first place. Multi-core scalability was not part of its original design. Adopting OpenBSDs pf would mean redoing all of the locking work Gleb did years ago. Given the differences in OpenBSD’s pf (e.g. they keep states in a tree, not a hash table) it’s not a matter of replaying the previous work on a new pf version. This is a from the ground up introduction of fine grained locking in a code base that assumes a single giant lock. As I understand it the OpenBSD people are working on the problem as well, but I’ve not seen any diffs yet. If they’ve made significant progress we may be able to base our work on theirs. I don’t think it’d be acceptable to not have this, as it’d mean a very large performance regression. For reference, before I did the pfsync work we essentially had a single-threaded pf when pfsync was enabled. On my test hardware this meant a throughput of ~1.1Mpps, rather than the ~3.9Mpps without pfsync. I’d expect OpenBSDs pf to perform at around that ~1.1Mpps number without locking work. The second issue is one of syntax, and that’s what I assume is the main reason people want OpenBSDs pf. We’re still on an older iteration of the pf syntax, but changing that would inevitably mean breaking old configurations. That might be acceptable on a major version update (i.e. going into 13), but would mean the new work could never be backported. That’s probably the only way forward though. I’m playing with importing the ‘match’ keyword and not breaking the ‘scrub’ syntax at the same time, but it’s a non-trivial problem, and that’s only one of the steps along the way. Finally there’s vimage. That’s a FreeBSD-only feature, so naturally OpenBSDs pf is not aware of this. I expect that to be relatively easy to add back in, but it’s another obstacle. As vimage is what lets us have the pf tests we’ve got now I’d be very reluctant to let it go. It’s a supported feature in 12.0, so users will start to rely on it as well. TL;DR: It’s possible, but *hard*. Expect is to be many person-months of effort, and there’s no way it’s going to be a smooth ride. One thing I’ve thought of trying, and that might be an interesting stepping stone, is to create a port (/usr/ports/net/opf or whatever) of OpenBSD’s pf. In that version it’d be acceptable to not fix any of the above issues. It’d still give users to option of getting the new syntax. I’d expect this to be a relatively straightforward exercise. In principle there’s nothing to stop us from doing that same work in base, but we’re **NOT** going to import a fourth firewall. We’re just not. Currently it seems to be at the version from OpenBSD 4.1. That’s so simplistic as to be outright wrong. We’ve not done wholesale imports from OpenBSD in a long time, yes, but FreeBSD’s pf is maintained, and regularly gets new features and bug fixes. Fixes even flow in both directions between OpenBSD and FreeBSD. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: pf and dummynet
On 29 Jul 2019, at 22:15, Rodney W. Grimes wrote: >> On 29 Jul 2019, at 20:22, mike tancsa wrote: >>> On 7/29/2019 1:51 PM, Kristof Provost wrote: >> In general I?d expect quality of service and bandwidth limits to only >> be effective in the upstream direction (when going from a fast link to a >> slow one). There?s no good way to limit how much traffic other >> machines send to you. > > Though dummynet is most effective in on the outbound > stream (absolute control) it can be used to good effect > on an incoming stream due to the end-to-end paradigm of > the internet and the fact that congestion must be dealt > with. > > If dummynet holds packets and parcels them into a box at > a lower rate for things like TCP you'll end up reducing > the congestion window and hence the senders rate. Or you > can get into the ACK clock situation here the sender simply > does not send any more data until it gets an ack back as > it already has filled the congestion window. > > I have been using dummynet for decades in this way, > and it more or less "just works." > True, with the caveat that that’s only for TCP of course. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: pf and dummynet
On 29 Jul 2019, at 20:22, mike tancsa wrote: On 7/29/2019 1:51 PM, Kristof Provost wrote: Also beware of gotchas with things like IPv6 fragment handling or route-to. I do not consider mixing firewalls to be a supported configuration. If it breaks you get to keep the pieces. Thanks, I was worried about that! Is there a way to get altq to limit inbound traffic directed to a server ? I would prefer not mixing and matching, but I dont see any other way other than going to ipfw which I would rather not I don’t know. I’m not very familiar with altq. In general I’d expect quality of service and bandwidth limits to only be effective in the upstream direction (when going from a fast link to a slow one). There’s no good way to limit how much traffic other machines send to you. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: pf and dummynet
> On 2019-07-29 18:44:00 (+0100), Paul Webster via freebsd-pf > wrote: > > > > Sent from Mail for Windows 10 > > > > From: mike tancsa > > Sent: 29 July 2019 17:06 > > To: freebsd-pf@freebsd.org > > Subject: pf and dummynet > > > > I have a box I need to shape inbound and outbound traffic. It seems altq > > can only shape outbound packets and not limit inbound ? If thats the > > case, what is the current state of mixing ipfw, dummynet and pf ? > > Writing large complex firewall rules works better from a readability POV > > (for us anyways) so I really prefer to use it. But I need to prevent zfs > > replication eating up BW over some WAN links, and dummynet seems to > > "just work" > > > > For ipfw I have > > > > > > 00010 6640359 9959147882 pipe 1 tcp from 192.168.128.0/20 to any > > 01000 3486901 228480912 allow ip from any to any > > > > and then checking my pf.conf rules, it seems to block and pass traffic > > as expected. > > > > Is there anything I should explicitly check ? > > > You can mix ipfw and pf, but beware of the order they are loaded (The > first one loaded is inside the second one loaded) – it may be better > in fact to compile them both in the kernel. > > You basically end up with: (pf)(ipfw)(system)(ipfw)(pf) – assuming pf > was loaded first Also beware of gotchas with things like IPv6 fragment handling or route-to. I do not consider mixing firewalls to be a supported configuration. If it breaks you get to keep the pieces. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: passthru ethernet interface
> On 22 May 2019, at 04:53, The Doctor via freebsd-questions > wrote: > > How do I set up in /etc/rc.conf an eth1 that will > attach ifself to the back of the router in my packet filtering (pf) box, > namely the $ext_if ? > You want to set a macro in your pf.conf, as documented in its man page. — Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: svn commit: r345760 - in head: contrib/pf sys/netpfil/pf sbin/pfctl
On 1 Apr 2019, at 23:06, Rodney W. Grimes wrote: On 1 Apr 2019, at 18:47, Rodney W. Grimes wrote: Those are: - scalability The project funding source is OS agnostic, would it help if the OpenBSD pf implementation was redone in a way that it had fine grained locking. Would it be possible to apply Gleb's work as a upstreaming operation to get this work in both implementations? It might be useful to help OpenBSD do their locking work first. Or at least talk to them and ask what the state of it is. I had a very quick look at their current tree yesterday evening, and found that while they’re working on the locking there’s a long way left to go. We can’t just take our existing locking work and apply it to OpenBSD’s tree. There are too many differences for that to be viable. One example is the state lookup. In our code that’s a hash table, and we lock the individual hash rows. In OpenBSD’s tree it’s a red-black tree, which can’t be trivially split up as we’ve done. The second issue is one of syntax, and that?s what I assume is the main reason people want OpenBSDs pf. We?re still on an older iteration of the pf syntax, but changing that would inevitably mean breaking old configurations. That might be acceptable on a major version update (i.e. going into 13), but would mean the new work could never be backported. That?s probably the only way forward though. I?m playing with importing the ?match? keyword and not breaking the ?scrub? syntax at the same time, but it?s a non-trivial problem, and that?s only one of the steps along the way. Isnt there more than just syntax? I thought OpenBSD pf has features that are not present in FreeBSD's pf implementation. There are feature differences too, yes, but the syntax is the most visible point. We can easily add extensions to the pf.conf language without worrying about breaking existing configurations. That’s not the case with the syntax changes. I’ve already brought over a couple of smaller features. That’s relatively straightforward. (Hence also my question about what users actually want. If that’s smaller features it’s a *much* easier and less invasive process.) Finally there?s vimage. That?s a FreeBSD-only feature, so naturally OpenBSDs pf is not aware of this. I expect that to be relatively easy to add back in, but it?s another obstacle. As vimage is what lets us have the pf tests we?ve got now I?d be very reluctant to let it go. It?s a supported feature in 12.0, so users will start to rely on it as well. TL;DR: It?s possible, but *hard*. Expect is to be many person-months of effort, and there?s no way it?s going to be a smooth ride. So your quantifying this at man months and not man years, that helps a bunch, that is clearly within the scope of the funding. Probably, yes. With the usual caveats and disclaimers. One thing I?ve thought of trying, and that might be an interesting stepping stone, is to create a port (/usr/ports/net/opf or whatever) of OpenBSD?s pf. In that version it?d be acceptable to not fix any of the above issues. It?d still give users to option of getting the new syntax. I?d expect this to be a relatively straightforward exercise. In principle there?s nothing to stop us from doing that same work in base, but we?re **NOT** going to import a fourth firewall. We?re just not. But 4, its nice, its a power of 2, infact it is the second power of 2, such nice round numbers :-) Don’t make me roll up the newspaper. We’re not adding firewalls. Or I’m not, in any case. I do like the idea of a first implementation of pf as a port, that being a fast path to update it, but would not want that as a long run solution. It’d likely also be a helpful step in adding the locking to the OpenBSD code. I’m not sufficiently familiar with OpenBSD to know if they’ve got equivalents to our lockstats/pmcstat/dtrace tools, and those are very nice to have for locking work. Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: svn commit: r345760 - in head: contrib/pf sys/netpfil/pf sbin/pfctl
On 1 Apr 2019, at 18:47, Rodney W. Grimes wrote: I know for a fact that there is desire, with financials avaliable, to get our code updated. I do not think there is any specific criteria desired, other than moved closer to the OpenBSD version. It’s a good goal, but there are three major issues along the way to importing the latest OpenBSD version. (And I’m sure a whole bunch of smaller ones.) Those are: - scalability - syntax - vimage The scalability issue is the obvious difference: OpenBSD’s pf is still very much single-core oriented, whereas the FreeBSD pf version can cope with multiple cores (somewhat) and is significantly faster on multicore hardware. Our version is by no means perfect, but it’s much faster than OpenBSD’s version. Much of the imperfections we have now is there because pf was designed in a giant locked kernel in the first place. Multi-core scalability was not part of its original design. Adopting OpenBSDs pf would mean redoing all of the locking work Gleb did years ago. Given the differences in OpenBSD’s pf (e.g. they keep states in a tree, not a hash table) it’s not a matter of replaying the previous work on a new pf version. This is a from the ground up introduction of fine grained locking in a code base that assumes a single giant lock. As I understand it the OpenBSD people are working on the problem as well, but I’ve not seen any diffs yet. If they’ve made significant progress we may be able to base our work on theirs. I don’t think it’d be acceptable to not have this, as it’d mean a very large performance regression. For reference, before I did the pfsync work we essentially had a single-threaded pf when pfsync was enabled. On my test hardware this meant a throughput of ~1.1Mpps, rather than the ~3.9Mpps without pfsync. I’d expect OpenBSDs pf to perform at around that ~1.1Mpps number without locking work. The second issue is one of syntax, and that’s what I assume is the main reason people want OpenBSDs pf. We’re still on an older iteration of the pf syntax, but changing that would inevitably mean breaking old configurations. That might be acceptable on a major version update (i.e. going into 13), but would mean the new work could never be backported. That’s probably the only way forward though. I’m playing with importing the ‘match’ keyword and not breaking the ‘scrub’ syntax at the same time, but it’s a non-trivial problem, and that’s only one of the steps along the way. Finally there’s vimage. That’s a FreeBSD-only feature, so naturally OpenBSDs pf is not aware of this. I expect that to be relatively easy to add back in, but it’s another obstacle. As vimage is what lets us have the pf tests we’ve got now I’d be very reluctant to let it go. It’s a supported feature in 12.0, so users will start to rely on it as well. TL;DR: It’s possible, but *hard*. Expect is to be many person-months of effort, and there’s no way it’s going to be a smooth ride. One thing I’ve thought of trying, and that might be an interesting stepping stone, is to create a port (/usr/ports/net/opf or whatever) of OpenBSD’s pf. In that version it’d be acceptable to not fix any of the above issues. It’d still give users to option of getting the new syntax. I’d expect this to be a relatively straightforward exercise. In principle there’s nothing to stop us from doing that same work in base, but we’re **NOT** going to import a fourth firewall. We’re just not. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: rdr pass for proto tcp sometimes creates states with expire time zero and so breaking connections
On 19 Feb 2019, at 22:53, Andreas Longwitz wrote: Kristof Provost wrote: Because fetching a counter is a rather expansive function we should use counter_u64_fetch() in pf_state_expires() only when necessary. A "rdr pass" rule should not cause more effort than separate "rdr" and "pass" rules. For rules with adaptive timeout values the call of counter_u64_fetch() should be accepted, but otherwise not. For a small gain in performance especially for "rdr pass" rules I suggest something like --- pf.c.orig 2019-02-18 17:49:22.944751000 +0100 +++ pf.c 2019-02-18 17:55:07.396163000 +0100 @@ -1558,7 +1558,7 @@ if (!timeout) timeout = V_pf_default_rule.timeout[state->timeout]; start = state->rule.ptr->timeout[PFTM_ADAPTIVE_START]; - if (start) { + if (start && state->rule.ptr != &V_pf_default_rule) { end = state->rule.ptr->timeout[PFTM_ADAPTIVE_END]; states = counter_u64_fetch(state->rule.ptr->states_cur); } else { I think that looks correct. Do you have any performance measurements on this? No Although presumably it only really matters in cases where there’s no explicit catch-all rule, so I do wonder if it’s worth it. Sorry, but I do not understand this argument. From manpage: The adaptive timeout values can be defined both globally and for each rule. When used on a per-rule basis, the values relate to the number of states created by the rule, otherwise to the total number of states. This handling of adaptive timeouts is done in pf_state_expires(): start = state->rule.ptr->timeout[PFTM_ADAPTIVE_START]; if (start) { end = state->rule.ptr->timeout[PFTM_ADAPTIVE_END]; states = counter_u64_fetch(state->rule.ptr->states_cur); } else { start = V_pf_default_rule.timeout[PFTM_ADAPTIVE_START]; end = V_pf_default_rule.timeout[PFTM_ADAPTIVE_END]; states = V_pf_status.states; } The following calculation needs three values: start, end and states. 1. Normal rules "pass .." without adaptive setting meaning "start = 0" runs in the else-section of the code snippet and therefore takes "start" and "end" from the global default settings and sets "states" to pf_status.states (= total number of states). 2. Special rules like "pass .. keep state (adaptive.start 500 adaptive.end 1000)" have start != 0, run in the if-section of the code snippet and take "start" and "end" from the rule and set "states" to the number of states created by their rule using counter_u64_fetch(). Thats all ok, but there is a third case without special handling in the above code snippet: 3. All "rdr/nat pass .." statements use together the pf_default_rule. Therefore we have "start != 0" in this case and we run the if-section of the code snippet but we better should run the else-section in this case and do not fetch the counter of the pf_default_rule but take the total number of states. Thats what the patch does. Thank you, that makes sense. I’d missed the third case. The patch is in my queue and should get committed soon. Your explanation makes a great commit message. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: rdr pass for proto tcp sometimes creates states with expire time zero and so breaking connections
On 18 Feb 2019, at 18:30, Andreas Longwitz wrote: Ok, thanks, I will commit the patch shortly. I do not see a point in waiting for two more weeks, sure report me if anything goes wrong. your patch for counter(9) on i386 definitely solves my problem discussed in this thread. Because fetching a counter is a rather expansive function we should use counter_u64_fetch() in pf_state_expires() only when necessary. A "rdr pass" rule should not cause more effort than separate "rdr" and "pass" rules. For rules with adaptive timeout values the call of counter_u64_fetch() should be accepted, but otherwise not. For a small gain in performance especially for "rdr pass" rules I suggest something like --- pf.c.orig 2019-02-18 17:49:22.944751000 +0100 +++ pf.c2019-02-18 17:55:07.396163000 +0100 @@ -1558,7 +1558,7 @@ if (!timeout) timeout = V_pf_default_rule.timeout[state->timeout]; start = state->rule.ptr->timeout[PFTM_ADAPTIVE_START]; - if (start) { + if (start && state->rule.ptr != &V_pf_default_rule) { end = state->rule.ptr->timeout[PFTM_ADAPTIVE_END]; states = counter_u64_fetch(state->rule.ptr->states_cur); } else { I think that looks correct. Do you have any performance measurements on this? Although presumably it only really matters in cases where there’s no explicit catch-all rule, so I do wonder if it’s worth it. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: routing LAN traffic through/around a pf gateway
On 25 Jan 2019, at 9:37, James B. Byrne via freebsd-pf wrote: I have limited knowledge of PF being in the process of transitioning from 20+ years of RHEL/CentOS to FreeBSD. Neither do I possess a great fund of knowledge respecting IP routing. That said this is my problem: On a small test LAN I have three hosts, W44, W4 and G5: network layout, gateway address 216.185.71.5 W44 G5 w4 216.185.71.44 > 216.185.71.5216.185.71.4 int_if IP 192.168.150.44 192.168.150.5 > 192.168.150.4 int_if IP alias Using ssh and with PF running on the gateway, when I connect from 216.185.71.44 to 216.185.71.4 then the ssh session operates normally. However, if instead I connect from 216.185.71.44 to 192.168.150.4 then the initial connection is made but the ssh session remains responsive for a brief time before it becomes non-responsive. If I terminate the PF running on the gateway the ssh session again becomes responsive. If I do not terminate PF then eventually the ssh session client disconnects with a timeout error. Besides macros the entire active contents of pf.conf on G5 are: scrub inall no-df max-mss 1440 fragment reassemble block return out log all block dropin log all pass log on $int_if passinet proto icmp all \ icmp-type $icmp_types keep state pass out quick on $ext_if inet proto udp \ from any \ toany port 33433 >< 33626 keep state Which results in these rules when PF is running: @0 scrub in all no-df max-mss 1440 fragment reassemble @1 block return out log all @2 block drop in log all @3 pass log on em0 all flags S/SA keep state @4 pass inet proto icmp all icmp-type echoreq keep state @5 pass inet proto icmp all icmp-type unreach keep state @6 pass out quick on em1 inet proto udp from any to any port 33433 >< 33626 keep state You don’t appear to have a rule permitting the SSH traffic to pass through your router. I’m a more than little surprised you manage to establish a connection in the first place. Unless the connection existed before you started pf, of course. Try adding something like: pass inet porto tcp port 22 Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: VNET jails and PF service
On 2018-12-13 13:06:00 (+0100), Kristof Provost wrote: > On 2018-12-13 12:35:05 (+0100), Goran Mekić wrote: > > On Thu, Dec 13, 2018 at 09:30:12AM +0100, Kristof Provost wrote: > > > On 2018-12-13 01:02:32 (+0100), Goran Mekić wrote: > > > > I can't start PF as service from vnet jail. I have devfs rule to unhide > > > > bpf (for dhclient) and pf that the jail is using. I can run "pfctl -e -f > > > > /etc/pf.conf" but "service pf start" fails with: > > > > > > > > kldload: can't load pf: Operation not permitted > > > > /etc/rc.d/pf: WARNING: Unable to load kernel module pf > > > > > > > Yes, jails can't load kernel modules, for obvious reasons. > > > Your host needs to load the pf module, then the jail will be able to use > > > it. > > > > I did load it on the host, that's why "pfctl -e -f /etc/pf.conf" works > > in the jail, but "service pf start" doesn't. > > I can't seem to reproduce that. How did you start your jail? > > (The output of 'jls -na' might be helpful too) > At least on CURRENT that all does what I'd expect it to do: % sudo kldload pfsync % sudo jail -c name=alcatraz persist vnet % sudo jexec alcatraz /bin/sh # service pf onestart Enabling pf. # pfctl -s all FILTER RULES: scrub in all fragment reassemble pass out all flags S/SA keep state block drop in log all pass in inet proto tcp from any to any port = ssh flags S/SA keep state INFO: Status: Enabled for 0 days 00:00:03 Debug: Urgent ... Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: VNET jails and PF service
On 2018-12-13 12:35:05 (+0100), Goran Mekić wrote: > On Thu, Dec 13, 2018 at 09:30:12AM +0100, Kristof Provost wrote: > > On 2018-12-13 01:02:32 (+0100), Goran Mekić wrote: > > > I can't start PF as service from vnet jail. I have devfs rule to unhide > > > bpf (for dhclient) and pf that the jail is using. I can run "pfctl -e -f > > > /etc/pf.conf" but "service pf start" fails with: > > > > > > kldload: can't load pf: Operation not permitted > > > /etc/rc.d/pf: WARNING: Unable to load kernel module pf > > > > > Yes, jails can't load kernel modules, for obvious reasons. > > Your host needs to load the pf module, then the jail will be able to use > > it. > > I did load it on the host, that's why "pfctl -e -f /etc/pf.conf" works > in the jail, but "service pf start" doesn't. I can't seem to reproduce that. How did you start your jail? (The output of 'jls -na' might be helpful too) Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: VNET jails and PF service
On 2018-12-13 01:02:32 (+0100), Goran Mekić wrote: > I can't start PF as service from vnet jail. I have devfs rule to unhide > bpf (for dhclient) and pf that the jail is using. I can run "pfctl -e -f > /etc/pf.conf" but "service pf start" fails with: > > kldload: can't load pf: Operation not permitted > /etc/rc.d/pf: WARNING: Unable to load kernel module pf > Yes, jails can't load kernel modules, for obvious reasons. Your host needs to load the pf module, then the jail will be able to use it. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: rdr pass for proto tcp sometimes creates states with expire time zero and so breaking connections
On 13 Nov 2018, at 22:01, Andreas Longwitz wrote: Are there any hints why the counter pf_default_rule->states_cur could get a negative value ? I’m afraid I have no idea right now. OK, in the meantime I did some more research and I am now quite sure the problem with the bogus pf_default_rule->states_cur counter is not a problem in pf. I am convinced it is a problem in counter(9) on i386 server. The critical code is the machine instruction cmpxchg8b used in /sys/i386/include/counter.h. I’m always happy to hear problems aren’t my fault :) From intel instruction set reference manual: Zhis instruction can be used with a LOCK prefix allow the instruction to be executed atomically. We have two other sources in kernel using cmpxchg8b: /sys/i386/include/atomic.h and /sys/cddl/contrib/opensolaris/common/atomic/i386/opensolaris_atomic.S Both make use of the LOCK feature, in atomic.h a detailed explanation is given. Because counter.h lacks the LOCK prefix I propose the following patch to get around the leak: --- counter.h.orig 2015-07-03 16:45:36.0 +0200 +++ counter.h 2018-11-13 16:07:20.329053000 +0100 @@ -60,6 +60,7 @@ "movl %%edx,%%ecx\n\t" "addl (%%edi),%%ebx\n\t" "adcl 4(%%edi),%%ecx\n\t" + "lock \n\t" "cmpxchg8b %%fs:(%%esi)\n\t" "jnz1b" : @@ -76,6 +77,7 @@ __asm __volatile( "movl %%eax,%%ebx\n\t" "movl %%edx,%%ecx\n\t" + "lock \n\t" "cmpxchg8b (%2)" : "=a" (res_lo), "=d"(res_high) : "SD" (p) @@ -121,6 +123,7 @@ "xorl %%ebx,%%ebx\n\t" "xorl %%ecx,%%ecx\n\t" "1:\n\t" + "lock \n\t" "cmpxchg8b (%0)\n\t" "jnz1b" : That looks very plausible. I’m somewhat out of my depth here, so I’d like the authors of the counter code to take a look at it. Best regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
[Differential] D1309: VIMAGE PF fixes #1
kristof closed this revision. kristof added a comment. Herald added subscribers: farrokhi, ae. Assorted pf VIMAGE fixes have been done, and pf is now usable inside VIMAGE jails. CHANGES SINCE LAST ACTION https://reviews.freebsd.org/D1309/new/ REVISION DETAIL https://reviews.freebsd.org/D1309 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: rodrigc, #network, trociny, glebius, gnn, bz, zec Cc: ae, farrokhi, kristof, robak, emaste, freebsd-virtualization-list, freebsd-pf-list, freebsd-net-list ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: rdr pass for proto tcp sometimes creates states with expire time zero and so breaking connections
On 27 Oct 2018, at 5:22, Andreas Longwitz wrote: Thanks very much for answer especially for the hint to openbsd. I wonder if there’s an integer overflow in the of_state_expires() calculation. The OpenBSD people have a cast to u_int64_t in their version: |timeout = (u_int64_t)timeout * (end - states) / (end - start); | Perhaps this would fix your problem? (Untested, not even compiled) |if (end && states > start && start < end) { if (states < end) { timeout = (uint64_t)timeout * (end - states) / (end - start); return (state->expire + timeout; } else return (time_uptime); } return (state->expire + timeout); I can confirm the patch of the openbsd people adding the uint64_t cast makes sense. If timeout * (end - states) becomes bigger than UINT32_MAX (I am on i386) the cast prevents the overflow of this product and the result of the adaptive calculation will always be correct. Example: start=6000, end=12000, timeout=86400 * 5 (5 days), states=100 result 140972, result with cast patch 856800. In the problem I have reported for states of "rdr pass" rules I see start=6000, end=12000, timeout=86400 and (obviously erroneous, probably negative) states=0xffd0. I have no idea how that can happen. Just to make sure I understand: you know that states is negative here because of a printf() or SDT addition in pf_expire_states(), right? Further the counter variable for states_cur of pf_default_rule is used für all "rdr/nat/binat pass" rules together. This was a little bit suprising for me, but I think this is intended behaviour. Correct ? Yes. Are there any hints why the counter pf_default_rule->states_cur could get a negative value ? I’m afraid I have no idea right now. Best regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: rdr pass for proto tcp sometimes creates states with expire time zero and so breaking connections
On 15 Oct 2018, at 15:26, Andreas Longwitz wrote: On two of my FreeBSD 10 (r338093) firewall servers some incoming ssh connections stopped to work because pf started to create states with expire time zero (instead of 86400 sec) for translation statements like rdr pass on em0 proto tcp from any to myip port 8022 --> 10.0.0.254 Therefore a command given on a remote server like ssh -p 8022 myip sleep 15 did not return, because the created state for the connection was purged by pf (interval 10 sec) before 15 seconds. If I replace the rdr pass rule with a rdr rule and a separate filter pass rule then the created state always has expire time 24h and everything is ok. I have tried to analyze the bug in the rdr pass rule. Immediately after starting the above ssh command on the remote sever I see with pfctl -vss the sreated state on my firewall machine: all tcp 10.0.0.254:8022 (myip:8022) <- remoteip:59233 ESTABLISHED:ESTABLISHED [1443872812 + 66608] wscale 6 [1365307391 + 66560] wscale 3 age 00:00:00, expires in 00:00:00, 13:12 pkts, 2955:3306 bytes and a DTrace script running at the same time gives 3 19323pfsync_state_export:entry creation=4491391. expire=4491391, state_timeout=2 3 16459 pf_state_expires:entry state_timeout=86400, start_timeout=6000, end_timeout=12000 states=882 3 17624 counter_u64_fetch:entry 3 17625 counter_u64_fetch:return returncode (states_cur)=4294967248 = 0xffd0 3 16460 pf_state_expires:return returncode=4491391 3 19324 pfsync_state_export:return creation=0. expire=0, syncstate_timeout=2 0 12730 pfioctl:return returncode=0, time_uptime=4491391 So pf_state_expires() returns the value time_update and therefore pfsync_state_export() gives expire zero. Reason for this is the very big (means negative) value returned by the function counter_u64_fetch() for states_cur. This variable is of type uint64_t and only incremented and decremented by the macros STATE_INC_COUNTERS and STATE_DEC_COUNTERS. I wonder if there’s an integer overflow in the of_state_expires() calculation. The OpenBSD people have a cast to u_int64_t in their version: timeout = (u_int64_t)timeout * (end - states) / (end - start); Perhaps this would fix your problem? (Untested, not even compiled) if (end && states > start && start < end) { if (states < end) { timeout = (uint64_t)timeout * (end - states) / (end - start); return (state->expire + timeout; } else return (time_uptime); } return (state->expire + timeout); } Best regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: pf tables locking
On 14 Aug 2018, at 0:32, Kajetan Staszkiewicz wrote: On Monday, 13 August 2018 17:59:15 CEST Kristof Provost wrote: How about this? https://github.com/innogames/freebsd/commit/ d44a0d9487285fac8ed1d7372cc99cca83f616e6 That looks good to me. There’s a few minor issues, things like inconsistent indentation and overly long lines, but that’s about the only criticism I have. Do you have a bit more information about your use case? What are you trying to accomplish with this change? I have a loadbalancer which uses pf and route-to targets. After a server is added to a pool, I want this server to immediately take over much traffic. With round-robin the server receives new clients rather slowly. If kernel could measure amount of states per table entry, I could send new clients to this new server until it serves as many clients as other servers. I see. I’m not quite sure yet if that’s a feature we want to import or not, but at least your ‘support’ patches should probably go in. The above one certainly. There are some more issues I found around pf_map_addr. Some of them I mentioned in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=229092. Some more came out while working on this least-states loadbalancing. I will group them into something meaningful and make another PR for them. Yeah, that bug is still on my todo list somewhere, but things are extremely hectic at the moment, and I can’t make any promises about when I’ll have time for it. I thought that was rather on my todo :) I’m not going to stop you. I love it when other people do the work ;) Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: pf tables locking
On 13 Aug 2018, at 17:06, Kajetan Staszkiewicz wrote: > On Monday, 13 August 2018 15:22:33 CEST Kristof Provost wrote: >> rules (and associated tables) won’t just go away while there’s still >> state, > > This is mostly what I wanted to ask about in this message. How is it ensured > that table and counters are gone only after everybody stops using them? What > if I delete a table, then change ruleset, but there is still active connection > keeping a state? I really had hard time finding how this is guarded in source. > pf keeps rules around until there are no more states left referencing the rule. Look at pf_commit_rules(): The old rules are unlinked rather than removed. They’re kept on the V_pf_unlinked rules list. Every so often pf runs through all states (in pf_purge_thread()) to mark their associated rules as still referenced. Only rules which are not referenced by any state are removed. This means that while there’s still a state which was created by the rule (and can thus put packets towards its table), the rule will exist. Once the state goes away it’ll still take one full iteration through all states before the rule can be freed. Hence my statement that it’s highly unlikely (pretty much impossible) for us to run into a situation where the rule no longer exists. >> I don’t want to re-take the rules lock for this, so my current >> thinking is that the best approach would be to already get rid of the >> potential memory leak by just always allocating the pfrke_counters when >> the table is created (i.e. when the rule is first set). That might waste >> a little memory if we didn’t need it, but it should simplify things a >> bit. > >> We can resolve the counting issue by using the counter_u64_*() functions >> for them. We should be able to get away with not locking this. > > Sure, I can use counter(9). The question, as always with my patches, is what > can go to FreeBSD and what won't go. > > My current goal is to modify round-robin pf target to always point to table > entry with least amount of states. > > As I see it for now: > 1. Modify pfrke_counters to be always allocated. > 2. Rewrite pfrke_counters to use counter(9). > 3. Provide state counter in pfrke_counters. > 4. Modify round-robin target. > > 1. and 2. make a good PR. I'm not sure about 3. Do you want patches for least- > connections target too? I want to just replace existing round-robin but if > there is any chance of getting it into kernel code, I could make it work as > new target in pf.conf. > Do you have a bit more information about your use case? What are you trying to accomplish with this change? > There are some more issues I found around pf_map_addr. Some of them I > mentioned in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=229092. Some > more came out while working on this least-states loadbalancing. I will group > them into something meaningful and make another PR for them. > Yeah, that bug is still on my todo list somewhere, but things are extremely hectic at the moment, and I can’t make any promises about when I’ll have time for it. Regards, Kristof signature.asc Description: OpenPGP digital signature
Re: pf tables locking
On 13 Aug 2018, at 0:09, Kajetan Staszkiewicz wrote: Hello group, Can anybody help me iwth pf_table.c and all operations on tables, especially pfr_update_stats? I'm working on implementing stats for redirection targets, that is for nat or route-to. I'm going through the code and I've found out that many table-related function are guarded by lock on pf ruleset. But that is not true for pfr_update_stats. This function is called from pf_test only after PF_RULES_RUNLOCK(). I think you’re right, this does look wrong. It’s very unlikely that this will actually lead to a crash, because rules (and associated tables) won’t just go away while there’s still state, but we could theoretically lose memory (in the pfrke_counters allocation), and miscount. I don’t want to re-take the rules lock for this, so my current thinking is that the best approach would be to already get rid of the potential memory leak by just always allocating the pfrke_counters when the table is created (i.e. when the rule is first set). That might waste a little memory if we didn’t need it, but it should simplify things a bit. We can resolve the counting issue by using the counter_u64_*() functions for them. We should be able to get away with not locking this. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Possible bug: 11.2-RELEASE guest with vtnet and PF
On 2 Jul 2018, at 16:17, Kristof Provost wrote: Hi Jakub, On 30 Jun 2018, at 17:07, Jakub Chromy wrote: I've just installed a 11.2-RELEASE guest under bhyve (hypervisor is 11.1-RELEASE)... and I cant use Virtio network interface with PF: odine:/boot/kernel# /sbin/pfctl -n -f ~/local/tmp/pf.work *pfctl: pfi_get_ifaces: Bad file descriptor* the file contains the following single line only: pass out quick on vtnet0 proto tcp from any to any keep state I’m pretty sure this is a pf bug rather than an issue with vtnet. Does this still happen if you don’t specify ‘-n’? I suspect this might be related to r333181, but that’s included in CURRENT too, and I’ve not been able to reproduce this on my CURRENT box. I’m updating my stable/11 test VM now, but that’ll take a while. Ah, I think I see the problem. I think you don’t have the pf module loaded, which is apparently not treated as a fatal error if ‘-n’ is specified, but the change in r333181 can’t cope with that. We should probably fix that, but it’s not a particularly critical problem. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: pf reload/resync and skipped interface groups on 11.2-RELEASE
On 2 Jul 2018, at 16:44, Felix J. Ogris wrote: this is a fresh install of 11.2-RELEASE amd64 with a minimal pf rule set. After the first reload/resync, any traffic on an interface that is skipped via an interface group statement in pf.conf is rejected: Thanks for the report. I think that’s the same issue as described in PR 229241, in which case it’s on my todo list already. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Possible bug: 11.2-RELEASE guest with vtnet and PF
Hi Jakub, On 30 Jun 2018, at 17:07, Jakub Chromy wrote: I've just installed a 11.2-RELEASE guest under bhyve (hypervisor is 11.1-RELEASE)... and I cant use Virtio network interface with PF: odine:/boot/kernel# /sbin/pfctl -n -f ~/local/tmp/pf.work *pfctl: pfi_get_ifaces: Bad file descriptor* the file contains the following single line only: pass out quick on vtnet0 proto tcp from any to any keep state I’m pretty sure this is a pf bug rather than an issue with vtnet. Does this still happen if you don’t specify ‘-n’? I suspect this might be related to r333181, but that’s included in CURRENT too, and I’ve not been able to reproduce this on my CURRENT box. I’m updating my stable/11 test VM now, but that’ll take a while. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: "egress" group
On 25 Jun 2018, at 22:12, Joseph Ward wrote: My current pf.conf contains the following lines (with a lot of other stuff redacted for irrelevance): ext_if="em0" ... block log all pass in on $ext_if proto tcp from any to any port 22 flags S/SA keep state and it works great; ssh is able to get in. However, when I change "$ext_if" to "egress", it no longer works. From the various documentation I've found online, egress should automatically be the interface which has the default route, and netstat -rn gives me: ‘egress’ exists in OpenBSD’s pf, but not in FreeBSD. My goal is for this pf.conf to be able to be used on multiple systems which unfortunately have different network cards, so the interface names are different. If "egress" isn't going to work, is there another way to accomplish that goal? You could rename your network card (ifconfig em0 name foo). That’d let you hide the difference from pf (but you’d have to cope with it in /etc/rc.conf) Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: pfr_update_stats: assertion failed.
On 24 Jun 2018, at 21:07, Marek Zarychta wrote: On Sun, Jun 24, 2018 at 01:56:07PM +0200, Kristof Provost wrote: On 23 Jun 2018, at 18:46, Marek Zarychta wrote: On Sat, Jun 23, 2018 at 05:27:29PM +0200, Marek Zarychta wrote: On Sun, Oct 16, 2016 at 08:17:13PM +0200, Marek Zarychta wrote: The issue occurred first two years ago, after upgrade from 8 to 9 branch. Now this i386 machine is running 11.0-STABLE and despite it was compiled with "WITHOUT_ASSERT_DEBUG=yes", still from time to time message buffer is fed with: pfr_update_stats: assertion failed. pfr_update_stats: assertion failed. pfr_update_stats: assertion failed. pfr_update_stats: assertion failed. pfr_update_stats: assertion failed. pfr_update_stats: assertion failed. pfr_update_stats: assertion failed. These messages are still filling system message buffer. According to pfctl (8) there is nothing wrong with incrementing "XPass" counters instead of the "Pass" counters. The message "pfr_update_stats: assertion failed" is printed for debugging purposes only. One could also compare the counters with the command "pfctl -sT -vv". OpenBSD converted printf()'s to DPFDEBUG() macro in their sources almost 8 years ago. Only this printf() in pf_table.c has been converted to the level of LOG_DEBUG [1]. Perhaps this line of code could be removed from FreeBSD PF sources? The previous patch was hastily prepared. It should rather look like this: --- sys/netpfil/pf/pf_table.orig.c 2018-06-23 16:40:14.876882000 +0200 +++ sys/netpfil/pf/pf_table.c 2018-06-23 18:17:49.35349 +0200 @@ -1984,9 +1984,7 @@ panic("%s: unknown address family %u", __func__, af); } - if ((ke == NULL || ke->pfrke_not) != notrule) { - if (op_pass != PFR_OP_PASS) - printf("pfr_update_stats: assertion failed.\n"); + if ((ke == NULL || ke->pfrke_not) != notrule) op_pass = PFR_OP_XPASS; - } kt->pfrkt_packets[dir_out][op_pass]++; kt->pfrkt_bytes[dir_out][op_pass] += len; We could delete those lines and that’d get rid of the dmesg noise, but I’m a bit worried that this demonstrates an actual problem. It’s not at all clear to me what’s going on in this bit of the code, and the OpenBSD repo doesn’t have any information about it either. This machine acts as a NAT/firewall gateway for about a hundred users. A few hundred of PF rules + 20 tables are used. The error appeared suddenly after upgrade from 8-STABLE to 10-STABLE 3 years ago. It never occurred when the firewall run PF on 8-STABLE. I don't remember whether firewall rules were changed at that time. If it is true then changes concerned only the compatibility with the newer version of PF. If it demonstrates an actual problem, then, please give me a clue how to debug it. On the other hand, ~6 years ago PF was significantly reworked. Is this piece of code still relevant there? I honestly don’t know. I’d be good to actually investigate this before simply removing the warning. Unfortunately I just don’t have the time to dig into this right now. The only thing I can suggest is to look at the code and work out where the op_pass value comes from (and perhaps also what it’s used for. Why is PRF_OP_XPASS better than !PFR_OP_PASS? It’s still present (though perhaps not logged) in OpenBSD too. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: pfr_update_stats: assertion failed.
On 23 Jun 2018, at 18:46, Marek Zarychta wrote: On Sat, Jun 23, 2018 at 05:27:29PM +0200, Marek Zarychta wrote: On Sun, Oct 16, 2016 at 08:17:13PM +0200, Marek Zarychta wrote: The issue occurred first two years ago, after upgrade from 8 to 9 branch. Now this i386 machine is running 11.0-STABLE and despite it was compiled with "WITHOUT_ASSERT_DEBUG=yes", still from time to time message buffer is fed with: pfr_update_stats: assertion failed. pfr_update_stats: assertion failed. pfr_update_stats: assertion failed. pfr_update_stats: assertion failed. pfr_update_stats: assertion failed. pfr_update_stats: assertion failed. pfr_update_stats: assertion failed. These messages are still filling system message buffer. According to pfctl (8) there is nothing wrong with incrementing "XPass" counters instead of the "Pass" counters. The message "pfr_update_stats: assertion failed" is printed for debugging purposes only. One could also compare the counters with the command "pfctl -sT -vv". OpenBSD converted printf()'s to DPFDEBUG() macro in their sources almost 8 years ago. Only this printf() in pf_table.c has been converted to the level of LOG_DEBUG [1]. Perhaps this line of code could be removed from FreeBSD PF sources? The previous patch was hastily prepared. It should rather look like this: --- sys/netpfil/pf/pf_table.orig.c 2018-06-23 16:40:14.876882000 +0200 +++ sys/netpfil/pf/pf_table.c 2018-06-23 18:17:49.35349 +0200 @@ -1984,9 +1984,7 @@ panic("%s: unknown address family %u", __func__, af); } - if ((ke == NULL || ke->pfrke_not) != notrule) { - if (op_pass != PFR_OP_PASS) - printf("pfr_update_stats: assertion failed.\n"); + if ((ke == NULL || ke->pfrke_not) != notrule) op_pass = PFR_OP_XPASS; - } kt->pfrkt_packets[dir_out][op_pass]++; kt->pfrkt_bytes[dir_out][op_pass] += len; We could delete those lines and that’d get rid of the dmesg noise, but I’m a bit worried that this demonstrates an actual problem. It’s not at all clear to me what’s going on in this bit of the code, and the OpenBSD repo doesn’t have any information about it either. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Is there an upper limit to PF's tables?
On 18 Jun 2018, at 0:19, Chris H wrote: Sorry. Looks like I might be coming to the party a little late. But I'm currently running a 9.3 box that runs as a IP (service) filter for much of a network. While I've patched the box well enough to keep it safe to continue running. I am reluctant to up(grade|date) it to 11, or CURRENT, based on some of the information related to topics like this thread. Currently, the 9.3 box maintains some 18 million entries *just* within the SPAM related table. The other tables contain no less that 1 million. As it stands I have *no* trouble loading pf(4) with all of the tables totaling some 20+ million entries, *even* when the BOX is working with as little 4Gb ram. Has something in pf(4) changed, since 9.3 that would now prevent me from continuing to use my current setup, and tables? No. There are no new limits in 11, and the only thing that *might* be an issue is validation improvements in 12. Still, anything that worked on 9 is expected to work on 12 (if not, report a bug). Please don’t keep running unsupported versions. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Is there an upper limit to PF's tables?
On 14 Jun 2018, at 19:40, Dave Horsfall wrote: I can't get access to kernel sauce right now, but I'm hitting over 1,000 entries from woodpeckers[*] etc; is there some upper limit, or is it just purely dynamic? aneurin% freebsd-version 10.4-RELEASE-p9 Ian already gave some good information, but it’s important to note that there are a number of different limits, and the maximum number of states is different from the limit on table sizes. There’s no immediate limit to the number of addresses in a table. It mostly depends on having enough memory. On 12 you may start to run into issues loading it in one go once you have more than 65k entries. If you do run into that, that particular limit can be tuned using `sysctl net.pf.request_maxcount` Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: pf nat log does not show source and destination port
Hi Fatemeh, On 11 Jun 2018, at 7:51, Fatemeh Mehdizadeh wrote: Hi all, I'm using pf to create nat. I'm on FreeBSD9.2. Note that FreeBSD 9.2 is not a supported version. It went out of support at the end of 2014. (See https://www.freebsd.org/security/unsupported.html) I would strongly recommend upgrading to a supported version: https://www.freebsd.org/security/security.html#sup I want enable logs for nat translations, so in pf.cpnf: table { 20.20.20.2,20.20.20.3,20.20.20.4,20.20.20.5 } nat log on 'eth0' from { 10.10.10.0/24} to any -> round-robin sticky-address After ping request I have a log: # tcpdump -t -r pflog IP 20.20.20.3 > 20.20.20.1: ICMP echo request, id 4147, seq 0, length 64 pflog logs the entire packet (with a pf-specific header with information about the matched rules), so you can parse whatever information you want out of that. The problem is that I want my log shows the source port and destination port and NOT show id, seq and length. You may get enough information by simply telling tcpdump to be more verbose: # tcpdump -t -v -r pflog (Repeat the ‘-v’ flag for even more information.) Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Required modification for round robin napt with ip address prefixes
On 14 Mar 2018, at 18:30, Steven Crangle wrote: I was looking for some advice on the type of locking required to stop a box panicking that utilises both napt and ip address prefixes. My colleague made a post a while ago, and we ended up getting distracted fixing other panics that showed up. But we've now returned to try and figure out the issue. The relevant code is in pf_lb.c : 424 I’d recommend talking to glebius@. He did the locking code and wrote the comment block discussing the locking choices around PF_POOL_ROUNDROBIN. I suspect it’s a bit more complicated that a straightforward PF_RULES_WLOCK() would fix. The locking model for pf is pretty complex. I’ve not had the time to really dig into this, so I can’t give more advice right now. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Kernel Panic
On 1 Mar 2018, at 15:37, Joe Jones wrote: yes we use pfsync. Yesterday we tried with pfsync switched off, the box still locked up but this time without a panic. We make the DIOCRADDADDRS ioctl on the master and the backup (we use CARPed pairs). Interesting. It might be related to pfsync. Is is the master that panics or the backup? Or both? Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Kernel Panic
On 28 Feb 2018, at 9:52, Kristof Provost wrote: On 27 Feb 2018, at 20:40, Joe Jones wrote: we have a kernel panic after compiling with witness and invariant Feb 27 13:49:33 sovapn1 kernel: lock order reversal: Feb 27 13:49:33 sovapn1 kernel: 1st 0xfe000fed78b8 pf_idhash (pf_idhash) @ /usr/src/sys/netpfil/pf/pf.c:1078 Feb 27 13:49:33 sovapn1 kernel: 2nd 0xf8001e0474a8 pfsync (pfsync) @ /usr/src/sys/netpfil/pf/if_pfsync.c:1667 That’s a lock order reversal. It’s not good, but it should at worst result in a deadlock. Did the system stop after this? It also looks like a different problem from the panic you initially reported. Also, do you actively use pfsync in this setup? Does the panic happen on the box where you DIOCRADDADDRS or the other(s)? Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Kernel Panic
On 27 Feb 2018, at 20:40, Joe Jones wrote: we have a kernel panic after compiling with witness and invariant Feb 27 13:49:33 sovapn1 kernel: lock order reversal: Feb 27 13:49:33 sovapn1 kernel: 1st 0xfe000fed78b8 pf_idhash (pf_idhash) @ /usr/src/sys/netpfil/pf/pf.c:1078 Feb 27 13:49:33 sovapn1 kernel: 2nd 0xf8001e0474a8 pfsync (pfsync) @ /usr/src/sys/netpfil/pf/if_pfsync.c:1667 That’s a lock order reversal. It’s not good, but it should at worst result in a deadlock. Did the system stop after this? It also looks like a different problem from the panic you initially reported. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Kernel Panic
On 26 Feb 2018, at 17:06, Joe Jones wrote: Hi Kristof, we are not updating rules during the test although in production we will reload the rule set from time to time. We are constantly adding and removing from tables though, using the DIOCRADDADDRS and DIOCRDELADDRS ioctl, also DIOCKILLSTATES is being called a lot. These are all in response to RADIUS events. We tried using pfctl shell command rather than calling ioctl directly, to check that it wasn't a problem with how we are calling the ioctl. That’s interesting. The panic leads me to suspect something’s wrong with the kt->pfrkt_ipv4->rt, which would explain why we get the unexpected NULL result. My first guess at the cause would be a race condition, where it’s being modified (through one of the ioctls you do) while the pfr_pool_get() is walking it. I don’t immediately see where that’d happen though, because both DIOCRADDADDRS and DIOCRDELADDRS take the rules lock (and pfr_pool_get() takes it too). It might be interesting to run this with these extra asserts (and be sure to enable INVARIANTS). diff --git a/sys/netpfil/pf/pf_table.c b/sys/netpfil/pf/pf_table.c index 18342a94073..cad9b4ea89f 100644 --- a/sys/netpfil/pf/pf_table.c +++ b/sys/netpfil/pf/pf_table.c @@ -962,6 +962,8 @@ pfr_unroute_kentry(struct pfr_ktable *kt, struct pfr_kentry *ke) struct radix_node *rn; struct radix_head *head = NULL; + PF_RULES_WASSERT(); + if (ke->pfrke_af == AF_INET) head = &kt->pfrkt_ip4->rh; else if (ke->pfrke_af == AF_INET6) @@ -1855,6 +1859,8 @@ pfr_destroy_ktable(struct pfr_ktable *kt, int flushaddr) { struct pfr_kentryworkq addrq; + PF_RULES_WASSERT(); + if (flushaddr) { pfr_enqueue_addrs(kt, &addrq, NULL, 0); pfr_clean_node_mask(kt, &addrq); Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Kernel Panic
On 14 Feb 2018, at 23:47, Joe Jones wrote: Hi, we are running test traffic through our system, after between 1 and 12 hours we get a kernel panic, always in the pfr_pool_get function in /usr/src/sys/netpfil/pf/pf_table.c line 2140. After a bit of investigation I confirmed that ke2 is set to null on line 2122. It’d probably be interesting to know what the contents of uaddr/addr is here. From a very quick look at the code there’s supposed to be a route lookup there, and I’d expect there to always be a result. The code certainly expects it, because that looks to be what causes the panic. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: problems with tftp-proxy in 11.1?
On 14 Dec 2017, at 8:16, Kristof Provost wrote: On 7 Dec 2017, at 18:02, John Jasen wrote: On Wed, Dec 6, 2017 at 5:01 PM, Kristof Provost wrote: On 6 Dec 2017, at 21:25, John Jasen wrote: On 12/04/2017 02:47 PM, Kristof Provost wrote: Okay, so this is interesting: 25013: ioctl(4,0xc04c4417 { IORW 0x44('D'), 23, 76 },0x7fffe5b0) ERR#2 'No such file or directory' The DIOCNATLOOK ioctl() fails with ENOENT, which happens if the state can’t be found. Of course, I have no idea why that would happen. Does this affect some tftp connections or all of them? Some, where the amount seems somewhat random. I *think* I’ve reproduced this. It seems to happen only to the first tftp packet, but at least that’s something to go on. I’ll try to investigate further soon. Do you have ‘set skip on lo0’ set by any chance? I only see issues with that set, not once I unset it. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: problems with tftp-proxy in 11.1?
On 7 Dec 2017, at 18:02, John Jasen wrote: On Wed, Dec 6, 2017 at 5:01 PM, Kristof Provost wrote: On 6 Dec 2017, at 21:25, John Jasen wrote: On 12/04/2017 02:47 PM, Kristof Provost wrote: Okay, so this is interesting: 25013: ioctl(4,0xc04c4417 { IORW 0x44('D'), 23, 76 },0x7fffe5b0) ERR#2 'No such file or directory' The DIOCNATLOOK ioctl() fails with ENOENT, which happens if the state can’t be found. Of course, I have no idea why that would happen. Does this affect some tftp connections or all of them? Some, where the amount seems somewhat random. I *think* I’ve reproduced this. It seems to happen only to the first tftp packet, but at least that’s something to go on. I’ll try to investigate further soon. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: problems with tftp-proxy in 11.1?
On 6 Dec 2017, at 21:25, John Jasen wrote: On 12/04/2017 02:47 PM, Kristof Provost wrote: On 4 Dec 2017, at 19:57, John Jasen wrote: Depending on circumstances, we see a lot or a very few of the following messages: "pf connection lookup failed (no rdr?)" That means the state lookup (using ioctl(DIOCNATLOOK)) failed. There seem to be a couple of possible reasons why that might happen. One of which is that there’s no state at all. Can you check how many states you’ve got (and what the limits are)? The state tables should be fine. They're currently in the 30k range, set to alert in nagios at 250k. I've attached truss snippets and log snippets from a failed connection. truss was obtained via truss -f -p $pid -o outfile, and grepping down via the failued pid as logged in syslog. Okay, so this is interesting: 25013: ioctl(4,0xc04c4417 { IORW 0x44('D'), 23, 76 },0x7fffe5b0) ERR#2 'No such file or directory' The DIOCNATLOOK ioctl() fails with ENOENT, which happens if the state can’t be found. Of course, I have no idea why that would happen. Does this affect some tftp connections or all of them? Can you post the outputs of `pfctl -s memory`, `pfctl -s info` and `sudo pfctl -s limits`? Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: problems with tftp-proxy in 11.1?
On 4 Dec 2017, at 19:57, John Jasen wrote: Depending on circumstances, we see a lot or a very few of the following messages: "pf connection lookup failed (no rdr?)" That means the state lookup (using ioctl(DIOCNATLOOK)) failed. There seem to be a couple of possible reasons why that might happen. One of which is that there’s no state at all. Can you check how many states you’ve got (and what the limits are)? It might also be worth checking what errno is when the ioctl failed. truss can help, or you can patch tftp-proxy: diff --git a/contrib/pf/tftp-proxy/filter.c b/contrib/pf/tftp-proxy/filter.c index e5a769a62a5..1802ac2c4d9 100644 --- a/contrib/pf/tftp-proxy/filter.c +++ b/contrib/pf/tftp-proxy/filter.c @@ -363,7 +363,10 @@ server_lookup4(struct sockaddr_in *client, struct sockaddr_in *proxy, pnl.dport = proxy->sin_port; if (ioctl(dev, DIOCNATLOOK, &pnl) == -1) + { + printf("DIOCTNATLOOK errno %d\n", errno); return (-1); + } memset(server, 0, sizeof(struct sockaddr_in)); server->sin_len = sizeof(struct sockaddr_in); Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: problems with tftp-proxy in 11.1?
On 2 Dec 2017, at 4:56, John Jasen wrote: > Attempts to run tftp-proxy across a freebsd system running pf result in > very slow performance and an endless amount of: > > "pf connection lookup failed (no rdr?)" > Is there something that has regressed in 11.1, or am I missing something? > I’m not aware of any such regressions, but that of course doesn’t mean the can’t be there. Can you post the relevant bits of your rules/configuration? A small test case would be ideal. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Jail isolation from internal network and host (pf, vnet (vimage), freebsd 11.1)
On 7 Nov 2017, at 23:43, irukandji via freebsd-pf wrote: > Hi Everyone, > > Problem: isolating jail away from internal network and host "hosting" > it. > Environment: jail with 192.168.1.100, host 192.168.1.200, VIMAGE > enabled kernel, VNET (vnet0:JID) over bridge interface (bridge0), > single network card on re0 > Can you show how you’ve started the jail and configured the network setup? Are you running a vnet jail? > I am unable prevent jail accessing host (192.168.1.200) for any other > ip it is working, i have configured VNET just to have separated stack > but host is still accessible from jail. > What pf rules do you have? > Am I missing something or this is just something that cant be > accomplished using pf? I am banging my head to the wall with this issue > for past few months going radical lately (kernel recompile ;) ) > but still without any result. > It should be possible to do this, but there’s a lot of ways to set this up. Also bear in mind that VIMAGE was experimental in 11.1. There are several important bugs that are not fixed in 11.1 (but are fixed in CURRENT), especially in combination with pf. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: NATted outbound traffic sometimes uses backup CARP IP on LACP/LAGG interface
On 15 Sep 2017, at 11:31, Dave Cottlehuber wrote: Can you explain what $if:0 resolves to, for example how does it relate to to the primary ipv4/6 addresses bound to that interface? I couldn't find a reference in the usual ifconfig manpages about this (ifname:#) format, the BNF grammar for pf.conf doesn't cover it either, and `pfctl -vnf ...` simply shows (lagg0:0). It tells pf to not use any of the alias addresses. It’s explained in the pf.conf man page: Interface names and interface group names can have modifiers appended: :network Translates to the network(s) attached to the interface. :broadcastTranslates to the interface's broadcast address(es). :peer Translates to the point-to-point interface's peer address(es). :0Do not include interface aliases. Host names may also have the :0 option appended to restrict the name resolution to the first of each v4 and v6 address found. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: NATted outbound traffic sometimes uses backup CARP IP on LACP/LAGG interface
On 14 Sep 2017, at 16:21, Dave Cottlehuber wrote: Outgoing traffic (from a jail) via PF NAT over a LAGG/LACP sometimes has the *backup* CARP IP address assigned to it. ### running configs ## pfctl indeed shows its a round-robin ### # pfctl -vnf /etc/pf.conf protocols = "{ tcp, udp, icmp }" extl_if = "lagg0" intl_if = "lo0" jail_if = "lo1" intl_net = "lo0:network" jail_net = "lo1:network" internet = "lagg0:network" set limit states 20 set limit frags 8 set limit src-nodes 8 set timeout adaptive.start 18 set timeout adaptive.end 20 scrub in all fragment reassemble nat on lagg0 inet proto tcp from 10.241.0.0/16 to any -> (lagg0) round-robin nat on lagg0 inet proto tcp from 10.241.0.1 to any -> (lagg0) round-robin I think this is your problem. You’re telling pf to nat to the IP address of lagg0, but lagg0 has multiple addresses assigned. ‘(lagg0:0)’ should work, or just use the IP address. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Is panic() the way to handle errors in pf?
On 1 Aug 2017, at 11:30, Kajetan Staszkiewicz wrote: > Hey, group. > > A thought came to me: is it really the best thing to panic when errors are > encountered within pf? I understand there are situations where it is safer for > the kernel to not continue running like some low-level operations in memory > allocator or filesystems. But a firewall? Especially that a firewall handles > packets coming from the Interent which can be arbitrarily crafted. > pf does not use panic() to handle bad packets, but to handle **impossible** situations. Basically, what you see here are assertions (go count KASSERT() too), not error paths. If it were possible to trigger such a panic by sending a bad packet it would be a bug, yes, but that’s not what’s happening here. These panics document invariants. They are assertions. Once the impossible has happened there’s no sane way for the system to continue. It would be irresponsible to even try. Removing them would make pf **more** vulnerable to exploitation, not less. Regards, Kristof signature.asc Description: OpenPGP digital signature
Re: Bridge forward detection bug in 11.1-RELEASE?
On 30 Jul 2017, at 22:19, Heikki Paatela wrote: I was having kernel panics with 10.2-RELEASE earlier, caused by https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=202351. It would seem that some change that has happened between 11.0 and 11.1 has introduced a similar problem. The patch fixed the issue for me with 10-stable, and I installed 11.0-RELEASE via freebsd-update and had everything running nicely. Last week I updated to 11.1-RELEASE via freebsd-update, and started to again experience problems, where my bridge would stop forwarding traffic if my AirPort Extreme -base station was connected. This time I'm not experiencing kernel panics, and I'm not seeing anything peculiar in the log files. The bridge is headless computer, so I haven't been able to check if it's otherwise usable, when it stops forwarding traffic. I didn't report this on bugzilla, as I don't really think I have enough details on the matter. I have since downgraded back to 11.0-RELEASE, where everything is again working flawlessly. I’m not aware of relevant changes between 11.0 and 11.1. The closes I can see is the fix for PR 217883, but I’d be surprised if that’s the problem. It’d be useful if you could provide more details on your configuration, and if you could check if you’re seeing kernel panics. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: pfctl does not clear limit couters
On 14 Apr 2017, at 8:24, Max wrote: "pfctl -F info" command doesn't clear limit counters (shown in "pfctl -vsi" output). I think, should be --- sys/netpfil/pf/pf_ioctl.c.orig 2017-04-14 09:10:25.17138 +0300 +++ sys/netpfil/pf/pf_ioctl.c 2017-04-14 09:13:21.55365 +0300 @@ -1835,16 +1835,18 @@ case DIOCCLRSTATUS: { PF_RULES_WLOCK(); for (int i = 0; i < PFRES_MAX; i++) counter_u64_zero(V_pf_status.counters[i]); for (int i = 0; i < FCNT_MAX; i++) counter_u64_zero(V_pf_status.fcounters[i]); for (int i = 0; i < SCNT_MAX; i++) counter_u64_zero(V_pf_status.scounters[i]); + for (int i = 0; i < LCNT_MAX; i++) + counter_u64_zero(V_pf_status.lcounters[i]); V_pf_status.since = time_second; if (*V_pf_status.ifname) pfi_update_status(V_pf_status.ifname, NULL); PF_RULES_WUNLOCK(); break; } case DIOCNATLOOK: { This looks reasonable, but interestingly OpenBSD also don’t clear lcounters. I’ll dig into it a bit more in the next few days. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: When should I worry about performance tuning?
On 29 Mar 2017, at 22:06, Chris H wrote: OK. My association with FreeBSD has made me a prime target for every male hormone distributor on the net. Fact is; I can guarantee ~89 SPAM attempts in under 5 minutes, after creating a pr on bugzilla. At first I was angry, and frustrated. But decided to make it a challenge/contest, and see my way to thwarting their attacks. Long story short; I think I'm on the right track; In just over a month, I've managed to trap just under 3 million (2,961,264) *bonafide* SPAM sources. I've been honing, and tuning my approach to insure that there are zero false positives, and at the same time, make it more, and more efficient. So now that I'm dropping packets from *so* many IP's I'm wondering if it's not time to better tune pf(4). I've never worked pf hard enough to do any more than create a table, and a few simple rules. But I think I need to do more. Here's the bulk of what I'm using now: ### set loginterface re0 set block-policy drop set fingerprints "/etc/pf.os" scrub in all set skip on lo0 antispoof quick for lo0 antispoof for re0 inet table persist file "/etc/SPAMMERS" block in log quick on re0 proto tcp from to port {smtp, submission, pop3, imap, imaps} ### Would set optimization be warranted? Any thoughts, or advice greatly appreciated! If I’m reading the code right the table lookup already uses a radix table internally, so I would already expect this to perform as well as it’s going to. Arguably you could just drop all traffic from them on all interfaces, but I doubt that’ll make a huge difference. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: pf, ALTQ and 10G
On 28 Mar 2017, at 9:33, Eugene M. Zheganin wrote: I need to implement QoS on a 10G interface (ix(4)) with bandwidth of 4-5 Gbit/sec. In general I'm using pf on FreeBSD, since I like it more than ipfw. But I'm aware that it's kind of ancient and wasn't updated for a long time from the upstream (and the upstream still doesn't support SMP). So, my question is - is it worth to stick to pf/ALTQ on 10G interfaces ? Will pf carry such traffic ? Be aware that ALTQ will not let you configure queues with that sort of bandwidth. All of the datatypes used are 32-bit integers and top out at 2 or 4 Gbps. Unfortunately dummynet has exactly the same problem, so switching to ipfw won’t help. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Support for the enc(4) pseudo-interface
On 21 Mar 2017, at 12:44, Miroslav Lachman wrote: Kristof Provost wrote on 2017/03/21 10:18: On 21 Mar 2017, at 9:43, Marin Bernard wrote: If there is no SA, it is impossible for a peer to ping another. As soon as IKE creates a SA, however, ping starts working. As you can see, the last rule is explicitely bound to the inexistent enc0 interface, and yet is working fine. Can you try without the enc0 rule? I suspect that what’s happening here is that the IPSec traffic is bypassing the firewall altogether. If that's the case the your traffic will still flow, even without the pass on enc0 rule. If you want to filter on it it should work if you add ‘device enc’ to your kernel config. The man page suggests that should then allow you to filter IPSec traffic on enc0. Shouldn't it be included in GENERIC if IPSec is now part of it? It seems illogical to build own kernel for IPsec if IPSec was included in GENERIC for 11.0 ... but without enc. Yeah, perhaps it should be. I’ve not used it myself, so I don’t know if/how well it works now, but unless it breaks things or introduces significant performance regressions we should probably turn it on too. Martin, could you give us an idea of how well this works for you when you’ve got the time to set it up? Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Support for the enc(4) pseudo-interface
On 21 Mar 2017, at 9:43, Marin Bernard wrote: Thanks for answering. Yes, I know that pf accepts rules mentioning inexistent interfaces. What puzzles me here is that my ruleset is actually working. With peer0 = 1.2.3.4 and peer1 = 5.6.7.8, the following ruleset works as expected: - peers = "{1.2.3.4, 5.6.7.8}" set skip on lo block all # Allow IKE pass in proto {tcp, udp} from $peers to self port isakmp pass out proto {tcp, udp} from self to $peers port isakmp # Allow ICMPv4 echo requests only through IPsec pass in on enc0 proto icmp from $peers to self icmp-type echoreq - If there is no SA, it is impossible for a peer to ping another. As soon as IKE creates a SA, however, ping starts working. As you can see, the last rule is explicitely bound to the inexistent enc0 interface, and yet is working fine. Can you try without the enc0 rule? I suspect that what’s happening here is that the IPSec traffic is bypassing the firewall altogether. If that's the case the your traffic will still flow, even without the pass on enc0 rule. If you want to filter on it it should work if you add ‘device enc’ to your kernel config. The man page suggests that should then allow you to filter IPSec traffic on enc0. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Support for the enc(4) pseudo-interface
On 20 Mar 2017, at 23:08, Marin Bernard wrote: Yet, it appears that pf is able to handle references to enc(4) in its ruleset even if the kernel does not support it. Is it expected behaviour? Is it safe to use such a configuration on a production machine ? pf accepts rules for interfaces that don’t exist (yet), so this is expected, but it won’t do what you want it to do. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: sonewconn: pru_attach() failed and kernel panic in PF
So it turns out I shouldn't commit things when jet lagged. You want r314810 in head. The other one was mistakenly done in stable/11. It needed to go there sooner or later so I'm just going to leave it. Regards, Kristof > On 5 Mar 2017, at 22:19, Kristof Provost wrote: > >> On 5 Mar 2017, at 21:42, Kristof Provost wrote: >> There’s only a couple of calls to uma_zfree() in pf_get_translations(). >> >> These are: >> * uma_zfree(V_pf_state_key_z, skp); >> * uma_zfree(V_pf_state_key_z, *nkp); >> * uma_zfree(V_pf_state_key_z, *skp); >> >> Going by the inconsistent pointer use the first one is rather suspect. >> Looking a bit deeper, pf_get_translation() is only called from one place, >> and it always passes stack variables for skp and nkp, so the first call >> ends up trying to free that, which won’t work too well. >> >> That’s a bug (and I’ll fix it), but you’re only running into it because >> pf_state_key_clone() returned NULL, which will only happen under memory >> pressure. >> > The fix is done in r314702. > > Regards, > Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: sonewconn: pru_attach() failed and kernel panic in PF
On 5 Mar 2017, at 21:42, Kristof Provost wrote: There’s only a couple of calls to uma_zfree() in pf_get_translations(). These are: * uma_zfree(V_pf_state_key_z, skp); * uma_zfree(V_pf_state_key_z, *nkp); * uma_zfree(V_pf_state_key_z, *skp); Going by the inconsistent pointer use the first one is rather suspect. Looking a bit deeper, pf_get_translation() is only called from one place, and it always passes stack variables for skp and nkp, so the first call ends up trying to free that, which won’t work too well. That’s a bug (and I’ll fix it), but you’re only running into it because pf_state_key_clone() returned NULL, which will only happen under memory pressure. The fix is done in r314702. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: sonewconn: pru_attach() failed and kernel panic in PF
On 27 Feb 2017, at 21:08, Ross wrote: Hello One of my machines panics almost every day. It is always like this: first there is a number of messages about "sonewconn: pcb 0xf80085478740: pru_attach() failed" at the same time and then panic. Here's an example: ... many lines of sonewconn ... Feb 27 13:41:43 core kernel: sonewconn: pcb 0xf8008575bcb0: pru_attach() failed Feb 27 13:41:43 core kernel: I wonder if you’re running low on memory by any chance. I think I know why you’re crashing, but I suspect your root problem is that you’re running low on memory and that’s why you’re seeing the pru_attach() failures, and eventually running into the pf panic. Feb 27 13:41:43 core kernel: KDB: stack backtrace: Feb 27 13:41:43 core kernel: #0 0x80b312c7 at kdb_backtrace+0x67 Feb 27 13:41:43 core kernel: #1 0x80ae5c92 at vpanic+0x182 Feb 27 13:41:43 core kernel: #2 0x80ae5b03 at panic+0x43 Feb 27 13:41:43 core kernel: #3 0x80fd6d51 at trap_fatal+0x351 Feb 27 13:41:43 core kernel: #4 0x80fd6f43 at trap_pfault+0x1e3 Feb 27 13:41:43 core kernel: #5 0x80fd64ec at trap+0x26c Feb 27 13:41:43 core kernel: #6 0x80fb9d61 at calltrap+0x8 Feb 27 13:41:43 core kernel: #7 0x80e4185e at uma_zfree_arg+0x4fe Feb 27 13:41:43 core kernel: #8 0x82442165 at pf_get_translation+0x2c5 There’s only a couple of calls to uma_zfree() in pf_get_translations(). These are: * uma_zfree(V_pf_state_key_z, skp); * uma_zfree(V_pf_state_key_z, *nkp); * uma_zfree(V_pf_state_key_z, *skp); Going by the inconsistent pointer use the first one is rather suspect. Looking a bit deeper, pf_get_translation() is only called from one place, and it always passes stack variables for skp and nkp, so the first call ends up trying to free that, which won’t work too well. That’s a bug (and I’ll fix it), but you’re only running into it because pf_state_key_clone() returned NULL, which will only happen under memory pressure. What should I do to fix it? You’ll need to look at your system and figure out who’s running away with all of your memory. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: udp - weird behavior of reply-to
On 9 Jan 2017, at 18:25, Marek Zarychta wrote: On Sun, Jan 08, 2017 at 07:08:10PM +0100, Kristof Provost wrote: On 8 Jan 2017, at 15:55, Marek Zarychta wrote: The problem description doesn’t ring any bells with me, but I’m also not sure I’ve fully understood it. Can you document a minimal reproduction scenario, with a pf.conf and perhaps network captures documenting the problem? There’s certainly not been a conscious decision to break UDP reply-to. Let me apologize, the problem wasn't previously properly identified. It seems to be more problem of UDP protocol implementation than PF issue. UDP sockets are opened and bound to address of the outgoing interface (interface which has a route to the client). Because the socket is not bound to the incoming interface, the PF reply-to rules couldn't be evaluated. By the way, TCP sockets are bound to the interface where the traffic arrives and everything works fine. This machine is i386 running 11.0-STABLE r311772 The problem remains unresolved. Are there any corresponding sysctls correcting this behavior and enabling the opportunity to use PF assisted symmetric routing scenario again? Thinking about this a bit more, I think the behaviour you see is entirely correct and expected. We’re talking about datagram sockets, and as far as the kernel is concerned there’s no relationship between the packet you’ve just received from address X and the packet you send to host X. There’s no established connection. As a result it’s entirely free to choose its source address: you’re simply telling the kernel “Send this data to X”, you’re not adding “it’s from Y”. If you want this to behave differently I think you’ll have to convince your application to open a socket per interface (binding it to that interface), and reply using the correct socket. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: udp - weird behavior of reply-to
On 9 Jan 2017, at 18:25, Marek Zarychta wrote: On Sun, Jan 08, 2017 at 07:08:10PM +0100, Kristof Provost wrote: On 8 Jan 2017, at 15:55, Marek Zarychta wrote: The problem description doesn’t ring any bells with me, but I’m also not sure I’ve fully understood it. Can you document a minimal reproduction scenario, with a pf.conf and perhaps network captures documenting the problem? There’s certainly not been a conscious decision to break UDP reply-to. Let me apologize, the problem wasn't previously properly identified. It seems to be more problem of UDP protocol implementation than PF issue. UDP sockets are opened and bound to address of the outgoing interface (interface which has a route to the client). Because the socket is not bound to the incoming interface, the PF reply-to rules couldn't be evaluated. By the way, TCP sockets are bound to the interface where the traffic arrives and everything works fine. This machine is i386 running 11.0-STABLE r311772 The problem remains unresolved. Are there any corresponding sysctls correcting this behavior and enabling the opportunity to use PF assisted symmetric routing scenario again? How are your UDP listen sockets set up? Are they bound to a specific interface, or are they listening on 0.0.0.0? I’m afraid I’m still struggling to understand what your setup is, what you’re expecting to see and what you’re seeing instead. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: udp - weird behavior of reply-to
On 8 Jan 2017, at 15:55, Marek Zarychta wrote: Is it a bug to be officially submitted or it will not be possible to use reply-to for UDP traffic anymore? The problem description doesn’t ring any bells with me, but I’m also not sure I’ve fully understood it. Can you document a minimal reproduction scenario, with a pf.conf and perhaps network captures documenting the problem? There’s certainly not been a conscious decision to break UDP reply-to. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Slow NAT on 10.3-RELEASE
On 11 Oct 2016, at 10:34, Kamil Choudhury wrote: I've seen some mention of checksum issues on NAT limiting performance, but that seems to have been fixed as of 10.2 in an errata. Have I stumbled upon an actual problem, or have I misconfigured something? It’s worth trying the workaround (i.e. disable all checksum offloading on your interfaces). I’ve had at least one bug report indicating that the checksum patch is not 100% correct, but I’ve not had the time to investigate that in-depth. What virtualisation system are you using? Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: dscp set/get
On 6 Oct 2016, at 15:01, Mark Martinec wrote: Just adding recognition to a parser for a couple of DSCP constants to be mapped to TOS is not the solution. Keep in mind that DSCP is a 6-bit field, and TOS is an 8-bit field. The remaining two bits are used for ECN (Explicit Congestion Notification). Setting TOS field with the intention of changing DSCP clobbers ECN bits. Pf needs a proper support for DSCP, currently it is unusable for this purpose in FreeBSD (one can use the PF from OpenBSD). Hmm, that’s a good point. OpenBSD seem to just always preserve the ECN bits (so there’s no dscp keyword). Perhaps we should do the same. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: dscp set/get
On 6 Oct 2016, at 10:30, Franco Fichtner wrote: On 06 Oct 2016, at 10:10 AM, Kristof Provost wrote: On 6 Oct 2016, at 6:57, Eugene M. Zheganin wrote: pf still lacks the DSCP handling, will it be difficult/expensive to add this ? AFAIK ipfw got this recently. pf has set-tos and tos keywords. What is it not letting you do? I think it works already with the plain hex/decimal values given, though we're missing OpenBSD's parser support for special DSCP options with its funky naming: "This rule applies to packets with the specified TOS bits set. string may be one of critical, inetcontrol, lowdelay, netcontrol,throughput, reliability, or one of the DiffServ Code Points: ef, af11 ... af43, cs0 ... cs7; number may be either a hex or decimal number." http://man.openbsd.org/OpenBSD-current/man5/pf.conf.5 Looks like pfctl(8) is the only thing to "fix" here. I'll take a look. Oh, yes, if you want those constants that should be a straightforward port of their changes to parse.y. I have no objections to including that. It’s another (small) step in bringing our pf closer to openbsd pf. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: dscp set/get
On 6 Oct 2016, at 6:57, Eugene M. Zheganin wrote: > pf still lacks the DSCP handling, will it be difficult/expensive to add > this ? AFAIK ipfw got this recently. > pf has set-tos and tos keywords. What is it not letting you do? Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: pf fastroute tag removal reviewers needed
On 28 Sep 2016, at 13:53, Franco Fichtner wrote: The main culprit of pfil not working correctly is pf's route-to and reply-to (and the tag formerly known as fastroute) as they would call if_output directly on the ifnet and consume their packets this way. That transmit code is also copied from if_output() and should likely not be called from within pf, especially when there is a pfil hook chain to go through. Agreed, but there’s another culprit: the v6 fragment handling code. It needs to call ip6_output()/ip6_forward() because it generates multiple output packets. Dealing with that has been on my todo list for a while now, but I’ve not even found the time to make a start at it. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
[Differential] D1944: PF and VIMAGE fixes
kristof abandoned this revision. REVISION DETAIL https://reviews.freebsd.org/D1944 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: kristof, trociny, gnn, zec, rodrigc, glebius, eri, bz, nvass-gmx.com Cc: ryan_timewasted.me, mmoll, javier_ovi_yahoo.com, farrokhi, julian, robak, freebsd-virtualization-list, freebsd-pf-list, freebsd-net-list ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
[Differential] D1944: PF and VIMAGE fixes
kristof commandeered this revision. kristof edited reviewers, added: nvass-gmx.com; removed: kristof. kristof added a comment. I'm commandeering this so it can be closed, because the patch fro bz@ (https://reviews.freebsd.org/D6924) has been included. REVISION DETAIL https://reviews.freebsd.org/D1944 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: kristof, trociny, gnn, zec, rodrigc, glebius, eri, bz, nvass-gmx.com Cc: ryan_timewasted.me, mmoll, javier_ovi_yahoo.com, farrokhi, julian, robak, freebsd-virtualization-list, freebsd-pf-list, freebsd-net-list ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: How to set tos to 0
On 10 Aug 2016, at 16:23, Radek Krejča wrote: > this patch seems to be working. > Thanks for testing! > I will post bugreport. > The patch has already been committed to head (r303663). A bug would still be useful so I don’t forget to merge it back to 11 and 10. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Max altq bandwidth 4.26 Gbit
On 10 Aug 2016, at 14:38, Radek Krejča wrote: I have changed bandwidth to 100%, 90% or 95%. Syntax OK, but value stops at 1.27Gbit (it looks, that 1Gbit is default) When I give ifconfig, I see: media: Ethernet autoselect (10Gbase-SR ) It looks that "autodetection" of pf is broken to. I was afraid of that. I think the issue there is that when pf asks for the speed of the interface it puts a 64-bit value in a 32-bit field, so the resulting value is incorrect. Please do file a bug, because you’ve discovered a real problem and I’d hate for it to get forgotten about. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Max altq bandwidth 4.26 Gbit
On 10 Aug 2016, at 11:19, Radek Krejča wrote: That looks like you might be hitting the maximum of an unsigned integer. Try using relative specifications (i.e. as a percentage) instead. Yes, I think so. But I dont know, that I can say relative specification for inteface bandwidth. Could you show me how? I don’t run ALTQ myself, so all I can say is what the man page says: bandwidth ⟨bw⟩ The maximum bitrate for all queues on an interface may be specified using the bandwidth keyword. The value can be specified as an absolute value or as a percentage of the interface bandwidth. When using an absolute value, the suffixes b, Kb, Mb, and Gb are used to represent bits, kilobits, megabits, and gigabits per second, respectively. The value must not exceed the interface bandwidth. If bandwidth is not specified, the interface bandwidth is used (but take note that some interfaces do not know their bandwidth, or can adapt their bandwidth rates). I’d expect that ‘altq on $int_if cbq bandwidth 85% queue { default_nat.’ would do what you want. Looking at the code, I’m not at all sure that it’ll end up working either, but it’s worth a try. Fundamentally, we’ll have to change pf (and worse, the interface to user space) to use 64-bit integers to carry bandwidth information, not 32-bit ones. Can you file a bug so this doesn’t get forgotten? I have 10Gb line and I need to shape my client with exact bandwidth. How can I manage with your advice? You can only specify integer percentages (so 85%, not 85.5%), so I’m afraid you’ll lose some accuracy. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Max altq bandwidth 4.26 Gbit
On 10 Aug 2016, at 9:28, Radek Krejča wrote: I need to shape 10G traffic, but I cant make bandwidth higher than 4.26 Gbit: pfctl shows: altq on int0 cbq bandwidth 4.26Gb tbrsize 36000 queue { default_nat.. but in pf.conf is: altq on $int_if cbq bandwidth 8550Mb queue { default_nat.. or altq on $int_if cbq bandwidth 10Gb queue { default_nat That looks like you might be hitting the maximum of an unsigned integer. Try using relative specifications (i.e. as a percentage) instead. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: How to set tos to 0
On 31 Jul 2016, at 19:46, Radek Krejča wrote: I need to set TOS to 0 and remark it with rules. I am trying to use scrub to set tos to 0, but I have problem: scrub all fragment reassemble no-df set-tos 0 give Illegal value but scrub all fragment reassemble no-df set-tos 1 is working. I am trying 0x00, 0x0 and still the same. How can I set TOS to 0? I think you may have found a bug. Can you give this patch a try? diff --git a/sbin/pfctl/parse.y b/sbin/pfctl/parse.y index e0cfa3d..980976e 100644 --- a/sbin/pfctl/parse.y +++ b/sbin/pfctl/parse.y @@ -3593,8 +3593,8 @@ tos : STRING{ else if ($1[0] == '0' && $1[1] == 'x') $$ = strtoul($1, NULL, 16); else - $$ = 0; /* flag bad argument */ - if (!$$ || $$ > 255) { + $$ = 256; /* flag bad argument */ + if ($$ < 0 || $$ > 255) { yyerror("illegal tos value %s", $1); free($1); YYERROR; @@ -3603,7 +3603,7 @@ tos : STRING{ } | NUMBER{ $$ = $1; - if (!$$ || $$ > 255) { + if ($$ < 0 || $$ > 255) { yyerror("illegal tos value %s", $1); YYERROR; } Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: fragments processing
On 20 May 2016, at 18:57, Max wrote: 20.05.2016 11:53, Kristof Provost пишет: On 19 May 2016, at 19:49, Max wrote: The number of used frags (almost) never decreases. I don't have enough experience in programming. But I guess that the problem may be in "frag->fr_timeout = time_second;" in pf_fillup_fragment() (pf_norm.c). It should be "frag->fr_timeout = time_uptime;". Actually, I don't now the difference between those variables. So, correct me if I'm wrong. I think you’re right. If I’m reading the code right time_second is unix time, but time_uptime is the number of seconds the system has been up. I rebuilt the kernel. It seems the problem is gone. Awesome, thanks for testing. I’ve committed the fix, and will merge it back to stable/10 in a couple of days. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"