Re: Problem with state and PF on a 4.3 setup
On 2008-05-08, Jon Radel [EMAIL PROTECTED] wrote: You appear making use of the default pass rule for all your outbound traffic, as I didn't notice a single rule that applied to outbound traffic (other than your block port 0, CARP, PFSync, and ping rules). I don't believe that can be counted on to establish state. Unless I missed anything in the commits from the last couple of days, the implicit pass rules definitely do not establish state.
Re: Problem with state and PF on a 4.3 setup
Thanks for the information. This is the first time that I've used PF as a router based firewall and not with NAT. I didn't know that the state was on a per interface basis, and not global to the system. So this means that unless I want to allow all outbound traffic from my firewall, I need to have a matching pass out rule for all the pass in rules for which I want to restrict the inbound interface (ie for which I don't want to put just pass for)? The reason I need quick, especially on a few of these rules, is that the firewall will be establishing 3 to 6 thousand new sessions per second and managing betwee 300 000-500 000 state entries. This means that if it's one state entry per interface, this effectively doubles the state table size. Thanks for the other tips by other people for lists and the implicit keep state, I hadn't even realized I had omitted important lists and didn't know about the implicit keep state. Jon Radel wrote: You appear making use of the default pass rule for all your outbound traffic, as I didn't notice a single rule that applied to outbound traffic (other than your block port 0, CARP, PFSync, and ping rules). I don't believe that can be counted on to establish state. So a packet arrives on an interface, is allowed in with a pass in quick on XX and state is established. The packet is then routed out YY, which is allowed since there is no rule to block it. There is, however, no state established on interface YY, so the return packet is dropped unless you have a rule explicitly allowing that packet in. Try dropping a pass out all into the rule set to see if things get better. (As a test, think about the implications before you put that into production.) --Jon Radel
Re: Problem with state and PF on a 4.3 setup
On Thu, May 08, 2008 at 07:23:41AM -0400, Steve Johnson wrote: Thanks for the information. This is the first time that I've used PF as a router based firewall and not with NAT. I didn't know that the state was on a per interface basis, and not global to the system. So this means that unless I want to allow all outbound traffic from my firewall, I need to have a matching pass out rule for all the pass in rules for which I want to restrict the inbound interface (ie for which I don't want to put just pass for)? No, states are by default global and not tied to an interface. See man pf.conf. -Otto The reason I need quick, especially on a few of these rules, is that the firewall will be establishing 3 to 6 thousand new sessions per second and managing betwee 300 000-500 000 state entries. This means that if it's one state entry per interface, this effectively doubles the state table size. Thanks for the other tips by other people for lists and the implicit keep state, I hadn't even realized I had omitted important lists and didn't know about the implicit keep state. Jon Radel wrote: You appear making use of the default pass rule for all your outbound traffic, as I didn't notice a single rule that applied to outbound traffic (other than your block port 0, CARP, PFSync, and ping rules). I don't believe that can be counted on to establish state. So a packet arrives on an interface, is allowed in with a pass in quick on XX and state is established. The packet is then routed out YY, which is allowed since there is no rule to block it. There is, however, no state established on interface YY, so the return packet is dropped unless you have a rule explicitly allowing that packet in. Try dropping a pass out all into the rule set to see if things get better. (As a test, think about the implications before you put that into production.) --Jon Radel
Re: Problem with state and PF on a 4.3 setup
* Otto Moerbeek [EMAIL PROTECTED] [2008-05-08 13:47]: On Thu, May 08, 2008 at 07:23:41AM -0400, Steve Johnson wrote: Thanks for the information. This is the first time that I've used PF as a router based firewall and not with NAT. I didn't know that the state was on a per interface basis, and not global to the system. So this means that unless I want to allow all outbound traffic from my firewall, I need to have a matching pass out rule for all the pass in rules for which I want to restrict the inbound interface (ie for which I don't want to put just pass for)? No, states are by default global and not tied to an interface. See man pf.conf. now you conusd him even more :) while otto is right, contrary to your belief, the direction of creation is in the state. so given routing doesn't change they are effectively per-interface. it;s just that they can move onto another interfaces if routing changes. for your case, consider skipping filtering on one interface (set skip em5) -- Henning Brauer, [EMAIL PROTECTED], [EMAIL PROTECTED] BS Web Services, http://bsws.de Full-Service ISP - Secure Hosting, Mail and DNS Services Dedicated Servers, Rootservers, Application Hosting - Hamburg Amsterdam
Re: Problem with state and PF on a 4.3 setup
On 2008-05-08, Otto Moerbeek [EMAIL PROTECTED] wrote: On Thu, May 08, 2008 at 07:23:41AM -0400, Steve Johnson wrote: Thanks for the information. This is the first time that I've used PF as a router based firewall and not with NAT. I didn't know that the state was on a per interface basis, and not global to the system. So this means that unless I want to allow all outbound traffic from my firewall, I need to have a matching pass out rule for all the pass in rules for which I want to restrict the inbound interface (ie for which I don't want to put just pass for)? No, states are by default global and not tied to an interface. See man pf.conf. But they are sensitive to direction; if you keep state for a new incoming session on an interface, you: 1. *do* pass _return_ traffic associated with that connection, 2. *do not* pass the incoming traffic that created the state (or any following incoming traffic from the same connection) out of another interface to send to another machine For 2. you can either pass the outbound traffic separately, or you can tag the inbound traffic and pass outbound traffic that has been tagged. ... The reason I need quick, especially on a few of these rules, is that the firewall will be establishing 3 to 6 thousand new sessions per second You should read this set of articles: http://undeadly.org/cgi?action=articlesid=20060927091645
Re: Problem with state and PF on a 4.3 setup
Ah, that explains a lot! Thanks for the information. Yes, what Otto had mentioned was indeed confusing me :-) Especially when I look at the following statement from the faq: http://www.openbsd.org/faq/pf/filter.html#state When a rule creates state, the first packet matching the rule creates a state between the sender and receiver. Now, not only do packets going from the sender to receiver match the state entry and bypass ruleset evaluation, but so do the reply packets from receiver to sender. Plus, in the pf.conf(5) stated that floating was the default behaviour, which I now notice is what Otto pointed out, but that meant me think even more that the pass out would not have been needed, since the state was created with the pass in. I've added a pass out from ! self rule that should take care of everything that I've allowed to come in. It also explains why the state table will grow so much, if there's a state entry for each direction. I'll look more into the tag part which should be even better as well as that link on optimization. Is the state direction tracking something that changed at one point of the PF development or has it always been like that? Thanks again to all for the responses and references, Steve Stuart Henderson wrote: On 2008-05-08, Otto Moerbeek [EMAIL PROTECTED] wrote: On Thu, May 08, 2008 at 07:23:41AM -0400, Steve Johnson wrote: Thanks for the information. This is the first time that I've used PF as a router based firewall and not with NAT. I didn't know that the state was on a per interface basis, and not global to the system. So this means that unless I want to allow all outbound traffic from my firewall, I need to have a matching pass out rule for all the pass in rules for which I want to restrict the inbound interface (ie for which I don't want to put just pass for)? No, states are by default global and not tied to an interface. See man pf.conf. But they are sensitive to direction; if you keep state for a new incoming session on an interface, you: 1. *do* pass _return_ traffic associated with that connection, 2. *do not* pass the incoming traffic that created the state (or any following incoming traffic from the same connection) out of another interface to send to another machine For 2. you can either pass the outbound traffic separately, or you can tag the inbound traffic and pass outbound traffic that has been tagged. ... The reason I need quick, especially on a few of these rules, is that the firewall will be establishing 3 to 6 thousand new sessions per second You should read this set of articles: http://undeadly.org/cgi?action=articlesid=20060927091645
Re: Problem with state and PF on a 4.3 setup
* Steve Johnson [EMAIL PROTECTED] [2008-05-08 14:57]: Is the state direction tracking something that changed at one point of the PF development or has it always been like that? it has always been like that. it is the only sane thing to do. once you exceed that little 2 interfaces firewall scenario you'll see why... you put policies on interfaces, and anyonegoing fron netA to netB must pass the outbound policy on the netA facing interface and the inbound policy in the netB facing interface (to make things more confusing, the inbound policy is what gets written as pass -out- on... anyway). with the one state covering everything you bypass netB's inbound policy, which is both dangerous and stupid. ipfilter does it that way. -- Henning Brauer, [EMAIL PROTECTED], [EMAIL PROTECTED] BS Web Services, http://bsws.de Full-Service ISP - Secure Hosting, Mail and DNS Services Dedicated Servers, Rootservers, Application Hosting - Hamburg Amsterdam
Problem with state and PF on a 4.3 setup
Hi, I have a new setup with a 4.3 PF firewall that includes CARP addresses, trunked VLANs and HA. We've migrated from a different architecture, so the rules have never been tested on a different version before. I've tried to setup the first unit with my ruleset, but all forwarded packets seem to have problems with state. The packets come through, a state table entry is created, they reach the system, but when they come back, they are blocked by PF. I have keep state entries for all of my rules, so I don't know where the problem could be. The ruleset is available here: http://www.sjohnson.info/other/pf.conf The only thing I've removed from the ruleset are aliases and table definitions. When I check for specific entries in the state table, I see them as CLOSED:SYN_SENT. If I disable PF, the packets make it through properly, so it should not be any routing or IP forwarding issue. I also tried conservative instead of aggressive optimization, but it didn't change anything, as I expected. Here are the sysctl settings that I hace changed: net.inet.ip.forwarding=1 net.inet.tcp.recvspace=65536 net.inet.tcp.sendspace=65536 net.inet.carp.preempt=1 Any clue as to what could be the problem? Thanks a lot, Steve Johnson
Re: Problem with state and PF on a 4.3 setup
On Wed, May 07, 2008 at 04:09:43PM -0400, Steve Johnson wrote:
I have a new setup with a 4.3 PF firewall that includes CARP addresses,
trunked VLANs and HA. We've migrated from a different architecture, so the
rules have never been tested on a different version before. I've tried to
setup the first unit with my ruleset, but all forwarded packets seem to
have problems with state. The packets come through, a state table entry is
created, they reach the system, but when they come back, they are blocked
by PF.
I have keep state entries for all of my rules, so I don't know where the
problem could be. The ruleset is available here:
http://www.sjohnson.info/other/pf.conf
The only thing I've removed from the ruleset are aliases and table
definitions.
When I check for specific entries in the state table, I see them as
CLOSED:SYN_SENT. If I disable PF, the packets make it through properly,
so it should not be any routing or IP forwarding issue. I also tried
conservative instead of aggressive optimization, but it didn't change
anything, as I expected.
Here are the sysctl settings that I hace changed:
net.inet.ip.forwarding=1
net.inet.tcp.recvspace=65536
net.inet.tcp.sendspace=65536
net.inet.carp.preempt=1
Any clue as to what could be the problem?
Not really, I'm afraid, but some ideas:
- I see you've marked everything as block log - is there anything on
pflog0 (pflog(4), tcpdump(8))? If so, which rule is triggered?
- if pf is enabled, can the firewall access and be accessed by all hosts
involved in the testing? (That is, are you sure that routing is the
only thing that fails?)
- does this happen for all protocols (TCP/UDP/ICMP e.a.)?
- if nobody else has a good idea, could you create a dump with tcpdump
and post it (ASCII output should do, I believe)? On all involved
interfaces, please.
- your ruleset could be a lot more compact if you used { a, b, c }
everywhere (antispoof!), and omitted anything unnecessary (keep state
flags S/SA has been the default for several releases, and port =
http can be written as just port http). You might also wish to
reconsider using quick for every rule. But this is purely stylistic.
Joachim
--
TFMotD: pod2latex (1) - convert pod documentation to latex format
Re: Problem with state and PF on a 4.3 setup
Steve Johnson wrote: Hi, I have a new setup with a 4.3 PF firewall that includes CARP addresses, trunked VLANs and HA. We've migrated from a different architecture, so the rules have never been tested on a different version before. I've tried to setup the first unit with my ruleset, but all forwarded packets seem to have problems with state. The packets come through, a state table entry is created, they reach the system, but when they come back, they are blocked by PF. I have keep state entries for all of my rules, so I don't know where the problem could be. The ruleset is available here: http://www.sjohnson.info/other/pf.conf The only thing I've removed from the ruleset are aliases and table definitions. When I check for specific entries in the state table, I see them as CLOSED:SYN_SENT. If I disable PF, the packets make it through properly, so it should not be any routing or IP forwarding issue. I also tried conservative instead of aggressive optimization, but it didn't change anything, as I expected. Here are the sysctl settings that I hace changed: net.inet.ip.forwarding=1 net.inet.tcp.recvspace=65536 net.inet.tcp.sendspace=65536 net.inet.carp.preempt=1 Any clue as to what could be the problem? Thanks a lot, Steve Johnson You appear making use of the default pass rule for all your outbound traffic, as I didn't notice a single rule that applied to outbound traffic (other than your block port 0, CARP, PFSync, and ping rules). I don't believe that can be counted on to establish state. So a packet arrives on an interface, is allowed in with a pass in quick on XX and state is established. The packet is then routed out YY, which is allowed since there is no rule to block it. There is, however, no state established on interface YY, so the return packet is dropped unless you have a rule explicitly allowing that packet in. Try dropping a pass out all into the rule set to see if things get better. (As a test, think about the implications before you put that into production.) --Jon Radel [demime 1.01d removed an attachment of type application/x-pkcs7-signature which had a name of smime.p7s]
Re: Problem with state and PF on a 4.3 setup
Steve Johnson [EMAIL PROTECTED] writes: I have keep state entries for all of my rules, so I don't know where the problem could be. The ruleset is available here: http://www.sjohnson.info/other/pf.conf The only thing I've removed from the ruleset are aliases and table definitions. Leaving those definitions in there (suitably anonymized if need be) would have made it easier to play with for others. But anyway, the first thing that strikes me is that the ruleset logic is a bit hard to follow with all those pass quick rules and the block quick at the end. That final block could be a significant part of the problem, and unless my low caffeine level plays tricks on me, the only pass out I find is for ICMP traffic. If you want traffic through your gateway, you need to pass out to $somewhere as well (or where appropriate just pass from $foo to $bar). It's usually a lot better to start with a block all, then punch the holes you need with pass rules, and add quick only when there's a real need for it. And as Joachim mentioned, using lists and macros in a few places where your rule set now has blocks of very similar rules is extremely good for readability. -- Peter N. M. Hansteen, member of the first RFC 1149 implementation team http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/ Remember to set the evil bit on all malicious network traffic delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
