Re: [pfSense Support] Symmetrically routing connection with Multi-WAN and NAT [resolved]
Last message in my monologue now, for it is resolved!
I was half-way through hacking in some rudimentary support to add a
reply-to setting to individual firewall rules when I stumbled upon this
in /etc/inc/filter.inc - turns out somebody already solved this problem
before me!
if($rule['gateway'] == "" && $aline['direction'] <> "" &&
interface_has_gateway($rule['interface']) &&
!isset($config['system']['disablereplyto'])) {
$rg = get_interface_gateway($rule['interface']);
if(is_ipaddr($rg)) {
$aline['reply'] = "reply-to ( {$ifcfg['if']}
{$rg} ) ";
} else {
if($rule['interface'] <> "pptp") {
log_error("Could not find gateway for
interface({$rule['interface']}).");
}
}
}
After printlining the code I was able to determine that it wasn't adding
the reply-to clause because interface_has_gateway($rule['interface'])
was false. Turns out I had neglected to set the "Gateway" on the network
interfaces themselves, in the interface settings! So, finally, I was
able to get a point-and-click solution and I don't have to patch
anything. Yay!
-
To unsubscribe, e-mail: support-unsubscr...@pfsense.com
For additional commands, e-mail: support-h...@pfsense.com
Commercial support available - https://portal.pfsense.org
Re: [pfSense Support] Symmetrically routing connection with Multi-WAN and NAT
On 2011-04-18 13:31, Per von Zweigbergk wrote:
OK, so having researched this a little more, it seems that what I
really want is to put the reply-to option on either the firewall rule
or the NAT rule (haven't quite figured out which one of those it goes
on.) as per from the manual page of pf.conf(5):
/reply-to/
The/reply-to/ option is similar to/route-to/, but routes packets that
pass in the opposite direction (replies) to the specified inter-
face. Opposite direction is only defined in the context of a state
entry, and/reply-to/ is useful only in rules that create state. It
can be used on systems with multiple external connections to route
all outgoing packets of a connection through the interface the
incoming connection arrived through (symmetric routing enforce-
ment).
You can do route-to by using the "gateway" dropdown, but a "reply-to"
dropdown seems to be missing. This would seem to be an option that
needs to be added to the web UI if I want this to work, unless I'm
missing something.
Sorry to be having a mailing list monologue here, but I managed to get
this to work.
This is what I ended up doing:
[2.0-RC1][r...@edgefw.multiwanlab.ita.local]/tmp(88): diff -u
rules.debug rules.mod
--- rules.debug 2011-04-18 11:43:21.0 +
+++ rules.mod 2011-04-18 12:12:42.0 +
@@ -99,9 +99,9 @@
pass in quick on em2 proto tcp from any to (em2) port { 80 443 22 }
keep state label "anti-lockout rule"
# User-defined rules follow
-pass in log quick on $WAN1 proto tcp from any to 172.16.0.51
flags S/SA keep state label "USER_RULE: NAT chargen wan1"
+pass in log quick on $WAN1 reply-to (em1 172.16.1.2) proto tcp from
any to 172.16.0.51 flags S/SA keep state label "USER_RULE: NAT chargen
wan1"
pass in log quick on $LAN from 172.16.0.0/24 to any keep state
label "USER_RULE: Default allow LAN to any rule"
-pass in log quick on $WAN2 proto tcp from any to 172.16.0.51
flags S/SA keep state label "USER_RULE: NAT chargen wan2"
+pass in log quick on $WAN2 reply-to (em0 172.16.2.2) proto tcp from
any to 172.16.0.51 flags S/SA keep state label "USER_RULE: NAT chargen
wan2"
# VPN Rules
anchor "tftp-proxy/*"
Then I just loaded that modified ruleset with "pfctl -f /tmp/rules.mod".
I also removed all the ineffective policy routing / floating rules stuff
out of there. All that's needed is a reply-to on the incoming firewall
rule matching the NAT traffic to begin with.
Now of course, this change isn't persistent, it's going to break
whenever I reboot or make a change to the ruleset. So I figure I'll just
have to patch pfSense to add this feature to the editor for firewall
rules - assuming it's not too complicated.
Re: [pfSense Support] Symmetrically routing connection with Multi-WAN and NAT
OK, so having researched this a little more, it seems that what I really want is to put the reply-to option on either the firewall rule or the NAT rule (haven't quite figured out which one of those it goes on.) as per from the manual page of pf.conf(5): /reply-to/ The/reply-to/ option is similar to/route-to/, but routes packets that pass in the opposite direction (replies) to the specified inter- face. Opposite direction is only defined in the context of a state entry, and/reply-to/ is useful only in rules that create state. It can be used on systems with multiple external connections to route all outgoing packets of a connection through the interface the incoming connection arrived through (symmetric routing enforce- ment). You can do route-to by using the "gateway" dropdown, but a "reply-to" dropdown seems to be missing. This would seem to be an option that needs to be added to the web UI if I want this to work, unless I'm missing something.
Re: [pfSense Support] Symmetrically routing connection with Multi-WAN and NAT
On 2011-04-18 12:38, Chris Buechler wrote:
You can do that with floating rules. Check your resulting floating
rules in /tmp/rules.debug to ensure you have them configured
correctly, and enable logging on all your rules so you can determine
which rule matched.
You're right. It is possible to add those specific rules using Floating
Rules. Thanks for telling me about /tmp/rules.debug, this is very much
more helpful than trying to make sense of whatever's dumped by pfctl.
They do appear in that file as follows:
GWWAN2 = " route-to ( em0 172.16.2.2 ) "
pass out log on { em1 } $GWWAN2 from 172.16.2.1 to any keep state
label "USER_RULE"
I have pasted the full /tmp/rules.debug at the end of this message for
the full listing. And this seems to be what I asked for - with the
exception of "keep state". I have tried with "no state" as well, and got
the same result.
As you can see I've enabled logging on this particular rule as well. But
the rule never seems to match, at least not when I look in the Web UI.
The only rule I see matched is this:
@40 pass in log quick on em0 inet proto tcp from any to 172.16.0.51
flags S/SA keep state label "USER_RULE: NAT chargen wan2"
My current instinct is that the return packets never even hit the
firewall rules so that they may be policy routed, since the packet is
part of a state. But I have no idea how I would verify that, or what I'd
do to remedy that situation.
---
[2.0-RC1][r...@edgefw.multiwanlab.ita.local]/root(27): cat /tmp/rules.debug
#System aliases
loopback = "{ lo0 }"
WAN1 = "{ em1 }"
LAN = "{ em2 }"
WAN2 = "{ em0 }"
#SSH Lockout Table
table persist
table persist
#pfSnortSam tables
table
table
table
table
# User Aliases
# Gateways
GWWAN2 = " route-to ( em0 172.16.2.2 ) "
GWWAN1 = " route-to ( em1 172.16.1.2 ) "
set loginterface em1
set loginterface em2
set loginterface em0
set optimization normal
set limit states 22000
set limit src-nodes 22000
set skip on pfsync0
scrub in on $WAN1 allfragment reassemble
scrub in on $LAN allfragment reassemble
scrub in on $WAN2 allfragment reassemble
nat-anchor "natearly/*"
nat-anchor "natrules/*"
# Outbound NAT rules
nat on $WAN1 from 172.16.0.0/24 to any -> 172.16.1.1/32 port 1024:65535
nat on $WAN2 from 172.16.0.0/24 to any -> 172.16.2.1/32 port 1024:65535
# Load balancing anchor
rdr-anchor "relayd/*"
# TFTP proxy
rdr-anchor "tftp-proxy/*"
table { 172.16.1.0/24 172.16.0.0/24 172.16.2.0/24 }
# NAT Inbound Redirects
rdr on em1 proto tcp from any to 172.16.1.1 port 19 -> 172.16.0.51
rdr on em0 proto tcp from any to 172.16.2.1 port 19 -> 172.16.0.51
# UPnPd rdr anchor
rdr-anchor "miniupnpd"
anchor "relayd/*"
#---
# default deny rules
#---
block in log all label "Default deny rule"
block out log all label "Default deny rule"
# We use the mighty pf, we cannot be fooled.
block quick proto { tcp, udp } from any port = 0 to any
block quick proto { tcp, udp } from any to any port = 0
# Block all IPv6
block in quick inet6 all
block out quick inet6 all
# pfSnortSam
block quick from to any label "Block snort2c hosts"
block quick from any to label "Block snort2c hosts"
block quick from to any label "Block pfSnortSamOut hosts"
block quick from any to label "Block pfSnortSamIn hosts"
# SSH lockout
block in log quick proto tcp from to any port 22 label
"sshlockout"
# webConfigurator lockout
block in log quick proto tcp from to any port
443 label "webConfiguratorlockout"
block in quick from to any label "virusprot overload table"
antispoof for em1
antispoof for em2
# allow access to DHCP server on LAN
pass in on $LAN proto udp from any port = 68 to 255.255.255.255 port =
67 label "allow access to DHCP server"
pass in on $LAN proto udp from any port = 68 to 172.16.0.1 port = 67
label "allow access to DHCP server"
pass out on $LAN proto udp from 172.16.0.1 port = 67 to any port = 68
label "allow access to DHCP server"
antispoof for em0
# loopback
pass in on $loopback all label "pass loopback"
pass out on $loopback all label "pass loopback"
# let out anything from the firewall host itself and decrypted IPsec traffic
pass out all keep state allow-opts label "let out anything from firewall
host itself"
# make sure the user cannot lock himself out of the webConfigurator or SSH
pass in quick on em2 proto tcp from any to (em2) port { 80 443 22 }
keep state label "anti-lockout rule"
# User-defined rules follow
pass out log on { em1 } from 172.16.2.1 to no state label
"NEGATE_ROUTE: Negate policy route for vpn(s)"
pass out log on { em1 } $GWWAN2 from 172.16.2.1 to any no state
pass out log on { em0 } from 172.16.1.1 to no state lab
pass out log on { em0 } $GWWAN1 from 172.16.1.1 to any no state
pass in log quick on $WAN1 proto tcp from any to 172.16.0.51 fl
pass in log quick on
Re: [pfSense Support] Symmetrically routing connection with Multi-WAN and NAT
On Mon, Apr 18, 2011 at 6:14 AM, Per von Zweigbergk wrote: > I have the following set up in a lab: > > [WinXP](LAN)[edgefw](WAN1)(Link1)[mock- ](WAN)[to my "real" LAN] > [ ](WAN2)(Link2)[router](LAN)[Win7] > > > The WinXP box has a chargen server running for testing (I just installed the > Windows XP Simple Internet Services). > > Edgefw and mockrouter are running pfSense 2.0-RC1. > > What this setup is intended to simulate is a Multi-WAN scenario, where > edgefw is the router for a LAN which has two different WAN connections that > are NATed. > > Mockrouter is inserted into the mix simply to simulate the two seperate > Internet connections (Link1 and Link2). Mockrouter's WAN connection to my > real LAN is just a convenience for me so I can still access the web on my > lab machines and not strictly relevant to the lab. > > Edgefw is configured with two WAN connections, and has a default route out > of both of them. The specific part of this setup I'm having difficulty with > is routing reply packets for TCP connections. > > What I have done is that I've made two port forwards on edgefw, from the > WAN1 edge IP and the WAN2 edge IP into the WinXP machine on the TCP chargen > port. > > When WAN1 is the default gateway, if I connect from my Win7 box to edgefw's > WAN1 IP on the chargen port packets in both directions flow through WAN1. > This is to be expected. > > If I instead connect the same way, but with WAN2's IP, the packets going > from Win7 to WinXP flow through WAN2, which is to be expected. However, > packets returning on the same connection will exit on WAN1. Which is > expected, but not desired - WAN1 is the default route after all, and it's > not like the kernel makes routing decisions based on pf's state table. > > To solve this problem, I googled, and I turned up with the following > solution that applies to hand-written pf that I believe would work in my > scenario: > > pass out on $ext_if1 from $ext_if2 route-to ($ext_if2 $ext_gw2) > pass out on $ext_if2 from $ext_if1 route-to ($ext_if1 $ext_gw1) > You can do that with floating rules. Check your resulting floating rules in /tmp/rules.debug to ensure you have them configured correctly, and enable logging on all your rules so you can determine which rule matched. - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org
[pfSense Support] Symmetrically routing connection with Multi-WAN and NAT
I have the following set up in a lab: [WinXP](LAN)[edgefw](WAN1)(Link1)[mock- ](WAN)[to my "real" LAN] [ ](WAN2)(Link2)[router](LAN)[Win7] The WinXP box has a chargen server running for testing (I just installed the Windows XP Simple Internet Services). Edgefw and mockrouter are running pfSense 2.0-RC1. What this setup is intended to simulate is a Multi-WAN scenario, where edgefw is the router for a LAN which has two different WAN connections that are NATed. Mockrouter is inserted into the mix simply to simulate the two seperate Internet connections (Link1 and Link2). Mockrouter's WAN connection to my real LAN is just a convenience for me so I can still access the web on my lab machines and not strictly relevant to the lab. Edgefw is configured with two WAN connections, and has a default route out of both of them. The specific part of this setup I'm having difficulty with is routing reply packets for TCP connections. What I have done is that I've made two port forwards on edgefw, from the WAN1 edge IP and the WAN2 edge IP into the WinXP machine on the TCP chargen port. When WAN1 is the default gateway, if I connect from my Win7 box to edgefw's WAN1 IP on the chargen port packets in both directions flow through WAN1. This is to be expected. If I instead connect the same way, but with WAN2's IP, the packets going from Win7 to WinXP flow through WAN2, which is to be expected. However, packets returning on the same connection will exit on WAN1. Which is expected, but not desired - WAN1 is the default route after all, and it's not like the kernel makes routing decisions based on pf's state table. To solve this problem, I googled, and I turned up with the following solution that applies to hand-written pf that I believe would work in my scenario: pass out on $ext_if1 from $ext_if2 route-to ($ext_if2 $ext_gw2) pass out on $ext_if2 from $ext_if1 route-to ($ext_if1 $ext_gw1) This is taken from the PF User's Guide (http://www.openbsd.org/faq/pf/pools.html) So, I tried to replicate this very set of rules using the Floating Rules functionality. I set a rule that would match packets exiting WAN1 with WAN2's source IP address, and perform policy routing on it to shove it out over WAN2 instead. And vice versa. This however did nothing. I tried enabling logging on these rules, and it seems that they are never matched. I'm at a loss as to how to solve this at the moment. I'm suspecting that perhaps the return packets are never subject to firewall rules to begin with (since they are part of a known state) or perhaps, that the floating rules are applied too early (before NATing) or too late (never reached?) in the complete ruleset. What I'm trying to accomplish does not seem too far out of the ordinary. Am I missing something, or is this something that is not acheivable with pfSense? Do I need to build a generic BSD box instead and hand-roll my rules? - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org
