Advanced IPFW2 Forward rule problem / bug / misunderstanding
Hello,
I have searched the lists for information pertaining to this problem,
but I haven't been able to find anything relevant to my attempted usage
of IPFWs forward action. If there are any preexisting threads that
address my concern, please direct me to them.
I have also sent this message to the freebsd-ipfw list, as this seems
consistent with some other threads I see in their archive.
Here's the situation :
I have two ISPs, each providing two IPs. One of these ISPs is providing
IPs on totally different subnets, and is MAC sensitive. I have two
internal servers (Actually, just one listening on two addresses), and I
want this server to be available externally to both ISPs. (We're
migrating ISPs, and we don't want any interruption in service). I am
using port forwarding in NATd to allow the necessary ports through to
the server. My problem comes with the replies - FreeBSD has only one
default gateway, and all traffic going out, regardless of which external
IP address it is from, goes to that gateway. Since ISP2 doesen't care
much for routing traffic from ISP1, and vice-versa, I have a problem.
I should note here that I am not trying to load balance - I am perfectly
happy with all outbound LAN connections being NATted over one link, I
just need the ability to service inbound connections on all four IPs.
I am using forward rules in my firewall to match packets belonging to
these other interfaces, to forward them to the appropriate gateway.
According to the manpage for ipfw, If /ipaddr/ is not a local address,
then the port number (if specified) is ignored, and the packet will be
forwarded to the remote address, using the route as found in the local
routing table for that IP. I interpret that as The packet's next hop
will be compared to the routing table, and routed out the appropriate
interface to reach that next hop. The problem is that doesen't seem to
be happening. I have tried fiddling a few knobs to no effect -
specifically net.inet.ip.fastforwarding, net.inet.ip.sourceroute and
net.inet.ip.accept_sourceroute.
Telus is the legacy ISP, so when I'm trying these rules all the
inbound server requests are from the two Telus interfaces. I have
numbered a rule here 42000. This rule will catch all kinds of packets
outbound from ${ext1_ip}:80 and ${ext1_ip}:443 to clients on the internet.
This tells me my inbound NAT translation is working, the packets are
getting to the server, replies are coming back, they're matching my
forward rules, but still going out the wrong interface anyways!
As attached as I am to the idea of doing this via ipfw, if anyone has
any suggestions on alternate methods to achieve the same results, I'd
love to hear them!!!
On to the technical details - I have obscured IP addresses here, but the
networks and subnet masks remain the same
---
bsdbox# uname -a
FreeBSD bsdbox 6.0-RELEASE-p1 FreeBSD 6.0-RELEASE-p1 #1: Mon Jan 9
08:15:08 PST 2006 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/BSDBOX i386
---
bsdbox# cat /usr/src/sys/i386/conf/BSDBOX
.
.
.
### FIREWALLING
options IPFIREWALL
options IPFIREWALL_FORWARD
options IPFIREWALL_FORWARD_EXTENDED ( I just did this to test -
it made no difference)
options IPDIVERT
---
bsdbox# cat /etc/rc.conf
.
.
.
##
## Networking
##
gateway_enable=YES
## Ensure interface configuration and Firewall script remain consistent!!
defaultrouter=24.85.92.1
ifconfig_rl0=192.168.1.1
ifconfig_vr0=142.179.109.xxx netmask 255.255.248.0
ifconfig_vr1=216.232.85.xxx netmask 255.255.254.0
ifconfig_rue0=24.85.9x.xxx netmask 255.255.252.0
ifconfig_rue0_alias0=24.85.9x.xxx netmask 255.255.255.255
natd_enable=NO
firewall_enable=YES
firewall_script=/usr/local/etc/firewall.telus+shaw-test
-
bsdbox# cat /usr/local/etc/firewall.telus+shaw-test
# firewall.telus+shaw 0.9.8
# Aquire variables from /etc/rc.conf
if [ -r /etc/rc.conf ]; then
. /etc/rc.conf
fi
fwcmd=/sbin/ipfw -q
##
## THIS SCRIPT REQUIRES THE FOLLOWING VARIABLES
## TO BE CORRECTLY DEFINED!
##
##
# PRIMARY external interface (Telus)
ext1=vr0 # Device name
ext1_ip=142.179.109.xxx # IP Address
ext1_gw=142.179.104.254 # IP Gateway
ext1_bc=142.179.111.255 # Broadcast Address
ext1_srv=192.168.1.10 # Server IP Address
##
##
# SECONDARY external interface (Telus)
ext2=vr1 # Device name
ext2_ip=216.232.85.xxx # IP Address
ext2_nm=255.255.254.0 # Network Mask
ext2_bc=216.232.85.255 # Broadcast Address
ext2_gw=216.232.84.254 # IP Gateway
ext2_srv=192.168.1.11 # IP Address of internal server
##
##
# Shaw Cable Interface(s)
# PRIMARY IP
shaw=rue0 # Device Name
shaw_ip=24.85.93.xxx # IP Address
shaw_nm=255.255.252.0 # Network Mask
shaw_bc=24.85.95.255 # Broadcast
Re: Advanced IPFW2 Forward rule problem / bug / misunderstanding
This should get you most of the way there or at least give you a good
idea of what's required.
options IPFIREWALL_FORWARD_EXTENDED
I'm pretty sure this will be required.
defaultrouter=24.85.92.1
ifconfig_rl0=192.168.1.1
ifconfig_vr0=142.179.109.xxx netmask 255.255.248.0
ifconfig_vr1=216.232.85.xxx netmask 255.255.254.0
ifconfig_rue0=24.85.9x.xxx netmask 255.255.252.0
ifconfig_rue0_alias0=24.85.9x.xxx netmask 255.255.255.255
Telus
ext1_ip=142.179.109.xxx # IP Address
ext1_gw=142.179.104.254 # IP Gateway
ext2_ip=216.232.85.xxx # IP Address
ext2_gw=216.232.84.254 # IP Gateway
Shaw Cable
shaw_ip=24.85.93.xxx # IP Address
shaw_gw=24.85.92.1 # IP Gateway
srv2_ext=24.85.93.xxx # External IP of server
INTERNAL
int_ip=192.168.1.1 # IP Address
# And run our new NATd
/sbin/natd -log_ipfw_denied -i ${nat_in} -o ${nat_out} -s -m -u -n
${shaw} -punch_fw 36000:100 -redirect_port tcp ${ext1_srv}:22
${ext1_ip}: -redirect_port tcp ${ext1_srv}:53 ${ext1_ip}:53
-redirect_port tcp ${ext1_srv}:80 ${ext1_ip}:80 -redirect_port tcp
${ext1_srv}:443 ${ext1_ip}:443 -redirect_port udp ${ext1_srv}:53
${ext1_ip}:53 -redirect_port tcp ${ext2_srv}:80 ${ext2_ip}:80
-redirect_port tcp ${ext2_srv}:443 ${ext2_ip}:443 -redirect_port tcp
${srv1_int}:22 ${shaw_ip}: -redirect_port tcp ${srv1_int}:53
${shaw_ip}:53 -redirect_port udp ${srv1_int}:53 ${shaw_ip}:53
-redirect_port tcp ${srv1_int}:80 ${shaw_ip}:80 -redirect_port tcp
${srv1_int}:443 ${shaw_ip}:443 -redirect_port tcp ${srv2_int}:80
${srv2_ext}:80 -redirect_port tcp ${srv2_int}:443 ${srv2_ext}:443
That's a hefty nat command. Let's simplfy by putting it in a file. I
leave the port forwarding to you.
/etc/rc.conf
natd_enable=yes
natd_flags=-f /etc/natd.conf
/etc/natd.conf
instance default
interface vr0
port 8668
instance telus2
interface vr1
port 8669
instance shaw1
alias_address 24.85.93.xxx
port 8670
instance shaw2
alias_address 24.85.93.xxx
port 8671
globalport 8672
I see that your firewall is based on rc.firewall. Forget rc.firewall, it
is junk. Base your firewall on this structure.
1. Public Interface NAT Diversion
2. check-state
3. Public Interface Leak Prevention
3.1 deny egress from internal hosts
3.2 deny ingress to internal hosts
4. Antispoof
4.1 allow via loopback interface
4.2 deny ingress from router
4.3 deny ingress from internal hosts via public interface
5. Router
5.1 allow egress
5.2 deny egress
5.3 allow ingress
5.4 deny ingress
6. Internal Hosts
6.1 allow egress
6.2 deny egress
6.3 allow ingress
6.4 deny ingress
7. Default Deny
/etc/rc.conf
firewall_enable=yes
firewall_type=/etc/ipfw.rules
/etc/ipfw.rules
-f flush
add divert 8668 ip from any to any in via vr0
add divert 8669 ip from any to any in via vr1
add divert 8670 ip from any to 24.85.93.xxx in via rue0
add divert 8671 ip from any to 24.85.93.xxx in via rue0
#have never known the globalport to work on inbound
add divert 8672 ip from any to any out via { vr0 or vr1 or rue0 }
#not sure if that 'or' will work... may need to split it up
check-state
add deny ip from 192.168.1.1/24 to any via { vr0 or vr1 or rue0 }
add allow ip from me to me via lo0 keep-state
add deny ip from me to any in
add allow ip from me to { me or 192.168.1.1/24 or 142.179.109.xxx/21 or
216.232.85.xxx/23 or 24.85.9x.xxx/22 } keep-state
add forward 142.179.104.254 ip from 142.179.109.xxx to any keep-state
add forward 216.232.84.254 ip from 216.232.85.xxx to any keep-state
add forward 24.85.92.1 ip from 24.85.9x.xxx to any keep-state
add forward 24.85.92.1 ip from 24.85.9x.xxx to any keep-state
add allow ip from me to any keep-state
add deny ip from me to any
add allow icmp from any to me icmptypes 3,4,8,11 keep-state
add deny ip from any to me
add allow ip from 192.168.1.1/24 to any keep-state
add deny ip from 192.168.1.1/24 to any
add allow icmp from any to 192.168.1.1/24 icmptypes 3,4,11 keep-state
add deny ip from any to 192.168.1.1/24
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Advanced IPFW2 Forward rule problem / bug / misunderstanding
Dennis Olvany wrote:
This should get you most of the way there or at least give you a good
idea of what's required.
options IPFIREWALL_FORWARD_EXTENDED
I'm pretty sure this will be required.
defaultrouter=24.85.92.1
ifconfig_rl0=192.168.1.1
ifconfig_vr0=142.179.109.xxx netmask 255.255.248.0
ifconfig_vr1=216.232.85.xxx netmask 255.255.254.0
ifconfig_rue0=24.85.9x.xxx netmask 255.255.252.0
ifconfig_rue0_alias0=24.85.9x.xxx netmask 255.255.255.255
Telus
ext1_ip=142.179.109.xxx # IP Address
ext1_gw=142.179.104.254 # IP Gateway
ext2_ip=216.232.85.xxx # IP Address
ext2_gw=216.232.84.254 # IP Gateway
Shaw Cable
shaw_ip=24.85.93.xxx # IP Address
shaw_gw=24.85.92.1 # IP Gateway
srv2_ext=24.85.93.xxx # External IP of server
INTERNAL
int_ip=192.168.1.1 # IP Address
# And run our new NATd
/sbin/natd -log_ipfw_denied -i ${nat_in} -o ${nat_out} -s -m -u -n
${shaw} -punch_fw 36000:100 -redirect_port tcp ${ext1_srv}:22
${ext1_ip}: -redirect_port tcp ${ext1_srv}:53 ${ext1_ip}:53
-redirect_port tcp ${ext1_srv}:80 ${ext1_ip}:80 -redirect_port tcp
${ext1_srv}:443 ${ext1_ip}:443 -redirect_port udp ${ext1_srv}:53
${ext1_ip}:53 -redirect_port tcp ${ext2_srv}:80 ${ext2_ip}:80
-redirect_port tcp ${ext2_srv}:443 ${ext2_ip}:443 -redirect_port tcp
${srv1_int}:22 ${shaw_ip}: -redirect_port tcp ${srv1_int}:53
${shaw_ip}:53 -redirect_port udp ${srv1_int}:53 ${shaw_ip}:53
-redirect_port tcp ${srv1_int}:80 ${shaw_ip}:80 -redirect_port tcp
${srv1_int}:443 ${shaw_ip}:443 -redirect_port tcp ${srv2_int}:80
${srv2_ext}:80 -redirect_port tcp ${srv2_int}:443 ${srv2_ext}:443
That's a hefty nat command. Let's simplfy by putting it in a file. I
leave the port forwarding to you.
/etc/rc.conf
natd_enable=yes
natd_flags=-f /etc/natd.conf
/etc/natd.conf
instance default
interface vr0
port 8668
instance telus2
interface vr1
port 8669
instance shaw1
alias_address 24.85.93.xxx
port 8670
instance shaw2
alias_address 24.85.93.xxx
port 8671
globalport 8672
I see that your firewall is based on rc.firewall. Forget rc.firewall, it
is junk. Base your firewall on this structure.
1. Public Interface NAT Diversion
2. check-state
3. Public Interface Leak Prevention
3.1 deny egress from internal hosts
3.2 deny ingress to internal hosts
4. Antispoof
4.1 allow via loopback interface
4.2 deny ingress from router
4.3 deny ingress from internal hosts via public interface
5. Router
5.1 allow egress
5.2 deny egress
5.3 allow ingress
5.4 deny ingress
6. Internal Hosts
6.1 allow egress
6.2 deny egress
6.3 allow ingress
6.4 deny ingress
7. Default Deny
/etc/rc.conf
firewall_enable=yes
firewall_type=/etc/ipfw.rules
/etc/ipfw.rules
-f flush
add divert 8668 ip from any to any in via vr0
add divert 8669 ip from any to any in via vr1
add divert 8670 ip from any to 24.85.93.xxx in via rue0
add divert 8671 ip from any to 24.85.93.xxx in via rue0
#have never known the globalport to work on inbound
add divert 8672 ip from any to any out via { vr0 or vr1 or rue0 }
#not sure if that 'or' will work... may need to split it up
check-state
add deny ip from 192.168.1.1/24 to any via { vr0 or vr1 or rue0 }
add allow ip from me to me via lo0 keep-state
add deny ip from me to any in
add allow ip from me to { me or 192.168.1.1/24 or 142.179.109.xxx/21 or
216.232.85.xxx/23 or 24.85.9x.xxx/22 } keep-state
add forward 142.179.104.254 ip from 142.179.109.xxx to any keep-state
add forward 216.232.84.254 ip from 216.232.85.xxx to any keep-state
add forward 24.85.92.1 ip from 24.85.9x.xxx to any keep-state
add forward 24.85.92.1 ip from 24.85.9x.xxx to any keep-state
add allow ip from me to any keep-state
add deny ip from me to any
add allow icmp from any to me icmptypes 3,4,8,11 keep-state
add deny ip from any to me
add allow ip from 192.168.1.1/24 to any keep-state
add deny ip from 192.168.1.1/24 to any
add allow icmp from any to 192.168.1.1/24 icmptypes 3,4,11 keep-state
add deny ip from any to 192.168.1.1/24
Thank for the quick reply.
I just want to clarify a few things here Given that I only want
outbound NAT on one interface, is it really necessary to run four
instances of NATd? Can't one instance handle outbound NAT + inbound
sessions on all interfaces, as I have it setup? Also, you're using a
whole bunch of options and features here that are not documented on the
natd man page. I found a writeup by the author of these features, but
I'm not certain if that's in the -STABLE branch or not. (IE, will these
options work with a 6.0-RELEASE natd?)
Also, I'm not certain how your forward rules would work when mine do
not, as you're doing the same thing I did - NAT Translation, then
forward to the appropriate gateway. My experience is that forwarding
packets to the appropriate gateway *does* *not* *work*, as they all
leave via the default route's interface
Re: Advanced IPFW2 Forward rule problem / bug / misunderstanding
Andrew Fremantle wrote:
Dennis Olvany wrote:
This should get you most of the way there or at least give you a
good idea of what's required.
options IPFIREWALL_FORWARD_EXTENDED
I'm pretty sure this will be required.
defaultrouter=24.85.92.1 ifconfig_rl0=192.168.1.1
ifconfig_vr0=142.179.109.xxx netmask 255.255.248.0
ifconfig_vr1=216.232.85.xxx netmask 255.255.254.0
ifconfig_rue0=24.85.9x.xxx netmask 255.255.252.0
ifconfig_rue0_alias0=24.85.9x.xxx netmask 255.255.255.255
Telus ext1_ip=142.179.109.xxx # IP Address
ext1_gw=142.179.104.254 # IP Gateway ext2_ip=216.232.85.xxx
# IP Address ext2_gw=216.232.84.254 # IP Gateway
Shaw Cable shaw_ip=24.85.93.xxx # IP Address
shaw_gw=24.85.92.1 # IP Gateway srv2_ext=24.85.93.xxx #
External IP of server
INTERNAL int_ip=192.168.1.1 # IP Address
# And run our new NATd /sbin/natd -log_ipfw_denied -i ${nat_in}
-o ${nat_out} -s -m -u -n ${shaw} -punch_fw 36000:100
-redirect_port tcp ${ext1_srv}:22 ${ext1_ip}: -redirect_port
tcp ${ext1_srv}:53 ${ext1_ip}:53 -redirect_port tcp
${ext1_srv}:80 ${ext1_ip}:80 -redirect_port tcp ${ext1_srv}:443
${ext1_ip}:443 -redirect_port udp ${ext1_srv}:53 ${ext1_ip}:53
-redirect_port tcp ${ext2_srv}:80 ${ext2_ip}:80 -redirect_port
tcp ${ext2_srv}:443 ${ext2_ip}:443 -redirect_port tcp
${srv1_int}:22 ${shaw_ip}: -redirect_port tcp ${srv1_int}:53
${shaw_ip}:53 -redirect_port udp ${srv1_int}:53 ${shaw_ip}:53
-redirect_port tcp ${srv1_int}:80 ${shaw_ip}:80 -redirect_port
tcp ${srv1_int}:443 ${shaw_ip}:443 -redirect_port tcp
${srv2_int}:80 ${srv2_ext}:80 -redirect_port tcp ${srv2_int}:443
${srv2_ext}:443
That's a hefty nat command. Let's simplfy by putting it in a file.
I leave the port forwarding to you.
/etc/rc.conf
natd_enable=yes natd_flags=-f /etc/natd.conf
/etc/natd.conf
instance default interface vr0 port 8668
instance telus2 interface vr1 port 8669
instance shaw1 alias_address 24.85.93.xxx port 8670
instance shaw2 alias_address 24.85.93.xxx port 8671
globalport 8672
I see that your firewall is based on rc.firewall. Forget
rc.firewall, it is junk. Base your firewall on this structure.
1. Public Interface NAT Diversion
2. check-state
3. Public Interface Leak Prevention 3.1 deny egress from internal
hosts 3.2 deny ingress to internal hosts
4. Antispoof 4.1 allow via loopback interface 4.2 deny ingress from
router 4.3 deny ingress from internal hosts via public interface
5. Router 5.1 allow egress 5.2 deny egress 5.3 allow ingress 5.4
deny ingress
6. Internal Hosts 6.1 allow egress 6.2 deny egress 6.3 allow
ingress 6.4 deny ingress
7. Default Deny
/etc/rc.conf
firewall_enable=yes firewall_type=/etc/ipfw.rules
/etc/ipfw.rules
-f flush
add divert 8668 ip from any to any in via vr0 add divert 8669 ip
from any to any in via vr1 add divert 8670 ip from any to
24.85.93.xxx in via rue0 add divert 8671 ip from any to
24.85.93.xxx in via rue0 #have never known the globalport to work
on inbound add divert 8672 ip from any to any out via { vr0 or vr1
or rue0 } #not sure if that 'or' will work... may need to split it
up
check-state
add deny ip from 192.168.1.1/24 to any via { vr0 or vr1 or rue0 }
add allow ip from me to me via lo0 keep-state add deny ip from me
to any in
add allow ip from me to { me or 192.168.1.1/24 or
142.179.109.xxx/21 or 216.232.85.xxx/23 or 24.85.9x.xxx/22 }
keep-state add forward 142.179.104.254 ip from 142.179.109.xxx to
any keep-state add forward 216.232.84.254 ip from 216.232.85.xxx to
any keep-state add forward 24.85.92.1 ip from 24.85.9x.xxx to any
keep-state add forward 24.85.92.1 ip from 24.85.9x.xxx to any
keep-state add allow ip from me to any keep-state add deny ip from
me to any add allow icmp from any to me icmptypes 3,4,8,11
keep-state add deny ip from any to me
add allow ip from 192.168.1.1/24 to any keep-state add deny ip from
192.168.1.1/24 to any add allow icmp from any to 192.168.1.1/24
icmptypes 3,4,11 keep-state add deny ip from any to 192.168.1.1/24
Thank for the quick reply.
I just want to clarify a few things here Given that I only want
outbound NAT on one interface, is it really necessary to run four
instances of NATd? Can't one instance handle outbound NAT + inbound
sessions on all interfaces, as I have it setup?
I'm sure you'll need one instance for each public IP from which you wish
to originate traffic.
Also, you're using a whole bunch of options and features here that
are not documented on the natd man page. I found a writeup by the
author of these features, but I'm not certain if that's in the
-STABLE branch or not. (IE, will these options work with a
6.0-RELEASE natd?)
Yes. I use these features with 6.0-release.
Also, I'm not certain how your forward rules would work when mine do
not, as you're doing the same thing I did - NAT
