Please "reply to all" as I'm not currently subscribed to the list.. thanks!
I have a problem with pf (whatever is bundled with FreeBSD 6.1,
updating from RELENG_6_1 to RELENG_6 now) that I've googled for and
experimented with all night, but cannot find a solid answer to.
I have a series of internal apache servers on a private network,
192.168.0.0/16. My FreeBSD 6 server exists on the internal network as
192.168.0.1 (fxp0), and the external network.
The external network is a "poor mans" multi-homed solution, with
several consumer-grade lines from different ISPs that will not pass
"spoofed" traffic, so I have the perennial problem of needing the
traffic to exit the same interface it came in on. Assume the public
networks are 10.0.0.0/24 and 10.0.1.0/24; 10.0.0.1 and 10.0.1.1 belong
to their respective ISP devices, and the remaining addresses are
assigned to the FreeBSD box (fxp1). The networks aren't actually that
large of course, but it makes explanation easier.
The twist in my case is that the internal webservers need the packets
to come from a "real" source IP, so NAT or a load balancer like haproxy
is out of the question here. I don't write the software, I just try to
make it work.
My first attempt was a simple set of two RDR rules, and two pass rules.
The RDR rules are exactly what you'd expect.
rdr on fxp1 proto tcp from any to $ext_net_a port 80 -> $int_net_a port 80
rdr on fxp1 proto tcp from any to $ext_net_b port 80 -> $int_net_a port 80
The pass rules I thought would be equally simple.
pass in quick on fxp0 route-to (fxp1 $ext_gw_a) from $ext_net_a to any
pass in quick on fxp0 route-to (fxp1 $ext_gw_b) from $ext_net_b to any
pass all
Unfortunately this simply does not work. Only traffic from the outside
to external IPs on whichever line the FreeBSD default route is set to
work, leading me to believe the route-to lines are never tripped.
pfctl -ss shows this to be the case with packet and bytecounts of 0 for
those rules.
I guess my question is, is there a better way to do this? RDR appears
to not be 'stateful' and so the packets aren't being 'un-RDRed' on
their way in prior to testing against the pass rule. In fact, with the
rule counts being 0, I'm not sure how it's even semi-working as it is.
Probably some trickery by the ISP devices.
Any help would be much appreciated. As it is, I've resorted to
assigning different "mental" subnets to the internal webservers, such
as 192.168.1.* for stuff from ISP-A, 192.168.2.* for ISP-B, etc, and
adjusting the rules to reflect this. It's an ugly solution, it
'works', but it's causing other problems.
Thanks for any hints!
