I'm trying to diagnose the problem in our new firewall setup. I've
drawn a digram below. We have two IP ranges, one serviced by an
IPCop Linux distro, another by a CARPed OpenBSD pf pair (currently
OpenBSD 3.8). Currently our old windows web server is assigned
addresses from the first range, and the two clustered (CARPed
FreeBSD) are behind the OpenBSD pair. (The issue occurs with the
windows server too though.)
The aim is to relegate the IPCop server to a spam filter in front of
the internal network. Currently all internal traffic goes through
IPCop, even that destined for hosts filtered by the OpenBSD boxes.
Basicaly I have made pf rules that seem to allow traffic through, and
after reading through them hundreds of times, even my inexperienced
eyes are beginning to think they must be correct (they're in my
previous email though.)
Here are my observations of the problem:
- access through the OpenBSD firewalls is REALLY slow, giving a
noticeable delay on web sites
- despite this MOST traffic goes through
- however, a small number of connections are blocked by the main
"block all log" rule, seen in a tcpdump of pflog0:
- the connections from the internet are blocked IN on the dmz
interface (em0)
- and the wierd bit! traffic from out internal network is blocked
going OUT on the external interface (vr0)
- I have even seen packets dropped on pflog where there was
apparently a state for that connection - I might have to sanity
check that but I'm fairly sure it's not me going mad
Here are my thoughts about the likely cause of the problem:
- I don't think it's the firewall rules, as they work 90% of the time
- I don't think it's any of the physical networking as the other
machines run fine
- Could it be hardware incompatibility?
- I saw this in the em man page:
There is a known compatibility issue where time to link is slow
or link
is not established between 82541/82547 controllers and some
switches.
Known switches include:
I-O Data ETG-SH8
Planex FXG-08TE
- Also it is brand new hardware, Intel board and onboard ethernet
Unforunately I'm at home now and can't find the exact hardware
description of the machines from here. I don't know whether the
bizarre pf logs showing different failures from our internal requests
to external (which are ALL external as far as the firewalls are
concerned) are evidence for or against it being hardware (or driver)
related.
Tomorrow I plan to rebuild the firewalls with OpenBSD 3.9 in the hope
it is a recently-fixed bug. Failing that I will be forced to find an
old desktop and try installing one on that.
I'm hoping someone will recognise the symptoms as that might point me
in the right direction and save me time (although I ran out of that
days ago!!!)
Thanks
Ashley
internet
|
|
------------------
| ISP Cisco Router |
------------------
|
|
---->--- eth switch ---->----
^ |
| x.x.1.x v x.x.2.x
------- -<-- eth switch ----
--->--| IPCop | | |
| ------- v vr0 | vr
| | ------------ ------------
| | | OpenBSD/pf | | OpenBSD/pf |
^ | ------------ ------------
| | em0| |em1________em0|__switch_|em1
| | v | |
| | -->- eth switch -- |
| | | |
| | ________<__________v |
| | DMZ | |
| --------------------------- |
^ | webserv1 (win) [ipcop] | .
| | webserv2a (fbsd) [obsd] | .
| | webserv2b (fbsd) [obsd] | .
| --------------------------- .
| .
----<-------------------<-------------------<-----internal network
arrows show route from internal network to new webservers
(On the plus side, drawing the above piece of ASCII art was very
theraputic.)
