Hi... can anyone work out what is wrong with my PF rules? We have a DMZ and internal corporate network. Externally, we have to IP ranges with 28 bit netmasks. Currently, we have an IPCop server handling the old range in the DMZ (say a.b.c.d, which is rdr'd to 10.0.x.x inside the DMZ) and the internal network, and two CARPed/pfsynced OpenBSD 3.8 servers handling the new range (say e.f.g.h).
Yesterday morning the Linux box randomly corrupted and we tried to move the new IP range onto the OpenBSD firewalls and switch the windows webserver over. We set the web server to use the cluster as the default gateway, and moved the aliases from IPCop to the carp0 interface on the primary OpenBSD firewall. We swapped the IPs on the internal interface so that our internal network was using the cluster to reach the internet. However we get two big problems: - network connections from internal to the DMZ or the internet were really slow (despite minimal CPU load on the firewalls) - we get loads of tcpdump block logs of the form: Jul 17 12:21:51.070264 rule 0/(match) block in on em0: 10.0.0.13.80 > 62.6.139.10.15309: [|tcp] Jul 17 12:21:51.144867 rule 0/(match) block in on em0: 10.0.0.13.80 > 84.71.160.155.1603: [|tcp] Jul 17 12:21:52.063020 rule 0/(match) block in on em0: 10.0.0.13.80 > 159.168.7.200.21039: [|tcp] Jul 17 12:21:52.611955 rule 0/(match) block in on em0: 10.0.0.13.80 > 86.134.106.43.2013: [|tcp] I've just seen this: Jul 17 17:36:17.307019 rule 0/(match) block in on em0: 10.0.100.1.22 > 211.137.76.105.27953: [|tcp] (DF) Presumably it's just a skiddie attack, but mainly it shows it's not just HTTP traffic causing problems. The block logs really puzzle me, because all web traffic should create a state to let the return traffic back out. What's strange is that you can still access the sites through the firewalls despite the random errors. Anyone got any ideas? I'm kinda at the end of my tether now... I'm not the network admin here (although I seem to do most of his work lately) so these rules might not be very well written. They appeared to work ok in testing but have blown up live. Thanks for any advice... Ashley ########################## # INTERFACES # ########################## ext_if = "vr0" dmz_if = "em0" int_if = "em1" pfsync_phys_if = "em1" pfsync_secure_if = "enc0" all_if = "{ vr0, em0, em1 }" # can't antispoof on em1 because enc0 (created by ipsec) shares an IP range # not critical as this is on the internal interface anyway antispoof_if = "{ vr0, em0 }" ########################## # ADDRESSES AND SERVICES # ########################## ### External table <public> persist { a.b.c.d/28, e.f.g.h/28 } ### DMZ dmz_ad="10.0.0.0/16" dmz_tcp_services_out = "{ http, https, ftp, ntp, domain, 5999 }" # 5999 is cvsup (FreeBSD) dmz_udp_services_out = "{ ntp, domain }" ## webserv1 webserv1_ext_ad = "x.x.x.x" webserv1_dmz_ad = "10.0.0.12" gr8_ext_ad = "x.x.x.x" gr8_dmz_ad = "10.0.0.13" codeweavers_secure_ext_ad = "x.x.x.x" codeweavers_secure_dmz_ad = "10.0.0.14" dealersystem_ext_ad = "x.x.x.x" dealersystem_dmz_ad = "10.0.0.15" easidrive_ext_ad = "x.x.x.x" easidrive_dmz_ad = "10.0.0.21" ## webserv2 (cluster) # primary dmz address is "physical address", others are CARPED webserv2_ext_ad = "x.x.x.x" webserv2_dmz_primary_ad = "{ 10.0.1.1, 10.0.1.2 }" webserv2_dmz_ad = "{ 10.0.100.1, 10.0.101.1 }" # applies to all webservers webserver_tcp_services = "{ http, https, ssh }" webserv1_extra_tcp_services = "{ smtp, 3389 }" ## database servers magneto_dmz_ad = "10.0.2.1" mystique_dmz_ad = "10.0.2.2" dbserv_ext_ad = "x.x.x.x" dbserv_tcp_services = "{ 2222, 2223 }" # Internal table <internal> persist { 192.168.136.0/24, 192.168.0.0/24 } intranet_ext_ad = "x.x.x.x" jigsawfirewall_ad = "192.168.136.251" jigsawfirewall_tcp_services_in = "{ smtp }" # include both firewalls here to save maintaining separate # scripts for each server # note: these are the IPs used over the internal interface firewall_ad = "{ 192.168.136.253, 192.168.136.252, 192.168.254.254, 192.168.254.253 }" # Spam table <spammers> persist ############ # DEFAULTS # ############ # dont filter on loopback: set skip on lo0 ############# # SCRUBBING # ############# # clean all packets: # - random-id: helps prevent OS identification and NAT host counting # - reassemble tcp: used with fragment reassemble for NAT # - fragment reassemble: makes sure packet fragments are reassembled before # sending through the network scrub all reassemble tcp scrub in all fragment reassemble scrub out all random-id ################### # NAT/REDIRECTION # ################### ### DMZ nat on $ext_if inet proto { tcp, udp, icmp } \ from $webserv1_dmz_ad -> $webserv1_ext_ad nat on $ext_if inet proto { tcp, udp, icmp } \ from $webserv2_dmz_primary_ad -> $webserv2_ext_ad nat on $ext_if inet proto { tcp, udp, icmp } \ from { $magneto_dmz_ad, $mystique_dmz_ad } -> $dbserv_ext_ad ## webserv1 rdr on { $ext_if, $int_if } inet proto tcp \ from any to $webserv1_ext_ad port { http, https, ssh, smtp, 3389 } \ -> $webserv1_dmz_ad rdr on { $ext_if, $int_if } inet proto tcp \ from any to $codeweavers_secure_ext_ad port { http, https } \ -> $codeweavers_secure_dmz_ad rdr on { $ext_if, $int_if } inet proto tcp \ from any to $dealersystem_ext_ad port { http, https } \ -> $dealersystem_dmz_ad rdr on { $ext_if, $int_if } inet proto tcp \ from any to $gr8_ext_ad port { http, https } \ -> $gr8_dmz_ad rdr on { $ext_if, $int_if } inet proto tcp \ from any to $easidrive_ext_ad port { http, https } \ -> $easidrive_dmz_ad ## webserv2a/b (load balancing) rdr on { $ext_if, $int_if } inet proto tcp \ from any to $webserv2_ext_ad port http -> $webserv2_dmz_ad port http \ round-robin sticky-address rdr on { $ext_if, $int_if } inet proto tcp \ from any to $webserv2_ext_ad port https -> $webserv2_dmz_ad port https \ round-robin sticky-address rdr on { $ext_if, $int_if } inet proto tcp \ from any to $webserv2_ext_ad port ssh -> $webserv2_dmz_ad port ssh \ round-robin sticky-address rdr on { $ext_if, $int_if } inet proto tcp \ from any to $dbserv_ext_ad port 2222 -> $magneto_dmz_ad port ssh rdr on { $ext_if, $int_if } inet proto tcp \ from any to $dbserv_ext_ad port 2223 -> $mystique_dmz_ad port ssh # Internal nat on $ext_if proto { tcp, udp, icmp } \ from <internal> -> $intranet_ext_ad rdr on $ext_if proto { tcp, udp, icmp } \ from any to $intranet_ext_ad -> $jigsawfirewall_ad # Spam rdr pass on $ext_if inet proto tcp \ from <spammers> to port smtp -> 127.0.0.1 port spamd # FTP proxy rdr on $dmz_if proto tcp \ from $dmz_ad to any port ftp -> localhost port 8021 rdr on $int_if proto tcp \ from <internal> to any port ftp -> localhost port 8021 ############# # FILTERING # ############# #default policy: block log antispoof log quick for $antispoof_if # pass out anything we've deemed safe to leave the opposite interface pass out quick inet tagged SANITISED modulate state # --- pfsync (ipsec virtual interface) # just allow anything, nothing unpleasant is going to pass over the interface pass quick on $pfsync_secure_if # --- external # fuzz any 'nmap' attempt # (see http://monkey.org/openbsd/archive/misc/0208/msg00877.html) block in log quick on $ext_if inet proto tcp from any to any flags FUP/FUP block in log quick on $ext_if inet proto tcp from any to any flags SF/SFRA block in log quick on $ext_if inet proto tcp from any to any flags /SFRA # to intranet pass in on $ext_if inet proto tcp \ from any to $jigsawfirewall_ad port $jigsawfirewall_tcp_services_in \ synproxy state tag SANITISED # to web servers pass in on $ext_if inet proto tcp \ from any to $webserv1_dmz_ad port $webserver_tcp_services \ synproxy state tag SANITISED pass in on $ext_if inet proto tcp \ from any to $webserv1_dmz_ad port $webserv1_extra_tcp_services \ synproxy state tag SANITISED # webserv1 extra sites pass in on $ext_if inet proto tcp \ from any to $codeweavers_secure_dmz_ad port { http, https } \ synproxy state tag SANITISED pass in on $ext_if inet proto tcp \ from any to $dealersystem_dmz_ad port { http, https } \ synproxy state tag SANITISED pass in on $ext_if inet proto tcp \ from any to $gr8_dmz_ad port { http, https } \ synproxy state tag SANITISED pass in on $ext_if inet proto tcp \ from any to $easidrive_dmz_ad port { http, https } \ synproxy state tag SANITISED # webserv2 pass in on $ext_if inet proto tcp \ from any to $webserv2_dmz_ad port $webserver_tcp_services \ synproxy state tag SANITISED # allow the firewall iteself to make requests pass out on $ext_if inet proto { tcp, udp, icmp } \ from ($ext_if) to any modulate state ## - NTP #pass out log on $ext_if inet proto { tcp, udp } \ # from ($ext_if) to any port { ntp } keep state ## - DNS #pass out on $ext_if inet proto { tcp, udp } \ # from ($ext_if) to any port { domain } keep state # allow FTP traffic pass in on $ext_if inet proto tcp \ from any to <public> port > 49151 keep state pass out on $ext_if inet proto { tcp } \ from ($ext_if) to any port { ftp } keep state # WORMS: block known worms explicitly so we don't log the packets # Sasser block in quick on $ext_if proto tcp to port { 445, 5554, 9996 } # Blaster block in quick on $ext_if proto tcp to port 135 # God Message block in quick on $ext_if proto tcp to port 139 # Messenger spam bug block in quick on $ext_if proto udp to port 135 block in quick on $ext_if proto udp to port 1025 >< 1032 # --- internal # allow firewall to reach the internal network pass out on $int_if inet proto { tcp, udp, icmp } \ from ($int_if) to <internal> keep state # allow traffic from internal network to anywhere pass in on $int_if inet proto { tcp, udp, icmp } modulate state tag SANITISED # don't need to let this out of the internal network block in on $int_if inet proto igmp # allow SSH traffic from internal to firewall itself # TODO: not needed because we're allowing internal full access? # ...maybe look at restricting internal access to DMZ? pass in on $int_if inet proto tcp \ from <internal> to ($int_if) port ssh keep state # allow SSH, ISAKMP, ICMP traffic between firewalls over internal interface pass out on $int_if inet proto tcp \ from ($int_if) to $firewall_ad \ port ssh keep state pass in on $int_if inet proto tcp \ from $firewall_ad to ($int_if) \ port ssh keep state pass out on $int_if inet proto { tcp, udp } \ from ($int_if) to $firewall_ad \ port isakmp keep state pass in on $int_if inet proto { tcp, udp } \ from $firewall_ad to ($int_if) \ port isakmp keep state pass out on $int_if inet proto icmp \ from ($int_if) to $firewall_ad keep state pass out on $int_if inet proto icmp \ from $firewall_ad to ($int_if) keep state # allow encrypted traffic (ipsec tunnel for pfsync) pass in on $int_if inet proto esp from any to ($int_if) pass out on $int_if inet proto esp from ($int_if) to any # --- dmz # allow firewall to reach the DMZ servers pass out on $dmz_if inet proto { tcp, udp, icmp } \ from ($dmz_if) to $dmz_ad keep state # allow traffic from the DMZ to selected services, but # not to the internal network pass in on $dmz_if inet proto tcp \ from $dmz_ad to any port $dmz_tcp_services_out modulate state tag SANITISED pass in on $dmz_if inet proto udp \ from $dmz_ad to any port $dmz_udp_services_out keep state tag SANITISED pass in on $dmz_if proto icmp \ from $dmz_ad to any keep state tag SANITISED # allow access to FTP proxy # don't ask me why we need the second rule but we do! # it was deduced from the pflog output pass in on $dmz_if inet proto tcp \ from $dmz_ad to localhost port { 8021 } keep state pass in on $dmz_if inet proto tcp \ from $dmz_ad to <public> keep state # TODO: configure NTP server in the DMZ and remove NTP access here pass in on $dmz_if inet proto { tcp, udp } \ to port ntp modulate state tag SANITISED # don't allow connections to internal initiated from the DMZ block in log on $dmz_if from $dmz_ad to <internal> # block netbios traffic from the DMZ block in on $dmz_if proto { tcp, udp } \ from any to any port { netbios-ns, netbios-dgm, netbios-ssn } # --- CARP pass on $all_if inet proto carp keep state -- "If you do it the stupid way, you will have to do it again" - Gregory Chudnovsky