Hi, I've got an interesting problem that I'd really appreciate some input on. I am in the process of migrating our Linux router-on-a-stick to an OpenBSD router, and have configured an OpenBSD 4.7 (GENERIC) box with an IP on each VLAN. At present, no devices are configured to use the OpenBSD box as gateway, so all I should be seeing is broadcast traffic on each vlan interface.
For some reason however, on one particular VLAN the switch is erroneously forwarding traffic from a particular host (203.135.184.10) to the OpenBSD box. The traffic is forwarded even when the destination MAC address is not that of the OpenBSD box. So there's something broken on my switch, I need to fix it, fair enough. The strange thing is that occasionally, the OpenBSD box will reply to the gratuitous traffic with a spoofed TCP RST. For example, see [1] - a TCP connection was initiated from 203.135.184.10 (an OSX server) to 203.135.184.6 (a Linux server), which is on the same subnet. The connection is immediately closed, apparently by 203.135.184.6 - but if you look at the MAC addresses it's a spoofed packet from the OpenBSD box, which is normally 203.135.184.33. The *even weirder* thing is that if I run tcpdump on the appropriate interface on the OpenBSD router, in an attempt to diagnose the problem, the issue goes away. I can happily initiate connections all day long and the OpenBSD box will behave. Running tcpdump on any other interface does not affect the problem. Traffic from 203.135.184.10 to other hosts not sent via the OpenBSD box is also affected; for example my SSH sessions to 203.135.184.10 will drop out occasionally when I'm producing lots of traffic. Traffic from 203.135.184.10 to other hosts sent via the OpenBSD box is *not* affected; for example SSHing to 203.135.184.10 from another subnet using the OpenBSD box as the router works fine. The problem is also not consistently reproducible (though it does happen more than not) - see [2]. In my pf.conf I have "match in all scrub (reassemble tcp)" and "antispoof log for $interfaces" and nothing else that isn't a simple pass/block or NAT rule. I'm not ruling out some sort of config error here, because I'm pretty new to OpenBSD and pf, though my understanding is that the above won't cause RSTs to be sent for layer-two traffic not sent to the OpenBSD box in question. Does anyone have any ideas on what could be happening here? While I can fix the switch, I'd like to understand why this is happening. Let me know if there's any output I can provide that would help. Cheers, Patrick -- http://www.labyrinthdata.net.au - WA Backup, Web and VPS Hosting [1] tcpdump running on 203.135.184.10, while running 'nc 203.135.184.6 25' in another vty. Note the appearance of the RST packets from the OpenBSD box at 11:59:33.538154. Relevant MAC addresses are: 203.135.184.10: 00:24:36:f2:9d:9c 203.135.184.6: 00:50:56:96:11:f3 203.135.184.33 (OpenBSD router): 00:04:23:c9:bd:d0 spartacus$ sudo tcpdump -i vlan21 -ne host 203.135.184.6 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on vlan21, link-type EN10MB (Ethernet), capture size 65535 bytes 11:59:33.537765 00:24:36:f2:9d:9c > 00:50:56:96:11:f3, ethertype IPv4 (0x0800), length 78: 203.135.184.10.55790 > 203.135.184.6.25: Flags [S], seq 1966388277, win 65535, options [mss 1460,nop,wscale 1,nop,nop,TS val 532321133 ecr 0,sackOK,eol], length 0 11:59:33.538028 00:50:56:96:11:f3 > 00:24:36:f2:9d:9c, ethertype IPv4 (0x0800), length 74: 203.135.184.6.25 > 203.135.184.10.55790: Flags [S.], seq 3218134397, ack 1966388278, win 5792, options [mss 1460,sackOK,TS val 215574207 ecr 532321133,nop,wscale 7], length 0 11:59:33.538044 00:24:36:f2:9d:9c > 00:50:56:96:11:f3, ethertype IPv4 (0x0800), length 66: 203.135.184.10.55790 > 203.135.184.6.25: Flags [.], ack 1, win 33304, options [nop,nop,TS val 532321133 ecr 215574207], length 0 11:59:33.538154 00:04:23:c9:bd:d0 > 00:24:36:f2:9d:9c, ethertype IPv4 (0x0800), length 56: 203.135.184.6.25 > 203.135.184.10.55790: Flags [R.], seq 1076832899, ack 1, win 0, length 0 11:59:33.538160 00:04:23:c9:bd:d0 > 00:24:36:f2:9d:9c, ethertype IPv4 (0x0800), length 56: 203.135.184.6.25 > 203.135.184.10.55790: Flags [R.], seq 1076832899, ack 1, win 0, length 0 11:59:33.538332 00:04:23:c9:bd:d0 > 00:24:36:f2:9d:9c, ethertype IPv4 (0x0800), length 56: 203.135.184.6.25 > 203.135.184.10.55790: Flags [R.], seq 1, ack 1, win 0, length 0 11:59:33.538339 00:04:23:c9:bd:d0 > 00:24:36:f2:9d:9c, ethertype IPv4 (0x0800), length 56: 203.135.184.6.25 > 203.135.184.10.55790: Flags [R.], seq 1, ack 1, win 0, length 0 11:59:33.694294 00:50:56:96:11:f3 > 00:24:36:f2:9d:9c, ethertype IPv4 (0x0800), length 102: 203.135.184.6.25 > 203.135.184.10.55790: Flags [P.], seq 1:37, ack 1, win 46, options [nop,nop,TS val 215574246 ecr 532321133], length 36 11:59:33.694308 00:24:36:f2:9d:9c > 00:50:56:96:11:f3, ethertype IPv4 (0x0800), length 54: 203.135.184.10.55790 > 203.135.184.6.25: Flags [R], seq 1966388278, win 0, length 0 ^C 9 packets captured 116 packets received by filter 0 packets dropped by kernel spartacus$ [2] The problem is not consistent and occurs at different stages of the TCP connection: spartacus$ telnet 203.135.184.6 25 Trying 203.135.184.6... telnet: connect to address 203.135.184.6: Connection refused telnet: Unable to connect to remote host spartacus$ telnet 203.135.184.6 25 Trying 203.135.184.6... Connected to io.ccgs.wa.edu.au. Escape character is '^]'. Connection closed by foreign host. spartacus$ telnet 203.135.184.6 25 Trying 203.135.184.6... telnet: connect to address 203.135.184.6: Connection refused telnet: Unable to connect to remote host spartacus$ telnet 203.135.184.6 25 Trying 203.135.184.6... Connected to io.ccgs.wa.edu.au. Escape character is '^]'. Connection closed by foreign host. [four identical attempts trimmed] spartacus$ telnet 203.135.184.6 25 Trying 203.135.184.6... Connected to io.ccgs.wa.edu.au. Escape character is '^]'. 220 mail.ccgs.wa.edu.au ESMTP CCGS ^[[Aquit 502 5.5.2 Error: command not recognized quit 221 2.0.0 Bye Connection closed by foreign host. spartacus$ telnet 203.135.184.6 25 Trying 203.135.184.6... Connected to io.ccgs.wa.edu.au. Escape character is '^]'. 220 mail.ccgs.wa.edu.au ESMTP CCGS quit 221 2.0.0 Bye Connection closed by foreign host.
