[Bug 1327442] [NEW] docker multicast packets origination addresses are mangled
Public bug reported: Although I've filed this as a docker bug, it is far more likely to be a kernel (bridge) bug. I have enabled nothing extra in terms of docker networking. Both of these two ubuntu72:~/monitor/src/cma $ uname -a Linux ubuntu72 3.13.0-27-generic #50-Ubuntu SMP Thu May 15 18:06:16 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux ubuntu72:~/monitor/src/cma $ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description:Ubuntu 14.04 LTS Release:14.04 Codename: trusty ubuntu72:~/monitor/src/cma $ brctl show docker0 bridge name bridge id STP enabled interfaces docker0 8000.d6b7b71c12ab no veth3d9d vetha7dd vethade0 I have a couple of docker instances. One is a client with IP address 172.17.0.4, and one a server with address 172.17.0.3. The client sends out a multicast UDP packet to a certain address (224.0.2.5) reserved for the use of this software. The server dutifully listens for the packet, and it receives it. HOWEVER, the source address of the packet is 172.17.42.1 - which is the address of the docker interface on the host (and of course, not the proper source address). The tcpdump trace of this packet being sent is below: 20:28:39.474887 IP 172.17.42.1.bb 224.0.2.5.bb: UDP, length 713 Needless to say, this is extraordinarily confusing to my software - and when it attempts to reply, nothing good happens... You can see that it gets 'port unreachable' for that attempt to reply (as it should). The subsequent packet sent to the (correct) 0.4 address happens because that address is in the _content_ of initial multicast packet, and by then the software is operating on the contents of the packet, not the (incorrect) origination address.. Below is the full tcpdump trace of what's going on here from the server perspective: # tcpdump -i eth0 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes 20:28:35.933507 IP 7db9eb0e89ee igmp.mcast.net: igmp v3 report, 1 group record(s) 20:28:35.934144 IP 7db9eb0e89ee.42650 10.10.10.20.domain: 18534+ PTR? 22.0.0.224.in-addr.arpa. (41) 20:28:35.960407 IP 10.10.10.20.domain 7db9eb0e89ee.42650: 18534 1/6/0 PTR igmp.mcast.net. (181) 20:28:35.960737 IP 7db9eb0e89ee.47869 10.10.10.20.domain: 19922+ PTR? 20.10.10.10.in-addr.arpa. (42) 20:28:35.986002 IP 10.10.10.20.domain 7db9eb0e89ee.47869: 19922 0/1/0 (101) 20:28:36.113623 IP 7db9eb0e89ee igmp.mcast.net: igmp v3 report, 1 group record(s) 20:28:39.474887 IP 172.17.42.1.bb 224.0.2.5.bb: UDP, length 713 20:28:39.475003 IP 7db9eb0e89ee.33748 10.10.10.20.domain: 3617+ PTR? 5.2.0.224.in-addr.arpa. (40) 20:28:39.525434 IP 7db9eb0e89ee.bb 172.17.42.1.bb: UDP, length 1091 20:28:39.525488 IP 172.17.42.1 7db9eb0e89ee: ICMP 172.17.42.1 udp port bb unreachable, length 556 20:28:39.529629 IP 7db9eb0e89ee.bb 172.17.0.4.bb: UDP, length 924 20:28:39.545780 IP 172.17.0.4.bb 7db9eb0e89ee.bb: UDP, length 70 20:28:39.568680 IP 10.10.10.20.domain 7db9eb0e89ee.33748: 3617 NXDomain 0/1/0 (97) 20:28:39.568878 IP 7db9eb0e89ee.44605 10.10.10.20.domain: 47116+ PTR? 1.42.17.172.in-addr.arpa. (42) 20:28:39.593858 IP 10.10.10.20.domain 7db9eb0e89ee.44605: 47116 0/1/0 (101) 20:28:39.594120 IP 7db9eb0e89ee.36687 10.10.10.20.domain: 20755+ PTR? 4.0.17.172.in-addr.arpa. (41) 20:28:39.620179 IP 10.10.10.20.domain 7db9eb0e89ee.36687: 20755 0/1/0 (100) 20:28:40.941627 ARP, Request who-has 172.17.42.1 tell 7db9eb0e89ee, length 28 20:28:40.941687 ARP, Reply 172.17.42.1 is-at d6:b7:b7:1c:12:ab (oui Unknown), length 28 20:28:41.638772 IP 7db9eb0e89ee.bb 172.17.42.1.bb: UDP, length 1091 20:28:41.638811 IP 172.17.42.1 7db9eb0e89ee: ICMP 172.17.42.1 udp port bb unreachable, length 556 20:28:43.639166 IP 7db9eb0e89ee.bb 172.17.42.1.bb: UDP, length 1091 20:28:43.639193 IP 172.17.42.1 7db9eb0e89ee: ICMP 172.17.42.1 udp port bb unreachable, length 556 On the client side, the packet was sent correctly (according to tcpdump): 20:48:13.277353 IP 172.17.0.4.bb 224.0.2.5.bb: UDP, length 713 So, somebody somewhere is screwing over my source IP addresses... This could _conceivably_ be considered a security bug I suppose because information being misrouted might be security-sensitive, and would be visible to the wrong party. Below is some misc networking configuration for the host and the two containers: ubuntu72:~/monitor/src/cma $ uname -n; ifconfig; route -n ubuntu72 docker0 Link encap:Ethernet HWaddr d6:b7:b7:1c:12:ab inet addr:172.17.42.1 Bcast:0.0.0.0 Mask:255.255.0.0 inet6 addr: fe80::e8cd:69ff:feb7:d419/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:786010 errors:0 dropped:0 overruns:0 frame:0 TX packets:1180631 errors:0 dropped:0
[Bug 1327442] [NEW] docker multicast packets origination addresses are mangled
Public bug reported: Although I've filed this as a docker bug, it is far more likely to be a kernel (bridge) bug. I have enabled nothing extra in terms of docker networking. Both of these two ubuntu72:~/monitor/src/cma $ uname -a Linux ubuntu72 3.13.0-27-generic #50-Ubuntu SMP Thu May 15 18:06:16 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux ubuntu72:~/monitor/src/cma $ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description:Ubuntu 14.04 LTS Release:14.04 Codename: trusty ubuntu72:~/monitor/src/cma $ brctl show docker0 bridge name bridge id STP enabled interfaces docker0 8000.d6b7b71c12ab no veth3d9d vetha7dd vethade0 I have a couple of docker instances. One is a client with IP address 172.17.0.4, and one a server with address 172.17.0.3. The client sends out a multicast UDP packet to a certain address (224.0.2.5) reserved for the use of this software. The server dutifully listens for the packet, and it receives it. HOWEVER, the source address of the packet is 172.17.42.1 - which is the address of the docker interface on the host (and of course, not the proper source address). The tcpdump trace of this packet being sent is below: 20:28:39.474887 IP 172.17.42.1.bb 224.0.2.5.bb: UDP, length 713 Needless to say, this is extraordinarily confusing to my software - and when it attempts to reply, nothing good happens... You can see that it gets 'port unreachable' for that attempt to reply (as it should). The subsequent packet sent to the (correct) 0.4 address happens because that address is in the _content_ of initial multicast packet, and by then the software is operating on the contents of the packet, not the (incorrect) origination address.. Below is the full tcpdump trace of what's going on here from the server perspective: # tcpdump -i eth0 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes 20:28:35.933507 IP 7db9eb0e89ee igmp.mcast.net: igmp v3 report, 1 group record(s) 20:28:35.934144 IP 7db9eb0e89ee.42650 10.10.10.20.domain: 18534+ PTR? 22.0.0.224.in-addr.arpa. (41) 20:28:35.960407 IP 10.10.10.20.domain 7db9eb0e89ee.42650: 18534 1/6/0 PTR igmp.mcast.net. (181) 20:28:35.960737 IP 7db9eb0e89ee.47869 10.10.10.20.domain: 19922+ PTR? 20.10.10.10.in-addr.arpa. (42) 20:28:35.986002 IP 10.10.10.20.domain 7db9eb0e89ee.47869: 19922 0/1/0 (101) 20:28:36.113623 IP 7db9eb0e89ee igmp.mcast.net: igmp v3 report, 1 group record(s) 20:28:39.474887 IP 172.17.42.1.bb 224.0.2.5.bb: UDP, length 713 20:28:39.475003 IP 7db9eb0e89ee.33748 10.10.10.20.domain: 3617+ PTR? 5.2.0.224.in-addr.arpa. (40) 20:28:39.525434 IP 7db9eb0e89ee.bb 172.17.42.1.bb: UDP, length 1091 20:28:39.525488 IP 172.17.42.1 7db9eb0e89ee: ICMP 172.17.42.1 udp port bb unreachable, length 556 20:28:39.529629 IP 7db9eb0e89ee.bb 172.17.0.4.bb: UDP, length 924 20:28:39.545780 IP 172.17.0.4.bb 7db9eb0e89ee.bb: UDP, length 70 20:28:39.568680 IP 10.10.10.20.domain 7db9eb0e89ee.33748: 3617 NXDomain 0/1/0 (97) 20:28:39.568878 IP 7db9eb0e89ee.44605 10.10.10.20.domain: 47116+ PTR? 1.42.17.172.in-addr.arpa. (42) 20:28:39.593858 IP 10.10.10.20.domain 7db9eb0e89ee.44605: 47116 0/1/0 (101) 20:28:39.594120 IP 7db9eb0e89ee.36687 10.10.10.20.domain: 20755+ PTR? 4.0.17.172.in-addr.arpa. (41) 20:28:39.620179 IP 10.10.10.20.domain 7db9eb0e89ee.36687: 20755 0/1/0 (100) 20:28:40.941627 ARP, Request who-has 172.17.42.1 tell 7db9eb0e89ee, length 28 20:28:40.941687 ARP, Reply 172.17.42.1 is-at d6:b7:b7:1c:12:ab (oui Unknown), length 28 20:28:41.638772 IP 7db9eb0e89ee.bb 172.17.42.1.bb: UDP, length 1091 20:28:41.638811 IP 172.17.42.1 7db9eb0e89ee: ICMP 172.17.42.1 udp port bb unreachable, length 556 20:28:43.639166 IP 7db9eb0e89ee.bb 172.17.42.1.bb: UDP, length 1091 20:28:43.639193 IP 172.17.42.1 7db9eb0e89ee: ICMP 172.17.42.1 udp port bb unreachable, length 556 On the client side, the packet was sent correctly (according to tcpdump): 20:48:13.277353 IP 172.17.0.4.bb 224.0.2.5.bb: UDP, length 713 So, somebody somewhere is screwing over my source IP addresses... This could _conceivably_ be considered a security bug I suppose because information being misrouted might be security-sensitive, and would be visible to the wrong party. Below is some misc networking configuration for the host and the two containers: ubuntu72:~/monitor/src/cma $ uname -n; ifconfig; route -n ubuntu72 docker0 Link encap:Ethernet HWaddr d6:b7:b7:1c:12:ab inet addr:172.17.42.1 Bcast:0.0.0.0 Mask:255.255.0.0 inet6 addr: fe80::e8cd:69ff:feb7:d419/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:786010 errors:0 dropped:0 overruns:0 frame:0 TX packets:1180631 errors:0 dropped:0