[Yahoo-eng-team] [Bug 1462154] Re: With DVR Pings to floating IPs replied with fixed-ips
I tested it and i was able to reproduce. In my setup VM1 is 10.0.0.3 and VM2 is 10.0.0.5 and with fip 172.168.1.9 - both hosted in the same compute node. In the q-router namespace, there is a DNAT rule (shown below) Chain neutron-l3-agent-PREROUTING (1 references) pkts bytes target prot opt in out source destination 0 0 REDIRECT tcp -- qr-+ * 0.0.0.0/0 169.254.169.254 tcp dpt:80 redir ports 9697 12 1008 DNAT all -- * * 0.0.0.0/0 172.168.1.9 to:10.0.0.5 Because of which, the ping packet destined to the floating ip (172.168.1.9) is not received by the snat namespace of the controller node. Below is the tcpdump of the q-router interface 15:48:51.418852 fa:16:3e:48:fa:e5 > fa:16:3e:01:b5:31, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 20248, offset 0, flags [DF], proto ICMP (1), length 84) 10.0.0.3 > 172.168.1.9: ICMP echo request, id 29185, seq 0, length 64 15:48:51.418920 fa:16:3e:01:b5:31 > Broadcast, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.5 tell 10.0.0.1, length 28 15:48:51.419430 fa:16:3e:ef:ce:6b > fa:16:3e:01:b5:31, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.5 is-at fa:16:3e:ef:ce:6b, length 28 15:48:51.419446 fa:16:3e:01:b5:31 > fa:16:3e:ef:ce:6b, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 20248, offset 0, flags [DF], proto ICMP (1), length 84) 10.0.0.3 > 10.0.0.5: ICMP echo request, id 29185, seq 0, length 64 15:48:52.418927 fa:16:3e:48:fa:e5 > fa:16:3e:01:b5:31, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 20480, offset 0, flags [DF], proto ICMP (1), length 84) 10.0.0.3 > 172.168.1.9: ICMP echo request, id 29185, seq 1, length 64 15:48:52.418996 fa:16:3e:01:b5:31 > fa:16:3e:ef:ce:6b, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 20480, offset 0, flags [DF], proto ICMP (1), length 84) I manually deleted the DNAT rule from iptables and it seems to work fine initially. But it had side effects. I am not sure if its worth fixing it. Thanks Numan ** Changed in: neutron Status: In Progress => Opinion ** Changed in: neutron Assignee: Numan Siddique (numansiddique) => (unassigned) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1462154 Title: With DVR Pings to floating IPs replied with fixed-ips Status in OpenStack Neutron (virtual network service): Opinion Bug description: On my single node devstack setup, there are 2 VMs hosted. VM1 has no floating IP assigned. VM2 has a floating IP assigned. From VM1, ping to VM2 using the floating IP. Ping output reports the replies comes from VM2's fixed ip address. The reply should be from VM2's floating ip address. This is a DVR problem as it doesn't happen when the L3 agent's mode is 'legacy'. This may be a problem with the NAT rules defined by the DVR L3-agent. I used the latest neutron code on the master branch to reproduce, The agent_mode is set to 'dvr_snat'. Here is how the problem is reproduced: VM1 and VM2 runs on the same host. VM1 has fixed IP of 10.11.12.4, no floating-ip associated. VM2 has fixed IP of 10.11.12.5 floating-ip=10.127.10.226 Logged into VM1 from the qrouter namespace. From VM1, ping to 10.127.10.226, ping output at VM1 reports ping replies are from the VM2's fixed IP address # ssh cirros@10.11.12.4 cirros@10.11.12.4's password: $ ping 10.127.10.226 PING 10.127.10.226 (10.127.10.226): 56 data bytes 64 bytes from 10.11.12.5: seq=0 ttl=64 time=4.189 ms 64 bytes from 10.11.12.5: seq=1 ttl=64 time=1.254 ms 64 bytes from 10.11.12.5: seq=2 ttl=64 time=2.386 ms 64 bytes from 10.11.12.5: seq=3 ttl=64 time=2.064 ms ^C --- 10.127.10.226 ping statistics --- 4 packets transmitted, 4 packets received, 0% packet loss round-trip min/avg/max = 1.254/2.473/4.189 ms $ If I associate a floating IP on VM1 then repeat the same test, ping reports the replies comes from VM2's floating IP: # ssh cirros@10.11.12.4 cirros@10.11.12.4's password: $ ping 10.127.10.226 PING 10.127.10.226 (10.127.10.226): 56 data bytes 64 bytes from 10.127.10.226: seq=0 ttl=63 time=16.750 ms 64 bytes from 10.127.10.226: seq=1 ttl=63 time=2.417 ms 64 bytes from 10.127.10.226: seq=2 ttl=63 time=1.558 ms 64 bytes from 10.127.10.226: seq=3 ttl=63 time=1.042 ms 64 bytes from 10.127.10.226: seq=4 ttl=63 time=2.770 ms ^C --- 10.127.10.226 ping statistics --- 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max = 1.042/4.907/16.750 ms $ To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1462154/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yaho
[Yahoo-eng-team] [Bug 1462154] Re: With DVR Pings to floating IPs replied with fixed-ips
** Changed in: neutron Status: Opinion => New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1462154 Title: With DVR Pings to floating IPs replied with fixed-ips Status in OpenStack Neutron (virtual network service): New Bug description: On my single node devstack setup, there are 2 VMs hosted. VM1 has no floating IP assigned. VM2 has a floating IP assigned. From VM1, ping to VM2 using the floating IP. Ping output reports the replies comes from VM2's fixed ip address. The reply should be from VM2's floating ip address. This is a DVR problem as it doesn't happen when the L3 agent's mode is 'legacy'. This may be a problem with the NAT rules defined by the DVR L3-agent. I used the latest neutron code on the master branch to reproduce, The agent_mode is set to 'dvr_snat'. Here is how the problem is reproduced: VM1 and VM2 runs on the same host. VM1 has fixed IP of 10.11.12.4, no floating-ip associated. VM2 has fixed IP of 10.11.12.5 floating-ip=10.127.10.226 Logged into VM1 from the qrouter namespace. From VM1, ping to 10.127.10.226, ping output at VM1 reports ping replies are from the VM2's fixed IP address # ssh cirros@10.11.12.4 cirros@10.11.12.4's password: $ ping 10.127.10.226 PING 10.127.10.226 (10.127.10.226): 56 data bytes 64 bytes from 10.11.12.5: seq=0 ttl=64 time=4.189 ms 64 bytes from 10.11.12.5: seq=1 ttl=64 time=1.254 ms 64 bytes from 10.11.12.5: seq=2 ttl=64 time=2.386 ms 64 bytes from 10.11.12.5: seq=3 ttl=64 time=2.064 ms ^C --- 10.127.10.226 ping statistics --- 4 packets transmitted, 4 packets received, 0% packet loss round-trip min/avg/max = 1.254/2.473/4.189 ms $ If I associate a floating IP on VM1 then repeat the same test, ping reports the replies comes from VM2's floating IP: # ssh cirros@10.11.12.4 cirros@10.11.12.4's password: $ ping 10.127.10.226 PING 10.127.10.226 (10.127.10.226): 56 data bytes 64 bytes from 10.127.10.226: seq=0 ttl=63 time=16.750 ms 64 bytes from 10.127.10.226: seq=1 ttl=63 time=2.417 ms 64 bytes from 10.127.10.226: seq=2 ttl=63 time=1.558 ms 64 bytes from 10.127.10.226: seq=3 ttl=63 time=1.042 ms 64 bytes from 10.127.10.226: seq=4 ttl=63 time=2.770 ms ^C --- 10.127.10.226 ping statistics --- 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max = 1.042/4.907/16.750 ms $ To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1462154/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1462154] Re: With DVR Pings to floating IPs replied with fixed-ips if VMs are on the same network
Reviewed: https://review.openstack.org/289172 Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=a388f78c8cb4b1c860bfc11029b5210955f1932d Submitter: Jenkins Branch:master commit a388f78c8cb4b1c860bfc11029b5210955f1932d Author: Hong Hui Xiao Date: Thu May 12 05:48:15 2016 + DVR: Pings to floatingip returns with fixed-ip on same network Pinging a floatingip of VM1 from a second VM(VM2) which has SNAT enabled connected to a DVR router on the same network returns with fixed-ip address rather than the floatingip address. The NAT forwarding rules for floatingip in the router namespace does not check for the in coming port and tries to add the rule for all in coming ports. This causes the packets that are originating from the router namespace to be modified and forwarded directly to the VM2 fixed ip instead of forwarding the traffic to the SNAT namespace. The fix in here will make sure that for all routers, the floatingip forwarding rules will be applied only to the 'rfp-' internal ports and not to all ports. Change-Id: I9453beffd94bf685afd74b0820506fb6b7c996c4 Closes-Bug: #1462154 Co-Authored-By: Hong Hui Xiao ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1462154 Title: With DVR Pings to floating IPs replied with fixed-ips if VMs are on the same network Status in neutron: Fix Released Bug description: On my single node devstack setup, there are 2 VMs hosted. VM1 has no floating IP assigned. VM2 has a floating IP assigned. From VM1, ping to VM2 using the floating IP. Ping output reports the replies comes from VM2's fixed ip address. The reply should be from VM2's floating ip address. This is a DVR problem as it doesn't happen when the L3 agent's mode is 'legacy'. This may be a problem with the NAT rules defined by the DVR L3-agent. I used the latest neutron code on the master branch to reproduce, The agent_mode is set to 'dvr_snat'. Here is how the problem is reproduced: VM1 and VM2 runs on the same host. VM1 has fixed IP of 10.11.12.4, no floating-ip associated. VM2 has fixed IP of 10.11.12.5 floating-ip=10.127.10.226 Logged into VM1 from the qrouter namespace. From VM1, ping to 10.127.10.226, ping output at VM1 reports ping replies are from the VM2's fixed IP address # ssh cirros@10.11.12.4 cirros@10.11.12.4's password: $ ping 10.127.10.226 PING 10.127.10.226 (10.127.10.226): 56 data bytes 64 bytes from 10.11.12.5: seq=0 ttl=64 time=4.189 ms 64 bytes from 10.11.12.5: seq=1 ttl=64 time=1.254 ms 64 bytes from 10.11.12.5: seq=2 ttl=64 time=2.386 ms 64 bytes from 10.11.12.5: seq=3 ttl=64 time=2.064 ms ^C --- 10.127.10.226 ping statistics --- 4 packets transmitted, 4 packets received, 0% packet loss round-trip min/avg/max = 1.254/2.473/4.189 ms $ If I associate a floating IP on VM1 then repeat the same test, ping reports the replies comes from VM2's floating IP: # ssh cirros@10.11.12.4 cirros@10.11.12.4's password: $ ping 10.127.10.226 PING 10.127.10.226 (10.127.10.226): 56 data bytes 64 bytes from 10.127.10.226: seq=0 ttl=63 time=16.750 ms 64 bytes from 10.127.10.226: seq=1 ttl=63 time=2.417 ms 64 bytes from 10.127.10.226: seq=2 ttl=63 time=1.558 ms 64 bytes from 10.127.10.226: seq=3 ttl=63 time=1.042 ms 64 bytes from 10.127.10.226: seq=4 ttl=63 time=2.770 ms ^C --- 10.127.10.226 ping statistics --- 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max = 1.042/4.907/16.750 ms $ To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1462154/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
