Hello, I'm trying to test the floating ip functionality on an 3.2 opencontrail instalation (all in one contrail+devstack). I have read the 3.2 user guide that is describing how to configure simple gateway and how to use and debug floating ip. I've also read some older threads on these aspects in the opencontrail email lists (for example these 2 http://lists.opencontrail.org/pipermail/dev_lists.opencontrail.org/2014-August/001397.html http://lists.opencontrail.org/pipermail/dev_lists.opencontrail.org/2013-December/000377.html ) but even so I'm not able to make it work properly.
My use case is one of the simplest. I run contrail+devstack inside a VM whith just one eth interface - eth0 with ip 10.241.13.231. I'm starting one cirros VM using open-stack commands inside the contrail VM having an ip from virtual network 20.20.20.x (20.20.20.100) and try to assign to it a floating ip from 10.241.13.x subnet. I want to be able to ping that VM from the host and also from other servers with interfaces in same subnet 10.241.13.x which are able to ping the host contrail VM. >From this link -> >https://github.com/Juniper/contrail-controller/wiki/How-to-setup-floating-ip-with-Neutron I understood that I need to add this to my localrc (under contrail-installer): CONTRAIL_VGW_INTERFACE=vgw CONTRAIL_VGW_PUBLIC_SUBNET=10.241.13.0/24 CONTRAIL_VGW_PUBLIC_NETWORK=default-domain:admin:public:public After booting up the contrail VM (bofore starting contrail and devstack) the routing table looks like this: userid@crl32debugVM:~$ route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 10.241.13.254 0.0.0.0 UG 0 0 0 eth0 10.241.13.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 192.168.122.0 0.0.0.0 255.255.255.0 U 0 0 0 virbr0 I do have 2 interfaces - one eth0 and another one virbr0 created by virtlib daemon. Than I start the contrail and vhost0 is created as clone to eth0 and another route is added for 10.241.13.0 over vgw which takes precedence over the existing route on eth0 which gets moved to vhost0. And the new route table looks like this -> userid@crl32debugVM:~$ route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 10.241.13.254 0.0.0.0 UG 0 0 0 vhost0 10.241.13.0 0.0.0.0 255.255.255.0 U 0 0 0 vgw 10.241.13.0 0.0.0.0 255.255.255.0 U 0 0 0 vhost0 192.168.122.0 0.0.0.0 255.255.255.0 U 0 0 0 virbr0 So the new simple gateway route over vgw is overwriting the existing route over vhost0 and now I'm not able anymore to ping from contrail VM the "outside" public network 10.241.13.0. Even so the ping to the floating iP is not working either from contrail VM. Even if I delete the route trough vhost0 ping will not work. I have tried a second approach - configure in localrc only a small subnet of the 10.241.13.0/24 as vgw subnet - same subnet which later I will add it to the public contrail network -> CONTRAIL_VGW_INTERFACE=vgw CONTRAIL_VGW_PUBLIC_SUBNET=10.241.13.240/29 CONTRAIL_VGW_PUBLIC_NETWORK=default-domain:admin:public:public and reconfigured and restarted contrail Now route table on host VM looks like this userid@crl32debugVM:~/contrail-installer$ route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 10.241.13.254 0.0.0.0 UG 0 0 0 vhost0 10.241.13.0 0.0.0.0 255.255.255.0 U 0 0 0 vhost0 10.241.13.240 0.0.0.0 255.255.255.248 U 0 0 0 vgw 192.168.122.0 0.0.0.0 255.255.255.0 U 0 0 0 virbr0 I can ping from host VM the public gateway 10.241.13.254 Also if I run a script to create a virtual network, launch a cirros VM with ip 20.20.20.100 and install the floating ip for it (10.241.13.240) - this is getting installed and the ping works between the host and cirros VM on floating ip and ssh is working either. But with one observation when I ping and ssh the virbr0 interface is used as source interface not the vhost0 ... as I expected. Also when I log into the cirros VM I cannot ping the host ip 10.241.13.231. And cannot ping in fact any address from 10.241.13.x. But I can ping the virbr0 IP of the host. But the intention was not this. Was to be able to attach to the cirros VM an ip from 10.241.13.0/24 subnet that can be pinged from all servers in that subnet. Also from other servers in same public subnet I'm not able to ping/ssh on the cirros VM, only from the host contrail VM. userid@crl32debugVM:~/sorint/scripts$ neutron floatingip-list +--------------------------------------+------------------+---------------------+--------------------------------------+ | id | fixed_ip_address | floating_ip_address | port_id | +--------------------------------------+------------------+---------------------+--------------------------------------+ | 39df3e06-2818-43f1-b6df-48012d0f4bf6 | 20.20.20.100 | 10.241.13.240 | 2b75c16e-5d6e-4499-b9e3-07dce6c60315 | +--------------------------------------+------------------+---------------------+--------------------------------------+ userid@crl32debugVM:~/sorint/scripts$ ping 10.241.13.240 PING 10.241.13.240 (10.241.13.240) 56(84) bytes of data. 64 bytes from 10.241.13.240: icmp_seq=1 ttl=63 time=0.985 ms 64 bytes from 10.241.13.240: icmp_seq=2 ttl=63 time=0.481 ms ^C --- 10.241.13.240 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1000ms rtt min/avg/max/mdev = 0.481/0.733/0.985/0.252 ms userid@crl32debugVM:~/sorint/scripts$ ssh [email protected] The authenticity of host '10.241.13.240 (10.241.13.240)' can't be established. RSA key fingerprint is c8:4a:9a:67:4d:b4:17:d0:48:22:89:c9:7e:d1:4b:ad. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '10.241.13.240' (RSA) to the list of known hosts. [email protected]'s password: Permission denied, please try again. [email protected]'s password: $ ifconfig eth0 Link encap:Ethernet HWaddr 02:2B:75:C1:6E:5D inet addr:20.20.20.100 Bcast:20.20.20.255 Mask:255.255.255.0 inet6 addr: fe80::2b:75ff:fec1:6e5d/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:389 errors:0 dropped:0 overruns:0 frame:0 TX packets:196 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:23575 (23.0 KiB) TX bytes:12394 (12.1 KiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) $ ping 10.241.13.231 PING 10.241.13.231 (10.241.13.231): 56 data bytes ^C --- 10.241.13.231 ping statistics --- 7 packets transmitted, 0 packets received, 100% packet loss $ ping 192.168.122.1 PING 192.168.122.1 (192.168.122.1): 56 data bytes 64 bytes from 192.168.122.1: seq=0 ttl=63 time=2.056 ms 64 bytes from 192.168.122.1: seq=1 ttl=63 time=0.582 ms 64 bytes from 192.168.122.1: seq=2 ttl=63 time=0.742 ms When ping the floating ip (10.241.13.240) from host VM I did some captures on vgw interface and the tap interface that connects the cirros VM. userid@crl32debugVM:~/contrail-installer$ sudo tcpdump -i vgw -XX tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on vgw, link-type EN10MB (Ethernet), capture size 262144 bytes 17:13:06.931265 IP 192.168.122.1 > 10.241.13.240: ICMP echo request, id 31534, seq 20, length 64 0x0000: 0000 5e00 0100 0000 5e00 0100 0800 4500 ..^.....^.....E. 0x0010: 0054 ecf6 4000 4001 fa27 c0a8 7a01 0af1 .T..@.@..'..z... 0x0020: 0df0 0800 5719 7b2e 0014 0243 d158 0000 ....W.{....C.X.. 0x0030: 0000 8535 0e00 0000 0000 1011 1213 1415 ...5............ 0x0040: 1617 1819 1a1b 1c1d 1e1f 2021 2223 2425 ...........!"#$% 0x0050: 2627 2829 2a2b 2c2d 2e2f 3031 3233 3435 &'()*+,-./012345 0x0060: 3637 67 17:13:06.931908 IP 10.241.13.240 > 192.168.122.1: ICMP echo reply, id 31534, seq 20, length 64 0x0000: 0000 0000 0001 0000 5e00 0100 0800 4500 ........^.....E. 0x0010: 0054 d844 0000 3f01 4fda 0af1 0df0 c0a8 .T.D..?.O....... 0x0020: 7a01 0000 5f19 7b2e 0014 0243 d158 0000 z..._.{....C.X.. 0x0030: 0000 8535 0e00 0000 0000 1011 1213 1415 ...5............ 0x0040: 1617 1819 1a1b 1c1d 1e1f 2021 2223 2425 ...........!"#$% 0x0050: 2627 2829 2a2b 2c2d 2e2f 3031 3233 3435 &'()*+,-./012345 userid@crl32debugVM:~/contrail-installer$ sudo tcpdump -i tap2b75c16e-5d -XX tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on tap2b75c16e-5d, link-type EN10MB (Ethernet), capture size 262144 bytes 17:14:12.953122 IP 192.168.122.1 > 20.20.20.100: ICMP echo request, id 31534, seq 86, length 64 0x0000: 022b 75c1 6e5d 0000 5e00 0100 0800 4500 .+u.n]..^.....E. 0x0010: 0054 0bb0 4000 3f01 ccd7 c0a8 7a01 1414 .T..@.?.....z... 0x0020: 1464 0800 ba81 7b2e 0056 4443 d158 0000 .d....{..VDC.X.. 0x0030: 0000 df8a 0e00 0000 0000 1011 1213 1415 ................ 0x0040: 1617 1819 1a1b 1c1d 1e1f 2021 2223 2425 ...........!"#$% 0x0050: 2627 2829 2a2b 2c2d 2e2f 3031 3233 3435 &'()*+,-./012345 0x0060: 3637 67 17:14:12.953787 IP 20.20.20.100 > 192.168.122.1: ICMP echo reply, id 31534, seq 86, length 64 0x0000: 0000 5e00 0100 022b 75c1 6e5d 0800 4500 ..^....+u.n]..E. 0x0010: 0054 f743 0000 4001 2044 1414 1464 c0a8 [email protected].. 0x0020: 7a01 0000 c281 7b2e 0056 4443 d158 0000 z.....{..VDC.X.. 0x0030: 0000 df8a 0e00 0000 0000 1011 1213 1415 ................ 0x0040: 1617 1819 1a1b 1c1d 1e1f 2021 2223 2425 ...........!"#$% 0x0050: 2627 2829 2a2b 2c2d 2e2f 3031 3233 3435 &'()*+,-./012345 0x0060: 3637 67 This is showing me that the ping uses as source ip address the virbr0 ip address (192.168.122.1) and not the 10.241.13.231 which used to be the contrail VM ip from public network and now is the vhost0 ip. Also looks that the "17:13:06.931265 IP 192.168.122.1 > 10.241.13.240: ICMP echo request" packet uses smac==dmac==vgw mac address. If I delete virbr0 interface and restart contrail and the script, the ping from contrail VM to cirros VM will not work anymore and ping request packets are discarded in vrouter due to invalid source reason (sudo dropstats) So I do not understand how can I use the simple gateway+ floating ip in order to be able to ping from fabric (public) network the VMs inside virtual networks. Bellow is the script I'm using to setup the public and virtual network and assign a floating ip to the VM -> #! /bin/bash export OS_PROJECT_DOMAIN_NAME=default export OS_USER_DOMAIN_NAME=default export OS_PROJECT_NAME=admin export OS_USERNAME=admin export OS_PASSWORD=contrail123 export OS_AUTH_URL=http://127.0.0.1:35357/v3 export OS_IDENTITY_API_VERSION=3 export OS_IMAGE_API_VERSION=2 #Setup public network in neutron neutron net-create public --router:external=True #public_id=`neutron net-list | awk '/public/{print $2}'` neutron subnet-create public --name public-subnet1 10.241.13.0/24 --allocation-pool start=10.241.13.240,end=10.241.13.245 --gateway 10.241.13.231 --disable-dhcp #Setup floating ip pool in contrail python /opt/stack/contrail/controller/src/config/utils/create_floating_pool.py --public_vn_name default-domain:admin:public --floating_ip_pool_name floatingip_pool --api_server_ip 10.241.13.231 --api_server_port 8082 --admin_user admin --admin_password contrail123 --admin_tenant_name admin python /opt/stack/contrail/controller/src/config/utils/use_floating_pool.py --project_name default-domain:admin --floating_ip_pool_name default-domain:admin:public:floatingip_pool #Setup Security Group nova secgroup-list nova secgroup-list-rules default nova secgroup-add-rule default tcp 22 22 0.0.0.0/0 nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0 nova secgroup-list-rules default neutron floatingip-create public neutron net-create --provider:network_type vlan --provider:physical_network netvlan --provider:segmentation_id 2 data_vn neutron subnet-create data_vn --name data_vn_subnet --allocation-pool start=20.20.20.100,end=20.20.20.200 --gateway 20.20.20.1 20.20.20.0/24 neutron router-create router1 neutron router-interface-add router1 data_vn_subnet neutron router-gateway-set router1 public sleep 5 echo "Create images:" NET_ID=`neutron net-show data_vn | grep " id " | awk '{print $4}'` echo $NET_ID nova boot --flavor m1.tiny --image cirros-0.3.4-x86_64-uec --nic net-id=$NET_ID data_vn_img1 sleep 10 #attach the floating IP to the VM interface port_id=$(neutron port-list | grep "20.20.20.100" | sed -ne 's/| \([-0-9a-f]*\)[ \t]*|[ \t]*.*'"$vm_net_ip"'.*/\1/p') #port_id=`echo $TMP_VAL | awk '{print $2}'` echo "port_id=" $port_id TMP_VAL=`neutron floatingip-list | awk '{print $2}'` echo $TMP_VAL floatingip_id=`echo $TMP_VAL | awk '{print $2}'` echo "floatingip_id=" $floatingip_id neutron floatingip-associate $floatingip_id $port_id I have also tried out other combinations as well Started the contrail VM with eth0 (10.241.13.0/24) and eth1 (10.100.100.0/24) up and running and started contrail+devstack on 10.100.100.x network with vgw gateway and floating ip setup first on 10.241.13.0/24 and later on 10.100.100.0/24 but in neither 2 cases manage to ping the cirros VM from host contrail VM. Thank you, Sorin.
_______________________________________________ Dev mailing list [email protected] http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org
