Realized that if I add these 2 lines inside VPC router: root@r-109-VM:~# diff /opt/cloud/bin/cs_ip.py /opt/cloud/bin/cs_ip.py.back 37,38d36 < if nic_dev_id is None: < nic_dev_id=1
and edit /etc/cloudstack/ips.json properly once, everything begins to work, everything begins to work (even all new operations in GUI) Not clear whether this is bug of some sort, or our config is wrong to begin with.. Thanks Roman -----Original Message----- From: Roman Ledovskiy [mailto:roman.ledovs...@exalit.com] Sent: Friday, 16 June, 2017 11:45 PM To: users@cloudstack.apache.org Subject: RE: ethNone problem with VPC Hi Dag, Not using OVS, using Linux bridges.. So I assume that cloud0 should be managed by cloudstack.. Thanks Roman -----Original Message----- From: Dag Sonstebo [mailto:dag.sonst...@shapeblue.com] Sent: Friday, 16 June, 2017 11:34 PM To: users@cloudstack.apache.org Subject: Re: ethNone problem with VPC Hi Roman, In my experience when using OVS the cloud0 bridge sometimes has to be configured manually. Check my blog post from last year on this, it has some examples on the cloud0 bridge setup: http://www.shapeblue.com/networking-kvm-for-cloudstack/ Regards, Dag Sonstebo Cloud Architect ShapeBlue On 16/06/2017, 15:45, "Roman Ledovskiy" <roman.ledovs...@exalit.com> wrote: I'm not sure if it is actually related to cloud0... I'm troubleshooting this way now: Copying one of /var/cache/cloud/processed/ip_associations.json.* to /var/cache/cloud/ip_associations.json inside VPC Running from the host /usr/share/cloudstack-common/scripts/network/domr/router_proxy.sh update_config.py 169.254.1.227 ip_associations.json I'm immediately getting the error (I added few print lines and trying to figure out where exactly in /opt/cloud/bin/* interface becomes ethNone) https://pastebin.com/NECSej8Z Thanks Roman -----Original Message----- From: Roman Ledovskiy [mailto:roman.ledovs...@exalit.com] Sent: Friday, 16 June, 2017 10:28 PM To: users@cloudstack.apache.org Subject: RE: ethNone problem with VPC Simon, Thanks for your help. I posted all my configs and print screens with labels (URL - first line in pastebin) here: https://pastebin.com/NAuxN9ms Thanks Roman -----Original Message----- From: Simon Weller [mailto:swel...@ena.com.INVALID] Sent: Friday, 16 June, 2017 9:24 PM To: users@cloudstack.apache.org Subject: Re: ethNone problem with VPC Interface and bridge configs would be helpful as well. ________________________________ From: Simon Weller Sent: Friday, June 16, 2017 8:14 AM To: users@cloudstack.apache.org Subject: Re: ethNone problem with VPC Can you provide your /etc/cloudstack/agent/agent.properties file on the host? Can you also screenshot your traffic label configuration for each network type? ________________________________ From: Roman Ledovskiy <roman.ledovs...@exalit.com> Sent: Friday, June 16, 2017 7:59 AM To: users@cloudstack.apache.org Subject: RE: ethNone problem with VPC Simon, I saw this but I thought that "cloud0" is something I have no control over? (Some internal cloudstack bridge that it creates and manages completely automatically) Do you have any idea on how this can be fixed? Thanks Roman -----Original Message----- From: Simon Weller [mailto:swel...@ena.com.INVALID] Sent: Friday, 16 June, 2017 8:45 PM To: users@cloudstack.apache.org Subject: Re: ethNone problem with VPC Roman, Take a look at these logs: 1. 017-06-16 14:55:21,410 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:49d03fbe) failing to get physical interface from bridge cloud0, did not find an eth*, bond*, team*, vlan*, em*, p*p*, ens*, eno*, enp*, or enx* in /sys/devices/virtual/net/cloud0/brif 2. 2017-06-16 14:55:21,410 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:49d03fbe) getting broadcast uri for pif and bridge cloud0 3. 2017-06-16 14:55:21,410 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) (logid:49d03fbe) failed to get vNet id from bridge cloud0attached to physical interface You may need to change your underlying interface name to match one of the supported types above. I'd fix that before you troubleshoot further and see if it has any affect. - Si ________________________________ From: Roman Ledovskiy <roman.ledovs...@exalit.com> Sent: Friday, June 16, 2017 2:38 AM To: users@cloudstack.apache.org Subject: RE: ethNone problem with VPC Simon, Yes, I have traffic labels setup for my networks (2 guest networks with different tags and 1 public). Advanced network setup / Centos-7.x on hypervisors. I enabled debug on hypervisor and tried to restart router VM via Cloudstack GUI. Since its config overwritten I presume something happens exactly at this stage.. Here it is: https://pastebin.com/6VRHyRnJ [https://pastebin.com/i/facebook.png]<https://pastebin.com/6VRHyRnJ> 2017-06-16 14:54:57,513 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:null) - Pastebin.com<https://pastebin.com/6VRHyRnJ> pastebin.com I really can't see anything wrong in there. It correctly associates VPC public interface with my bridge for public network (brenp6s0-904) Thanks Roman -----Original Message----- From: Simon Weller [mailto:swel...@ena.com.INVALID] Sent: Friday, 16 June, 2017 5:40 AM To: users@cloudstack.apache.org Subject: Re: ethNone problem with VPC Roman, Do you have a traffic label setup for your public network? Can you put the host in debug mode: https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+agent+debug - Si ________________________________ From: Roman Ledovskiy <roman.ledovs...@exalit.com> Sent: Wednesday, June 14, 2017 7:59 AM To: users@cloudstack.apache.org Subject: ethNone problem with VPC Hi Everyone, Does anyone encounter "ethNone" interfaces in VPC routers? I'm running cloudstack 4.9.2.0 with KVM hypervisor and every time I create VPC router it fails to do proper setup, with errors: ==== Traceback (most recent call last): File "/opt/cloud/bin/configure.py", line 931, in main config.address().process() File "/opt/cloud/bin/cs/CsAddress.py", line 103, in process ip = CsIP(dev, self.config) File "/opt/cloud/bin/cs/CsAddress.py", line 256, in __init__ self.dnum = hex(int(dev[3:])) ValueError: invalid literal for int() with base 10: 'None' ==== Only local link IP (eth0) is up. /etc/cloudstack/ips.json inside my VPC routers always look like this (I guess this explains above): ==== { "eth0": [ { "add": true, "broadcast": "169.254.255.255", "cidr": "169.254.1.238/16", "device": "eth0", ..... ... } ], "ethNone": [ { "add": true, "broadcast": "X.Y.Z.255", "cidr": "Y.Y.Z.43/24", "device": "ethNone", ..... .... } ], "id": "ips" } ==== X.Y.Z.0/24 is my public network If I manually edit /etc/cloudstack/ips.json (change ethNone to eth1) - then restart VPC router manually (via virsh) - everything works, interfaces/iptables are up.. Until next time I do anything with this VPC from GUI - then ethNone reappears and everything is broken again.. Appreciate if anyone has any thoughts. Thanks Roman dag.sonst...@shapeblue.com www.shapeblue.com 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue