Solved my own problem, I think. From what I can tell, the issue relates back to my decision to use openvswitch when setting up KVM on my hosts. If you read the install guide for KVM (http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/kvm.html) it goes over two different network setups – plain old bridging, and openvswitch. Reading that page, and having used openvswitch before, I decided to deploy it over traditional bridging. The network becomes software defined, and life is good.
Well, except if you want to create a VPC. If you go to the documentation page for the OVS plugin (http://docs.cloudstack.apache.org/en/latest/networking/ovs-plugin.html) and read all the way to the very very bottom it says "OVS plugin does not work with VPC at that time”. To test my theory, I built another host, setup the networking the bridge way, and successfully deployed a VPC. Am I correct in my assumption that because I used OVS to setup the host networking that was the cause of my VPC deployment issues? If so, shouldn’t the KVM setup docs be updated to clearly state that if you use OVS then VPCs break? Or is it supposed to work and I found a bug? On 1/21/16, 2:52 PM, "Matthew M. Gamble" <mgam...@thoughtfire.ca> wrote: >Trying to debug this again, I got the ip_associations.json that is being >passed to the router on create: > >{ > "ip_address":[ > { > "public_ip":"XXX.XXX.XXX.188", > "source_nat":true, > "add":true, > "one_to_one_nat":false, > "first_i_p":false, > "gateway":"1XXX.XXX.XXX.161", > "netmask":"255.255.255.224", > "vif_mac_address”:”XX:XX:XX:00:00:80", > "new_nic":false > } > ], > "type":"ips" >} > > >The one thing I noticed is the trackback is an issue parsing the nic_dev_id >field, which makes sense, since the nic_dev_id field doesn’t exist in the json >file. So my question is, why is this field missing from the file and how to I >backtrace this issue further? > > > >On 1/4/16, 6:00 PM, "Matthew M. Gamble" <mgam...@thoughtfire.ca> wrote: > >>Sure – I just recreated the issue and uploaded a full log snippet from the >>management server: http://pastebin.com/MLR9JYQ3 >> >>From: Glenn Wagner >><glenn.wag...@shapeblue.com<mailto:glenn.wag...@shapeblue.com>> >>Reply-To: "users@cloudstack.apache.org<mailto:users@cloudstack.apache.org>" >><users@cloudstack.apache.org<mailto:users@cloudstack.apache.org>> >>Date: Monday, January 4, 2016 at 2:25 PM >>To: "users@cloudstack.apache.org<mailto:users@cloudstack.apache.org>" >><users@cloudstack.apache.org<mailto:users@cloudstack.apache.org>> >>Subject: Re: Cannot create VPC >> >>Hi, >> >>Could you send us a big section of the log file , before and after the error >> >>Thanks >> >>Glenn Wagner >>Senior Consultant, South Africa >> >> >> >>M: +27 73 917 4111 >> >> >>[ShapeBlue]<http://www.shapeblue.com> >>Glenn Wagner >>Senior Consultant , ShapeBlue >> >> >>d: | s: +27 21 527 0091<tel:|%20s:%20+27%2021%20527%200091> | >> m: +27 73 917 4111<tel:+27%2073%20917%204111> >> >>e: glenn.wag...@shapeblue.com | t: >><mailto:glenn.wag...@shapeblue.com%20|%20t:> | w: >>www.shapeblue.com<http://www.shapeblue.com> >> >>a: 2nd Floor, Oudehuis Centre, 122 Main Rd, Somerset West Cape Town 7130 >>South Africa >> >> >>[cid:image3c9f62.png@b7e7b1d4.4e889590] >> >> >>Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue >>Services India LLP is a company incorporated in India and is operated under >>license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company >>incorporated in Brasil and is operated under license from Shape Blue Ltd. >>ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa >>and is traded under license from Shape Blue Ltd. ShapeBlue is a registered >>trademark. >>This email and any attachments to it may be confidential and are intended >>solely for the use of the individual to whom it is addressed. Any views or >>opinions expressed are solely those of the author and do not necessarily >>represent those of Shape Blue Ltd or related companies. If you are not the >>intended recipient of this email, you must neither take any action based upon >>its contents, nor copy or show it to anyone. Please contact the sender if you >>believe you have received this email in error. >> >> >> >> >>________________________________________ >>From: Matthew M. Gamble >><mgam...@thoughtfire.ca<mailto:mgam...@thoughtfire.ca>> >>Sent: Wednesday, December 30, 2015 8:44 PM >>To: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org> >>Subject: Cannot create VPC >> >>When attempting to create a VPC using 4.7 (with the 4.6 system images) under >>KVM, I’m getting the following error on the agent when it attempts to deploy >>the virtual router: >> >>{"results":["null - success: >>null","null - failed: [INFO] update_config.py :: Processing incoming >>file => ip_associations.json[INFO] Processing JSON file >>ip_associations.jsonTraceback (most recent call last): File >>\"/opt/cloud/bin/update_config.py\", line 140, in <module> >>process_file() File \"/opt/cloud/bin/update_config.py\", line 52, in >>process_file qf.load(None) File \"/opt/cloud/bin/merge.py\", line >>258, in load proc = updateDataBag(self) File >>\"/opt/cloud/bin/merge.py\", line 91, in __init__ self.process() >>File \"/opt/cloud/bin/merge.py\", line 103, in process dbag = >>self.processIP(self.db.getDataBag()) File \"/opt/cloud/bin/merge.py\", >>line 190, in processIP dbag = cs_ip.merge(dbag, ip) File >>\"/opt/cloud/bin/cs_ip.py\", line 32, in merge ip['device'] = 'eth' >>+ str(ip['nic_dev_id'])KeyError: >>'nic_dev_id'"],"result":false,"wait":0}},{"com.cloud.agent.api.Answer": >>{"result":false,"details":"Stopped by previous >>failure","wait":0}},{"com.cloud.agent.api.Answer":{"result":false," >>details":"Stopped by previous failure","wait":0}}] } >> >>Any ideas where to start debugging this problem? It looks like the nic_dev_id >>isn’t being passed correctly to the VM? >> >> >>Find out more about ShapeBlue and our range of CloudStack related services: >>IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//> >>| CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/> >>CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> | >>CloudStack Software >>Engineering<http://shapeblue.com/cloudstack-software-engineering/> >>CloudStack Infrastructure >>Support<http://shapeblue.com/cloudstack-infrastructure-support/> | CloudStack >>Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>