>>> Perhaps you could check the API being called, via the networks tab in the >>> browser developer tools. The failed request generally responds with whats >>> wrong. >>> Another reason it could fail is a missing or invalid parameter. It doesn't >>> throw an exception in the logs but is more of a one-liner like this : >>> >>> 2020-09-09 01:53:00,858 INFO [c.c.a.ApiServer] >>> (qtp1136419747-3235:ctx-be37c54f ctx-5a9ade88) (logid:ec603079) Unable to >>> execute API command createpod due to missing parameter name >>> >>> The logs would be helpful. >>> >>> Thanks, >>> David >>> ________________________________ >>> >>> I tried creating the zone again and this is what I'm seeing in the API log. The UI reported starting the pod creation around 2020 05:58:44 PM UTC and it reported the error around 2020 05:59:14 PM UTC
2020-09-09 17:58:43,789 INFO [a.c.c.a.ApiServer] (qtp1984975621-521:ctx-6447166c ctx-ef2b62b9) (logid:1b2dca03) (userId=2 accountId=2 sessionId=node01xmngwypx9ttd196tkm74q241j13) 172.16.44.18 -- GET jobId=e1490fe7-aabd-436c-ad0d-7776a374de79&command=queryAsyncJobResult&response=json 200 {"queryasyncjobresultresponse":{"accountid":"98b877a6-e0b9-11ea-b3a7-525400cadf4e","userid":"98bb271a-e0b9-11ea-b3a7-525400cadf4e","cmd":"org.apache.cloudstack.api.command.admin.network.UpdateNetworkServiceProviderCmd","jobstatus":1,"jobprocstatus":0,"jobresultcode":0,"jobresulttype":"object","jobresult":{"networkserviceprovider":{"name":"InternalLbVm","physicalnetworkid":"7b1eff21-425e-451a-89e2-94f7c2128f9e","state":"Enabled","id":"6c25618d-b64e-4a6a-89a9-b30ad1a42f72","servicelist":["Lb"],"canenableindividualservice":true}},"created":"2020-09-09T17:58:42+0000","completed":"2020-09-09T17:58:42+0000","jobid":"e1490fe7-aabd-436c-ad0d-7776a374de79"}} 2020-09-09 17:58:43,848 INFO [a.c.c.a.ApiServer] (qtp1984975621-474:ctx-754fb2aa ctx-d89d3c6b) (logid:493ef4f9) (userId=2 accountId=2 sessionId=node01xmngwypx9ttd196tkm74q241j13) 172.16.44.18 -- GET jobId=dcd1e698-1134-4cc9-8d24-291a1884f9f2&command=queryAsyncJobResult&response=json 200 {"queryasyncjobresultresponse":{"accountid":"98b877a6-e0b9-11ea-b3a7-525400cadf4e","userid":"98bb271a-e0b9-11ea-b3a7-525400cadf4e","cmd":"org.apache.cloudstack.api.command.admin.network.UpdateNetworkServiceProviderCmd","jobstatus":1,"jobprocstatus":0,"jobresultcode":0,"jobresulttype":"object","jobresult":{"networkserviceprovider":{"name":"VpcVirtualRouter","physicalnetworkid":"7b1eff21-425e-451a-89e2-94f7c2128f9e","state":"Enabled","id":"b781fe6e-57ae-45b8-9a08-575f616d2af5","servicelist":["Vpn","Dhcp","Dns","Gateway","Lb","SourceNat","StaticNat","PortForwarding","UserData"],"canenableindividualservice":true}},"created":"2020-09-09T17:58:42+0000","completed":"2020-09-09T17:58:43+0000","jobid":"dcd1e698-1134-4cc9-8d24-291a1884f9f2"}} 2020-09-09 18:00:17,323 INFO [a.c.c.a.ApiServer] (qtp1984975621-499:ctx-57c32f54 ctx-57989bcd) (logid:76a262cc) (userId=2 accountId=2 sessionId=node01xmngwypx9ttd196tkm74q241j13) 172.16.44.18 -- GET name=sdn.ovs.controller&command=listConfigurations&response=json 200 {"listconfigurationsresponse":{"count":1,"configuration":[{"category":"Network","name":"sdn.ovs.controller.default.label","value":"cloud-public","description":"Default network label to be used when fetching interface for GRE endpoints","isdynamic":false}]}} 2020-09-09 18:00:22,155 INFO [a.c.c.a.ApiServer] (qtp1984975621-476:ctx-e8338538 ctx-0e36fba5) (logid:2223a11e) (userId=2 accountId=2 sessionId=node01xmngwypx9ttd196tkm74q241j13) 172.16.44.18 -- GET zoneId=0bf0afee-1bce-4840-9283-e090ccdefdb9&name=POD1&gateway=10.2.20.1&netmask=255.255.252.0&startIp=10.2.21.65&endIp=10.2.21.80&command=createPod&response=json 200 {"createpodresponse":{"pod":{"id":"fcb7d090-d29b-4bab-8b6d-ca7db008e387","name":"POD1","zoneid":"0bf0afee-1bce-4840-9283-e090ccdefdb9","zonename":"BL1","gateway":"10.2.20.1","netmask":"255.255.252.0","startip":["10.2.21.65"],"endip":["10.2.21.80"],"forsystemvms":["0"],"vlanid":["vlan://untagged"],"allocationstate":"Enabled"}}} 2020-09-09 18:00:31,824 INFO [a.c.c.a.ApiServer] (qtp1984975621-512:ctx-b0c3fb79 ctx-f9d2b41a) (logid:28023bad) (userId=2 accountId=2 sessionId=node01xmngwypx9ttd196tkm74q241j13) 172.16.44.18 -- GET listall=true&page=1&pagesize=10&command=listSystemVms&response=json 200 {"listsystemvmsresponse":{}} 2020-09-09 18:00:40,222 INFO [a.c.c.a.ApiServer] (qtp1984975621-476:ctx-4b638a90 ctx-d49c860a) (logid:fbffa4f6) (userId=2 accountId=2 sessionId=node01xmngwypx9ttd196tkm74q241j13) 172.16.44.18 -- GET command=listZones&response=json 200 {"listzonesresponse":{"count":1,"zone":[{"id":"0bf0afee-1bce-4840-9283-e090ccdefdb9","name":"BL1","dns1":"8.8.8.8","dns2":"8.8.4.4","ip6dns1":"2001:4860:4860::8888","ip6dns2":"2001:4860:4860::8888","internaldns1":"10.2.81.1","internaldns2":"10.2.81.2","guestcidraddress":"192.168.0.0/24","networktype":"Advanced","securitygroupsenabled":false,"allocationstate":"Disabled","zonetoken":"4f4f38e0-1f72-3a59-b15b-3d7e7389cec4","dhcpprovider":"VirtualRouter","localstorageenabled":false,"tags":[]}]}} 2020-09-09 18:00:40,272 INFO [a.c.c.a.ApiServer] (qtp1984975621-474:ctx-222c137e ctx-89a6547d) (logid:44f5fcac) (userId=2 accountId=2 sessionId=node01xmngwypx9ttd196tkm74q241j13) 172.16.44.18 -- GET page=1&pagesize=6&listall=true&command=listEvents&response=json 200 {"listeventsresponse":{"count":375,"event":[{"id":"0bfee4b7-2b72-414a-a418-be7ab4883756","username":"admin","type":"SERVICE.PROVIDER.UPDATE","level":"INFO","description":"Updating physical network ServiceProvider: 106","account":"system","domainid":"76a00e1c-daa0-11ea-b3a7-525400cadf4e","domain":"ROOT","created":"2020-09-09T17:58:42+0000","state":"Scheduled"},{"id":"2c8d237a-b85d-4776-bc65-7c4b15261d16","username":"admin","type":"SERVICE.PROVIDER.UPDATE","level":"INFO","description":"Updating physical network ServiceProvider: 108","account":"system","domainid":"76a00e1c-daa0-11ea-b3a7-525400cadf4e","domain":"ROOT","created":"2020-09-09T17:58:42+0000","state":"Scheduled"},{"id":"762a9b71-43d4-48fa-ac90-fe0dfbb3628b","username":"admin","type":"SERVICE.PROVIDER.UPDATE","level":"INFO","description":"Updating physical network ServiceProvider","account":"admin","domainid":"76a00e1c-daa0-11ea-b3a7-525400cadf4e","domain":"ROOT","created":"2020-09-09T17:58:42+0000","state":"Started","parentid":"2c8d237a-b85d-4776-bc65-7c4b15261d16"},{"id":"c6c59730-d0c0-428d-a4a0-99a662751318","username":"admin","type":"SERVICE.PROVIDER.UPDATE","level":"INFO","description":"Updating physical network ServiceProvider","account":"admin","domainid":"76a00e1c-daa0-11ea-b3a7-525400cadf4e","domain":"ROOT","created":"2020-09-09T17:58:42+0000","state":"Started","parentid":"0bfee4b7-2b72-414a-a418-be7ab4883756"},{"id":"30e4bcf6-df08-402e-8e56-df9eaffbf001","username":"admin","type":"SERVICE.PROVIDER.UPDATE","level":"INFO","description":"Successfully completed Updating physical network ServiceProvider","account":"admin","domainid":"76a00e1c-daa0-11ea-b3a7-525400cadf4e","domain":"ROOT","created":"2020-09-09T17:58:42+0000","state":"Completed","parentid":"2c8d237a-b85d-4776-bc65-7c4b15261d16"},{"id":"add1db88-bd07-45f5-91ed-ec5e12ad2c78","username":"admin","type":"SERVICE.PROVIDER.UPDATE","level":"INFO","description":"Successfully completed Updating physical network ServiceProvider","account":"admin","domainid":"76a00e1c-daa0-11ea-b3a7-525400cadf4e","domain":"ROOT","created":"2020-09-09T17:58:42+0000","state":"Completed","parentid":"0bfee4b7-2b72-414a-a418-be7ab4883756"}]}} 2020-09-09 18:00:40,462 INFO [a.c.c.a.ApiServer] (qtp1984975621-512:ctx-b76c3e22 ctx-f6908bc6) (logid:5265e7a5) (userId=2 accountId=2 sessionId=node01xmngwypx9ttd196tkm74q241j13) 172.16.44.18 -- GET zoneid=0bf0afee-1bce-4840-9283-e090ccdefdb9&fetchlatest=false&command=listCapacity&response=json 200 {"listcapacityresponse":{"count":3,"capacity":[{"type":2,"name":"STORAGE","zoneid":"0bf0afee-1bce-4840-9283-e090ccdefdb9","zonename":"BL1","capacityused":0,"capacitytotal":0,"percentused":"0"},{"type":6,"name":"SECONDARY_STORAGE","zoneid":"0bf0afee-1bce-4840-9283-e090ccdefdb9","zonename":"BL1","capacityused":0,"capacitytotal":0,"percentused":"0"},{"type":19,"name":"GPU","zoneid":"0bf0afee-1bce-4840-9283-e090ccdefdb9","zonename":"BL1","capacityused":0,"capacitytotal":0,"percentused":"0"}]}} 2020-09-09 18:00:43,610 INFO [a.c.c.a.ApiServer] (qtp1984975621-499:ctx-25cca7c4 ctx-ef843394) (logid:8cee0b72) (userId=2 accountId=2 sessionId=node01xmngwypx9ttd196tkm74q241j13) 172.16.44.18 -- GET command=listInfrastructure&response=json 200 {"listinfrastructureresponse":{"infrastructure":{"zones":1,"pods":1,"clusters":0,"hosts":0,"storagepools":0,"imagestores":0,"systemvms":0,"routers":0,"ilbvms":0,"cpusockets":0,"managementservers":2,"alerts":7}}} 2020-09-09 18:00:49,710 INFO [a.c.c.a.ApiServer] (qtp1984975621-512:ctx-8b1d1c63 ctx-6a8e54a6) (logid:34939e81) (userId=2 accountId=2 sessionId=node01xmngwypx9ttd196tkm74q241j13) 172.16.44.18 -- GET listall=true&page=1&pagesize=10&command=listZonesMetrics&response=json 200 {"listzonesmetricsresponse":{"count":1,"zone":[{"state":"Disabled","clusters":"0 / 0","cputotal":"0.00 Ghz","memorytotal":"0.00 GB","id":"0bf0afee-1bce-4840-9283-e090ccdefdb9","name":"BL1","dns1":"8.8.8.8","dns2":"8.8.4.4","ip6dns1":"2001:4860:4860::8888","ip6dns2":"2001:4860:4860::8888","internaldns1":"10.2.81.1","internaldns2":"10.2.81.2","guestcidraddress":"192.168.0.0/24","networktype":"Advanced","securitygroupsenabled":false,"allocationstate":"Disabled","zonetoken":"4f4f38e0-1f72-3a59-b15b-3d7e7389cec4","dhcpprovider":"VirtualRouter","localstorageenabled":false,"tags":[]}]}} 2020-09-09 18:00:57,136 INFO [a.c.c.a.ApiServer] (qtp1984975621-499:ctx-53a62fb1 ctx-9c860007) (logid:0cd6a3a6) (userId=2 accountId=2 sessionId=node01xmngwypx9ttd196tkm74q241j13) 172.16.44.18 -- GET listall=true&page=1&pagesize=10&command=listManagementServers&response=json 200 {"listmanagementserversresponse":{"count":2,"managementserver":[{"id":"0c9c71a5-776f-4fad-be53-31a86ca6e18d","name":"bllcloudctl01.harmonywave.cloud","state":"Up","version":"4.14.0.0"},{"id":"642b3206-e131-4c20-be72-981dcb6dfafe","name":"bllcloudctl02.harmonywave.cloud","state":"Down","version":"4.14.0.0"}]}} It looks like the pod is created successfully. When I exit the wizard I do see the pod listed in the infrastructure, but no system VM's are created. Is there a way to try and force the system VM's to be created/started to see if I can diagnose if there is a problem with them? -- Thanks, Joshua Schaeffer