Cloudstack appears to not obey physical network tags.
I'm trying to get a ACS environment running with two different guest
networks (one using VXLAN, the other using VLAN).
I have two physical network interfaces (cloudbr0 and cloudbr1) and intended
to use cloudbr0 for vlan isolation and cloudbr1 for vxlan isolation.
I have configured interface tags on the physical nics and network
offerings, and have confirmed in the logs that the vxlan interface on the
router is attempting to create the network on cloudbr1 as shown:
"nics": [
{
"broadcastType": "Vlan",
"broadcastUri": "vlan://901",
"defaultNic": true,
"deviceId": 2,
"dns1": "66.194.167.3",
"dns2": "66.194.167.4",
"gateway": "66.194.167.1",
"ip": "66.194.167.142",
"isSecurityGroupEnabled": false,
"isolationUri": "vlan://901",
"mac": "06:c4:e6:00:00:18",
"name": "cloudbr0",
"netmask": "255.255.255.0",
"networkRateMbps": 200,
"nicUuid": "843263cf-d69d-496f-84fd-b6bfbf55dbf9",
"pxeDisable": true,
"type": "Public",
"uuid": "47b26175-781e-47d8-8d37-577f6d1e1213"
},
{
"*broadcastType": "Vxlan", *
*"broadcastUri": "vxlan://1712", *
"defaultNic": false,
"deviceId": 0,
"dns1": "66.194.167.3",
"dns2": "66.194.167.4",
"ip": "10.1.1.1",
"isSecurityGroupEnabled": false,
"isolationUri": "vxlan://1712",
"mac": "02:00:01:47:00:02",
*"name": "cloudbr1", *
"netmask": "255.255.255.0",
"networkRateMbps": 200,
"nicUuid": "be7d02f5-2ef4-464d-8713-9823b3ae8bb7",
"pxeDisable": true,
"type": "Guest",
"uuid": "5e510ece-0afe-44a3-9cbc-9d4c587a0dce"
},
{
"broadcastType": "LinkLocal",
"defaultNic": false,
"deviceId": 1,
"gateway": "169.254.0.1",
"ip": "169.254.2.250",
"isSecurityGroupEnabled": false,
"mac": "0e:00:a9:fe:02:fa",
"netmask": "255.255.0.0",
"networkRateMbps": -1,
"nicUuid": "269431ed-c997-472c-95da-2ac26e129335",
"pxeDisable": true,
"type": "Control",
"uuid": "05975418-d018-449c-af0e-8948243aa76d"
}
],
For context, the full log entry is here: http://pastebin.com/TazJKFqd
However, when I look on the host, the following occurs:
ip -d link show vxlan1712
41: vxlan1712: mtu 1450 qdisc noqueue
state UNKNOWN
link/ether 8e:05:2c:dd:87:22 brd ff:ff:ff:ff:ff:ff
vxlan id 1712 group 239.0.6.176 dev cloudbr0 port 32768 61000 ttl 10
ageing 300
Anyone have any idea what is up with this discrepancy?
I'm on ACS 4.5.2 and my hypervisors are KVM on Centos 6
Re: VPC Private gateway not being removed
Daan,
I've created the issue at
https://issues.apache.org/jira/browse/CLOUDSTACK-8400 . Please let me
know if you need me to gather any more information from my
environment. Thanks.
On Thu, Apr 23, 2015 at 1:46 AM, Daan Hoogland daan.hoogl...@gmail.com wrote:
Ok it resambles a bug in an older version. Can you create a bug report in
jira. You can assign to me or give me the issue id. I'll investigate.
mobile dev with bilingual spelling checker used (read at your own risk)
Op 22 apr. 2015 23:48 schreef Grayson Head gray...@graysonhead.net:
I attempted to re-create the issue on a newly-created VPC, (Since I
had removed the old one due to this issue). The issue happens
regardless of whether I attempt to delete a gateway with associated
static routes or not. (I tried it both ways, same result).
And once again, I don't see any errors in the logs, it just seems like
the static route gets removed from the VR (does not show up in the
VR's routing table) as well as the UI, but persists in the database in
a Removing state.
I'm on 4.4.2 by the way.
On Wed, Apr 22, 2015 at 4:26 PM, Daan Hoogland daan.hoogl...@gmail.com
wrote:
Grayson, quick stab at this: in older versions you couldn't delete a
gateway if it still had a static route on it. What version are you
running
and did the gateway have a route associated?
mobile dev with bilingual spelling checker used (read at your own risk)
Op 21 apr. 2015 15:46 schreef Grayson Head gray...@graysonhead.net:
Unfortunately, I fat fingered a VLAN in a VPC Private gateway I was
setting up. I removed it using the UI without any errors, but when I
go to create the new one, I get the following error: Private network
for the vlan: vlan://4000 and cidr 10.233.5.0/24 already exists for
Vpc 16 in zone AusZone1
I also tried removing it via cloudmonkey.
(local) mycloudmonkey delete privategateway
id=340b56bb-ee01-49be-84b4-9e54fd2af83d
.Error
Async job ed15f016-fd7e-48fb-b92c-0ea07302bffb failed
Error 530, Invalid private gateway id
accountid = a9d7423c-a02e-11e4-b7f0-00505695fae7
cmd =
org.apache.cloudstack.api.command.admin.vpc.DeletePrivateGatewayCmd
created = 2015-04-21T06:38:36-0700
jobid = ed15f016-fd7e-48fb-b92c-0ea07302bffb
jobprocstatus = 0
jobresult:
errorcode = 530
errortext = Invalid private gateway id
jobresultcode = 530
jobresulttype = object
jobstatus = 2
userid = a9d755d8-a02e-11e4-b7f0-00505695fae7
Looking at the database, the problem becomes somewhat clear:
mysql select id,uuid,domain_id,state,removed,ip4_address from
vpc_gateways;
++--+---+--+-+--+
| id | uuid | domain_id | state|
removed | ip4_address |
++--+---+--+-+--+
| 1 | 63a9f6f7-dbe2-4825-a3e2-17e3892c0ded | 1 | Ready|
NULL| 10.233.51.92 |
| 2 | 340b56bb-ee01-49be-84b4-9e54fd2af83d | 1 | Removing |
2015-04-20 21:28:53 | 10.233.5.11 |
++--+---+--+-+--+
2 rows in set (0.00 sec)
Unfortunately I don't have the logs from initially removing the VPC. I
can try and re-create this If needed, but I need to get this
environment fixed before I do anything else. Will setting the
state=Removed on the gateway entry in the database correct this?
Here are the logs from the management-server.log when trying to remove
the VPC via the UI:
2015-04-21 06:36:16,036 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-15:null) SeqA 9-315848: Processing Seq 9-315848:
{ Cmd , MgmtId: -1, via: 9, Ver: v1, Flags: 11,
[{com.cloud.agent.api.ConsoleProxyLoadReportCommand:{_proxyVmId:78,_loadInfo:{\n
\connections\: []\n},wait:0}}] }
2015-04-21 06:36:16,049 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-15:null) SeqA 9-315848: Sending Seq 9-315848: {
Ans: , MgmtId: 345050016868, via: 9, Ver: v1, Flags: 100010,
[{com.cloud.agent.api.AgentControlAnswer:{result:true,wait:0}}]
}
2015-04-21 06:36:16,853 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-2:null) Ping from 15
2015-04-21 06:36:16,877 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-4:null) Ping from 9
2015-04-21 06:36:20,715 DEBUG [c.c.a.ApiServlet]
(catalina-exec-6:ctx-62b2ea0a) ===START=== 10.233.51.176 -- GET
command=createPrivateGatewaysourcenatsupported=falseresponse=jsonsessionkey=sFXaEOKvIpqQArupbqZuPX6wYOo%3Dphysicalnetworkid=63a1c53c-b799-452a-ad15-3e0ef582c456vpcid=0fef5e73-b916-4505-8760-4321a3763bf6ipaddress=10.233.5.12gateway=10.233.5.1netmask=255.255.255.0vlan=4000aclid=9599b3fe-a02e-11e4-b7f0-00505695fae7_=1429623380716
2015-04-21 06:36:20,761 DEBUG [c.c.n.v.VpcManagerImpl]
(catalina-exec-6:ctx-62b2ea0a ctx-ebd9cd79
VPC Private gateway not being removed
Unfortunately, I fat fingered a VLAN in a VPC Private gateway I was
setting up. I removed it using the UI without any errors, but when I
go to create the new one, I get the following error: Private network
for the vlan: vlan://4000 and cidr 10.233.5.0/24 already exists for
Vpc 16 in zone AusZone1
I also tried removing it via cloudmonkey.
(local) mycloudmonkey delete privategateway
id=340b56bb-ee01-49be-84b4-9e54fd2af83d
.Error
Async job ed15f016-fd7e-48fb-b92c-0ea07302bffb failed
Error 530, Invalid private gateway id
accountid = a9d7423c-a02e-11e4-b7f0-00505695fae7
cmd = org.apache.cloudstack.api.command.admin.vpc.DeletePrivateGatewayCmd
created = 2015-04-21T06:38:36-0700
jobid = ed15f016-fd7e-48fb-b92c-0ea07302bffb
jobprocstatus = 0
jobresult:
errorcode = 530
errortext = Invalid private gateway id
jobresultcode = 530
jobresulttype = object
jobstatus = 2
userid = a9d755d8-a02e-11e4-b7f0-00505695fae7
Looking at the database, the problem becomes somewhat clear:
mysql select id,uuid,domain_id,state,removed,ip4_address from vpc_gateways;
++--+---+--+-+--+
| id | uuid | domain_id | state|
removed | ip4_address |
++--+---+--+-+--+
| 1 | 63a9f6f7-dbe2-4825-a3e2-17e3892c0ded | 1 | Ready|
NULL| 10.233.51.92 |
| 2 | 340b56bb-ee01-49be-84b4-9e54fd2af83d | 1 | Removing |
2015-04-20 21:28:53 | 10.233.5.11 |
++--+---+--+-+--+
2 rows in set (0.00 sec)
Unfortunately I don't have the logs from initially removing the VPC. I
can try and re-create this If needed, but I need to get this
environment fixed before I do anything else. Will setting the
state=Removed on the gateway entry in the database correct this?
Here are the logs from the management-server.log when trying to remove
the VPC via the UI:
2015-04-21 06:36:16,036 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-15:null) SeqA 9-315848: Processing Seq 9-315848:
{ Cmd , MgmtId: -1, via: 9, Ver: v1, Flags: 11,
[{com.cloud.agent.api.ConsoleProxyLoadReportCommand:{_proxyVmId:78,_loadInfo:{\n
\connections\: []\n},wait:0}}] }
2015-04-21 06:36:16,049 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-15:null) SeqA 9-315848: Sending Seq 9-315848: {
Ans: , MgmtId: 345050016868, via: 9, Ver: v1, Flags: 100010,
[{com.cloud.agent.api.AgentControlAnswer:{result:true,wait:0}}]
}
2015-04-21 06:36:16,853 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-2:null) Ping from 15
2015-04-21 06:36:16,877 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-4:null) Ping from 9
2015-04-21 06:36:20,715 DEBUG [c.c.a.ApiServlet]
(catalina-exec-6:ctx-62b2ea0a) ===START=== 10.233.51.176 -- GET
command=createPrivateGatewaysourcenatsupported=falseresponse=jsonsessionkey=sFXaEOKvIpqQArupbqZuPX6wYOo%3Dphysicalnetworkid=63a1c53c-b799-452a-ad15-3e0ef582c456vpcid=0fef5e73-b916-4505-8760-4321a3763bf6ipaddress=10.233.5.12gateway=10.233.5.1netmask=255.255.255.0vlan=4000aclid=9599b3fe-a02e-11e4-b7f0-00505695fae7_=1429623380716
2015-04-21 06:36:20,761 DEBUG [c.c.n.v.VpcManagerImpl]
(catalina-exec-6:ctx-62b2ea0a ctx-ebd9cd79) Creating Private gateway
for VPC [VPC [16-RSIDevNet]
2015-04-21 06:36:20,762 INFO [c.c.n.v.VpcManagerImpl]
(catalina-exec-6:ctx-62b2ea0a ctx-ebd9cd79) creating new network for
vpc [VPC [16-RSIDevNet] using broadcast uri: 4000
2015-04-21 06:36:20,774 DEBUG [c.c.n.NetworkServiceImpl]
(catalina-exec-6:ctx-62b2ea0a ctx-ebd9cd79) Private network already
exists: Ntwk[2a99ce65-36b0-432e-8fd8-f748a26aabdf|Guest|5]
2015-04-21 06:36:20,776 DEBUG [c.c.u.d.T.Transaction]
(catalina-exec-6:ctx-62b2ea0a ctx-ebd9cd79) Rolling back the
transaction: Time = 15 Name = catalina-exec-6; called by
-TransactionLegacy.rollback:900-TransactionLegacy.removeUpTo:843-TransactionLegacy.close:667-Transaction.execute:41-NetworkServiceImpl.createPrivateNetwork:3991-NativeMethodAccessorImpl.invoke0:-2-NativeMethodAccessorImpl.invoke:57-DelegatingMethodAccessorImpl.invoke:43-Method.invoke:606-AopUtils.invokeJoinpointUsingReflection:317-ReflectiveMethodInvocation.invokeJoinpoint:183-ReflectiveMethodInvocation.proceed:150
2015-04-21 06:36:20,790 INFO [c.c.a.ApiServer]
(catalina-exec-6:ctx-62b2ea0a ctx-ebd9cd79) Private network for the
vlan: vlan://4000 and cidr 10.233.5.0/24 already exists for Vpc 16
in zone AusZone1
2015-04-21 06:36:20,790 DEBUG [c.c.a.ApiServlet]
(catalina-exec-6:ctx-62b2ea0a ctx-ebd9cd79) ===END=== 10.233.51.176
-- GET
KVM CPU Overprovisioning issues.
I'm trying to figure out how CPU over-provisioning works in
cloudstack. It seems that no matter how much I change the
over-provisioning factor value in global settings, I cannot
over-provision CPU allocation on the hosts. I suspect that I'm just
having a lack of understanding on my part, but it seems like I should
be able to increase the number of VMs I can deploy by increasing the
CPU over-provisioning value (given that I'm not close to any other
resource limits). But this doesn't seem to be the case in practice.
https://screencloud.net/v/l1X7
https://screencloud.net/v/9ifx
Error message when trying to create additional machine via API.
grayson@gworkstation:~$ cloudmonkey deploy virtualmachine
zoneid=a4c20770-f17e-425e-bb21-413dd610b975
serviceofferingid=4b391121-49c7-4434-883d-ffe569f63a3b
templateid=e5701f73-b0c3-4dd7-8b2d-b8bf78875a85 keypair=key1
networkids=86e6ded0-6155-449b-a4bd-03411bf0a9cd name=tvmsman1
displayname
[1] 3907
Async job 09148c21-3c42-4334-93b2-428b48fb1fa2 failed
Error 533, Unable to create a deployment for VM[User|i-15-1151-VM]
accountid = a4e6eaed-9b06-4806-98f9-4fcc781b773f
cmd = org.apache.cloudstack.api.command.user.vm.DeployVMCmd
created = 2015-04-17T13:46:22-0700
jobid = 09148c21-3c42-4334-93b2-428b48fb1fa2
jobprocstatus = 0
jobresult:
errorcode = 533
errortext = Unable to create a deployment for VM[User|i-15-1151-VM]
jobresultcode = 530
jobresulttype = object
jobstatus = 2
userid = c7a0889e-fcec-4095-9d07-d29ea3c1e5d9
(local) mycloudmonkey list hosts
filter=name,clustername,cpuallocated,cpunumber,cpusockets,cpuspeed,cpuused,cpuwithoverprovisioning,
name = v-78-VM
name = s-871-VM
name = cl2hyper2
clustername = CL2
cpuallocated = 100.3%
cpunumber = 8
cpusockets = 2
cpuspeed = 1994
cpuused = 1.2%
cpuwithoverprovisioning = 63808.0
name = v-1-VM
name = cl2hyper1
clustername = CL2
cpuallocated = 81.45%
cpunumber = 8
cpusockets = 2
cpuspeed = 1995
cpuused = 0.6%
cpuwithoverprovisioning = 63840.0
2015-04-17 13:46:22,517 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(catalina-exec-25:ctx-7b0eb43a ctx-751b2eca ctx-5c8a75fc) submit async
job-7312, details: AsyncJobVO {id:7312, userId: 15, accountId: 15,
instanceType: VirtualMachine, instanceId: 1151, cmd:
org.apache.cloudstack.api.command.user.vm.DeployVMCmd, cmdInfo:
{serviceofferingid:4b391121-49c7-4434-883d-ffe569f63a3b,cmdEventType:VM.CREATE,ctxUserId:15,zoneid:a4c20770-f17e-425e-bb21-413dd610b975,signatureversion:3,httpmethod:GET,templateid:e5701f73-b0c3-4dd7-8b2d-b8bf78875a85,networkids:86e6ded0-6155-449b-a4bd-03411bf0a9cd,apiKey:wrxDdS8yi2kJPe8UrNdOdLtp55mCSKskk3MyTJ1ucIApdMoYnOWsd-dmigPywD1KC-6BVfVmKiJVenEM_kovpw,id:1151,response:json,ctxDetails:{\com.cloud.template.VirtualMachineTemplate\:\e5701f73-b0c3-4dd7-8b2d-b8bf78875a85\,\com.cloud.vm.VirtualMachine\:\6b3ace85-8e4c-4467-afdf-b511136ca6b1\,\com.cloud.network.Network\:\86e6ded0-6155-449b-a4bd-03411bf0a9cd\,\com.cloud.dc.DataCenter\:\a4c20770-f17e-425e-bb21-413dd610b975\,\com.cloud.offering.ServiceOffering\:\4b391121-49c7-4434-883d-ffe569f63a3b\},expires:2015-04-17T20:56:22+,keypair:key1,name:tvmsman1,uuid:6b3ace85-8e4c-4467-afdf-b511136ca6b1,ctxAccountId:15,ctxStartEventId:11922,signature:8Ebg1+qjfnk5nv7xz1GtdSju2ws\u003d},
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
result: null, initMsid: 345050015385, completeMsid: null, lastUpdated:
null, lastPolled: null, created: null}
2015-04-17 13:46:22,518 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(API-Job-Executor-16:ctx-e039eccb job-7312) Add job-7312 into job
monitoring
2015-04-17 13:46:22,518 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-16:ctx-e039eccb job-7312) Executing AsyncJobVO
{id:7312, userId: 15, accountId: 15, instanceType: VirtualMachine,
instanceId: 1151, cmd:
org.apache.cloudstack.api.command.user.vm.DeployVMCmd, cmdInfo:
Re: Unable to remove or modify VPC
Sanjeev,
I have opened a bug: https://issues.apache.org/jira/browse/CLOUDSTACK-8371
Thanks.
On Tue, Apr 7, 2015 at 4:01 AM, Sanjeev N sanj...@apache.org wrote:
Looks like a bug to me. Please open a bug in https://issues.apache.org/
On Tue, Apr 7, 2015 at 1:57 AM, Grayson Head gray...@graysonhead.net
wrote:
One of my coworkers has created a VPC that we are unable to modify or
remove.
The coworker said he was testing the site-to-site VPN functionality,
and ended up having to create a seperate VPC instance to get it
working. Now we are unable to remove this VPC instance using the web
interface or the API. I suspect based on the log outputs that the
site-to-site VPN is what is causing this problem, but the VPC shows
that it doesn't have any site-to-site connections configured.
The VPC still has one external IP address allocated to it, which I
think was the one he was using as the VPN gateway. I am also unable to
remove this IP address, it fails with an internal server error. The
following is the management-server.log when attempting to release this
IP address.
2015-04-06 13:18:42,427 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(API-Job-Executor-96:ctx-428fa98b job-6262) Add job-6262 into job
monitoring
2015-04-06 13:18:42,427 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-96:ctx-428fa98b job-6262) Executing AsyncJobVO
{id:6262, userId: 3, accountId: 3, instanceType: IpAddress,
instanceId: 9, cmd:
org.apache.cloudstack.api.command.user.address.DisassociateIPAddrCmd,
cmdInfo:
{response:json,id:3d2205a4-41e9-4cf7-9261-d1ae63ce863f,sessionkey:gSf69NNJJZ0SzrIEz1fwOQJpRTo\u003d,ctxDetails:{\com.cloud.network.IpAddress\:\3d2205a4-41e9-4cf7-9261-d1ae63ce863f\},cmdEventType:NET.IPRELEASE,ctxUserId:3,httpmethod:GET,_:1428351522388,uuid:3d2205a4-41e9-4cf7-9261-d1ae63ce863f,ctxAccountId:3,ctxStartEventId:7705},
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
result: null, initMsid: 345050016868, completeMsid: null, lastUpdated:
null, lastPolled: null, created: null}
2015-04-06 13:18:42,427 DEBUG [c.c.a.ApiServlet]
(catalina-exec-14:ctx-91e999f7 ctx-49efc96a) ===END=== 10.233.51.176
-- GET
command=disassociateIpAddressresponse=jsonsessionkey=gSf69NNJJZ0SzrIEz1fwOQJpRTo%3Did=3d2205a4-41e9-4cf7-9261-d1ae63ce863f_=1428351522388
2015-04-06 13:18:42,451 DEBUG [c.c.n.IpAddressManagerImpl]
(API-Job-Executor-96:ctx-428fa98b job-6262 ctx-bbf8738d) Revoking all
Firewallrules as a part of public IP id=9 release...
2015-04-06 13:18:42,462 DEBUG [c.c.n.f.FirewallManagerImpl]
(API-Job-Executor-96:ctx-428fa98b job-6262 ctx-bbf8738d) Releasing 0
firewall rules for ip id=9
2015-04-06 13:18:42,464 DEBUG [c.c.n.f.FirewallManagerImpl]
(API-Job-Executor-96:ctx-428fa98b job-6262 ctx-bbf8738d) There are no
firewall rules to apply
2015-04-06 13:18:42,466 DEBUG [c.c.n.f.FirewallManagerImpl]
(API-Job-Executor-96:ctx-428fa98b job-6262 ctx-bbf8738d) Successfully
released firewall rules for ip id=9 and # of rules now = 0
2015-04-06 13:18:42,475 DEBUG [c.c.n.IpAddressManagerImpl]
(API-Job-Executor-96:ctx-428fa98b job-6262 ctx-bbf8738d) Revoking all
PortForwarding/StaticNat rules as a part of public IP id=9 release...
2015-04-06 13:18:42,477 DEBUG [c.c.n.r.RulesManagerImpl]
(API-Job-Executor-96:ctx-428fa98b job-6262 ctx-bbf8738d) Releasing 0
port forwarding rules for ip id=9
2015-04-06 13:18:42,478 DEBUG [c.c.n.r.RulesManagerImpl]
(API-Job-Executor-96:ctx-428fa98b job-6262 ctx-bbf8738d) Releasing 0
static nat rules for ip id=9
2015-04-06 13:18:42,480 DEBUG [c.c.n.r.RulesManagerImpl]
(API-Job-Executor-96:ctx-428fa98b job-6262 ctx-bbf8738d) There are no
port forwarding rules to apply for ip id=9
2015-04-06 13:18:42,482 DEBUG [c.c.n.r.RulesManagerImpl]
(API-Job-Executor-96:ctx-428fa98b job-6262 ctx-bbf8738d) There are no
static nat rules to apply for ip id=9
2015-04-06 13:18:42,483 DEBUG [c.c.n.r.RulesManagerImpl]
(API-Job-Executor-96:ctx-428fa98b job-6262 ctx-bbf8738d) Source ip
id=Ip[66.194.167.148-1] is not one to one nat
2015-04-06 13:18:42,487 DEBUG [c.c.n.r.RulesManagerImpl]
(API-Job-Executor-96:ctx-428fa98b job-6262 ctx-bbf8738d) Successfully
released rules for ip id=9 and # of rules now = 0
2015-04-06 13:18:42,487 DEBUG [c.c.n.IpAddressManagerImpl]
(API-Job-Executor-96:ctx-428fa98b job-6262 ctx-bbf8738d) Revoking all
LoadBalancing rules as a part of public IP id=9 release...
2015-04-06 13:18:42,488 DEBUG [c.c.n.l.LoadBalancingRulesManagerImpl]
(API-Job-Executor-96:ctx-428fa98b job-6262 ctx-bbf8738d) Found 0 lb
rules to cleanup
2015-04-06 13:18:42,488 DEBUG [c.c.n.IpAddressManagerImpl]
(API-Job-Executor-96:ctx-428fa98b job-6262 ctx-bbf8738d) Cleaning up
remote access vpns as a part of public IP id=9 release...
2015-04-06 13:18:42,523 ERROR [c.c.a.ApiAsyncJobDispatcher]
(API-Job-Executor-96:ctx-428fa98b job-6262) Unexpected exception while
executing
org.apache.cloudstack.api.command.user.address.DisassociateIPAddrCmd
Unable to remove or modify VPC
One of my coworkers has created a VPC that we are unable to modify or remove.
The coworker said he was testing the site-to-site VPN functionality,
and ended up having to create a seperate VPC instance to get it
working. Now we are unable to remove this VPC instance using the web
interface or the API. I suspect based on the log outputs that the
site-to-site VPN is what is causing this problem, but the VPC shows
that it doesn't have any site-to-site connections configured.
The VPC still has one external IP address allocated to it, which I
think was the one he was using as the VPN gateway. I am also unable to
remove this IP address, it fails with an internal server error. The
following is the management-server.log when attempting to release this
IP address.
2015-04-06 13:18:42,427 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(API-Job-Executor-96:ctx-428fa98b job-6262) Add job-6262 into job
monitoring
2015-04-06 13:18:42,427 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-96:ctx-428fa98b job-6262) Executing AsyncJobVO
{id:6262, userId: 3, accountId: 3, instanceType: IpAddress,
instanceId: 9, cmd:
org.apache.cloudstack.api.command.user.address.DisassociateIPAddrCmd,
cmdInfo:
{response:json,id:3d2205a4-41e9-4cf7-9261-d1ae63ce863f,sessionkey:gSf69NNJJZ0SzrIEz1fwOQJpRTo\u003d,ctxDetails:{\com.cloud.network.IpAddress\:\3d2205a4-41e9-4cf7-9261-d1ae63ce863f\},cmdEventType:NET.IPRELEASE,ctxUserId:3,httpmethod:GET,_:1428351522388,uuid:3d2205a4-41e9-4cf7-9261-d1ae63ce863f,ctxAccountId:3,ctxStartEventId:7705},
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
result: null, initMsid: 345050016868, completeMsid: null, lastUpdated:
null, lastPolled: null, created: null}
2015-04-06 13:18:42,427 DEBUG [c.c.a.ApiServlet]
(catalina-exec-14:ctx-91e999f7 ctx-49efc96a) ===END=== 10.233.51.176
-- GET
command=disassociateIpAddressresponse=jsonsessionkey=gSf69NNJJZ0SzrIEz1fwOQJpRTo%3Did=3d2205a4-41e9-4cf7-9261-d1ae63ce863f_=1428351522388
2015-04-06 13:18:42,451 DEBUG [c.c.n.IpAddressManagerImpl]
(API-Job-Executor-96:ctx-428fa98b job-6262 ctx-bbf8738d) Revoking all
Firewallrules as a part of public IP id=9 release...
2015-04-06 13:18:42,462 DEBUG [c.c.n.f.FirewallManagerImpl]
(API-Job-Executor-96:ctx-428fa98b job-6262 ctx-bbf8738d) Releasing 0
firewall rules for ip id=9
2015-04-06 13:18:42,464 DEBUG [c.c.n.f.FirewallManagerImpl]
(API-Job-Executor-96:ctx-428fa98b job-6262 ctx-bbf8738d) There are no
firewall rules to apply
2015-04-06 13:18:42,466 DEBUG [c.c.n.f.FirewallManagerImpl]
(API-Job-Executor-96:ctx-428fa98b job-6262 ctx-bbf8738d) Successfully
released firewall rules for ip id=9 and # of rules now = 0
2015-04-06 13:18:42,475 DEBUG [c.c.n.IpAddressManagerImpl]
(API-Job-Executor-96:ctx-428fa98b job-6262 ctx-bbf8738d) Revoking all
PortForwarding/StaticNat rules as a part of public IP id=9 release...
2015-04-06 13:18:42,477 DEBUG [c.c.n.r.RulesManagerImpl]
(API-Job-Executor-96:ctx-428fa98b job-6262 ctx-bbf8738d) Releasing 0
port forwarding rules for ip id=9
2015-04-06 13:18:42,478 DEBUG [c.c.n.r.RulesManagerImpl]
(API-Job-Executor-96:ctx-428fa98b job-6262 ctx-bbf8738d) Releasing 0
static nat rules for ip id=9
2015-04-06 13:18:42,480 DEBUG [c.c.n.r.RulesManagerImpl]
(API-Job-Executor-96:ctx-428fa98b job-6262 ctx-bbf8738d) There are no
port forwarding rules to apply for ip id=9
2015-04-06 13:18:42,482 DEBUG [c.c.n.r.RulesManagerImpl]
(API-Job-Executor-96:ctx-428fa98b job-6262 ctx-bbf8738d) There are no
static nat rules to apply for ip id=9
2015-04-06 13:18:42,483 DEBUG [c.c.n.r.RulesManagerImpl]
(API-Job-Executor-96:ctx-428fa98b job-6262 ctx-bbf8738d) Source ip
id=Ip[66.194.167.148-1] is not one to one nat
2015-04-06 13:18:42,487 DEBUG [c.c.n.r.RulesManagerImpl]
(API-Job-Executor-96:ctx-428fa98b job-6262 ctx-bbf8738d) Successfully
released rules for ip id=9 and # of rules now = 0
2015-04-06 13:18:42,487 DEBUG [c.c.n.IpAddressManagerImpl]
(API-Job-Executor-96:ctx-428fa98b job-6262 ctx-bbf8738d) Revoking all
LoadBalancing rules as a part of public IP id=9 release...
2015-04-06 13:18:42,488 DEBUG [c.c.n.l.LoadBalancingRulesManagerImpl]
(API-Job-Executor-96:ctx-428fa98b job-6262 ctx-bbf8738d) Found 0 lb
rules to cleanup
2015-04-06 13:18:42,488 DEBUG [c.c.n.IpAddressManagerImpl]
(API-Job-Executor-96:ctx-428fa98b job-6262 ctx-bbf8738d) Cleaning up
remote access vpns as a part of public IP id=9 release...
2015-04-06 13:18:42,523 ERROR [c.c.a.ApiAsyncJobDispatcher]
(API-Job-Executor-96:ctx-428fa98b job-6262) Unexpected exception while
executing org.apache.cloudstack.api.command.user.address.DisassociateIPAddrCmd
java.lang.NullPointerException
at
com.cloud.network.NetworkModelImpl.getPhysicalNetworkId(NetworkModelImpl.java:1374)
at
com.cloud.network.element.VirtualRouterElement.canHandle(VirtualRouterElement.java:152)
at
com.cloud.network.element.VirtualRouterElement.stopVpn(VirtualRouterElement.java:443)
at
