Hi everybody
I have a problem with console proxy.
After a long time I restarted it because of upgrading SSL certificates.
Unfortunately console proxy virtual machine won't start anymore.
I can see this in the libvirt logs:
=============================
2014-09-30 21:25:28.903+0000: 13547: error : virCommandWait:2399 :
internal error: Child process (ovs-vsctl --timeout=5 -- --may-exist
add-port management vnet1 -- set Interface vnet1
'external-ids:attached-mac="06:59:5a:00:00:1f"' -- set Interface vnet1
'external-ids:iface-id="3eac9df5-8288-4ca5-b76a-545b1b96e4cf"' -- set
Interface vnet1
'external-ids:vm-id="5c97bbfd-2161-441c-a6a6-68a1ec78402c"' -- set
Interface vnet1 external-ids:iface-status=active) unexpected exit status
1: ovs-vsctl: no bridge named management
And this in agent log:
=============================
2014-09-30 23:11:16,779 WARN [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-5:null) LibvirtException
org.libvirt.LibvirtException: Unable to add port vnet1 to OVS bridge
management: Operation not permitted
at org.libvirt.ErrorHandler.processError(Unknown Source)
at org.libvirt.Connect.processError(Unknown Source)
at org.libvirt.Connect.processError(Unknown Source)
at org.libvirt.Connect.domainCreateXML(Unknown Source)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.startVM(LibvirtComputingResource.java:1213)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:3659)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1307)
at com.cloud.agent.Agent.processRequest(Agent.java:498)
at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:806)
at com.cloud.utils.nio.Task.run(Task.java:83)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2014-09-30 23:11:16,780 DEBUG [kvm.storage.KVMStoragePoolManager]
(agentRequest-Handler-5:null) Disconnecting disk
ab234ec9-d8cf-410a-b849-0ec3859c2a06
2014-09-30 23:11:17,769 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-5:null) Seq 3-1904869421: { Ans: , MgmtId:
186916991823178, via: 3, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.StartAnswer":{"vm":{"id":405,"name":"v-405-VM","type":"ConsoleProxy","cpus":1,"minSpeed":454,"maxSpeed":500,"minRam":1073741824,"maxRam":1073741824,"arch":"x86_64","os":"Debian
GNU/Linux 7(64-bit)","bootArgs":" template=domP type=consoleproxy
host=10.0.1.1 port=8250 name=v-405-VM premium=true zone=1 pod=1
guid=Proxy.405 proxy_vm=405 disable_rp_filter=true
eth2ip=XXX.YYY.248.144 eth2mask=255.255.255.0 gateway=XXX.YYY.248.254
eth0ip=169.254.1.24 eth0mask=255.255.0.0 eth1ip=10.0.1.33
eth1mask=255.255.255.0 mgmtcidr=10.0.1.0/24 localgw=10.0.1.1
internaldns1=10.0.1.1 dns1=XXX.YYY.248.1
dns2=8.8.8.8","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"c21eae05c175155f","vncAddr":"10.0.1.104","params":{"memoryOvercommitRatio":"1","cpuOvercommitRatio":"1.1"},"uuid":"5c97bbfd-2161-441c-a6a6-68a1ec78402c","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"ab234ec9-d8cf-410a-b849-0ec3859c2a06","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"0fe7281f-bfde-3260-964b-45a9d461006a","id":201,"poolType":"RBD","host":"ceph-int.xxxxxxxxxxxxx.com","path":"rbd","port":6789,"url":"RBD://ceph-int.xxxxxxxxxxxxx.com/rbd/?ROLE=Primary&STOREUUID=0fe7281f-bfde-3260-964b-45a9d461006a"}},"name":"ROOT-405","size":2621440000,"path":"ab234ec9-d8cf-410a-b849-0ec3859c2a06","volumeId":480,"vmName":"v-405-VM","accountId":1,"format":"RAW","id":480,"deviceId":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"ab234ec9-d8cf-410a-b849-0ec3859c2a06","type":"ROOT","_details":{"managed":"false","storagePort":"6789","storageHost":"ceph-int.xxxxxxxxxxxxx.com","volumeSize":"2621440000"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"uuid":"ab6fca92-0aba-49bc-8024-08eba4686d8d","ip":"XXX.YYY.248.144","netmask":"255.255.255.0","gateway":"XXX.YYY.248.254","mac":"06:2e:e4:00:00:20","dns1":"XXX.YYY.248.1","dns2":"8.8.8.8","broadcastType":"Vlan","type":"Public","broadcastUri":"vlan://untagged","isolationUri":"vlan://untagged","isSecurityGroupEnabled":false,"name":"cloudbr1"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"uuid":"1ec8fc52-a8ac-4c02-b389-7b6575ad4a0e","ip":"169.254.1.24","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:01:18","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"uuid":"86779b11-a440-4b2d-ab4b-21128b0acaa0","ip":"10.0.1.33","netmask":"255.255.255.0","gateway":"10.0.1.1","mac":"06:91:ee:00:00:0e","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"cloudbr0"}]},"result":false,"details":"Unable
to add port vnet1 to OVS bridge management: Operation not
permitted","wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Stopped
by previous failure","wait":0}}] }
Other data are:
=============================
ovs-vsctl show :
a2dcba6a-3aba-424a-9c8a-5c19f689ba69
Bridge "cloud0"
Port "vnet0"
Interface "vnet0"
Port "cloud0"
Interface "cloud0"
type: internal
Bridge "cloudbr0"
Port "cloudbr0"
Interface "cloudbr0"
type: internal
Port bondBE
Interface "eth1"
Interface "eth2"
Port management
Interface management
type: internal
Port storage
Interface storage
type: internal
Port "vnet1"
Interface "vnet1"
Bridge "cloudbr1"
Port bondFE
Interface "eth3"
Interface "eth0"
Port "vnet2"
Interface "vnet2"
Port "cloudbr1"
Interface "cloudbr1"
type: internal
ovs_version: "2.0.1"
=============================
agent.properties:
#Storage
#Tue Sep 30 23:43:36 CEST 2014
guest.network.device=cloudbr1
workers=5
private.network.device=management
network.bridge.type=openvswitch
port=8250
resource=com.cloud.hypervisor.kvm.resource.LibvirtComputingResource
pod=1
storage.network.device=storage
zone=1
guid=5fba1db3-d7b2-379b-816f-0e6d0be99594
public.network.device=cloudbr1
cluster=1
local.storage.uuid=0c3487b4-0a53-4302-af9a-07d640826cc1
domr.scripts.dir=scripts/network/domr/kvm
LibvirtComputingResource.id=3
host=10.0.1.1
=============================
I tried a workaround changing agent.properties with
<< private.network.device=management
private.network.device=cloudbr0
With this, console proxy starts on that host, but the host use the
storage network as management network (assign the same ip to storage and
management networks)
In addiction all VMs started on that host use storage network as
management network.
Practically changing that setting in agent.properties cannot be a
solution because of unwantend effects
Any idea about what happens and how to start console proxy correctly?
--
Gerolamo Valcamonica
