dpthakar opened a new issue, #12043:
URL: https://github.com/apache/cloudstack/issues/12043
Hello,
**Problem:-**
SSVM Fails to start.
**Versions:-**
Cloudstack agent 4.22
Cloudstack Management 4.22
ubuntu 24.04
**Error:-**
```
==> /var/log/cloudstack/agent/agent.log <==
2025-11-11 16:07:12,873 INFO [kvm.storage.LibvirtStorageAdaptor]
(AgentRequest-Handler-1:[]) (logid:6786196e) Trying to fetch storage pool
3b4573e2-11b4-3b02-81b3-a7d8f8c97101 from libvirt
2025-11-11 16:07:12,942 WARN [kvm.resource.BridgeVifDriver]
(AgentRequest-Handler-1:[]) (logid:6786196e) Execution of process [41853] for
command [/usr/share/cloudstack-common/scripts/vm/network/vnet/modifyvlan.sh -v
18 -p bond1 -b brbond1-18 -o add ] failed.
2025-11-11 16:07:12,943 WARN [kvm.resource.BridgeVifDriver]
(AgentRequest-Handler-1:[]) (logid:6786196e) Process [41853] for command
[/usr/share/cloudstack-common/scripts/vm/network/vnet/modifyvlan.sh -v 18 -p
bond1 -b brbond1-18 -o add ] encountered the error: [Error: 8021q: VLAN device
already
exists./usr/share/cloudstack-common/scripts/vm/network/vnet/modifyvlan.sh: line
37: /proc/sys/net/ipv6/conf/bond1.18/disable_ipv6: No such file or
directoryCannot find device "bond1.18"Failed to create vlan 18 on pif: bond1.].
2025-11-11 16:07:12,943 WARN [resource.wrapper.LibvirtStartCommandWrapper]
(AgentRequest-Handler-1:[]) (logid:6786196e) InternalErrorException
com.cloud.exception.InternalErrorException: Failed to create vnet 18: Error:
8021q: VLAN device already
exists./usr/share/cloudstack-common/scripts/vm/network/vnet/modifyvlan.sh: line
37: /proc/sys/net/ipv6/conf/bond1.18/disable_ipv6: No such file or
directoryCannot find device "bond1.18"Failed to create vlan 18 on pif: bond1.
at
com.cloud.hypervisor.kvm.resource.BridgeVifDriver.createVnet(BridgeVifDriver.java:338)
at
com.cloud.hypervisor.kvm.resource.BridgeVifDriver.createVnetBr(BridgeVifDriver.java:320)
at
com.cloud.hypervisor.kvm.resource.BridgeVifDriver.createStorageVnetBridgeIfNeeded(BridgeVifDriver.java:203)
at
com.cloud.hypervisor.kvm.resource.BridgeVifDriver.plug(BridgeVifDriver.java:272)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.createVif(LibvirtComputingResource.java:3861)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.createVifs(LibvirtComputingResource.java:3438)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStartCommandWrapper.execute(LibvirtStartCommandWrapper.java:86)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStartCommandWrapper.execute(LibvirtStartCommandWrapper.java:52)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:2280)
at com.cloud.agent.Agent.processRequest(Agent.java:813)
at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1295)
at com.cloud.utils.nio.Task.call(Task.java:83)
at com.cloud.utils.nio.Task.call(Task.java:29)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
```
Earlier SSVM was working fine on CS Agent 4.20, after upgrading CS Agent to
4.22 AND CS Mgmt to 4,22 we are getting above mentioned error.
_Below is current network config._
```
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group
default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host noprefixroute
valid_lft forever preferred_lft forever
2: ens3f0np0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group
default qlen 1000
link/ether 7c:c2:55:42:8d:f4 brd ff:ff:ff:ff:ff:ff
altname enp99s0f0np0
3: ens3f1np1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group
default qlen 1000
link/ether 7c:c2:55:42:8d:f5 brd ff:ff:ff:ff:ff:ff
altname enp99s0f1np1
4: ens2f0np0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9214 qdisc mq
master bond0 state UP group default qlen 1000
link/ether 0a:b7:ba:ab:5b:13 brd ff:ff:ff:ff:ff:ff permaddr
7c:c2:55:42:8b:54
altname enp68s0f0np0
5: ens2f1np1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 9214 qdisc mq state
DOWN group default qlen 1000
link/ether 7c:c2:55:42:8b:55 brd ff:ff:ff:ff:ff:ff
altname enp68s0f1np1
6: enp3s0f0np0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group
default qlen 1000
link/ether 7c:c2:55:42:8e:34 brd ff:ff:ff:ff:ff:ff
7: enp3s0f1np1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group
default qlen 1000
link/ether 7c:c2:55:42:8e:35 brd ff:ff:ff:ff:ff:ff
8: enp36s0f0np0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9214 qdisc mq
master bond0 state UP group default qlen 1000
link/ether 0a:b7:ba:ab:5b:13 brd ff:ff:ff:ff:ff:ff permaddr
7c:c2:55:42:85:44
9: enp36s0f1np1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9214 qdisc mq
master bond1 state UP group default qlen 1000
link/ether ca:9f:8f:ec:8e:1f brd ff:ff:ff:ff:ff:ff permaddr
7c:c2:55:42:85:45
10: enp227s0f0np0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
group default qlen 1000
link/ether 7c:c2:55:42:85:24 brd ff:ff:ff:ff:ff:ff
11: enp51s0f0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state
DOWN group default qlen 1000
link/ether 7c:c2:55:7b:58:d6 brd ff:ff:ff:ff:ff:ff
12: enp227s0f1np1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9214 qdisc mq
master bond1 state UP group default qlen 1000
link/ether ca:9f:8f:ec:8e:1f brd ff:ff:ff:ff:ff:ff permaddr
7c:c2:55:42:85:25
13: enp196s0f0np0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9214 qdisc mq
master bond0 state UP group default qlen 1000
link/ether 0a:b7:ba:ab:5b:13 brd ff:ff:ff:ff:ff:ff permaddr
7c:c2:55:42:84:84
14: enp196s0f1np1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 9214 qdisc mq
state DOWN group default qlen 1000
link/ether 7c:c2:55:42:84:85 brd ff:ff:ff:ff:ff:ff
15: enp131s0f0np0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
group default qlen 1000
link/ether 7c:c2:55:42:85:34 brd ff:ff:ff:ff:ff:ff
16: enp131s0f1np1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
group default qlen 1000
link/ether 7c:c2:55:42:85:35 brd ff:ff:ff:ff:ff:ff
17: enp164s0f0np0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9214 qdisc mq
master bond0 state UP group default qlen 1000
link/ether 0a:b7:ba:ab:5b:13 brd ff:ff:ff:ff:ff:ff permaddr
7c:c2:55:42:85:54
18: enp164s0f1np1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 9214 qdisc mq
state DOWN group default qlen 1000
link/ether 7c:c2:55:42:85:55 brd ff:ff:ff:ff:ff:ff
19: enp51s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master
hostbr0 state UP group default qlen 1000
link/ether 7c:c2:55:7b:58:d7 brd ff:ff:ff:ff:ff:ff
20: enxbe3af2b6059f: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
group default qlen 1000
link/ether be:3a:f2:b6:05:9f brd ff:ff:ff:ff:ff:ff
21: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 9214 qdisc noqueue
master cloudbr0 state UP group default qlen 1000
link/ether 0a:b7:ba:ab:5b:13 brd ff:ff:ff:ff:ff:ff
22: bond1: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 9214 qdisc noqueue
state UP group default qlen 1000
link/ether ca:9f:8f:ec:8e:1f brd ff:ff:ff:ff:ff:ff
inet6 fe80::c89f:8fff:feec:8e1f/64 scope link
valid_lft forever preferred_lft forever
23: cloudbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9214 qdisc noqueue state
UP group default qlen 1000
link/ether 76:55:ab:fd:45:64 brd ff:ff:ff:ff:ff:ff
inet6 fe80::7455:abff:fefd:4564/64 scope link
valid_lft forever preferred_lft forever
24: csmgmtbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9214 qdisc noqueue
state UP group default qlen 1000
link/ether da:30:cd:f0:20:0a brd ff:ff:ff:ff:ff:ff
inet 192.168.10.131/23 metric 100 brd 192.168.11.255 scope global
dynamic csmgmtbr0
valid_lft 7773596sec preferred_lft 7773596sec
inet6 fe80::d830:cdff:fef0:200a/64 scope link
valid_lft forever preferred_lft forever
25: hostbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state
UP group default qlen 1000
link/ether fe:e0:4f:7c:cb:40 brd ff:ff:ff:ff:ff:ff
inet 192.168.14.116/23 metric 100 brd 192.168.15.255 scope global
dynamic hostbr0
valid_lft 7773572sec preferred_lft 7773572sec
inet6 fe80::fce0:4fff:fe7c:cb40/64 scope link
valid_lft forever preferred_lft forever
26: strbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9214 qdisc noqueue state
UP group default qlen 1000
link/ether 7e:44:09:e3:06:76 brd ff:ff:ff:ff:ff:ff
inet 192.168.18.105/24 brd 192.168.18.255 scope global strbr0
valid_lft forever preferred_lft forever
inet6 fe80::7c44:9ff:fee3:676/64 scope link
valid_lft forever preferred_lft forever
27: vlan18@bond1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9214 qdisc noqueue
master strbr0 state UP group default qlen 1000
link/ether ca:9f:8f:ec:8e:1f brd ff:ff:ff:ff:ff:ff
28: vlan10@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9214 qdisc noqueue
master csmgmtbr0 state UP group default qlen 1000
link/ether 0a:b7:ba:ab:5b:13 brd ff:ff:ff:ff:ff:ff
29: cloud0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state
UP group default qlen 1000
link/ether 4a:3b:6e:d2:39:6d brd ff:ff:ff:ff:ff:ff
inet 169.254.0.1/16 scope global cloud0
valid_lft forever preferred_lft forever
inet6 fe80::483b:6eff:fed2:396d/64 scope link
valid_lft forever preferred_lft forever
42: bond0.4090@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9214 qdisc
noqueue master brbond0-4090 state UP group default qlen 1000
link/ether 0a:b7:ba:ab:5b:13 brd ff:ff:ff:ff:ff:ff
43: brbond0-4090: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9214 qdisc noqueue
state UP group default qlen 1000
link/ether c2:88:40:83:d0:27 brd ff:ff:ff:ff:ff:ff
44: vnet6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master
cloud0 state UNKNOWN group default qlen 1000
link/ether fe:00:a9:fe:3c:68 brd ff:ff:ff:ff:ff:ff
inet6 fe80::fc00:a9ff:fefe:3c68/64 scope link
valid_lft forever preferred_lft forever
45: vnet7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9214 qdisc noqueue master
csmgmtbr0 state UNKNOWN group default qlen 1000
link/ether fe:00:48:00:00:0b brd ff:ff:ff:ff:ff:ff
inet6 fe80::fc00:48ff:fe00:b/64 scope link
valid_lft forever preferred_lft forever
46: vnet8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9214 qdisc noqueue master
brbond0-4090 state UNKNOWN group default qlen 1000
link/ether fe:00:9f:00:00:0c brd ff:ff:ff:ff:ff:ff
inet6 fe80::fc00:9fff:fe00:c/64 scope link
valid_lft forever preferred_lft forever
```
To fix it, we tried following but did not help.
Reboot all hosts & mgmgt
Downgrade CS Agent to 4.20
Erase CS Agent package & re-add host
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
