xuanyuanaosheng opened a new issue, #7930:
URL: https://github.com/apache/cloudstack/issues/7930
##### ISSUE TYPE
* Config / Usage problem
##### COMPONENT NAME
* Virtual routers
##### CLOUDSTACK VERSION
CloudStack 4.18.0.0
##### OS / ENVIRONMENT
OS: oracle linux 8
##### CONFIGURATION
- Advanced networking, using **VXLAN**
(http:docs.cloudstack.apache.org/en/latest/plugins/vxlan.html?highlight=vxlan),
having three blades.
1. _**kvm001**_ on blade enclosures BL12
2. _**kvm002**_ on blade enclosures BL14
3. _**kvm003**_ on blade enclosures BL14
Our env is
- The Guest Isolation method is using `VXLAN and the VLAN/VNI: 2800-2900`
- The Public Network: VLAN: `2227 (10.71.227.0/24)`
- The management Network: `VLAN: 2128`
- The blade enclosures have enabled the following `VLANs: 2128, 2230, 2227`
- The switch has enabled the following `VLANs: 2128, 2230, 2227`
- The hosts network config:
```
|---------------- cloudbr0: 10.26.128.22 ( VLAN 2128)
kvm001 ------
|---------------- cloudbr1: 10.71.231.42 ( VLAN 2230)
|---------------- cloudbr0: 10.26.128.23 ( VLAN 2128)
kvm002 ------
|---------------- cloudbr1: 10.71.231.43 ( VLAN 2230)
|---------------- cloudbr0: 10.26.128.25 ( VLAN 2128)
kvm003 ------
|---------------- cloudbr1: 10.71.231.41 ( VLAN 2230)
```
The hosts can ping each other using cloudbr1.
The hosts can ping each other using cloudbr0.
The host network config, using kvm003 as en example:
```
# cat ifcfg-eno49
TYPE=Ethernet
BOOTPROTO=none
NAME=eno49
UUID=0650d63c-0244-4852-b0aa-ca5d8a64d8cb
DEVICE=eno49
ONBOOT=yes
# cat ifcfg-eno49.2128
NAME=eno49.2128
DEVICE=eno49.2128
ONBOOT=yes
HOTPLUG=no
BOOTPROTO=none
VLAN=yes
BRIDGE=cloudbr0
# cat ifcfg-cloudbr0
NAME=cloudbr0
DEVICE=cloudbr0
TYPE=Bridge
BOOTPROTO=none
ONBOOT=yes
IPADDR=10.26.128.25
GATEWAY=10.26.128.254
NETMASK=255.255.255.0
HOTPLUG=no
DELAY=5
STP=no
-------------------------------------------------------------------------------------
# cat ifcfg-eno50
TYPE=Ethernet
BOOTPROTO=none
NAME=eno50
UUID=46da1a8f-615e-4649-be64-fc8e1c7dd264
DEVICE=eno50
ONBOOT=yes
# cat ifcfg-eno50.2230
NAME=eno50.2230
DEVICE=eno50.2230
ONBOOT=yes
HOTPLUG=no
BOOTPROTO=none
VLAN=yes
BRIDGE=cloudbr1
# cat ifcfg-cloudbr1
NAME=cloudbr1
DEVICE=cloudbr1
TYPE=BRIDGE
ONBOOT=yes
BOOTPROTO=static
IPADDR=10.71.231.41
NETMASK=255.255.255.0
IPV6INIT=no
IPV6_AUTOCONF=no
HOTPLUG=no
DELAY=5
STP=no
```
The VPC is using `Default VPC offering`
The VR r-30996-VM (10.71.227.33) is on kvm002, and the Virtual routers
Health checks are all OK.
ubuntu221 (10.28.22.112) is on kvm001, set the mtu to 1400
ubuntu231 (10.28.22.19) is on kvm002, set the mtu to 1400
ubuntu251 (10.28.22.162) is on kvm003, set the mtu to 1400
The host VXLAN info using `vxlan2864` as an example:
```
# ip -d link show cloudbr0
7: cloudbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state
UP mode DEFAULT group default qlen 1000
link/ether 20:67:7c:19:67:70 brd ff:ff:ff:ff:ff:ff promiscuity 0 minmtu
68 maxmtu 65535
bridge forward_delay 1500 hello_time 200 max_age 2000 ageing_time 30000
stp_state 0 priority 32768 vlan_filtering 0 vlan_protocol 802.1Q bridge_id
8000.20:67:7c:19:67:70 designated_root 8000.20:67:7c:19:67:70 root_port 0
root_path_cost 0 topology_change 0 topology_change_detected 0 hello_timer
0.00 tcn_timer 0.00 topology_change_timer 0.00 gc_timer 65.71
vlan_default_pvid 1 vlan_stats_enabled 0 vlan_stats_per_port 0 group_fwd_mask 0
group_address 01:80:c2:00:00:00 mcast_snooping 1 mcast_router 1
mcast_query_use_ifaddr 0 mcast_querier 0 mcast_hash_elasticity 16
mcast_hash_max 4096 mcast_last_member_count 2 mcast_startup_query_count 2
mcast_last_member_interval 100 mcast_membership_interval 26000
mcast_querier_interval 25500 mcast_query_interval 12500
mcast_query_response_interval 1000 mcast_startup_query_interval 3125
mcast_stats_enabled 0 mcast_igmp_version 2 mcast_mld_version 1 nf_call_iptables
0 nf_call_ip6tables 0 nf_call_arptables 0 addrgenmode eui64 numtxqueues
1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
# ip -d link show cloudbr1
6: cloudbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state
UP mode DEFAULT group default qlen 1000
link/ether 20:67:7c:19:67:78 brd ff:ff:ff:ff:ff:ff promiscuity 0 minmtu
68 maxmtu 65535
bridge forward_delay 1500 hello_time 200 max_age 2000 ageing_time 30000
stp_state 0 priority 32768 vlan_filtering 0 vlan_protocol 802.1Q bridge_id
8000.20:67:7c:19:67:78 designated_root 8000.20:67:7c:19:67:78 root_port 0
root_path_cost 0 topology_change 0 topology_change_detected 0 hello_timer
0.00 tcn_timer 0.00 topology_change_timer 0.00 gc_timer 60.14
vlan_default_pvid 1 vlan_stats_enabled 0 vlan_stats_per_port 0 group_fwd_mask 0
group_address 01:80:c2:00:00:00 mcast_snooping 1 mcast_router 1
mcast_query_use_ifaddr 0 mcast_querier 0 mcast_hash_elasticity 16
mcast_hash_max 4096 mcast_last_member_count 2 mcast_startup_query_count 2
mcast_last_member_interval 100 mcast_membership_interval 26000
mcast_querier_interval 25500 mcast_query_interval 12500
mcast_query_response_interval 1000 mcast_startup_query_interval 3125
mcast_stats_enabled 0 mcast_igmp_version 2 mcast_mld_version 1 nf_call_iptables
0 nf_call_ip6tables 0 nf_call_arptables 0 addrgenmode eui64 numtxqueues
1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
# ip -d link show vxlan2864
76: vxlan2864: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue
master brvx-2864 state UNKNOWN mode DEFAULT group default qlen 1000
link/ether f6:01:07:83:28:a9 brd ff:ff:ff:ff:ff:ff promiscuity 1 minmtu
68 maxmtu 65535
vxlan id 2864 group 239.0.11.48 dev cloudbr1 srcport 0 0 dstport 8472
ttl 10 ageing 300 udpcsum noudp6zerocsumtx noudp6zerocsumrx
bridge_slave state forwarding priority 32 cost 100 hairpin off guard off
root_block off fastleave off learning on flood on port_id 0x8001 port_no 0x1
designated_port 32769 designated_cost 0 designated_bridge 8000.f6:1:7:83:28:a9
designated_root 8000.f6:1:7:83:28:a9 hold_timer 0.00 message_age_timer
0.00 forward_delay_timer 0.00 topology_change_ack 0 config_pending 0
proxy_arp off proxy_arp_wifi off mcast_router 1 mcast_fast_leave off
mcast_flood on bcast_flood on mcast_to_unicast off neigh_suppress off
group_fwd_mask 0 group_fwd_mask_str 0x0 vlan_tunnel off isolated off
addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs
65535
# ip -d link show brvx-2864
77: brvx-2864: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue
state UP mode DEFAULT group default qlen 1000
link/ether f6:01:07:83:28:a9 brd ff:ff:ff:ff:ff:ff promiscuity 0 minmtu
68 maxmtu 65535
bridge forward_delay 1500 hello_time 200 max_age 2000 ageing_time 30000
stp_state 0 priority 32768 vlan_filtering 0 vlan_protocol 802.1Q bridge_id
8000.f6:1:7:83:28:a9 designated_root 8000.f6:1:7:83:28:a9 root_port 0
root_path_cost 0 topology_change 0 topology_change_detected 0 hello_timer
0.00 tcn_timer 0.00 topology_change_timer 0.00 gc_timer 200.93
vlan_default_pvid 1 vlan_stats_enabled 0 vlan_stats_per_port 0 group_fwd_mask 0
group_address 01:80:c2:00:00:00 mcast_snooping 1 mcast_router 1
mcast_query_use_ifaddr 0 mcast_querier 0 mcast_hash_elasticity 16
mcast_hash_max 4096 mcast_last_member_count 2 mcast_startup_query_count 2
mcast_last_member_interval 100 mcast_membership_interval 26000
mcast_querier_interval 25500 mcast_query_interval 12500
mcast_query_response_interval 1000 mcast_startup_query_interval 3125
mcast_stats_enabled 0 mcast_igmp_version 2 mcast_mld_version 1 nf_call_iptables
0 nf_call_ip6tables 0 nf_call_arptables 0 addrgenmode eui64 numtxqueues 1 n
umrxqueues 1 gso_max_size 65536 gso_max_segs 65535
```
Now the vm on one blade enclosures can ping each other, But the vm on
different blade enclosures can not ping each other.
I do not know why this happen, Could you please take a look and give some
advices?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
