On 10/31/23 03:59, Liqi An wrote:
> Hi ,
> Ok , I will continue to consult SUSE support on this issue .
>
> About the new workaround, I hope bond1 work as before ,
>
> A> Old solution(bond1 in bridge: br-oam will lost when restarting network of
> host):
> cluster12-b:/etc/sysconfig/network # cat ifcfg-bond1
> DEVICE='bond1'
> BONDING_MASTER='yes'
> BONDING_MODULE_OPTS='mode=active-backup miimon=100 use_carrier=0'
> BONDING_SLAVE0='eth1'
> BONDING_SLAVE1='eth5'
> BOOTPROTO='static'
> BORADCAST=''
> ETHTOOL_OPTIONS=''
> IPADDR=''
> MTU=''
> NAME=''
> NETMASK=''
> NETWORK=''
> REMOTE_IPADDR=''
> STARTMODE='auto'
> USERCONTROL='no'
> BONDING_SKIP_REMOVE_WORKAROUND='yes'
> ZONE=public
>
> # /usr/bin/ovs-vsctl add-port br-oam bond1 trunk=3932,3933
>
> Bridge br-oam
> Port "2.11-SC-2-eth1"
> tag: 3932
> Interface "2.11-SC-2-eth1"
> Port bond1
> trunks: [3932, 3933]
> Interface bond1
> Port "2.11-SC-2-eth2"
> tag: 3933
> Interface "2.11-SC-2-eth2"
> Port br-oam
> Interface br-oam
> type: internal
>
>
>
>
> B> New solution:
> # ovs-vsctl add-bond br-oam bond1 eth1 eth5 trunk=3932,3933
>
> Bridge br-oam
> Port br-oam
> Interface br-oam
> type: internal
> Port bond1
> trunks: [3932, 3933]
> Interface eth5
> Interface eth1
> Port "2.11-SC-2-eth2"
> tag: 3933
> Interface "2.11-SC-2-eth2"
> Port "2.11-SC-2-eth1"
> tag: 3932
> Interface "2.11-SC-2-eth1"
>
> Would bond1 in solution B> work in the same way as bond1 in solution A>
> , Especially the work of eth1ð5
> as ' BONDING_MODULE_OPTS='mode=active-backup miimon=100 use_carrier=0''? Or
> more parameter configuration is required in
> command"# ovs-vsctl add-bond br-oam bond1 eth1 eth5 trunk=3932,3933" ?
By default the bonding mode is active-backup, but the default failure
detection mode is carrier, so you'll need to change that in order to
match the previous configuration. For example:
ovs-vsctl set port bond1 other_config:bond-detect-mode=miimon \
other_config:bond-miimon-interval=100
>
>
> //An
>
> -----Original Message-----
> From: Ilya Maximets <[email protected]>
> Sent: Monday, October 30, 2023 7:09 PM
> To: Liqi An <[email protected]>; [email protected]
> Cc: [email protected]; Cheng Chi <[email protected]>; Jonas Yi
> <[email protected]>; Yawei Lu <[email protected]>
> Subject: Re: [ovs-discuss] Restarting the network triggers the deletion of
> one ovs port
>
> On 10/30/23 10:28, Liqi An wrote:
>> Hi ,
>> Is there any update about this issue , It's been bothering me for
>> over two weeks,thx~
>
> Hi. As you saw in the log, something is calling ovs-vsctl to remove the port
> from OVS:
>
> 2023-10-16T13:07:17.668420+08:00 cluster12-b ovs-vsctl:
> ovs|00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl del-port br-oam bond1
>
> OVS is not doing that on its own. You need to find what is calling this
> command in order to fix the problem. Likely candidates are network-scripts,
> NetworkManager or something similar.
>
> Best regards, Ilya Maximets.
>
>>
>> //An
>>
>> -----Original Message-----
>> From: Liqi An
>> Sent: Wednesday, October 18, 2023 5:13 PM
>> To: Ilya Maximets <[email protected]>; [email protected]
>> Cc: Cheng Chi <[email protected]>; Jonas Yi
>> <[email protected]>; Yawei Lu <[email protected]>; IPW AQUA
>> team (CBC) <[email protected]>
>> Subject: RE: [ovs-discuss] Restarting the network triggers the
>> deletion of one ovs port
>>
>> Hi ,
>> We added bond1 to br-oam with below command before:
>>
>> # ovs-vsctl add-port br-oam bond1 trunk=3932,3933
>>
>> In my opinion , , once it add bond1 to the virtual switch
>> (openvswtich) successfully , this configuration should be saved into ovs db
>> , What I'm wondering is why the network management on the host actively
>> triggers the deletion, That's not supposed to be the job of SUSE15-SP5 , The
>> ovs-vsctl command is used to manually add or delete these configurations
>> before.
>>
>>
>> By the way , I tried another way , delete bond1 configuration of
>> previous host , and add eth1ð5 as bond1 to openvswitch directly:
>> WA:
>> cluster12-b:/etc/sysconfig/network # ip link set bond1 down
>> cluster12-b:/etc/sysconfig/network # rm -f
>> /etc/sysconfig/network/ifcfg-bond1
>> cluster12-b:/etc/sysconfig/network # service network restart
>> cluster12-b:/etc/sysconfig/network # ip link set eth1 up
>> cluster12-b:/etc/sysconfig/network # ip link set eth5 up
>>
>> cluster12-b:/etc/sysconfig/network # ovs-vsctl add-bond br-oam bond1 eth1
>> eth5 trunk=3932,3933 cluster12-b:/etc/sysconfig/network # ovs-vsctl show
>> 2e9bf291-50ac-4c3a-ac55-2d590df1880d
>> Bridge br-oam
>> Port br-oam
>> Interface br-oam
>> type: internal
>> Port bond1
>> trunks: [3932, 3933]
>> Interface eth1
>> Interface eth5
>> ovs_version: "2.14.2"
>> cluster12-b:/etc/sysconfig/network # service network restart
>> cluster12-b:/etc/sysconfig/network # ovs-vsctl show
>> 2e9bf291-50ac-4c3a-ac55-2d590df1880d
>> Bridge br-oam
>> Port br-oam
>> Interface br-oam
>> type: internal
>> Port bond1
>> trunks: [3932, 3933]
>> Interface eth1
>> Interface eth5
>> ovs_version: "2.14.2"
>>
>> After initial testing, this scheme works temporarily and does not lose the
>> relevant network configuration after restarting the network and host ,But I
>> not sure how eth1ð5 works in this configuration , I hope them work as
>> before:
>>
>> cluster12-b:~ # cat /etc/sysconfig/network/ifcfg-bond1
>> DEVICE='bond1'
>> BORADCAST=''
>> NETWORK=''
>> USERCONTROL='no'
>> BONDING_SKIP_REMOVE_WORKAROUND='yes'
>> BOOTPROTO='static'
>> STARTMODE='auto'
>> ZONE='public'
>> BONDING_MASTER='yes'
>> BONDING_SLAVE0='eth5'
>> BONDING_SLAVE1='eth1'
>> BONDING_MODULE_OPTS='mode=active-backup miimon=100 use_carrier=0'
>>
>> do you have any idea about this solution or give some advice, thx ~
>>
>> //An
>>
>> -----Original Message-----
>> From: Ilya Maximets <[email protected]>
>> Sent: Tuesday, October 17, 2023 7:34 PM
>> To: Liqi An <[email protected]>; [email protected]
>> Cc: [email protected]; Cheng Chi <[email protected]>; Jonas Yi
>> <[email protected]>; Yawei Lu <[email protected]>
>> Subject: Re: [ovs-discuss] Restarting the network triggers the
>> deletion of one ovs port
>>
>> On 10/17/23 07:53, Liqi An wrote:
>>> Hi experts ,
>>> I simplified the issue repetition steps ,
>>>
>>> cluster12-b: # cat ovs-network.xml
>>> <?xml version="1.0"?>
>>> <network>
>>> <name>2.11-ovs-network</name>
>>> <forward mode="bridge"/>
>>> <bridge name="br-oam"/>
>>> <virtualport type="openvswitch"/>
>>> <portgroup name="2.11-vlan-01" default="yes">
>>> </portgroup>
>>> <portgroup name="2.11-SC-eth1">
>>> <vlan>
>>> <tag id="3932"/>
>>> </vlan>
>>> </portgroup>
>>> <portgroup name="2.11-SC-eth2">
>>> <vlan>
>>> <tag id="3933"/>
>>> </vlan>
>>> </portgroup>
>>> <portgroup name="2.11-vlan-all">
>>> <vlan trunk="yes">
>>> <tag id="3932"/>
>>> <tag id="3933"/>
>>> </vlan>
>>> </portgroup>
>>> </network>
>>> cluster12-b: # virsh list --all
>>> Id Name State
>>> --------------------
>>>
>>> cluster12-b: # virsh net-list --all
>>> Name State Autostart Persistent
>>> ----------------------------------------
>>>
>>> cluster12-b: # virsh net-define ovs-network.xml Network
>>> 2.11-ovs-network defined from ovs-network.xml
>>>
>>> cluster12-b: # virsh net-list --all
>>> Name State Autostart Persistent
>>> -------------------------------------------------------
>>> 2.11-ovs-network inactive no yes
>>>
>>> cluster12-b: # virsh net-start 2.11-ovs-network Network
>>> 2.11-ovs-network started
>>>
>>> cluster12-b: # virsh net-list --all
>>> Name State Autostart Persistent
>>> -----------------------------------------------------
>>> 2.11-ovs-network active no yes
>>>
>>> cluster12-b: # ovs-vsctl show
>>> 2e9bf291-50ac-4c3a-ac55-2d590df1880d
>>> ovs_version: "2.14.2"
>>> cluster12-b: # ovs-vsctl add-br br-oam
>>> cluster12-b: # ovs-vsctl show
>>> 2e9bf291-50ac-4c3a-ac55-2d590df1880d
>>> Bridge br-oam
>>> Port br-oam
>>> Interface br-oam
>>> type: internal
>>> ovs_version: "2.14.2"
>>> cluster12-b: # ovs-vsctl add-port br-oam bond1 trunk=3932,3933
>>> cluster12-b: # ovs-vsctl show
>>> 2e9bf291-50ac-4c3a-ac55-2d590df1880d
>>> Bridge br-oam
>>> Port br-oam
>>> Interface br-oam
>>> type: internal
>>> Port bond1
>>> trunks: [3932, 3933]
>>> Interface bond1
>>> ovs_version: "2.14.2"
>>> cluster12-b: # date
>>> Tue Oct 17 13:47:02 CST 2023
>>> cluster12-b: # service network restart
>>> cluster12-b: # ovs-vsctl show
>>> 2e9bf291-50ac-4c3a-ac55-2d590df1880d
>>> Bridge br-oam
>>> Port br-oam
>>> Interface br-oam
>>> type: internal
>>> ovs_version: "2.14.2"
>>> cluster12-b: #
>>>
>>> it seems like a common issue .
>>
>> You have in your log "kernel: [340557.486796][ T2447] bonding: bond1 is
>> being deleted", it means something is destroying a bonding device in the
>> kernel. And it makes some sense because you're restarting the network.
>> Does this bond1 interface exist after the network restart? What creates it
>> back? network-scripts? If so, you should check these network-scripts. And
>> the port-add command should be somewhere in the same network scripts.
>> And the bridge should likely be created there. Otherwise, the configuration
>> you created is not going to be persistent.
>>
>>>
>>> //An
>>>
>>> -----Original Message-----
>>> From: Ilya Maximets <[email protected]>
>>> Sent: Monday, October 16, 2023 6:54 PM
>>> To: Liqi An <[email protected]>; [email protected]
>>> Cc: Cheng Chi <[email protected]>; Jonas Yi
>>> <[email protected]>; Yawei Lu <[email protected]>;
>>> [email protected]
>>> Subject: Re: [ovs-discuss] Restarting the network triggers the
>>> deletion of one ovs port
>>>
>>> On 10/16/23 07:38, Liqi An via discuss wrote:
>>>> Hi experts,
>>>>
>>>> I am having a very strange problem with matching virtual machines
>>>> installations with openvswitch. My operating system is suse15-sp4;
>>>>
>>>> cluster12-b:~ # cat /etc/os-release
>>>> /NAME="SLES"/
>>>> /VERSION="15-SP4"/
>>>> /VERSION_ID="15.4"/
>>>> /PRETTY_NAME="SUSE Linux Enterprise Server 15 SP4"/ /ID="sles"/
>>>> /ID_LIKE="suse"/ /ANSI_COLOR="0;32"/
>>>> /CPE_NAME="cpe:/o:suse:sles:15:sp4"/
>>>> /DOCUMENTATION_URL=https://documentation.suse.com/
>>>> <https://documentation.suse.com/>/
>>>>
>>>> cluster12-b:~ # rpm -qa |grep openvswitch
>>>> /openvswitch-2.14.2-150400.22.23.x86_64/
>>>>
>>>> cluster12-b:~ # virsh net-list --all
>>>>
>>>> /Name State Autostart Persistent/
>>>> /-----------------------------------------------------/
>>>> /2.11-ovs-network active yes yes/
>>>>
>>>> bond1 was used by the VMs:
>>>> …
>>>> Bridge br-oam
>>>> Port bond1
>>>> trunks: [3932, 3933]
>>>> Interface bond1
>>>> Port "2.11-SC-2-eth1"
>>>> tag: 3932
>>>> Interface "2.11-SC-2-eth1"
>>>> Port br-oam
>>>> Interface br-oam
>>>> type: internal
>>>> Port "2.11-SC-2-eth2"
>>>> tag: 3933
>>>> Interface "2.11-SC-2-eth2"
>>>>
>>>> But when I restarted the network service by command: # service
>>>> network restart , this port bond1 lost in the bridge br-oam , and
>>>> there are some abnormal log in systemlog, Detailed operation logs
>>>> are attached
>>>>
>>>> …
>>>> /25302 2023-10-16T13:07:12.708071+08:00 cluster12-b kernel:
>>>> [340552.475586][ T2447] device eth1 left promiscuous mode/
>>>> /25303 2023-10-16T13:07:12.824022+08:00 cluster12-b kernel:
>>>> [340552.593298][ T2447] bonding: bond0 is being deleted.../
>>>> /25304 2023-10-16T13:07:12.824045+08:00 cluster12-b kernel:
>>>> [340552.593393][ T2447] bond0 (unregistering): Released all slaves/
>>>> /25305 2023-10-16T13:07:12.881576+08:00 cluster12-b systemd[1]:
>>>> Starting Generate issue file for login session.../
>>>> /25306 2023-10-16T13:07:12.905589+08:00 cluster12-b systemd[1]:
>>>> issue-generator.service: Deactivated successfully./
>>>> /25307 2023-10-16T13:07:12.905662+08:00 cluster12-b systemd[1]:
>>>> Finished Generate issue file for login session./
>>>> /25308 2023-10-16T13:07:17.668420+08:00 cluster12-b ovs-vsctl:
>>>> ovs|00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl del-port br-oam
>>>> bond1/
>>>> /25309 2023-10-16T13:07:17.676015+08:00 cluster12-b kernel:
>>>> [340557.444150][ T2261] device bond1 left promiscuous mode/
>>>> /25310 2023-10-16T13:07:17.720080+08:00 cluster12-b kernel:
>>>> [340557.486796][ T2447] bonding: bond1 is being deleted.../
>>>> /25311 2023-10-16T13:07:17.720097+08:00 cluster12-b kernel:
>>>> [340557.486891][ T2447] bond1 (unregistering): Released all slaves/
>>>
>>> IIUC, the 'bond1' is some sort of a kernel bonding device configured
>>> outside of OVS. And it is getting removed.
>>> When you restart the network, the system will execute whatever network
>>> configuration is in your system settings, e.g. stuff from
>>> /etc/sysconfig/network-scripts, maybe NetworkManager is going to re-apply
>>> its configuration or netplan, I don't really know what SUSE is using.
>>> So, you should look in these places for things that manage the bond1
>>> interface.
>>>
>>> Best regards, Ilya Maximets.
>>>
>>>>
>>>> It seemed that Restarting the host's network service automatically
>>>> triggered behavior: /as /usr/bin/ovs-vsctl del-port br-oam bond1/
>>>>
>>>> Also, I restart host which causes the same issue, would you please
>>>> help check and give some advice, thx~
>>>>
>>>> //An
_______________________________________________
discuss mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
