Re: [ovs-discuss] OVN SB DB from RAFT cluster to Relay DB

[Sri kor via discuss]

Is there a way to configure, ovn-controller subscribing to only specific SB
DB updates?
I

On Wed, Mar 6, 2024 at 1:45 AM Felix Huettner 
wrote:

> On Wed, Mar 06, 2024 at 10:29:29AM +0300, Vladislav Odintsov wrote:
> > Hi Felix,
> >
> > > On 6 Mar 2024, at 10:16, Felix Huettner via discuss <
> ovs-discuss@openvswitch.org> wrote:
> > >
> > > Hi Srini,
> > >
> > > i can share what works for us for ~1k hypervisors:
> > >
> > > On Tue, Mar 05, 2024 at 09:51:43PM -0800, Sri kor via discuss wrote:
> > >> Hi Team,
> > >>
> > >>
> > >> Currently , we are using OVN in RAFT cluster mode. We have 3 NB and SB
> > >> ovsdb-servers operating in RAFT cluster mode. Currently we have 500
> > >> hypervisors connected to this RAFT cluster.
> > >>
> > >> For our next deployment, our scale would increase to 3000
> hypervisors. To
> > >> accommodate this scaled hypervisors, we are migrating to DB relay with
> > >> multigroup deployment model. This increase helps with OVN SB DB read
> > >> transactions. But for write transactions, only the leader in the RAFT
> > >> cluster can update the DB. This creates a load on the leader of RAFT.
> Is
> > >> there a way to address the load on the RAFT cluster leader?
> > >
> > > We do the following:
> > > * If you need TLS on the ovsdb path, separate it out to some
> > >  reverseproxy that can do just L4 TLS Termination (e.g. traefik, or so)
> >
> > Do I understand correctly that with such TLS "offload" you can’t use
> RBAC for hypervisors?
> >
>
> yes, that is the unfortunate side effect
>
> > > * Have nobody besides northd connect to the SB DB directly, everyone
> > >  else needs to use a relay
> > > * Do not run backups on the cluster leader, but on one of the current
> > >  followers
> > > * Increase the raft election timeout significantly (we have 120s in
> > >  there). However there is a patch afaik in 3.3 that makes that better
> > > * If you create metrics or so from database content generate these on
> > >  the relays instead of the raft cluster
> > >
> > > Overall when our southbound db had issues most of the time it was some
> > > client constantly reconnecting to it and thereby pulling always a full
> > > DB dump.
> > >
> > >>
> > >>
> > >> As the scale increases, number updates coming to the ovn-controller
> from
> > >> OVN SB increases. that creates pressure on ovn-controller. Is there a
> way
> > >> to minimize the load on ovn-controller?
> > >
> > > Did not see any kind of issue there yet.
> > > However if you are using some python tooling outside of OVN (e.g.
> > > Openstack) ensure that you have JSON parsing using a C library
> avaialble
> > > in the ovs lib. This brings significant performance benefts if you have
> > > a lot of updates.
> > > You can check with `python3 -c "import ovs.json;
> print(ovs.json.PARSER)"`
> > > which should return "C".
> > >
> > >>
> > >> I wish there is a way for ovn-controller to subscribe to updates
> specific
> > >> to this hypervisor. Are there any known ovn-contrller subscription
> methods
> > >> available and being used OVS community?
> > >
> > > Yes, they do that per default. However for us we saw that this creates
> > > increased load on the relays due to the needed additional filtering and
> > > json serializing per target node. So we turned it of and thereby trade
> > > less ovsdb load for more network bandwidth.
> > > Relevant setting is `external_ids:ovn-monitor-all`.
> > >
> > > Thanks
> > > Felix
> > >
> > >>
> > >>
> > >> How can I optimize the load on the leader node in an OVN RAFT cluster
> to
> > >> handle increased write transactions?
> > >>
> > >>
> > >>
> > >> Thanks,
> > >>
> > >> Srini
> > >
> > >> ___
> > >> discuss mailing list
> > >> disc...@openvswitch.org
> > >> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
> > >
> > > ___
> > > discuss mailing list
> > > disc...@openvswitch.org 
> > > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
> >
> >
> > Regards,
> > Vladislav Odintsov
> >
>
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss

Re: [ovs-discuss] OVN: Configuring Flow_Table (vswitchd.config.db) settings through OVN

[Naveen Yerramneni via discuss]

> On 07-Mar-2024, at 9:10 PM, Numan Siddique  wrote:
> 
> On Wed, Mar 6, 2024 at 9:41 PM Naveen Yerramneni via discuss
>  wrote:
>> 
>> 
>> 
>>> On 07-Mar-2024, at 6:02 AM, Numan Siddique  wrote:
>>> 
>>> On Wed, Mar 6, 2024 at 3:07 PM Naveen Yerramneni via discuss
>>>  wrote:
 
 Hi All,
 
 We are exploring the possibility of doing some Flow_Table settings (like 
 classifier optimizations)  through OVN.
 
 One possible option could be to expose this in ovn-nb config and propagate 
 it to ovn-sb.
 - Add new table with name “Flow_Config” which stores settings (similar to 
 Flow_Table in  vwitchd.conf.db)
 - Add new columns “flow_table_in_settings” and “flow_table_out_settings” 
 in NB_Global and SB_Global tables.
  The type of these columns is map of : where key 
 is logical pipeline stage number and
 value points to a row entry in Flow_Config table.
 
 OVN controller uses this information and configures vwitchd.config.db.
 - Flow_Table rows in vswitchd.conf.db are populated using Flow_Config 
 table in southbound.
 - Bridge table's flow_tables column is populated using keys (logical table 
 numbers) in flow_table_in_settings and
 flow_table_out_settings columns of SB_Global table . During configuration, 
 OVN controller adds offset
 OFTABLE_LOG_INGRESS_PIPELINE for ingress tables and 
 OFTABLE_LOG_EGRESS_PIPELINE for egress pipelines.
 
 Probably a new command can be added to northd to dump the logical switch 
 and logical router
 ingress and egress pipeline stage table names and numbers for reference.
 
 Please share your thoughts/inputs on this.
>>> 
>>> Generally,  to configure anything which is chassis related,  we have
>>> used the local openvswitch table.  Each ovn-controller would read
>>> that and configure accordingly.  One example is - 
>>> ovn-openflow-probe-interval.
>>> 
>>> Can't we do something similar here ?  I understand that this config
>>> needs to be done on each chassis,  but if it is a one time thing,
>>> then perhaps it should not be a big concern.  Does this approach work for 
>>> you ?
>>> 
>>> Thanks
>>> Numan
>> 
>> Hi Numan,
>> 
>> Thanks for the reply.
>> 
>> The reason why I thought of putting this config in northbound is:
>>  - Logical table numbers and physical table numbers can potentially change 
>> release to release.
>>  - If we have this config in northbound, it is possible to add some 
>> automation in CMS plug-in to reconfigure
>>the flow_table_settings on the new logical table numbers when northd gets 
>> upgraded. CMS plug-in can
>>have its own logic to find out the logical table numbers.
>>Ex: CMS plug-in  can get the logical table numbers either by parsing the 
>> northd new command
>>output that dumps logical pipeline table names and numbers (or) by other 
>> means.
>> 
>> 
>> If the recommendation is to get this done on the chassis side then, I can 
>> think of below alternative.
>>  - Update northd to dump "logical pipeline stage name: logical table number” 
>> in options:logical-table-mapping
>>of SB_Global table.
>>  - Update OVN controller to dump the "logical pipeline stage name: physical 
>> table number" mapping
>>to the external_ids:oftable-mapping of openvswitch table whenever SB 
>> entry get updated. Additionally, we can
>>possibly add to new command to ovn-controller to dump this 
>> oftable-mapping.
>> - Some automation can be done on the chassis side to use the table mapping 
>> information that ovn-controller dumps
>>   and configure the vswitchd.conf.db.
>> 
>> 
>> Please let me know your suggestions.
> 
> Ok.  I was not aware of the "Flow_Table" feature of OVS.  I think it
> makes sense to put the config in Northbound db
> and propagate it down to OVS via ovn-controller.
> 
> This is what I think can be done:
> 
> 1.  Add new Northbound tables - Switch_Pipeline_Config and
> Router_Pipeline_Config.
> 2.  ovn-northd will create rows in these tables for each stage.
> 3.  CMS will set the config for each row (if it wants to)
> 4. ovn-northd will replicate these to Southbound db.
> 5. ovn-controller will aggregate the switch stage and router stage
> configs since both the stages share the same table number.
> 6. ovn-controller will configure the OVS Flow_Table.
> 
> CMS doesn't need to know the actual physical flow table numbers.
> 
> Switch_Pipeline_Config table can have columns like
> 
> Switch_Pipeline_Config
>-  name  (stage name)
>-  config  (smap)
> 
> 
> What do you think ?   It's worth taking opinion and feedback from
> others before taking this approach.

This approach looks fine.
Do you think exposing the tables as Ingress_Pipeline_Config and 
Egress_Pipeline_Config
is better since logical switch and router share same table number ?

Ingress_Pipeline_Config/Egress_Pipeline_Config table schema can look like:
  - config  (smap)
  - options  (ls_stage_name=<>, lr_stage_name=<>)

If 

Re: [ovs-discuss] OVN: Configuring Flow_Table (vswitchd.config.db) settings through OVN

[Numan Siddique via discuss]

On Wed, Mar 6, 2024 at 9:41 PM Naveen Yerramneni via discuss
 wrote:
>
>
>
> > On 07-Mar-2024, at 6:02 AM, Numan Siddique  wrote:
> >
> > On Wed, Mar 6, 2024 at 3:07 PM Naveen Yerramneni via discuss
> >  wrote:
> >>
> >> Hi All,
> >>
> >> We are exploring the possibility of doing some Flow_Table settings (like 
> >> classifier optimizations)  through OVN.
> >>
> >> One possible option could be to expose this in ovn-nb config and propagate 
> >> it to ovn-sb.
> >> - Add new table with name “Flow_Config” which stores settings (similar to 
> >> Flow_Table in  vwitchd.conf.db)
> >> - Add new columns “flow_table_in_settings” and “flow_table_out_settings” 
> >> in NB_Global and SB_Global tables.
> >>   The type of these columns is map of : where key 
> >> is logical pipeline stage number and
> >>  value points to a row entry in Flow_Config table.
> >>
> >> OVN controller uses this information and configures vwitchd.config.db.
> >> - Flow_Table rows in vswitchd.conf.db are populated using Flow_Config 
> >> table in southbound.
> >> - Bridge table's flow_tables column is populated using keys (logical table 
> >> numbers) in flow_table_in_settings and
> >>  flow_table_out_settings columns of SB_Global table . During 
> >> configuration, OVN controller adds offset
> >> OFTABLE_LOG_INGRESS_PIPELINE for ingress tables and 
> >> OFTABLE_LOG_EGRESS_PIPELINE for egress pipelines.
> >>
> >> Probably a new command can be added to northd to dump the logical switch 
> >> and logical router
> >> ingress and egress pipeline stage table names and numbers for reference.
> >>
> >> Please share your thoughts/inputs on this.
> >
> > Generally,  to configure anything which is chassis related,  we have
> > used the local openvswitch table.  Each ovn-controller would read
> > that and configure accordingly.  One example is - 
> > ovn-openflow-probe-interval.
> >
> > Can't we do something similar here ?  I understand that this config
> > needs to be done on each chassis,  but if it is a one time thing,
> > then perhaps it should not be a big concern.  Does this approach work for 
> > you ?
> >
> > Thanks
> > Numan
>
> Hi Numan,
>
> Thanks for the reply.
>
> The reason why I thought of putting this config in northbound is:
>   - Logical table numbers and physical table numbers can potentially change 
> release to release.
>   - If we have this config in northbound, it is possible to add some 
> automation in CMS plug-in to reconfigure
> the flow_table_settings on the new logical table numbers when northd gets 
> upgraded. CMS plug-in can
> have its own logic to find out the logical table numbers.
> Ex: CMS plug-in  can get the logical table numbers either by parsing the 
> northd new command
> output that dumps logical pipeline table names and numbers (or) by other 
> means.
>
>
> If the recommendation is to get this done on the chassis side then, I can 
> think of below alternative.
>   - Update northd to dump "logical pipeline stage name: logical table number” 
> in options:logical-table-mapping
> of SB_Global table.
>   - Update OVN controller to dump the "logical pipeline stage name: physical 
> table number" mapping
> to the external_ids:oftable-mapping of openvswitch table whenever SB 
> entry get updated. Additionally, we can
> possibly add to new command to ovn-controller to dump this 
> oftable-mapping.
>  - Some automation can be done on the chassis side to use the table mapping 
> information that ovn-controller dumps
>and configure the vswitchd.conf.db.
>
>
> Please let me know your suggestions.

Ok.  I was not aware of the "Flow_Table" feature of OVS.  I think it
makes sense to put the config in Northbound db
and propagate it down to OVS via ovn-controller.

This is what I think can be done:

1.  Add new Northbound tables - Switch_Pipeline_Config and
Router_Pipeline_Config.
2.  ovn-northd will create rows in these tables for each stage.
3.  CMS will set the config for each row (if it wants to)
4. ovn-northd will replicate these to Southbound db.
5. ovn-controller will aggregate the switch stage and router stage
configs since both the stages share the same table number.
6. ovn-controller will configure the OVS Flow_Table.

CMS doesn't need to know the actual physical flow table numbers.

Switch_Pipeline_Config table can have columns like

Switch_Pipeline_Config
-  name  (stage name)
-  config  (smap)


What do you think ?   It's worth taking opinion and feedback from
others before taking this approach.

Thanks
Numan



>
> Thanks,
> Naveen
>
>
> >>
> >> Thanks,
> >> Naveen
> >> ___
> >> discuss mailing list
> >> disc...@openvswitch.org
> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_mailman_listinfo_ovs-2Ddiscuss=DwIFaQ=s883GpUCOChKOHiocYtGcg=2PQjSDR7A28z1kXE1ptSm6X36oL_nCq1XxeEt7FkLmA=U830KSvmakLVHWvUoXFV_ohX9oM93MLYKIx1e1QXRv5yv5ftXaYXFm2eWao0W2pd=tyS_z11uBp8uDvvlAJ7bUUVP_Qw5RMp3p3lAac52fm8=
>
> 

[ovs-discuss] OVN: Configuring Flow_Table (vswitchd.config.db) settings through OVN

[Naveen Yerramneni via discuss]

Hi All,
 
We are exploring the possibility of doing some Flow_Table settings (like 
classifier optimizations)  through OVN.

One possible option could be to expose this in ovn-nb config and propagate it 
to ovn-sb.
  - Add new table with name “Flow_Config” which stores settings (similar to 
Flow_Table in  vwitchd.conf.db)
  - Add new columns “flow_table_in_settings” and “flow_table_out_settings” in 
NB_Global and SB_Global tables.
The type of these columns is map of : where key is 
logical pipeline stage number and 
   value points to a row entry in Flow_Config table.
 
OVN controller uses this information and configures vwitchd.config.db.
  - Flow_Table rows in vswitchd.conf.db are populated using Flow_Config table 
in southbound.
  - Bridge table's flow_tables column is populated using keys (logical table 
numbers) in flow_table_in_settings and
   flow_table_out_settings columns of SB_Global table . During configuration, 
OVN controller adds offset
  OFTABLE_LOG_INGRESS_PIPELINE for ingress tables and 
OFTABLE_LOG_EGRESS_PIPELINE for egress pipelines.

Probably a new command can be added to northd to dump the logical switch and 
logical router
ingress and egress pipeline stage table names and numbers for reference.

Please share your thoughts/inputs on this.

Thanks,
Naveen
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss

Re: [ovs-discuss] Mirror: ovs-tcpdump cannot capture vlan packets on the port with tag

[Ilya Maximets via discuss]

On 3/7/24 04:50, Zhangweiwei wrote:
> I have configured vlan-limit to 0, but the packets still are not delivered to 
> mitapVm72.
> Here is the track information:
> 
> [root@localhost ~]# ovs-vsctl set o . other_config:vlan-limit=0
> [root@localhost ~]#
> [root@localhost ~]# ovs-appctl ofproto/trace vds1-br  in_port=tapVm72 
> 52540067d5615254009abfed81640800455458ad4000400151f502024602020246010800dde3067a0078e030e96588c00200101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f3031323334353637
> Flow: 
> icmp,in_port=6,dl_vlan=100,dl_vlan_pcp=0,vlan_tci1=0x,dl_src=52:54:00:9a:bf:ed,dl_dst=52:54:00:67:d5:61,nw_src=2.2.70.2,nw_dst=2.2.70.1,nw_tos=0,nw_ecn=0,nw_ttl=64,nw_frag=no,icmp_type=8,icmp_code=0
> 
> bridge("vds1-br")
> -
>  0. in_port=6, priority 32768
>   dropping VLAN 100 tagged packet received on port tapVm72 configured 
> as VLAN 0 access port 
> output:3
>   dropping VLAN 100 tagged packet received on port tapVm71 configured 
> as VLAN 0 access port 
> 
> Final flow: unchanged
> Megaflow: recirc_id=0,eth,ip,in_port=6,nw_frag=no
> Datapath actions: 6
> 
> 
> Packets are mirrored in this function. input_vid_is_valid() checked failed
> when a packet with vlan ingress on tapVm72 , because tapVm72 is an access 
> port.

I see.  That makes sense.  The issue is caused by the mix of the NORMAL
pipeline processing and direct forwarding.  There are two ways you can
fix the issue:

1. Remove the tag 0 from the ports, so they are not access ports anymore.
   Is there a reason for them to be access ports if you're not using the
   NORMAL action?  Also, the tag 0 is a special value that can be ignored,
   so I'd advise to not use it.

2. Change the type from access to dot1q-tunnel.  This way double tagging
   will be allowed and the packets should not be dropped anymore.

I don't think the vlan check can be easily removed from the mirroring
code, because we should not mirror a packet if it is going to be dropped
by a NORMAL pipleline later.

Best regrads, Ilya Maximets.

> mirror_packet(struct xlate_ctx *ctx, struct xbundle *xbundle,
>   mirror_mask_t mirrors)
> {
> struct xvlan in_xvlan;
> struct xvlan xvlan;
> 
> /* Figure out what VLAN the packet is in (because mirrors can select
>  * packets on basis of VLAN). */
> xvlan_extract(>xin->flow, _xvlan);
> if (!input_vid_is_valid(ctx, in_xvlan.v[0].vid, xbundle)) {
> return;
> }
> 
> -邮件原件-
> 发件人: Ilya Maximets [mailto:i.maxim...@ovn.org]
> 发送时间: 2024年3月6日 19:43
> 收件人: zhangweiwei (RD) ; ovs-discuss@openvswitch.org
> 抄送: i.maxim...@ovn.org
> 主题: Re: [ovs-discuss] Mirror: ovs-tcpdump cannot capture vlan packets on the 
> port with tag
> 
> On 3/6/24 08:54, Zhangweiwei via discuss wrote:
>> Hi,
>>
>> I set tag 0 on port tapVm72and tapVm71, and then send ping packets
>> with vlan
>> 100 from tapVm72to tapVm71. But ovs-tcpdump cannot capture any packets
>> with vlan on tapVm72. It seems that vlan check is failed in
>> mirror_packet(), because
>> tapVm72 is an access port and the vlan packets are dropped. This is
>> not reasonable because OVS does not use NORMAL forward. When using
>> custom OpenFlow tables, mirror action should not consider tag configuration.
> 
> I don't think that is related to tapVm72 being an access port.
> Could you, please, run ovs-appctl ofproto/trace on a packet arriving from 
> tapVm72 ?
> 
> Note that since mitapVm72 is not in vlan 0, mirrored traffic will have both 
> vlan tags pushed to the packet.  For this to work the vlan-limit 
> configuration should be 2 or 0 (unlimited).  Default value is 1 and that may 
> be one reason why packets are not delivered to mitapVm72.  ofproto/trace 
> should be able to confirm if that is the case.
> 
> Best regards, Ilya Maximets.
> 
>>
>> 1、ovs version: 3.2.1
>> 2、Bridge
>> [root@localhost openvswitch-3.2.1]# ovs-vsctl show
>> Bridge vds1-br
>> Controller "tcp:172.20.66.228:6633"
>> is_connected: true
>> Controller "tcp:172.20.66.229:6633"
>> is_connected: true
>> fail_mode: secure
>> datapath_type: netdev
>> Port vxlan_vds1-br
>> Interface vxlan_vds1-br
>> type: vxlan
>> options: {key=flow, local_ip="3.3.3.70",
>> remote_ip=flow, tos=inherit}
>> Port tapVm72
>> tag: 0
>> Interface tapVm72
>> type: dpdkvhostuserclient
>> options:
>> {vhost-server-path="/var/run/openvswitch/tapVm72"}
>> Port mitapVm72
>> Interface mitapVm72
>> Port tapVm71
>> tag: 0
>> Interface tapVm71
>> type: dpdkvhostuserclient
>> options:
>> {vhost-server-path="/var/run/openvswitch/tapVm71"}
>> Port vds1-br
>> Interface vds1-br
>> type: internal
>> ovs_version: "3.2.1"
>>
>> 3、dpcls:
>>
>>