[ovs-discuss] OVS dropping NSH packets that are correctly encapsulated
Hello,
I'm trying to use OVS 2.11.9 (without controller) in Ubuntu 14.04 to push NSH
and ETH headers to an ICMP packet. The topology is as follows:
Client ---> (p1p1) Classifier (p1p2) ---> Server
where p1p1 and p1p2 interfaces on the classifier are bound to the OVS bridge
br0.
The problem is that after a packet enters classifier's p1p1, it gets dropped by
OVS datapath before it reaches br0.
dmesg shows error message as
[ 3538.861851] openvswitch: netlink: Flow actions may not be safe on all
matching packets.
ovs-vswitchd.log shows
2019-06-20T13:05:13.942Z|03925|dpif(handler7)|WARN|system@ovs-system: failed to
put[create] (Invalid argument) ufid:7f5e3b96-75ba-4a36-8385-fcb4ee4cda93
recirc_id(0),dp_hash(0/0),skb_priority(0/0),in_port(2),skb_mark(0/0),eth(src=a0:36:9f:a3:5e:f0,dst=a0:36:9f:a3:5c:30),eth_type(0x0800),ipv4(src=10.0.0.1/0.0.0.0,dst=13.0.0.1,proto=1/0,tos=0/0,ttl=64/0,frag=no),icmp(type=8/0,code=0/0),
actions:push_nsh(flags=2,ttl=63,mdtype=1,np=3,spi=0x1234,si=255,c1=0x11223344,c2=0x11223344,c3=0x11223344,c4=0x11223344),push_eth(src=a0:36:9f:a3:5c:30,dst=a0:36:9f:8e:4b:48),1
Any suggestion please?
== dump-flows output ==
root@ovs-br-1:/home/chang# ovs-ofctl -O Openflow13 dump-flows br0
cookie=0x0, duration=669.284s, table=0, n_packets=669, n_bytes=65562,
ip,in_port=p1p1,nw_dst=13.0.0.1
actions=encap(nsh(md_type=1)),set_field:2->nsh_flags,set_field:0x1234->nsh_spi,set_field:255->nsh_si,set_field:63->nsh_ttl,set_field:0x11223344->nsh_c1,set_field:0x11223344->nsh_c2,set_field:0x11223344->nsh_c3,set_field:0x11223344->nsh_c4,goto_table:4
cookie=0x0, duration=669.281s, table=0, n_packets=16, n_bytes=960,
reset_counts priority=1000,arp,arp_tpa=10.0.0.10,arp_op=1
actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[],set_field:a0:36:9f:a3:5c:30->eth_src,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[],load:0xa0369fa35c30->NXM_NX_ARP_SHA[],move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[],load:0xa0a->NXM_OF_ARP_SPA[],IN_PORT
cookie=0x0, duration=669.252s, table=4, n_packets=669, n_bytes=65562,
packet_type=(1,0x894f)
actions=encap(ethernet),set_field:a0:36:9f:a3:5c:30->eth_src,set_field:a0:36:9f:8e:4b:48->eth_dst,goto_table:10
cookie=0x0, duration=552.056s, table=10, n_packets=552, n_bytes=54096,
dl_type=0x894f actions=output:p1p2
== appctl trace output ==
Flow:
ip,in_port=1,vlan_tci=0x,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,nw_src=0.0.0.0,nw_dst=13.0.0.1,nw_proto=0,nw_tos=0,nw_ecn=0,nw_ttl=0
bridge("br0")
-
0. ip,in_port=1,nw_dst=13.0.0.1, priority 32768
encap(nsh(md_type=1))
set_field:2->nsh_flags
set_field:0x1234->nsh_spi
set_field:255->nsh_si
set_field:63->nsh_ttl
set_field:0x11223344->nsh_c1
set_field:0x11223344->nsh_c2
set_field:0x11223344->nsh_c3
set_field:0x11223344->nsh_c4
goto_table:4
4. packet_type=(1,0x894f), priority 32768
encap(ethernet)
set_field:a0:36:9f:a3:5c:30->eth_src
set_field:a0:36:9f:8e:4b:48->eth_dst
goto_table:10
10. dl_type=0x894f, priority 32768
output:2
Final flow:
in_port=1,vlan_tci=0x,dl_src=a0:36:9f:a3:5c:30,dl_dst=a0:36:9f:8e:4b:48,dl_type=0x894f,nsh_flags=2,nsh_ttl=63,nsh_mdtype=1,nsh_np=3,nsh_spi=0x1234,nsh_si=255,nsh_c1=0x11223344,nsh_c2=0x11223344,nsh_c3=0x11223344,nsh_c4=0x11223344,nw_proto=0,nw_tos=0,nw_ecn=0,nw_ttl=0
Megaflow:
recirc_id=0,eth,ip,in_port=1,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,nw_dst=13.0.0.1,nw_frag=no
Datapath actions:
push_nsh(flags=2,ttl=63,mdtype=1,np=3,spi=0x1234,si=255,c1=0x11223344,c2=0x11223344,c3=0x11223344,c4=0x11223344),push_eth(src=a0:36:9f:a3:5c:30,dst=a0:36:9f:8e:4b:48),1
== ovs-vsctl output ==
9e1192fd-3f02-497f-90e5-64d24b24932f
Bridge "br0"
Port "p1p1"
Interface "p1p1"
Port "br0"
Interface "br0"
type: internal
Port "p1p2"
Interface "p1p2"
ovs_version: "2.11.90"
Regards,
Chang
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] Bridge datapath_type gets reset after couple of reboots
On Thu, Jun 20, 2019 at 12:34:31AM +, Shriroop Joshi (shrirjos) via discuss wrote: > I encountered a strange problem with OVS-DPDK. I created a bridge and changed > its datapath_type to netdev after initializing DPDK. > After a couple of reboots, it's datapath_type gets reset with DPDK > initialized. Also, vnics connected to the bridge get disconnected. We don't > have any interfaces connected to the bridge. It's usually possible to track down the cause of database changes with "ovsdb-tool -mm show-log", which shows what program made a change and when and sometimes additional comments. ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] Bridge datapath_type gets reset after couple of reboots
The ovsdb-tool.log shows several bridges being inserted but only one of
them has its datapath_type set to netdev. Is that intentional?
On Thu, Jun 20, 2019 at 11:46:09PM +, Alex Yeh (ayeh) wrote:
> Hi Ben,
> Thanks for your reply and the suggestion.
> We looked at the output of the command and don't seen command
> explicitly modified the datapath_type on ayeh-br1 bridge. I turned on the
> debug on ovs-vswitchd for the reboot hopefully it can provide some
> information. I'm attaching the output from the command and the debug log from
> ovs-vswitchd.
>
> Thanks much for any tip anyone can spare,
> Alex
> (\__/)
> (='.'=)
> (")_(")
>
> -Original Message-
> From: Ben Pfaff
> Sent: Thursday, June 20, 2019 9:26 AM
> To: Shriroop Joshi (shrirjos)
> Cc: ovs-discuss@openvswitch.org; Alex Yeh (ayeh) ; Tobias
> Hofmann -T (tohofman - AAP3 INC at Cisco)
> Subject: Re: [ovs-discuss] Bridge datapath_type gets reset after couple of
> reboots
>
> On Thu, Jun 20, 2019 at 12:34:31AM +, Shriroop Joshi (shrirjos) via
> discuss wrote:
> > I encountered a strange problem with OVS-DPDK. I created a bridge and
> > changed its datapath_type to netdev after initializing DPDK.
> > After a couple of reboots, it's datapath_type gets reset with DPDK
> > initialized. Also, vnics connected to the bridge get disconnected. We don't
> > have any interfaces connected to the bridge.
>
> It's usually possible to track down the cause of database changes with
> "ovsdb-tool -mm show-log", which shows what program made a change and when
> and sometimes additional comments.
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] Bridge datapath_type gets reset after couple of reboots
Ben,
Yes, it is intentional, ayeh-br1 has netdev set.
Thanks
Alex
(\__/)
(='.'=)
(")_(")
-Original Message-
From: Ben Pfaff
Sent: Thursday, June 20, 2019 5:29 PM
To: Alex Yeh (ayeh)
Cc: Shriroop Joshi (shrirjos) ;
ovs-discuss@openvswitch.org; Tobias Hofmann -T (tohofman - AAP3 INC at Cisco)
Subject: Re: [ovs-discuss] Bridge datapath_type gets reset after couple of
reboots
The ovsdb-tool.log shows several bridges being inserted but only one of them
has its datapath_type set to netdev. Is that intentional?
On Thu, Jun 20, 2019 at 11:46:09PM +, Alex Yeh (ayeh) wrote:
> Hi Ben,
> Thanks for your reply and the suggestion.
> We looked at the output of the command and don't seen command
> explicitly modified the datapath_type on ayeh-br1 bridge. I turned on the
> debug on ovs-vswitchd for the reboot hopefully it can provide some
> information. I'm attaching the output from the command and the debug log from
> ovs-vswitchd.
>
> Thanks much for any tip anyone can spare, Alex
> (\__/)
> (='.'=)
> (")_(")
>
> -Original Message-
> From: Ben Pfaff
> Sent: Thursday, June 20, 2019 9:26 AM
> To: Shriroop Joshi (shrirjos)
> Cc: ovs-discuss@openvswitch.org; Alex Yeh (ayeh) ;
> Tobias Hofmann -T (tohofman - AAP3 INC at Cisco)
> Subject: Re: [ovs-discuss] Bridge datapath_type gets reset after
> couple of reboots
>
> On Thu, Jun 20, 2019 at 12:34:31AM +, Shriroop Joshi (shrirjos) via
> discuss wrote:
> > I encountered a strange problem with OVS-DPDK. I created a bridge and
> > changed its datapath_type to netdev after initializing DPDK.
> > After a couple of reboots, it's datapath_type gets reset with DPDK
> > initialized. Also, vnics connected to the bridge get disconnected. We don't
> > have any interfaces connected to the bridge.
>
> It's usually possible to track down the cause of database changes with
> "ovsdb-tool -mm show-log", which shows what program made a change and when
> and sometimes additional comments.
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] [OVN] ovn-controller Incremental Processing scale testing
On Tue, Jun 11, 2019 at 9:16 AM Daniel Alvarez Sanchez
wrote:
>
> Thanks a lot Han for the answer!
>
> On Tue, Jun 11, 2019 at 5:57 PM Han Zhou wrote:
> >
> >
> >
> >
> > On Tue, Jun 11, 2019 at 5:12 AM Dumitru Ceara wrote:
> > >
> > > On Tue, Jun 11, 2019 at 10:40 AM Daniel Alvarez Sanchez
> > > wrote:
> > > >
> > > > Hi Han, all,
> > > >
> > > > Lucas, Numan and I have been doing some 'scale' testing of OpenStack
> > > > using OVN and wanted to present some results and issues that we've
> > > > found with the Incremental Processing feature in ovn-controller.
Below
> > > > is the scenario that we executed:
> > > >
> > > > * 7 baremetal nodes setup: 3 controllers (running
> > > > ovn-northd/ovsdb-servers in A/P with pacemaker) + 4 compute nodes.
OVS
> > > > 2.10.
> > > > * The test consists on:
> > > > - Create openstack network (OVN LS), subnet and router
> > > > - Attach subnet to the router and set gw to the external network
> > > > - Create an OpenStack port and apply a Security Group (ACLs to
allow
> > > > UDP, SSH and ICMP).
> > > > - Bind the port to one of the 4 compute nodes (randomly) by
> > > > attaching it to a network namespace.
> > > > - Wait for the port to be ACTIVE in Neutron ('up == True' in NB)
> > > > - Wait until the test can ping the port
> > > > * Running browbeat/rally with 16 simultaneous process to execute the
> > > > test above 150 times.
> > > > * When all the 150 'fake VMs' are created, browbeat will delete all
> > > > the OpenStack/OVN resources.
> > > >
> > > > We first tried with OVS/OVN 2.10 and pulled some results which
showed
> > > > 100% success but ovn-controller is quite loaded (as expected) in all
> > > > the nodes especially during the deletion phase:
> > > >
> > > > - Compute node: https://imgur.com/a/tzxfrIR
> > > > - Controller node (ovn-northd and ovsdb-servers):
https://imgur.com/a/8ffKKYF
> > > >
> > > > After conducting the tests above, we replaced ovn-controller in all
7
> > > > nodes by the one with the current master branch (actually from last
> > > > week). We also replaced ovn-northd and ovsdb-servers but the
> > > > ovs-vswitchd has been left untouched (still on 2.10). The expected
> > > > results were to get less ovn-controller CPU usage and also better
> > > > times due to the Incremental Processing feature introduced recently.
> > > > However, the results don't look very good:
> > > >
> > > > - Compute node: https://imgur.com/a/wuq87F1
> > > > - Controller node (ovn-northd and ovsdb-servers):
https://imgur.com/a/99kiyDp
> > > >
> > > > One thing that we can tell from the ovs-vswitchd CPU consumption is
> > > > that it's much less in the Incremental Processing (IP) case which
> > > > apparently doesn't make much sense. This led us to think that
perhaps
> > > > ovn-controller was not installing the necessary flows in the switch
> > > > and we confirmed this hypothesis by looking into the dataplane
> > > > results. Out of the 150 VMs, 10% of them were unreachable via ping
> > > > when using ovn-controller from master.
> > > >
> > > > @Han, others, do you have any ideas as of what could be happening
> > > > here? We'll be able to use this setup for a few more days so let me
> > > > know if you want us to pull some other data/traces, ...
> > > >
> > > > Some other interesting things:
> > > > On each of the compute nodes, (with an almost evenly distributed
> > > > number of logical ports bound to them), the max amount of logical
> > > > flows in br-int is ~90K (by the end of the test, right before
deleting
> > > > the resources).
> > > >
> > > > It looks like with the IP version, ovn-controller leaks some memory:
> > > > https://imgur.com/a/trQrhWd
> > > > While with OVS 2.10, it remains pretty flat during the test:
> > > > https://imgur.com/a/KCkIT4O
> > >
> > > Hi Daniel, Han,
> > >
> > > I just sent a small patch for the ovn-controller memory leak:
> > > https://patchwork.ozlabs.org/patch/1113758/
> > >
> > > At least on my setup this is what valgrind was pointing at.
> > >
> > > Cheers,
> > > Dumitru
> > >
> > > >
> > > > Looking forward to hearing back :)
> > > > Daniel
> > > >
> > > > PS. Sorry for my previous email, I sent it by mistake without the
subject
> > > > ___
> > > > discuss mailing list
> > > > disc...@openvswitch.org
> > > > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
> >
> > Thanks Daniel for the testing and reporting, and thanks Dumitru for
fixing the memory leak.
> >
> > Currently ovn-controller incremental processing only handles below SB
changes incrementally:
> > - logical_flow
> > - port_binding (for regular VIF binding NOT on current chassis)
> > - mc_group
> > - address_set
> > - port_group
> > - mac_binding
> >
> > So, in test scenario you described, since each iteration creates
network (SB datapath changes) and router ports (port_binding changes for
non VIF), the incremental processing would not help much, because most
steps in your test should trigger rec
Re: [ovs-discuss] [OVN] ovn-controller Incremental Processing scale testing
On Fri, Jun 21, 2019, 11:47 AM Han Zhou wrote:
>
>
> On Tue, Jun 11, 2019 at 9:16 AM Daniel Alvarez Sanchez <
> dalva...@redhat.com> wrote:
> >
> > Thanks a lot Han for the answer!
> >
> > On Tue, Jun 11, 2019 at 5:57 PM Han Zhou wrote:
> > >
> > >
> > >
> > >
> > > On Tue, Jun 11, 2019 at 5:12 AM Dumitru Ceara
> wrote:
> > > >
> > > > On Tue, Jun 11, 2019 at 10:40 AM Daniel Alvarez Sanchez
> > > > wrote:
> > > > >
> > > > > Hi Han, all,
> > > > >
> > > > > Lucas, Numan and I have been doing some 'scale' testing of
> OpenStack
> > > > > using OVN and wanted to present some results and issues that we've
> > > > > found with the Incremental Processing feature in ovn-controller.
> Below
> > > > > is the scenario that we executed:
> > > > >
> > > > > * 7 baremetal nodes setup: 3 controllers (running
> > > > > ovn-northd/ovsdb-servers in A/P with pacemaker) + 4 compute nodes.
> OVS
> > > > > 2.10.
> > > > > * The test consists on:
> > > > > - Create openstack network (OVN LS), subnet and router
> > > > > - Attach subnet to the router and set gw to the external network
> > > > > - Create an OpenStack port and apply a Security Group (ACLs to
> allow
> > > > > UDP, SSH and ICMP).
> > > > > - Bind the port to one of the 4 compute nodes (randomly) by
> > > > > attaching it to a network namespace.
> > > > > - Wait for the port to be ACTIVE in Neutron ('up == True' in NB)
> > > > > - Wait until the test can ping the port
> > > > > * Running browbeat/rally with 16 simultaneous process to execute
> the
> > > > > test above 150 times.
> > > > > * When all the 150 'fake VMs' are created, browbeat will delete all
> > > > > the OpenStack/OVN resources.
> > > > >
> > > > > We first tried with OVS/OVN 2.10 and pulled some results which
> showed
> > > > > 100% success but ovn-controller is quite loaded (as expected) in
> all
> > > > > the nodes especially during the deletion phase:
> > > > >
> > > > > - Compute node: https://imgur.com/a/tzxfrIR
> > > > > - Controller node (ovn-northd and ovsdb-servers):
> https://imgur.com/a/8ffKKYF
> > > > >
> > > > > After conducting the tests above, we replaced ovn-controller in
> all 7
> > > > > nodes by the one with the current master branch (actually from last
> > > > > week). We also replaced ovn-northd and ovsdb-servers but the
> > > > > ovs-vswitchd has been left untouched (still on 2.10). The expected
> > > > > results were to get less ovn-controller CPU usage and also better
> > > > > times due to the Incremental Processing feature introduced
> recently.
> > > > > However, the results don't look very good:
> > > > >
> > > > > - Compute node: https://imgur.com/a/wuq87F1
> > > > > - Controller node (ovn-northd and ovsdb-servers):
> https://imgur.com/a/99kiyDp
> > > > >
> > > > > One thing that we can tell from the ovs-vswitchd CPU consumption is
> > > > > that it's much less in the Incremental Processing (IP) case which
> > > > > apparently doesn't make much sense. This led us to think that
> perhaps
> > > > > ovn-controller was not installing the necessary flows in the switch
> > > > > and we confirmed this hypothesis by looking into the dataplane
> > > > > results. Out of the 150 VMs, 10% of them were unreachable via ping
> > > > > when using ovn-controller from master.
> > > > >
> > > > > @Han, others, do you have any ideas as of what could be happening
> > > > > here? We'll be able to use this setup for a few more days so let me
> > > > > know if you want us to pull some other data/traces, ...
> > > > >
> > > > > Some other interesting things:
> > > > > On each of the compute nodes, (with an almost evenly distributed
> > > > > number of logical ports bound to them), the max amount of logical
> > > > > flows in br-int is ~90K (by the end of the test, right before
> deleting
> > > > > the resources).
> > > > >
> > > > > It looks like with the IP version, ovn-controller leaks some
> memory:
> > > > > https://imgur.com/a/trQrhWd
> > > > > While with OVS 2.10, it remains pretty flat during the test:
> > > > > https://imgur.com/a/KCkIT4O
> > > >
> > > > Hi Daniel, Han,
> > > >
> > > > I just sent a small patch for the ovn-controller memory leak:
> > > > https://patchwork.ozlabs.org/patch/1113758/
> > > >
> > > > At least on my setup this is what valgrind was pointing at.
> > > >
> > > > Cheers,
> > > > Dumitru
> > > >
> > > > >
> > > > > Looking forward to hearing back :)
> > > > > Daniel
> > > > >
> > > > > PS. Sorry for my previous email, I sent it by mistake without the
> subject
> > > > > ___
> > > > > discuss mailing list
> > > > > disc...@openvswitch.org
> > > > > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
> > >
> > > Thanks Daniel for the testing and reporting, and thanks Dumitru for
> fixing the memory leak.
> > >
> > > Currently ovn-controller incremental processing only handles below SB
> changes incrementally:
> > > - logical_flow
> > > - port_binding (for regular VIF binding NOT on current chassis)
>
Re: [ovs-discuss] [OVN] ovn-controller Incremental Processing scale testing
Hi Han,
> On 21 Jun 2019, at 08:16, Han Zhou wrote:
>
>
>
> On Tue, Jun 11, 2019 at 9:16 AM Daniel Alvarez Sanchez
> wrote:
> >
> > Thanks a lot Han for the answer!
> >
> > On Tue, Jun 11, 2019 at 5:57 PM Han Zhou wrote:
> > >
> > >
> > >
> > >
> > > On Tue, Jun 11, 2019 at 5:12 AM Dumitru Ceara wrote:
> > > >
> > > > On Tue, Jun 11, 2019 at 10:40 AM Daniel Alvarez Sanchez
> > > > wrote:
> > > > >
> > > > > Hi Han, all,
> > > > >
> > > > > Lucas, Numan and I have been doing some 'scale' testing of OpenStack
> > > > > using OVN and wanted to present some results and issues that we've
> > > > > found with the Incremental Processing feature in ovn-controller. Below
> > > > > is the scenario that we executed:
> > > > >
> > > > > * 7 baremetal nodes setup: 3 controllers (running
> > > > > ovn-northd/ovsdb-servers in A/P with pacemaker) + 4 compute nodes. OVS
> > > > > 2.10.
> > > > > * The test consists on:
> > > > > - Create openstack network (OVN LS), subnet and router
> > > > > - Attach subnet to the router and set gw to the external network
> > > > > - Create an OpenStack port and apply a Security Group (ACLs to allow
> > > > > UDP, SSH and ICMP).
> > > > > - Bind the port to one of the 4 compute nodes (randomly) by
> > > > > attaching it to a network namespace.
> > > > > - Wait for the port to be ACTIVE in Neutron ('up == True' in NB)
> > > > > - Wait until the test can ping the port
> > > > > * Running browbeat/rally with 16 simultaneous process to execute the
> > > > > test above 150 times.
> > > > > * When all the 150 'fake VMs' are created, browbeat will delete all
> > > > > the OpenStack/OVN resources.
> > > > >
> > > > > We first tried with OVS/OVN 2.10 and pulled some results which showed
> > > > > 100% success but ovn-controller is quite loaded (as expected) in all
> > > > > the nodes especially during the deletion phase:
> > > > >
> > > > > - Compute node: https://imgur.com/a/tzxfrIR
> > > > > - Controller node (ovn-northd and ovsdb-servers):
> > > > > https://imgur.com/a/8ffKKYF
> > > > >
> > > > > After conducting the tests above, we replaced ovn-controller in all 7
> > > > > nodes by the one with the current master branch (actually from last
> > > > > week). We also replaced ovn-northd and ovsdb-servers but the
> > > > > ovs-vswitchd has been left untouched (still on 2.10). The expected
> > > > > results were to get less ovn-controller CPU usage and also better
> > > > > times due to the Incremental Processing feature introduced recently.
> > > > > However, the results don't look very good:
> > > > >
> > > > > - Compute node: https://imgur.com/a/wuq87F1
> > > > > - Controller node (ovn-northd and ovsdb-servers):
> > > > > https://imgur.com/a/99kiyDp
> > > > >
> > > > > One thing that we can tell from the ovs-vswitchd CPU consumption is
> > > > > that it's much less in the Incremental Processing (IP) case which
> > > > > apparently doesn't make much sense. This led us to think that perhaps
> > > > > ovn-controller was not installing the necessary flows in the switch
> > > > > and we confirmed this hypothesis by looking into the dataplane
> > > > > results. Out of the 150 VMs, 10% of them were unreachable via ping
> > > > > when using ovn-controller from master.
> > > > >
> > > > > @Han, others, do you have any ideas as of what could be happening
> > > > > here? We'll be able to use this setup for a few more days so let me
> > > > > know if you want us to pull some other data/traces, ...
> > > > >
> > > > > Some other interesting things:
> > > > > On each of the compute nodes, (with an almost evenly distributed
> > > > > number of logical ports bound to them), the max amount of logical
> > > > > flows in br-int is ~90K (by the end of the test, right before deleting
> > > > > the resources).
> > > > >
> > > > > It looks like with the IP version, ovn-controller leaks some memory:
> > > > > https://imgur.com/a/trQrhWd
> > > > > While with OVS 2.10, it remains pretty flat during the test:
> > > > > https://imgur.com/a/KCkIT4O
> > > >
> > > > Hi Daniel, Han,
> > > >
> > > > I just sent a small patch for the ovn-controller memory leak:
> > > > https://patchwork.ozlabs.org/patch/1113758/
> > > >
> > > > At least on my setup this is what valgrind was pointing at.
> > > >
> > > > Cheers,
> > > > Dumitru
> > > >
> > > > >
> > > > > Looking forward to hearing back :)
> > > > > Daniel
> > > > >
> > > > > PS. Sorry for my previous email, I sent it by mistake without the
> > > > > subject
> > > > > ___
> > > > > discuss mailing list
> > > > > disc...@openvswitch.org
> > > > > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
> > >
> > > Thanks Daniel for the testing and reporting, and thanks Dumitru for
> > > fixing the memory leak.
> > >
> > > Currently ovn-controller incremental processing only handles below SB
> > > changes incrementally:
> > > - logical_flow
> > > - port_binding (for regular VIF binding NOT on current chassis)
