Hi IIya,
Pls find the replies inline.
> -----Original Message-----
> From: Ilya Maximets <i.maxim...@ovn.org>
> Sent: Tuesday, September 28, 2021 6:22 PM
> To: Amber, Kumar <kumar.am...@intel.com>; Ilya Maximets
> <i.maxim...@ovn.org>; ovs-dev@openvswitch.org;
> tony.vanderp...@alliedtelesis.co.nz
> Cc: Stokes, Ian <ian.sto...@intel.com>; Van Haaren, Harry
> <harry.van.haa...@intel.com>
> Subject: Re: Unit Test Failure Report to OVS ML
>
> On 9/28/21 13:30, Amber, Kumar wrote:
> > Hi llya,
> >
> > The test-case fails with the following build command on the master branch.
> >
> > Pass:
> > ./configure --with-dpdk=static CFLAGS=""
> > Fails:
> > ./configure --with-dpdk=static CFLAGS="-msse4.2"
> >
> > Testing on ovs-master branch, running test case like this $ make check
> > TESTSUITEFLAGS="779"
> >
> > Based on the above build tests it was identified that "-msse4.2" is causing
> > the
> unit-test to fail. Note that OVS changes its hashing implementation based on
> SSE4.2 being available at compile-time. This switches between murmur hash SW
> implementation, and SSE CRC32 instruction.
> >
> > This test seems to expect *a specific value* of a hash result, causing it to
> pass/fail based on hashing implementation selected at ./configure time.
>
> Hi. Thanks for details. That make sense.
>
> Could you try following patch:
>
> diff --git a/tests/tunnel-push-pop.at b/tests/tunnel-push-pop.at index
> 12fc1ef91..cf021c0cc 100644
> --- a/tests/tunnel-push-pop.at
> +++ b/tests/tunnel-push-pop.at
> @@ -628,20 +628,22 @@ AT_CHECK([
> AT_CHECK([ovs-vsctl -- set Interface p0 options:tx_pcap=p0.pcap])
>
> packet=50540000000a505400000009123
> -
> encap=f8bc124434b6aa55aa55000008004500003200004000401134060101025
> 80101025c83a917c1001e00000000655800007b00
> +dnl Source port is based on a packet hash, so it might be different
> +depending dnl on compiler flags and CPU. Masked with '....'.
> +encap=f8bc124434b6aa55aa5500000800450000320000400040113406010102
> 5801010
> +25c....17c1001e00000000655800007b00
>
> dnl Output to tunnel from a int-br internal port.
> dnl Checking that the packet arrived and it was correctly encapsulated.
> AT_CHECK([ovs-ofctl add-flow int-br
> "in_port=LOCAL,actions=debug_slow,output:2"])
> AT_CHECK([ovs-appctl netdev-dummy/receive int-br "${packet}4"]) -
> OVS_WAIT_UNTIL([test `ovs-pcap p0.pcap | grep "${encap}${packet}4" | wc -l` -
> ge 1])
> +OVS_WAIT_UNTIL([test `ovs-pcap p0.pcap | egrep "${encap}${packet}4" |
> +wc -l` -ge 1])
> dnl Sending again to exercise the non-miss upcall path.
> AT_CHECK([ovs-appctl netdev-dummy/receive int-br "${packet}4"]) -
> OVS_WAIT_UNTIL([test `ovs-pcap p0.pcap | grep "${encap}${packet}4" | wc -l` -
> ge 2])
> +OVS_WAIT_UNTIL([test `ovs-pcap p0.pcap | egrep "${encap}${packet}4" |
> +wc -l` -ge 2])
>
> dnl Output to tunnel from the controller.
> AT_CHECK([ovs-ofctl -O OpenFlow13 packet-out int-br CONTROLLER
> "debug_slow,output:2" "${packet}5"]) -OVS_WAIT_UNTIL([test `ovs-pcap
> p0.pcap | grep "${encap}${packet}5" | wc -l` -ge 1])
> +OVS_WAIT_UNTIL([test `ovs-pcap p0.pcap | egrep "${encap}${packet}5" |
> +wc -l` -ge 1])
>
> dnl Datapath actions should not have tunnel push action.
> AT_CHECK([ovs-appctl dpctl/dump-flows | grep -q tnl_push], [1])
> ---
>
> I can send it formally once I'm back from PTO.
>
The patch works for the "-msse4.2" flag and now the test-case passes for all
the builds.
> One other thing that I noticed is that all tests are failing with '-msse4.2'
> and without dpdk due to extra warnings from an AVX512 code. I think we need
> to lower them down to DBG, as they are not very useful, but breaks tests and
> make issues like this drain in the flood of test failures.
> If you can fix that, that would be great.
>
The fix is pushed on the patchworks.
http://patchwork.ozlabs.org/project/openvswitch/list/?series=264765
Regards
Amber
> Best regards, Ilya Maximets.
>
> >
> > Regards
> > Amber
> >
> >> -----Original Message-----
> >> From: Ilya Maximets <i.maxim...@ovn.org>
> >> Sent: Tuesday, September 21, 2021 7:00 PM
> >> To: Amber, Kumar <kumar.am...@intel.com>; Ilya Maximets
> >> <i.maxim...@ovn.org>; ovs-dev@openvswitch.org;
> >> tony.vanderp...@alliedtelesis.co.nz
> >> Cc: Stokes, Ian <ian.sto...@intel.com>; Van Haaren, Harry
> >> <harry.van.haa...@intel.com>
> >> Subject: Re: Unit Test Failure Report to OVS ML
> >>
> >> On 9/21/21 14:51, Amber, Kumar wrote:
> >>> Hi Ilya,
> >>>
> >>> The Test-case failure is not related to AVX512 or any patches we are
> >>> directly
> >> failing on "master" latest of OVS with no patches on top of it.
> >>> I am still trying to figure out or root cause the issue, we tested
> >>> the master on 4
> >> different servers, and all fails on the same test-case.
> >>
> >> This sounds very weird. How do you build it?
> >>
> >>>
> >>> Regards
> >>> Amber
> >>>
> >>>> -----Original Message-----
> >>>> From: Ilya Maximets <i.maxim...@ovn.org>
> >>>> Sent: Monday, September 20, 2021 5:05 PM
> >>>> To: Amber, Kumar <kumar.am...@intel.com>; ovs-dev@openvswitch.org;
> >>>> i.maxim...@ovn.org; tony.vanderp...@alliedtelesis.co.nz
> >>>> Cc: Stokes, Ian <ian.sto...@intel.com>; Van Haaren, Harry
> >>>> <harry.van.haa...@intel.com>
> >>>> Subject: Re: Unit Test Failure Report to OVS ML
> >>>>
> >>>> On 9/20/21 12:35, Amber, Kumar wrote:
> >>>>> Hi all,
> >>>>>
> >>>>> The following commit ID with the following description added a
> >>>>> test case for
> >>>> "tunnel-push-pop" test-suit by the name: "tunnel_push_pop -
> >>>> packet_out debug_slow" has been found to be failing on the latest
> >>>> master
> >> branch.
> >>>>>
> >>>>> ## ------------------------------- ## ## openvswitch 2.16.90 test
> >>>>> suite. ## ## ------------------------------- ##
> >>>>> 779: tunnel_push_pop - packet_out debug_slow FAILED
> >>>>> (ovs-macros.at:242)
> >>>>>
> >>>>> ## ------------- ##
> >>>>> ## Test results. ##
> >>>>> ## ------------- ##
> >>>>>
> >>>>> ERROR: 1 test was run,
> >>>>> 1 failed unexpectedly.
> >>>>>
> >>>>> We did some investigation, and the matching is the cause of the failure.
> >>>>>
> >>>>> ./ovs-macros.at:242: hard failure
> >>>>>
> >>>>> 779. tunnel-push-pop.at:598: 779. tunnel_push_pop - packet_out
> >>>>> debug_slow (tunnel-push-pop.at:598): FAILED (ovs-macros.at:242)
> >>>>>
> >>>>> Commit patch: 7e6b41ac8d9d183655be96795b529adeb33aeb47
> >>>>>
> >>>>> dpif-netdev: Fix crash when PACKET_OUT is metered.
> >>>>>
> >>>>> When a PACKET_OUT has output port of OFPP_TABLE, and the rule
> >>>>> table includes a meter and this causes the packet to be deleted,
> >>>>> execute with a clone of the packet, restoring the original packet
> >>>>> if it is changed by the execution.
> >>>>>
> >>>>> Add tests to verify the original issue is fixed, and that the fix
> >>>>> doesn't break tunnel processing.
> >>>>>
> >>>>> Would the authors of the patch investigate why the test is failing?
> >>>>>
> >>>>> Regards
> >>>>> Amber
> >>>>
> >>>> Hi.
> >>>>
> >>>> I can't reproduce the issue. I re-run the test 10 times on 2 of my
> >>>> systems and it works 10/10 without any issues. And none of our CI
> >>>> systems has issues with this test.
> >>>>
> >>>> The patch that added the test should not affect packet matching as
> >>>> it only changes the execution of actions, just to avoid the crash
> >>>> under certain conditions, and it tries to do that with least amount
> >>>> of side effects possible. So, this patch should not be a root
> >>>> cause. Maybe the new test case just uncovered a different issue in
> >>>> packet
> matching?
> >>>>
> >>>> The test itself was carefully crafted to catch a particular issue
> >>>> where packet is not encapsulated, while it should be. And the test
> >>>> itself
> >> seems solid.
> >>>>
> >>>> Does it still fail for you, if you revert code changes from the
> >>>> patch but keep the aforementioned unit test (this test is not for
> >>>> the crash itself, so it should pass without the change in the patch)?
> >>>>
> >>>> Anyway, what does "the matching is the cause of the failure" mean?
> >>>> Are you testing with avx512 enabled? If so, doesn't autovalidator
> >>>> tell you what the issue is?
> >>>>
> >>>> Best regards, Ilya Maximets.
> >
_______________________________________________
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
