A few points worth noting: 1/ This test only uses the ping application and there are no advanced options, so my first guess is distro should not matter, but I could be wrong.
2/ This will not be fixed by increasing the sleep time before the check below; there are extra drops for broadcast packets. There are 6 extra lines of them for this failure – see the below o/p. The trailing checks in the test may fail of course. If these are noise, then they can be filtered. But an explanation would be needed. I assume you added sleep because you want to make sure datapath flows are established and stats are as expected Again, adding more sleep for this part will not help here. 3/ Your group needs to be able to reproduce the issue in house. We used standard Ubuntu 14.04 and 16.04 and also Ubuntu with upgraded kernel. 4/ The system tests should pass the far majority of the time. Portion of test: # Ping between N1 and N3, via the L2 GRE tunnel between br-in1 and br-in3 NS_CHECK_EXEC([ns1], [ping -q -c 3 -i 0.3 -w 2 $N3_IP | FORMAT_PING], [0], [dnl 3 packets transmitted, 3 received, 0% packet loss, time 0ms ]) sleep 1 AT_CHECK([ ovs-appctl dpctl/dump-flows | strip_used | grep -v ipv6 | grep -v arp |sort ], [0], [flow-dump from non-dpdk interfaces: recirc_id(0),in_port(10),packet_type(ns=0,id=0),eth(dst=aa:55:00:00:00:03),eth_type(0x0800),ipv4(src=10.0.0.1,dst=10.0.0.3,proto=47,frag=no), packets:2, bytes:272, used:0.0s, actions:set(ipv4(src=30.0.0.1,dst=30.0.0.3)),tnl_pop(14) recirc_id(0),in_port(11),packet_type(ns=0,id=0),eth(dst=aa:55:00:00:00:03),eth_type(0x0800),ipv4(frag=no), packets:2, bytes:272, used:0.0s, actions:13 recirc_id(0),in_port(12),packet_type(ns=0,id=0),eth(dst=aa:55:00:00:00:01),eth_type(0x0800),ipv4(frag=no), packets:2, bytes:244, used:0.0s, actions:11 recirc_id(0),in_port(13),packet_type(ns=0,id=0),eth(dst=aa:55:00:00:00:02),eth_type(0x0800),ipv4(frag=no), packets:2, bytes:244, used:0.0s, actions:12 relevant error output lines recirc_id(0),in_port(10),packet_type(ns=0,id=0),eth(dst=aa:55:00:00:00:03),eth_type(0x0800),ipv4(src=10.0.0.1,dst=10.0.0.3,proto=47,frag=no), packets:2, bytes:272, used:0.0s, actions:set(ipv4(src=30.0.0.1,dst=30.0.0.3)),tnl_pop(14) +recirc_id(0),in_port(10),packet_type(ns=0,id=0),eth_type(0x0800),ipv4(dst=255.255.255.255/128.0.0.0,frag=no), packets:2, bytes:684, used:0.0s, actions:drop recirc_id(0),in_port(11),packet_type(ns=0,id=0),eth(dst=aa:55:00:00:00:03),eth_type(0x0800),ipv4(frag=no), packets:2, bytes:272, used:0.0s, actions:13 +recirc_id(0),in_port(11),packet_type(ns=0,id=0),eth(dst=ff:ff:ff:ff:ff:ff),eth_type(0x0800),ipv4(frag=no), packets:2, bytes:684, used:0.0s, actions:drop recirc_id(0),in_port(12),packet_type(ns=0,id=0),eth(dst=aa:55:00:00:00:01),eth_type(0x0800),ipv4(frag=no), packets:2, bytes:244, used:0.0s, actions:11 +recirc_id(0),in_port(12),packet_type(ns=0,id=0),eth(dst=ff:ff:ff:ff:ff:ff),eth_type(0x0800),ipv4(frag=no), packets:2, bytes:684, used:0.0s, actions:drop recirc_id(0),in_port(13),packet_type(ns=0,id=0),eth(dst=aa:55:00:00:00:02),eth_type(0x0800),ipv4(frag=no), packets:2, bytes:244, used:0.0s, actions:12 +recirc_id(0),in_port(13),packet_type(ns=0,id=0),eth(dst=ff:ff:ff:ff:ff:ff),eth_type(0x0800),ipv4(frag=no), packets:2, bytes:684, used:0.0s, actions:drop recirc_id(0),in_port(15),packet_type(ns=0,id=0),eth_type(0x0800),ipv4(dst=192.168.10.30,tos=0/0x3,frag=no), packets:2, bytes:196, used:0.0s, actions:tnl_push(tnl_port(14),header(size=38,type=3,eth(dst=aa:55:00:00:00:03,src=aa:55:00:00:00:01,dl_type=0x0800),ipv4(src=10.0.0.1,dst=10.0.0.3,proto=47,tos=0,ttl=64,frag=0x4000),gre((flags=0x0,proto=0x6558))),out_port(5)) recirc_id(0),in_port(17),packet_type(ns=0,id=0),eth_type(0x0800),ipv4(dst=192.168.10.10,tos=0/0x3,frag=no), packets:2, bytes:196, used:0.0s, actions:pop_eth,tnl_push(tnl_port(14),header(size=38,type=3,eth(dst=aa:55:00:00:00:02,src=aa:55:00:00:00:03,dl_type=0x0800),ipv4(src=30.0.0.3,dst=30.0.0.2,proto=47,tos=0,ttl=64,frag=0x4000),gre((flags=0x0,proto=0x800))),out_port(7)) recirc_id(0),in_port(5),packet_type(ns=0,id=0),eth_type(0x0800),ipv4(frag=no), packets:2, bytes:272, used:0.0s, actions:8 recirc_id(0),in_port(6),packet_type(ns=0,id=0),eth_type(0x0800),ipv4(frag=no), packets:2, bytes:244, used:0.0s, actions:9 recirc_id(0),in_port(7),packet_type(ns=0,id=0),eth_type(0x0800),ipv4(frag=no), packets:2, bytes:244, used:0.0s, actions:10 recirc_id(0),in_port(8),packet_type(ns=0,id=0),eth(dst=aa:55:00:00:00:01),eth_type(0x0800),ipv4(src=20.0.0.2,dst=20.0.0.1,proto=47,frag=no), packets:2, bytes:244, used:0.0s, actions:set(ipv4(src=10.0.0.2,dst=10.0.0.1)),tnl_pop(14) +recirc_id(0),in_port(8),packet_type(ns=0,id=0),eth_type(0x0800),ipv4(dst=255.255.255.255/128.0.0.0,frag=no), packets:2, bytes:684, used:0.0s, actions:drop recirc_id(0),in_port(9),packet_type(ns=0,id=0),eth(dst=aa:55:00:00:00:02),eth_type(0x0800),ipv4(src=30.0.0.3,dst=30.0.0.2,proto=47,frag=no), packets:2, bytes:244, used:0.0s, actions:set(ipv4(src=20.0.0.3,dst=20.0.0.2)),tnl_pop(14) +recirc_id(0),in_port(9),packet_type(ns=0,id=0),eth_type(0x0800),ipv4(dst=255.255.255.255/128.0.0.0,frag=no), packets:3, bytes:1026, used:0.0s, actions:drop tunnel(src=10.0.0.2,dst=10.0.0.1,flags(-df-csum)),recirc_id(0),in_port(14),packet_type(ns=1,id=0x800),ipv4(dst=192.168.10.10,frag=no), packets:2, bytes:168, used:0.0s, actions:push_eth(src=00:00:00:00:00:00,dst=aa:55:aa:55:00:01),15 tunnel(src=20.0.0.3,dst=20.0.0.2,flags(-df-csum)),recirc_id(0),in_port(14),packet_type(ns=1,id=0x800),ipv4(dst=192.168.10.10,tos=0/0x3,frag=no), packets:2, bytes:168, used:0.0s, actions:tnl_push(tnl_port(14),header(size=38,type=3,eth(dst=aa:55:00:00:00:01,src=aa:55:00:00:00:02,dl_type=0x0800),ipv4(src=20.0.0.2,dst=20.0.0.1,proto=47,tos=0,ttl=64,frag=0x4000),gre((flags=0x0,proto=0x800))),out_port(6)) tunnel(src=30.0.0.1,dst=30.0.0.3,flags(-df-csum)),recirc_id(0),in_port(14),packet_type(ns=0,id=0),eth(dst=aa:55:aa:55:00:03),eth_type(0x0800),ipv4(dst=192.168.10.30,frag=no), packets:2, bytes:196, used:0.0s, actions:17 On 7/7/17, 12:23 AM, "Zoltán Balogh" <zoltan.bal...@ericsson.com> wrote: Hi, The script seems to be stopped at line 344 in system-userspace-packet-type-aware.at. Checking root privilege happens before that. There is a "sleep 1" command before line 344. Maybe it's too short on some systems. Darell, could you share the log file of this test case, please? It should be : tests/system-userspace-testsuite.dir/94/system-userspace-testsuite.log If the log file too large, then you could try to grep for the relevant part: grep -A 40 "\--- " tests/system-userspace-testsuite.dir/94/system-userspace-testsuite.log Thank you, Zoltan > -----Original Message----- > From: ovs-dev-boun...@openvswitch.org [mailto:ovs-dev-boun...@openvswitch.org] On Behalf Of Yang, Yi Y > Sent: Friday, July 07, 2017 3:36 AM > To: Darrell Ball <db...@vmware.com>; Eric Garver <e...@erig.me> > Cc: 'd...@openvswitch.org' <d...@openvswitch.org>; simon.hor...@netronome.com; Jiri Benc <jb...@redhat.com> > Subject: Re: [ovs-dev] [PATCH v4 0/7] Packet type aware pipeline > > I think it needs root privilege to run. > > -----Original Message----- > From: ovs-dev-boun...@openvswitch.org [mailto:ovs-dev-boun...@openvswitch.org] On Behalf Of Darrell Ball > Sent: Friday, July 7, 2017 5:06 AM > To: Eric Garver <e...@erig.me> > Cc: 'd...@openvswitch.org' <d...@openvswitch.org>; simon.hor...@netronome.com; Jiri Benc <jb...@redhat.com> > Subject: Re: [ovs-dev] [PATCH v4 0/7] Packet type aware pipeline > > btw, I asked Andy to check on his Trusty system and he observes the same failure. > > On 7/6/17, 1:39 PM, "Darrell Ball" <db...@vmware.com> wrote: > > > > On 7/6/17, 1:22 PM, "Eric Garver" <e...@erig.me> wrote: > > On Thu, Jul 06, 2017 at 03:59:51PM +0000, Darrell Ball wrote: > > Patch 4 added unit tests > > One test was added to system userspace. > > I ran it many times and it has never passed. > > Is something missing in the test or supporting code ? > > Works for me. Maybe there is a timing problem? > > Thanks > There are races in the system tests, but those tests will pass on a second attempt. > This is 100% failure. > > OVS: current master (fb16fec66498) > > Tip of master - same for me > > Kernel: 4.12.0-rc7+ > > I tried 2 kernels 4.2.0 and 3.19 – always fails in both. > > OS: RHEL 7.4-ish > > I really hope the distro is not relevant here > If you have reason to think so, pls speak up > > > > > > ## ------------------------------ ## > > ## openvswitch 2.7.90 test suite. ## > > ## ------------------------------ ## > > 94: ptap - triangle bridge setup with L2 and L3 GRE tunnels FAILED (system-userspace-packet-type- > aware.at:344) > > > > ## ------------- ## > > ## Test results. ## > > ## ------------- ## > > > > ERROR: 1 test was run, > > 1 failed unexpectedly. > > ## ------------------------------------------- ## > > ## system-userspace-testsuite.log was created. ## > > ## ------------------------------------------- ## > > > > Please send `tests/system-userspace-testsuite.log' and all information you think might help: > > > > To: <b...@openvswitch.org> > > Subject: [openvswitch 2.7.90] system-userspace-testsuite: 94 failed > > > > You may investigate any problem if you feel able to do so, in which > > case the test suite provides a good starting point. Its output may > > be found below `tests/system-userspace-testsuite.dir'. > > > > make: *** [check-system-userspace] Error 1 > > make: Leaving directory `/home/dball/openvswitch/ovs/_gcc' > > > > > > > > > > On 6/27/17, 2:29 PM, "ovs-dev-boun...@openvswitch.org on behalf of Ben Pfaff" <ovs-dev- > boun...@openvswitch.org on behalf of b...@ovn.org> wrote: > > > > On Fri, Jun 23, 2017 at 04:47:48PM +0000, Zoltán Balogh wrote: > > > > > > This series was started by Ben Pfaff, v3 can be found here: > > > https://urldefense.proofpoint.com/v2/url?u=https- > 3A__patchwork.ozlabs.org_patch_778070_&d=DwIFAw&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih- > uZnsw&m=YZOYZFR6QorkvkAZXXYPq7Z9SWuK0lZg9EH6ND2me04&s=01fMZLg-sq_Rf_2X9TcxmtDN0Vx2hS2v4BABA77en0w&e= > > > https://urldefense.proofpoint.com/v2/url?u=https- > 3A__patchwork.ozlabs.org_patch_778071_&d=DwIFAw&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih- > uZnsw&m=YZOYZFR6QorkvkAZXXYPq7Z9SWuK0lZg9EH6ND2me04&s=5fpQZh33OQ_0AGjRuJNQqvS-1E6yhFoD5TxUoLayzwM&e= > > > https://urldefense.proofpoint.com/v2/url?u=https- > 3A__patchwork.ozlabs.org_patch_778076_&d=DwIFAw&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih- > uZnsw&m=YZOYZFR6QorkvkAZXXYPq7Z9SWuK0lZg9EH6ND2me04&s=rqnhEoIuJ9rECQaXZafZGrAW9us8y184GCexqK8VUxQ&e= > > > https://urldefense.proofpoint.com/v2/url?u=https- > 3A__patchwork.ozlabs.org_patch_778072_&d=DwIFAw&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih- > uZnsw&m=YZOYZFR6QorkvkAZXXYPq7Z9SWuK0lZg9EH6ND2me04&s=p5Fnh2SQ8fUrNF0V8ChNRQyQLmbjiuvKRd1UUwE93s8&e= > > > https://urldefense.proofpoint.com/v2/url?u=https- > 3A__patchwork.ozlabs.org_patch_778074_&d=DwIFAw&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih- > uZnsw&m=YZOYZFR6QorkvkAZXXYPq7Z9SWuK0lZg9EH6ND2me04&s=02CPoSKkiGGsFX4neL8TpGa75VUPaaeNYWz_2WbqBsQ&e= > > > https://urldefense.proofpoint.com/v2/url?u=https- > 3A__patchwork.ozlabs.org_patch_778073_&d=DwIFAw&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih- > uZnsw&m=YZOYZFR6QorkvkAZXXYPq7Z9SWuK0lZg9EH6ND2me04&s=eMl7m4NpJSD07nXTN_oOdQFvysmJnclSRw39GgGg9BQ&e= > > > https://urldefense.proofpoint.com/v2/url?u=https- > 3A__patchwork.ozlabs.org_patch_778075_&d=DwIFAw&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih- > uZnsw&m=YZOYZFR6QorkvkAZXXYPq7Z9SWuK0lZg9EH6ND2me04&s=71zSrD_kcUjZwuKvjfeqam_rdCpeXuxtm4TW_SsZ_3w&e= > > > > > > Ben's series is based on this one: > > > https://urldefense.proofpoint.com/v2/url?u=https- > 3A__patchwork.ozlabs.org_patch_770490_&d=DwIFAw&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih- > uZnsw&m=YZOYZFR6QorkvkAZXXYPq7Z9SWuK0lZg9EH6ND2me04&s=W1AcbVTr4qKOBEfQuMSmVH-Illfhs0R3TYJs1s9HtfU&e= > > > https://urldefense.proofpoint.com/v2/url?u=https- > 3A__patchwork.ozlabs.org_patch_770487_&d=DwIFAw&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih- > uZnsw&m=YZOYZFR6QorkvkAZXXYPq7Z9SWuK0lZg9EH6ND2me04&s=EHhVEHj-Pvdl0I609BkK_Z44CV-641MVWlLy7rZH6Qo&e= > > > https://urldefense.proofpoint.com/v2/url?u=https- > 3A__patchwork.ozlabs.org_patch_770495_&d=DwIFAw&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih- > uZnsw&m=YZOYZFR6QorkvkAZXXYPq7Z9SWuK0lZg9EH6ND2me04&s=izZUroix8O9Bcg5XNCmvsEseCctZfho3ovxvZRz0rwI&e= > > > https://urldefense.proofpoint.com/v2/url?u=https- > 3A__patchwork.ozlabs.org_patch_770498_&d=DwIFAw&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih- > uZnsw&m=YZOYZFR6QorkvkAZXXYPq7Z9SWuK0lZg9EH6ND2me04&s=VFf2BDMVyA1OU5h0hCBJFboZtp_U2NlqmPYkLDGnWnE&e= > > > https://urldefense.proofpoint.com/v2/url?u=https- > 3A__patchwork.ozlabs.org_patch_770488_&d=DwIFAw&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih- > uZnsw&m=YZOYZFR6QorkvkAZXXYPq7Z9SWuK0lZg9EH6ND2me04&s=3_rtktraZwXdr6VkYz8ujpi_eV9MDKc8EwCDk43nnLg&e= > > > https://urldefense.proofpoint.com/v2/url?u=https- > 3A__patchwork.ozlabs.org_patch_770489_&d=DwIFAw&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih- > uZnsw&m=YZOYZFR6QorkvkAZXXYPq7Z9SWuK0lZg9EH6ND2me04&s=1OhDZfiGpq3GF4Vie9KKwKudHEAHpG8nj8qTMqlmKV8&e= > > > > > > v1->v2: > > > - Squash fixup patches. > > > - Apply changes agreed with Jan. > > > - Not yet done: Figure out whether to really show packet_type in (some) > > > match_format() output. > > > - New patch at the end unsuccessfully tries to re-enable packet-aware > > > test. Either I don't have enough insight yet, or it just reveals a > > > bug or two. > > > - 4 new patches at beginning. First one is trivial. Next 3 are intended > > > to make it easier to debug the packet aware test that is still failing. > > > Jan, you don't have to feel obligated to review these if you feel they > > > are off-topic; I will get separate reviews. > > > > > > v2->v3: > > > - Drop first two patches, which have been applied to master. > > > - Drop patches 3 and 4, which have been moved to a new series. > > > - Drop last patch, which was incomplete. I'll hope that Jan or Zoltan > > > can pick it back up. > > > - Apply Jan's and Zoltan's comments on a few other patches. > > > > > > v3->v4: > > > - Rebase to recent origin/master (0722f3410). > > > - Re-introduce packet-type aware unit tests. > > > - Unwildcard dl_type only for packet_type=PT_ETH. > > > - Apply comments. > > > > Thanks. I applied this series to master. > > _______________________________________________ > > dev mailing list > > d...@openvswitch.org > > https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_mailman_listinfo_ovs- > 2Ddev&d=DwIFAw&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih- > uZnsw&m=YZOYZFR6QorkvkAZXXYPq7Z9SWuK0lZg9EH6ND2me04&s=eFnoeVgPHfExQ3X4kGuiRhqxnn-iw7rq2RoY9fvwj4I&e= > > > > > > _______________________________________________ > > dev mailing list > > d...@openvswitch.org > > https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_mailman_listinfo_ovs- > 2Ddev&d=DwIDAw&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih- > uZnsw&m=msEftCAccuIB4BWrvoXOXgDDsp7LUK5CHuIscL6JtsY&s=wbZ8iJo4O-gC9UmULTSz6NABL3Dmz-9d-Pg_28JNlmE&e= > > > > > _______________________________________________ > dev mailing list > d...@openvswitch.org > https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_mailman_listinfo_ovs-2Ddev&d=DwIGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=dGZmbKhBG9tJHY4odedsGA&m=L7nAsK74d2EKUrXtQVIVX1ExwBLsoe6W3XwyyU6lvi0&s=lEqDjzoYGuE8UKddAndoqJC_x_HwlzECADVds-2P0uU&e= > _______________________________________________ > dev mailing list > d...@openvswitch.org > https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_mailman_listinfo_ovs-2Ddev&d=DwIGaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=dGZmbKhBG9tJHY4odedsGA&m=L7nAsK74d2EKUrXtQVIVX1ExwBLsoe6W3XwyyU6lvi0&s=lEqDjzoYGuE8UKddAndoqJC_x_HwlzECADVds-2P0uU&e= _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev