On 29 Jun 2021, at 20:49, Amber, Kumar wrote:
> Hi Flavio,
>
> Comments Inline.
>
>> -----Original Message-----
>> From: Flavio Leitner <f...@sysclose.org>
>> Sent: Tuesday, June 29, 2021 11:49 PM
>> To: Amber, Kumar <kumar.am...@intel.com>
>> Cc: Eelco Chaudron <echau...@redhat.com>; d...@openvswitch.org;
>> i.maxim...@ovn.org
>> Subject: Re: [ovs-dev] [v4 07/12] test/sytem-dpdk: Add unit test for mfex
>> autovalidator
>>
>> On Tue, Jun 29, 2021 at 05:11:00PM +0000, Amber, Kumar wrote:
>>> Hi Eelco, Flavio,
>>>
>>> Pls find my replies Inline
>>>
>>>> -----Original Message-----
>>>> From: Flavio Leitner <f...@sysclose.org>
>>>> Sent: Tuesday, June 29, 2021 7:51 PM
>>>> To: Eelco Chaudron <echau...@redhat.com>
>>>> Cc: Amber, Kumar <kumar.am...@intel.com>; Van Haaren, Harry
>>>> <harry.van.haa...@intel.com>; d...@openvswitch.org;
>>>> i.maxim...@ovn.org
>>>> Subject: Re: [ovs-dev] [v4 07/12] test/sytem-dpdk: Add unit test for
>>>> mfex autovalidator
>>>>
>>>> On Tue, Jun 29, 2021 at 03:50:22PM +0200, Eelco Chaudron wrote:
>>>>>
>>>>>
>>>>> On 28 Jun 2021, at 4:57, Flavio Leitner wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>>
>>>>>> On Thu, Jun 17, 2021 at 09:57:49PM +0530, Kumar Amber wrote:
>>>>>>> Tests:
>>>>>>> 6: OVS-DPDK - MFEX Autovalidator
>>>>>>> 7: OVS-DPDK - MFEX Autovalidator Fuzzy
>>>>>>>
>>>>>>> Added a new directory to store the PCAP file used in the tests
>>>>>>> and a script to generate the fuzzy traffic type pcap to be used
>>>>>>> in fuzzy unit test.
>>>>>>
>>>>>>
>>>>>> I haven't tried this yet but am I right that these tests are
>>>>>> going to pass a pcap to send traffic in a busy loop for 5
>>>>>> seconds in the first case and 20 seconds in the second case?
>>>>>>
>>>>>> I see that when autovalidator is set OVS will crash if one
>>>>>> implementation returns a different value, so I wonder why we
>>>>>> need to run for that long.
>>>>>
>>>>> I think we should remove the assert (already suggested by Harry),
>>>>> so it will not crass by accident if someone selects autovalidator
>>>>> in the field (and runs into an issue).
>>>>> Failure will then be detected by the ERROR log entries on shutdown.
>>>>
>>>> That's true for the testsuite, but not in production as there is
>>>> nothing to disable that.
>>>>
>>>> Perhaps if autovalidator detects an issue, it should log an ERROR
>>>> level log to report to testsuite, disable the failing mode and make
>>>> sure OVS is either in default or in another functional mode.
>>>
>>> So I have put the following :
>>> Removed the assert
>>> Allow the Auto-validator to run for all implementation and for a full
>> batch
>>> Document error via Vlog_Error
>>> Set the auto-validator to default {Scalar} when returning out in case
>> of failure.
>>
>> Sounds like a plan to me.
>> Is that okay with you Eelco?
ACK, this sounds good to me.
>>>>> I’m wondering if there is another way than a simple delay, as
>>>>> these tend to
>>>> cause issues later on. Can we check packets processed or something?
>>>>
>>>> Yeah, maybe we can pass all packets like 5x at least.
>>>
>>> Sure I will try to find something to do it more nicely.
>>> But just a thought keeping it 20sec allows for a full-stabilization and also
>> thorough testing of stability as well.
>>> So keeping it may not be just a bad idea.
>>
>> The issue is that if every test decides to delay seconds, the testsuite
>> becomes impractical. We have removed 'sleep' over time. Instead, we have
>> functions to wait for a certain cmdline output, or some event.
>> Yes, there are still some left to be fixed.
>>
>> Back to the point, maybe there is a signal of some sort we can get that
>> indicates the stability you're looking for.
>>
>
> I agree to the point and I am looking for a singal but currently due to
> assert removal we don’t have any marker.
> To Minimize the time, I did analysis of the time taken in each test-case :
>
> 1) for the simple test-case we don’t need the 5sec wait time as PCAP only
> contains one traffic or each type.
> 2) for fuzzy we do need at least 5sec for all 10k packets to be sent at-least
> 2x and also stability.
Guess you know the number of packets, so you could check in a 1sec delay loop
the number of received packets for the flows, and exit if you saw enough. And
just to be sure, you terminate after 5~10 seconds?
_______________________________________________
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
