Akhil Goyal <gak...@marvell.com> writes: > Hi Ciara, > > I see that CI is reporting one error in autotest. Can you please check > if it is because of your patchset? > > 9/96 DPDK:fast-tests / cycles_autotest FAIL 1.42s (exit status 255 or > signal 127 SIGinvalid)
Just to note, I'm looking into that test case a bit more. It would be nice if the failures were asserts that listed the values. I also wonder if it's possible that we hit some kind of delay on wake up (especially on busy systems like CI systems) and should have a flag to disable this. I can see how it's possible for this test case to give a false positive in some system configurations but not sure how best to remedy that situation. > 12:33:15 DPDK_TEST=cycles_autotest MALLOC_PERTURB_=44 > /home/jenkins-local/jenkins-agent/workspace/Ubuntu18.04-Unit-Test-DPDK/dpdk/build/app/test/dpdk-test > '-l 0-3' --file-prefix=cycles_autotest > ----------------------------------- output ----------------------------------- > stdout: > RTE>>cycles_autotest > > delay_us is not accurate: too long > Test Failed > > > Regards, > Akhil > >> -----Original Message----- >> From: Ciara Power <ciara.po...@intel.com> >> Sent: Wednesday, May 12, 2021 5:07 PM >> To: dev@dpdk.org >> Cc: declan.dohe...@intel.com; Akhil Goyal <gak...@marvell.com>; >> acon...@redhat.com; hemant.agra...@nxp.com; Anoob Joseph >> <ano...@marvell.com>; ruifeng.w...@arm.com; asoma...@amd.com; >> ajit.khapa...@broadcom.com; g.si...@nxp.com; roy.fan.zh...@intel.com; >> Ciara Power <ciara.po...@intel.com> >> Subject: [EXT] [PATCH v4 0/7] test: refactor crypto unit test framework >> >> External Email >> >> ---------------------------------------------------------------------- >> The current crypto unit test framework is not granular enough to >> accurately track unit test results. This is caused by one testcase >> in a suite actually running multiple testcases, but only returning >> one result. >> >> The approach taken in this patchset allows a test suite have a >> list of sub-testsuites, and/or a list of testcases as previously used. >> The unit test suite runner can then recursively iterate and run the >> sub-testsuites, until it reaches a suite with testcases, >> and it then runs each testcase as it had done previously. >> In the case of a testsuite with both testcases and sub-testsuites, >> the testcases are executed first before iterating through the >> sub-testsuites. >> >> By allowing this further breakdown into sub-testsuites, >> a refactor of the crypto unit tests solves the issue of inaccurate >> reporting, as sub-testsuites can be used in place of the testcases >> that had multiple testcases hidden on a sub level. >> The blockcipher tests previously had these hidden testcases, >> but are now sub-testsuites that are dynamically created and added to a >> parent test suite, allowing for each testcase status to be reported >> directly to the runner. >> The cryptodev test suite is broken down into smaller suites that are >> used as sub-testsuites, which allows for more flexibility choosing which >> sub-testsuites should run for the current device. >> The introduction of sub-testsuites also allows for more precise >> setup/teardown functions, that can check the capabilities required to >> run its testcases. >> >> For example, when running the cryptodev_aesni_mb_autotest, >> the parent Cryptodev Test Suite is executed. >> Various sub-testsuites are added to the parent test suite, such as >> the static suites of testcases that were once in the cryptodev_testsuite, >> and blockcipher suites. >> The unit test runner can then run the Cryptodev parent test suite, >> which in turn will run the sub-testsuites. >> >> The user is now required to create vdevs via EAL commandline args, >> this is no longer done within the test app for crypto autotests. >> >> Documentation will need to be added at a later stage, >> adding to the test document that isn't yet merged. [1] >> >> [1] https://urldefense.proofpoint.com/v2/url?u=https- >> 3A__patchwork.dpdk.org_project_dpdk_patch_20210309155757.615536- >> 2D1-2Daconole- >> 40redhat.com_&d=DwIDAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=DnL7Si2wl_PR >> wpZ9TWey3eu68gBzn7DkPwuqhd6WNyo&m=XItKx7UnOATOiVC3kRLlV3Woh >> rLeZbmlqShu_py452U&s=BpOMigsNnvWMV739- >> fMxEvVERJieNSnfnnmZdiMLMJg&e= >> >> >> v4: >> - Fixed some return value bugs related to patch 4. >> - Rebased onto main. >> v3: >> - Added support for a testsuite having both a list of testcases, >> and a list of sub-testsuites. >> - Replaced PMD based parent testsuites with a cryptodev testsuite >> used by all autotests, with the exception of scheduler autotest. >> - Setup functions were added for all sub-testsuites, within which >> required capability support is checked. >> - The setup functions no longer create vdevs if needed, >> this must be done by the user when running the test. >> - Patch added to standardise return values for skipped testcases. >> v2: >> - Added macro in place of testcase/testsuite loops. >> - Added more detail in the summary output. >> - Moved testcase counts to the testsuite structure. >> - Flattened testsuite structure to remove union. >> - Added patch for fix of blockcipher test return value. >> - Squashed release note into last patch. >> >> Ciara Power (7): >> app/test: refactor of unit test suite runner >> test: introduce parent testsuite format >> test/crypto: refactor to use sub-testsuites >> test/crypto: replace unsupported with skipped >> test/crypto: move testsuite params to header file >> test/crypto: fix return value on test skipped >> test/crypto: dynamically build blockcipher suite >> >> app/test/test.c | 176 +- >> app/test/test.h | 23 +- >> app/test/test_cryptodev.c | 2333 ++++++++++++++---------- >> app/test/test_cryptodev.h | 32 + >> app/test/test_cryptodev_asym.c | 111 +- >> app/test/test_cryptodev_blockcipher.c | 423 ++++- >> app/test/test_cryptodev_blockcipher.h | 12 +- >> app/test/test_ipsec.c | 32 +- >> doc/guides/rel_notes/release_21_05.rst | 5 + >> 9 files changed, 2004 insertions(+), 1143 deletions(-) >> >> -- >> 2.25.1
