It has been observed that useful information in regression report can be drowned in huge regression lists which are often false-positives (for example, a whole set of tests has been temporarily disabled). Additionally, the regression report usually starts with not-so-useful information when searching for regressions (basically, big lists of tests which have not regressed).
This series aims to improve regression display by rearranging info order as well as bringing a default limit to how many changes are displayed per base/target comparison. This new version follows up suggestions discussed in [1]. One consequence of this series is the new available option for yocto_testresults_query: - yocto_testresults_query 4.3_M1 4.3_M2 -> will display at most 50 regressions per test type - yocto_testresults_query -l 10 4.3_M1 4.3_M2 -> override the display limit and reduce it to 10 regressions per test type - yocto_testresults_query -l 0 4.3_M1 4.3_M2 -> disable the display limit, print all regressions An example of regression report with those modifications can be found at [2] Note: this series does not take care yet of improving base/target pairs selection based on machine/distro (issue raised by Ross in [1]), because I still need to have a better look at this before tweaking anything there. Changes in v2: - use the display limit for each test type instead of using it as a global limit - display regressions summaries before detailed list - display regressions first, display matched/improved tests after, as "additional data" - Reduce less useful info size - fix testresult selftests [1] https://lore.kernel.org/yocto/90d7fe40-32eb-4d67-b099-9e0c764fc...@arm.com/ [2] https://pastebin.com/JHFLF0sK Alexis Lothoré (4): scripts/resulttool: limit the number of changes displayed per test scripts/resulttool: rearrange regressions report order scripts/resulttool: make additional info more compact scripts/yocto_testresults_query: add option to change display limit scripts/lib/resulttool/regression.py | 60 +++++++++++++++++++++++----- scripts/yocto_testresults_query.py | 13 ++++-- 2 files changed, 60 insertions(+), 13 deletions(-) -- 2.42.0
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#189606): https://lists.openembedded.org/g/openembedded-core/message/189606 Mute This Topic: https://lists.openembedded.org/mt/102120368/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-