Hello, These patches can make ptest test case work, RP has suggested we write a tool to do the regression check on ptest result, I think that the use case is like:
$ bitbake <image> -ctestiamge # Suppose we add ptest to default test cases in the future # Upgrade a recipe form V1.0 to V1.1 $ bitbake <image> -ctestiamge # Run the test again Then the regression check tool can report what's different (passed, failed, skipped) between V1.0 and V1.1. Currently, I'm not sure about where to save the ptest results, I saved it to ${WORKDIR}/testimage/ptest_log atm, e.g.: tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/testimage/ptest_log But it will be removed after -cclean, then no regression check can be made, so I'd like to save the ptest results to DEPLOY_DIR_IMAGE if no objections, and make -cclean not remove them. (Or only keep the latest 2 results). And I'm not sure where to add the regression check/tool, maybe one of: 1) Add a separate tool in oe-core/scripts, this can make it easy to do regression check among different build directories, and runtime/cases/_ptest.py can invoke it. 2) Add it to runtime/cases/_ptest.py directly 3) Add it to buildhistory I prefer the first one, please feel free to give your comments. // Robert The following changes since commit ef68005a8c527e9b1d05b7769f0ec8ebe9ec3f91: webkitgtk: Upgrade to 2.16.5 (2017-07-17 13:49:04 +0100) are available in the git repository at: git://git.openembedded.org/openembedded-core-contrib rbt/ptest http://cgit.openembedded.org/openembedded-core-contrib/log/?h=rbt/ptest Robert Yang (5): oeqa/targetcontrol.py: simplify checking for qemu_use_kvm runtime/cases/_ptest.py: revive it oeqa/utils/logparser.py: add skip status runtime/cases/_ptest.py: add skip status runtime/cases/_ptest.py: rename it to ptest.py meta/lib/oeqa/runtime/cases/_ptest.py | 103 ---------------------------------- meta/lib/oeqa/runtime/cases/ptest.py | 77 +++++++++++++++++++++++++ meta/lib/oeqa/targetcontrol.py | 5 +- meta/lib/oeqa/utils/logparser.py | 5 +- 4 files changed, 82 insertions(+), 108 deletions(-) delete mode 100644 meta/lib/oeqa/runtime/cases/_ptest.py create mode 100644 meta/lib/oeqa/runtime/cases/ptest.py -- 2.11.0 -- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core