On 07/21/2017 08:32 PM, Richard Purdie wrote:
On Thu, 2017-07-20 at 10:09 +0800, Robert Yang wrote:
On 07/19/2017 04:16 PM, Robert Yang wrote:
Hello,
These patches can make ptest test case work, RP has suggested we
write a tool to
do the regression check on ptest result, I think that the use case
is like:
$ bitbake <image> -ctestiamge # Suppose we add ptest to default
test cases in the future
# Upgrade a recipe form V1.0 to V1.1
$ bitbake <image> -ctestiamge # Run the test again
Then the regression check tool can report what's different (passed,
failed,
skipped) between V1.0 and V1.1.
Currently, I'm not sure about where to save the ptest results, I
saved it to
${WORKDIR}/testimage/ptest_log atm, e.g.:
tmp/work/qemux86-poky-linux/core-image-minimal/1.0-
r0/testimage/ptest_log
I still have to think about more about where to put the test result.
Would buildhistory make sense for these since we already have a
mechanism to store results there between builds?
Thanks, I will add it to buildhistory.
But it will be removed after -cclean, then no regression check can
be made, so
I'd like to save the ptest results to DEPLOY_DIR_IMAGE if no
objections, and
make -cclean not remove them. (Or only keep the latest 2 results).
And I'm not sure where to add the regression check/tool, maybe one
of:
1) Add a separate tool in oe-core/scripts, this can make it easy to
do regression
check among different build directories, and
runtime/cases/_ptest.py can invoke it.
I'm leaning to add a script called ptest-regression-check which is
more flexible, the cases/ptest.py or buildhistory (also the user) can
run it when needed.
I'd either add a script, or perhaps an option to the buildhistory-diff?
Paul might have some adivce on that. I would like to understand if
there is a good reason we can't use buildhistory to at least help with
some of this?
Thanks, the script (I named it prc -- Ptest Result Compare or Ptest Regression
Compile) already done, I will investigate buildhistory to see how to merge it
into buildhistory.
// Robert
Cheers,
Richard
--
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
