>-----Original Message-----
>From: richard.pur...@linuxfoundation.org <richard.pur...@linuxfoundation.org>
>Sent: Tuesday, 23 April, 2019 10:31 PM
>To: Jain, Sangeeta <sangeeta.j...@intel.com>; 'yocto@yoctoproject.org'
><yocto@yoctoproject.org>; Eggleton, Paul <paul.eggle...@intel.com>; 'Michael
>Halstead' <mhalst...@linuxfoundation.org>; Erway, Tracey M
><tracey.m.er...@intel.com>; 'sjolley.yp...@gmail.com'
><sjolley.yp...@gmail.com>; 'openembedded-core-
>requ...@lists.openembedded.org' <openembedded-core-
>requ...@lists.openembedded.org>
>Cc: Sangal, Apoorv <apoorv.san...@intel.com>; Yeoh, Ee Peng
><ee.peng.y...@intel.com>; Graydon, Tracy <tracy.gray...@intel.com>;
>Tummalapalli, Vineela <vineela.tummalapa...@intel.com>
>Subject: Re: QA cycle report for 2.5.3 RC3
>
>On Thu, 2019-04-18 at 04:58 +0000, Jain, Sangeeta wrote:
>>
>> QA cycle report for 2.5.3 RC3:
>>
>> No high milestone defects.
>> Test results are available at following location:
>> · For results of all automated tests, refer to results at
>> public AB [1].
>> · For other test results, refer to attachment [2].
>> · For test report for test cases run by Intel and WR team,
>> refer attachment [3]
>> · For full test report, refer attachment [4]
>> · For ptest results, please refer to results at public AB [5]
>> · For ptest report, refer to attachment [6]
>> No new defects are found in this cycle.
>> Number of existing issues observed in this release is 2- toaster [7]
>> and Build-appliance [8] For ptest, regression data is not available
>> for this release. No timeout issues.
>> Test result report on Public AB shows no failures.
>
>I've been discussing with Tracy and Vineela how to best handle the test
>results for
>the release for 2.5.3. In the end I:
>
>a) Copied in the ptest results to the right place. This will happen
>automagically in
>all future builds:
>
>https://autobuilder.yocto.io/pub/releases/yocto-2.5.3/testresults/qemux86-64-
>ptest/
>
>b) Copied in the intel test results to their own directory:
>
>https://autobuilder.yocto.io/pub/releases/yocto-2.5.3/testresults-intel/
>
>c) Manually generated an updated test report for the combined results with the
>commands:
>
>$ cd /srv/autobuilder/autobuilder.yoctoproject.org/pub/releases/yocto-2.5.3
>$resulttool report . > testreport.txt
>
>d) Added a header to the report which consisted of the report from QA with
>details of the bugs etc.
>
>This means we have a top level testreport file which contains all the test
>information about the release.
>
>I'd like to make this the standard procedure for release. The release notes and
>announcement can refer to the test report included with the release and the
>test
>results as its there all together.
>
>Ultimately I'd like to improve the formatting of the report (separate out ptest
>regressions from the other regressions, maybe html, maybe graphs, better
>regression information) but that is something for the future.
>
>Does that work for everyone?
That looks great to me.
As a further improvement for sharing of test results, I am working to create a
wiki page for consolidated
test results. For 2.5.3, it can be viewed at:
https://wiki.yoctoproject.org/wiki/WW16_-_2019-04-18_-_Full_Test_Cycle_2.5.3_RC3
After you have added more results on AB, I will change some existing links on
this web page with the new ones.
Another thought I have is that since for any release, all data will be removed
from public AB after few days of release,
Should I include here a link to "yocto-testresults" git repo? It will make
this wiki page more useful for reference in future as well.
>
>Cheers,
>
>Richard
>
Thanks & Regards,
Sangeeta Jain
--
_______________________________________________
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
