Re: [yocto] QA notification for completed autobuilder build (yocto-2.6.3.rc1)
Hello All, Intel and WR YP QA is planning for QA execution for YP build yocto-2.6.3.rc1. We are planning to execute following tests for this cycle: OEQA-manual tests for following module: 1. OE-Core 2. BSP-hw 3. BSP-Qemu Runtime auto test for following platforms: 1. MinnowTurbot 32-bit 2. NUC 7 3. NUC 6 4. Edgerouter 5. MPC8315e-rdb 6. Beaglebone ETA for completion is next Wednesday, August 07. -Original Message- From: Pokybuild User [mailto:pokybu...@debian10-ty-3.yocto.io] Sent: Thursday, August 1, 2019 11:04 PM To: yocto@yoctoproject.org Cc: ota...@ossystems.com.br; yi.z...@windriver.com; Sangal, Apoorv ; Yeoh, Ee Peng ; Chan, Aaron Chun Yew ; Ang, Chin Huat ; richard.pur...@linuxfoundation.org; akuster...@gmail.com; sjolley.yp...@gmail.com; Jain, Sangeeta Subject: QA notification for completed autobuilder build (yocto-2.6.3.rc1) A build flagged for QA (yocto-2.6.3.rc1) was completed on the autobuilder and is available at: https://autobuilder.yocto.io/pub/releases/yocto-2.6.3.rc1 Build hash information: bitbake: 4ce92f43eeac6a4bfd06e8567fa6891614b5b3b0 meta-gplv2: aabc30f3bd03f97326fb8596910b94639fea7575 meta-intel: 278f0b5fa2f1836635178bdc770e035d77ae03c7 meta-mingw: 6556cde16fbdf42c7edae89bb186e5ae4982eff5 oecore: d3d3f443039b03f1200a14bfe99f985592632018 poky: cb26830f765f03309c0663352cc5849491271be8 This is an automated message from the Yocto Project Autobuilder Git: git://git.yoctoproject.org/yocto-autobuilder2 Email: richard.pur...@linuxfoundation.org -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
[yocto] QA cycle report for 2.8 M2 RC1
Hello All, This is the full report for 2.8 M2 RC1: https://git.yoctoproject.org/cgit/cgit.cgi/yocto-testresults-contrib/tree/?h=zeus=d550d40646e1ca78a4b5d106e1b20831208b4c37 === Summary No high milestone defects. Two new defects are found in this cycle, both beaglebone and genericx86-64 facing shutdown hanging (BUG id:13461) and ptest failed (BUG id:13465). One timeout issues observed in ptests was mdadm (BUG id: 13368). === Bugs https://bugzilla.yoctoproject.org/show_bug.cgi?id=13461 https://bugzilla.yoctoproject.org/show_bug.cgi?id=13465 https://bugzilla.yoctoproject.org/show_bug.cgi?id=13368 -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] QA notification for completed autobuilder build (yocto-2.8_M2.rc1)
Hello All, Intel and WR YP QA is planning for QA execution for YP build yocto-2.8_M2.rc1. We are planning to execute following tests for this cycle: OEQA-manual tests for following module: 1. OE-Core 2. BSP-hw 3. BSP-Qemu Runtime auto test for following platforms: 1. MinnowTurbot 32-bit 2. Coffee Lake 3. NUC 7 4. NUC 6 5. Edgerouter 6. MPC8315e-rdb 7. Beaglebone ETA for completion is next Wednesday, July 31. -Original Message- From: pokybuild@centos7-ty-3.localdomain [mailto:pokybuild@centos7-ty-3.localdomain] Sent: Wednesday, July 24, 2019 11:21 AM To: yocto@yoctoproject.org Cc: ota...@ossystems.com.br; yi.z...@windriver.com; Sangal, Apoorv ; Yeoh, Ee Peng ; Chan, Aaron Chun Yew ; Ang, Chin Huat ; richard.pur...@linuxfoundation.org; akuster...@gmail.com; sjolley.yp...@gmail.com; Jain, Sangeeta Subject: QA notification for completed autobuilder build (yocto-2.8_M2.rc1) A build flagged for QA (yocto-2.8_M2.rc1) was completed on the autobuilder and is available at: https://autobuilder.yocto.io/pub/releases/yocto-2.8_M2.rc1 Build hash information: bitbake: f5ea06fc2b6713c9f8e85ecf7cb981ae9a84d896 meta-gplv2: 1e2480e50f34e55bdfd5e06f98441e03a3752d5a meta-intel: 3227874941bb8f9b706d6057b1de3997881bdffd meta-mingw: 3fa43aa92f1a1c90304f6f4f49270915d1392cce oecore: e0c3436241afca93f107e325d1b9ffcdebf706cd poky: 835f7eac0610325e906591cd81890bebe8627580 This is an automated message from the Yocto Project Autobuilder Git: git://git.yoctoproject.org/yocto-autobuilder2 Email: richard.pur...@linuxfoundation.org -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] QA notification for completed autobuilder build (yocto-2.8_M1.rc1)
Richard, Thanks for informing! Best regards, Yeoh Ee Peng -Original Message- From: richard.pur...@linuxfoundation.org [mailto:richard.pur...@linuxfoundation.org] Sent: Tuesday, June 11, 2019 6:18 AM To: Poky Build User ; yocto@yoctoproject.org Cc: ota...@ossystems.com.br; yi.z...@windriver.com; Sangal, Apoorv ; Yeoh, Ee Peng ; Chan, Aaron Chun Yew ; Ang, Chin Huat ; akuster...@gmail.com; sjolley.yp...@gmail.com; Jain, Sangeeta Subject: Re: QA notification for completed autobuilder build (yocto-2.8_M1.rc1) On Mon, 2019-06-10 at 21:38 +, Poky Build User wrote: > A build flagged for QA (yocto-2.8_M1.rc1) was completed on the > autobuilder and is available at: > > > https://autobuilder.yocto.io/pub/releases/yocto-2.8_M1.rc1 > I've triggered an rc2 since one of the autobuilder workers messed up the mips builds. Cheers, Richard -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] QA notification for completed autobuilder build (yocto-2.6.2.rc3)
Hi Richard, Sorry for the confusion. Yes, public autobuilder had already covered all automated runtime qemu tests. We do not plan to run any automated runtime qemu tests. For automated selftest, we plan to run a subset of selftest tests for a list of qemu architecture (qemu-shutdown & meta-ide). For these subset of selftest, could we enabled them to be run by pulic autobuilder as it only required changing MACHINE configuration to cover all qemu architectures? Hope this help to clarify the confusion. Please let us know if you have any more questions. Best regards, Yeoh Ee Peng -Original Message- From: Jain, Sangeeta Sent: Thursday, April 4, 2019 10:19 AM To: richard.pur...@linuxfoundation.org; Poky Build User ; yocto@yoctoproject.org; sjolley.yp...@gmail.com; Yi Zhao Cc: ota...@ossystems.com.br; Chan, Aaron Chun Yew ; akuster...@gmail.com; Yeoh, Ee Peng ; Sangal, Apoorv Subject: RE: [yocto] QA notification for completed autobuilder build (yocto-2.6.2.rc3) Hi Richard, >-Original Message- >From: richard.pur...@linuxfoundation.org > >Sent: Wednesday, 3 April, 2019 5:20 PM >To: Jain, Sangeeta ; Poky Build User >; yocto@yoctoproject.org; >sjolley.yp...@gmail.com; Yi Zhao >Cc: ota...@ossystems.com.br; Chan, Aaron Chun Yew >; akuster...@gmail.com; Yeoh, Ee Peng >; Sangal, Apoorv >Subject: Re: [yocto] QA notification for completed autobuilder build >(yocto- >2.6.2.rc3) > >Hi, > >This is good information and helps a lot, thanks! I had a question below. > >On Wed, 2019-04-03 at 08:06 +, Jain, Sangeeta wrote: >> Intel and WR YP QA is now working on QA execution for YP build 2.6.2 >> RC3. We are planning to execute following tests for this cycle: >> >> OEQA-Manual tests for following modules: >> 1. SDK >> 2. Eclipse-plugin >> 3. Kernel >> 4. Toaster >> 5. Bitbake (oe-core) >> 6. Build-appliance >> 7. Crops >> 8. Compliance-test >> 9. Bsp-hw >> 10. Bsp-qemu >> >> Runtime auto test for following platforms: >> >> 1. MinnowTurbo 32bit >> 2. MinnowTurbot 64 bit >> 3. NUC 7 >> 4. NUC 6 >> 5. Edgerouter >> 6. MPC8315e-rdb >> 7. Beaglebone >> 8. Qemuarm >> 9. Qemuarm-64 >> 10. Qemuppc >> 11. Qemumips >> 12. Qemumips64 > >I'm wondering if there is any difference between items 8-12 here and >the runtime testing we run on the public autobuilder? Is there some >testing we're missing on the public autobuilder? We will be running oe-selftest for machines 8-12, which is missing in autobuilder results. Also, I don't see any ptest results for 2.6.2.rc3 on public autobuilder. Do you have any plans to run them and share the results on autobuilder? > >Cheers, > >Richard > Thanks & Regards, Sangeeta Jain -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] Proposed new QA process
Hi Richard, Agreed to you inputs! I had created a new QA process wiki page to compile all inputs. https://wiki.yoctoproject.org/wiki/New_QA_process > Do we need to add something to the results to indicate which results are from > "who"? (i.e. from > the public autobuilder, Intel QA, WR QA, any other > sources?). We may want to add something >such as a parameter to the > resulttool store command so we can add this info to results? I had enhanced the resulttool following your suggestion. Patches submitted to openembedded-core mailing list. Best regards, Yeoh Ee Peng -Original Message- From: richard.pur...@linuxfoundation.org [mailto:richard.pur...@linuxfoundation.org] Sent: Wednesday, April 3, 2019 4:18 AM To: Yeoh, Ee Peng ; 'yocto@yoctoproject.org' Subject: Re: Proposed new QA process This is a good start. I've filled out some details below and had some questions. On Tue, 2019-04-02 at 03:28 +, Yeoh, Ee Peng wrote: > Given the new QA tooling (resulttool) available to manage QA test > results and reporting, here was the proposed new QA process. > > The new QA process consists below: > · Test Trigger > · Test Planning > · Test Execution > · Test Result Store > · Test Monitoring & Reporting > · Release Decision > > Test Trigger: Each QA team will subscribe to QA notification email > (request through Richard Purdie). The list of notifications is maintained on config.json in the yocto- autobuilder-helper which has a branch per release. > Test Planning: The lead QA team no longer need to setup Testopia and > wiki page. Each QA team (eg. intel, windriver, etc) will perform > planning on what extra tests they plan to run and when they'll send > the results back, then send these planning information as acknowledge > email to QA stakeholders (eg. Richard Purdie, Stephen Jolley) and the > lead QA team. Each QA team can refer to OEQA for automated and manual > test cases for their planning. What form will these notifications take? Is there a time limit for when they'll be received after a QA notification email? Can we agree to include an estimate of execution time in this notification? > Test Execution: Each QA team will execute the planned extra tests. To > make sure test result from the test execution could fully integrated > to the new QA tooling (resulttool for test result management and > reporting/regression), execute OEQA automated tests and OEQA manual > tests through resulttool (refer > https://wiki.yoctoproject.org/wiki/Resulttool#manualexecution). > > Test Result Store: Each QA team will store test result to the remote > yocto-testresults git repository using resulttool (refer > https://wiki.yoctoproject.org/wiki/Resulttool#store), then send the QA > completion email (include new defects information) to both QA > stakeholder and the lead QA team. Each QA team will request write > access to remote yocto-testresults git repository (request through > Richard Purdie). Ultimately, yes but I want to have things working before we have multiple people pushing things there. Need to document the commands used here to add the results too. Do we need to add something to the results to indicate which results are from "who"? (i.e. from the public autobuilder, Intel QA, WR QA, any other sources?). We may want to add something such as a parameter to the resulttool store command so we can add this info to results? > Test Monitoring & Reporting: QA stakeholder will monitor testing > progress from remote yocto-testresults git repository using resulttool > (refer https://wiki.yoctoproject.org/wiki/Resulttool#report). Once > every QA team completed the test execution, the lead QA team will > create QA test report and regression using resulttool. Send email > report to QA stakeholder and public yocto mailing list. We should document the command to do this. I'm also wondering where the list of stakeholders would be maintained? Is Intel volunteering to help with this role for the time being or does someone else need to start doing this. A key thing we need to document here is that someone, somewhere in this process needs to: a) Open a bug for each unique QA test failure b) List the bugs found in the QA report c) Notice any ptest timeouts and file bugs for those too > Release Decision: QA stakeholder will make the final decision for > release. The release decision will ultimately be made by the Yocto Project TSC who will review the responses to the QA report (including suggestions from QA) and make a go/nogo decision on that information (exact process to be agreed by the TSC). Cheers, Richard --- Begin Message --- v01: Enable merging results where both base and target are directory. v02: Follow suggestion from RP to reused the
Re: [yocto] QA notification for completed autobuilder build (yocto-2.6.2.rc3)
Hi Armin, For 2.6.2.rc3, QA team shall try to run the execution by referring to the manual testcases from master. Thanks, Yeoh Ee Peng -Original Message- From: akuster808 [mailto:akuster...@gmail.com] Sent: Wednesday, April 3, 2019 5:50 PM To: Jain, Sangeeta ; Poky Build User ; yocto@yoctoproject.org; richard.pur...@linuxfoundation.org; sjolley.yp...@gmail.com; Yi Zhao Cc: ota...@ossystems.com.br; Chan, Aaron Chun Yew ; Yeoh, Ee Peng ; Sangal, Apoorv Subject: Re: [yocto] QA notification for completed autobuilder build (yocto-2.6.2.rc3) Hello, On 4/3/19 1:06 AM, Jain, Sangeeta wrote: > Intel and WR YP QA is now working on QA execution for YP build 2.6.2 RC3. We > are planning to execute following tests for this cycle: > > OEQA-Manual tests for following modules: > 1.SDK > 2.Eclipse-plugin > 3.Kernel > 4.Toaster > 5.Bitbake (oe-core) > 6.Build-appliance > 7.Crops > 8.Compliance-test > 9.Bsp-hw > 10. Bsp-qemu I did not backport the manual test from master or thud. Are these the original set? regards, Armin > Runtime auto test for following platforms: > > 1.MinnowTurbo 32bit > 2.MinnowTurbot 64 bit > 3.NUC 7 > 4.NUC 6 > 5.Edgerouter > 6.MPC8315e-rdb > 7.Beaglebone > 8.Qemuarm > 9.Qemuarm-64 > 10. Qemuppc > 11. Qemumips > 12. Qemumips64 > > Other auto tests: > > 1.Qemu-selftest > 2.Qemu-meta-ide > > ETA for completion is Wednesday, 10 April. > > Thanks & Regards, > Sangeeta Jain > >> -Original Message- >> From: yocto-boun...@yoctoproject.org >> On Behalf Of Poky Build User >> Sent: Thursday, 28 March, 2019 11:30 AM >> To: yocto@yoctoproject.org >> Cc: ota...@ossystems.com.br; Chan, Aaron Chun Yew >> ; akuster...@gmail.com >> Subject: [yocto] QA notification for completed autobuilder build >> (yocto-2.6.2.rc3) >> >> >> A build flagged for QA (yocto-2.6.2.rc3) was completed on the >> autobuilder and is available at: >> >> >>https://autobuilder.yocto.io/pub/releases/yocto-2.6.2.rc3 >> >> >> Build hash information: >> >> bitbake: 7c1eb51d1e8a4c5f39bf9dddf05fb0b3598da72b >> eclipse-poky-neon: cd86f167be58a11b289af4ef236b4adec57ec316 >> eclipse-poky-oxygen: 210c58c5a7147127f9c840718c6cd2a56e871718 >> meta-gplv2: aabc30f3bd03f97326fb8596910b94639fea7575 >> meta-intel: 27dadcfc7bc0de70328b02fecb841608389d22fc >> meta-mingw: 6556cde16fbdf42c7edae89bb186e5ae4982eff5 >> meta-qt3: 23d7543ebd7e82ba95cbe19043ae4229bdb3b6b1 >> meta-qt4: b37d8b93924b314df3591b4a61e194ff3feb5517 >> oecore: 45032e30be70503faeee468159b216031b729309 >> poky: faeb366bc3eb322f5f203cfe08dc4cf529a822e9 >> >> >> >> This is an automated message from the Yocto Project Autobuilder >> Git: git://git.yoctoproject.org/yocto-autobuilder2 >> Email: richard.pur...@linuxfoundation.org >> >> >> >> -- >> ___ >> yocto mailing list >> yocto@yoctoproject.org >> https://lists.yoctoproject.org/listinfo/yocto -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
[yocto] Proposed new QA process
Given the new QA tooling (resulttool) available to manage QA test results and reporting, here was the proposed new QA process. The new QA process consists below: * Test Trigger * Test Planning * Test Execution * Test Result Store * Test Monitoring & Reporting * Release Decision Test Trigger: Each QA team will subscribe to QA notification email (request through Richard Purdie). Test Planning: The lead QA team no longer need to setup Testopia and wiki page. Each QA team (eg. intel, windriver, etc) will perform planning on what extra tests they plan to run and when they'll send the results back, then send these planning information as acknowledge email to QA stakeholders (eg. Richard Purdie, Stephen Jolley) and the lead QA team. Each QA team can refer to OEQA for automated and manual test cases for their planning. Test Execution: Each QA team will execute the planned extra tests. To make sure test result from the test execution could fully integrated to the new QA tooling (resulttool for test result management and reporting/regression), execute OEQA automated tests and OEQA manual tests through resulttool (refer https://wiki.yoctoproject.org/wiki/Resulttool#manualexecution). Test Result Store: Each QA team will store test result to the remote yocto-testresults git repository using resulttool (refer https://wiki.yoctoproject.org/wiki/Resulttool#store), then send the QA completion email (include new defects information) to both QA stakeholder and the lead QA team. Each QA team will request write access to remote yocto-testresults git repository (request through Richard Purdie). Test Monitoring & Reporting: QA stakeholder will monitor testing progress from remote yocto-testresults git repository using resulttool (refer https://wiki.yoctoproject.org/wiki/Resulttool#report). Once every QA team completed the test execution, the lead QA team will create QA test report and regression using resulttool. Send email report to QA stakeholder and public yocto mailing list. Release Decision: QA stakeholder will make the final decision for release. -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
[yocto] QA cycle report for 2.7 M1 RC1
Hello All, This is the full report for 2.7 M1 RC1: https://wiki.yoctoproject.org/wiki/WW01_-_2019-01-02_-_Full_Test_Cycle_2.7_M1_RC1 === Summary All planned tests were executed. Total Test Executed - 3659 Passed Test - 3638 Failed Test - 18 Blocked Test - 3 There were zero high milestone defect. Team had found 3 new defects coming from Toaster [1] [2] [3]. For ptest, team had confirmed that there was no tests which passed in 2.6 M4 rc1 while failed in current 2.7 M1 rc1, but team had found that the qa-tools/ptest-parser script used to generate comparison table was not working as expected (ticket created to fix this). For CROPS manual testing, team was able to execute CROPS testing automatically by utilizing the enhancement from testimage that enable qemu slirp to run eSDK automated tests for CROPS. Propose to remove manual crops testcases inside oeqa/manual directory. === QA-Hints No high milestone defect. === Bugs New Bugs [1] Bug 13102 - [Bug][QA 2.7 M1 rc1][Toaster] All layers are not getting populated after clicking "View compatible layers" on project page https://bugzilla.yoctoproject.org/show_bug.cgi?id=13102 [2] Bug 13103 - [Bug][QA 2.7 M1 rc1][Toaster] "Recipes" table and "machines" table are not getting populated after clicking on imported layer as well as after clicking Machines Tab on project page https://bugzilla.yoctoproject.org/show_bug.cgi?id=13103 [3] Bug 13104 - [Bug][QA 2.7 M1 rc1][Toaster] All "Image Recipes" are not getting populated after clicking "Image Recipes" on project page https://bugzilla.yoctoproject.org/show_bug.cgi?id=13104 Regards Ee Peng -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
[yocto] QA cycle report for 2.5.1 RC1
Hello All, This is the full report for 2.5.1 RC1: https://wiki.yoctoproject.org/wiki/WW33_-_2018-08-16-_Full_Test_Cycle_-_2.5.1_rc1 === Summary All planned tests were executed except the SDK automated tests were skipped in qemumips (testrun#9868-9869), syslog automated tests were skipped due to sysklogd not installed (testrun#9882-9883, 9878-9880), and xorg automated test was skipped on qemuarm64 (testrun#9867). There were zero high milestone defect. Team had found 4 new defects, where selftest wic test cases produce random results on debian9 [1], audio not working on monitor with hdmi [2], toaster failed to create new custom image after adding new layer [3], and build appliance stuck at bitbake parsing after manual set ip after bootup build appliance image [4]. For busybox ptest, the passed % decreased due to the same failure detected during 2.6 M1 RC1, where the workaround provided to resolve the proxy issue had fixed the failure. === QA-Hints No high milestone defect. === Bugs New Bugs [1] Bug 12885 - [2.5.1 rc1] selftest wic test cases produce random results on debian9 https://bugzilla.yoctoproject.org/show_bug.cgi?id=12885 [2] Bug 12893 - [QA 2.5.1.rc1][BSP][NUC7][Case 266 & 267]HDMI audio for .wav and .ogg file not audible on monitor's audio port with HDMI https://bugzilla.yoctoproject.org/show_bug.cgi?id=12893 [3] Bug 12887 - [QA 2.5.1.rc1 ][Toaster]:Search for rpi-basic-image, click on 'add layer' button https://bugzilla.yoctoproject.org/show_bug.cgi?id=12887 [4] Bug 12894 - [Bug][QA 2.5 rc1 ][Build Appliance]:bitbake stuck at parsing https://bugzilla.yoctoproject.org/show_bug.cgi?id=12894 Regards Ee Peng -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] QA cycle report for 2.6 M2 RC1
Hi Richard, There are 23 tests were not executed. 13 of these tests were related to SDK automated tests (part of it belong to galculator test). 6 of these tests (selftest/cases/distrodata.py), where it was configured to be skipped by Autobuilder (we will removed these tests from planned tests starting new cycle). 2 of these tests (runtime/cases/oe_syslog.py), were skipped due to tests can't run with sysklogd installed. 1 of the runtime/xorg tests running on qemuarm64 were not executed by the testimage. For the eSDK automated tests issue, we had filed for bug. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12875 Please let me know if any more question. Thanks, Yeoh Ee Peng -Original Message- From: richard.pur...@linuxfoundation.org [mailto:richard.pur...@linuxfoundation.org] Sent: Thursday, August 9, 2018 4:29 PM To: Yeoh, Ee Peng ; 'yocto@yoctoproject.org' ; Jolley, Stephen K ; Eggleton, Paul Cc: Sangal, Apoorv ; Kirkiris, Nectar Subject: Re: QA cycle report for 2.6 M2 RC1 On Thu, 2018-08-09 at 05:57 +, Yeoh, Ee Peng wrote: > === Summary > > All planned tests were executed except a few SDK automated tests were > skipped in Autobuilder (testrun# 9830-9836, 9796-9797). Team found > that these SDK automated tests were running successfully in core- > image-sato-sdk image but not the existing core-image-sato used by > Autobuilder, investigating why these automated tests were skipped in > core-image-sato and no message available in testsdk logfile to explain > why it was skipped. I think Ross just sent a patch for this, assuming it was the galculator test? > Team also found that eSDK devtool automated testcase(s) were skipped > unexpectedly due to the unknown error on testcase dependency logic, > temporary removing the dependency logic and successfully executed the > devtool tests. We should probably have a bug to track and resolve this issue as it hasn't been resolved yet as far as I know. I believe its a problem introduced by the parallelism changes. We may have to rework the dependency logic. > New Bugs > [1] Bug 12866 - [2.6 M2 rc1] wic test_fix_size failed > https://bugzilla.yoctoproject.org/show_bug.cgi?id=12866 > > [2] Bug 12864 - [2.6 M2 rc1] buildhistorydifftest failed > https://bugzilla.yoctoproject.org/show_bug.cgi?id=12864 > > [3] Bug 12869 - [2.6 M2 rc1] libxml test cases failed > https://bugzilla.yoctoproject.org/show_bug.cgi?id=12869 The good news is that these have all already been fixed in master and are understood issues with good resolutions. I don't believe any of them are enough to warrant an rc2. My recommendation is we release rc1 subject to my question below. Why does the QA report not show 100% of tests were completed in all cases? We do need to take the list of bugs in the QA report itself and see if we can resolve some of them for the final release, Stephen, your help in driving/tracking that would be appreciated. Cheers, Richard -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
[yocto] QA cycle report for 2.6 M2 RC1
Hello All, This is the full report for 2.6 M2 RC1: https://wiki.yoctoproject.org/wiki/WW32_-_2018-08-07_-_Full_Test_Cycle_2.6_M2_rc1 === Summary All planned tests were executed except a few SDK automated tests were skipped in Autobuilder (testrun# 9830-9836, 9796-9797). Team found that these SDK automated tests were running successfully in core-image-sato-sdk image but not the existing core-image-sato used by Autobuilder, investigating why these automated tests were skipped in core-image-sato and no message available in testsdk logfile to explain why it was skipped. Team also found that eSDK devtool automated testcase(s) were skipped unexpectedly due to the unknown error on testcase dependency logic, temporary removing the dependency logic and successfully executed the devtool tests. There were zero high milestone defect. Team had found 3 new defects, where buildhistorydifftest and wic test_fix_size failure [9] [10] and ptest for libxml failed in current release but passed in previous 2.6 M1 rc1. === QA-Hints No high milestone defect. === Bugs New Bugs [1] Bug 12866 - [2.6 M2 rc1] wic test_fix_size failed https://bugzilla.yoctoproject.org/show_bug.cgi?id=12866 [2] Bug 12864 - [2.6 M2 rc1] buildhistorydifftest failed https://bugzilla.yoctoproject.org/show_bug.cgi?id=12864 [3] Bug 12869 - [2.6 M2 rc1] libxml test cases failed https://bugzilla.yoctoproject.org/show_bug.cgi?id=12869 Regards Yeoh Ee Peng -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
[yocto] QA cycle report for 2.2.4 RC1
Hello All, This is the full report for 2.2.4.rc1: https://wiki.yoctoproject.org/wiki/WW28_-_2018-07-13_-_Full_Test_Cycle_2.2.4_rc1 === Summary All planned tests were executed. There were zero high milestone defect. Team had found 3 new defects [1] [2] [3] where runtime test_parselogs testcase failed with error message inside dmesg log. While executing the runtime smart package manager tests on target platform to install psplash, team discovered that the target platform screen was splash with Yocto logo and the Yocto logo remain after the tests completed, need further investigation to verify if this was caused by post installation script from psplash. For performance test, regression on 2.2.3.rc2 faced with error where SRCREV defined for distcc was no longer available. For pTest, both glib-2.0 and util-linux passrate decreased as compared to 2.2.3.rc2 as some new tests available in 2.2.4.rc1 were failing. === QA-Hints Found 3 non critical defects. === Bugs New Bugs [1] Bug 12854 - [2.2.4.rc1][BSP Auto] [genericx86-64 on MMAX64bit] [test_parselogs] blk_update_request: I/O error, dev loop0 https://bugzilla.yoctoproject.org/show_bug.cgi?id=12854 [2] Bug 12853 - [2.2.4.rc1][BSP Auto] [genericx86-64 on NUC6][test_parselogs] drm:intel_dp_link_training_clock_recovery] *ERROR* too many voltage retries, give up https://bugzilla.yoctoproject.org/show_bug.cgi?id=12853 [3] Bug 12852 - [2.2.4.rc1][BSP Auto] [genericx86-64-WIC on NUC6][test_parselogs] snd_hda_intel :00:1f.3: failed to add i915 component master https://bugzilla.yoctoproject.org/show_bug.cgi?id=12852 Regards Ee Peng -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
[yocto] QA cycle report for 2.3.4 RC1
Hello All, This is the full report for 2.3.4.rc1: https://wiki.yoctoproject.org/wiki/WW25_-_2018-06-22-_Full_Test_Cycle_-_2.3.4_rc1 === Summary All planned tests were executed. There were zero high milestone defect. Team had found 3 new defects where yocto-bsp failed to create bootable image with new bsp layer [1], QEMU ended unexpectedly during do_testimage:core-image-lsb-sdk [2], & valgrind ptest passed in 2.3.3.rc1 failed at current release [3]. === QA-Hints Found 3 non critical defects. === Bugs New Bugs [1] Bug 12846 - [QA 2.3.4rc1][Meta-Yocto][Case 309, Case 310]: After creating BSP layer using yocto-bsp, the image created not able to boot up as local variable 'qbsys' referenced before assignment https://bugzilla.yoctoproject.org/show_bug.cgi?id=12846 [2] Bug 12812 - [2.3.4 rc1] Qemu ended unexpectedly during do_testimage:core-image-lsb-sdk https://bugzilla.yoctoproject.org/show_bug.cgi?id=12812 [3] Bug 12847 - [2.3.4 rc1] valgrind ptest failed https://bugzilla.yoctoproject.org/show_bug.cgi?id=12847 Regards Ee Peng -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
[yocto] QA cycle report for 2.6 M1 RC1
Hello All, This is the full report for 2.6 M1 RC1: https://wiki.yoctoproject.org/wiki/WW27_-_2018-07-02_-_Full_Test_Cycle_2.6_M1_rc1 === Summary 99% of planned tests were executed except automated SDK test for buildgalculator (testrun# 9747-9753) and automated SDK image tests for core-image-sato-sdk-qemumips (testrun# 9713-9714), where those tests were excluded from the automated test suites and it was still under investigation why they were excluded. There were zero high milestone defect. Team had found 9 new defects where Edgerouter [1] & Beaglebone Black [3] defects were resolved. Team found that pTest for busybox and valgrind had testcases passed in previous 2.5 M4 rc1 but failed during this release [8] [9]. Team had also found that existing automated xorg test was passed successfully while it was not able to catch matchbox window manager issue on qemumips [4], thus team had added manual graphic tests as temporary solution. === QA-Hints Two medium+ defects where 1 was resolved and 1 under design. === Bugs New Bugs [1] Bug 12790 - [2.6 M1] Edgerouter can not boot https://bugzilla.yoctoproject.org/show_bug.cgi?id=12790 [2] Bug 12804 - [QA 2.6 M1 rc1 ][Build Appliance]: proxy issue in VM ware https://bugzilla.yoctoproject.org/show_bug.cgi?id=12804 [3] Bug 12795 - [2.6 M1] gcc/g++ doesn't work on beaglebone black https://bugzilla.yoctoproject.org/show_bug.cgi?id=12795 [4] Bug 12806 - [2.6 M1 rc1 ][BSP][Test Run 9708]: Qemuppc image was not booting with graphic even though runtime/xorg.py test passed https://bugzilla.yoctoproject.org/show_bug.cgi?id=12806 [5] Bug 12832 - [ 2.6 M1 rc1 ][BSP][Test case 267]: audio and video does not play in media player[Mturbot x86-64 and NUC7] https://bugzilla.yoctoproject.org/show_bug.cgi?id=12832 [6] Bug 12802 - [Yocto-2.6_M1.RC1] Crosstap doesn't work on 2.6 M1 RC1 https://bugzilla.yoctoproject.org/show_bug.cgi?id=12802 [7] Bug 12813 - [QA 2.6 M1 rc1 ][Toaster]:Build stopped without error, to terminate by ctrl+c https://bugzilla.yoctoproject.org/show_bug.cgi?id=12813 [8] Bug 12836 - [2.6 M1 RC1] busybox ptest failed https://bugzilla.yoctoproject.org/show_bug.cgi?id=12836 [9] Bug 12837 - [2.6 M1 RC1] valgrind ptest failed https://bugzilla.yoctoproject.org/show_bug.cgi?id=12837 Regards Ee Peng -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
[yocto] QA cycle report for 2.4.3 RC2
Hello All, This is the full report for 2.4.3 RC2: https://wiki.yoctoproject.org/wiki/WW24_-_2018-06-12-_Full_Test_Cycle_-_2.4.3_rc2 === Summary The QA cycle for release 2.4.3 RC2 was completed. Team had found that virtual/kernel build time had increased by 10% and 18% accordingly on Centos 7 and Ubuntu 1604 performance machines. Team found that libxml2 pTest was failed. No other new bugs were discovered. === QA-Hints QA shows no major bug. === Bugs New Bugs [1] Bug 12782 - [2.4.3.rc2] libxml2 ptest failed https://bugzilla.yoctoproject.org/show_bug.cgi?id=12782 Regards Ee Peng -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] Release Candidate Build for yocto-2.2.4.rc1 now available.
Hi all, Build for 2.2.4.rc1 is ready. Please get ready for the QA. Testopia and wiki page were ready. https://wiki.yoctoproject.org/wiki/2.2_QA_Status Thanks, Yeoh Ee Peng -Original Message- From: richard.pur...@intel.com [mailto:richard.pur...@intel.com] Sent: Thursday, May 24, 2018 9:33 PM To: yocto Cc: pi...@toganlabs.com; ota...@ossystems.com.br; Zhao, Yi (Wind River) ; Graydon, Tracy ; stephano.cet...@linux.intel.com; Sangal, Apoorv ; Yeoh, Ee Peng ; Chan, Aaron Chun Yew ; Chang, Rebecca Swee Fun ; Ang, Chin Huat ; Jolley, Stephen K ; Jordan, Robin L Subject: Release Candidate Build for yocto-2.2.4.rc1 now available. A release candidate build for yocto-2.2.4.rc1 is now available at: https://autobuilder.yocto.io/pub/releases/yocto-2.2.4.rc1 Please begin QA on this build as soon as possible. Build hash information: meta-intel : c781510a5a6b45e60cc32b6614ddcce3f1452121 meta-qt4 : f389368dc86e745df14cab9eeb9a94bc02bd273e meta-mingw : f5f676d9ee9f8e205363eace7a846a0360998634 meta-qt3 : f33b73a9563f2dfdfd0ee37b61d65d90197a456f meta-gplv2 : d7687d404bbc9ba3f44ec43ea8828d9071033513 poky : df76669bdd70aa0d903c4211dde731221c7e756d This is an automated message from\nThe Yocto Project Autobuilder Git: git://git.yoctoproject.org/yocto-autobuilder -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] Release Candidate Build for yocto-2.3.4.rc1 now available.
Hi all, Build for 2.3.4.rc1 is ready. Please get ready for the QA. Testopia and wiki page were ready. https://wiki.yoctoproject.org/wiki/2.3_QA_Status Thanks, Ee Peng -Original Message- From: richard.pur...@linuxfoundation.org [mailto:richard.pur...@linuxfoundation.org] Sent: Tuesday, May 22, 2018 11:36 PM To: yocto Cc: pi...@toganlabs.com; ota...@ossystems.com.br; Zhao, Yi (Wind River) ; Graydon, Tracy ; stephano.cet...@linux.intel.com; Sangal, Apoorv ; Yeoh, Ee Peng ; Chan, Aaron Chun Yew ; Chang, Rebecca Swee Fun ; Ang, Chin Huat ; Jolley, Stephen K ; Jordan, Robin L Subject: Release Candidate Build for yocto-2.3.4.rc1 now available. A release candidate build for yocto-2.3.4.rc1 is now available at: https://autobuilder.yocto.io/pub/releases/yocto-2.3.4.rc1 Please begin QA on this build as soon as possible. Build hash information: meta-intel : 9b37952d6af36358b6397cedf3dd53ec8962b6bf meta-qt4 : 88989ae3abe98b30089e7518d3adabe990c40a10 meta-mingw : c6db1934fc43fb06b93aff3e5634d89cc2b65b77 meta-qt3 : f33b73a9563f2dfdfd0ee37b61d65d90197a456f meta-gplv2 : 627147994943dd562cc294b7ccfd44325744956d poky : ebb42af2829edfca1a23c7a51a431c656ffc2090 This is an automated message from The Yocto Project Autobuilder Git: git://git.yoctoproject.org/yocto-autobuilder -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] Release Candidate Build for yocto-2.4.3.rc2 now available.
Hi all, Build for 2.4.3.rc2 is ready. Please get ready for the QA. Thanks, Yeoh, Ee Peng -Original Message- From: richard.pur...@linuxfoundation.org [mailto:richard.pur...@linuxfoundation.org] Sent: Thursday, May 24, 2018 4:16 PM To: yocto Cc: pi...@toganlabs.com; ota...@ossystems.com.br; Zhao, Yi (Wind River) ; Graydon, Tracy ; stephano.cet...@linux.intel.com; Sangal, Apoorv ; Yeoh, Ee Peng ; Chan, Aaron Chun Yew ; Chang, Rebecca Swee Fun ; Ang, Chin Huat ; Jolley, Stephen K ; Jordan, Robin L Subject: Release Candidate Build for yocto-2.4.3.rc2 now available. A release candidate build for yocto-2.4.3.rc2 is now available at: https://autobuilder.yocto.io/pub/releases/yocto-2.4.3.rc2 Please begin QA on this build as soon as possible. Build hash information: meta-intel : f66ce51d059d291a441d896854be8db70de5a554 meta-qt4 : e290738759ef3f39c9e079eaa9b606a62107e5ba meta-mingw : 1cc620b38f6f30a0bdd181783297998fe073387f meta-qt3 : 8ef7261a331e0869a0461ab2fb44e39980cedc02 meta-gplv2 : f875c60ecd6f30793b80a431a2423c4b98e51548 poky : 7e7ee662f5dea4d090293045f7498093322802cc This is an automated message from The Yocto Project Autobuilder Git: git://git.yoctoproject.org/yocto-autobuilder -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
[yocto] QA cycle report for 2.5 M4 RC1
This is the full report for 2.5 M4 RC1: https://wiki.yoctoproject.org/wiki/WW19_-_2018-05-10-_Full_Test_Cycle_-_2.5_M4_rc1 === Summary The QA cycle for release 2.5 M4 RC1 was completed. Team had discovered 2 new bugs. 1. eSDK toolchain were configured with SSTATE_MIRRORS but "devtool sdk-install libxml2" failed [1]. 2. oe-selftest failed at runtime_test.TestImage.test_testimage_dnf [2]. For high & medium+ bugs identified during 2.5 M3 RC1, all 3 high bugs (12647, 12665, 12655) and 2 medium+ bugs (12633 & 12641) were resolved/verified. === QA-Hints No significant bugs were discovered. === New Bugs [1] Bug 12725 - devtool sdk-install unable to add component for libxml2 https://bugzilla.yoctoproject.org/show_bug.cgi?id=12725 [2] Bug 12736 - 2.5 rc1: CentOS 7 oe-selftest fail at runtime_test.TestImage.test_testimage_dnf https://bugzilla.yoctoproject.org/show_bug.cgi?id=12736 Regards Ee Peng -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
[yocto] QA cycle report for 2.5 M3 RC1
Hello All, Update the report with bitbake task explorer, where it has an expected error message while overall functionality working as expected. This is the full report for 2.5 M3 RC1: https://wiki.yoctoproject.org/wiki/WW15_-_2018-04-09-_Full_Test_Cycle_-_2.5_M3_rc1 === Summary The QA cycle for release 2.5 M3 RC1 was completed. Team had discovered 8 new bugs. There were 3 bugs related to runtime [4] [6] [7], where core-image-lsb-sdk unable to boot on Minnowboard Turbot, runtime tests failed on OpenSUSE42.3 host machine, & connect host with serial console failed to display login prompt for core-image-sato-sdk-genericx86-64 on Minnowboard Turbot. There were 2 bugs related to selftest [1] [2], 1 bug for kernel development [5], and 1 bug for bitbake taskexp [8]. This QA cycle were executed with additional manual testing on the images/artifacts created by the new Autobuilder codebase (http://autobuilder.yoctoproject.org/pub/yocto-2.5_M3.rc1/), where the images/artifacts were downloaded and manually tested. There was 1 bug discovered from new Autobuilder codebase where Build Appliance failed [3] to build image. Testcase#202 (syslog) were skipped in multiple BSP automated runtime tests where it cannot run with sysklogd installed. Performance tests shown that build virtual/kernel time for this cycle increased by ~6% compared to 2.5 M2 RC1. This QA cycle had automated package management runtime tests as well as most of the meta-ide-support and SDK toolchain using both existing and new automated tests available. === QA-Hints No significant bugs were discovered. === New Bugs [1] Bug 12647 - [2.5 M3 rc1][OE-Core:Debian9] recipe xcursor-transport-theme went backwards, breaks package feeds [2] Bug 12655 - [2.5 M3 rc1][OE Core:Debian9] layerappend.LayerAppendTests.test_layer_appends - Testcase 1196: ERROR [3] Bug 12637 - [2.5 M3 RC1] Build appliance failed on glibc-initial do_configure when bitbake core-image-minimal [4] Bug 12633 - [2.5 M3 RC1] BSP-QEMU:core-image-lsb-sdk-x86.wic unable to boot on Minnowboard Turbot [5] Bug 12641 - [2.5 M3 RC1] Kernel Development Build external modules (hello-mod) test case: Function failed do_rootfs [6] Bug 12665 - [2.5 M3 rc1] OpenSUSE42.3 runtime test fail at runtime_test.TextExport, runtime_test.TestImage, runtime_test.TestImage and oe_runmake [7] Bug 12666 - [2.5 M3 RC1] BSP-QEMU:core-image-sato-sdk-genericx86-64.hddimg unable to connect to console with serial on Minnowboard Turbot [8] Bug 12670 - [2.5 M3 RC1]MANUAL_BitBake:test dependancy explorer is launched with failed logs Links = 1. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12647 [1] 2. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12655 [2] 3. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12637 [3] 4. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12633 [4] 5. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12641 [5] 6. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12665 [6] 7. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12666 [7] 8. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12670 [8] Regards Ee Peng -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
[yocto] QA cycle report for 2.5 M3 RC1
Hello All, This is the full report for 2.5 M3 RC1: https://wiki.yoctoproject.org/wiki/WW15_-_2018-04-09-_Full_Test_Cycle_-_2.5_M3_rc1 === Summary The QA cycle for release 2.5 M3 RC1 was completed. Team had discovered 7 new bugs. There were 3 bugs related to runtime [4] [6] [7], where core-image-lsb-sdk unable to boot on Minnowboard Turbot, runtime tests failed on OpenSUSE42.3 host machine, & connect host with serial console failed to display login prompt for core-image-sato-sdk-genericx86-64 on Minnowboard Turbot. There were 3 bugs related to selftest [1] [2] and kernel development [5]. This QA cycle were executed with additional manual testing on the images/artifacts created by the new Autobuilder codebase (http://autobuilder.yoctoproject.org/pub/yocto-2.5_M3.rc1/), where the images/artifacts were downloaded and manually tested. There was 1 bug discovered from new Autobuilder codebase where Build Appliance failed [3] to build image. Testcase#202 (syslog) were skipped in multiple BSP automated runtime tests where it cannot run with sysklogd installed. Testcase#1550 (bitbake dependency explorer) was failing and it was still undergoing regression testing. Performance tests shown that build virtual/kernel time for this cycle increased by ~6% compared to 2.5 M2 RC1. This QA cycle had automated package management runtime tests as well as most of the meta-ide-support and SDK toolchain using both existing and new automated tests available. === QA-Hints No significant bugs were discovered. === New Bugs [1] Bug 12647 - [2.5 M3 rc1][OE-Core:Debian9] recipe xcursor-transport-theme went backwards, breaks package feeds [2] Bug 12655 - [2.5 M3 rc1][OE Core:Debian9] layerappend.LayerAppendTests.test_layer_appends - Testcase 1196: ERROR [3] Bug 12637 - [2.5 M3 RC1] Build appliance failed on glibc-initial do_configure when bitbake core-image-minimal [4] Bug 12633 - [2.5 M3 RC1] BSP-QEMU:core-image-lsb-sdk-x86.wic unable to boot on Minnowboard Turbot [5] Bug 12641 - [2.5 M3 RC1] Kernel Development Build external modules (hello-mod) test case: Function failed do_rootfs [6] Bug 12665 - [2.5 M3 rc1] OpenSUSE42.3 runtime test fail at runtime_test.TextExport, runtime_test.TestImage, runtime_test.TestImage and oe_runmake [7] Bug 12666 - [2.5 M3 RC1] BSP-QEMU:core-image-sato-sdk-genericx86-64.hddimg unable to connect to console with serial on Minnowboard Turbot Links = 1. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12647 [1] 2. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12655 [2] 3. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12637 [3] 4. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12633 [4] 5. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12641 [5] 6. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12665 [6] 7. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12666 [7] Regards Ee Peng -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] Resend QA cycle report for 2.4.2 RC2
Hi Stephen, Those idle test cases are actually belongs to the outdated testcases inside Testopia, GDC had opened a Bugzilla ticket to remove and replace outdated and duplicated test cases. We are in progress of removing ducplicated test cases, next step is to remove outdated test cases using automated script that utilize testopia python api. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12435 For testrun# 9206-9212, testopia testcases not 100% completed as there are some outdated testcases that were no longer available in oeqa/selftest testsuite, pending testopia data cleanup. Please let me know if any more question. Thanks, Ee Peng -Original Message- From: Jolley, Stephen K Sent: Friday, March 9, 2018 12:48 AM To: Yeoh, Ee Peng <ee.peng.y...@intel.com>; 'yocto@yoctoproject.org' <yocto@yoctoproject.org>; Purdie, Richard <richard.pur...@intel.com>; Jordan, Robin L <robin.l.jor...@intel.com>; Zhang, Jessica <jessica.zh...@intel.com>; Lock, Joshua G <joshua.g.l...@intel.com>; Eggleton, Paul <paul.eggle...@intel.com> Cc: Sangal, Apoorv <apoorv.san...@intel.com> Subject: RE: Resend QA cycle report for 2.4.2 RC2 Why are we showing 62 (2%) of the test cases as Idle. (Not Run). Were they blocked? What happened to them? Thanks, Stephen -Original Message- From: Yeoh, Ee Peng Sent: Thursday, March 08, 2018 1:41 AM To: 'yocto@yoctoproject.org' <yocto@yoctoproject.org>; Purdie, Richard <richard.pur...@intel.com>; Jolley, Stephen K <stephen.k.jol...@intel.com>; Jordan, Robin L <robin.l.jor...@intel.com>; Zhang, Jessica <jessica.zh...@intel.com>; Lock, Joshua G <joshua.g.l...@intel.com>; Eggleton, Paul <paul.eggle...@intel.com> Cc: Sangal, Apoorv <apoorv.san...@intel.com> Subject: Resend QA cycle report for 2.4.2 RC2 Hello All, Resend the full report for 2.4.2 RC2 with updated information: https://wiki.yoctoproject.org/wiki/WW10_-_2018-03-05-_Full_Test_Cycle_-_2.4.2_rc2 === Summary The QA cycle for release 2.4.2 RC2 was completed including the performance test. Team had discovered 3 new bugs (oeselftest.sstatetests.test_sstate_sametune_samesigs failed, x11perf failed on minnowmax and nuc, toaster run again button does not behaved as expected). For testrun# 9206-9212, testopia testcases not 100% completed as there are some outdated testcases that were no longer available in oeqa/selftest testsuite, pending testopia data cleanup. Performance tests were executed for both 2.4.2 RC2 and 2.4.1 RC1 (as regression and benchmark) on these new machines in linux foundation. === QA-Hints QA shows no major bug. === Bugs New Bugs [1] 12587 - [2.4.2 rc2] test_sstate_sametune_samesigs (sstatetests.SStateTests) [2] 12590 - [2.4.2 rc2] No. of Chars/sec For Yocto Image is very Low To that of Ubuntu [3] 12591 - [2.4.2 rc2] Run again button from all builds page must run the specified task Links = 1. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12587 2. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12590 3. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12591 Regards Ee Peng -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
[yocto] Resend QA cycle report for 2.4.2 RC2
Hello All, Resend the full report for 2.4.2 RC2 with updated information: https://wiki.yoctoproject.org/wiki/WW10_-_2018-03-05-_Full_Test_Cycle_-_2.4.2_rc2 === Summary The QA cycle for release 2.4.2 RC2 was completed including the performance test. Team had discovered 3 new bugs (oeselftest.sstatetests.test_sstate_sametune_samesigs failed, x11perf failed on minnowmax and nuc, toaster run again button does not behaved as expected). For testrun# 9206-9212, testopia testcases not 100% completed as there are some outdated testcases that were no longer available in oeqa/selftest testsuite, pending testopia data cleanup. Performance tests were executed for both 2.4.2 RC2 and 2.4.1 RC1 (as regression and benchmark) on these new machines in linux foundation. === QA-Hints QA shows no major bug. === Bugs New Bugs [1] 12587 - [2.4.2 rc2] test_sstate_sametune_samesigs (sstatetests.SStateTests) [2] 12590 - [2.4.2 rc2] No. of Chars/sec For Yocto Image is very Low To that of Ubuntu [3] 12591 - [2.4.2 rc2] Run again button from all builds page must run the specified task Links = 1. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12587 2. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12590 3. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12591 Regards Ee Peng -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
[yocto] QA cycle report for 2.4.2 RC2
Hello All, This is the full report for 2.4.2 RC2: https://wiki.yoctoproject.org/wiki/WW10_-_2018-03-05-_Full_Test_Cycle_-_2.4.2_rc2 === Summary The QA cycle for release 2.4.2 RC2 was completed including the performance test. Team had not discovered any new bug. Performance tests were executed for both 2.4.2 RC2 and 2.4.1 RC1 (as regression and benchmark) on these new machines in linux foundation. === QA-Hints QA shows no new bug, this release is stable for release. === Bugs N/A Links = N/A Regards Ee Peng -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
[yocto] QA cycle report for 2.2.3 RC2
Hello All, This is the full report for 2.2.3.RC2: https://wiki.yoctoproject.org/wiki/WW06_-_2018-02-08_-_Full_Point_Release_Test_Cycle_2.2.3_rc2 === Summary The QA cycle for release 2.2.3.RC2 was completed. Team had discovered 3 new bugs. Performance tests executed on GDC show that eSDK time was longer than expected on both Fedora and Ubuntu machines, where GDC Fedora machine was identified to be causing longer eSDK time previously in 2.5 M1 rc3. Suspect that GDC Ubuntu machine was facing same issue as the Fedora machine which causing the longer than expected eSDK time. === QA-Hints The build is stable enough to be released, need to rerun performance tests on new machines from Linux Foundation. === Bugs New Bugs - 12519 [1] [2.2.3 rc2] ccache-native-3.2.5-r0: oe-runmake failed - 12520 [2] [2.2.3 rc2] grub-efi-native-2.00-r3 oe_runmake failed - 12521 [3] [2.2.3 rc2] setUpClass - gpg fail to build skey array skipping testcases (auto) Links = 1. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12519 [1] 2. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12520 [2] 3. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12521 [3] Regards Ee Peng -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
[yocto] QA cycle report for 2.5 M2 RC1
Hello All, This is the full report for 2.5 M2 RC1: https://wiki.yoctoproject.org/wiki/WW07_-_2018-02-14-_Full_Test_Cycle_-_2.5_M2_rc1 === Summary The QA cycle for release 2.5 M2 RC1 was completed including the performance test. Team had discovered a new bug [1] where sdk toolchain downloaded from Public Autobuilder failed when executing runqemu command. YP QA recently requested the performance host machines from linux foundation to replace old machines, where initial setup completed at WW06 and going through testing. Performance tests were executed for both 2.5 M2 RC1 and 2.5 M1 RC3 (as regression and benchmark) on these new machines in linux foundation. === QA-Hints Need to fix the sdk toolchain bug in order to enable runqemu. Performance results show that no significant differences in performance tests. === Bugs New Bugs - 12545 [1] [2.5 M2 RC1] SDK runqemu failed with python import logging Links = 1. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12545 [1] Regards Ee Peng -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
[yocto] Preliminary QA cycle report for 2.5 M2 RC1
Hello All, This is the preliminary report for 2.5 M2 RC1 (while waiting for performance test data to be ready): https://wiki.yoctoproject.org/wiki/WW07_-_2018-02-14-_Full_Test_Cycle_-_2.5_M2_rc1 === Summary The QA cycle for release 2.5 M2 RC1 was completed except performance test. Team had discovered a new bug [1] where sdk toolchain downloaded from Public Autobuilder failed when executing runqemu command. YP QA recently requested the performance host machines from linux foundation to replace old machines, where initial setup completed at WW06 and going through testing. Currently, running performance testing trial run to collect data. Expected data collection for regression and 2.5 M2 rc1 to be completed by WW08.1. === QA-Hints Waiting performance test data to be ready, the build is stable, need to fix the sdk toolchain bug in order to enable runqemu. === Bugs New Bugs - 12545 [1] [2.5 M2 RC1] SDK runqemu failed with python import logging Links = 1. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12545 [1] Regards Ee Peng -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
[yocto] QA cycle report for 2.5 M1 RC3
Hello All, Enjoy viewing the full Report for 2.5 M1 RC3: https://wiki.yoctoproject.org/wiki/WW03_-_2018-1-18-_Full_Test_Cycle_-_2.5_M1_rc3 === Summary The QA cycle for release 2.5 M1 RC3 was completed. Team had confirmed that qemu frozen (hung up) during OE-selftest on Fedora 26/27 after applied latest kernel patches will be resolved by using the default kernel (4.14.5-200.fc26.x86_64 on Fedora26 and 4.13.9-300.fc27.x86_64 on Fedora27). Team discovered 3 new bugs, build appliance failed to build core-image-sato & build-appliance-image [1], qemuarm failed during shutdown [2], and LTP test running on core-image-lsb-sdk not able to complete full test [3]. Team decided to drop Tumbleweed from QA coverage. === QA-Hints The build is stable enough to be released, need to further investigate on build appliance issue [1] and LTP testing issue [3]. === Bugs New Bugs - 12497 [1] Build Appliance : failed to build core-image-sato image & build-appliance-image - 12499 [2] qemuarm: failed to shutdown (2.5 M1 rc3) - 12504 [3] LTP Test: LTP tests hung up and not able to complete full run (2.5 M1 RC3) Full Bug Report: https://wiki.yoctoproject.org/wiki/WW03_-_2018-1-18-_Full_Test_Cycle_-_2.5_M1_rc3#Bugs_Found_during_QA_Test Links = 1. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12497 [1] 2. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12499 [2] 3.https://bugzilla.yoctoproject.org/show_bug.cgi?id=12504 [3] Regards Ee Peng -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
[yocto] QA cycle report for 2.5 M1 RC1
Hello All, Enjoy viewing the full Report for 2.5 M1 RC1: https://wiki.yoctoproject.org/wiki/WW02_-_2018-1-8-_Full_Test_Cycle_-_2.5_M1_rc1 === Summary The QA cycle for release 2.5 M1 RC1 is complete. There are 2 new bugs and 1 concern. QA has 1 concern on the unexpected QEMU hung up on Fedora 26 & Tumbleweed. QA discovered that Minnow Turbot igb ethernet controller failed when boot up with core-image-sato-sdk image. Performance report shows that GDC fedora performance machine was still facing longer than expected eSDK deploy time which were identified to be related to the machine itself. === QA-Hints The build is no stable enough to be released, need to fix the Minnow Turbot ethernet and qemu hung up issues. === Bugs New Bugs -12457 [1] [QA 2.5 M1 rc1] Minnow Turbot with igb ethernet controller failed for network functional testing after boot up with core-image-sato-sdk image - 12481 [2] ptest-runner libxml2 failed to execute for 2.5 M1 rc1 Full Bug Report: https://wiki.yoctoproject.org/wiki/WW02_-_2018-1-8-_Full_Test_Cycle_-_2.5_M1_rc1#Bugs_Found_during_QA_Test Links = 1. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12457 [1] 2. https://bugzilla.yoctoproject.org/show_bug.cgi?id= 12481 [2] Regards Ee Peng -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
[yocto] yocto-2.5_M1.rc3 failed at runqemu for x86
Hi Joshua, I noticed for 2.5 M1 rc3, nightly-x86 failed during testimage (Sanity Test), due to runqemu error. Is this expected? Shall QA move on to initialize the QA for this rc3? Thank you very much for your attention and help! Thanks, Yeoh Ee Peng https://autobuilder.yocto.io/builders/nightly-x86/builds/703 ERROR: core-image-sato-sdk-1.0-r0 do_testimage: Qemu pid didn't appear in 120 seconds (01/04/18 17:54:45) ERROR: core-image-sato-1.0-r0 do_testimage: Qemu pid didn't appear in 120 seconds (01/04/18 17:54:45) WARNING: core-image-sato-sdk-1.0-r0 do_testimage: Qemu ended unexpectedly, dump data from host is in /tmp/oe-saved-tests/201801041754_qemu WARNING: core-image-sato-1.0-r0 do_testimage: Qemu ended unexpectedly, dump data from host is in /tmp/oe-saved-tests/201801041754_qemu ERROR: core-image-sato-sdk-1.0-r0 do_testimage: Output from runqemu: runqemu - INFO - Continuing with the following parameters: runqemu - INFO - Acquiring lockfile /tmp/qemu-tap-locks/tap0.lock failed: [Errno 11] Resource temporarily unavailable runqemu - INFO - Acquiring lockfile /tmp/qemu-tap-locks/tap1.lock failed: [Errno 11] Resource temporarily unavailable runqemu - INFO - Using preconfigured tap device tap2 runqemu - INFO - If this is not intended, touch /tmp/qemu-tap-locks/tap2.skip to make runqemu skip tap2. runqemu - INFO - Network configuration: 192.168.7.6::192.168.7.5:255.255.255.0 runqemu - INFO - Running /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86/build/build/tmp/work/x86_64-linux/qemu-helper-native/1.0-r1/recipe-sysroot-native/usr/bin/qemu-system-i386 -device virtio-net-pci,netdev=net0,mac=52:54:00:12:34:06 -netdev tap,id=net0,ifname=tap2,script=no,downscript=no -drive file=/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86/build/build/tmp/deploy/images/qemux86/core-image-sato-sdk-qemux86.ext4,if=virtio,format=raw -vga vmware -show-cursor -usb -device usb-tablet -device virtio-rng-pci -serial tcp:127.0.0.1:57033 -pidfile pidfile_32833 -cpu pentium2 -enable-kvm -m 256 -serial tcp:127.0.0.1:33999 -snapshot -kernel /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86/build/build/tmp/deploy/images/qemux86/bzImage--4.12.18+git0+b66a4f9730_558fe84d69-r0.2-qemux86-20180102174658.bin -append 'root=/dev/vda rw highres=off clocksource=kvm-clock hpet=disable noapic nolapic mem=256M ip=192.168.7.6::192.168.7.5:255.255.255.0 vga=0 uvesafb.mode_option=640x480-32 oprofile.timer=1 uvesafb.task_timeout=-1 console=tty1 console=ttyS0,115200n8 printk.time=1' runqemu - ERROR - Failed to run qemu: ioctl(KVM_CREATE_VM) failed: 12 Cannot allocate memory qemu-system-i386: failed to initialize KVM: Cannot allocate memory -Original Message- From: Lock, Joshua G Sent: Thursday, January 4, 2018 11:42 PM To: pokybu...@debian8.yocto.io; yocto@yoctoproject.org Cc: pi...@toganlabs.com; Perez Carranza, Jose <jose.perez.carra...@intel.com>; Cruz, Libertad <libertad.c...@intel.com>; ota...@ossystems.com.br; Zhao, Yi (Wind River) <yi.z...@windriver.com>; Graydon, Tracy <tracy.gray...@intel.com>; Sangal, Apoorv <apoorv.san...@intel.com>; Yeoh, Ee Peng <ee.peng.y...@intel.com>; Chan, Aaron Chun Yew <aaron.chun.yew.c...@intel.com>; Chang, Rebecca Swee Fun <rebecca.swee.fun.ch...@intel.com>; Ang, Chin Huat <chin.huat@intel.com> Subject: Re: Release Candidate Build for yocto-2.5_M1.rc2 now available. This build had a couple of failures, I'm going to spin another rc. QA: please don't take any action with this build. Joshua On 04/01/18 15:35, pokybu...@debian8.yocto.io wrote: > > A release candidate build for yocto-2.5_M1.rc2 is now available at: > > > https://autobuilder.yocto.io/pub/releases/yocto-2.5_M1.rc2 > > > Please begin QA on this build as soon as possible. > > > Build hash information: > meta-intel : 1580913674bc84c8e6c10cf855dbf38d2ee8f3d2 > meta-qt4 : f313dbee2ac3d5fcc9801407947d3cb6cfb90b5d > meta-mingw : a2c5fb82e5595d5bb56b579b7c93147c4fb065a5 > meta-qt3 : f33b73a9563f2dfdfd0ee37b61d65d90197a456f > meta-gplv2 : a0f8716d07324e945f0462b82f9309fa0934af17 > poky : 8f436af94648c8c14244b71183e7da5f6cd09813 > > > This is an automated message from > The Yocto Project Autobuilder > Git: git://git.yoctoproject.org/yocto-autobuilder > Email: joshua.g.l...@intel.com > -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] QA cycle report for 2.4.1 RC1
Hi Richard, We had configured GDC Fedora Performance Machine to run 2.4 M4 rc3 [1]. Based on the new performance data we collected, the average time for eSDK deploy is 237 seconds (3 min 57 sec). In comparison, data collected WW43 2017, the average time for eSDK deploy was 161 seconds (2 min 41 sec). In summary, while running the same commit, 2.4 M4 rc3, the eSDK time is now significantly higher (47% higher). This clearly show that the performance machine was the root cause of the longer eSDK time during recent QA for 2.4.1.rc1 [2] but we have not identify why the performance machine was contributing to this trouble. Any suggestion on what shall be our next step? Hi Jose, Any idea what could be the reason why performance machine take longer time to run eSDK deploy? How about restart the Fedora Performance Machine? Thank you very much for your attention! Thanks, Yeoh Ee Peng eSDK Deploy Time – Run on WW01 2018 [cid:image001.jpg@01D38649.9DBA1930] eSDK Deploy Time – Run on WW43 2017 [cid:image002.jpg@01D38649.9DBA1930] [1] https://wiki.yoctoproject.org/wiki/WW43_-_2017-10-23_-_Full_Test_Cycle_2.4_RC3 [2] https://wiki.yoctoproject.org/wiki/WW51_-_2017-12-21-_Full_Test_Cycle_-_2.4.1_rc1 -Original Message- From: Richard Purdie [mailto:richard.pur...@linuxfoundation.org] Sent: Tuesday, January 2, 2018 7:27 PM To: Yeoh, Ee Peng <ee.peng.y...@intel.com>; Cruz, Libertad <libertad.c...@intel.com>; yocto@yoctoproject.org; Lock, Joshua G <joshua.g.l...@intel.com>; Jolley, Stephen K <stephen.k.jol...@intel.com>; Zhang, Jessica <jessica.zh...@intel.com> Cc: Perez Carranza, Jose <jose.perez.carra...@intel.com>; Sangal, Apoorv <apoorv.san...@intel.com> Subject: Re: QA cycle report for 2.4.1 RC1 Hi, On Fri, 2017-12-22 at 01:40 -0800, Yeoh, Ee Peng wrote: > Enjoy viewing the full Report for 2.4.1 > RC1: https://wiki.yoctoproject.org/wiki/WW51_-_2017-12-21- > _Full_Test_Cycle_-_2.4.1_rc1 > > === Summary > > The QA cycle for release 2.4.1 RC1 is complete. There are 5 new bugs > from which so far none of them are high. QA has 1 concern on the > performance testing. > > Performance report shows an increase of 23% build time on the eSDK in > fedora 23, bug has not been created until further investigation with > setup outside of the GDC environment. Was there any further update on this? It may be worth running the test for 2.4.0 and checking that you get the same result as we had at time of release, just to double check there isn't some other contributing cause here... Cheers, Richard -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] QA cycle report for 2.4.1 RC1
Hi Richard, We had just configured Fedora Performance Machine to run 2.4.rc3 after we had collected enough performance data for 2.5 M1 rc1. Please give us 2 days to collect 10 data points to verify this issue. [cid:image003.jpg@01D3847B.47E721D0] Hi Jose, I had configured ypperf01 (Fedora Performance Machine) to run 2.4.rc3. Please let me know if anything that concern you. [cid:image004.jpg@01D3847B.47E721D0] Thank you very much! Thanks, Yeoh Ee Peng -Original Message- From: Richard Purdie [mailto:richard.pur...@linuxfoundation.org] Sent: Tuesday, January 2, 2018 7:27 PM To: Yeoh, Ee Peng <ee.peng.y...@intel.com>; Cruz, Libertad <libertad.c...@intel.com>; yocto@yoctoproject.org; Lock, Joshua G <joshua.g.l...@intel.com>; Jolley, Stephen K <stephen.k.jol...@intel.com>; Zhang, Jessica <jessica.zh...@intel.com> Cc: Perez Carranza, Jose <jose.perez.carra...@intel.com>; Sangal, Apoorv <apoorv.san...@intel.com> Subject: Re: QA cycle report for 2.4.1 RC1 Hi, On Fri, 2017-12-22 at 01:40 -0800, Yeoh, Ee Peng wrote: > Enjoy viewing the full Report for 2.4.1 > RC1: https://wiki.yoctoproject.org/wiki/WW51_-_2017-12-21- > _Full_Test_Cycle_-_2.4.1_rc1 > > === Summary > > The QA cycle for release 2.4.1 RC1 is complete. There are 5 new bugs > from which so far none of them are high. QA has 1 concern on the > performance testing. > > Performance report shows an increase of 23% build time on the eSDK in > fedora 23, bug has not been created until further investigation with > setup outside of the GDC environment. Was there any further update on this? It may be worth running the test for 2.4.0 and checking that you get the same result as we had at time of release, just to double check there isn't some other contributing cause here... Cheers, Richard -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
[yocto] QA cycle report for 2.4.1 RC1
Hello All, Enjoy viewing the full Report for 2.4.1 RC1: https://wiki.yoctoproject.org/wiki/WW51_-_2017-12-21-_Full_Test_Cycle_-_2.4.1_rc1 === Summary The QA cycle for release 2.4.1 RC1 is complete. There are 5 new bugs from which so far none of them are high. QA has 1 concern on the performance testing. Performance report shows an increase of 23% build time on the eSDK in fedora 23, bug has not been created until further investigation with setup outside of the GDC environment. https://wiki.yoctoproject.org/wiki/WW51_-_2017-12-21-_Full_Test_Cycle_-_2.4.1_rc1#Build_Performance_Test === QA-Hints The build is stable enough to be released, provided the performance issue is investigated and root-cause identified. === Bugs New Bugs -12444 [1] [Test Case 1542] test_2_logrotate fails in MinnowMax with core-image-lsb-sdk-genericx86-64 image -12417 [2] Eclipse installer steps only apply to latest Eclipse release -12416 [3] autobuilder.yocto.io directory index does not display the full long filename -12443 [4] [Test Case 312,313] Bitbake failed after using yocto-kernel add patch -12445 [5] [Test Case 309,310,311] Bitbake fails after add layer with yocto-bsp create Full Bug Report: https://wiki.yoctoproject.org/wiki/WW51_-_2017-12-21-_Full_Test_Cycle_-_2.4.1_rc1#Bugs_Found_during_QA_Test Links = 1. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12444 [1] 2. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12417 [2] 3. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12416 [3] 4. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12443 [4] 5. https://bugzilla.yoctoproject.org/show_bug.cgi?id=12445 [5] Regards Ee Peng -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
