On 6/14/23 12:31, Richard Purdie wrote: > On Wed, 2023-06-14 at 10:56 +0200, Alexis Lothoré via > lists.yoctoproject.org wrote: >> From: Alexis Lothoré <alexis.loth...@bootlin.com> >> >> This series is a follow-up for the 4.3_M1.rc1 regression report issue. >> >> It has been observed that the report is empty. This issue is linked to >> configuration description in yocto-autobuilder-helper, and has been >> identified through the following steps: >> - empty report is supposed to be a comparison between yocto-4.2 (4.2.rc3) >> and 4.3_M1.rc1 >> - yocto-4.2 results are almost empty: we only find test results from Intel >> QA (pushed _after_ the AB build) and not the AB test results >> - tests results are managed by send-qa-email.send-qa-email uses resulttool >> to systematically gather and store test results in local git directory >> - however, it looks for basebranch/comparebranch to know if those results >> can be pushed onto git server, and those variables depend on config.json >> content >> - yocto-4.2 (4.2.rc3) has been built on release branch mickledore >> (https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/5212) >> - since mickledore is not yet described in config.json, send-qa-email >> considers it as a "work" branch (contrary to a "release" branch) and does >> not push test results >> >> As a consequence: >> - first commit brings in python logger >> - second commit adds a warning when such case happen, since we are able to >> detect it >> - third fix actually adds mickledore as a release branch to properly store >> again test results >> >> There must be a more robust rework to do (because the issue will likely >> happen on each major delivery), but I aimed for the quick and small fix to >> quickly bring back tests results storage without breaking other things in >> the process >> >> Alexis Lothoré (3): >> scripts/send-qa-email: use logger instead of raw prints >> scripts/send-qa-email: print warning when test results are not stored >> config.json: add mickledore as direct push branch for test results > > Thanks for the analysis. I agree we need to somehow fix this properly. > One solution might be to always push for poky if the branch name > doesn't end with -next?
That might work indeed. If we are sure enough that no custom/feature branch will be used in poky with send-qa-email (ie, only in poky-contrib), I can do the fix this way > > Since we have the release artefacts for the release, could we add the > test results after the fact now?> > Id' be interested to see the 4.3 M1 to 4.2 comparison rerun with that > added. I am not sure about where to find those artifacts for yocto-4.2 ? If you are referring to https://autobuilder.yocto.io/pub/, yocto-4.2 has already been removed from there. And if you are referring to the archived release on main site (https://downloads.yoctoproject.org/releases/yocto/yocto-4.2/poky-21790e71d55f417f27cd51fae9dd47549758d4a0.tar.bz2), it does contain a single, 40 line testresults.json, so that's definitely not the full AB tests results. > > Cheers, > > Richard > > > > > -- Alexis Lothoré, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60300): https://lists.yoctoproject.org/g/yocto/message/60300 Mute This Topic: https://lists.yoctoproject.org/mt/99523809/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
