[OE-core] [meta-oe][PATCH 1/3] testimage: Refactoring and fixing.
The testimage.bbclass invocation of controllers besides
simpletarget and qemutarget was broken. Setting TEST_TARGET
to class names did not work, only hardcoded values set in
context.py were allowed. On the other hand the other
TEST_TARGETs available were supposedly available through the
class name convetion but none of them were working. See:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=12842
In this commit qemuwrapper and simpletarget TEST_TARGETS are
now not available through these names, but through the
common convention of class names.
The code for layer defined controllers was outdated and
broken because the example controllers in meta-yocto-bsp as
well as the controllers available in masterimage were
dependent only on the datastore d being available for
__init__ contruction, when the testimage.bbclass and
context.py, which initialize test controllers never passed
the datastore. This commit fixes all the openembedded
controllers to not use the datastore, but the testdata json
file that is loaded by the testimage.bbclass. This way the
tests can still be exported to be ran outside bitbake. The
kwargs and logger is still passed in the constructor and
there maybe some redundancies between the kwargs and the td
test datstore, but this is not addressed in this commit.
In masterimage.py there was a duplicate SystemdbootTarget
which was removed. It is my opinion that all targets should
go to meta-yocto-bsp or meta/ and should not get special
treatment whether they come from poky or other layer.
In this commit functionality that was lost in masterimage
about retrieving a fresh BB_ORIGENV. The test data json file
does not store this variable, so the build-time enviromental
variables cannot be retrieved later on, when running the
tests.
The deploy() method invocation of the test cases was removed
in the refactoring and this commit re-adds it in the same
logical place. The deploy method was added as an abstract
method so that all Test controller classes have this method.
This method had been lost and was not used in the
simpleremote and qemuwrapper, because these controllers do
not need the functionality, but almost any real board
requires certain steps to deploy the new images, before the
tests can be ran. An example of this need was visible in the
master image and subclasses in meta-yocto-bsp.
A dependency on images fstypes of qemu was removed from
testimage. The allowed file system types were not relevant
for other controllers, and made it so that there was an
import of the OEQemuTarget python class applicable to all
possible controllers, including non qemu.
The SimpleRemoteTarget, OEQemuTarget and a custom
controller, developed according to the manual were
successfully tested.
Signed-off-by: Paulo Neves
---
meta/classes/testimage.bbclass | 18 +---
meta/lib/oeqa/controllers/masterimage.py | 107 ++---
.../target/qemu.py => controllers/qemutarget.py} | 30 --
meta/lib/oeqa/controllers/simpleremote.py | 33 +++
.../target/ssh.py => controllers/sshtarget.py} | 9 +-
meta/lib/oeqa/core/target/__init__.py | 4 +
meta/lib/oeqa/runtime/context.py | 32 +-
meta/lib/oeqa/targetcontrol.py | 97 ++-
8 files changed, 137 insertions(+), 193 deletions(-)
rename meta/lib/oeqa/{core/target/qemu.py => controllers/qemutarget.py} (58%)
create mode 100644 meta/lib/oeqa/controllers/simpleremote.py
rename meta/lib/oeqa/{core/target/ssh.py => controllers/sshtarget.py} (98%)
diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 6fa901b..2aa59b0 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -75,7 +75,7 @@ DEFAULT_TEST_SUITES_remove_qemumips64 = "${MIPSREMOVE}"
TEST_SUITES ?= "${DEFAULT_TEST_SUITES}"
TEST_QEMUBOOT_TIMEOUT ?= "1000"
-TEST_TARGET ?= "qemu"
+TEST_TARGET ?= "OEQemuTarget"
TESTIMAGEDEPENDS = ""
TESTIMAGEDEPENDS_qemuall = "qemu-native:do_populate_sysroot
qemu-helper-native:do_populate_sysroot
qemu-helper-native:do_addto_recipe_sysroot"
@@ -147,7 +147,6 @@ def testimage_main(d):
from oeqa.core.utils.misc import updateTestData
from oeqa.runtime.context import OERuntimeTestContext
from oeqa.runtime.context import OERuntimeTestContextExecutor
-from oeqa.core.target.qemu import supported_fstypes
from oeqa.core.utils.test import getSuiteCases
from oeqa.utils import make_logger_bitbake_compatible
@@ -189,15 +188,6 @@ def testimage_main(d):
# Get machine
machine = d.getVar("MACHINE")
-# Get rootfs
-fstypes = [fs for fs in d.getVar('IMAGE_FSTYPES').split(' ')
- if fs in supported_fstypes]
-if not fstypes:
-bb.fatal('Unsupported image type built. Add a comptible image to '
- 'IMAGE_FSTYPES. Supported types: %s' %
- ', '.join(supported_fstypes))
-rootfs = '%s.%s' % (image_name,
Re: [OE-core] [meta-oe][PATCH 1/3] testimage: Refactoring and fixing.
On Fri, 2018-08-10 at 21:37 +0200, Paulo Neves wrote: > The testimage.bbclass invocation of controllers besides > simpletarget and qemutarget was broken. Setting TEST_TARGET > to class names did not work, only hardcoded values set in > context.py were allowed. On the other hand the other > TEST_TARGETs available were supposedly available through the > class name convetion but none of them were working. See: > https://bugzilla.yoctoproject.org/show_bug.cgi?id=12842 > > In this commit qemuwrapper and simpletarget TEST_TARGETS are > now not available through these names, but through the > common convention of class names. There are some good cleanups in here, thanks! It is hard to review due to the number/size of changes. I put it into a test run on the autobuilder and it failed: https://autobuilder.yocto.io/builders/nightly-musl/builds/1220/steps/Running%20Sanity%20Tests/logs/stdio The rest of the builds have't gotten this far yet but I suspect we'll see further similar failures and these will need to be fixed before we can consider merging the patches. Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [meta-oe][PATCH 1/3] testimage: Refactoring and fixing.
I found a bug in my code where changing a variable, for example in
layer.conf does not re-trigger the generation of the json file
containing the data store at the time of image creation. The json file
is generated in
ROOTFS_POSTPROCESS_COMMAND += "write_image_test_data ; " @
meta/classes/rootfs-postcommands.bbclass.
Can anybody tell me if changing a variable in the datastore and
accessing it in the postprocess_command will never trigger the
postprocess_command? It happens that in this case this variable is not
important for the rootfs task itself but only for the post-process.
Paulo Neves
On Fri, Aug 10, 2018 at 9:38 PM Paulo Neves wrote:
>
> The testimage.bbclass invocation of controllers besides
> simpletarget and qemutarget was broken. Setting TEST_TARGET
> to class names did not work, only hardcoded values set in
> context.py were allowed. On the other hand the other
> TEST_TARGETs available were supposedly available through the
> class name convetion but none of them were working. See:
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=12842
>
> In this commit qemuwrapper and simpletarget TEST_TARGETS are
> now not available through these names, but through the
> common convention of class names.
>
> The code for layer defined controllers was outdated and
> broken because the example controllers in meta-yocto-bsp as
> well as the controllers available in masterimage were
> dependent only on the datastore d being available for
> __init__ contruction, when the testimage.bbclass and
> context.py, which initialize test controllers never passed
> the datastore. This commit fixes all the openembedded
> controllers to not use the datastore, but the testdata json
> file that is loaded by the testimage.bbclass. This way the
> tests can still be exported to be ran outside bitbake. The
> kwargs and logger is still passed in the constructor and
> there maybe some redundancies between the kwargs and the td
> test datstore, but this is not addressed in this commit.
>
> In masterimage.py there was a duplicate SystemdbootTarget
> which was removed. It is my opinion that all targets should
> go to meta-yocto-bsp or meta/ and should not get special
> treatment whether they come from poky or other layer.
>
> In this commit functionality that was lost in masterimage
> about retrieving a fresh BB_ORIGENV. The test data json file
> does not store this variable, so the build-time enviromental
> variables cannot be retrieved later on, when running the
> tests.
>
> The deploy() method invocation of the test cases was removed
> in the refactoring and this commit re-adds it in the same
> logical place. The deploy method was added as an abstract
> method so that all Test controller classes have this method.
> This method had been lost and was not used in the
> simpleremote and qemuwrapper, because these controllers do
> not need the functionality, but almost any real board
> requires certain steps to deploy the new images, before the
> tests can be ran. An example of this need was visible in the
> master image and subclasses in meta-yocto-bsp.
>
> A dependency on images fstypes of qemu was removed from
> testimage. The allowed file system types were not relevant
> for other controllers, and made it so that there was an
> import of the OEQemuTarget python class applicable to all
> possible controllers, including non qemu.
>
> The SimpleRemoteTarget, OEQemuTarget and a custom
> controller, developed according to the manual were
> successfully tested.
>
> Signed-off-by: Paulo Neves
> ---
> meta/classes/testimage.bbclass | 18 +---
> meta/lib/oeqa/controllers/masterimage.py | 107
> ++---
> .../target/qemu.py => controllers/qemutarget.py} | 30 --
> meta/lib/oeqa/controllers/simpleremote.py | 33 +++
> .../target/ssh.py => controllers/sshtarget.py} | 9 +-
> meta/lib/oeqa/core/target/__init__.py | 4 +
> meta/lib/oeqa/runtime/context.py | 32 +-
> meta/lib/oeqa/targetcontrol.py | 97 ++-
> 8 files changed, 137 insertions(+), 193 deletions(-)
> rename meta/lib/oeqa/{core/target/qemu.py => controllers/qemutarget.py} (58%)
> create mode 100644 meta/lib/oeqa/controllers/simpleremote.py
> rename meta/lib/oeqa/{core/target/ssh.py => controllers/sshtarget.py} (98%)
>
> diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
> index 6fa901b..2aa59b0 100644
> --- a/meta/classes/testimage.bbclass
> +++ b/meta/classes/testimage.bbclass
> @@ -75,7 +75,7 @@ DEFAULT_TEST_SUITES_remove_qemumips64 = "${MIPSREMOVE}"
> TEST_SUITES ?= "${DEFAULT_TEST_SUITES}"
>
> TEST_QEMUBOOT_TIMEOUT ?= "1000"
> -TEST_TARGET ?= "qemu"
> +TEST_TARGET ?= "OEQemuTarget"
>
> TESTIMAGEDEPENDS = ""
> TESTIMAGEDEPENDS_qemuall = "qemu-native:do_populate_sysroot
> qemu-helper-native:do_populate_sysroot
> qemu-helper-native:do_addto_recipe_sysroot"
> @@ -147,7 +147,6 @@ def testimage_main(d)
Re: [OE-core] [meta-oe][PATCH 1/3] testimage: Refactoring and fixing.
On Sun, 2018-08-12 at 21:49 +0200, Paulo Neves wrote: > I found a bug in my code where changing a variable, for example in > layer.conf does not re-trigger the generation of the json file > containing the data store at the time of image creation. The json > file is generated in > ROOTFS_POSTPROCESS_COMMAND += "write_image_test_data ; " @ > meta/classes/rootfs-postcommands.bbclass. > > Can anybody tell me if changing a variable in the datastore and > accessing it in the postprocess_command will never trigger the > postprocess_command? It happens that in this case this variable is > not important for the rootfs task itself but only for the post- > process. The system has no insight into which variables that command might be accessing so it doesn't know when to rerun it. Adding a dependency on all variables would mean the rootfs would get continually rewritten. I don't like the json usage in testimage for this reason as it effectively breaks our data dependencies and handling in order to be able to run standalone. Pros and cons I guess, this is one of the downsides. Not sure there is much that can be done about it. Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [meta-oe][PATCH 1/3] testimage: Refactoring and fixing.
Ping.
Does anybody have any comments? Also I noticed that my commit once was
in the master-next but it was removed. Without it being in master I
cannot submit my changes to poky mailing list with the fixes for the
remaining targets.
Paulo Neves
On Sun, Aug 12, 2018 at 9:49 PM Paulo Neves wrote:
>
> I found a bug in my code where changing a variable, for example in
> layer.conf does not re-trigger the generation of the json file
> containing the data store at the time of image creation. The json file
> is generated in
> ROOTFS_POSTPROCESS_COMMAND += "write_image_test_data ; " @
> meta/classes/rootfs-postcommands.bbclass.
>
> Can anybody tell me if changing a variable in the datastore and
> accessing it in the postprocess_command will never trigger the
> postprocess_command? It happens that in this case this variable is not
> important for the rootfs task itself but only for the post-process.
>
> Paulo Neves
> On Fri, Aug 10, 2018 at 9:38 PM Paulo Neves wrote:
> >
> > The testimage.bbclass invocation of controllers besides
> > simpletarget and qemutarget was broken. Setting TEST_TARGET
> > to class names did not work, only hardcoded values set in
> > context.py were allowed. On the other hand the other
> > TEST_TARGETs available were supposedly available through the
> > class name convetion but none of them were working. See:
> > https://bugzilla.yoctoproject.org/show_bug.cgi?id=12842
> >
> > In this commit qemuwrapper and simpletarget TEST_TARGETS are
> > now not available through these names, but through the
> > common convention of class names.
> >
> > The code for layer defined controllers was outdated and
> > broken because the example controllers in meta-yocto-bsp as
> > well as the controllers available in masterimage were
> > dependent only on the datastore d being available for
> > __init__ contruction, when the testimage.bbclass and
> > context.py, which initialize test controllers never passed
> > the datastore. This commit fixes all the openembedded
> > controllers to not use the datastore, but the testdata json
> > file that is loaded by the testimage.bbclass. This way the
> > tests can still be exported to be ran outside bitbake. The
> > kwargs and logger is still passed in the constructor and
> > there maybe some redundancies between the kwargs and the td
> > test datstore, but this is not addressed in this commit.
> >
> > In masterimage.py there was a duplicate SystemdbootTarget
> > which was removed. It is my opinion that all targets should
> > go to meta-yocto-bsp or meta/ and should not get special
> > treatment whether they come from poky or other layer.
> >
> > In this commit functionality that was lost in masterimage
> > about retrieving a fresh BB_ORIGENV. The test data json file
> > does not store this variable, so the build-time enviromental
> > variables cannot be retrieved later on, when running the
> > tests.
> >
> > The deploy() method invocation of the test cases was removed
> > in the refactoring and this commit re-adds it in the same
> > logical place. The deploy method was added as an abstract
> > method so that all Test controller classes have this method.
> > This method had been lost and was not used in the
> > simpleremote and qemuwrapper, because these controllers do
> > not need the functionality, but almost any real board
> > requires certain steps to deploy the new images, before the
> > tests can be ran. An example of this need was visible in the
> > master image and subclasses in meta-yocto-bsp.
> >
> > A dependency on images fstypes of qemu was removed from
> > testimage. The allowed file system types were not relevant
> > for other controllers, and made it so that there was an
> > import of the OEQemuTarget python class applicable to all
> > possible controllers, including non qemu.
> >
> > The SimpleRemoteTarget, OEQemuTarget and a custom
> > controller, developed according to the manual were
> > successfully tested.
> >
> > Signed-off-by: Paulo Neves
> > ---
> > meta/classes/testimage.bbclass | 18 +---
> > meta/lib/oeqa/controllers/masterimage.py | 107
> > ++---
> > .../target/qemu.py => controllers/qemutarget.py} | 30 --
> > meta/lib/oeqa/controllers/simpleremote.py | 33 +++
> > .../target/ssh.py => controllers/sshtarget.py} | 9 +-
> > meta/lib/oeqa/core/target/__init__.py | 4 +
> > meta/lib/oeqa/runtime/context.py | 32 +-
> > meta/lib/oeqa/targetcontrol.py | 97
> > ++-
> > 8 files changed, 137 insertions(+), 193 deletions(-)
> > rename meta/lib/oeqa/{core/target/qemu.py => controllers/qemutarget.py}
> > (58%)
> > create mode 100644 meta/lib/oeqa/controllers/simpleremote.py
> > rename meta/lib/oeqa/{core/target/ssh.py => controllers/sshtarget.py} (98%)
> >
> > diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
> > index 6fa901b..2aa59b0 100644
> > --- a/meta/classes/
Re: [OE-core] [meta-oe][PATCH 1/3] testimage: Refactoring and fixing.
On Sat, 2018-08-18 at 18:40 +0200, Paulo Neves wrote: > Ping. > > Does anybody have any comments? Also I noticed that my commit once > was in the master-next but it was removed. Without it being in master > I cannot submit my changes to poky mailing list with the fixes for > the remaining targets. I tried adding it to master-next, it broke things, I did reply about it, I dropped it from -next due to the problems. My reply is here: http://lists.openembedded.org/pipermail/openembedded-core/2018-August/154037.html Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
