from:"Stefan Stanacar"

[OE-core] [rocko][PATCH] default-versions.inc: use weak assignments to allow overwrite

2018-06-19 Thread Stefan Stanacar

default-versions.inc is included by defaultsetup.conf which in bitbake.conf
is included after local/machine/distro.conf.
If these are hard assignments distros can's overwrite them.

Signed-off-by: Stefan Stanacar 
---
 meta/conf/distro/include/default-versions.inc | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/conf/distro/include/default-versions.inc 
b/meta/conf/distro/include/default-versions.inc
index 868073843a..3d88e8f656 100644
--- a/meta/conf/distro/include/default-versions.inc
+++ b/meta/conf/distro/include/default-versions.inc
@@ -2,6 +2,6 @@
 # Default preferred versions
 #
 
-PREFERRED_VERSION_openssl = "1.0.%"
-PREFERRED_VERSION_openssl-native = "1.0.%"
-PREFERRED_VERSION_nativesdk-openssl = "1.0.%"
+PREFERRED_VERSION_openssl ?= "1.0.%"
+PREFERRED_VERSION_openssl-native ?= "1.0.%"
+PREFERRED_VERSION_nativesdk-openssl ?= "1.0.%"
-- 
2.17.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [sumo][PATCH] default-versions.inc: use weak assignments to allow overwrite

2018-06-19 Thread Stefan Stanacar

default-versions.inc is included by defaultsetup.conf which in bitbake.conf
is included after local/machine/distro.conf.
If these are hard assignments distros can's overwrite them.

Signed-off-by: Stefan Stanacar 
---
 meta/conf/distro/include/default-versions.inc | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/conf/distro/include/default-versions.inc 
b/meta/conf/distro/include/default-versions.inc
index 868073843a..3d88e8f656 100644
--- a/meta/conf/distro/include/default-versions.inc
+++ b/meta/conf/distro/include/default-versions.inc
@@ -2,6 +2,6 @@
 # Default preferred versions
 #
 
-PREFERRED_VERSION_openssl = "1.0.%"
-PREFERRED_VERSION_openssl-native = "1.0.%"
-PREFERRED_VERSION_nativesdk-openssl = "1.0.%"
+PREFERRED_VERSION_openssl ?= "1.0.%"
+PREFERRED_VERSION_openssl-native ?= "1.0.%"
+PREFERRED_VERSION_nativesdk-openssl ?= "1.0.%"
-- 
2.17.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] default-versions.inc: use weak assignments to allow overwrite

2018-06-13 Thread Stefan Stanacar

default-versions.inc is included by defaultsetup.conf which in bitbake.conf
is included after local/machine/distro.conf.
If these are hard assignments distros can's overwrite them.

Signed-off-by: Stefan Stanacar 
---
 meta/conf/distro/include/default-versions.inc | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/conf/distro/include/default-versions.inc 
b/meta/conf/distro/include/default-versions.inc
index 868073843a..3d88e8f656 100644
--- a/meta/conf/distro/include/default-versions.inc
+++ b/meta/conf/distro/include/default-versions.inc
@@ -2,6 +2,6 @@
 # Default preferred versions
 #
 
-PREFERRED_VERSION_openssl = "1.0.%"
-PREFERRED_VERSION_openssl-native = "1.0.%"
-PREFERRED_VERSION_nativesdk-openssl = "1.0.%"
+PREFERRED_VERSION_openssl ?= "1.0.%"
+PREFERRED_VERSION_openssl-native ?= "1.0.%"
+PREFERRED_VERSION_nativesdk-openssl ?= "1.0.%"
-- 
2.17.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] kernel.bbclass: Fix symlink creation when using externalsrc

2018-02-06 Thread Stefan Stanacar

Ping?

On Mon, Jan 29, 2018 at 4:38 PM, Stefan Stanacar <sst...@gmail.com> wrote:

> do_unpack is by default in SRCTREECOVEREDTASKS so this append can't run,
> since
> do_unpack gets removed by when externalsrc is enabled.
>
> This was hidden because externalsrc does actually run do_fetch and
> do_unpack if
> there are type=kmeta or file:// entries in the SRC_URI value of the kernel
> recipe.
> (which is the case for linux-yocto).
>
> Make this a separate task so that it actually gets run for kernel recipes
> with
> no file:// or type=kmeta in SRC_URI.
>
> Signed-off-by: Stefan Stanacar <sst...@gmail.com>
> ---
>  meta/classes/kernel.bbclass | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
> index 2f6eca3..6a8d6e8 100644
> --- a/meta/classes/kernel.bbclass
> +++ b/meta/classes/kernel.bbclass
> @@ -125,7 +125,7 @@ inherit ${KERNEL_CLASSES}
>  # the symlink.
>  do_unpack[cleandirs] += " ${S} ${STAGING_KERNEL_DIR} ${B}
> ${STAGING_KERNEL_BUILDDIR}"
>  do_clean[cleandirs] += " ${S} ${STAGING_KERNEL_DIR} ${B}
> ${STAGING_KERNEL_BUILDDIR}"
> -base_do_unpack_append () {
> +python do_symlink_staging_dir () {
>  s = d.getVar("S")
>  if s[-1] == '/':
>  # drop trailing slash, so that os.symlink(kernsrc, s) doesn't use
> s as directory name and fail
> @@ -142,6 +142,7 @@ base_do_unpack_append () {
>  shutil.move(s, kernsrc)
>  os.symlink(kernsrc, s)
>  }
> +addtask do_symlink_staging_dir after do_unpack before do_patch
> do_configure
>
>  inherit kernel-arch deploy
>
> --
> 2.7.4
>
>
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [morty][PATCH] kernel.bbclass: Fix symlink creation when using externalsrc

2018-01-29 Thread Stefan Stanacar

do_unpack is by default in SRCTREECOVEREDTASKS so this append can't run, since
do_unpack gets removed by when externalsrc is enabled.

This was hidden because externalsrc does actually run do_fetch and do_unpack if
there are type=kmeta or file:// entries in the SRC_URI value of the kernel 
recipe.
(which is the case for linux-yocto).

Make this a separate task so that it actually gets run for kernel recipes with
no file:// or type=kmeta in SRC_URI.

Signed-off-by: Stefan Stanacar <sst...@gmail.com>
---
 meta/classes/kernel.bbclass | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index f8318b8..3f2d796 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -102,7 +102,7 @@ inherit ${KERNEL_CLASSES}
 # the symlink.
 do_unpack[cleandirs] += " ${S} ${STAGING_KERNEL_DIR} ${B} 
${STAGING_KERNEL_BUILDDIR}"
 do_clean[cleandirs] += " ${S} ${STAGING_KERNEL_DIR} ${B} 
${STAGING_KERNEL_BUILDDIR}"
-base_do_unpack_append () {
+python do_symlink_staging_dir () {
 s = d.getVar("S", True)
 if s[-1] == '/':
 # drop trailing slash, so that os.symlink(kernsrc, s) doesn't use s as 
directory name and fail
@@ -119,6 +119,7 @@ base_do_unpack_append () {
 shutil.move(s, kernsrc)
 os.symlink(kernsrc, s)
 }
+addtask do_symlink_staging_dir after do_unpack before do_patch do_configure
 
 inherit kernel-arch deploy
 
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [morty][PATCH] kernelsrc/perf: fix dependency on non existant task when using externalsrc

2018-01-29 Thread Stefan Stanacar

When externalsrc is enabled for kernel, do_patch doesn't exist since is in
SRCTREECOVEREDTASKS, so make these depend on a real task.

Fixes:
ERROR: Task do_unpack in /data/yocto/poky/meta/recipes-kernel/perf/perf.bb
depends upon non-existent task do_patch in 
/data/yocto/poky/meta/recipes-kernel/linux/linux-yocto_4.8.bb

Signed-off-by: Stefan Stanacar <sst...@gmail.com>
---
 meta/classes/kernelsrc.bbclass   | 2 +-
 meta/recipes-kernel/perf/perf.bb | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/classes/kernelsrc.bbclass b/meta/classes/kernelsrc.bbclass
index 9efd46a..ce6c999 100644
--- a/meta/classes/kernelsrc.bbclass
+++ b/meta/classes/kernelsrc.bbclass
@@ -1,6 +1,6 @@
 S = "${STAGING_KERNEL_DIR}"
 do_fetch[noexec] = "1"
-do_unpack[depends] += "virtual/kernel:do_patch"
+do_unpack[depends] += "virtual/kernel:do_configure"
 do_unpack[noexec] = "1"
 do_patch[noexec] = "1"
 do_package[depends] += "virtual/kernel:do_populate_sysroot"
diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb
index 03ae446..145774b 100644
--- a/meta/recipes-kernel/perf/perf.bb
+++ b/meta/recipes-kernel/perf/perf.bb
@@ -47,7 +47,7 @@ export PYTHON_SITEPACKAGES_DIR
 #kernel 3.1+ supports WERROR to disable warnings as errors
 export WERROR = "0"
 
-do_populate_lic[depends] += "virtual/kernel:do_patch"
+do_populate_lic[depends] += "virtual/kernel:do_configure"
 
 # needed for building the tools/perf Perl binding
 inherit perlnative cpan-base
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] kernel.bbclass: Fix symlink creation when using externalsrc

2018-01-29 Thread Stefan Stanacar

do_unpack is by default in SRCTREECOVEREDTASKS so this append can't run, since
do_unpack gets removed by when externalsrc is enabled.

This was hidden because externalsrc does actually run do_fetch and do_unpack if
there are type=kmeta or file:// entries in the SRC_URI value of the kernel 
recipe.
(which is the case for linux-yocto).

Make this a separate task so that it actually gets run for kernel recipes with
no file:// or type=kmeta in SRC_URI.

Signed-off-by: Stefan Stanacar <sst...@gmail.com>
---
 meta/classes/kernel.bbclass | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 2f6eca3..6a8d6e8 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -125,7 +125,7 @@ inherit ${KERNEL_CLASSES}
 # the symlink.
 do_unpack[cleandirs] += " ${S} ${STAGING_KERNEL_DIR} ${B} 
${STAGING_KERNEL_BUILDDIR}"
 do_clean[cleandirs] += " ${S} ${STAGING_KERNEL_DIR} ${B} 
${STAGING_KERNEL_BUILDDIR}"
-base_do_unpack_append () {
+python do_symlink_staging_dir () {
 s = d.getVar("S")
 if s[-1] == '/':
 # drop trailing slash, so that os.symlink(kernsrc, s) doesn't use s as 
directory name and fail
@@ -142,6 +142,7 @@ base_do_unpack_append () {
 shutil.move(s, kernsrc)
 os.symlink(kernsrc, s)
 }
+addtask do_symlink_staging_dir after do_unpack before do_patch do_configure
 
 inherit kernel-arch deploy
 
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [morty][PATCH] kernel.bbclass: Fix symlink creation when using externalsrc

2018-01-18 Thread Stefan Stanacar

On Wed, Jan 17, 2018 at 9:04 PM, akuster808 <akuster...@gmail.com> wrote:

>
>
> On 01/11/2018 08:07 AM, Stefan Stanacar wrote:
> > do_unpack is by default in SRCTREECOVEREDTASKS so this append can't run,
> since
> > this tasks gets removed by externalsrc when it's enabled.
> >
> > However this was hidden because externalsrc does run do_fetch and
> do_unpack if
> > there are type=kmeta or file:// entries in the SRC_URI value of the
> kernel recipe.
> > (e.g linux-yocto).
> >
> > Make this a separate task so that it actually gets run for kernel
> recipes with
> > no file:// or type=kmeta in SRC_URI.
>
> Is this fixed in Master?
>
>
Nope, I've sent a patch for master as well:
http://lists.openembedded.org/pipermail/openembedded-core/2018-January/146355.html
but it wasn't merged (yet?)

Cheers,
Stefan


> - armin
> > Signed-off-by: Stefan Stanacar <sst...@gmail.com>
> > ---
> >  meta/classes/kernel.bbclass | 4 +++-
> >  1 file changed, 3 insertions(+), 1 deletion(-)
> >
> > diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
> > index f8318b8..3eaae03 100644
> > --- a/meta/classes/kernel.bbclass
> > +++ b/meta/classes/kernel.bbclass
> > @@ -102,7 +102,7 @@ inherit ${KERNEL_CLASSES}
> >  # the symlink.
> >  do_unpack[cleandirs] += " ${S} ${STAGING_KERNEL_DIR} ${B}
> ${STAGING_KERNEL_BUILDDIR}"
> >  do_clean[cleandirs] += " ${S} ${STAGING_KERNEL_DIR} ${B}
> ${STAGING_KERNEL_BUILDDIR}"
> > -base_do_unpack_append () {
> > +python do_symlink_staging_dir () {
> >  s = d.getVar("S", True)
> >  if s[-1] == '/':
> >  # drop trailing slash, so that os.symlink(kernsrc, s) doesn't
> use s as directory name and fail
> > @@ -119,6 +119,8 @@ base_do_unpack_append () {
> >  shutil.move(s, kernsrc)
> >  os.symlink(kernsrc, s)
> >  }
> > +addtask do_symlink_staging_dir after do_unpack before do_patch
> do_configure
> > +
> >
> >  inherit kernel-arch deploy
> >
>
>
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [morty][PATCH] kernelsrc/perf: fix dependency on non existant task when using externalsrc

2018-01-12 Thread Stefan Stanacar

On Fri, Jan 12, 2018 at 6:21 PM, Joshua Watt <jpewhac...@gmail.com> wrote:

> On Thu, 2018-01-11 at 16:07 +0000, Stefan Stanacar wrote:
> > When externalsrc is enabled for kernel, do_patch doesn't exist since
> > is in
> > SRCTREECOVEREDTASKS, so make these depend on a real task.
> >
> > Fixes:
> > ERROR: Task do_unpack in /data/yocto/poky/meta/recipes-
> > kernel/perf/perf.bb
> > depends upon non-existent task do_patch in
> > /data/yocto/poky/meta/recipes-kernel/linux/linux-yocto_4.8.bb
> >
> > Signed-off-by: Stefan Stanacar <sst...@gmail.com>
> > ---
> >  meta/classes/kernelsrc.bbclass   | 2 +-
> >  meta/recipes-kernel/perf/perf.bb | 2 +-
> >  2 files changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/meta/classes/kernelsrc.bbclass
> > b/meta/classes/kernelsrc.bbclass
> > index 9efd46a..ce6c999 100644
> > --- a/meta/classes/kernelsrc.bbclass
> > +++ b/meta/classes/kernelsrc.bbclass
> > @@ -1,6 +1,6 @@
> >  S = "${STAGING_KERNEL_DIR}"
> >  do_fetch[noexec] = "1"
> > -do_unpack[depends] += "virtual/kernel:do_patch"
> > +do_unpack[depends] += "virtual/kernel:do_configure"
>
> I was looking at fixing this as well, but I had another approach.
> Instead of perpetually changing these to handle externalsrc corner
> cases, perhaps externalsrc.bbclass should not delete the tasks, but
> instead mark them as noexec as follows (I'll push this up for real if
> in a moment...). Then recipes should be able to work transparently
> regardless of externalsrc being used or not.
>
>

+1
Yup, I like this, it's a better fix. I've sent another patch for
kernel.bbclass (both for master and morty) which I think is still needed
even with your patch.

Cheers,
Stefan



> diff --git a/meta/classes/externalsrc.bbclass
> b/meta/classes/externalsrc.bbclass
> index 65dd13ddc1f..12046d74e17 100644
> --- a/meta/classes/externalsrc.bbclass
> +++ b/meta/classes/externalsrc.bbclass
> @@ -102,7 +102,7 @@ python () {
>  for task in d.getVar("SRCTREECOVEREDTASKS").split():
>  if local_srcuri and task in fetch_tasks:
>  continue
> -bb.build.deltask(task, d)
> +d.setVarFlag(task, 'noexec', '1')
>
>  d.prependVarFlag('do_compile', 'prefunc
> s', "externalsrc_compile_prefunc ")
>  d.prependVarFlag('do_configure', 'prefuncs',
> "externalsrc_configure_prefunc ")
>
>
> >  do_unpack[noexec] = "1"
> >  do_patch[noexec] = "1"
> >  do_package[depends] += "virtual/kernel:do_populate_sysroot"
> > diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-
> > kernel/perf/perf.bb
> > index 03ae446..145774b 100644
> > --- a/meta/recipes-kernel/perf/perf.bb
> > +++ b/meta/recipes-kernel/perf/perf.bb
> > @@ -47,7 +47,7 @@ export PYTHON_SITEPACKAGES_DIR
> >  #kernel 3.1+ supports WERROR to disable warnings as errors
> >  export WERROR = "0"
> >
> > -do_populate_lic[depends] += "virtual/kernel:do_patch"
> > +do_populate_lic[depends] += "virtual/kernel:do_configure"
> >
> >  # needed for building the tools/perf Perl binding
> >  inherit perlnative cpan-base
> > --
> > 2.7.4
> >
>
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] kernel.bbclass: Fix symlink creation when using externalsrc

2018-01-12 Thread Stefan Stanacar

do_unpack is by default in SRCTREECOVEREDTASKS so this append can't run, since
this tasks gets removed by externalsrc when it's enabled.

However this was hidden because externalsrc does run do_fetch and do_unpack if
there are type=kmeta or file:// entries in the SRC_URI value of the kernel 
recipe.
(e.g linux-yocto).

Make this a separate task so that it actually gets run for kernel recipes with
no file:// or type=kmeta in SRC_URI.

Signed-off-by: Stefan Stanacar <sst...@gmail.com>
---
 meta/classes/kernel.bbclass | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index f7b612f..6f0a117 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -125,7 +125,7 @@ inherit ${KERNEL_CLASSES}
 # the symlink.
 do_unpack[cleandirs] += " ${S} ${STAGING_KERNEL_DIR} ${B} 
${STAGING_KERNEL_BUILDDIR}"
 do_clean[cleandirs] += " ${S} ${STAGING_KERNEL_DIR} ${B} 
${STAGING_KERNEL_BUILDDIR}"
-base_do_unpack_append () {
+python do_symlink_staging_dir () {
 s = d.getVar("S")
 if s[-1] == '/':
 # drop trailing slash, so that os.symlink(kernsrc, s) doesn't use s as 
directory name and fail
@@ -142,6 +142,8 @@ base_do_unpack_append () {
 shutil.move(s, kernsrc)
 os.symlink(kernsrc, s)
 }
+addtask do_symlink_staging_dir after do_unpack before do_patch do_configure
+
 
 inherit kernel-arch deploy
 
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [morty][PATCH] kernel.bbclass: Fix symlink creation when using externalsrc

2018-01-11 Thread Stefan Stanacar

do_unpack is by default in SRCTREECOVEREDTASKS so this append can't run, since
this tasks gets removed by externalsrc when it's enabled.

However this was hidden because externalsrc does run do_fetch and do_unpack if
there are type=kmeta or file:// entries in the SRC_URI value of the kernel 
recipe.
(e.g linux-yocto).

Make this a separate task so that it actually gets run for kernel recipes with
no file:// or type=kmeta in SRC_URI.

Signed-off-by: Stefan Stanacar <sst...@gmail.com>
---
 meta/classes/kernel.bbclass | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index f8318b8..3eaae03 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -102,7 +102,7 @@ inherit ${KERNEL_CLASSES}
 # the symlink.
 do_unpack[cleandirs] += " ${S} ${STAGING_KERNEL_DIR} ${B} 
${STAGING_KERNEL_BUILDDIR}"
 do_clean[cleandirs] += " ${S} ${STAGING_KERNEL_DIR} ${B} 
${STAGING_KERNEL_BUILDDIR}"
-base_do_unpack_append () {
+python do_symlink_staging_dir () {
 s = d.getVar("S", True)
 if s[-1] == '/':
 # drop trailing slash, so that os.symlink(kernsrc, s) doesn't use s as 
directory name and fail
@@ -119,6 +119,8 @@ base_do_unpack_append () {
 shutil.move(s, kernsrc)
 os.symlink(kernsrc, s)
 }
+addtask do_symlink_staging_dir after do_unpack before do_patch do_configure
+
 
 inherit kernel-arch deploy
 
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [morty][PATCH] kernelsrc/perf: fix dependency on non existant task when using externalsrc

2018-01-11 Thread Stefan Stanacar

When externalsrc is enabled for kernel, do_patch doesn't exist since is in
SRCTREECOVEREDTASKS, so make these depend on a real task.

Fixes:
ERROR: Task do_unpack in /data/yocto/poky/meta/recipes-kernel/perf/perf.bb
depends upon non-existent task do_patch in 
/data/yocto/poky/meta/recipes-kernel/linux/linux-yocto_4.8.bb

Signed-off-by: Stefan Stanacar <sst...@gmail.com>
---
 meta/classes/kernelsrc.bbclass   | 2 +-
 meta/recipes-kernel/perf/perf.bb | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/classes/kernelsrc.bbclass b/meta/classes/kernelsrc.bbclass
index 9efd46a..ce6c999 100644
--- a/meta/classes/kernelsrc.bbclass
+++ b/meta/classes/kernelsrc.bbclass
@@ -1,6 +1,6 @@
 S = "${STAGING_KERNEL_DIR}"
 do_fetch[noexec] = "1"
-do_unpack[depends] += "virtual/kernel:do_patch"
+do_unpack[depends] += "virtual/kernel:do_configure"
 do_unpack[noexec] = "1"
 do_patch[noexec] = "1"
 do_package[depends] += "virtual/kernel:do_populate_sysroot"
diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb
index 03ae446..145774b 100644
--- a/meta/recipes-kernel/perf/perf.bb
+++ b/meta/recipes-kernel/perf/perf.bb
@@ -47,7 +47,7 @@ export PYTHON_SITEPACKAGES_DIR
 #kernel 3.1+ supports WERROR to disable warnings as errors
 export WERROR = "0"
 
-do_populate_lic[depends] += "virtual/kernel:do_patch"
+do_populate_lic[depends] += "virtual/kernel:do_configure"
 
 # needed for building the tools/perf Perl binding
 inherit perlnative cpan-base
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH V2 6/7] controllers/masterimage.py: Make testimage kernel naming pattern universal

2014-06-07 Thread Stefan Stanacar

On Jun 6, 2014 8:11 PM, Corneliu Stoicescu corneliux.stoice...@intel.com
wrote:

 YB: #6254

 The QemuTargetControl class does not specify any kernel naming but the
runqemu script it uses uses the  KERNEL_IMAGETYPE + MACHINE + '.bin' 
naming scheme.
 Modifying the other major target controller class,
MasterImageHardwareTarget, to use the same kernel naming scheme.


You can't do that because not all targets (bbb, nuc, etc) use a bin
kernel...

 This is usefull also to outside scripts that want to anticipate the
kernel file name for all target controllers.

And what's wrong with the link? Bitbake will always take care of that and
point to the real file why can't the script dereference that?

Cheers,
Stefan


 Signed-off-by: Corneliu Stoicescu corneliux.stoice...@intel.com
 ---
  meta/lib/oeqa/controllers/masterimage.py | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

 diff --git a/meta/lib/oeqa/controllers/masterimage.py
b/meta/lib/oeqa/controllers/masterimage.py
 index b03c5bf..311f0cf 100644
 --- a/meta/lib/oeqa/controllers/masterimage.py
 +++ b/meta/lib/oeqa/controllers/masterimage.py
 @@ -52,7 +52,7 @@ class
MasterImageHardwareTarget(oeqa.targetcontrol.BaseTarget):
  # test rootfs + kernel
  self.image_fstype = self.get_image_fstype(d)
  self.rootfs = os.path.join(d.getVar(DEPLOY_DIR_IMAGE, True),
d.getVar(IMAGE_LINK_NAME, True) + '.' + self.image_fstype)
 -self.kernel = os.path.join(d.getVar(DEPLOY_DIR_IMAGE, True),
d.getVar(KERNEL_IMAGETYPE))
 +self.kernel = os.path.join(d.getVar(DEPLOY_DIR_IMAGE, True),
d.getVar(KERNEL_IMAGETYPE) + '-' + d.getVar('MACHINE') + '.bin')
  if not os.path.isfile(self.rootfs):
  # we could've checked that IMAGE_FSTYPES contains tar.gz but
the config for running testimage might not be
  # the same as the config with which the image was build, ie
 --
 1.8.3.2

 --
 ___
 Openembedded-core mailing list
 Openembedded-core@lists.openembedded.org
 http://lists.openembedded.org/mailman/listinfo/openembedded-core
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH V2 1/7] targetcontrol.py: make possible dynamical determination of rootfs type

2014-06-07 Thread Stefan Stanacar

Hi Corneliu,

On Fri, Jun 6, 2014 at 10:14 PM, Corneliu Stoicescu
corneliux.stoice...@intel.com wrote:
 YB: #6375

 Added a new method get_image_fstype() that autodetermines what fstype to use 
 for the rootfs file.
 This method uses a new list variable 'supported_image_fstypes' that contains 
 image fstypes supported by the target controller.

 This method is also a classmethod which means outside scripts can get the 
 image fstype.

 Signed-off-by: Corneliu Stoicescu corneliux.stoice...@intel.com
 ---
  meta/lib/oeqa/targetcontrol.py | 19 +--
  1 file changed, 17 insertions(+), 2 deletions(-)

 diff --git a/meta/lib/oeqa/targetcontrol.py b/meta/lib/oeqa/targetcontrol.py
 index ff1bb89..866c414 100644
 --- a/meta/lib/oeqa/targetcontrol.py
 +++ b/meta/lib/oeqa/targetcontrol.py
 @@ -44,6 +44,8 @@ class BaseTarget(object):

  __metaclass__ = ABCMeta

 +supported_image_fstypes = []
 +
  def __init__(self, d):
  self.connection = None
  self.ip = None
 @@ -70,6 +72,16 @@ class BaseTarget(object):
  def stop(self):
  pass

 +@classmethod
 +def get_image_fstype(self, d, image_fstypes=None):

You made this a static method so you can call it from an external
script? Hmm.. But then you wouldn't be able to pass d, so what's the
point? Adding support for more fs is nice, but you should do that for
testimage, ie make it configurable and let the use choose from
local.conf (with a reasonable default). Right now you are configuring
the test bits from outside...

Cheers,
Stefan

 +if not image_fstypes:
 +image_fstypes = d.getVar('IMAGE_FSTYPES', True).split(' ')
 +possible_image_fstypes = [fstype for fstype in 
 self.supported_image_fstypes if fstype in image_fstypes]
 +if possible_image_fstypes:
 +return possible_image_fstypes[0]
 +else:
 +bb.fatal(no possible image_fstype could not be determined. 
 IMAGE_FSTYPES=\%s\ and supported_image_fstypes=\%s\:  % (', 
 '.join(map(str, image_fstypes)), ', '.join(map(str, 
 self.supported_image_fstypes
 +
  def restart(self, params=None):
  self.stop()
  self.start(params)
 @@ -87,13 +99,16 @@ class BaseTarget(object):

  class QemuTarget(BaseTarget):

 +supported_image_fstypes = ['ext3']
 +
  def __init__(self, d):

  super(QemuTarget, self).__init__(d)

 +self.image_fstype = self.get_image_fstype(d)
  self.qemulog = os.path.join(self.testdir, qemu_boot_log.%s % 
 self.datetime)
 -self.origrootfs = os.path.join(d.getVar(DEPLOY_DIR_IMAGE, True),  
 d.getVar(IMAGE_LINK_NAME, True) + '.ext3')
 -self.rootfs = os.path.join(self.testdir, d.getVar(IMAGE_LINK_NAME, 
 True) + '-testimage.ext3')
 +self.origrootfs = os.path.join(d.getVar(DEPLOY_DIR_IMAGE, True),  
 d.getVar(IMAGE_LINK_NAME, True) + '.' + self.image_fstype)
 +self.rootfs = os.path.join(self.testdir, d.getVar(IMAGE_LINK_NAME, 
 True) + '-testimage.' + self.image_fstype)

  self.runner = QemuRunner(machine=d.getVar(MACHINE, True),
  rootfs=self.rootfs,
 --
 1.8.3.2

 --
 ___
 Openembedded-core mailing list
 Openembedded-core@lists.openembedded.org
 http://lists.openembedded.org/mailman/listinfo/openembedded-core
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH V2 6/7] controllers/masterimage.py: Make testimage kernel naming pattern universal

2014-06-07 Thread Stefan Stanacar

On Sat, Jun 7, 2014 at 1:54 PM, Stefan Stanacar sst...@gmail.com wrote:

 On Jun 6, 2014 8:11 PM, Corneliu Stoicescu corneliux.stoice...@intel.com
 wrote:

 YB: #6254

 The QemuTargetControl class does not specify any kernel naming but the
 runqemu script it uses uses the  KERNEL_IMAGETYPE + MACHINE + '.bin' 
 naming scheme.
 Modifying the other major target controller class,
 MasterImageHardwareTarget, to use the same kernel naming scheme.


 You can't do that because not all targets (bbb, nuc, etc) use a bin
 kernel...

Ah, misunderstood... right, there is .bin link too

 This is usefull also to outside scripts that want to anticipate the kernel
 file name for all target controllers.

 And what's wrong with the link? Bitbake will always take care of that and
 point to the real file why can't the script dereference that?

 Cheers,
 Stefan


 Signed-off-by: Corneliu Stoicescu corneliux.stoice...@intel.com
 ---
  meta/lib/oeqa/controllers/masterimage.py | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

 diff --git a/meta/lib/oeqa/controllers/masterimage.py
 b/meta/lib/oeqa/controllers/masterimage.py
 index b03c5bf..311f0cf 100644
 --- a/meta/lib/oeqa/controllers/masterimage.py
 +++ b/meta/lib/oeqa/controllers/masterimage.py
 @@ -52,7 +52,7 @@ class
 MasterImageHardwareTarget(oeqa.targetcontrol.BaseTarget):
  # test rootfs + kernel
  self.image_fstype = self.get_image_fstype(d)
  self.rootfs = os.path.join(d.getVar(DEPLOY_DIR_IMAGE, True),
 d.getVar(IMAGE_LINK_NAME, True) + '.' + self.image_fstype)
 -self.kernel = os.path.join(d.getVar(DEPLOY_DIR_IMAGE, True),
 d.getVar(KERNEL_IMAGETYPE))
 +self.kernel = os.path.join(d.getVar(DEPLOY_DIR_IMAGE, True),
 d.getVar(KERNEL_IMAGETYPE) + '-' + d.getVar('MACHINE') + '.bin')
  if not os.path.isfile(self.rootfs):
  # we could've checked that IMAGE_FSTYPES contains tar.gz but
 the config for running testimage might not be
  # the same as the config with which the image was build, ie
 --
 1.8.3.2

 --
 ___
 Openembedded-core mailing list
 Openembedded-core@lists.openembedded.org
 http://lists.openembedded.org/mailman/listinfo/openembedded-core
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH V2 7/7] scripts/test-remote-image: Add script for running runtime tests on remotely built images

2014-06-07 Thread Stefan Stanacar

Hi Corneliu,

I haven't carefully reviewed this series so if I didn't quite got the
point of some things, sorry...
At a first look, here are my thoughts (this is actually a reply to the
entire series, I seem to have lost the cover letter):

- It seems wierd that you instantiate the target controllers classes
here... You do that so you can get the extra files, but that has
nothing to do with testimage anyway. Let bitbake/testimage do it's
part and it's configuration. Similar for image fs types (see my other
reply)
- I wouldn't mix the download stuff with the test/deploy stuff
(bitbake's job). You already have command line configuration knobs,
I'm sure you could add one for extra files... Surely the user know
it's testing BBB and core-image-minimal and it needs to download the
dtbs (because you need the dtbs only for minimal).
- the support for other fs types should be separate from the test
external built image. E.g bbb uses tar.bz2, so even at this point you
can still test external images only with tar.gz (adding IMAGE_FSTYPES
+= tar.gz to the Autobuilder config is a easy task). What I'm
trying to say is that this is a user/build config option and this
script shouldn't deal with every little thing
- What's with the postconfig stuff? QA_MACHINE = MACHINE? Same for
distro.. I really don't get that part...
- I'm missing the bigger picture here with all the profiles stuff
(plus some classes are unnecessarily abstract ).

So the whole point of this was to replace (assume default local.conf
and MACHINE=beaglebone here)
this:
---
$ vi conf/local.conf
INHERIT += testimage
TEST_TARGET_IP = 10.11.12.2
TEST_TARGET = BeagleBoneTarget
TEST_SERIALCONTROL_CMD = picocom /dev/ttyUSB0 -b 115200
$ bitbake core-image-minimal # (or sato / sato-sdk)
$ bitbake image -c testimage
---
with this (except the local.conf part which should already be there,
it is not the job of this script to add that):

$ bitbake rpm psplash # rpm/smart tests require these, else skip
$ bitbake package-index
$ wget -r -l0 -nd -np -P tmp/deploy/images
http://autobuilder.yoctoproject.org/pub/nightly/20140606-3/machines/beaglebone/
$ bitbake image -c testimage
---
#yes, I cheated there, getting all the files, but you get the point..
alternatively:
wget -r -l0 -nd -np -P tmp/deploy/images -A *boneblack*.dtb
http://autobuilder.yoctoproject.org/pub/nightly/20140606-3/machines/beaglebone/
wget -P tmp/deploy/images
http://autobuilder.yoctoproject.org/pub/nightly/20140606-3/machines/beaglebone/core-image-minimal-beaglebone.manifest
wget -P tmp/deploy/images
http://autobuilder.yoctoproject.org/pub/nightly/20140606-3/machines/beaglebone/core-image-minimal-beaglebone.tar.gz
wget -P tmp/deploy/images
http://autobuilder.yoctoproject.org/pub/nightly/20140606-3/machines/beaglebone/uImage

It seems to me that you are doing a lot more with this series and
adding unnecessary complications. I guess I'm missing some pieces
here...

Cheers,
Stefan

On Fri, Jun 6, 2014 at 10:14 PM, Corneliu Stoicescu
corneliux.stoice...@intel.com wrote:
YB: #6254

Adding a new script that will fetch image files from a remote images
repository.
These images will then be used for local runtime testing.

Use the '-h' option for more details on usage.

Signed-off-by: Corneliu Stoicescu corneliux.stoice...@intel.com
---
scripts/test-remote-image | 340
++
1 file changed, 340 insertions(+)
create mode 100755 scripts/test-remote-image

diff --git a/scripts/test-remote-image b/scripts/test-remote-image
new file mode 100755
index 000..6da6672
--- /dev/null
+++ b/scripts/test-remote-image
@@ -0,0 +1,340 @@
+#!/usr/bin/env python
+
+# Copyright (c) 2014 Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+# DESCRIPTION
+# This script is used to test public autobuilder images on remote hardware.
+# The script is called from a machine that is able download the images from
the remote images repository and to connect to the test hardware.
+#
+# test-remote-image --image-type core-image-sato --repo-link
http://192.168.10.2/images --required-packages rpm psplash
+#
+# Translation: Build the 'rpm' and 'pslash' packages and test a remote
core-image-sato image using the http://192.168.10.2/images repository.
+#
+# You can also use the '-h' option to see some help information.
+
+import os

Re: [OE-core] [PATCH] runqemu-internal: add NIC to kernel parameters

2014-05-29 Thread Stefan Stanacar

On May 29, 2014 2:16 PM, Richard Purdie 
richard.pur...@linuxfoundation.org wrote:

 On Tue, 2014-05-27 at 12:22 +0200, Adrian Freihofer wrote:
  Fixes [YOCTO #6356]
 
  Problem: If QEMU parameters are extended to emulate additional
  network interfaces, NFS boot does not work. Example:
  runqemu ... qemuparams=-net nic,model=virtio... -net tap, ...
  This is solved by extending the kernel parameters by ::eth0:on
  ---
   scripts/runqemu-internal | 2 +-
   1 file changed, 1 insertion(+), 1 deletion(-)
 
  diff --git a/scripts/runqemu-internal b/scripts/runqemu-internal
  index d317dfd..30d6514 100755
  --- a/scripts/runqemu-internal
  +++ b/scripts/runqemu-internal
  @@ -249,7 +249,7 @@ else
   n1=$(($n0 * 2 + 1))
   n2=$(($n1 + 1))
 
  -
 KERNEL_NETWORK_CMD=ip=192.168.7.$n2::192.168.7.$n1:255.255.255.0
  +
 KERNEL_NETWORK_CMD=ip=192.168.7.$n2::192.168.7.$n1:255.255.255.0::eth0:on
   QEMU_TAP_CMD=-net
tap,vlan=0,ifname=$TAP,script=no,downscript=no
   if [ $KVM_ACTIVE = yes ]; then
   QEMU_NETWORK_CMD=-net nic,model=virtio
$QEMU_TAP_CMD,vhost=on

 Unfortunately this results in failures in all our automated sanity
 tests:


It should be noted that minimal images do have network so this might upset
connman...

Cheers,
Stefan

https://autobuilder.yoctoproject.org/main/builders/nightly-ipk/builds/94/steps/Running%20Sanity%20Tests_1/logs/stdio

 Cheers,

 Richard

 --
 ___
 Openembedded-core mailing list
 Openembedded-core@lists.openembedded.org
 http://lists.openembedded.org/mailman/listinfo/openembedded-core
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] oeqa/controllers/beaglebonetarget.py: fix conditions for files copied to /boot

2014-05-27 Thread Stefan Stanacar

On Tue, May 27, 2014 at 12:50 PM, Corneliu Stoicescu
corneliux.stoice...@intel.com wrote:

 ---
  meta-yocto-bsp/lib/oeqa/controllers/beaglebonetarget.py | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)


As this patch is for meta-yocto-bsp this should be sent to
p...@yoctoproject.org list...

 diff --git a/meta-yocto-bsp/lib/oeqa/controllers/beaglebonetarget.py 
 b/meta-yocto-bsp/lib/oeqa/controllers/beaglebonetarget.py
 index 53f454b..af56bda 100644
 --- a/meta-yocto-bsp/lib/oeqa/controllers/beaglebonetarget.py
 +++ b/meta-yocto-bsp/lib/oeqa/controllers/beaglebonetarget.py
 @@ -41,12 +41,12 @@ class BeagleBoneTarget(MasterImageHardwareTarget):
  'mount -L testrootfs /mnt/testrootfs',
  'rm -rf /mnt/testrootfs/*',
  'tar xzvf ~/test-rootfs.tar.gz -C /mnt/testrootfs',
 -'[ ! -e /mnt/testrootfs/boot/uImage ]  cp ~/test-kernel 
 /mnt/testrootfs/boot/uImage',
 +'if [ ! -e /mnt/testrootfs/boot/uImage ]; then cp 
 ~/test-kernel /mnt/testrootfs/boot/uImage; fi',

How about reversing the check: [ -e /mnt/testrootfs/boot/uImage] || cp
~/test-kernel /mnt/testrootfs/boot/uImage
That would work in both cases, if it exists it won't copy the kernel
and returns 0, if it doesn't it will copy and returns the result of
cp.

  ]

  for _, dtbfn in self.dtbs:
  # Kernel and dtb files may not be in the image, so copy them if 
 not
 -self.deploy_cmds.append('[ ! -e /mnt/testrootfs/boot/{0} ]  cp 
 ~/{0} /mnt/testrootfs/boot/'.format(dtbfn))
 +self.deploy_cmds.append('if [ ! -e /mnt/testrootfs/boot/{0} ]; 
 then cp ~/{0} /mnt/testrootfs/boot/'.format(dtbfn) + '; fi')


Same here.


Cheers,
Stefan

  if not self.serialcontrol_cmd:
  bb.fatal(This TEST_TARGET needs a TEST_SERIALCONTROL_CMD 
 defined in local.conf.)
 --
 1.8.3.2

 --
 ___
 Openembedded-core mailing list
 Openembedded-core@lists.openembedded.org
 http://lists.openembedded.org/mailman/listinfo/openembedded-core
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PTACH V2] oeqa/runtime: Update test names with testopia tc numbers.

2014-05-19 Thread Stefan Stanacar

Hi Lucian,

On Fri, May 16, 2014 at 11:44 AM, Lucian Musat
georgex.l.mu...@intel.com wrote:

  class SmartRepoTest(SmartTest):
 @@ -54,11 +54,12 @@ class SmartRepoTest(SmartTest):
  @classmethod
  def tearDownClass(self):
  self.repo_server.stop()
 +   smartsRepo = SmartRepoTest


Again, this line doesn't make sense and it probably shouldn't be here.

  class SyslogTest(oeRuntimeTest):

 -@skipUnlessPassed(test_ssh)
 -def test_syslog_help(self):
 -(status,output) = self.target.run('/sbin/syslogd --help')
 -self.assertEqual(status, 0, msg=status and output: %s and %s % 
 (status,output))
 +@skipUnlessPassed(test_ssh_tc_224)
 +def test_syslog_help_tc_202(self):
 +   (status,output) = self.target.run('/sbin/syslogd --help')
 +   self.assertEqual(status, 0, msg=status and output: %s and 
 %s % (status,output))


There are some whitespaces/tabs added here that probably shouldn't be.

Please also add a description why this patch is needed. Why do we need
to add testopia numbers to test names, how is it useful?
Also, isn't there any other way we could map these tests to testopia testcases?

Cheers,
Stefan
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] oeqa/runtime: Update test names with testopia tc numbers.

2014-05-14 Thread Stefan Stanacar

Hi Lucian,

I'm afraid that you added some lines in the smart.py module, that
shouldn't be there. Also a patch description would be nice, explaining
why this is needed.

On Tue, May 13, 2014 at 2:43 PM, Lucian Musat georgex.l.mu...@intel.com wrote:
 diff --git a/meta/lib/oeqa/runtime/smart.py b/meta/lib/oeqa/runtime/smart.py
 index 195f117..735cbca 100644
 --- a/meta/lib/oeqa/runtime/smart.py
 +++ b/meta/lib/oeqa/runtime/smart.py
 @@ -14,7 +14,7 @@ def setUpModule():

  class SmartRepoTest(SmartTest):
 @@ -54,11 +54,13 @@ class SmartRepoTest(SmartTest):
  @classmethod
  def tearDownClass(self):
  self.repo_server.stop()
 +   smartsRepo = SmartRepoTest
 +   smartsRepo.cleanup()


Those two lines shouldn't be here  and they don't seem to be correct anyway.


  class SyslogTest(oeRuntimeTest):

 -@skipUnlessPassed(test_ssh)
 -def test_syslog_help(self):
 -(status,output) = self.target.run('/sbin/syslogd --help')
 -self.assertEqual(status, 0, msg=status and output: %s and %s % 
 (status,output))
 -
 -@skipUnlessPassed(test_syslog_help)
 -def test_syslog_running(self):
 +@skipUnlessPassed(test_ssh_tc_224)
 +def test_syslog_help_tc_202(self):
 +(status,output) = self.target.run('ls -l /sbin/syslogd')
 +if ('busybox' in output):
 +   pass
 +   else:
 +   (status,output) = self.target.run('/sbin/syslogd --help')
 +   self.assertEqual(status, 0, msg=status and output: %s and 
 %s % (status,output))
 +
 +@skipUnlessPassed(test_syslog_help_tc_202)
 +def test_syslog_running_tc_202(self):
  (status,output) = self.target.run(oeRuntimeTest.pscmd + ' | grep -i 
 [s]yslogd')
  self.assertEqual(status, 0, msg=no syslogd process, ps output: %s 
 % self.target.run(oeRuntimeTest.pscmd)[1])

Again you are doing more changes that the summary of the patch
suggests, you are more than renaming methods here... And why is that
needed? Why did it worked until now and what triggered this change for
this test?


Cheers,
Stefan
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] scripts/cleanup-workdir: fix after binutils-cross name change

2014-05-04 Thread Stefan Stanacar

Signed-off-by: Stefan Stanacar sst...@gmail.com
---
 scripts/cleanup-workdir | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/scripts/cleanup-workdir b/scripts/cleanup-workdir
index 25fef97..cffac59 100755
--- a/scripts/cleanup-workdir
+++ b/scripts/cleanup-workdir
@@ -49,9 +49,10 @@ def run_command(cmd):
 
 def get_cur_arch_dirs(workdir, arch_dirs):
 pattern = workdir + '/(.*?)/'
-
+tunearch = run_command(bitbake -e | grep ^TUNE_ARCH=)
+tunearch = tunearch.split('')[1]
 # select thest 5 packages to get the dirs of current arch
-pkgs = ['hicolor-icon-theme', 'base-files', 'acl-native', 
'binutils-crosssdk', 'nativesdk-autoconf']
+pkgs = ['hicolor-icon-theme', 'base-files', 'acl-native', 
'binutils-crosssdk-' + tunearch, 'nativesdk-autoconf']
 
 for pkg in pkgs:
 cmd = bitbake -e  + pkg +  | grep ^IMAGE_ROOTFS=
-- 
1.9.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] oeqa/utils: sshcontrol: realtime logging of output

2014-04-25 Thread Stefan Stanacar

Log the output of the command as it runs not when it finished, else
tail -f 
tmp/work/minnow-poky-linux/core-image-sato/1.0-r0/testimage/ssh_target_log
isn't as useful as it could be.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/utils/sshcontrol.py | 19 +++
 1 file changed, 15 insertions(+), 4 deletions(-)

diff --git a/meta/lib/oeqa/utils/sshcontrol.py 
b/meta/lib/oeqa/utils/sshcontrol.py
index d355d5e..1c81795 100644
--- a/meta/lib/oeqa/utils/sshcontrol.py
+++ b/meta/lib/oeqa/utils/sshcontrol.py
@@ -29,8 +29,15 @@ class SSHProcess(object):
 self.output = None
 self.process = None
 self.starttime = None
+self.logfile = None
 
-def run(self, command, timeout=None):
+def log(self, msg):
+if self.logfile:
+with open(self.logfile, a) as f:
+   f.write(%s % msg)
+
+def run(self, command, timeout=None, logfile=None):
+self.logfile = logfile
 self.starttime = time.time()
 output = ''
 self.process = subprocess.Popen(command, **self.options)
@@ -45,8 +52,10 @@ class SSHProcess(object):
 eof = True
 else:
 output += data
+self.log(data)
 endtime = time.time() + timeout
 
+
 # process hasn't returned yet
 if not eof:
 self.process.terminate()
@@ -55,9 +64,12 @@ class SSHProcess(object):
 self.process.kill()
 except OSError:
 pass
-output += \nProcess killed - no output for %d seconds. Total 
running time: %d seconds. % (timeout, time.time() - self.starttime)
+lastline = \nProcess killed - no output for %d seconds. Total 
running time: %d seconds. % (timeout, time.time() - self.starttime)
+self.log(lastline)
+output += lastline
 else:
 output = self.process.communicate()[0]
+self.log(output.rstrip())
 
 self.status = self.process.wait()
 self.output = output.rstrip()
@@ -91,9 +103,8 @@ class SSHControl(object):
 self.log([Running]$ %s %  .join(command))
 
 proc = SSHProcess()
-status, output = proc.run(command, timeout)
+status, output = proc.run(command, timeout, logfile=self.logfile)
 
-self.log(%s % output)
 self.log([Command returned '%d' after %.2f seconds] % (status, 
time.time() - proc.starttime))
 
 if status and not ignore_status:
-- 
1.9.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2] scripts/send-error-report: use a real server as the default

2014-04-17 Thread Stefan Stanacar

Now that there is a public instance [1] of the error report web interface [2],
we should set it as the default.

[1] http://errors.yoctoproject.org/
[1] http://git.yoctoproject.org/cgit/cgit.cgi/error-report-web/

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 scripts/send-error-report | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/send-error-report b/scripts/send-error-report
index 5e3b2a2..3d1f7a4 100755
--- a/scripts/send-error-report
+++ b/scripts/send-error-report
@@ -106,4 +106,4 @@ if __name__ == '__main__':
 elif len(sys.argv) == 3:
 sendData(sys.argv[1], sys.argv[2])
 else:
-sendData(sys.argv[1], localhost:8000)
+sendData(sys.argv[1], errors.yoctoproject.org)
-- 
1.9.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v3] scripts/send-error-report: use a real server as the default

2014-04-17 Thread Stefan Stanacar

Now that there is a public instance [1] of the error report web interface [2],
we should set it as the default.

[1] http://errors.yoctoproject.org/
[2] http://git.yoctoproject.org/cgit/cgit.cgi/error-report-web/

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 scripts/send-error-report | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/send-error-report b/scripts/send-error-report
index 5e3b2a2..3d1f7a4 100755
--- a/scripts/send-error-report
+++ b/scripts/send-error-report
@@ -106,4 +106,4 @@ if __name__ == '__main__':
 elif len(sys.argv) == 3:
 sendData(sys.argv[1], sys.argv[2])
 else:
-sendData(sys.argv[1], localhost:8000)
+sendData(sys.argv[1], errors.yoctoproject.org)
-- 
1.9.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] selftest: DiskMonTest: use a high value for free space

2014-04-11 Thread Stefan Stanacar

Using the free space of the host works when
oe-selftest is the only build running, but if something else
on the host remove things this will fail (as seen on AB).
Using an absurdly high value should fix this.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/selftest/buildoptions.py | 8 +++-
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/meta/lib/oeqa/selftest/buildoptions.py 
b/meta/lib/oeqa/selftest/buildoptions.py
index 8ff40ba..27fc452 100644
--- a/meta/lib/oeqa/selftest/buildoptions.py
+++ b/meta/lib/oeqa/selftest/buildoptions.py
@@ -51,17 +51,15 @@ class ImageOptionsTests(oeSelfTest):
 class DiskMonTest(oeSelfTest):
 
 def test_stoptask_behavior(self):
-result = runCmd(df -Pk %s % os.getcwd())
-size = result.output.split(\n)[1].split()[3]
-self.write_config('BB_DISKMON_DIRS = STOPTASKS,${TMPDIR},%sK,4510K' 
% size)
+self.write_config('BB_DISKMON_DIRS = 
STOPTASKS,${TMPDIR},10G,100K')
 res = bitbake(m4, ignore_status = True)
 self.assertTrue('ERROR: No new tasks can be executed since the disk 
space monitor action is STOPTASKS!' in res.output)
 self.assertEqual(res.status, 1)
-self.write_config('BB_DISKMON_DIRS = ABORT,${TMPDIR},%sK,4510K' % 
size)
+self.write_config('BB_DISKMON_DIRS = ABORT,${TMPDIR},10G,100K')
 res = bitbake(m4, ignore_status = True)
 self.assertTrue('ERROR: Immediately abort since the disk space monitor 
action is ABORT!' in res.output)
 self.assertEqual(res.status, 1)
-self.write_config('BB_DISKMON_DIRS = WARN,${TMPDIR},%sK,4510K' % 
size)
+self.write_config('BB_DISKMON_DIRS = WARN,${TMPDIR},10G,100K')
 res = bitbake(m4)
 self.assertTrue('WARNING: The free space' in res.output)
 
-- 
1.9.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] scripts/send-error-report: simple hack to use proxy from the enviroment

2014-04-11 Thread Stefan Stanacar

People behind a proxy couldn't send an error report to an upstream server,
this should fix the issue if they use a proxy that doesn't require 
authentication,
and it's correctly exported in the enviroment.
There are probbaly better ways to fix this (urllib2, proxy connect,
a command line option for proxy) but those require more changes than
I feel confortable doing now.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 scripts/send-error-report | 27 +--
 1 file changed, 21 insertions(+), 6 deletions(-)

diff --git a/scripts/send-error-report b/scripts/send-error-report
index 3734b61..0bce8b6 100755
--- a/scripts/send-error-report
+++ b/scripts/send-error-report
@@ -8,6 +8,24 @@
 
 
 import httplib, urllib, os, sys, json
+from urllib2 import _parse_proxy as parseproxy
+
+
+def handle_connection(server, data):
+params = urllib.urlencode({'data': data})
+headers = {Content-type: application/json}
+
+if os.environ.get(http_proxy) or os.environ.get(HTTP_PROXY):
+proxy = parseproxy(os.environ.get(http_proxy) or 
os.environ.get(HTTP_PROXY))
+if proxy[1] or proxy[2]:
+raise Exception(http proxies that require 
authentication are not supported for now.)
+conn = httplib.HTTPConnection(proxy[3])
+conn.request(POST, http://%s/ClientPost/; % server, params, 
headers)
+else:
+conn = httplib.HTTPConnection(server)
+conn.request(POST, /ClientPost/, params, headers)
+
+return conn
 
 
 def sendData(json_file, server):
@@ -45,10 +63,7 @@ def sendData(json_file, server):
 return
 
 try:
-params = urllib.urlencode({'data': data})
-headers = {Content-type: application/json}
-conn = httplib.HTTPConnection(server)
-conn.request(POST, /ClientPost/, params, headers)
+conn = handle_connection(server, data)
 response = conn.getresponse()
 print response.status, response.reason
 res = response.read()
@@ -59,8 +74,8 @@ def sendData(json_file, server):
 with open(%s.response.html % json_file, w) as f:
 f.write(res)
 conn.close()
-except:
-print(Server connection failed)
+except Exception as e:
+print(Server connection failed: %s % e)
 
 else:
 print(No data file found.)
-- 
1.9.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] scripts/send-error-report: use the official server as the default

2014-04-11 Thread Stefan Stanacar

Now that we have an YP instance of the error report web interface [1],
that should be set as the default instead of localhost.

[1] http://git.yoctoproject.org/cgit/cgit.cgi/error-report-web/

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 scripts/send-error-report | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/send-error-report b/scripts/send-error-report
index 8eea3cf..3734b61 100755
--- a/scripts/send-error-report
+++ b/scripts/send-error-report
@@ -77,4 +77,4 @@ if __name__ == '__main__':
 elif len(sys.argv) == 3:
 sendData(sys.argv[1], sys.argv[2])
 else:
-sendData(sys.argv[1], localhost:8000)
+sendData(sys.argv[1], error-report-web.yoctoproject.org)
-- 
1.9.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2] scripts/send-error-report: simple hack to use proxy from the enviroment

2014-04-11 Thread Stefan Stanacar

People behind a proxy couldn't send an error report to an upstream server,
this should fix the issue if they use a proxy that doesn't require 
authentication,
or one that uses basic http authentication and it's correctly exported in the 
enviroment.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 scripts/send-error-report | 43 ---
 1 file changed, 36 insertions(+), 7 deletions(-)

diff --git a/scripts/send-error-report b/scripts/send-error-report
index 8eea3cf..5e3b2a2 100755
--- a/scripts/send-error-report
+++ b/scripts/send-error-report
@@ -7,7 +7,39 @@
 
 
 
-import httplib, urllib, os, sys, json
+import httplib, urllib, os, sys, json, base64
+from urllib2 import _parse_proxy as parseproxy
+
+
+def handle_connection(server, data):
+params = urllib.urlencode({'data': data})
+headers = {Content-type: application/json}
+proxyrequired = False
+if os.environ.get(http_proxy) or os.environ.get(HTTP_PROXY):
+proxyrequired = True
+# we need to check that the server isn't a local one, as in no_proxy
+try:
+temp = httplib.HTTPConnection(server, strict=True, timeout=5)
+temp.request(GET, /)
+tempres = temp.getresponse()
+if tempres.status == 200:
+proxyrequired = False
+temp.close()
+except:
+pass
+
+if proxyrequired:
+proxy = parseproxy(os.environ.get(http_proxy) or 
os.environ.get(HTTP_PROXY))
+if proxy[1] and proxy[2]:
+auth = base64.encodestring(%s:%s % (proxy[1], proxy[2]))
+headers[Authorization] = Basic %s % auth
+conn = httplib.HTTPConnection(proxy[3])
+conn.request(POST, http://%s/ClientPost/; % server, params, headers)
+else:
+conn = httplib.HTTPConnection(server)
+conn.request(POST, /ClientPost/, params, headers)
+
+return conn
 
 
 def sendData(json_file, server):
@@ -45,10 +77,7 @@ def sendData(json_file, server):
 return
 
 try:
-params = urllib.urlencode({'data': data})
-headers = {Content-type: application/json}
-conn = httplib.HTTPConnection(server)
-conn.request(POST, /ClientPost/, params, headers)
+conn = handle_connection(server, data)
 response = conn.getresponse()
 print response.status, response.reason
 res = response.read()
@@ -59,8 +88,8 @@ def sendData(json_file, server):
 with open(%s.response.html % json_file, w) as f:
 f.write(res)
 conn.close()
-except:
-print(Server connection failed)
+except Exception as e:
+print(Server connection failed: %s % e)
 
 else:
 print(No data file found.)
-- 
1.9.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] qemux86-64: re-enable paravirt guest, bring it in sync with qemux86

2014-04-01 Thread Stefan Stanacar

Seems wrong that qemux86 has this enabled and qemux86-64 doesn't.
Also this will allow people to use kvm with -cpu=host.
Right now, runqemu qemux86-64 kvm uses -cpu=kvm64 because without this
feature you can't use cpu=host on newer host kernels (= 3.8).

This basically reverts poky e6149ec6c4c14d3a939022cdc29ac94dced836dd /
oe-core 64749308fadabb4aa7c39f360c6395827bc5eb3a
The reason of that commit (which is more than a year old) was that on
the AB running old kernels (2.6.37) we would see occasional shutdown failures.
(but if memory serves me right the fix might have been just a coincidence).

I've tested this change with by:
  - run tests on runqemu qemux86-64 (without kvm) (300 seconds)
  - run tests on runqemu qemux86-64 kvm (which uses by default cpu=kvm64) - 
(tests
took 20 seconds)
  - run tests on runqemu qemux86-64 kvm with cpu=host - (tests took 18 seconds)

[ YOCTO #5956 ]

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/recipes-kernel/linux/linux-yocto_3.14.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-kernel/linux/linux-yocto_3.14.bb 
b/meta/recipes-kernel/linux/linux-yocto_3.14.bb
index d5202cd..46c4260 100644
--- a/meta/recipes-kernel/linux/linux-yocto_3.14.bb
+++ b/meta/recipes-kernel/linux/linux-yocto_3.14.bb
@@ -33,5 +33,5 @@ COMPATIBLE_MACHINE = 
qemuarm|qemux86|qemuppc|qemumips|qemumips64|qemux86-64
 KERNEL_EXTRA_FEATURES ?= features/netfilter/netfilter.scc
 KERNEL_FEATURES_append =  ${KERNEL_EXTRA_FEATURES}
 KERNEL_FEATURES_append_qemux86= cfg/sound.scc cfg/paravirt_kvm.scc
-KERNEL_FEATURES_append_qemux86-64= cfg/sound.scc
+KERNEL_FEATURES_append_qemux86-64= cfg/sound.scc cfg/paravirt_kvm.scc
 KERNEL_FEATURES_append =  ${@bb.utils.contains(TUNE_FEATURES, mx32,  
cfg/x32.scc,  ,d)}
-- 
1.9.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] image.bbclass: add postprocess func to disable DNS lookups for openssh

2014-03-31 Thread Stefan Stanacar

In the default config openssh does reverse DNS resolution, but
if there is no DNS configured on the target (or even worse the
DNS server is unreachable) there is a long timeout when connecting
over ssh. This is most visible on core-image-sato-sdk builds on AB, where the
tests take a long time because each ssh command spends 15 seconds
just connecting to the target, disabling the reverse DNS resolution halves
the total test time.
This isn't a qemu specific problem however and instead of changing the default
config I think it's worth disabling this when debug-tweaks is enabled.

Patch based on a previous version sent by Saul Wold s...@linux.intel.com

[YOCTO #5954]

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/classes/image.bbclass | 10 +-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 29309f5..846c768 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -146,6 +146,8 @@ MACHINE_POSTPROCESS_COMMAND ?= 
 ROOTFS_POSTPROCESS_COMMAND += '${@base_contains(IMAGE_FEATURES, 
debug-tweaks, ssh_allow_empty_password; , ,d)}'
 # Enable postinst logging if debug-tweaks is enabled
 ROOTFS_POSTPROCESS_COMMAND += '${@base_contains(IMAGE_FEATURES, 
debug-tweaks, postinst_enable_logging; , ,d)}'
+# Disable DNS lookups if debug-tweaks is enabled
+ROOTFS_POSTPROCESS_COMMAND += '${@base_contains(IMAGE_FEATURES, 
debug-tweaks, ssh_disable_dns_lookup; , ,d)}'
 # Write manifest
 IMAGE_MANIFEST = ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.manifest
 ROOTFS_POSTPROCESS_COMMAND =+ write_image_manifest ; 
@@ -304,6 +306,12 @@ ssh_allow_empty_password () {
fi
 }
 
+ssh_disable_dns_lookup () {
+if [ -e ${IMAGE_ROOTFS}${sysconfdir}/ssh/sshd_config ]; then
+sed -i -e 's:#UseDNS yes:UseDNS no:' 
${IMAGE_ROOTFS}${sysconfdir}/ssh/sshd_config
+fi
+}
+
 # Enable postinst logging if debug-tweaks is enabled
 postinst_enable_logging () {
mkdir -p ${IMAGE_ROOTFS}${sysconfdir}/default
@@ -374,7 +382,7 @@ rootfs_sysroot_relativelinks () {
sysroot-relativelinks.py ${SDK_OUTPUT}/${SDKTARGETSYSROOT}
 }
 
-EXPORT_FUNCTIONS zap_empty_root_password remove_init_link do_rootfs 
make_zimage_symlink_relative set_image_autologin rootfs_update_timestamp 
rootfs_no_x_startup
+EXPORT_FUNCTIONS zap_empty_root_password remove_init_link do_rootfs 
make_zimage_symlink_relative set_image_autologin rootfs_update_timestamp 
rootfs_no_x_startup ssh_disable_dns_lookup
 
 do_fetch[noexec] = 1
 do_unpack[noexec] = 1
-- 
1.9.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/3] initrdscripts: add install scripts with a second rootfs

2014-03-30 Thread Stefan Stanacar

These are meant to be used by a master image, for a simple
initial setup.
The install scripts are similar to the default ones, but:
- custom partitioning, replaces the swap partiton with a second root filesystem
- adds labels to the partitions
- preconfigures a boot loader entry for the second rootfs

Part of [YOCTO #5614]

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 .../initrdscripts/files/init-install-efi-testfs.sh | 199 +++
 .../initrdscripts/files/init-install-testfs.sh | 211 +
 .../initramfs-live-install-efi-testfs_1.0.bb   |  14 ++
 .../initramfs-live-install-testfs_1.0.bb   |  14 ++
 4 files changed, 438 insertions(+)
 create mode 100644 
meta/recipes-core/initrdscripts/files/init-install-efi-testfs.sh
 create mode 100644 meta/recipes-core/initrdscripts/files/init-install-testfs.sh
 create mode 100644 
meta/recipes-core/initrdscripts/initramfs-live-install-efi-testfs_1.0.bb
 create mode 100644 
meta/recipes-core/initrdscripts/initramfs-live-install-testfs_1.0.bb

diff --git a/meta/recipes-core/initrdscripts/files/init-install-efi-testfs.sh 
b/meta/recipes-core/initrdscripts/files/init-install-efi-testfs.sh
new file mode 100644
index 000..2fea761
--- /dev/null
+++ b/meta/recipes-core/initrdscripts/files/init-install-efi-testfs.sh
@@ -0,0 +1,199 @@
+#!/bin/sh -e
+#
+# Copyright (c) 2012, Intel Corporation.
+# All rights reserved.
+#
+# install.sh [device_name] [rootfs_name]
+#
+
+PATH=/sbin:/bin:/usr/sbin:/usr/bin
+
+# We need 200 Mb for the boot partition
+boot_size=200
+
+# 50% for the second rootfs
+testfs_ratio=50
+
+found=no
+
+echo Searching for a hard drive...
+for device in 'hda' 'hdb' 'sda' 'sdb' 'mmcblk0' 'mmcblk1'
+do
+if [ -e /sys/block/${device}/removable ]; then
+if [ $(cat /sys/block/${device}/removable) = 0 ]; then
+found=yes
+
+while true; do
+# Try sleeping here to avoid getting kernel messages
+# obscuring/confusing user
+sleep 5
+echo Found drive at /dev/${device}. Do you want to install 
this image there ? [y/n]
+read answer
+if [ $answer = y ] ; then
+break
+fi
+
+if [ $answer = n ] ; then
+found=no
+break
+fi
+
+echo Please answer y or n
+done
+fi
+fi
+
+if [ $found = yes ]; then
+break;
+fi
+
+done
+
+if [ $found = no ]; then
+exit 1
+fi
+
+echo Installing image on /dev/${device}
+
+#
+# The udev automounter can cause pain here, kill it
+#
+rm -f /etc/udev/rules.d/automount.rules
+rm -f /etc/udev/scripts/mount*
+
+#
+# Unmount anything the automounter had mounted
+#
+umount /dev/${device}* 2 /dev/null || /bin/true
+
+mkdir -p /tmp
+cat /proc/mounts  /etc/mtab
+
+disk_size=$(parted /dev/${device} unit mb print | grep Disk | cut -d  -f 3 | 
sed -e s/MB//)
+
+testfs_size=$((disk_size*testfs_ratio/100))
+rootfs_size=$((disk_size-boot_size-testfs_size))
+
+rootfs_start=$((boot_size))
+rootfs_end=$((rootfs_start+rootfs_size))
+testfs_start=$((rootfs_end))
+
+# MMC devices are special in a couple of ways
+# 1) they use a partition prefix character 'p'
+# 2) they are detected asynchronously (need rootwait)
+rootwait=
+part_prefix=
+if [ ! ${device#mmcblk} = ${device} ]; then
+part_prefix=p
+rootwait=rootwait
+fi
+bootfs=/dev/${device}${part_prefix}1
+rootfs=/dev/${device}${part_prefix}2
+testfs=/dev/${device}${part_prefix}3
+
+echo *
+echo Boot partition size:   $boot_size MB ($bootfs)
+echo Rootfs partition size: $rootfs_size MB ($rootfs)
+echo Testfs partition size:   $testfs_size MB ($testfs)
+echo *
+echo Deleting partition table on /dev/${device} ...
+dd if=/dev/zero of=/dev/${device} bs=512 count=2
+
+echo Creating new partition table on /dev/${device} ...
+parted /dev/${device} mklabel gpt
+
+echo Creating boot partition on $bootfs
+parted /dev/${device} mkpart primary 0% $boot_size
+parted /dev/${device} set 1 boot on
+
+echo Creating rootfs partition on $rootfs
+parted /dev/${device} mkpart primary $rootfs_start $rootfs_end
+
+echo Creating testfs partition on $testfs
+parted /dev/${device} mkpart primary $testfs_start 100%
+
+parted /dev/${device} print
+
+echo Formatting $bootfs to vfat...
+mkfs.vfat -n boot $bootfs
+
+echo Formatting $rootfs to ext3...
+mkfs.ext3 -L platform $rootfs
+
+echo Formatting $testfs to ext3...
+mkfs.ext3 -L testrootfs $testfs
+
+mkdir /ssd
+mkdir /rootmnt
+mkdir /bootmnt
+
+mount $rootfs /ssd
+mount -o rw,loop,noatime,nodiratime /media/$1/$2 /rootmnt
+
+echo Copying rootfs files...
+cp -a /rootmnt/* /ssd
+
+touch /ssd/etc/masterimage
+
+if [ -d /ssd/etc/ ] ; then
+# We dont want udev to mount our root device while we're booting...
+if [ -d /ssd/etc/udev/ ] ; then
+echo /dev/${device}  /ssd/etc/udev/mount.blacklist
+fi

[OE-core] [PATCH 2/3] recipes-extended: add master image for testing purposes

2014-03-30 Thread Stefan Stanacar

Add a custom recipe with custom initramfs and installers, that makes
it easy to deploy a master image for testing purposes.
We need a master image running on the target hardware, that should be a
known good build, with a set of utilities installed so that we use it
to deploy the images under test.

This core-image-testmaster recipe isn't a requirement per se, any image can
be used as long as the required conditions are met.
The test code assumes:
 - that the device has a second rootfs labeled as testrootfs
 - it has a properly configured bootloader entry (called test) for the second 
kernel and rootfs
 - the master image has a /etc/masterimage file so it can differentiate between 
master
and test images
 - the master image has tar, mount, bash (basically the normal linux utilities 
not the busybox
ones)

[YOCTO #5614]

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 .../images/core-image-testmaster-initramfs.bb | 19 +++
 meta/recipes-extended/images/core-image-testmaster.bb | 18 ++
 2 files changed, 37 insertions(+)
 create mode 100644 
meta/recipes-extended/images/core-image-testmaster-initramfs.bb
 create mode 100644 meta/recipes-extended/images/core-image-testmaster.bb

diff --git a/meta/recipes-extended/images/core-image-testmaster-initramfs.bb 
b/meta/recipes-extended/images/core-image-testmaster-initramfs.bb
new file mode 100644
index 000..563260d
--- /dev/null
+++ b/meta/recipes-extended/images/core-image-testmaster-initramfs.bb
@@ -0,0 +1,19 @@
+DESCRIPTION = Small image capable of booting a device with custom install 
scripts, \
+adding a second rootfs, used for testing.
+
+# use -testfs live-install scripts
+PACKAGE_INSTALL = initramfs-live-boot initramfs-live-install-testfs 
initramfs-live-install-efi-testfs busybox udev base-passwd 
${ROOTFS_BOOTSTRAP_INSTALL}
+
+# Do not pollute the initrd image with rootfs features
+IMAGE_FEATURES = 
+
+export IMAGE_BASENAME = core-image-testmaster-initramfs
+IMAGE_LINGUAS = 
+
+LICENSE = MIT
+
+IMAGE_FSTYPES = ${INITRAMFS_FSTYPES}
+inherit core-image
+
+IMAGE_ROOTFS_SIZE = 8192
+BAD_RECOMMENDATIONS += busybox-syslog
diff --git a/meta/recipes-extended/images/core-image-testmaster.bb 
b/meta/recipes-extended/images/core-image-testmaster.bb
new file mode 100644
index 000..ac2fdbe
--- /dev/null
+++ b/meta/recipes-extended/images/core-image-testmaster.bb
@@ -0,0 +1,18 @@
+DESCRIPTION = A master image to be deployed on a target useful for testing 
other images
+
+IMAGE_FEATURES += ssh-server-openssh package-management
+
+inherit core-image
+
+# the deploy code requires bash and
+# normal linux utilities not busybox ones
+IMAGE_INSTALL += \
+bash coreutils util-linux tar gzip kmod \
+python-modules python-misc \
+e2fsprogs e2fsprogs-mke2fs parted \
+
+# we need a particular initramfs for live images
+# that pulls custom install scripts which take
+# care of partitioning for us
+INITRD_IMAGE = core-image-testmaster-initramfs
+
-- 
1.8.5.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/3] lib/oeqa: add a test target controller for EFI targets

2014-03-30 Thread Stefan Stanacar

The purpose of this module is to deploy a test image on a EFI-enabled hardware
and run our runtime tests. A bit of background:
 - testimage.bbclass uses the concept of TEST_TARGET which is a class name
that is responsible for target deploying. A layer can provide
it's own TEST_TARGET. Right now has OE-core has a QemuTarget and a 
SimpleRemoteTarget
(ssh into an already up and running machine and run tests), the default one 
being qemu.
 - basically testimage does something like:
target.deploy()
try:
target.start()
runTests()
finally:
target.stop()

This module assumes a running EFI machine with gummiboot as bootloader and
core-image-testmaster installed (or similar). Also your hardware under test has
to be in a DHCP-enabled network that gives it the same IP for each reboot.

One time setup (master image):
 - build core-image-testmaster with EFI_PROVIDER = gummiboot
 - install the image on the target

Test image setup:
 - build your test image, e.g core-image-sato as you usually do, but with these 
in local.conf:
IMAGE_FSTYPES += tar.gz
 - Now run the tests:
INHERIT += testimage
TEST_TARGET = GummibootTarget
TEST_TARGET_IP = 192.168.2.3
bitbake core-image-sato -c testimage

Other notes:
 - TEST_POWERCONTROL_CMD (togheter with TEST_POWERCONTROL_EXTRA_ARGS) can be a 
command that runs on the host and does power cycling.
The test code passes one argument to that command: off, on or cycle (off then 
on). In my case I use something like
TEST_POWERCONTROL_CMD=powercontrol.exp test 10.11.12.1 nuc1 in local.conf.
Basically my expect script does: 'ssh test@10.11.12.1 pyctl nuc1 arg and 
runs a python script there that controls power for a label called nuc1'.
The reason why my expect script has to ssh into another machine is because of 
network topology, and that machine is the one actually connected
to the test rack and the power strip. That's why TEST_POWERCONTROL_CMD and 
_ARGS need to be customized for one's setup, the only requirement being
that it accepts: on/off/cycle as the last argument.
 - if no command is defined it would use classic reboot. This is fine as long 
as the machine
actually reboots (as in the ssh test hasn't failed), but it's useful for 
simple-setup-with-one-board-on-the-desk scenario, where
some manual interaction is okay from time to time.

[YOCTO #5614]

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/controllers/masterimage.py | 133 +++
 meta/lib/oeqa/runtime/ssh.py |   2 +
 2 files changed, 135 insertions(+)
 create mode 100644 meta/lib/oeqa/controllers/masterimage.py

diff --git a/meta/lib/oeqa/controllers/masterimage.py 
b/meta/lib/oeqa/controllers/masterimage.py
new file mode 100644
index 000..188c630
--- /dev/null
+++ b/meta/lib/oeqa/controllers/masterimage.py
@@ -0,0 +1,133 @@
+import os
+import bb
+import traceback
+import time
+
+import oeqa.targetcontrol
+import oeqa.utils.sshcontrol as sshcontrol
+import oeqa.utils.commands as commands
+
+class GummibootTarget(oeqa.targetcontrol.SimpleRemoteTarget):
+
+def __init__(self, d):
+# let our base class do the ip thing
+super(GummibootTarget, self).__init__(d)
+
+# test rootfs + kernel
+self.rootfs = os.path.join(d.getVar(DEPLOY_DIR_IMAGE, True), 
d.getVar(IMAGE_LINK_NAME, True) + '.tar.gz')
+self.kernel = os.path.join(d.getVar(DEPLOY_DIR_IMAGE, True), 
d.getVar(KERNEL_IMAGETYPE))
+if not os.path.isfile(self.rootfs):
+# we could've checked that IMAGE_FSTYPES contains tar.gz but the 
config for running testimage might not be
+# the same as the config with which the image was build, ie
+# you bitbake core-image-sato with IMAGE_FSTYPES += tar.gz
+# and your autobuilder overwrites the config, adds the test bits 
and runs bitbake core-image-sato -c testimage
+bb.fatal(No rootfs found. Did you build the image ?\nIf yes, did 
you build it with IMAGE_FSTYPES += \tar.gz\ ? \
+  \nExpected path: %s % self.rootfs)
+if not os.path.isfile(self.kernel):
+bb.fatal(No kernel found. Expected path: %s % self.kernel)
+
+# if the user knows what he's doing, then by all means...
+# test-rootfs.tar.gz and test-kernel are hardcoded names in other 
places
+# they really have to be used like that in commands though
+cmds = d.getVar(TEST_DEPLOY_CMDS, True)
+
+# this the value we need to set in the LoaderEntryOneShot EFI variable
+# so the system boots the 'test' bootloader label and not the default
+# The first four bytes are EFI bits, and the rest is an utf-16le string
+# (EFI vars values need to be utf-16)
+# $ echo -en test\0 | iconv -f ascii -t utf-16le | hexdump -C
+#   74 00 65 00 73 00 74 00  00 00
|t.e.s.t...|
+self.efivarvalue = 
r'\x07\x00\x00\x00\x74\x00\x65

[OE-core] [PATCH 0/3] Master image generation and hw testing

2014-03-30 Thread Stefan Stanacar

Hello,

This is what I currently use for doing automated deployment
for a genericx86-64 image on NUC. More details on the commit
messages. There is nothing particular to genericx86-64,
it should work on any EFI-enabled device with gummiboot as bootloader.
Also see YB#1596 and it's sub-bug #5614 for a bit of background.

Cheers,
Stefan

V4: this is almost identical to the RFC sent but:
  - squased the first two patches
  - adress Otavio's comments - rename the class and use a constant for one
of the deploy commands
  - cosmetic changes, mostly comments/typos, nothing functional


The following changes since commit 790cc61a754d156c15f77c95539e17ec4d9f2c41:

  bitbake: user-manual-execution.xml: Added how BB processes curly braces. 
(2014-03-30 10:19:49 +0100)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib stefans/masterimage
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=stefans/masterimage

Stefan Stanacar (3):
  initrdscripts: add install scripts with a second rootfs
  recipes-extended: add master image for testing purposes
  lib/oeqa: add a test target controller for EFI targets

 meta/lib/oeqa/controllers/masterimage.py   | 133 +
 meta/lib/oeqa/runtime/ssh.py   |   2 +
 .../initrdscripts/files/init-install-efi-testfs.sh | 199 +++
 .../initrdscripts/files/init-install-testfs.sh | 211 +
 .../initramfs-live-install-efi-testfs_1.0.bb   |  14 ++
 .../initramfs-live-install-testfs_1.0.bb   |  14 ++
 .../images/core-image-testmaster-initramfs.bb  |  19 ++
 .../images/core-image-testmaster.bb|  18 ++
 8 files changed, 610 insertions(+)
 create mode 100644 meta/lib/oeqa/controllers/masterimage.py
 create mode 100644 
meta/recipes-core/initrdscripts/files/init-install-efi-testfs.sh
 create mode 100644 meta/recipes-core/initrdscripts/files/init-install-testfs.sh
 create mode 100644 
meta/recipes-core/initrdscripts/initramfs-live-install-efi-testfs_1.0.bb
 create mode 100644 
meta/recipes-core/initrdscripts/initramfs-live-install-testfs_1.0.bb
 create mode 100644 
meta/recipes-extended/images/core-image-testmaster-initramfs.bb
 create mode 100644 meta/recipes-extended/images/core-image-testmaster.bb

-- 
1.8.5.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [RFC PATCH 0/4] Master image generation and testing

2014-03-20 Thread Stefan Stanacar


Hello,

This is what I currently use for doing automated deployment
for a genericx86-64 image. More details on the commit
messages. There is nothing particular to genericx86-64,
it should work on any EFI-enabled device with gummiboot as bootloader.

Also see YB#1596 and it's sub-bug #5614 for a bit of background.
Any comments, questions, most welcome.

Cheers,
Stefan

--

Note that this is actually v3.
v1 and v2 [1] were sent on the poky mailing list, as everything was added to 
meta-yocto-bsp layer.
Nothing much changed since v2 except moving the files around.
Changes in this version include:
 - rebase on top of meta
 - add to the master image only the required utilities instead of an entire 
packagegroup
 - use TEST_POWERCONTROL_EXTRA_ARGS with TEST_POWERCONTROL_CMD
 - split the second patch in the earlier series in two

[1] https://lists.yoctoproject.org/pipermail/poky/2014-March/009631.html






Please review the following changes for suitability for inclusion. If you have
any objections or suggestions for improvement, please respond to the patches. If
you agree with the changes, please provide your Acked-by.

The following changes since commit 49aad7da07e187f206e963001844605731b01247:

  bitbake: runqueue: Remove use of waitpid on worker processes (2014-03-19 
17:46:33 +)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib stefans/masterimage
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=stefans/masterimage

Stefan Stanacar (4):
  initrdscripts: copy of the default install scripts
  initrdscripts: add install scripts with a second rootfs
  recipes-extended: add master image for testing purposes
  lib/oeqa: add a test target controller for EFI targets

 meta/lib/oeqa/controllers/masterimage.py   | 124 
 meta/lib/oeqa/runtime/ssh.py   |   2 +
 .../initrdscripts/files/init-install-efi-testfs.sh | 199 +++
 .../initrdscripts/files/init-install-testfs.sh | 211 +
 .../initramfs-live-install-efi-testfs_1.0.bb   |  20 ++
 .../initramfs-live-install-testfs_1.0.bb   |  20 ++
 .../images/core-image-testmaster-initramfs.bb  |  20 ++
 .../images/core-image-testmaster.bb|  13 ++
 8 files changed, 609 insertions(+)
 create mode 100644 meta/lib/oeqa/controllers/masterimage.py
 create mode 100644 
meta/recipes-core/initrdscripts/files/init-install-efi-testfs.sh
 create mode 100644 meta/recipes-core/initrdscripts/files/init-install-testfs.sh
 create mode 100644 
meta/recipes-core/initrdscripts/initramfs-live-install-efi-testfs_1.0.bb
 create mode 100644 
meta/recipes-core/initrdscripts/initramfs-live-install-testfs_1.0.bb
 create mode 100644 
meta/recipes-extended/images/core-image-testmaster-initramfs.bb
 create mode 100644 meta/recipes-extended/images/core-image-testmaster.bb

-- 
1.8.5.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [RFC PATCH 2/4] initrdscripts: add install scripts with a second rootfs

2014-03-20 Thread Stefan Stanacar

The install scripts are similar to the default ones, but:
- custom partitioning, replaces the swap partiton with a second root filesystem
- adds labels to the partitions
- preconfigures a boot loader entry for the second
Other arches not using installers would simply deploy the image as they do,
they just need to follow the same schema.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 .../initrdscripts/files/init-install-efi-testfs.sh | 36 --
 .../initrdscripts/files/init-install-testfs.sh | 33 +++-
 .../initramfs-live-install-efi-testfs_1.0.bb   | 20 
 .../initramfs-live-install-testfs_1.0.bb   | 20 
 4 files changed, 77 insertions(+), 32 deletions(-)
 create mode 100644 
meta/recipes-core/initrdscripts/initramfs-live-install-efi-testfs_1.0.bb
 create mode 100644 
meta/recipes-core/initrdscripts/initramfs-live-install-testfs_1.0.bb

diff --git a/meta/recipes-core/initrdscripts/files/init-install-efi-testfs.sh 
b/meta/recipes-core/initrdscripts/files/init-install-efi-testfs.sh
index ed3221b..2fea761 100644
--- a/meta/recipes-core/initrdscripts/files/init-install-efi-testfs.sh
+++ b/meta/recipes-core/initrdscripts/files/init-install-efi-testfs.sh
@@ -8,11 +8,11 @@
 
 PATH=/sbin:/bin:/usr/sbin:/usr/bin
 
-# We need 20 Mb for the boot partition
-boot_size=20
+# We need 200 Mb for the boot partition
+boot_size=200
 
-# 5% for swap
-swap_ratio=5
+# 50% for the second rootfs
+testfs_ratio=50
 
 found=no
 
@@ -71,12 +71,12 @@ cat /proc/mounts  /etc/mtab
 
 disk_size=$(parted /dev/${device} unit mb print | grep Disk | cut -d  -f 3 | 
sed -e s/MB//)
 
-swap_size=$((disk_size*swap_ratio/100))
-rootfs_size=$((disk_size-boot_size-swap_size))
+testfs_size=$((disk_size*testfs_ratio/100))
+rootfs_size=$((disk_size-boot_size-testfs_size))
 
 rootfs_start=$((boot_size))
 rootfs_end=$((rootfs_start+rootfs_size))
-swap_start=$((rootfs_end))
+testfs_start=$((rootfs_end))
 
 # MMC devices are special in a couple of ways
 # 1) they use a partition prefix character 'p'
@@ -89,12 +89,12 @@ if [ ! ${device#mmcblk} = ${device} ]; then
 fi
 bootfs=/dev/${device}${part_prefix}1
 rootfs=/dev/${device}${part_prefix}2
-swap=/dev/${device}${part_prefix}3
+testfs=/dev/${device}${part_prefix}3
 
 echo *
 echo Boot partition size:   $boot_size MB ($bootfs)
 echo Rootfs partition size: $rootfs_size MB ($rootfs)
-echo Swap partition size:   $swap_size MB ($swap)
+echo Testfs partition size:   $testfs_size MB ($testfs)
 echo *
 echo Deleting partition table on /dev/${device} ...
 dd if=/dev/zero of=/dev/${device} bs=512 count=2
@@ -109,19 +109,19 @@ parted /dev/${device} set 1 boot on
 echo Creating rootfs partition on $rootfs
 parted /dev/${device} mkpart primary $rootfs_start $rootfs_end
 
-echo Creating swap partition on $swap
-parted /dev/${device} mkpart primary $swap_start 100%
+echo Creating testfs partition on $testfs
+parted /dev/${device} mkpart primary $testfs_start 100%
 
 parted /dev/${device} print
 
 echo Formatting $bootfs to vfat...
-mkfs.vfat $bootfs
+mkfs.vfat -n boot $bootfs
 
 echo Formatting $rootfs to ext3...
-mkfs.ext3 $rootfs
+mkfs.ext3 -L platform $rootfs
 
-echo Formatting swap partition...($swap)
-mkswap $swap
+echo Formatting $testfs to ext3...
+mkfs.ext3 -L testrootfs $testfs
 
 mkdir /ssd
 mkdir /rootmnt
@@ -133,9 +133,9 @@ mount -o rw,loop,noatime,nodiratime /media/$1/$2 /rootmnt
 echo Copying rootfs files...
 cp -a /rootmnt/* /ssd
 
-if [ -d /ssd/etc/ ] ; then
-echo $swapswap swap   defaults
  0  0  /ssd/etc/fstab
+touch /ssd/etc/masterimage
 
+if [ -d /ssd/etc/ ] ; then
 # We dont want udev to mount our root device while we're booting...
 if [ -d /ssd/etc/udev/ ] ; then
 echo /dev/${device}  /ssd/etc/udev/mount.blacklist
@@ -184,6 +184,8 @@ if [ -d /media/$1/loader ]; then
 sed -i s/ root=[^ ]*/ / $GUMMIBOOT_CFGS
 # add the root= and other standard boot options
 sed -i s@options *@options root=$rootfs rw $rootwait quiet @ 
$GUMMIBOOT_CFGS
+# Add the test label
+echo -ne title test\nlinux /test-kernel\noptions root=$testfs rw 
$rootwait quiet\n  /ssd/loader/entries/test.conf
 fi
 
 umount /ssd
diff --git a/meta/recipes-core/initrdscripts/files/init-install-testfs.sh 
b/meta/recipes-core/initrdscripts/files/init-install-testfs.sh
index 8e433d5..c35fd2a 100644
--- a/meta/recipes-core/initrdscripts/files/init-install-testfs.sh
+++ b/meta/recipes-core/initrdscripts/files/init-install-testfs.sh
@@ -8,10 +8,10 @@
 PATH=/sbin:/bin:/usr/sbin:/usr/bin
 
 # We need 20 Mb for the boot partition
-boot_size=20
+boot_size=200
 
-# 5% for the swap
-swap_ratio=5
+# 50% for the the test partition
+testfs_ratio=50
 
 # Get a list of hard drives
 hdnamelist=
@@ -94,12 +94,12 @@ cat /proc/mounts  /etc/mtab
 
 disk_size=$(parted /dev/${device} unit mb print | grep Disk | cut -d  -f 3 | 
sed -e s/MB//)
 
-swap_size=$((disk_size

[OE-core] [RFC PATCH 3/4] recipes-extended: add master image for testing purposes

2014-03-20 Thread Stefan Stanacar

Add a custom recipe with custom initramfs and installers, that makes
it easy to deploy a master image for testing purposes.

We need a master image running on the target hardware, that should be a
known good build, with a set of utilities installed so that we use it
to deploy the images under test.

This core-image-testmaster recipe isn't a requirement per se, any image can
be used as long as the required conditions are met. The test code assumes:
 - that the device has a second rootfs labeled as testrootfs
 - it has a properly configured bootloader entry for the second kernel and 
rootfs
 - the master image has a /etc/masterimage file so it can differentiate between 
master
and test images
 - the master image has tar, mount, bash (basically the normal linux utilities 
not the busybox
ones)

[YOCTO #5614]

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 .../images/core-image-testmaster-initramfs.bb| 20 
 .../recipes-extended/images/core-image-testmaster.bb | 13 +
 2 files changed, 33 insertions(+)
 create mode 100644 
meta/recipes-extended/images/core-image-testmaster-initramfs.bb
 create mode 100644 meta/recipes-extended/images/core-image-testmaster.bb

diff --git a/meta/recipes-extended/images/core-image-testmaster-initramfs.bb 
b/meta/recipes-extended/images/core-image-testmaster-initramfs.bb
new file mode 100644
index 000..f4de80a
--- /dev/null
+++ b/meta/recipes-extended/images/core-image-testmaster-initramfs.bb
@@ -0,0 +1,20 @@
+DESCRIPTION = Small image capable of booting a device. The kernel includes \
+the Minimal RAM-based Initial Root Filesystem (initramfs), which finds the \
+first 'init' program more efficiently.
+
+PACKAGE_INSTALL = initramfs-live-boot initramfs-live-install-testfs 
initramfs-live-install-efi-testfs busybox udev base-passwd 
${ROOTFS_BOOTSTRAP_INSTALL}
+
+# Do not pollute the initrd image with rootfs features
+IMAGE_FEATURES = 
+
+export IMAGE_BASENAME = core-image-testmaster-initramfs
+IMAGE_LINGUAS = 
+
+LICENSE = MIT
+
+IMAGE_FSTYPES = ${INITRAMFS_FSTYPES}
+inherit core-image
+
+IMAGE_ROOTFS_SIZE = 8192
+
+BAD_RECOMMENDATIONS += busybox-syslog
diff --git a/meta/recipes-extended/images/core-image-testmaster.bb 
b/meta/recipes-extended/images/core-image-testmaster.bb
new file mode 100644
index 000..7a564d4
--- /dev/null
+++ b/meta/recipes-extended/images/core-image-testmaster.bb
@@ -0,0 +1,13 @@
+DESCRIPTION = A master image to be deployed on a target useful for testing 
other images
+
+IMAGE_FEATURES += ssh-server-openssh package-management
+
+inherit core-image
+
+IMAGE_INSTALL += \
+bash coreutils util-linux tar gzip kmod \
+python-modules python-misc \
+e2fsprogs e2fsprogs-mke2fs parted \
+
+INITRD_IMAGE = core-image-testmaster-initramfs
+
-- 
1.8.5.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [RFC PATCH 4/4] lib/oeqa: add a test target controller for EFI targets

2014-03-20 Thread Stefan Stanacar

The purpose of this module is to deploy a test image on a EFI-enabled hardware
and run our runtime tests.
A bit of background:
 - testimage.bbclass uses the concept of TEST_TARGET which is a class name
that is responsible for target deploying. A layer can provide
it's own TEST_TARGET. Right now has OE-core has a QemuTarget and a 
SimpleRemoteTarget
(ssh into an already up and running machine and run tests), the default one 
being qemu.
 - basically testimage does something like:
  - target.deploy()
  - target.start()
  - runTests()
  - target.stop()

This module assumes a running EFI (and gummiboot as bootloader) machine with
core-image-testmaster installed (or similar).
In order to use this Master Image mechanism there are some hard requirements:
 - it only works for EFI-enabled hardware with EFI_PROVIDER = gummiboot
 - your hardware under test has to be in DHCP-enabled network that gives it the 
same IP for each reboot

One time setup (master image):
 - build core-image-testmaster with EFI_PROVIDER = gummiboot
 - TEMP: for now you'll need SRCREV_meta_pn-linux-yocto = ${AUTOREV}
in local.conf so that some kernel configs required by gummiboot get enabled
 - install the image on the target

Test image setup:
 - build your test image, e.g core-image-sato as you usually do, but with these 
in local.conf:
IMAGE_FSTYPES += tar.gz
   (similar to above you need SRCREV_meta_pn-linux-yocto = ${AUTOREV})
 - Now run the tests:
INHERIT += testimage
TEST_TARGET = GenericEfi
TEST_TARGET_IP = 192.168.2.3
bitbake core-image-sato -c testimage

Other notes:
 - TEST_POWERCONTROL_CMD (togheter with TEST_POWERCONTROL_EXTRA_ARGS) can be a 
command that runs on the host and does power cycling.
   The test code passes one argument to that command: off, on or cycle (off 
then on). In my case I use something like
   TEST_POWERCONTROL_CMD=/home/stefans/powercontrol.exp test 192.168.2.1 nuc1 
in local.conf.
   Basically my expect script does: 'ssh test@192.168.2.1 and runs a python 
script there that controls power for a label called nuc1'.
 - if no command is defined it would use classic reboot. This is fine as long 
as the machine
   actually reboots (as in the ssh test hasn't failed), but it's useful for 
simple-setup-with-one-board-on-the-desk scenario, where
   some manual interaction is okay from time to time.

[YOCTO #5614]

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/controllers/masterimage.py | 124 +++
 meta/lib/oeqa/runtime/ssh.py |   2 +
 2 files changed, 126 insertions(+)
 create mode 100644 meta/lib/oeqa/controllers/masterimage.py

diff --git a/meta/lib/oeqa/controllers/masterimage.py 
b/meta/lib/oeqa/controllers/masterimage.py
new file mode 100644
index 000..028208b
--- /dev/null
+++ b/meta/lib/oeqa/controllers/masterimage.py
@@ -0,0 +1,124 @@
+import os
+import bb
+import traceback
+import time
+
+import oeqa.targetcontrol
+import oeqa.utils.sshcontrol as sshcontrol
+import oeqa.utils.commands as commands
+
+class GenericEfi(oeqa.targetcontrol.SimpleRemoteTarget):
+
+def __init__(self, d):
+# let our base class do the ip thing
+super(GenericEfi, self).__init__(d)
+
+# test rootfs + kernel
+self.rootfs = os.path.join(d.getVar(DEPLOY_DIR_IMAGE, True), 
d.getVar(IMAGE_LINK_NAME, True) + '.tar.gz')
+self.kernel = os.path.join(d.getVar(DEPLOY_DIR_IMAGE, True), 
d.getVar(KERNEL_IMAGETYPE))
+if not os.path.isfile(self.rootfs):
+# we could've checked that IMAGE_FSTYPES contains tar.gz but the 
config for running testimage might not be
+# the same as the config with which the image was build, ie
+# you bitbake core-image-sato with IMAGE_FSTYPES += tar.gz
+# and your autobuilder overwrites the config, adds the test bits 
and runs bitbake core-image-sato -c testimage
+bb.fatal(No rootfs found. Did you build the image ?\nIf yes, did 
you build it with IMAGE_FSTYPES += \tar.gz\ ? \
+  \nExpected path: %s % self.rootfs)
+if not os.path.isfile(self.kernel):
+bb.fatal(No kernel found. Expected path: %s % self.kernel)
+
+# if the user knows what he's doing, then by all means...
+# test-rootfs.tar.gz and test-kernel are hardcoded names in other 
places
+# they really have to be used like that in commands though
+cmds = d.getVar(TEST_DEPLOY_CMDS, True)
+if cmds:
+self.deploy_cmds = cmds.split(\n)
+else:
+self.deploy_cmds = [
+'mount -L boot /boot',
+'mkdir -p /mnt/testrootfs',
+'mount -L testrootfs /mnt/testrootfs',
+'modprobe efivarfs',
+'mount -t efivarfs efivarfs /sys/firmware/efi/efivars',
+'cp ~/test-kernel /boot',
+'rm -rf /mnt/testrootfs/*',
+'tar xzvf ~/test

[OE-core] [RFC PATCH 1/4] initrdscripts: copy of the default install scripts

2014-03-20 Thread Stefan Stanacar

These are the install files for a live image, an exact copy of init-install.sh
and init-install-efi.sh from core. There are renamed here with -testfs.
This is just a dummy commit, just to make it easier to see the diff
in the next commit.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 .../initrdscripts/files/init-install-efi-testfs.sh | 197 +++
 .../initrdscripts/files/init-install-testfs.sh | 208 +
 2 files changed, 405 insertions(+)
 create mode 100644 
meta/recipes-core/initrdscripts/files/init-install-efi-testfs.sh
 create mode 100644 meta/recipes-core/initrdscripts/files/init-install-testfs.sh

diff --git a/meta/recipes-core/initrdscripts/files/init-install-efi-testfs.sh 
b/meta/recipes-core/initrdscripts/files/init-install-efi-testfs.sh
new file mode 100644
index 000..ed3221b
--- /dev/null
+++ b/meta/recipes-core/initrdscripts/files/init-install-efi-testfs.sh
@@ -0,0 +1,197 @@
+#!/bin/sh -e
+#
+# Copyright (c) 2012, Intel Corporation.
+# All rights reserved.
+#
+# install.sh [device_name] [rootfs_name]
+#
+
+PATH=/sbin:/bin:/usr/sbin:/usr/bin
+
+# We need 20 Mb for the boot partition
+boot_size=20
+
+# 5% for swap
+swap_ratio=5
+
+found=no
+
+echo Searching for a hard drive...
+for device in 'hda' 'hdb' 'sda' 'sdb' 'mmcblk0' 'mmcblk1'
+do
+if [ -e /sys/block/${device}/removable ]; then
+if [ $(cat /sys/block/${device}/removable) = 0 ]; then
+found=yes
+
+while true; do
+# Try sleeping here to avoid getting kernel messages
+# obscuring/confusing user
+sleep 5
+echo Found drive at /dev/${device}. Do you want to install 
this image there ? [y/n]
+read answer
+if [ $answer = y ] ; then
+break
+fi
+
+if [ $answer = n ] ; then
+found=no
+break
+fi
+
+echo Please answer y or n
+done
+fi
+fi
+
+if [ $found = yes ]; then
+break;
+fi
+
+done
+
+if [ $found = no ]; then
+exit 1
+fi
+
+echo Installing image on /dev/${device}
+
+#
+# The udev automounter can cause pain here, kill it
+#
+rm -f /etc/udev/rules.d/automount.rules
+rm -f /etc/udev/scripts/mount*
+
+#
+# Unmount anything the automounter had mounted
+#
+umount /dev/${device}* 2 /dev/null || /bin/true
+
+mkdir -p /tmp
+cat /proc/mounts  /etc/mtab
+
+disk_size=$(parted /dev/${device} unit mb print | grep Disk | cut -d  -f 3 | 
sed -e s/MB//)
+
+swap_size=$((disk_size*swap_ratio/100))
+rootfs_size=$((disk_size-boot_size-swap_size))
+
+rootfs_start=$((boot_size))
+rootfs_end=$((rootfs_start+rootfs_size))
+swap_start=$((rootfs_end))
+
+# MMC devices are special in a couple of ways
+# 1) they use a partition prefix character 'p'
+# 2) they are detected asynchronously (need rootwait)
+rootwait=
+part_prefix=
+if [ ! ${device#mmcblk} = ${device} ]; then
+part_prefix=p
+rootwait=rootwait
+fi
+bootfs=/dev/${device}${part_prefix}1
+rootfs=/dev/${device}${part_prefix}2
+swap=/dev/${device}${part_prefix}3
+
+echo *
+echo Boot partition size:   $boot_size MB ($bootfs)
+echo Rootfs partition size: $rootfs_size MB ($rootfs)
+echo Swap partition size:   $swap_size MB ($swap)
+echo *
+echo Deleting partition table on /dev/${device} ...
+dd if=/dev/zero of=/dev/${device} bs=512 count=2
+
+echo Creating new partition table on /dev/${device} ...
+parted /dev/${device} mklabel gpt
+
+echo Creating boot partition on $bootfs
+parted /dev/${device} mkpart primary 0% $boot_size
+parted /dev/${device} set 1 boot on
+
+echo Creating rootfs partition on $rootfs
+parted /dev/${device} mkpart primary $rootfs_start $rootfs_end
+
+echo Creating swap partition on $swap
+parted /dev/${device} mkpart primary $swap_start 100%
+
+parted /dev/${device} print
+
+echo Formatting $bootfs to vfat...
+mkfs.vfat $bootfs
+
+echo Formatting $rootfs to ext3...
+mkfs.ext3 $rootfs
+
+echo Formatting swap partition...($swap)
+mkswap $swap
+
+mkdir /ssd
+mkdir /rootmnt
+mkdir /bootmnt
+
+mount $rootfs /ssd
+mount -o rw,loop,noatime,nodiratime /media/$1/$2 /rootmnt
+
+echo Copying rootfs files...
+cp -a /rootmnt/* /ssd
+
+if [ -d /ssd/etc/ ] ; then
+echo $swapswap swap   defaults
  0  0  /ssd/etc/fstab
+
+# We dont want udev to mount our root device while we're booting...
+if [ -d /ssd/etc/udev/ ] ; then
+echo /dev/${device}  /ssd/etc/udev/mount.blacklist
+fi
+fi
+
+umount /ssd
+umount /rootmnt
+
+echo Preparing boot partition...
+mount $bootfs /ssd
+
+EFIDIR=/ssd/EFI/BOOT
+mkdir -p $EFIDIR
+cp /media/$1/vmlinuz /ssd
+# Copy the efi loader
+cp /media/$1/EFI/BOOT/*.efi $EFIDIR
+
+if [ -f /media/$1/EFI/BOOT/grub.cfg ]; then
+GRUBCFG=$EFIDIR/grub.cfg
+cp /media/$1/EFI/BOOT/grub.cfg $GRUBCFG
+# Update grub config for the installed image

[OE-core] [PATCH] oeqa/utils/qemurunner: use the right kill call so we don't send SIGTERM to bitbake-worker

2014-03-11 Thread Stefan Stanacar

After recent bitbake-worker changes [1] this was killing the bitbake-worker,
which isn't what we want and kill(-pid) was the wrong call anyway.

runqemu.pid is the right PGID as the process was started with preexec_fn=setpgrp
(so no need to do os.getpgid(self.runqemu.pid))

[1] 
http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=15688798520896690561824b2fdc227c8a365c82

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/utils/qemurunner.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oeqa/utils/qemurunner.py 
b/meta/lib/oeqa/utils/qemurunner.py
index c664d00..f1a7e24 100644
--- a/meta/lib/oeqa/utils/qemurunner.py
+++ b/meta/lib/oeqa/utils/qemurunner.py
@@ -162,13 +162,13 @@ class QemuRunner:
 
 if self.runqemu:
 bb.note(Sending SIGTERM to runqemu)
-os.kill(-self.runqemu.pid,signal.SIGTERM)
+os.killpg(self.runqemu.pid, signal.SIGTERM)
 endtime = time.time() + self.runqemutime
 while self.runqemu.poll() is None and time.time()  endtime:
 time.sleep(1)
 if self.runqemu.poll() is None:
 bb.note(Sending SIGKILL to runqemu)
-os.kill(-self.runqemu.pid,signal.SIGKILL)
+os.killpg(self.runqemu.pid, signal.SIGKILL)
 self.runqemu = None
 if self.server_socket:
 self.server_socket.close()
-- 
1.8.5.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] lsbtest: fix comparison bashism

2014-03-11 Thread Stefan Stanacar

== is a bashism use = instead.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/recipes-extended/lsb/lsbtest/LSB_Test.sh | 4 ++--
 meta/recipes-extended/lsb/lsbtest_1.0.bb  | 8 
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/meta/recipes-extended/lsb/lsbtest/LSB_Test.sh 
b/meta/recipes-extended/lsb/lsbtest/LSB_Test.sh
index 12f0054..ab79985 100644
--- a/meta/recipes-extended/lsb/lsbtest/LSB_Test.sh
+++ b/meta/recipes-extended/lsb/lsbtest/LSB_Test.sh
@@ -272,12 +272,12 @@ then
 fi
 
 if ! `grep -F -q noarch-suse ${PLATFORM_FILE}`; then
-   if [ ${ARCH} == i686 ];then
+   if [ ${ARCH} = i686 ];then
echo i486-suse  ${PLATFORM_FILE}
echo i486-noarch  ${PLATFORM_FILE}
echo i486-pc  ${PLATFORM_FILE}
echo noarch-suse  ${PLATFORM_FILE}
-   elif [ ${ARCH} == x86_64 ]; then
+   elif [ ${ARCH} = x86_64 ]; then
echo i486-suse  ${PLATFORM_FILE}
echo i486-noarch  ${PLATFORM_FILE}
echo i486-pc  ${PLATFORM_FILE}
diff --git a/meta/recipes-extended/lsb/lsbtest_1.0.bb 
b/meta/recipes-extended/lsb/lsbtest_1.0.bb
index a877beb..f61251e 100644
--- a/meta/recipes-extended/lsb/lsbtest_1.0.bb
+++ b/meta/recipes-extended/lsb/lsbtest_1.0.bb
@@ -19,15 +19,15 @@ do_install() {
install -d  ${D}/opt/lsb-test
install -m 0644 ${S}/packages_list ${D}/opt/lsb-test/packages_list
install -m 0644 ${S}/session ${D}/opt/lsb-test/session
-   if [ ${TARGET_ARCH} == i586 ];then
+   if [ ${TARGET_ARCH} = i586 ];then
sed -i -e 's/lsbarch/ia32/g' -e 's/targetarch/i486/g' 
${D}/opt/lsb-test/packages_list
sed -i -e 's/targetarch/x86/g' ${D}/opt/lsb-test/session
fi
-   if [ ${TARGET_ARCH} == x86_64 ];then
+   if [ ${TARGET_ARCH} = x86_64 ];then
sed -i -e 's/lsbarch/amd64/g' -e 's/targetarch/x86_64/g' 
${D}/opt/lsb-test/packages_list
sed -i -e 's/targetarch/x86-64/g' ${D}/opt/lsb-test/session
fi
-   if [ ${TARGET_ARCH} == powerpc ];then
+   if [ ${TARGET_ARCH} = powerpc ];then
sed -i -e 's/lsbarch/ppc32/g' -e 's/targetarch/ppc/g' 
${D}/opt/lsb-test/packages_list
sed -i -e 's/targetarch/PPC32/g' ${D}/opt/lsb-test/session
fi
@@ -37,7 +37,7 @@ do_install() {
# in the package_list when MLIB=lib64 is being used.
# Otherwise, by default, the ppc32 LSB packages
# will be downloaded by LSB_Test.sh
-   if [ ${TARGET_ARCH} == powerpc64 ];then
+   if [ ${TARGET_ARCH} = powerpc64 ];then
if [ ${PN} != ${BPN} ];then
sed -i -e 's/lsbarch/ppc64/g' -e 's/targetarch/ppc64/g' 
${D}/opt/lsb-test/packages_list
sed -i -e 's/targetarch/PPC64/g' 
${D}/opt/lsb-test/session
-- 
1.8.5.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 4/4] classes: Add gummiboot class

2014-03-09 Thread Stefan Stanacar

Adds a gummiboot class similar to grub-efi class and makes the necessary
changes so it can be used for live/hddimg images as well.

One can set EFI_PROVIDER = gummiboot in local.conf to use gummiboot instead 
of grub-efi.
Gummiboot requires some kernel options that are not enabled by default, so one 
has to build
with KERNEL_FEATURES_append =  cfg/efi-ext.

The install scripts have been updated too, keeping the old behaviour around,
but accounting for the new boot loader config files (if they exist).
It can be argued that the installer and bootimg are a bit wierd and not 
necessarily correct,
but I wanted to have the exact same behviour with gummiboot.
With the default EFI_PROVIDER = grub-efi nothing changes, everthing should be 
just as before.

I've tested live boot, install and normal boot on:
- FRI2
- genericx86-64 on NUC
with:
  EFI_PROVIDER = gummiboot
  KERNEL_FEATURES_append =  cfg/efi-ext
in local.conf.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/classes/gummiboot.bbclass | 114 +
 .../initrdscripts/files/init-install-efi.sh|  51 ++---
 2 files changed, 149 insertions(+), 16 deletions(-)
 create mode 100644 meta/classes/gummiboot.bbclass

diff --git a/meta/classes/gummiboot.bbclass b/meta/classes/gummiboot.bbclass
new file mode 100644
index 000..0214652
--- /dev/null
+++ b/meta/classes/gummiboot.bbclass
@@ -0,0 +1,114 @@
+# Copyright (C) 2014 Intel Corporation
+#
+# Released under the MIT license (see COPYING.MIT)
+
+# gummiboot.bbclass - equivalent of grub-efi.bbclass
+# Set EFI_PROVIDER = gummiboot to use gummiboot on your live images instead 
of grub-efi
+# (images built by bootimage.bbclass or boot-directdisk.bbclass)
+
+do_bootimg[depends] += gummiboot:do_deploy
+do_bootdirectdisk[depends] += gummiboot:do_deploy
+
+EFIDIR = /EFI/BOOT
+
+GUMMIBOOT_CFG ?= ${S}/loader.conf
+GUMMIBOOT_ENTRIES ?= 
+GUMMIBOOT_TIMEOUT ?= 10
+
+efi_populate() {
+DEST=$1
+
+EFI_IMAGE=gummibootia32.efi
+DEST_EFI_IMAGE=bootia32.efi
+if [ ${TARGET_ARCH} = x86_64 ]; then
+EFI_IMAGE=gummibootx64.efi
+DEST_EFI_IMAGE=bootx64.efi
+fi
+
+install -d ${DEST}${EFIDIR}
+# gummiboot requires these paths for configuration files
+# they are not customizable so no point in new vars
+install -d ${DEST}/loader
+install -d ${DEST}/loader/entries
+install -m 0644 ${DEPLOY_DIR_IMAGE}/${EFI_IMAGE} 
${DEST}${EFIDIR}/${DEST_EFI_IMAGE}
+install -m 0644 ${GUMMIBOOT_CFG} ${DEST}/loader/loader.conf
+for i in ${GUMMIBOOT_ENTRIES}; do
+install -m 0644 ${i} ${DEST}/loader/entries
+done
+}
+
+efi_iso_populate() {
+iso_dir=$1
+efi_populate $iso_dir
+mkdir -p ${EFIIMGDIR}/${EFIDIR}
+cp $iso_dir/${EFIDIR}/* ${EFIIMGDIR}${EFIDIR}
+cp $iso_dir/vmlinuz ${EFIIMGDIR}
+echo ${DEST_EFI_IMAGE}  ${EFIIMGDIR}/startup.nsh
+if [ -f $iso_dir/initrd ] ; then
+cp $iso_dir/initrd ${EFIIMGDIR}
+fi
+}
+
+efi_hddimg_populate() {
+efi_populate $1
+}
+
+python build_efi_cfg() {
+s = d.getVar(S, True)
+labels = d.getVar('LABELS', True)
+if not labels:
+bb.debug(1, LABELS not defined, nothing to do)
+return
+
+if labels == []:
+bb.debug(1, No labels, nothing to do)
+return
+
+cfile = d.getVar('GUMMIBOOT_CFG', True)
+try:
+ cfgfile = open(cfile, 'w')
+except OSError:
+raise bb.build.funcFailed('Unable to open %s' % (cfile))
+
+cfgfile.write('# Automatically created by OE\n')
+cfgfile.write('default %s\n' % (labels.split()[0]))
+timeout = d.getVar('GUMMIBOOT_TIMEOUT', True)
+if timeout:
+cfgfile.write('timeout %s\n' % timeout)
+else:
+cfgfile.write('timeout 10\n')
+cfgfile.close()
+
+for label in labels.split():
+localdata = d.createCopy()
+
+overrides = localdata.getVar('OVERRIDES', True)
+if not overrides:
+raise bb.build.FuncFailed('OVERRIDES not defined')
+
+entryfile = %s/%s.conf % (s, label)
+d.appendVar(GUMMIBOOT_ENTRIES,   + entryfile)
+try:
+entrycfg = open(entryfile, w)
+except OSError:
+raise bb.build.funcFailed('Unable to open %s' % (entryfile))
+localdata.setVar('OVERRIDES', label + ':' + overrides)
+bb.data.update_data(localdata)
+
+entrycfg.write('title %s\n' % label)
+entrycfg.write('linux /vmlinuz\n')
+
+append = localdata.getVar('APPEND', True)
+initrd = localdata.getVar('INITRD', True)
+
+if initrd:
+entrycfg.write('initrd /initrd\n')
+lb = label
+if label == install:
+lb = install-efi
+entrycfg.write('options LABEL=%s ' % lb)
+if append:
+entrycfg.write('%s' % append)
+entrycfg.write('\n

[OE-core] [PATCH 2/4] recipes-bsp: Add gummiboot recipe

2014-03-09 Thread Stefan Stanacar

gummiboot is a simple UEFI boot manager.

Recipe imported from meta-intel with these changes:
 - drop PR and update configure options
 - upgraded to latest version

A couple of notes:
 - If you wish you can install the gummiboot package on the target and
use 'gummiboot install' to add the payload to the ESP (see gummiboot --help,
just make sure the ESP partition has the boot flag on as gummiboot won't accept 
it
otherwise). However the point of this recipe is to be used by bootimg.bbclass 
and
generate images with gummiboot instead of grub-efi.
 - You need a kernel which has CONFIG_EFI_STUB=y at least
 - The default linux-yocto kernel config does not enable that, easiest way
to enable is to build with KERNEL_FEATURES_append =  cfg/efi-ext in local.conf

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/recipes-bsp/gummiboot/gummiboot_git.bb | 25 +
 1 file changed, 25 insertions(+)
 create mode 100644 meta/recipes-bsp/gummiboot/gummiboot_git.bb

diff --git a/meta/recipes-bsp/gummiboot/gummiboot_git.bb 
b/meta/recipes-bsp/gummiboot/gummiboot_git.bb
new file mode 100644
index 000..5868a23
--- /dev/null
+++ b/meta/recipes-bsp/gummiboot/gummiboot_git.bb
@@ -0,0 +1,25 @@
+SUMMARY = Gummiboot is a simple UEFI boot manager which executes configured 
EFI images.
+HOMEPAGE = http://freedesktop.org/wiki/Software/gummiboot;
+
+LICENSE = LGPLv2.1
+LIC_FILES_CHKSUM = file://LICENSE;md5=4fbd65380cdd255951079008b364516c
+
+DEPENDS = gnu-efi util-linux
+
+inherit autotools
+inherit deploy
+
+PV = 43+git${SRCPV}
+SRCREV = 4062c51075ba054d4949c714fe06123f9ad3097d
+SRC_URI = git://anongit.freedesktop.org/gummiboot
+
+S = ${WORKDIR}/git
+
+EXTRA_OECONF = --disable-manpages --with-efi-includedir=${STAGING_INCDIR} \
+--with-efi-ldsdir=${STAGING_LIBDIR} \
+--with-efi-libdir=${STAGING_LIBDIR}
+
+do_deploy () {
+install ${B}/gummiboot*.efi ${DEPLOYDIR}
+}
+addtask deploy before do_build after do_compile
-- 
1.8.5.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/4] recipes-bsp: add gnu-efi recipe

2014-03-09 Thread Stefan Stanacar

gnu-efi is required by gummiboot.
Imported from meta-intel, PR dropped, no new version available.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 .../gnu-efi/gnu-efi/parallel-make-archives.patch   | 48 ++
 .../gnu-efi/gnu-efi/parallel-make.patch| 22 ++
 meta/recipes-bsp/gnu-efi/gnu-efi_3.0u.bb   | 33 +++
 3 files changed, 103 insertions(+)
 create mode 100644 
meta/recipes-bsp/gnu-efi/gnu-efi/parallel-make-archives.patch
 create mode 100644 meta/recipes-bsp/gnu-efi/gnu-efi/parallel-make.patch
 create mode 100644 meta/recipes-bsp/gnu-efi/gnu-efi_3.0u.bb

diff --git a/meta/recipes-bsp/gnu-efi/gnu-efi/parallel-make-archives.patch 
b/meta/recipes-bsp/gnu-efi/gnu-efi/parallel-make-archives.patch
new file mode 100644
index 000..e5b47c1
--- /dev/null
+++ b/meta/recipes-bsp/gnu-efi/gnu-efi/parallel-make-archives.patch
@@ -0,0 +1,48 @@
+Fix parallel make failure for archives
+
+Upstream-Status: Pending
+
+The lib and gnuefi makefiles were using the lib.a() form which compiles
+and ar's as a pair instead of compiling all and then ar'ing which can
+parallelize better. This was resulting in build failures on larger values
+of -j.
+
+See http://www.chemie.fu-berlin.de/chemnet/use/info/make/make_toc.html#TOC105
+for details.
+
+Signed-off-by: Saul Wold s...@linux.intel.com
+Signed-off-by: Darren Hart dvh...@linux.intel.com
+---
+---
+ gnuefi/Makefile |3 ++-
+ lib/Makefile|3 ++-
+ 2 files changed, 4 insertions(+), 2 deletions(-)
+
+Index: gnu-efi-3.0/lib/Makefile
+===
+--- gnu-efi-3.0.orig/lib/Makefile
 gnu-efi-3.0/lib/Makefile
+@@ -66,7 +66,8 @@ all: libsubdirs libefi.a
+ libsubdirs:
+   for sdir in $(SUBDIRS); do mkdir -p $$sdir; done
+ 
+-libefi.a: $(patsubst %,libefi.a(%),$(OBJS))
++libefi.a: $(OBJS)
++  $(AR) rv $@ $(OBJS)
+ 
+ clean:
+   rm -f libefi.a *~ $(OBJS) */*.o
+Index: gnu-efi-3.0/gnuefi/Makefile
+===
+--- gnu-efi-3.0.orig/gnuefi/Makefile
 gnu-efi-3.0/gnuefi/Makefile
+@@ -51,7 +51,8 @@ TARGETS  = crt0-efi-$(ARCH).o libgnuefi.a
+ 
+ all:  $(TARGETS)
+ 
+-libgnuefi.a: $(patsubst %,libgnuefi.a(%),$(OBJS))
++libgnuefi.a: $(OBJS)
++  $(AR) rv $@ $(OBJS)
+ 
+ clean:
+   rm -f $(TARGETS) *~ *.o $(OBJS)
diff --git a/meta/recipes-bsp/gnu-efi/gnu-efi/parallel-make.patch 
b/meta/recipes-bsp/gnu-efi/gnu-efi/parallel-make.patch
new file mode 100644
index 000..27c94e8
--- /dev/null
+++ b/meta/recipes-bsp/gnu-efi/gnu-efi/parallel-make.patch
@@ -0,0 +1,22 @@
+Fix parallel make failure
+
+Upstream-Status: Submitted [Maintainer directly]
+
+Add a missing dependency which resulted in a race leading to failure
+on larger values of -j.
+
+Signed-off-by: Darren Hart dvh...@linux.intel.com
+
+Index: gnu-efi-3.0/Makefile
+===
+--- gnu-efi-3.0.orig/Makefile
 gnu-efi-3.0/Makefile
+@@ -42,6 +42,8 @@ include $(SRCDIR)/Make.defaults
+ 
+ SUBDIRS = lib gnuefi inc apps
+ 
++gnuefi: lib
++
+ all:  check_gcc $(SUBDIRS)
+ 
+ $(SUBDIRS):
diff --git a/meta/recipes-bsp/gnu-efi/gnu-efi_3.0u.bb 
b/meta/recipes-bsp/gnu-efi/gnu-efi_3.0u.bb
new file mode 100644
index 000..0dbdba2
--- /dev/null
+++ b/meta/recipes-bsp/gnu-efi/gnu-efi_3.0u.bb
@@ -0,0 +1,33 @@
+SUMMARY = Libraries for producing EFI binaries
+HOMEPAGE = http://sourceforge.net/projects/gnu-efi/;
+SECTION = devel
+LICENSE = GPLv2+
+LIC_FILES_CHKSUM = 
file://debian/copyright;md5=5fb358a180f484b285b0d99acdc29666
+
+SRC_URI = http://downloads.sourceforge.net/gnu-efi/gnu-efi_3.0u.orig.tar.gz \
+   file://parallel-make.patch \
+   file://parallel-make-archives.patch \
+  
+SRC_URI[md5sum] = d15d3c700e79a1e2938544d73edc572d
+SRC_URI[sha256sum] = 
3c0d450d5829204ca05dcb3b2aae772e52c379b7c7e09146759c6315606f934e
+
+COMPATIBLE_HOST = (x86_64.*|i.86.*)-linux
+
+S = ${WORKDIR}/gnu-efi-3.0
+
+def gnu_efi_arch(d):
+import re
+tarch = d.getVar(TARGET_ARCH, True)
+if re.match(i[3456789]86, tarch):
+return ia32
+return tarch
+
+EXTRA_OEMAKE = 'ARCH=${@gnu_efi_arch(d)}' 'CC=${CC}' 'AS=${AS}' 'LD=${LD}' 
'AR=${AR}' \
+'RANLIB=${RANLIB}' 'OBJCOPY=${OBJCOPY}' 'PREFIX=${prefix}'\
+
+
+do_install() {
+oe_runmake install INSTALLROOT=${D}
+}
+
+FILES_${PN} += ${libdir}/*.lds
-- 
1.8.5.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/4] bootimg/grub-efi.bbclass: allow using a different class for EFI images

2014-03-09 Thread Stefan Stanacar

Abstract away some names so one can select using EFI_PROVIDER a different
class than grub-efi for populating live images, basically allowing the use
of a different bootloader than grub-efi.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/classes/boot-directdisk.bbclass |  7 ---
 meta/classes/bootimg.bbclass |  9 +
 meta/classes/grub-efi.bbclass| 12 ++--
 3 files changed, 15 insertions(+), 13 deletions(-)

diff --git a/meta/classes/boot-directdisk.bbclass 
b/meta/classes/boot-directdisk.bbclass
index 42b3415..88e5c52 100644
--- a/meta/classes/boot-directdisk.bbclass
+++ b/meta/classes/boot-directdisk.bbclass
@@ -32,7 +32,8 @@ BOOTDD_VOLUME_ID   ?= boot
 BOOTDD_EXTRA_SPACE ?= 16384
 
 EFI = ${@base_contains(MACHINE_FEATURES, efi, 1, 0, d)}
-EFI_CLASS = ${@base_contains(MACHINE_FEATURES, efi, grub-efi, , d)}
+EFI_PROVIDER ?= grub-efi
+EFI_CLASS = ${@base_contains(MACHINE_FEATURES, efi, ${EFI_PROVIDER}, 
, d)}
 
 # Include legacy boot if MACHINE_FEATURES includes pcbios or if it does not
 # contain efi. This way legacy is supported by default if neither is
@@ -87,7 +88,7 @@ build_boot_dd() {
syslinux_hddimg_populate $HDDDIR
fi
if [ ${EFI} = 1 ]; then
-   grubefi_hddimg_populate $HDDDIR
+   efi_hddimg_populate $HDDDIR
fi
 
if [ ${IS_VMDK} = true ]; then
@@ -154,7 +155,7 @@ python do_bootdirectdisk() {
 if d.getVar(PCBIOS, True) == 1:
 bb.build.exec_func('build_syslinux_cfg', d)
 if d.getVar(EFI, True) == 1:
-bb.build.exec_func('build_grub_cfg', d)
+bb.build.exec_func('build_efi_cfg', d)
 bb.build.exec_func('build_boot_dd', d)
 }
 
diff --git a/meta/classes/bootimg.bbclass b/meta/classes/bootimg.bbclass
index c370bd6..b13eef9 100644
--- a/meta/classes/bootimg.bbclass
+++ b/meta/classes/bootimg.bbclass
@@ -42,7 +42,8 @@ BOOTIMG_VOLUME_ID   ?= boot
 BOOTIMG_EXTRA_SPACE ?= 512
 
 EFI = ${@base_contains(MACHINE_FEATURES, efi, 1, 0, d)}
-EFI_CLASS = ${@base_contains(MACHINE_FEATURES, efi, grub-efi, , d)}
+EFI_PROVIDER ?= grub-efi
+EFI_CLASS = ${@base_contains(MACHINE_FEATURES, efi, ${EFI_PROVIDER}, 
, d)}
 
 # Include legacy boot if MACHINE_FEATURES includes pcbios or if it does not
 # contain efi. This way legacy is supported by default if neither is
@@ -89,7 +90,7 @@ build_iso() {
syslinux_iso_populate ${ISODIR}
fi
if [ ${EFI} = 1 ]; then
-   grubefi_iso_populate ${ISODIR}
+   efi_iso_populate ${ISODIR}
build_fat_img ${EFIIMGDIR} ${ISODIR}/efi.img
fi
 
@@ -206,7 +207,7 @@ build_hddimg() {
syslinux_hddimg_populate ${HDDDIR}
fi
if [ ${EFI} = 1 ]; then
-   grubefi_hddimg_populate ${HDDDIR}
+   efi_hddimg_populate ${HDDDIR}
fi
 
build_fat_img ${HDDDIR} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.hddimg
@@ -227,7 +228,7 @@ python do_bootimg() {
 if d.getVar(PCBIOS, True) == 1:
 bb.build.exec_func('build_syslinux_cfg', d)
 if d.getVar(EFI, True) == 1:
-bb.build.exec_func('build_grub_cfg', d)
+bb.build.exec_func('build_efi_cfg', d)
 bb.build.exec_func('build_hddimg', d)
 bb.build.exec_func('build_iso', d)
 }
diff --git a/meta/classes/grub-efi.bbclass b/meta/classes/grub-efi.bbclass
index 71bd00f..5c80c17 100644
--- a/meta/classes/grub-efi.bbclass
+++ b/meta/classes/grub-efi.bbclass
@@ -26,7 +26,7 @@ GRUB_OPTS ?= serial --unit=0 --speed=115200 --word=8 
--parity=no --stop=1
 
 EFIDIR = /EFI/BOOT
 
-grubefi_populate() {
+efi_populate() {
# DEST must be the root of the image so that EFIDIR is not
# nested under a top level directory.
DEST=$1
@@ -42,9 +42,9 @@ grubefi_populate() {
install -m 0644 ${GRUBCFG} ${DEST}${EFIDIR}
 }
 
-grubefi_iso_populate() {
+efi_iso_populate() {
iso_dir=$1
-   grubefi_populate $iso_dir
+   efi_populate $iso_dir
# Build a EFI directory to create efi.img
mkdir -p ${EFIIMGDIR}/${EFIDIR}
cp $iso_dir/${EFIDIR}/* ${EFIIMGDIR}${EFIDIR}
@@ -55,11 +55,11 @@ grubefi_iso_populate() {
fi
 }
 
-grubefi_hddimg_populate() {
-   grubefi_populate $1
+efi_hddimg_populate() {
+   efi_populate $1
 }
 
-python build_grub_cfg() {
+python build_efi_cfg() {
 import sys
 
 workdir = d.getVar('WORKDIR', True)
-- 
1.8.5.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 1/4] recipes-bsp: add gnu-efi recipe

2014-03-09 Thread Stefan Stanacar

gnu-efi is required by gummiboot.
Imported from meta-intel, PR dropped, no new version available.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 .../gnu-efi/gnu-efi/parallel-make-archives.patch   | 48 ++
 .../gnu-efi/gnu-efi/parallel-make.patch| 22 ++
 meta/recipes-bsp/gnu-efi/gnu-efi_3.0u.bb   | 33 +++
 3 files changed, 103 insertions(+)
 create mode 100644 
meta/recipes-bsp/gnu-efi/gnu-efi/parallel-make-archives.patch
 create mode 100644 meta/recipes-bsp/gnu-efi/gnu-efi/parallel-make.patch
 create mode 100644 meta/recipes-bsp/gnu-efi/gnu-efi_3.0u.bb

diff --git a/meta/recipes-bsp/gnu-efi/gnu-efi/parallel-make-archives.patch 
b/meta/recipes-bsp/gnu-efi/gnu-efi/parallel-make-archives.patch
new file mode 100644
index 000..e5b47c1
--- /dev/null
+++ b/meta/recipes-bsp/gnu-efi/gnu-efi/parallel-make-archives.patch
@@ -0,0 +1,48 @@
+Fix parallel make failure for archives
+
+Upstream-Status: Pending
+
+The lib and gnuefi makefiles were using the lib.a() form which compiles
+and ar's as a pair instead of compiling all and then ar'ing which can
+parallelize better. This was resulting in build failures on larger values
+of -j.
+
+See http://www.chemie.fu-berlin.de/chemnet/use/info/make/make_toc.html#TOC105
+for details.
+
+Signed-off-by: Saul Wold s...@linux.intel.com
+Signed-off-by: Darren Hart dvh...@linux.intel.com
+---
+---
+ gnuefi/Makefile |3 ++-
+ lib/Makefile|3 ++-
+ 2 files changed, 4 insertions(+), 2 deletions(-)
+
+Index: gnu-efi-3.0/lib/Makefile
+===
+--- gnu-efi-3.0.orig/lib/Makefile
 gnu-efi-3.0/lib/Makefile
+@@ -66,7 +66,8 @@ all: libsubdirs libefi.a
+ libsubdirs:
+   for sdir in $(SUBDIRS); do mkdir -p $$sdir; done
+ 
+-libefi.a: $(patsubst %,libefi.a(%),$(OBJS))
++libefi.a: $(OBJS)
++  $(AR) rv $@ $(OBJS)
+ 
+ clean:
+   rm -f libefi.a *~ $(OBJS) */*.o
+Index: gnu-efi-3.0/gnuefi/Makefile
+===
+--- gnu-efi-3.0.orig/gnuefi/Makefile
 gnu-efi-3.0/gnuefi/Makefile
+@@ -51,7 +51,8 @@ TARGETS  = crt0-efi-$(ARCH).o libgnuefi.a
+ 
+ all:  $(TARGETS)
+ 
+-libgnuefi.a: $(patsubst %,libgnuefi.a(%),$(OBJS))
++libgnuefi.a: $(OBJS)
++  $(AR) rv $@ $(OBJS)
+ 
+ clean:
+   rm -f $(TARGETS) *~ *.o $(OBJS)
diff --git a/meta/recipes-bsp/gnu-efi/gnu-efi/parallel-make.patch 
b/meta/recipes-bsp/gnu-efi/gnu-efi/parallel-make.patch
new file mode 100644
index 000..27c94e8
--- /dev/null
+++ b/meta/recipes-bsp/gnu-efi/gnu-efi/parallel-make.patch
@@ -0,0 +1,22 @@
+Fix parallel make failure
+
+Upstream-Status: Submitted [Maintainer directly]
+
+Add a missing dependency which resulted in a race leading to failure
+on larger values of -j.
+
+Signed-off-by: Darren Hart dvh...@linux.intel.com
+
+Index: gnu-efi-3.0/Makefile
+===
+--- gnu-efi-3.0.orig/Makefile
 gnu-efi-3.0/Makefile
+@@ -42,6 +42,8 @@ include $(SRCDIR)/Make.defaults
+ 
+ SUBDIRS = lib gnuefi inc apps
+ 
++gnuefi: lib
++
+ all:  check_gcc $(SUBDIRS)
+ 
+ $(SUBDIRS):
diff --git a/meta/recipes-bsp/gnu-efi/gnu-efi_3.0u.bb 
b/meta/recipes-bsp/gnu-efi/gnu-efi_3.0u.bb
new file mode 100644
index 000..0dbdba2
--- /dev/null
+++ b/meta/recipes-bsp/gnu-efi/gnu-efi_3.0u.bb
@@ -0,0 +1,33 @@
+SUMMARY = Libraries for producing EFI binaries
+HOMEPAGE = http://sourceforge.net/projects/gnu-efi/;
+SECTION = devel
+LICENSE = GPLv2+
+LIC_FILES_CHKSUM = 
file://debian/copyright;md5=5fb358a180f484b285b0d99acdc29666
+
+SRC_URI = http://downloads.sourceforge.net/gnu-efi/gnu-efi_3.0u.orig.tar.gz \
+   file://parallel-make.patch \
+   file://parallel-make-archives.patch \
+  
+SRC_URI[md5sum] = d15d3c700e79a1e2938544d73edc572d
+SRC_URI[sha256sum] = 
3c0d450d5829204ca05dcb3b2aae772e52c379b7c7e09146759c6315606f934e
+
+COMPATIBLE_HOST = (x86_64.*|i.86.*)-linux
+
+S = ${WORKDIR}/gnu-efi-3.0
+
+def gnu_efi_arch(d):
+import re
+tarch = d.getVar(TARGET_ARCH, True)
+if re.match(i[3456789]86, tarch):
+return ia32
+return tarch
+
+EXTRA_OEMAKE = 'ARCH=${@gnu_efi_arch(d)}' 'CC=${CC}' 'AS=${AS}' 'LD=${LD}' 
'AR=${AR}' \
+'RANLIB=${RANLIB}' 'OBJCOPY=${OBJCOPY}' 'PREFIX=${prefix}'\
+
+
+do_install() {
+oe_runmake install INSTALLROOT=${D}
+}
+
+FILES_${PN} += ${libdir}/*.lds
-- 
1.8.5.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 2/4] recipes-bsp: Add gummiboot recipe

2014-03-09 Thread Stefan Stanacar

gummiboot is a simple UEFI boot manager.

Recipe imported from meta-intel with these changes:
 - drop PR and update configure options
 - upgraded to latest version

A couple of notes:
 - If you wish you can install the gummiboot package on the target and
use 'gummiboot install' to add the payload to the ESP (see gummiboot --help,
just make sure the ESP partition has the boot flag on as gummiboot won't accept 
it
otherwise). However the point of this recipe is to be used by bootimg.bbclass 
and
generate images with gummiboot instead of grub-efi.
 - You need a kernel which has CONFIG_EFI_STUB=y at least
 - The default linux-yocto kernel config does not enable that, easiest way
to enable is to build with KERNEL_FEATURES_append =  cfg/efi-ext in local.conf

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/recipes-bsp/gummiboot/gummiboot_git.bb | 25 +
 1 file changed, 25 insertions(+)
 create mode 100644 meta/recipes-bsp/gummiboot/gummiboot_git.bb

diff --git a/meta/recipes-bsp/gummiboot/gummiboot_git.bb 
b/meta/recipes-bsp/gummiboot/gummiboot_git.bb
new file mode 100644
index 000..5868a23
--- /dev/null
+++ b/meta/recipes-bsp/gummiboot/gummiboot_git.bb
@@ -0,0 +1,25 @@
+SUMMARY = Gummiboot is a simple UEFI boot manager which executes configured 
EFI images.
+HOMEPAGE = http://freedesktop.org/wiki/Software/gummiboot;
+
+LICENSE = LGPLv2.1
+LIC_FILES_CHKSUM = file://LICENSE;md5=4fbd65380cdd255951079008b364516c
+
+DEPENDS = gnu-efi util-linux
+
+inherit autotools
+inherit deploy
+
+PV = 43+git${SRCPV}
+SRCREV = 4062c51075ba054d4949c714fe06123f9ad3097d
+SRC_URI = git://anongit.freedesktop.org/gummiboot
+
+S = ${WORKDIR}/git
+
+EXTRA_OECONF = --disable-manpages --with-efi-includedir=${STAGING_INCDIR} \
+--with-efi-ldsdir=${STAGING_LIBDIR} \
+--with-efi-libdir=${STAGING_LIBDIR}
+
+do_deploy () {
+install ${B}/gummiboot*.efi ${DEPLOYDIR}
+}
+addtask deploy before do_build after do_compile
-- 
1.8.5.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 3/4] bootimg/grub-efi.bbclass: allow using a different class for EFI images

2014-03-09 Thread Stefan Stanacar

Abstract away some names so one can select using EFI_PROVIDER a different
class than grub-efi for populating live images, basically allowing the use
of a different bootloader than grub-efi.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/classes/boot-directdisk.bbclass |  7 ---
 meta/classes/bootimg.bbclass |  9 +
 meta/classes/grub-efi.bbclass| 12 ++--
 3 files changed, 15 insertions(+), 13 deletions(-)

diff --git a/meta/classes/boot-directdisk.bbclass 
b/meta/classes/boot-directdisk.bbclass
index 42b3415..88e5c52 100644
--- a/meta/classes/boot-directdisk.bbclass
+++ b/meta/classes/boot-directdisk.bbclass
@@ -32,7 +32,8 @@ BOOTDD_VOLUME_ID   ?= boot
 BOOTDD_EXTRA_SPACE ?= 16384
 
 EFI = ${@base_contains(MACHINE_FEATURES, efi, 1, 0, d)}
-EFI_CLASS = ${@base_contains(MACHINE_FEATURES, efi, grub-efi, , d)}
+EFI_PROVIDER ?= grub-efi
+EFI_CLASS = ${@base_contains(MACHINE_FEATURES, efi, ${EFI_PROVIDER}, 
, d)}
 
 # Include legacy boot if MACHINE_FEATURES includes pcbios or if it does not
 # contain efi. This way legacy is supported by default if neither is
@@ -87,7 +88,7 @@ build_boot_dd() {
syslinux_hddimg_populate $HDDDIR
fi
if [ ${EFI} = 1 ]; then
-   grubefi_hddimg_populate $HDDDIR
+   efi_hddimg_populate $HDDDIR
fi
 
if [ ${IS_VMDK} = true ]; then
@@ -154,7 +155,7 @@ python do_bootdirectdisk() {
 if d.getVar(PCBIOS, True) == 1:
 bb.build.exec_func('build_syslinux_cfg', d)
 if d.getVar(EFI, True) == 1:
-bb.build.exec_func('build_grub_cfg', d)
+bb.build.exec_func('build_efi_cfg', d)
 bb.build.exec_func('build_boot_dd', d)
 }
 
diff --git a/meta/classes/bootimg.bbclass b/meta/classes/bootimg.bbclass
index c370bd6..b13eef9 100644
--- a/meta/classes/bootimg.bbclass
+++ b/meta/classes/bootimg.bbclass
@@ -42,7 +42,8 @@ BOOTIMG_VOLUME_ID   ?= boot
 BOOTIMG_EXTRA_SPACE ?= 512
 
 EFI = ${@base_contains(MACHINE_FEATURES, efi, 1, 0, d)}
-EFI_CLASS = ${@base_contains(MACHINE_FEATURES, efi, grub-efi, , d)}
+EFI_PROVIDER ?= grub-efi
+EFI_CLASS = ${@base_contains(MACHINE_FEATURES, efi, ${EFI_PROVIDER}, 
, d)}
 
 # Include legacy boot if MACHINE_FEATURES includes pcbios or if it does not
 # contain efi. This way legacy is supported by default if neither is
@@ -89,7 +90,7 @@ build_iso() {
syslinux_iso_populate ${ISODIR}
fi
if [ ${EFI} = 1 ]; then
-   grubefi_iso_populate ${ISODIR}
+   efi_iso_populate ${ISODIR}
build_fat_img ${EFIIMGDIR} ${ISODIR}/efi.img
fi
 
@@ -206,7 +207,7 @@ build_hddimg() {
syslinux_hddimg_populate ${HDDDIR}
fi
if [ ${EFI} = 1 ]; then
-   grubefi_hddimg_populate ${HDDDIR}
+   efi_hddimg_populate ${HDDDIR}
fi
 
build_fat_img ${HDDDIR} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.hddimg
@@ -227,7 +228,7 @@ python do_bootimg() {
 if d.getVar(PCBIOS, True) == 1:
 bb.build.exec_func('build_syslinux_cfg', d)
 if d.getVar(EFI, True) == 1:
-bb.build.exec_func('build_grub_cfg', d)
+bb.build.exec_func('build_efi_cfg', d)
 bb.build.exec_func('build_hddimg', d)
 bb.build.exec_func('build_iso', d)
 }
diff --git a/meta/classes/grub-efi.bbclass b/meta/classes/grub-efi.bbclass
index 71bd00f..5c80c17 100644
--- a/meta/classes/grub-efi.bbclass
+++ b/meta/classes/grub-efi.bbclass
@@ -26,7 +26,7 @@ GRUB_OPTS ?= serial --unit=0 --speed=115200 --word=8 
--parity=no --stop=1
 
 EFIDIR = /EFI/BOOT
 
-grubefi_populate() {
+efi_populate() {
# DEST must be the root of the image so that EFIDIR is not
# nested under a top level directory.
DEST=$1
@@ -42,9 +42,9 @@ grubefi_populate() {
install -m 0644 ${GRUBCFG} ${DEST}${EFIDIR}
 }
 
-grubefi_iso_populate() {
+efi_iso_populate() {
iso_dir=$1
-   grubefi_populate $iso_dir
+   efi_populate $iso_dir
# Build a EFI directory to create efi.img
mkdir -p ${EFIIMGDIR}/${EFIDIR}
cp $iso_dir/${EFIDIR}/* ${EFIIMGDIR}${EFIDIR}
@@ -55,11 +55,11 @@ grubefi_iso_populate() {
fi
 }
 
-grubefi_hddimg_populate() {
-   grubefi_populate $1
+efi_hddimg_populate() {
+   efi_populate $1
 }
 
-python build_grub_cfg() {
+python build_efi_cfg() {
 import sys
 
 workdir = d.getVar('WORKDIR', True)
-- 
1.8.5.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 0/4] Add gummiboot boot manager alongside grub-efi

2014-03-09 Thread Stefan Stanacar

(I've sent this earlier but didn't add v2 to subject, this is resend only)

Hello,

This series adds recipes for gummiboot and a class allowing it to be used
as the boot loader for live/hddimg images.
Without changing EFI_PROVIDER, we have the same behaviour for EFI machines.
Using EFI_PROVIDER = gummiboot an installed image will use gummiboot instead
of grub-efi as it's boot manager.

I've tested this on FRI2, NUC and minnow (live USB boot and install from USB 
and boot) 
with  EFI_PROVIDER = gummiboot and  KERNEL_FEATURES_append =  cfg/efi-ext
in local.conf

This is a partial fix for YB#4082.

Changes in V2:
 - split the first patch in two
 - used ${B} instead of ${S} in gummiboot do_deploy
 - change the function names in bootimg/grub-efi instead of using a
prefix

I've tested again the changes on top of master.

Cheers,
Stefan


The following changes since commit abe417e22b1b55c60f722ca434f9d382ef87:

  classes/uboot-config: ignore doc varflag (2014-03-07 15:05:09 +)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib stefans/gummiboot
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=stefans/gummiboot

Stefan Stanacar (4):
  recipes-bsp: add gnu-efi recipe
  recipes-bsp: Add gummiboot recipe
  bootimg/grub-efi.bbclass: allow using a different class for EFI images
  classes: Add gummiboot class

 meta/classes/boot-directdisk.bbclass   |   7 +-
 meta/classes/bootimg.bbclass   |   9 +-
 meta/classes/grub-efi.bbclass  |  12 +--
 meta/classes/gummiboot.bbclass | 114 +
 .../gnu-efi/gnu-efi/parallel-make-archives.patch   |  48 +
 .../gnu-efi/gnu-efi/parallel-make.patch|  22 
 meta/recipes-bsp/gnu-efi/gnu-efi_3.0u.bb   |  33 ++
 meta/recipes-bsp/gummiboot/gummiboot_git.bb|  25 +
 .../initrdscripts/files/init-install-efi.sh|  51 ++---
 9 files changed, 292 insertions(+), 29 deletions(-)
 create mode 100644 meta/classes/gummiboot.bbclass
 create mode 100644 
meta/recipes-bsp/gnu-efi/gnu-efi/parallel-make-archives.patch
 create mode 100644 meta/recipes-bsp/gnu-efi/gnu-efi/parallel-make.patch
 create mode 100644 meta/recipes-bsp/gnu-efi/gnu-efi_3.0u.bb
 create mode 100644 meta/recipes-bsp/gummiboot/gummiboot_git.bb

-- 
1.8.5.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 4/4] classes: Add gummiboot class

2014-03-09 Thread Stefan Stanacar

Adds a gummiboot class similar to grub-efi class and makes the necessary
changes so it can be used for live/hddimg images as well.

One can set EFI_PROVIDER = gummiboot in local.conf to use gummiboot instead 
of grub-efi.
Gummiboot requires some kernel options that are not enabled by default, so one 
has to build
with KERNEL_FEATURES_append =  cfg/efi-ext.

The install scripts have been updated too, keeping the old behaviour around,
but accounting for the new boot loader config files (if they exist).
It can be argued that the installer and bootimg are a bit wierd and not 
necessarily correct,
but I wanted to have the exact same behviour with gummiboot.
With the default EFI_PROVIDER = grub-efi nothing changes, everthing should be 
just as before.

I've tested live boot, install and normal boot on:
- FRI2
- genericx86-64 on NUC
with:
  EFI_PROVIDER = gummiboot
  KERNEL_FEATURES_append =  cfg/efi-ext
in local.conf.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/classes/gummiboot.bbclass | 114 +
 .../initrdscripts/files/init-install-efi.sh|  51 ++---
 2 files changed, 149 insertions(+), 16 deletions(-)
 create mode 100644 meta/classes/gummiboot.bbclass

diff --git a/meta/classes/gummiboot.bbclass b/meta/classes/gummiboot.bbclass
new file mode 100644
index 000..0214652
--- /dev/null
+++ b/meta/classes/gummiboot.bbclass
@@ -0,0 +1,114 @@
+# Copyright (C) 2014 Intel Corporation
+#
+# Released under the MIT license (see COPYING.MIT)
+
+# gummiboot.bbclass - equivalent of grub-efi.bbclass
+# Set EFI_PROVIDER = gummiboot to use gummiboot on your live images instead 
of grub-efi
+# (images built by bootimage.bbclass or boot-directdisk.bbclass)
+
+do_bootimg[depends] += gummiboot:do_deploy
+do_bootdirectdisk[depends] += gummiboot:do_deploy
+
+EFIDIR = /EFI/BOOT
+
+GUMMIBOOT_CFG ?= ${S}/loader.conf
+GUMMIBOOT_ENTRIES ?= 
+GUMMIBOOT_TIMEOUT ?= 10
+
+efi_populate() {
+DEST=$1
+
+EFI_IMAGE=gummibootia32.efi
+DEST_EFI_IMAGE=bootia32.efi
+if [ ${TARGET_ARCH} = x86_64 ]; then
+EFI_IMAGE=gummibootx64.efi
+DEST_EFI_IMAGE=bootx64.efi
+fi
+
+install -d ${DEST}${EFIDIR}
+# gummiboot requires these paths for configuration files
+# they are not customizable so no point in new vars
+install -d ${DEST}/loader
+install -d ${DEST}/loader/entries
+install -m 0644 ${DEPLOY_DIR_IMAGE}/${EFI_IMAGE} 
${DEST}${EFIDIR}/${DEST_EFI_IMAGE}
+install -m 0644 ${GUMMIBOOT_CFG} ${DEST}/loader/loader.conf
+for i in ${GUMMIBOOT_ENTRIES}; do
+install -m 0644 ${i} ${DEST}/loader/entries
+done
+}
+
+efi_iso_populate() {
+iso_dir=$1
+efi_populate $iso_dir
+mkdir -p ${EFIIMGDIR}/${EFIDIR}
+cp $iso_dir/${EFIDIR}/* ${EFIIMGDIR}${EFIDIR}
+cp $iso_dir/vmlinuz ${EFIIMGDIR}
+echo ${DEST_EFI_IMAGE}  ${EFIIMGDIR}/startup.nsh
+if [ -f $iso_dir/initrd ] ; then
+cp $iso_dir/initrd ${EFIIMGDIR}
+fi
+}
+
+efi_hddimg_populate() {
+efi_populate $1
+}
+
+python build_efi_cfg() {
+s = d.getVar(S, True)
+labels = d.getVar('LABELS', True)
+if not labels:
+bb.debug(1, LABELS not defined, nothing to do)
+return
+
+if labels == []:
+bb.debug(1, No labels, nothing to do)
+return
+
+cfile = d.getVar('GUMMIBOOT_CFG', True)
+try:
+ cfgfile = open(cfile, 'w')
+except OSError:
+raise bb.build.funcFailed('Unable to open %s' % (cfile))
+
+cfgfile.write('# Automatically created by OE\n')
+cfgfile.write('default %s\n' % (labels.split()[0]))
+timeout = d.getVar('GUMMIBOOT_TIMEOUT', True)
+if timeout:
+cfgfile.write('timeout %s\n' % timeout)
+else:
+cfgfile.write('timeout 10\n')
+cfgfile.close()
+
+for label in labels.split():
+localdata = d.createCopy()
+
+overrides = localdata.getVar('OVERRIDES', True)
+if not overrides:
+raise bb.build.FuncFailed('OVERRIDES not defined')
+
+entryfile = %s/%s.conf % (s, label)
+d.appendVar(GUMMIBOOT_ENTRIES,   + entryfile)
+try:
+entrycfg = open(entryfile, w)
+except OSError:
+raise bb.build.funcFailed('Unable to open %s' % (entryfile))
+localdata.setVar('OVERRIDES', label + ':' + overrides)
+bb.data.update_data(localdata)
+
+entrycfg.write('title %s\n' % label)
+entrycfg.write('linux /vmlinuz\n')
+
+append = localdata.getVar('APPEND', True)
+initrd = localdata.getVar('INITRD', True)
+
+if initrd:
+entrycfg.write('initrd /initrd\n')
+lb = label
+if label == install:
+lb = install-efi
+entrycfg.write('options LABEL=%s ' % lb)
+if append:
+entrycfg.write('%s' % append)
+entrycfg.write('\n

[OE-core] [PATCH] oeqa/targetcontrol: allow a custom port for SimpleRemoteTarget

2014-03-07 Thread Stefan Stanacar

We had the ability to use a custom port for the ssh connection,
but we weren't using it.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/targetcontrol.py | 9 +++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oeqa/targetcontrol.py b/meta/lib/oeqa/targetcontrol.py
index d8a6ac0..17871f5 100644
--- a/meta/lib/oeqa/targetcontrol.py
+++ b/meta/lib/oeqa/targetcontrol.py
@@ -144,7 +144,12 @@ class SimpleRemoteTarget(BaseTarget):
 
 def __init__(self, d):
 super(SimpleRemoteTarget, self).__init__(d)
-self.ip = d.getVar(TEST_TARGET_IP, True) or bb.fatal('Please set 
TEST_TARGET_IP with the IP address of the machine you want to run the tests 
on.')
+addr = d.getVar(TEST_TARGET_IP, True) or bb.fatal('Please set 
TEST_TARGET_IP with the IP address of the machine you want to run the tests 
on.')
+self.ip = addr.split(:)[0]
+try:
+self.port = addr.split(:)[1]
+except IndexError:
+self.port = None
 bb.note(Target IP: %s % self.ip)
 self.server_ip = d.getVar(TEST_SERVER_IP, True)
 if not self.server_ip:
@@ -158,7 +163,7 @@ class SimpleRemoteTarget(BaseTarget):
 super(SimpleRemoteTarget, self).deploy()
 
 def start(self, params=None):
-self.connection = SSHControl(self.ip, logfile=self.sshlog)
+self.connection = SSHControl(self.ip, logfile=self.sshlog, 
port=self.port)
 
 def stop(self):
 self.connection = None
-- 
1.8.5.3

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] oeqa/targetcontrol: fix loading a controller using a class name

2014-03-07 Thread Stefan Stanacar

This was wrong and if one would do TEST_TARGET = SimpleRemoteTarget
instead of TEST_TARGET = simpleremote it would complain
that there is no such controller when there is.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/targetcontrol.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/targetcontrol.py b/meta/lib/oeqa/targetcontrol.py
index d8a6ac0..490810d 100644
--- a/meta/lib/oeqa/targetcontrol.py
+++ b/meta/lib/oeqa/targetcontrol.py
@@ -9,6 +9,7 @@ import shutil
 import subprocess
 import bb
 import traceback
+import sys
 from oeqa.utils.sshcontrol import SSHControl
 from oeqa.utils.qemurunner import QemuRunner
 from oeqa.controllers.testtargetloader import TestTargetLoader
@@ -25,7 +26,7 @@ def get_target_controller(d):
 # use the class name
 try:
 # is it a core class defined here?
-controller = getattr(__name__, testtarget)
+controller = getattr(sys.modules[__name__], testtarget)
 except AttributeError:
 # nope, perhaps a layer defined one
 try:
-- 
1.8.5.3

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] oeqa/utils/qemurunner: search for login string in the entire boot log

2014-03-07 Thread Stefan Stanacar

If some service is to spammy we might miss the login, so search
in the entire log instead of just the last few lines.
Use qemu in regex too (to avoid a login string from some service).

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/utils/qemurunner.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/meta/lib/oeqa/utils/qemurunner.py 
b/meta/lib/oeqa/utils/qemurunner.py
index 5366a63..c664d00 100644
--- a/meta/lib/oeqa/utils/qemurunner.py
+++ b/meta/lib/oeqa/utils/qemurunner.py
@@ -133,8 +133,7 @@ class QemuRunner:
 if data:
 self.log(data)
 self.bootlog += data
-lastlines = 
\n.join(self.bootlog.splitlines()[-2:])
-if re.search(login:, lastlines):
+if re.search(qemu.* login:, self.bootlog):
 stopread = True
 reachedlogin = True
 bb.note(Reached login banner)
-- 
1.8.5.3

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/3] bootimg/grub-efi.bbclass: allow using a different class for EFI images

2014-03-06 Thread Stefan Stanacar

Abstract away some names so one can select using EFI_PROVIDER a different
class than grub-efi for populating live images, basically allowing the use
of a different bootloader than grub-efi.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/classes/boot-directdisk.bbclass | 7 ---
 meta/classes/bootimg.bbclass | 9 +
 meta/classes/grub-efi.bbclass| 6 ++
 3 files changed, 15 insertions(+), 7 deletions(-)

diff --git a/meta/classes/boot-directdisk.bbclass 
b/meta/classes/boot-directdisk.bbclass
index 42b3415..c35b4bb 100644
--- a/meta/classes/boot-directdisk.bbclass
+++ b/meta/classes/boot-directdisk.bbclass
@@ -32,7 +32,8 @@ BOOTDD_VOLUME_ID   ?= boot
 BOOTDD_EXTRA_SPACE ?= 16384
 
 EFI = ${@base_contains(MACHINE_FEATURES, efi, 1, 0, d)}
-EFI_CLASS = ${@base_contains(MACHINE_FEATURES, efi, grub-efi, , d)}
+EFI_PROVIDER ?= grub-efi
+EFI_CLASS = ${@base_contains(MACHINE_FEATURES, efi, ${EFI_PROVIDER}, 
, d)}
 
 # Include legacy boot if MACHINE_FEATURES includes pcbios or if it does not
 # contain efi. This way legacy is supported by default if neither is
@@ -87,7 +88,7 @@ build_boot_dd() {
syslinux_hddimg_populate $HDDDIR
fi
if [ ${EFI} = 1 ]; then
-   grubefi_hddimg_populate $HDDDIR
+   ${EFICLASS_FUNC_PREFIX}_hddimg_populate $HDDDIR
fi
 
if [ ${IS_VMDK} = true ]; then
@@ -154,7 +155,7 @@ python do_bootdirectdisk() {
 if d.getVar(PCBIOS, True) == 1:
 bb.build.exec_func('build_syslinux_cfg', d)
 if d.getVar(EFI, True) == 1:
-bb.build.exec_func('build_grub_cfg', d)
+bb.build.exec_func('build_efi_cfg', d)
 bb.build.exec_func('build_boot_dd', d)
 }
 
diff --git a/meta/classes/bootimg.bbclass b/meta/classes/bootimg.bbclass
index c370bd6..f8cfd37 100644
--- a/meta/classes/bootimg.bbclass
+++ b/meta/classes/bootimg.bbclass
@@ -42,7 +42,8 @@ BOOTIMG_VOLUME_ID   ?= boot
 BOOTIMG_EXTRA_SPACE ?= 512
 
 EFI = ${@base_contains(MACHINE_FEATURES, efi, 1, 0, d)}
-EFI_CLASS = ${@base_contains(MACHINE_FEATURES, efi, grub-efi, , d)}
+EFI_PROVIDER ?= grub-efi
+EFI_CLASS = ${@base_contains(MACHINE_FEATURES, efi, ${EFI_PROVIDER}, 
, d)}
 
 # Include legacy boot if MACHINE_FEATURES includes pcbios or if it does not
 # contain efi. This way legacy is supported by default if neither is
@@ -89,7 +90,7 @@ build_iso() {
syslinux_iso_populate ${ISODIR}
fi
if [ ${EFI} = 1 ]; then
-   grubefi_iso_populate ${ISODIR}
+   ${EFICLASS_FUNC_PREFIX}_iso_populate ${ISODIR}
build_fat_img ${EFIIMGDIR} ${ISODIR}/efi.img
fi
 
@@ -206,7 +207,7 @@ build_hddimg() {
syslinux_hddimg_populate ${HDDDIR}
fi
if [ ${EFI} = 1 ]; then
-   grubefi_hddimg_populate ${HDDDIR}
+   ${EFICLASS_FUNC_PREFIX}_hddimg_populate ${HDDDIR}
fi
 
build_fat_img ${HDDDIR} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.hddimg
@@ -227,7 +228,7 @@ python do_bootimg() {
 if d.getVar(PCBIOS, True) == 1:
 bb.build.exec_func('build_syslinux_cfg', d)
 if d.getVar(EFI, True) == 1:
-bb.build.exec_func('build_grub_cfg', d)
+bb.build.exec_func('build_efi_cfg', d)
 bb.build.exec_func('build_hddimg', d)
 bb.build.exec_func('build_iso', d)
 }
diff --git a/meta/classes/grub-efi.bbclass b/meta/classes/grub-efi.bbclass
index 71bd00f..f7fe8d8 100644
--- a/meta/classes/grub-efi.bbclass
+++ b/meta/classes/grub-efi.bbclass
@@ -15,6 +15,8 @@
 # ${GRUB_OPTS} - additional options to add to the config, ';' delimited # 
(optional)
 # ${GRUB_TIMEOUT} - timeout before executing the deault label (optional)
 
+EFICLASS_FUNC_PREFIX = grubefi
+
 do_bootimg[depends] += grub-efi:do_deploy
 do_bootdirectdisk[depends] += grub-efi:do_deploy
 
@@ -139,3 +141,7 @@ python build_grub_cfg() {
 
 cfgfile.close()
 }
+
+python build_efi_cfg() {
+bb.build.exec_func(build_grub_cfg, d)
+}
-- 
1.8.5.3

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 0/3] Add gummiboot boot manager alongside grub-efi

2014-03-06 Thread Stefan Stanacar

Hello,

This series adds recipes for gummiboot and a class allowing it to be used
as the boot loader for live/hddimg images.
Without changing EFI_PROVIDER, we have the same behaviour for EFI machines.
Using EFI_PROVIDER = gummiboot an installed image will use gummiboot instead
of grub-efi as it's boot manager.

I've tested this on FRI2, NUC and minnow (live USB boot and install from USB 
and boot) 
with  EFI_PROVIDER = gummiboot and  KERNEL_FEATURES_append =  cfg/efi-ext
in local.conf

This is a partial fix for YB#4082.

Cheers,
Stefan

The following changes since commit a01af0202558e6ed9d16590b3a8d1dd1b95c0374:

  recipes: bump PRs (2014-03-05 17:36:37 +)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib stefans/gummiboot
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=stefans/gummiboot

Stefan Stanacar (3):
  recipes-bsp: Add gummiboot and gnu-efi recipes
  bootimg/grub-efi.bbclass: allow using a different class for EFI images
  classes: Add gummiboot class

 meta/classes/boot-directdisk.bbclass   |   7 +-
 meta/classes/bootimg.bbclass   |   9 +-
 meta/classes/grub-efi.bbclass  |   6 ++
 meta/classes/gummiboot.bbclass | 112 +
 .../gnu-efi/gnu-efi/parallel-make-archives.patch   |  48 +
 .../gnu-efi/gnu-efi/parallel-make.patch|  22 
 meta/recipes-bsp/gnu-efi/gnu-efi_3.0u.bb   |  33 ++
 meta/recipes-bsp/gummiboot/gummiboot_git.bb|  25 +
 .../initrdscripts/files/init-install-efi.sh|  51 +++---
 9 files changed, 290 insertions(+), 23 deletions(-)
 create mode 100644 meta/classes/gummiboot.bbclass
 create mode 100644 
meta/recipes-bsp/gnu-efi/gnu-efi/parallel-make-archives.patch
 create mode 100644 meta/recipes-bsp/gnu-efi/gnu-efi/parallel-make.patch
 create mode 100644 meta/recipes-bsp/gnu-efi/gnu-efi_3.0u.bb
 create mode 100644 meta/recipes-bsp/gummiboot/gummiboot_git.bb

-- 
1.8.5.3

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/3] classes: Add gummiboot class

2014-03-06 Thread Stefan Stanacar

Adds a gummiboot class similar to grub-efi class and makes the necessary
changes so it can be used for live/hddimg images as well.

One can set EFI_PROVIDER = gummiboot in local.conf to use gummiboot instead 
of grub-efi.
Gummiboot requires some kernel options that are not enabled by default, so one 
has to build
with KERNEL_FEATURES_append =  cfg/efi-ext.

The install scripts have been updated too, keeping the old behaviour around,
but accounting for the new boot loader config files (if they exist).
It can be argued that the installer and bootimg are a bit wierd and not 
necessarily correct,
but I wanted to have the exact same behviour with gummiboot.
With the default EFI_PROVIDER = grub-efi nothing changes, everthing should be 
just as before.

I've tested live boot, install and normal boot on:
- FRI2
- genericx86-64 on NUC
with:
  EFI_PROVIDER = gummiboot
  KERNEL_FEATURES_append =  cfg/efi-ext
in local.conf.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/classes/gummiboot.bbclass | 112 +
 .../initrdscripts/files/init-install-efi.sh|  51 +++---
 2 files changed, 147 insertions(+), 16 deletions(-)
 create mode 100644 meta/classes/gummiboot.bbclass

diff --git a/meta/classes/gummiboot.bbclass b/meta/classes/gummiboot.bbclass
new file mode 100644
index 000..5c11286
--- /dev/null
+++ b/meta/classes/gummiboot.bbclass
@@ -0,0 +1,112 @@
+EFICLASS_FUNC_PREFIX = gummiboot
+
+do_bootimg[depends] += gummiboot:do_deploy
+do_bootdirectdisk[depends] += gummiboot:do_deploy
+
+EFIDIR = /EFI/BOOT
+
+GUMMIBOOT_CFG ?= ${S}/loader.conf
+GUMMIBOOT_ENTRIES ?= 
+GUMMIBOOT_TIMEOUT ?= 10
+
+gummiboot_populate() {
+DEST=$1
+
+EFI_IMAGE=gummibootia32.efi
+DEST_EFI_IMAGE=bootia32.efi
+if [ ${TARGET_ARCH} = x86_64 ]; then
+EFI_IMAGE=gummibootx64.efi
+DEST_EFI_IMAGE=bootx64.efi
+fi
+
+install -d ${DEST}${EFIDIR}
+# gummiboot requires these paths for configuration files
+# they are not customizable so no point in new vars
+install -d ${DEST}/loader
+install -d ${DEST}/loader/entries
+install -m 0644 ${DEPLOY_DIR_IMAGE}/${EFI_IMAGE} 
${DEST}${EFIDIR}/${DEST_EFI_IMAGE}
+install -m 0644 ${GUMMIBOOT_CFG} ${DEST}/loader/loader.conf
+for i in ${GUMMIBOOT_ENTRIES}; do
+install -m 0644 ${i} ${DEST}/loader/entries
+done
+}
+
+gummiboot_iso_populate() {
+iso_dir=$1
+gummiboot_populate $iso_dir
+mkdir -p ${EFIIMGDIR}/${EFIDIR}
+cp $iso_dir/${EFIDIR}/* ${EFIIMGDIR}${EFIDIR}
+cp $iso_dir/vmlinuz ${EFIIMGDIR}
+echo ${DEST_EFI_IMAGE}  ${EFIIMGDIR}/startup.nsh
+if [ -f $iso_dir/initrd ] ; then
+cp $iso_dir/initrd ${EFIIMGDIR}
+fi
+}
+
+gummiboot_hddimg_populate() {
+gummiboot_populate $1
+}
+
+python build_gummiboot_cfg() {
+s = d.getVar(S, True)
+labels = d.getVar('LABELS', True)
+if not labels:
+bb.debug(1, LABELS not defined, nothing to do)
+return
+
+if labels == []:
+bb.debug(1, No labels, nothing to do)
+return
+
+cfile = d.getVar('GUMMIBOOT_CFG', True)
+try:
+ cfgfile = open(cfile, 'w')
+except OSError:
+raise bb.build.funcFailed('Unable to open %s' % (cfile))
+
+cfgfile.write('# Automatically created by OE\n')
+cfgfile.write('default %s\n' % (labels.split()[0]))
+timeout = d.getVar('GUMMIBOOT_TIMEOUT', True)
+if timeout:
+cfgfile.write('timeout %s\n' % timeout)
+else:
+cfgfile.write('timeout 10\n')
+cfgfile.close()
+
+for label in labels.split():
+localdata = d.createCopy()
+
+overrides = localdata.getVar('OVERRIDES', True)
+if not overrides:
+raise bb.build.FuncFailed('OVERRIDES not defined')
+
+entryfile = %s/%s.conf % (s, label)
+d.appendVar(GUMMIBOOT_ENTRIES,   + entryfile)
+try:
+entrycfg = open(entryfile, w)
+except OSError:
+raise bb.build.funcFailed('Unable to open %s' % (entryfile))
+localdata.setVar('OVERRIDES', label + ':' + overrides)
+bb.data.update_data(localdata)
+
+entrycfg.write('title %s\n' % label)
+entrycfg.write('linux /vmlinuz\n')
+
+append = localdata.getVar('APPEND', True)
+initrd = localdata.getVar('INITRD', True)
+
+if initrd:
+entrycfg.write('initrd /initrd\n')
+lb = label
+if label == install:
+lb = install-efi
+entrycfg.write('options LABEL=%s ' % lb)
+if append:
+entrycfg.write('%s' % append)
+entrycfg.write('\n')
+entrycfg.close()
+}
+
+python build_efi_cfg() {
+bb.build.exec_func(build_gummiboot_cfg, d)
+}
diff --git a/meta/recipes-core/initrdscripts/files/init-install-efi.sh 
b/meta/recipes-core/initrdscripts/files/init-install-efi.sh
index

[OE-core] [PATCH] testimage: add task level lock

2014-03-01 Thread Stefan Stanacar

For machines other than qemu it will not be okay to run
multiple testimage tasks in parallel.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/classes/testimage-auto.bbclass | 1 +
 meta/classes/testimage.bbclass  | 4 
 2 files changed, 5 insertions(+)

diff --git a/meta/classes/testimage-auto.bbclass 
b/meta/classes/testimage-auto.bbclass
index a5b8f7f..860599d 100644
--- a/meta/classes/testimage-auto.bbclass
+++ b/meta/classes/testimage-auto.bbclass
@@ -20,3 +20,4 @@ python do_testimage_auto() {
 }
 addtask testimage_auto before do_build after do_rootfs
 do_testimage_auto[depends] += ${TESTIMAGEDEPENDS}
+do_testimage_auto[lockfiles] += ${TESTIMAGELOCK}
diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 48e1032..691c7f6 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -42,12 +42,16 @@ TEST_SERVER_IP ?= 
 TESTIMAGEDEPENDS = 
 TESTIMAGEDEPENDS_qemuall = qemu-native:do_populate_sysroot 
qemu-helper-native:do_populate_sysroot
 
+TESTIMAGELOCK = ${TMPDIR}/testimage.lock
+TESTIMAGELOCK_qemuall = 
+
 python do_testimage() {
 testimage_main(d)
 }
 addtask testimage
 do_testimage[nostamp] = 1
 do_testimage[depends] += ${TESTIMAGEDEPENDS}
+do_testimage[lockfiles] += ${TESTIMAGELOCK}
 
 
 def get_tests_list(d):
-- 
1.8.5.3

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] oeqa/targetcontrol: make BaseTarget an abstract class

2014-02-27 Thread Stefan Stanacar

This should make it clear what methods a subclass
needs to redefine.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/targetcontrol.py | 16 
 1 file changed, 16 insertions(+)

diff --git a/meta/lib/oeqa/targetcontrol.py b/meta/lib/oeqa/targetcontrol.py
index 46e5f7f..d8a6ac0 100644
--- a/meta/lib/oeqa/targetcontrol.py
+++ b/meta/lib/oeqa/targetcontrol.py
@@ -12,6 +12,7 @@ import traceback
 from oeqa.utils.sshcontrol import SSHControl
 from oeqa.utils.qemurunner import QemuRunner
 from oeqa.controllers.testtargetloader import TestTargetLoader
+from abc import ABCMeta, abstractmethod
 
 def get_target_controller(d):
 testtarget = d.getVar(TEST_TARGET, True)
@@ -40,6 +41,8 @@ def get_target_controller(d):
 
 class BaseTarget(object):
 
+__metaclass__ = ABCMeta
+
 def __init__(self, d):
 self.connection = None
 self.ip = None
@@ -48,6 +51,7 @@ class BaseTarget(object):
 self.testdir = d.getVar(TEST_LOG_DIR, True)
 self.pn = d.getVar(PN, True)
 
+@abstractmethod
 def deploy(self):
 
 self.sshlog = os.path.join(self.testdir, ssh_target_log.%s % 
self.datetime)
@@ -57,6 +61,18 @@ class BaseTarget(object):
 os.symlink(self.sshlog, sshloglink)
 bb.note(SSH log file: %s %  self.sshlog)
 
+@abstractmethod
+def start(self, params=None):
+pass
+
+@abstractmethod
+def stop(self):
+pass
+
+@abstractmethod
+def restart(self, params=None):
+pass
+
 def run(self, cmd, timeout=None):
 return self.connection.run(cmd, timeout)
 
-- 
1.8.5.3

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] oeqa/targetcontrol: properly get the host ip

2014-02-25 Thread Stefan Stanacar

For the SimpleRemote target where we need the host ip and
it wasn't set in conf, we tried to determine it automatically.
However ip route output isn't the same for every network, we
need the last field from the first line.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/targetcontrol.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/targetcontrol.py b/meta/lib/oeqa/targetcontrol.py
index ba5e6e5..46e5f7f 100644
--- a/meta/lib/oeqa/targetcontrol.py
+++ b/meta/lib/oeqa/targetcontrol.py
@@ -133,7 +133,7 @@ class SimpleRemoteTarget(BaseTarget):
 self.server_ip = d.getVar(TEST_SERVER_IP, True)
 if not self.server_ip:
 try:
-self.server_ip = subprocess.check_output(['ip', 'route', 
'get', self.ip ]).split()[6]
+self.server_ip = subprocess.check_output(['ip', 'route', 
'get', self.ip ]).split(\n)[0].split()[-1]
 except Exception as e:
 bb.fatal(Failed to determine the host IP address 
(alternatively you can set TEST_SERVER_IP with the IP address of this machine): 
%s % e)
 bb.note(Server IP: %s % self.server_ip)
-- 
1.8.5.3

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] oeqa/runtime: smart: don't run in auto mode when rpm is not the pm

2014-02-12 Thread Stefan Stanacar

TEST_SUITES = auto would run smart tests even for non-rpm images,
which is pointless.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/runtime/smart.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/lib/oeqa/runtime/smart.py b/meta/lib/oeqa/runtime/smart.py
index 8299bf3..195f117 100644
--- a/meta/lib/oeqa/runtime/smart.py
+++ b/meta/lib/oeqa/runtime/smart.py
@@ -9,6 +9,8 @@ def setUpModule():
 skipModule(Image doesn't have package management feature)
 if not oeRuntimeTest.hasPackage(smart):
 skipModule(Image doesn't have smart installed)
+if package_rpm != oeRuntimeTest.tc.d.getVar(PACKAGE_CLASSES, 
True).split()[0]:
+skipModule(Rpm is not the primary package manager)
 
 class SmartTest(oeRuntimeTest):
 
-- 
1.8.5.3

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] testimage: fail if no package manifest is found

2014-02-09 Thread Stefan Stanacar

Sometimes we may forget to actually build the image
we want to test (when testimage task is called manually).
Instead of an ugly traceback we should fail nicely.
The manifest is written after the rootfs so this ensures
the image was actually built.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/classes/testimage.bbclass | 7 +--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 75ab716..48e1032 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -185,8 +185,11 @@ def testimage_main(d):
 self.imagefeatures = d.getVar(IMAGE_FEATURES, True).split()
 self.distrofeatures = d.getVar(DISTRO_FEATURES, True).split()
 manifest = os.path.join(d.getVar(DEPLOY_DIR_IMAGE, True), 
d.getVar(IMAGE_LINK_NAME, True) + .manifest)
-with open(manifest) as f:
-self.pkgmanifest = f.read()
+try:
+with open(manifest) as f:
+self.pkgmanifest = f.read()
+except IOError as e:
+bb.fatal(No package manifest file found. Did you build the 
image?\n%s % e)
 
 # test context
 tc = TestContext()
-- 
1.8.5.3

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] oeqa/utils: targetbuild: take proxy into account

2014-02-09 Thread Stefan Stanacar

A previous commit broke downloads
when proxies are involved, let's fix it.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/runtime/buildcvs.py  |  2 +-
 meta/lib/oeqa/runtime/buildiptables.py |  2 +-
 meta/lib/oeqa/runtime/buildsudoku.py   |  2 +-
 meta/lib/oeqa/utils/targetbuild.py | 20 ++--
 4 files changed, 21 insertions(+), 5 deletions(-)

diff --git a/meta/lib/oeqa/runtime/buildcvs.py 
b/meta/lib/oeqa/runtime/buildcvs.py
index 9bf764d..f1fbf19 100644
--- a/meta/lib/oeqa/runtime/buildcvs.py
+++ b/meta/lib/oeqa/runtime/buildcvs.py
@@ -10,7 +10,7 @@ class BuildCvsTest(oeRuntimeTest):
 
 @classmethod
 def setUpClass(self):
-self.project = TargetBuildProject(oeRuntimeTest.tc.target,
+self.project = TargetBuildProject(oeRuntimeTest.tc.target, 
oeRuntimeTest.tc.d,
 
http://ftp.gnu.org/non-gnu/cvs/source/feature/1.12.13/cvs-1.12.13.tar.bz2;)
 self.project.download_archive()
 
diff --git a/meta/lib/oeqa/runtime/buildiptables.py 
b/meta/lib/oeqa/runtime/buildiptables.py
index 50faf5d..f6061a7 100644
--- a/meta/lib/oeqa/runtime/buildiptables.py
+++ b/meta/lib/oeqa/runtime/buildiptables.py
@@ -10,7 +10,7 @@ class BuildIptablesTest(oeRuntimeTest):
 
 @classmethod
 def setUpClass(self):
-self.project = TargetBuildProject(oeRuntimeTest.tc.target,
+self.project = TargetBuildProject(oeRuntimeTest.tc.target, 
oeRuntimeTest.tc.d,
 
http://netfilter.org/projects/iptables/files/iptables-1.4.13.tar.bz2;)
 self.project.download_archive()
 
diff --git a/meta/lib/oeqa/runtime/buildsudoku.py 
b/meta/lib/oeqa/runtime/buildsudoku.py
index 61dc1ff..a754f1d 100644
--- a/meta/lib/oeqa/runtime/buildsudoku.py
+++ b/meta/lib/oeqa/runtime/buildsudoku.py
@@ -10,7 +10,7 @@ class SudokuTest(oeRuntimeTest):
 
 @classmethod
 def setUpClass(self):
-self.project = TargetBuildProject(oeRuntimeTest.tc.target,
+self.project = TargetBuildProject(oeRuntimeTest.tc.target, 
oeRuntimeTest.tc.d,
 
http://downloads.sourceforge.net/project/sudoku-savant/sudoku-savant/sudoku-savant-1.3/sudoku-savant-1.3.tar.bz2;)
 self.project.download_archive()
 
diff --git a/meta/lib/oeqa/utils/targetbuild.py 
b/meta/lib/oeqa/utils/targetbuild.py
index 77181b1..3229676 100644
--- a/meta/lib/oeqa/utils/targetbuild.py
+++ b/meta/lib/oeqa/utils/targetbuild.py
@@ -11,8 +11,9 @@ import subprocess
 
 class TargetBuildProject():
 
-def __init__(self, target, uri, foldername=None):
+def __init__(self, target, d, uri, foldername=None):
 self.target = target
+self.d = d
 self.uri = uri
 self.targetdir = ~/
 self.archive = os.path.basename(uri)
@@ -23,7 +24,22 @@ class TargetBuildProject():
 
 def download_archive(self):
 
-subprocess.check_call(wget -O %s %s % (self.localarchive, self.uri), 
shell=True)
+exportvars = ['HTTP_PROXY', 'http_proxy',
+  'HTTPS_PROXY', 'https_proxy',
+  'FTP_PROXY', 'ftp_proxy',
+  'FTPS_PROXY', 'ftps_proxy',
+  'NO_PROXY', 'no_proxy',
+  'ALL_PROXY', 'all_proxy',
+  'SOCKS5_USER', 'SOCKS5_PASSWD']
+
+cmd = ''
+for var in exportvars:
+val = self.d.getVar(var, True)
+if val:
+cmd = 'export ' + var + '=\%s\; %s' % (val, cmd)
+
+cmd = cmd + wget -O %s %s % (self.localarchive, self.uri)
+subprocess.check_call(cmd, shell=True)
 
 (status, output) = self.target.copy_to(self.localarchive, 
self.targetdir)
 if status != 0:
-- 
1.8.5.3

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] scripts/oe-selftest: return based on the test results

2014-02-07 Thread Stefan Stanacar

Regardless if the tests passed or not the script returned 0,
which isn't what one would expect.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 scripts/oe-selftest | 6 --
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/scripts/oe-selftest b/scripts/oe-selftest
index 29fe010..8c4ea92 100755
--- a/scripts/oe-selftest
+++ b/scripts/oe-selftest
@@ -144,8 +144,10 @@ def main():
 add_include()
 result = runner.run(suite)
 log.info(Finished)
-
-return 0
+if result.wasSuccessful():
+return 0
+else:
+return 1
 
 if __name__ == __main__:
 try:
-- 
1.8.5.3

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/3] testimage: add ability to export tests

2014-02-03 Thread Stefan Stanacar

Add the ability to export the tests so that they can run independently of
the build system, as is required if you want to be able to hand the test
execution off to a scheduler.
Booting/deployment of the target is still handled by the build system,
as before, only the execution of the tests happens outside of the build system.
Tests exported are the ones defined in TEST_SUITES.

No tests have been changed as interesting parts of the data store have been
exported and tests can continue to query them as before. Small adjustments were 
made
for a couple of oeqa modules though.

[YOCTO #5613]

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/classes/testimage.bbclass | 107 +++--
 meta/lib/oeqa/oetest.py|  17 +++
 2 files changed, 98 insertions(+), 26 deletions(-)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 4777e14..75ab716 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -25,15 +25,16 @@
 
 TEST_LOG_DIR ?= ${WORKDIR}/testimage
 
+TEST_EXPORT_DIR ?= ${TMPDIR}/testimage/${PN}
+TEST_EXPORT_ONLY ?= 0
+
 DEFAULT_TEST_SUITES = ping auto
 DEFAULT_TEST_SUITES_pn-core-image-minimal = ping
 DEFAULT_TEST_SUITES_pn-core-image-sato = ping ssh df connman syslog xorg scp 
vnc date rpm smart dmesg python
 DEFAULT_TEST_SUITES_pn-core-image-sato-sdk = ping ssh df connman syslog xorg 
scp vnc date perl ldd gcc rpm smart kernelmodule dmesg python
-
 TEST_SUITES ?= ${DEFAULT_TEST_SUITES}
 
 TEST_QEMUBOOT_TIMEOUT ?= 1000
-
 TEST_TARGET ?= qemu
 TEST_TARGET_IP ?= 
 TEST_SERVER_IP ?= 
@@ -85,6 +86,71 @@ def get_tests_list(d):
 
 return testslist
 
+
+def exportTests(d,tc):
+import json
+import shutil
+import pkgutil
+
+exportpath = d.getVar(TEST_EXPORT_DIR, True)
+
+savedata = {}
+savedata[d] = {}
+savedata[target] = {}
+for key in tc.__dict__:
+# special cases
+if key != d and key != target:
+savedata[key] = getattr(tc, key)
+savedata[target][ip] = tc.target.ip or d.getVar(TEST_TARGET_IP, True)
+savedata[target][server_ip] = tc.target.server_ip or 
d.getVar(TEST_SERVER_IP, True)
+
+keys = [ key for key in d.keys() if not key.startswith(_) and not 
key.startswith(BB) \
+and not key.startswith(B_pn) and not key.startswith(do_) and 
not d.getVarFlag(key, func)]
+for key in keys:
+try:
+savedata[d][key] = d.getVar(key, True)
+except bb.data_smart.ExpansionError:
+# we don't care about those anyway
+pass
+
+with open(os.path.join(exportpath, testdata.json), w) as f:
+json.dump(savedata, f, skipkeys=True, indent=4, sort_keys=True)
+
+# now start copying files
+# we'll basically copy everything under meta/lib/oeqa, with these 
exceptions
+#  - oeqa/targetcontrol.py - not needed
+#  - oeqa/selftest - something else
+# That means:
+#   - all tests from oeqa/runtime defined in TEST_SUITES (including from 
other layers)
+#   - the contents of oeqa/utils and oeqa/runtime/files
+#   - oeqa/oetest.py and oeqa/runexport.py (this will get copied to 
exportpath not exportpath/oeqa)
+#   - __init__.py files
+bb.utils.mkdirhier(os.path.join(exportpath, oeqa/runtime/files))
+bb.utils.mkdirhier(os.path.join(exportpath, oeqa/utils))
+# copy test modules, this should cover tests in other layers too
+for t in tc.testslist:
+mod = pkgutil.get_loader(t)
+shutil.copy2(mod.filename, os.path.join(exportpath, oeqa/runtime))
+# copy __init__.py files
+oeqadir = pkgutil.get_loader(oeqa).filename
+shutil.copy2(os.path.join(oeqadir, __init__.py), 
os.path.join(exportpath, oeqa))
+shutil.copy2(os.path.join(oeqadir, runtime/__init__.py), 
os.path.join(exportpath, oeqa/runtime))
+# copy oeqa/oetest.py and oeqa/runexported.py
+shutil.copy2(os.path.join(oeqadir, oetest.py), os.path.join(exportpath, 
oeqa))
+shutil.copy2(os.path.join(oeqadir, runexported.py), exportpath)
+# copy oeqa/utils/*.py
+for root, dirs, files in os.walk(os.path.join(oeqadir, utils)):
+for f in files:
+if f.endswith(.py):
+shutil.copy2(os.path.join(root, f), os.path.join(exportpath, 
oeqa/utils))
+# copy oeqa/runtime/files/*
+for root, dirs, files in os.walk(os.path.join(oeqadir, runtime/files)):
+for f in files:
+shutil.copy2(os.path.join(root, f), os.path.join(exportpath, 
oeqa/runtime/files))
+
+bb.plain(Exported tests to: %s % exportpath)
+
+
 def testimage_main(d):
 import unittest
 import os
@@ -94,7 +160,11 @@ def testimage_main(d):
 from oeqa.targetcontrol import get_target_controller
 
 pn = d.getVar(PN, True)
+export = oe.utils.conditional(TEST_EXPORT_ONLY, 1, True, False, d)
 bb.utils.mkdirhier(d.getVar(TEST_LOG_DIR, True))
+if export:
+bb.utils.remove(d.getVar(TEST_EXPORT_DIR, True

[OE-core] [PATCH 3/3] oeqa/utils: targetbuild: don't use bb.fetch anymore

2014-02-03 Thread Stefan Stanacar

When running tests outside of the build system we can't use
bb.fetch anymore. It was nice but tests and their modules
need to rely on the data storage only as that gets exported.
This module is used by the oeqa/runtime/build* tests.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/utils/targetbuild.py | 27 ---
 1 file changed, 8 insertions(+), 19 deletions(-)

diff --git a/meta/lib/oeqa/utils/targetbuild.py 
b/meta/lib/oeqa/utils/targetbuild.py
index 9b2cf53..77181b1 100644
--- a/meta/lib/oeqa/utils/targetbuild.py
+++ b/meta/lib/oeqa/utils/targetbuild.py
@@ -4,11 +4,9 @@
 
 # Provides a class for automating build tests for projects
 
-from oeqa.oetest import oeRuntimeTest
-import bb.fetch2
-import bb.data
 import os
 import re
+import subprocess
 
 
 class TargetBuildProject():
@@ -16,26 +14,16 @@ class TargetBuildProject():
 def __init__(self, target, uri, foldername=None):
 self.target = target
 self.uri = uri
-self.targetdir = /home/root/
-
-self.localdata = bb.data.createCopy(oeRuntimeTest.tc.d)
-bb.data.update_data(self.localdata)
-
-if not foldername:
-self.archive = os.path.basename(uri)
-self.fname = re.sub(r'.tar.bz2|tar.gz$', '', self.archive)
-else:
+self.targetdir = ~/
+self.archive = os.path.basename(uri)
+self.localarchive = /tmp/ + self.archive
+self.fname = re.sub(r'.tar.bz2|tar.gz$', '', self.archive)
+if foldername:
 self.fname = foldername
 
 def download_archive(self):
 
-try:
-self.localdata.delVar(BB_STRICT_CHECKSUM)
-fetcher = bb.fetch2.Fetch([self.uri], self.localdata)
-fetcher.download()
-self.localarchive = fetcher.localpath(self.uri)
-except bb.fetch2.BBFetchException:
-raise Exception(Failed to download archive: %s % self.uri)
+subprocess.check_call(wget -O %s %s % (self.localarchive, self.uri), 
shell=True)
 
 (status, output) = self.target.copy_to(self.localarchive, 
self.targetdir)
 if status != 0:
@@ -61,3 +49,4 @@ class TargetBuildProject():
 
 def clean(self):
 self.target.run('rm -rf %s' % self.targetdir)
+subprocess.call('rm -f %s' % self.localarchive, shell=True)
-- 
1.8.5.3

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/3] lib/oeqa: add module for running tests outside of the build system

2014-02-03 Thread Stefan Stanacar

This script will run the exported tests outside of the build system.

Simplest way to test this is with a qemu image that you manually start.
For an already build image use this in local.conf:
TEST_EXPORT_ONLY = 1
TEST_TARGET = simpleremote
TEST_TARGET_IP = 192.168.7.2
TEST_SERVER_IP = 192.168.7.1
Export the tests: bitbake core-image-sato -c testimage
Then: runqemu core-image-sato
And: cd build/tmp/testimage/core-image-sato
 ./runexported.py testdata.json

The contents of build/tmp/testimage/core-image-sato can be moved on another 
machine
as long as some paths are updated in the json.
The exported data contains paths to the build dir. We only care about 
DEPLOY_DIR/rpm (
if the rpm and smart tests are enabled), so running the tests on other machine
means that the user has to move the contents and call runexported with 
--deploy-dir PATH:
 ./runexported.py --deploy-dir /path/on/another/machine testdata.json
runexported.py accepts other arguments as well, see --help.

[YOCTO #5613]

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/runexported.py | 140 +++
 1 file changed, 140 insertions(+)
 create mode 100755 meta/lib/oeqa/runexported.py

diff --git a/meta/lib/oeqa/runexported.py b/meta/lib/oeqa/runexported.py
new file mode 100755
index 000..e1b6642
--- /dev/null
+++ b/meta/lib/oeqa/runexported.py
@@ -0,0 +1,140 @@
+#!/usr/bin/env python
+
+
+# Copyright (C) 2013 Intel Corporation
+#
+# Released under the MIT license (see COPYING.MIT)
+
+# This script should be used outside of the build system to run image tests.
+# It needs a json file as input as exported by the build.
+# E.g for an already built image:
+#- export the tests:
+#   TEST_EXPORT_ONLY = 1
+#   TEST_TARGET  = simpleremote
+#   TEST_TARGET_IP = 192.168.7.2
+#   TEST_SERVER_IP = 192.168.7.1
+# bitbake core-image-sato -c testimage
+# Setup your target, e.g for qemu: runqemu core-image-sato
+# cd build/tmp/testimage/core-image-sato
+# ./runexported.py testdata.json
+
+import sys
+import os
+import time
+from optparse import OptionParser
+
+try:
+import simplejson as json
+except ImportError:
+import json
+
+sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), 
oeqa)))
+
+from oeqa.oetest import runTests
+from oeqa.utils.sshcontrol import SSHControl
+
+# this isn't pretty but we need a fake target object
+# for running the tests externally as we don't care
+# about deploy/start we only care about the connection methods (run, copy)
+class FakeTarget(object):
+def __init__(self, d):
+self.connection = None
+self.ip = None
+self.server_ip = None
+self.datetime = time.strftime('%Y%m%d%H%M%S',time.gmtime())
+self.testdir = d.getVar(TEST_LOG_DIR, True)
+self.pn = d.getVar(PN, True)
+
+def exportStart(self):
+self.sshlog = os.path.join(self.testdir, ssh_target_log.%s % 
self.datetime)
+sshloglink = os.path.join(self.testdir, ssh_target_log)
+if os.path.islink(sshloglink):
+os.unlink(sshloglink)
+os.symlink(self.sshlog, sshloglink)
+print(SSH log file: %s %  self.sshlog)
+self.connection = SSHControl(self.ip, logfile=self.sshlog)
+
+def run(self, cmd, timeout=None):
+return self.connection.run(cmd, timeout)
+
+def copy_to(self, localpath, remotepath):
+return self.connection.copy_to(localpath, remotepath)
+
+def copy_from(self, remotepath, localpath):
+return self.connection.copy_from(remotepath, localpath)
+
+
+class MyDataDict(dict):
+def getVar(self, key, unused = None):
+return self.get(key, )
+
+class TestContext(object):
+def __init__(self):
+self.d = None
+self.target = None
+
+def main():
+
+usage = usage: %prog [options] json file
+parser = OptionParser(usage=usage)
+parser.add_option(-t, --target-ip, dest=ip, help=The IP address of 
the target machine. Use this to \
+overwrite the value determined from TEST_TARGET_IP at build time)
+parser.add_option(-s, --server-ip, dest=server_ip, help=The IP 
address of this machine. Use this to \
+overwrite the value determined from TEST_SERVER_IP at build time.)
+parser.add_option(-d, --deploy-dir, dest=deploy_dir, help=Full path 
to the package feeds, that this \
+the contents of what used to be DEPLOY_DIR on the build machine. 
If not specified it will use the value \
+specified in the json if that directory actually exists or it will 
error out.)
+parser.add_option(-l, --log-dir, dest=log_dir, help=This sets the 
path for TEST_LOG_DIR. If not specified \
+the current dir is used. This is used for usually creating a ssh 
log file and a scp test file.)
+
+(options, args) = parser.parse_args()
+if len(args) != 1:
+parser.error(Incorrect number of arguments. The one and only argument 
should be a json file exported by the build

[OE-core] [PATCH 0/3] testimage/oeqa: Add ability to export tests to decouple them from the build system

2014-02-03 Thread Stefan Stanacar

Hello,

These patches add the ability to export and run tests outside of the build.
As tests used to query values from the data storage this exports parts of d
and makes them avaiable in a json file, so tests don't need to change to
make them exportable. Tests exported are the ones defined in TEST_SUITES 
regardless
of layer.
Starting the target (in the default case qemu) is still handled by the build 
system, as before,
so keep that in mind.

Simplest way to test this is with a qemu image that you manually start.
For an already build image use this in local.conf:
TEST_EXPORT_ONLY = 1
TEST_TARGET = simpleremote
TEST_TARGET_IP = 192.168.7.2
TEST_SERVER_IP = 192.168.7.1
Export the tests: bitbake core-image-sato -c testimage 
Then: runqemu core-image-sato
And: cd tmp/testimage/core-image-sato  ./runexported.py testdata.json

In the default case (TEST_EXPORT_ONLY = 0) nothing really changes
so no tests should start failing because of this.

Cheers,
Stefan


The following changes since commit 76f52a81f600fe7dfee0987db361e973b9ab71f8:

  insane: Fix python function whitespace changes (2014-02-03 16:16:30 +)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib stefans/export
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=stefans/export

Stefan Stanacar (3):
  testimage: add ability to export tests
  lib/oeqa: add module for running tests outside of the build system
  oeqa/utils: targetbuild: don't use bb.fetch anymore

 meta/classes/testimage.bbclass | 107 
 meta/lib/oeqa/oetest.py|  17 ++---
 meta/lib/oeqa/runexported.py   | 140 +
 meta/lib/oeqa/utils/targetbuild.py |  27 +++
 4 files changed, 246 insertions(+), 45 deletions(-)
 create mode 100755 meta/lib/oeqa/runexported.py

-- 
1.8.5.3

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] oeqa/runtime: drop restartTarget method

2014-01-29 Thread Stefan Stanacar

Restart is a property of a specific target, not
of a test class, should a test really need to restart
the target the direct method should be called.
Also some tests used this to enforce more ram, which
makes sense only for qemu targets only (and the inital
reason this was needed isn't valid anymore, qemu machines had
the default ram size bumped a while ago).

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/oetest.py| 4 
 meta/lib/oeqa/runtime/buildcvs.py  | 2 --
 meta/lib/oeqa/runtime/buildiptables.py | 2 --
 meta/lib/oeqa/runtime/buildsudoku.py   | 2 --
 4 files changed, 10 deletions(-)

diff --git a/meta/lib/oeqa/oetest.py b/meta/lib/oeqa/oetest.py
index ed545b4..0db6cb8 100644
--- a/meta/lib/oeqa/oetest.py
+++ b/meta/lib/oeqa/oetest.py
@@ -78,10 +78,6 @@ class oeRuntimeTest(oeTest):
 self.target = oeRuntimeTest.tc.target
 super(oeRuntimeTest, self).__init__(methodName)
 
-@classmethod
-def restartTarget(self,params=None):
-oeRuntimeTest.tc.target.restart(params)
-
 
 def getmodule(pos=2):
 # stack returns a list of tuples containg frame information
diff --git a/meta/lib/oeqa/runtime/buildcvs.py 
b/meta/lib/oeqa/runtime/buildcvs.py
index f024dfa..9bf764d 100644
--- a/meta/lib/oeqa/runtime/buildcvs.py
+++ b/meta/lib/oeqa/runtime/buildcvs.py
@@ -10,7 +10,6 @@ class BuildCvsTest(oeRuntimeTest):
 
 @classmethod
 def setUpClass(self):
-self.restartTarget(-m 512)
 self.project = TargetBuildProject(oeRuntimeTest.tc.target,
 
http://ftp.gnu.org/non-gnu/cvs/source/feature/1.12.13/cvs-1.12.13.tar.bz2;)
 self.project.download_archive()
@@ -29,4 +28,3 @@ class BuildCvsTest(oeRuntimeTest):
 @classmethod
 def tearDownClass(self):
 self.project.clean()
-self.restartTarget()
diff --git a/meta/lib/oeqa/runtime/buildiptables.py 
b/meta/lib/oeqa/runtime/buildiptables.py
index 88ece3b..50faf5d 100644
--- a/meta/lib/oeqa/runtime/buildiptables.py
+++ b/meta/lib/oeqa/runtime/buildiptables.py
@@ -10,7 +10,6 @@ class BuildIptablesTest(oeRuntimeTest):
 
 @classmethod
 def setUpClass(self):
-self.restartTarget(-m 512)
 self.project = TargetBuildProject(oeRuntimeTest.tc.target,
 
http://netfilter.org/projects/iptables/files/iptables-1.4.13.tar.bz2;)
 self.project.download_archive()
@@ -29,4 +28,3 @@ class BuildIptablesTest(oeRuntimeTest):
 @classmethod
 def tearDownClass(self):
 self.project.clean()
-self.restartTarget()
diff --git a/meta/lib/oeqa/runtime/buildsudoku.py 
b/meta/lib/oeqa/runtime/buildsudoku.py
index 0a7306d..61dc1ff 100644
--- a/meta/lib/oeqa/runtime/buildsudoku.py
+++ b/meta/lib/oeqa/runtime/buildsudoku.py
@@ -10,7 +10,6 @@ class SudokuTest(oeRuntimeTest):
 
 @classmethod
 def setUpClass(self):
-self.restartTarget(-m 512)
 self.project = TargetBuildProject(oeRuntimeTest.tc.target,
 
http://downloads.sourceforge.net/project/sudoku-savant/sudoku-savant/sudoku-savant-1.3/sudoku-savant-1.3.tar.bz2;)
 self.project.download_archive()
@@ -26,4 +25,3 @@ class SudokuTest(oeRuntimeTest):
 @classmethod
 def tearDownClass(self):
 self.project.clean()
-self.restartTarget()
-- 
1.8.5.3

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] oeqa/runtime: smart: fix adding channels

2014-01-29 Thread Stefan Stanacar

Without this it won't add core2-64 and similar channels, as the
directory name in deploy_dir/rpm uses _ not - as the package arch.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/runtime/smart.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/runtime/smart.py b/meta/lib/oeqa/runtime/smart.py
index 7ef4b0e..8299bf3 100644
--- a/meta/lib/oeqa/runtime/smart.py
+++ b/meta/lib/oeqa/runtime/smart.py
@@ -59,7 +59,7 @@ class SmartRepoTest(SmartTest):
 def test_smart_channel_add(self):
 image_pkgtype = self.tc.d.getVar('IMAGE_PKGTYPE', True)
 deploy_url = 'http://%s:%s/%s' %(self.target.server_ip, 
self.repo_server.port, image_pkgtype)
-pkgarchs = self.tc.d.getVar('PACKAGE_ARCHS', True)
+pkgarchs = self.tc.d.getVar('PACKAGE_ARCHS', 
True).replace(-,_).split()
 for arch in os.listdir('%s/%s' % (self.repo_server.root_dir, 
image_pkgtype)):
 if arch in pkgarchs:
 self.smart('channel -y --add {a} type=rpm-md 
baseurl={u}/{a}'.format(a=arch, u=deploy_url))
-- 
1.8.5.3

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] oeqa/utils: sshcontrol: rewrite the SSHControl class

2014-01-28 Thread Stefan Stanacar

Split the class in two, one to handle the process and the
timeout based on output and one for the actual ssh/scp commands.
The ssh/scp methods now use the same run method.
It does the same thing as before but:
- it looks cleaner.
- adds support for using a different user than root
- optionally, raises an exception when exit code != 0
(that's useful for code outside of tests, where you wouldn't want
to check the return code every time as the tests do)

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/utils/sshcontrol.py | 175 --
 1 file changed, 92 insertions(+), 83 deletions(-)

diff --git a/meta/lib/oeqa/utils/sshcontrol.py 
b/meta/lib/oeqa/utils/sshcontrol.py
index 8913250..d355d5e 100644
--- a/meta/lib/oeqa/utils/sshcontrol.py
+++ b/meta/lib/oeqa/utils/sshcontrol.py
@@ -11,108 +11,117 @@ import time
 import os
 import select
 
-class SSHControl(object):
 
-def __init__(self, ip=None, timeout=300, logfile=None, port=None):
+class SSHProcess(object):
+def __init__(self, **options):
+
+self.defaultopts = {
+stdout: subprocess.PIPE,
+stderr: subprocess.STDOUT,
+stdin: None,
+shell: False,
+bufsize: -1,
+preexec_fn: os.setsid,
+}
+self.options = dict(self.defaultopts)
+self.options.update(options)
+self.status = None
+self.output = None
+self.process = None
+self.starttime = None
+
+def run(self, command, timeout=None):
+self.starttime = time.time()
+output = ''
+self.process = subprocess.Popen(command, **self.options)
+if timeout:
+endtime = self.starttime + timeout
+eof = False
+while time.time()  endtime and not eof:
+if select.select([self.process.stdout], [], [], 5)[0] != []:
+data = os.read(self.process.stdout.fileno(), 1024)
+if not data:
+self.process.stdout.close()
+eof = True
+else:
+output += data
+endtime = time.time() + timeout
+
+# process hasn't returned yet
+if not eof:
+self.process.terminate()
+time.sleep(5)
+try:
+self.process.kill()
+except OSError:
+pass
+output += \nProcess killed - no output for %d seconds. Total 
running time: %d seconds. % (timeout, time.time() - self.starttime)
+else:
+output = self.process.communicate()[0]
+
+self.status = self.process.wait()
+self.output = output.rstrip()
+return (self.status, self.output)
+
+
+class SSHControl(object):
+def __init__(self, ip, logfile=None, timeout=300, user='root', port=None):
 self.ip = ip
-self.timeout = timeout
-self._starttime = None
-self._out = ''
-self._ret = 126
+self.defaulttimeout = timeout
+self.ignore_status = True
 self.logfile = logfile
+self.user = user
 self.ssh_options = [
 '-o', 'UserKnownHostsFile=/dev/null',
 '-o', 'StrictHostKeyChecking=no',
 '-o', 'LogLevel=ERROR'
 ]
-self.ssh = ['ssh', '-l', 'root'] + self.ssh_options
+self.ssh = ['ssh', '-l', self.user ] + self.ssh_options
+self.scp = ['scp'] + self.ssh_options
 if port:
-self.ssh = self.ssh + ['-p', str(port)]
+self.ssh = self.ssh + [ '-p', port ]
+self.scp = self.scp + [ '-P', port ]
 
 def log(self, msg):
 if self.logfile:
 with open(self.logfile, a) as f:
 f.write(%s\n % msg)
 
-def _internal_run(self, cmd):
-# We need this for a proper PATH
-cmd = . /etc/profile;  + cmd
-command = self.ssh + [self.ip, cmd]
+def _internal_run(self, command, timeout=None, ignore_status = True):
 self.log([Running]$ %s %  .join(command))
-self._starttime = time.time()
-# ssh hangs without os.setsid
-proc = subprocess.Popen(command, shell=False, stdout=subprocess.PIPE, 
stderr=subprocess.STDOUT, preexec_fn=os.setsid)
-return proc
-
-def run(self, cmd, timeout=None):
-Run cmd and get it's return code and output.
-Let it run for timeout seconds and then terminate/kill it,
-if time is 0 will let cmd run until it finishes.
-Time can be passed to here or can be set per class instance.
-
-if self.ip:
-sshconn = self._internal_run(cmd)
-else:
-raise Exception(Remote IP hasn't been set, I can't run ssh 
without one.)
 
-# run the command forever
-if timeout == 0:
-output = sshconn.communicate()[0]
-else

[OE-core] [PATCH] oeqa/runtime: rpm: fix finding the test rpm file

2014-01-28 Thread Stefan Stanacar

Translate TUNE_PKGARCH to find the right file (this
used to work because tune for qemux86-64 used to be
x86_64 now it's core2-64)
Also, while using packagedata was nice, it's harder to make
the test exportable and runnable outside of the
build system. (where oe.packagedata isn't available)

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/runtime/rpm.py | 11 ++-
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/meta/lib/oeqa/runtime/rpm.py b/meta/lib/oeqa/runtime/rpm.py
index 154cad5..084d22f 100644
--- a/meta/lib/oeqa/runtime/rpm.py
+++ b/meta/lib/oeqa/runtime/rpm.py
@@ -1,8 +1,8 @@
 import unittest
 import os
+import fnmatch
 from oeqa.oetest import oeRuntimeTest, skipModule
 from oeqa.utils.decorators import *
-import oe.packagedata
 
 def setUpModule():
 if not oeRuntimeTest.hasFeature(package-management):
@@ -27,11 +27,12 @@ class RpmInstallRemoveTest(oeRuntimeTest):
 
 @classmethod
 def setUpClass(self):
-deploydir = os.path.join(oeRuntimeTest.tc.d.getVar('DEPLOY_DIR', 
True), rpm, oeRuntimeTest.tc.d.getVar('TUNE_PKGARCH', True))
-pkgdata = oe.packagedata.read_subpkgdata(rpm-doc, oeRuntimeTest.tc.d)
+pkgarch = oeRuntimeTest.tc.d.getVar('TUNE_PKGARCH', True).replace(-, 
_)
+rpmdir = os.path.join(oeRuntimeTest.tc.d.getVar('DEPLOY_DIR', True), 
rpm, pkgarch)
 # pick rpm-doc as a test file to get installed, because it's small and 
it will always be built for standard targets
-testrpmfile = rpm-doc-%s-%s.%s.rpm % (pkgdata[PKGV], 
pkgdata[PKGR], oeRuntimeTest.tc.d.getVar('TUNE_PKGARCH', True))
-oeRuntimeTest.tc.target.copy_to(os.path.join(deploydir,testrpmfile), 
/tmp/rpm-doc.rpm)
+for f in fnmatch.filter(os.listdir(rpmdir), rpm-doc-*.%s.rpm % 
pkgarch):
+testrpmfile = f
+oeRuntimeTest.tc.target.copy_to(os.path.join(rpmdir,testrpmfile), 
/tmp/rpm-doc.rpm)
 
 @skipUnlessPassed('test_rpm_help')
 def test_rpm_install(self):
-- 
1.8.5.3

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] scripts/runqemu-internal: use -cpu core2duo for qemux86-64

2014-01-28 Thread Stefan Stanacar

Now that the tune for qemux86-64 changed to core2-64 we need to
tell the emulator to use a proper CPU model. With the default setting
of qemu64 we'll get things like:

root@qemux86-64:~# smart --help
traps: python[758] trap invalid opcode ip:7f2af01f6be7 sp:7fff49466ef0 error:0 
in strop.so[7f2af01f5000+6000]
Illegal instruction

If the tune for qemux86 changes, that needs to be updated too.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 scripts/runqemu-internal | 7 ---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/scripts/runqemu-internal b/scripts/runqemu-internal
index e3b0729..0893107 100755
--- a/scripts/runqemu-internal
+++ b/scripts/runqemu-internal
@@ -388,6 +388,7 @@ fi
 
 if [ $MACHINE = qemux86-64 ]; then
 QEMU=qemu-system-x86_64
+CPU_SUBTYPE=core2duo
 if [ ! -z $vga_option ]; then
 QEMU_UI_OPTIONS=$QEMU_UI_OPTIONS
 else
@@ -395,7 +396,7 @@ if [ $MACHINE = qemux86-64 ]; then
 fi
 if [ ${FSTYPE:0:3} = ext -o $FSTYPE = btrfs ]; then
 KERNCMDLINE=vga=0 uvesafb.mode_option=640x480-32 root=$DROOT rw 
mem=$QEMU_MEMORY $KERNEL_NETWORK_CMD
-QEMUOPTIONS=$QEMU_NETWORK_CMD $ROOTFS_OPTIONS $QEMU_UI_OPTIONS
+QEMUOPTIONS=$QEMU_NETWORK_CMD -cpu $CPU_SUBTYPE $ROOTFS_OPTIONS 
$QEMU_UI_OPTIONS
 fi
 if [ $FSTYPE = nfs ]; then
 if [ x$ROOTFS = x ]; then
@@ -407,10 +408,10 @@ if [ $MACHINE = qemux86-64 ]; then
 return 1
 fi
 KERNCMDLINE=root=/dev/nfs nfsroot=$NFS_SERVER:$NFS_DIR,$UNFS_OPTS rw 
$KERNEL_NETWORK_CMD mem=$QEMU_MEMORY
-QEMUOPTIONS=$QEMU_NETWORK_CMD $QEMU_UI_OPTIONS
+QEMUOPTIONS=$QEMU_NETWORK_CMD -cpu $CPU_SUBTYPE $QEMU_UI_OPTIONS
 fi
 if [ $FSTYPE = vmdk ]; then
-QEMUOPTIONS=$QEMU_NETWORK_CMD $QEMU_UI_OPTIONS
+QEMUOPTIONS=$QEMU_NETWORK_CMD -cpu $CPU_SUBTYPE $QEMU_UI_OPTIONS
 fi
 # Currently oprofile's event based interrupt mode doesn't work(Bug #828) in
 # qemux86 and qemux86-64. We can use timer interrupt mode for now.
-- 
1.8.5.3

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] oeqa/runtime: syslog: update --help test

2014-01-27 Thread Stefan Stanacar

busybox 1.22 now returns exitcode 0 instead of 1 for
--help options, so this test needs to be updated when
busybox gets upgraded to 1.22.

https://bugs.busybox.net/show_bug.cgi?id=5612
http://git.busybox.net/busybox/commit/?id=efd0698f74caab0a0c8a51228b923ee142e8e278

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/runtime/syslog.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/runtime/syslog.py b/meta/lib/oeqa/runtime/syslog.py
index 91d7963..b95b361 100644
--- a/meta/lib/oeqa/runtime/syslog.py
+++ b/meta/lib/oeqa/runtime/syslog.py
@@ -11,7 +11,7 @@ class SyslogTest(oeRuntimeTest):
 @skipUnlessPassed(test_ssh)
 def test_syslog_help(self):
 (status,output) = self.target.run('/sbin/syslogd --help')
-self.assertEqual(status, 1, msg=status and output: %s and %s % 
(status,output))
+self.assertEqual(status, 0, msg=status and output: %s and %s % 
(status,output))
 
 @skipUnlessPassed(test_syslog_help)
 def test_syslog_running(self):
-- 
1.8.5.3

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] lib/oeqa: sshcontrol: fix false timeout failures

2014-01-17 Thread Stefan Stanacar

Ocasionally AB shows odd false fails like:
http://autobuilder.yoctoproject.org/main/builders/nightly-arm/builds/1/steps/Running%20Sanity%20Tests/logs/stdio
This should fix that by checking for eof instead of
polling the return code of the ssh process, because the process
might still be there.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/utils/sshcontrol.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oeqa/utils/sshcontrol.py 
b/meta/lib/oeqa/utils/sshcontrol.py
index a0dcf02..3e53ec3 100644
--- a/meta/lib/oeqa/utils/sshcontrol.py
+++ b/meta/lib/oeqa/utils/sshcontrol.py
@@ -77,7 +77,7 @@ class SSHControl(object):
 endtime = time.time() + tdelta
 
 # process hasn't returned yet
-if sshconn.poll() is None:
+if not eof:
 sshconn.terminate()
 time.sleep(3)
 try:
@@ -86,7 +86,7 @@ class SSHControl(object):
 pass
 output += \n[!!! SSH command killed - no output for %d 
seconds. Total running time: %d seconds. % (tdelta, time.time() - 
self._starttime)
 
-self._ret = sshconn.poll()
+self._ret = sshconn.wait()
 # strip the last LF so we can test the output
 self._out = output.rstrip()
 self.log(%s % self._out)
-- 
1.8.4.2

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] lib/oeqa: allow a layer to provide it's own TEST_TARGET class

2014-01-16 Thread Stefan Stanacar

Allows a layer to define new classes in layer/lib/oeqa/utils/controllers.py
and completely control or extend deployment of a target. (core currently
has QemuTarget and SimpleRemoteTarget).
The value of TEST_TARGET must be the name of the new class.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/targetcontrol.py  | 23 +++
 meta/lib/oeqa/utils/__init__.py |  3 +++
 2 files changed, 22 insertions(+), 4 deletions(-)

diff --git a/meta/lib/oeqa/targetcontrol.py b/meta/lib/oeqa/targetcontrol.py
index dee38ec..757f9d3 100644
--- a/meta/lib/oeqa/targetcontrol.py
+++ b/meta/lib/oeqa/targetcontrol.py
@@ -8,18 +8,33 @@ import os
 import shutil
 import subprocess
 import bb
-
+import traceback
 from oeqa.utils.sshcontrol import SSHControl
 from oeqa.utils.qemurunner import QemuRunner
 
 
 def get_target_controller(d):
-if d.getVar(TEST_TARGET, True) == qemu:
+testtarget = d.getVar(TEST_TARGET, True)
+# old, simple names
+if testtarget == qemu:
 return QemuTarget(d)
-elif d.getVar(TEST_TARGET, True) == simpleremote:
+elif testtarget == simpleremote:
 return SimpleRemoteTarget(d)
 else:
-bb.fatal(Please set a valid TEST_TARGET)
+# use the class name
+try:
+# is it a core class defined here?
+controller = getattr(__name__, testtarget)
+except AttributeError:
+# nope, perhaps a layer defined one
+try:
+module = __import__(oeqa.utils.controllers, globals(), 
locals(), [testtarget])
+controller = getattr(module, testtarget)
+except ImportError as e:
+bb.fatal(Failed to import oeqa.utils.controllers:\n%s % 
traceback.format_exc())
+except AttributeError:
+bb.fatal(\%s\ is not a valid value for TEST_TARGET % 
testtarget)
+return controller(d)
 
 
 class BaseTarget(object):
diff --git a/meta/lib/oeqa/utils/__init__.py b/meta/lib/oeqa/utils/__init__.py
index e69de29..8eda927 100644
--- a/meta/lib/oeqa/utils/__init__.py
+++ b/meta/lib/oeqa/utils/__init__.py
@@ -0,0 +1,3 @@
+# Enable other layers to have modules in the same named directory
+from pkgutil import extend_path
+__path__ = extend_path(__path__, __name__)
-- 
1.8.4.2

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] testimage: include traceback when loading a test fails

2013-12-18 Thread Stefan Stanacar

Makes it much easier to figure out where a syntax error is.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/classes/testimage.bbclass | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index add8009..1161e59 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -122,7 +122,8 @@ def testimage_main(d):
 try:
 loadTests(tc)
 except Exception as e:
-bb.fatal(Loading tests failed:\n %s % e)
+import traceback
+bb.fatal(Loading tests failed:\n%s % traceback.format_exc())
 
 target.deploy()
 
-- 
1.8.3.1

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] lib/oeqa/runtime: rewrite the systemd test module

2013-12-18 Thread Stefan Stanacar

They are basically the same tests but:
- they look cleaner, using one single method / assert
- output from unittest will be cleaner (and includes a verbose status when 
needed)
- they are better grouped and use a real, active, enabled service
  (machineid will be dropped and hostnamed was a static service)

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/runtime/systemd.py | 93 +++-
 1 file changed, 45 insertions(+), 48 deletions(-)

diff --git a/meta/lib/oeqa/runtime/systemd.py b/meta/lib/oeqa/runtime/systemd.py
index 31007df..eed29d3 100644
--- a/meta/lib/oeqa/runtime/systemd.py
+++ b/meta/lib/oeqa/runtime/systemd.py
@@ -1,4 +1,5 @@
 import unittest
+import re
 from oeqa.oetest import oeRuntimeTest, skipModule
 from oeqa.utils.decorators import *
 
@@ -9,57 +10,53 @@ def setUpModule():
 skipModule(systemd is not the init manager for this image)
 
 
-class SystemdBasicTest(oeRuntimeTest):
+class SystemdTest(oeRuntimeTest):
 
-@skipUnlessPassed('test_ssh')
-def test_systemd_version(self):
-(status, output) = self.target.run('systemctl --version')
-self.assertEqual(status, 0, msg=status and output: %s and %s % 
(status,output))
-
-class SystemdTests(oeRuntimeTest):
-
-@skipUnlessPassed('test_systemd_version')
-def test_systemd_failed(self):
-(status, output) = self.target.run('systemctl --failed | grep 0 
loaded units listed')
-if status != 0:
-print self.target.run('systemctl status --failed -l')[1]
-self.fail(Some systemd units failed.)
-
-@skipUnlessPassed('test_systemd_version')
-def test_systemd_service(self):
-(status, output) = self.target.run('systemctl list-unit-files | grep 
systemd-hostnamed.service')
-self.assertEqual(status, 0, msg=systemd-hostnamed.service service is 
not available.)
+def systemctl(self, action = '', target = '', expected = 0, verbose = 
False):
+command = 'systemctl %s %s' % (action, target)
+status, output = self.target.run(command)
+message = '\n'.join([command, output])
+if status != expected and verbose:
+message += self.target.run('systemctl status --full %s' % 
target)[1]
+self.assertEqual(status, expected, message)
+return output
 
-@skipUnlessPassed('test_systemd_service')
-def test_systemd_stop(self):
-self.target.run('systemctl stop systemd-hostnamed.service')
-(status, output) = self.target.run('systemctl show 
systemd-hostnamed.service | grep ActiveState | grep =inactive')
-if status != 0:
-(status, output) = self.target.run('systemctl status -l 
systemd-hostnamed.service')
-self.fail(msg=systemd-hostnamed.service service could not be 
stopped. Status:\n + output)
 
-@skipUnlessPassed('test_systemd_stop')
-@skipUnlessPassed('test_systemd_version')
-def test_systemd_start(self):
-self.target.run('systemctl start systemd-hostnamed.service')
-(status, output) = self.target.run('systemctl is-active 
systemd-hostnamed.service')
-if status != 0:
-(status, output) = self.target.run('systemctl status -l 
systemd-hostnamed.service')
-self.fail(msg=systemd-hostnamed.service service could not be 
started. Status:\n + output)
+class SystemdBasicTests(SystemdTest):
 
-@skipUnlessPassed('test_systemd_version')
-def test_systemd_enable(self):
-self.target.run('systemctl enable machineid.service')
-(status, output) = self.target.run('systemctl is-enabled 
machineid.service')
-self.assertEqual(output, 'enabled', msg=machineid.service service 
could not be enabled. Status and output: %s and %s % (status, output))
-
-@skipUnlessPassed('test_systemd_enable')
-def test_systemd_disable(self):
-self.target.run('systemctl disable machineid.service')
-(status, output) = self.target.run('systemctl is-enabled 
machineid.service')
-self.assertEqual(output, 'disabled', msg=machineid.service service 
could not be disabled. Status and output: %s and %s % (status, output))
+@skipUnlessPassed('test_ssh')
+def test_systemd_basic(self):
+self.systemctl('--version')
 
-@skipUnlessPassed('test_systemd_version')
+@skipUnlessPassed('test_system_basic')
 def test_systemd_list(self):
-(status, output) = self.target.run('systemctl list-unit-files')
-self.assertEqual(status, 0, msg=systemctl list-unit-files command 
failed. Status:  %s % status)
+self.systemctl('list-unit-files')
+
+@skipUnlessPassed('test_systemd_basic')
+def test_systemd_failed(self):
+output = self.systemctl('list-units', '--failed')
+match = re.search(0 loaded units listed, output)
+if not match:
+output += self.systemctl('status --full --failed')
+self.assertTrue(match, msg=Some systemd units failed:\n%s % output

[OE-core] [PATCH] lib/oeqa/selftest/base, scripts/oe-selftest: fix wrong remove path and do a complete cleanup at the end

2013-12-04 Thread Stefan Stanacar

The script should clean-up all the .inc files that might
have been created by tests regardless of the outcome or if
the script is interrupted. (currently the
last test will leave a conf/selftest.inc around, even
if it's not included anywhere)
Also fix delete_recipeinc to actually delete what's supposed to.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/selftest/base.py |  2 +-
 scripts/oe-selftest| 12 
 2 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/selftest/base.py b/meta/lib/oeqa/selftest/base.py
index 30a71e8..c3474a3 100644
--- a/meta/lib/oeqa/selftest/base.py
+++ b/meta/lib/oeqa/selftest/base.py
@@ -92,7 +92,7 @@ class oeSelfTest(unittest.TestCase):
 inc_file = os.path.join(self.testlayer_path, 'recipes-test', recipe, 
'test_recipe.inc')
 self.log.debug(Deleting file: %s % inc_file)
 try:
-os.remove(self.testinc_path)
+os.remove(inc_file)
 except OSError as e:
 if e.errno != errno.ENOENT:
 raise
diff --git a/scripts/oe-selftest b/scripts/oe-selftest
index db42e73..29fe010 100755
--- a/scripts/oe-selftest
+++ b/scripts/oe-selftest
@@ -97,6 +97,17 @@ def remove_include():
 ftools.remove_from_file(os.path.join(builddir, conf/local.conf), 
\
 #include added by oe-selftest.py\ninclude selftest.inc)
 
+
+def remove_inc_files():
+try:
+os.remove(os.path.join(os.environ.get(BUILDDIR), 
conf/selftest.inc))
+for root, _, files in os.walk(get_test_layer()):
+for f in files:
+if f == 'test_recipe.inc':
+os.remove(os.path.join(root, f))
+except OSError as e:
+pass
+
 def get_tests():
 testslist = []
 for x in sys.argv[1:]:
@@ -145,4 +156,5 @@ if __name__ == __main__:
 traceback.print_exc(5)
 finally:
 remove_include()
+remove_inc_files()
 sys.exit(ret)
-- 
1.8.3.1

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] lib/oeqa/selftest: buildoptions: small fixes for some tests

2013-12-04 Thread Stefan Stanacar

While harmless, we should overwrite the config not append to it,
and use m4 as target, otherwise the WARN check will
build an entire image and we are not interested in that.
Also add an output check for the WARN_QA test.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/selftest/buildoptions.py | 15 ---
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/meta/lib/oeqa/selftest/buildoptions.py 
b/meta/lib/oeqa/selftest/buildoptions.py
index f99dda7..5fb69d8 100644
--- a/meta/lib/oeqa/selftest/buildoptions.py
+++ b/meta/lib/oeqa/selftest/buildoptions.py
@@ -46,16 +46,16 @@ class DiskMonTest(oeSelfTest):
 def test_stoptask_behavior(self):
 result = runCmd(df -k %s % os.getcwd())
 size = result.output.split(\n)[1].split()[3]
-self.append_config('BB_DISKMON_DIRS = STOPTASKS,${TMPDIR},%sK,4510K' 
% size)
-res = bitbake(core-image-minimal, ignore_status = True)
+self.write_config('BB_DISKMON_DIRS = STOPTASKS,${TMPDIR},%sK,4510K' 
% size)
+res = bitbake(m4, ignore_status = True)
 self.assertTrue('ERROR: No new tasks can be executed since the disk 
space monitor action is STOPTASKS!' in res.output)
 self.assertEqual(res.status, 1)
-self.append_config('BB_DISKMON_DIRS = ABORT,${TMPDIR},%sK,4510K' % 
size)
-res = bitbake(core-image-minimal, ignore_status = True)
+self.write_config('BB_DISKMON_DIRS = ABORT,${TMPDIR},%sK,4510K' % 
size)
+res = bitbake(m4, ignore_status = True)
 self.assertTrue('ERROR: Immediately abort since the disk space monitor 
action is ABORT!' in res.output)
 self.assertEqual(res.status, 1)
-self.append_config('BB_DISKMON_DIRS = WARN,${TMPDIR},%sK,4510K' % 
size)
-res = bitbake(core-image-minimal)
+self.write_config('BB_DISKMON_DIRS = WARN,${TMPDIR},%sK,4510K' % 
size)
+res = bitbake(m4)
 self.assertTrue('WARNING: The free space' in res.output)
 
 class SanityOptionsTest(oeSelfTest):
@@ -74,9 +74,10 @@ class SanityOptionsTest(oeSelfTest):
 self.write_recipeinc('xcursor-transparent-theme', 'PACKAGES += 
\${PN}-dbg\')
 self.append_config('ERROR_QA_remove = packages-list')
 self.append_config('WARN_QA_append =  packages-list')
-bitbake(xcursor-transparent-theme)
+res = bitbake(xcursor-transparent-theme)
 bitbake(xcursor-transparent-theme -ccleansstate)
 self.delete_recipeinc('xcursor-transparent-theme')
+self.assertTrue(WARNING: QA Issue: xcursor-transparent-theme-dbg is 
listed in PACKAGES multiple times, this leads to packaging errors. in 
res.output)
 
 def test_sanity_userspace_dependency(self):
 self.append_config('WARN_QA_append =  unsafe-references-in-binaries 
unsafe-references-in-scripts')
-- 
1.8.3.1

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] lib/oeqa/selftest: add tests for PR service

2013-12-04 Thread Stefan Stanacar

From: Corneliu Stoicescu corneliux.stoice...@intel.com

Signed-off-by: Corneliu Stoicescu corneliux.stoice...@intel.com
Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/selftest/prservice.py | 113 
 1 file changed, 113 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/prservice.py

diff --git a/meta/lib/oeqa/selftest/prservice.py 
b/meta/lib/oeqa/selftest/prservice.py
new file mode 100644
index 000..789c05f
--- /dev/null
+++ b/meta/lib/oeqa/selftest/prservice.py
@@ -0,0 +1,113 @@
+import unittest
+import os
+import logging
+import re
+import shutil
+import datetime
+
+import oeqa.utils.ftools as ftools
+from oeqa.selftest.base import oeSelfTest
+from oeqa.utils.commands import runCmd, bitbake, get_bb_var
+
+class BitbakePrTests(oeSelfTest):
+
+def get_pr_version(self, package_name):
+pkgdata_dir = get_bb_var('PKGDATA_DIR')
+package_data_file = os.path.join(pkgdata_dir, 'runtime', package_name)
+package_data = ftools.read_file(package_data_file)
+find_pr = re.search(PKGR: r[0-9]+\.([0-9]+), package_data)
+self.assertTrue(find_pr)
+return int(find_pr.group(1))
+
+def get_task_stamp(self, package_name, recipe_task):
+stampdata = get_bb_var('STAMP', target=package_name).split('/')
+prefix = stampdata[-1]
+package_stamps_path = /.join(stampdata[:-1])
+stamps = []
+for stamp in os.listdir(package_stamps_path):
+find_stamp = re.match(%s\.%s\.([a-z0-9]{32}) % (prefix, 
recipe_task), stamp)
+if find_stamp:
+stamps.append(find_stamp.group(1))
+self.assertFalse(len(stamps) == 0, msg=Cound not find stamp for task 
%s for recipe %s % (recipe_task, package_name))
+self.assertFalse(len(stamps)  1, msg=Found multiple %s stamps for 
the %s recipe in the %s directory. % (recipe_task, package_name, 
package_stamps_path))
+return str(stamps[0])
+
+def increment_package_pr(self, package_name):
+inc_data = do_package_append() 
{\nbb.build.exec_func('do_test_prserv', d)\n}\ndo_test_prserv() {\necho \The 
current date is: %s\\n} % datetime.datetime.now()
+self.write_recipeinc(package_name, inc_data)
+bitbake(-ccleansstate %s % package_name)
+res = bitbake(package_name, ignore_status=True)
+self.delete_recipeinc(package_name)
+self.assertEqual(res.status, 0, msg=res.output)
+self.assertTrue(NOTE: Started PRServer with DBfile in res.output, 
msg=res.output)
+
+def config_pr_tests(self, package_name, package_type='rpm', 
pr_socket='localhost:0'):
+config_package_data = 'PACKAGE_CLASSES = package_%s' % package_type
+self.write_config(config_package_data)
+config_server_data = 'PRSERV_HOST = %s' % pr_socket
+self.append_config(config_server_data)
+
+def run_test_pr_service(self, package_name, package_type='rpm', 
track_task='do_package', pr_socket='localhost:0'):
+self.config_pr_tests(package_name, package_type, pr_socket)
+
+self.increment_package_pr(package_name)
+pr_1 = self.get_pr_version(package_name)
+stamp_1 = self.get_task_stamp(package_name, track_task)
+
+self.increment_package_pr(package_name)
+pr_2 = self.get_pr_version(package_name)
+stamp_2 = self.get_task_stamp(package_name, track_task)
+
+bitbake(-ccleansstate %s % package_name)
+self.assertTrue(pr_2 - pr_1 == 1)
+self.assertTrue(stamp_1 != stamp_2)
+
+def run_test_pr_export_import(self, package_name, replace_current_db=True):
+self.config_pr_tests(package_name)
+
+self.increment_package_pr(package_name)
+pr_1 = self.get_pr_version(package_name)
+
+exported_db_path = os.path.join(self.builddir, 'export.inc')
+export_result = runCmd(bitbake-prserv-tool export %s % 
exported_db_path, ignore_status=True)
+self.assertEqual(export_result.status, 0, msg=PR Service database 
export failed: %s % export_result.output)
+
+if replace_current_db:
+current_db_path = os.path.join(get_bb_var('PERSISTENT_DIR'), 
'prserv.sqlite3')
+self.assertTrue(os.path.exists(current_db_path), msg=Path to 
current PR Service database is invalid: %s % current_db_path)
+os.remove(current_db_path)
+
+import_result = runCmd(bitbake-prserv-tool import %s % 
exported_db_path, ignore_status=True)
+os.remove(exported_db_path)
+self.assertEqual(import_result.status, 0, msg=PR Service database 
import failed: %s % import_result.output)
+
+self.increment_package_pr(package_name)
+pr_2 = self.get_pr_version(package_name)
+
+bitbake(-ccleansstate %s % package_name)
+self.assertTrue(pr_2 - pr_1 == 1)
+
+
+def test_import_export_replace_db(self):
+self.run_test_pr_export_import('m4')
+
+def test_import_export_override_db(self

[OE-core] [PATCH] lib/oeqa: use the new manifest file for determining if a package is installed

2013-11-28 Thread Stefan Stanacar

Use the new manifest file instead of ${WORKDIR}/installed_pkgs.txt for 
determining
if an image has a certain package, because installed_pkgs.txt goes away with 
rm_work
enabled.
We can't use the IMAGE_MANIFEST var for the file path because that relies on 
IMAGE_NAME which
changes at every run (because of date), so we use the link which points to the 
last
one built.

[ YOCTO #5072 ]

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/oetest.py | 14 --
 1 file changed, 4 insertions(+), 10 deletions(-)

diff --git a/meta/lib/oeqa/oetest.py b/meta/lib/oeqa/oetest.py
index 3bb3589..9566150 100644
--- a/meta/lib/oeqa/oetest.py
+++ b/meta/lib/oeqa/oetest.py
@@ -56,17 +56,11 @@ class oeTest(unittest.TestCase):
 
 @classmethod
 def hasPackage(self, pkg):
-
-pkgfile = os.path.join(oeTest.tc.d.getVar(WORKDIR, True), 
installed_pkgs.txt)
-
-with open(pkgfile) as f:
-data = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ)
-match = re.search(pkg, data)
-data.close()
-
-if match:
+manifest = os.path.join(oeTest.tc.d.getVar(DEPLOY_DIR_IMAGE, True), 
oeTest.tc.d.getVar(IMAGE_LINK_NAME, True) + .manifest)
+with open(manifest) as f:
+data = f.read()
+if re.search(pkg, data):
 return True
-
 return False
 
 @classmethod
-- 
1.8.3.1

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/4] scripts/oe-selftest: script to run builds as unittest against bitbake or various scripts

2013-11-27 Thread Stefan Stanacar

The purpose of oe-selftest is to run unittest modules added from 
meta/lib/oeqa/selftest,
which are tests against bitbake tools.

Right now the script it's useful for simple tests like:
  - bitbake --someoption, change some metadata, bitbake X, check something 
type scenarios (PR service, error output, etc)
  - or bitbake-layers ... type scripts and yocto-bsp tools.

This commit also adds some helper modules that the tests will use and a base 
class.
Also, most of the tests will have a dependency on a meta-selftest layer
which contains specially modified recipes/bbappends/include files for the 
purpose of the tests.
The tests themselves will usually write to .inc files from the layer or in 
conf/selftest.inc
(which is added as an include in local.conf at the start and removed at the end)

It's a simple matter or sourcing the enviroment, adding the meta-selftest layer 
to bblayers.conf
and running: oe-selftest to get some results. It would finish faster if at 
least a core-image-minimal
was built before.

[ YOCTO #4740 ]

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/selftest/__init__.py |   2 +
 meta/lib/oeqa/selftest/base.py |  98 
 meta/lib/oeqa/utils/commands.py| 137 ++
 meta/lib/oeqa/utils/ftools.py  |  27 +++
 scripts/oe-selftest| 148 +
 5 files changed, 412 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/__init__.py
 create mode 100644 meta/lib/oeqa/selftest/base.py
 create mode 100644 meta/lib/oeqa/utils/commands.py
 create mode 100644 meta/lib/oeqa/utils/ftools.py
 create mode 100755 scripts/oe-selftest

diff --git a/meta/lib/oeqa/selftest/__init__.py 
b/meta/lib/oeqa/selftest/__init__.py
new file mode 100644
index 000..3ad9513
--- /dev/null
+++ b/meta/lib/oeqa/selftest/__init__.py
@@ -0,0 +1,2 @@
+from pkgutil import extend_path
+__path__ = extend_path(__path__, __name__)
diff --git a/meta/lib/oeqa/selftest/base.py b/meta/lib/oeqa/selftest/base.py
new file mode 100644
index 000..30a71e8
--- /dev/null
+++ b/meta/lib/oeqa/selftest/base.py
@@ -0,0 +1,98 @@
+# Copyright (c) 2013 Intel Corporation
+#
+# Released under the MIT license (see COPYING.MIT)
+
+
+# DESCRIPTION
+# Base class inherited by test classes in meta/lib/selftest
+
+import unittest
+import os
+import sys
+import logging
+import errno
+
+import oeqa.utils.ftools as ftools
+
+
+class oeSelfTest(unittest.TestCase):
+
+log = logging.getLogger(selftest.base)
+longMessage = True
+
+def __init__(self, methodName=runTest):
+self.builddir = os.environ.get(BUILDDIR)
+self.localconf_path = os.path.join(self.builddir, conf/local.conf)
+self.testinc_path = os.path.join(self.builddir, conf/selftest.inc)
+self.testlayer_path = oeSelfTest.testlayer_path
+super(oeSelfTest, self).__init__(methodName)
+
+def setUp(self):
+os.chdir(self.builddir)
+# we don't know what the previous test left around in config or inc 
files
+# if it failed so we need a fresh start
+try:
+os.remove(self.testinc_path)
+except OSError as e:
+if e.errno != errno.ENOENT:
+raise
+for root, _, files in os.walk(self.testlayer_path):
+for f in files:
+if f == 'test_recipe.inc':
+os.remove(os.path.join(root, f))
+# tests might need their own setup
+# but if they overwrite this one they have to call
+# super each time, so let's give them an alternative
+self.setUpLocal()
+
+def setUpLocal(self):
+pass
+
+def tearDown(self):
+self.tearDownLocal()
+
+def tearDownLocal(self):
+pass
+
+# write to builddir/conf/selftest.inc
+def write_config(self, data):
+self.log.debug(Writing to: %s\n%s\n % (self.testinc_path, data))
+ftools.write_file(self.testinc_path, data)
+
+# append to builddir/conf/selftest.inc
+def append_config(self, data):
+self.log.debug(Appending to: %s\n%s\n % (self.testinc_path, data))
+ftools.append_file(self.testinc_path, data)
+
+# remove data from builddir/conf/selftest.inc
+def remove_config(self, data):
+self.log.debug(Removing from: %s\n\%s\n % (self.testinc_path, data))
+ftools.remove_from_file(self.testinc_path, data)
+
+# write to meta-sefltest/recipes-test/recipe/test_recipe.inc
+def write_recipeinc(self, recipe, data):
+inc_file = os.path.join(self.testlayer_path, 'recipes-test', recipe, 
'test_recipe.inc')
+self.log.debug(Writing to: %s\n%s\n % (inc_file, data))
+ftools.write_file(inc_file, data)
+
+# append data to meta-sefltest/recipes-test/recipe/test_recipe.inc
+def append_recipeinc(self, recipe, data):
+inc_file = os.path.join(self.testlayer_path, 'recipes-test', recipe, 
'test_recipe.inc')
+self.log.debug

[OE-core] [PATCH 3/4] lib/oeqa/selftest: buildoptions.py: add simple image build tests

2013-11-27 Thread Stefan Stanacar

From: Alexandru Palalau alexandrux.pala...@intel.com

Build images and tests different build options like RM_OLD_IMAGE
and for WARN_QA/ERROR_QA behaviour.

Signed-off-by: Alexandru Palalau alexandrux.pala...@intel.com
Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/selftest/buildoptions.py | 86 ++
 1 file changed, 86 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/buildoptions.py

diff --git a/meta/lib/oeqa/selftest/buildoptions.py 
b/meta/lib/oeqa/selftest/buildoptions.py
new file mode 100644
index 000..f99dda7
--- /dev/null
+++ b/meta/lib/oeqa/selftest/buildoptions.py
@@ -0,0 +1,86 @@
+import unittest
+import os
+import logging
+import re
+
+from oeqa.selftest.base import oeSelfTest
+from oeqa.utils.commands import runCmd, bitbake, get_bb_var
+import oeqa.utils.ftools as ftools
+
+class ImageOptionsTests(oeSelfTest):
+
+def test_incremental_image_generation(self):
+bitbake(-c cleanall core-image-minimal)
+self.write_config('INC_RPM_IMAGE_GEN = 1')
+self.append_config('IMAGE_FEATURES += ssh-server-openssh')
+bitbake(core-image-minimal)
+res = runCmd(grep 'Installing openssh-sshd' %s % 
(os.path.join(get_bb_var(WORKDIR, core-image-minimal), 
temp/log.do_rootfs)), ignore_status=True)
+self.remove_config('IMAGE_FEATURES += ssh-server-openssh')
+self.assertEqual(0, res.status, msg=No match for openssh-sshd in 
log.do_rootfs)
+bitbake(core-image-minimal)
+res = runCmd(grep 'Removing openssh-sshd' %s 
%(os.path.join(get_bb_var(WORKDIR, core-image-minimal), 
temp/log.do_rootfs)),ignore_status=True)
+self.assertEqual(0, res.status, msg=openssh-sshd was not removed from 
image)
+
+def test_rm_old_image(self):
+bitbake(core-image-minimal)
+deploydir = get_bb_var(DEPLOY_DIR_IMAGE, target=core-image-minimal)
+imagename = get_bb_var(IMAGE_LINK_NAME, target=core-image-minimal)
+oldimgpath = os.path.realpath(os.path.join(deploydir, imagename + 
.ext3))
+self.append_config(RM_OLD_IMAGE = \1\)
+bitbake(core-image-minimal)
+self.assertFalse(os.path.exists(oldimgpath), msg=Old image path still 
exists: %s % oldimgpath)
+
+def test_ccache_tool(self):
+bitbake(ccache-native)
+
self.assertTrue(os.path.isfile(os.path.join(get_bb_var('STAGING_BINDIR_NATIVE', 
'ccache-native'), ccache)))
+self.write_config('INHERIT += ccache')
+bitbake(m4 -c cleansstate)
+bitbake(m4 -c compile)
+res = runCmd(grep ccache %s % 
(os.path.join(get_bb_var(WORKDIR,m4),temp/log.do_compile)), 
ignore_status=True)
+self.assertEqual(0, res.status, msg=No match for ccache in m4 
log.do_compile)
+bitbake(ccache-native -ccleansstate)
+
+
+class DiskMonTest(oeSelfTest):
+
+def test_stoptask_behavior(self):
+result = runCmd(df -k %s % os.getcwd())
+size = result.output.split(\n)[1].split()[3]
+self.append_config('BB_DISKMON_DIRS = STOPTASKS,${TMPDIR},%sK,4510K' 
% size)
+res = bitbake(core-image-minimal, ignore_status = True)
+self.assertTrue('ERROR: No new tasks can be executed since the disk 
space monitor action is STOPTASKS!' in res.output)
+self.assertEqual(res.status, 1)
+self.append_config('BB_DISKMON_DIRS = ABORT,${TMPDIR},%sK,4510K' % 
size)
+res = bitbake(core-image-minimal, ignore_status = True)
+self.assertTrue('ERROR: Immediately abort since the disk space monitor 
action is ABORT!' in res.output)
+self.assertEqual(res.status, 1)
+self.append_config('BB_DISKMON_DIRS = WARN,${TMPDIR},%sK,4510K' % 
size)
+res = bitbake(core-image-minimal)
+self.assertTrue('WARNING: The free space' in res.output)
+
+class SanityOptionsTest(oeSelfTest):
+
+def test_options_warnqa_errorqa_switch(self):
+bitbake(xcursor-transparent-theme -ccleansstate)
+
+if packages-list not in get_bb_var(ERROR_QA):
+self.write_config(ERROR_QA_append = \ packages-list\)
+
+self.write_recipeinc('xcursor-transparent-theme', 'PACKAGES += 
\${PN}-dbg\')
+res = bitbake(xcursor-transparent-theme, ignore_status=True)
+self.delete_recipeinc('xcursor-transparent-theme')
+self.assertTrue(ERROR: QA Issue: xcursor-transparent-theme-dbg is 
listed in PACKAGES multiple times, this leads to packaging errors. in 
res.output)
+self.assertEqual(res.status, 1)
+self.write_recipeinc('xcursor-transparent-theme', 'PACKAGES += 
\${PN}-dbg\')
+self.append_config('ERROR_QA_remove = packages-list')
+self.append_config('WARN_QA_append =  packages-list')
+bitbake(xcursor-transparent-theme)
+bitbake(xcursor-transparent-theme -ccleansstate)
+self.delete_recipeinc('xcursor-transparent-theme')
+
+def test_sanity_userspace_dependency(self):
+self.append_config('WARN_QA_append =  unsafe-references

[OE-core] [PATCH 4/4] lib/oeqa/selftest: add test modules for expected bitbake output and bitbake-layers

2013-11-27 Thread Stefan Stanacar

From: Corneliu Stoicescu corneliux.stoice...@intel.com

Tests for bitbake-layers and expected output for some bitbake options.

Signed-off-by: Corneliu Stoicescu corneliux.stoice...@intel.com
Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/selftest/bblayers.py | 37 +++
 meta/lib/oeqa/selftest/bbtests.py  | 97 ++
 2 files changed, 134 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/bblayers.py
 create mode 100644 meta/lib/oeqa/selftest/bbtests.py

diff --git a/meta/lib/oeqa/selftest/bblayers.py 
b/meta/lib/oeqa/selftest/bblayers.py
new file mode 100644
index 000..52aa4f8
--- /dev/null
+++ b/meta/lib/oeqa/selftest/bblayers.py
@@ -0,0 +1,37 @@
+import unittest
+import os
+import logging
+import re
+import shutil
+
+import oeqa.utils.ftools as ftools
+from oeqa.selftest.base import oeSelfTest
+from oeqa.utils.commands import runCmd
+
+class BitbakeLayers(oeSelfTest):
+
+def test_bitbakelayers_showcrossdepends(self):
+result = runCmd('bitbake-layers show-cross-depends')
+self.assertTrue('aspell' in result.output)
+
+def test_bitbakelayers_showlayers(self):
+result = runCmd('bitbake-layers show_layers')
+self.assertTrue('meta-selftest' in result.output)
+
+def test_bitbakelayers_showappends(self):
+result = runCmd('bitbake-layers show_appends')
+self.assertTrue('xcursor-transparent-theme_0.1.1.bbappend' in 
result.output, msg='xcursor-transparent-theme_0.1.1.bbappend file was not 
recognised')
+
+def test_bitbakelayers_showoverlayed(self):
+result = runCmd('bitbake-layers show_overlayed')
+self.assertTrue('aspell' in result.output, 
msg='xcursor-transparent-theme_0.1.1.bbappend file was not recognised')
+
+def test_bitbakelayers_flatten(self):
+self.assertFalse(os.path.isdir(os.path.join(self.builddir, 'test')))
+result = runCmd('bitbake-layers flatten test')
+bb_file = os.path.join(self.builddir, 
'test/recipes-graphics/xcursor-transparent-theme/xcursor-transparent-theme_0.1.1.bb')
+self.assertTrue(os.path.isfile(bb_file))
+contents = ftools.read_file(bb_file)
+find_in_contents = re.search(# bbappended from meta-selftest 
#\n(.*\n)*include test_recipe.inc, contents)
+shutil.rmtree(os.path.join(self.builddir, 'test'))
+self.assertTrue(find_in_contents)
diff --git a/meta/lib/oeqa/selftest/bbtests.py 
b/meta/lib/oeqa/selftest/bbtests.py
new file mode 100644
index 000..01e0099
--- /dev/null
+++ b/meta/lib/oeqa/selftest/bbtests.py
@@ -0,0 +1,97 @@
+import unittest
+import os
+import logging
+import re
+import shutil
+
+import oeqa.utils.ftools as ftools
+from oeqa.selftest.base import oeSelfTest
+from oeqa.utils.commands import runCmd, bitbake, get_bb_var
+
+class BitbakeTests(oeSelfTest):
+
+def test_run_bitbake_from_dir_1(self):
+os.chdir(os.path.join(self.builddir, 'conf'))
+bitbake('-e')
+
+def test_run_bitbake_from_dir_2(self):
+my_env = os.environ.copy()
+my_env['BBPATH'] = my_env['BUILDDIR']
+os.chdir(os.path.dirname(os.environ['BUILDDIR']))
+bitbake('-e', env=my_env)
+
+def test_event_handler(self):
+self.write_config(INHERIT += \test_events\)
+result = bitbake('m4-native')
+find_build_started = re.search(NOTE: Test for 
bb\.event\.BuildStarted(\n.*)*NOTE: Preparing runqueue, result.output)
+find_build_completed = re.search(Tasks Summary:.*(\n.*)*NOTE: Test 
for bb\.event\.BuildCompleted, result.output)
+self.assertTrue(find_build_started, msg = Match failed in:\n%s  % 
result.output)
+self.assertTrue(find_build_completed, msg = Match failed in:\n%s % 
result.output)
+self.assertFalse('Test for bb.event.InvalidEvent' in result.output)
+
+def test_local_sstate(self):
+bitbake('m4-native -ccleansstate')
+bitbake('m4-native')
+bitbake('m4-native -cclean')
+result = bitbake('m4-native')
+find_setscene = re.search(m4-native.*do_.*_setscene, result.output)
+self.assertTrue(find_setscene)
+
+def test_bitbake_invalid_recipe(self):
+result = bitbake('-b asdf', ignore_status=True)
+self.assertTrue(ERROR: Unable to find any recipe file matching 
'asdf' in result.output)
+
+def test_bitbake_invalid_target(self):
+result = bitbake('asdf', ignore_status=True)
+self.assertTrue(ERROR: Nothing PROVIDES 'asdf' in result.output)
+
+def test_warnings_errors(self):
+result = bitbake('-b asdf', ignore_status=True)
+find_warnings = re.search(Summary: There was [1-9][0-9]* WARNING 
message shown., result.output)
+find_errors = re.search(Summary: There was [1-9][0-9]* ERROR message 
shown., result.output)
+self.assertTrue(find_warnings)
+self.assertTrue(find_errors)
+
+def test_invalid_patch(self):
+self.write_recipeinc('man

[OE-core] [PATCH 0/4] Script and layer for running tests

2013-11-27 Thread Stefan Stanacar


Hello,

This series adds an oe-selftest script, some modules and a new layer 
meta-selftest.
which are meant to help in writing tests (using python unittest) for various
bitbake tools/scripts as well as simple output checks or do
complete builds with different options (with the emphasis that everything 
checked
is done outside of bitbake context, just like a human would do.) For more 
details,
please see YOCTO #4740.

Cheers,
Stefan

Changes since the RFC:
 - more tests added
 - tests are now in meta/lib/oeqa/selftest instead of scripts/lib/selftest/tests
 - small changes to some of the files in the meta-selftest layer
 - more cleanups done by default in the base class


The following changes since commit 32adaac34a614d106e6dd3e9f1130f4e94ff39ae:

  libpng: set reasonable SUMMARY (2013-11-27 11:51:25 +)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib stefans/selftest
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=stefans/selftest

Alexandru Palalau (1):
  lib/oeqa/selftest: buildoptions.py: add simple image build tests

Corneliu Stoicescu (2):
  meta-selftest: create a new test layer to be used by oe-selftest
script
  lib/oeqa/selftest: add test modules for expected bitbake output and
bitbake-layers

Stefan Stanacar (1):
  scripts/oe-selftest: script to run builds as unittest against bitbake
or various scripts

 .gitignore |   1 +
 meta-selftest/COPYING.MIT  |  17 +++
 meta-selftest/README   |   3 +
 meta-selftest/classes/test_events.bbclass  |  16 +++
 meta-selftest/conf/layer.conf  |  10 ++
 .../recipes-test/aspell/aspell_0.0.0.1.bb  |  28 
 .../recipes-test/aspell/aspell_0.60.6.1.bbappend   |   2 +
 meta-selftest/recipes-test/m4/m4_1.4.17.bbappend   |   2 +
 .../recipes-test/man/man/man-1.5h1-make.patch  |  16 +++
 meta-selftest/recipes-test/man/man_1.6g.bbappend   |   2 +
 .../xcursor-transparent-theme_0.1.1.bbappend   |   2 +
 meta/lib/oeqa/selftest/__init__.py |   2 +
 meta/lib/oeqa/selftest/base.py |  98 ++
 meta/lib/oeqa/selftest/bblayers.py |  37 ++
 meta/lib/oeqa/selftest/bbtests.py  |  97 ++
 meta/lib/oeqa/selftest/buildoptions.py |  86 
 meta/lib/oeqa/utils/commands.py| 137 +++
 meta/lib/oeqa/utils/ftools.py  |  27 
 scripts/oe-selftest| 148 +
 19 files changed, 731 insertions(+)
 create mode 100644 meta-selftest/COPYING.MIT
 create mode 100644 meta-selftest/README
 create mode 100644 meta-selftest/classes/test_events.bbclass
 create mode 100644 meta-selftest/conf/layer.conf
 create mode 100644 meta-selftest/recipes-test/aspell/aspell_0.0.0.1.bb
 create mode 100644 meta-selftest/recipes-test/aspell/aspell_0.60.6.1.bbappend
 create mode 100644 meta-selftest/recipes-test/m4/m4_1.4.17.bbappend
 create mode 100644 meta-selftest/recipes-test/man/man/man-1.5h1-make.patch
 create mode 100644 meta-selftest/recipes-test/man/man_1.6g.bbappend
 create mode 100644 
meta-selftest/recipes-test/xcursor-transparent-theme/xcursor-transparent-theme_0.1.1.bbappend
 create mode 100644 meta/lib/oeqa/selftest/__init__.py
 create mode 100644 meta/lib/oeqa/selftest/base.py
 create mode 100644 meta/lib/oeqa/selftest/bblayers.py
 create mode 100644 meta/lib/oeqa/selftest/bbtests.py
 create mode 100644 meta/lib/oeqa/selftest/buildoptions.py
 create mode 100644 meta/lib/oeqa/utils/commands.py
 create mode 100644 meta/lib/oeqa/utils/ftools.py
 create mode 100755 scripts/oe-selftest

-- 
1.8.3.1

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/4] meta-selftest: create a new test layer to be used by oe-selftest script

2013-11-27 Thread Stefan Stanacar

From: Corneliu Stoicescu corneliux.stoice...@intel.com

Everything in this layer is meant to be used by tests called by
scripts/oe-selftest. These are helper recipes/appends to test various bitbake
options or scripts.
Currently most of these files here only have include test_recipe.inc which
is the file tests will actually use.

Signed-off-by: Corneliu Stoicescu corneliux.stoice...@intel.com
Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 .gitignore |  1 +
 meta-selftest/COPYING.MIT  | 17 +
 meta-selftest/README   |  3 +++
 meta-selftest/classes/test_events.bbclass  | 16 +
 meta-selftest/conf/layer.conf  | 10 
 .../recipes-test/aspell/aspell_0.0.0.1.bb  | 28 ++
 .../recipes-test/aspell/aspell_0.60.6.1.bbappend   |  2 ++
 meta-selftest/recipes-test/m4/m4_1.4.17.bbappend   |  2 ++
 .../recipes-test/man/man/man-1.5h1-make.patch  | 16 +
 meta-selftest/recipes-test/man/man_1.6g.bbappend   |  2 ++
 .../xcursor-transparent-theme_0.1.1.bbappend   |  2 ++
 11 files changed, 99 insertions(+)
 create mode 100644 meta-selftest/COPYING.MIT
 create mode 100644 meta-selftest/README
 create mode 100644 meta-selftest/classes/test_events.bbclass
 create mode 100644 meta-selftest/conf/layer.conf
 create mode 100644 meta-selftest/recipes-test/aspell/aspell_0.0.0.1.bb
 create mode 100644 meta-selftest/recipes-test/aspell/aspell_0.60.6.1.bbappend
 create mode 100644 meta-selftest/recipes-test/m4/m4_1.4.17.bbappend
 create mode 100644 meta-selftest/recipes-test/man/man/man-1.5h1-make.patch
 create mode 100644 meta-selftest/recipes-test/man/man_1.6g.bbappend
 create mode 100644 
meta-selftest/recipes-test/xcursor-transparent-theme/xcursor-transparent-theme_0.1.1.bbappend

diff --git a/.gitignore b/.gitignore
index 0171597..b6755fe 100644
--- a/.gitignore
+++ b/.gitignore
@@ -21,3 +21,4 @@ bitbake/doc/manual/pdf/
 bitbake/doc/manual/txt/
 bitbake/doc/manual/xhtml/
 pull-*/
+!meta-selftest
diff --git a/meta-selftest/COPYING.MIT b/meta-selftest/COPYING.MIT
new file mode 100644
index 000..89de354
--- /dev/null
+++ b/meta-selftest/COPYING.MIT
@@ -0,0 +1,17 @@
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the Software), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED AS IS, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.
diff --git a/meta-selftest/README b/meta-selftest/README
new file mode 100644
index 000..11a6fee
--- /dev/null
+++ b/meta-selftest/README
@@ -0,0 +1,3 @@
+This layer is intended as test layer, used by scripts/oe-selftest
+and it's probably a mistake to include it in your builds (unless you
+want to run the script).
diff --git a/meta-selftest/classes/test_events.bbclass 
b/meta-selftest/classes/test_events.bbclass
new file mode 100644
index 000..35324eb
--- /dev/null
+++ b/meta-selftest/classes/test_events.bbclass
@@ -0,0 +1,16 @@
+python test1_eventhandler() {
+bb.note(Test for bb.event.BuildStarted)
+}
+python test2_eventhandler() {
+bb.note(Test for bb.event.BuildCompleted)
+}
+python test3_eventhandler() {
+bb.note(Test for bb.event.InvalidEvent)
+}
+
+addhandler test1_eventhandler
+test1_eventhandler[eventmask] = bb.event.BuildStarted
+addhandler test2_eventhandler
+test2_eventhandler[eventmask] = bb.event.BuildCompleted
+addhandler test3_eventhandler
+test3_eventhandler[eventmask] = bb.event.InvalidEvent
diff --git a/meta-selftest/conf/layer.conf b/meta-selftest/conf/layer.conf
new file mode 100644
index 000..a847b78
--- /dev/null
+++ b/meta-selftest/conf/layer.conf
@@ -0,0 +1,10 @@
+# We have a conf and classes directory, add to BBPATH
+BBPATH .= :${LAYERDIR}
+
+# We have recipes-* directories, add to BBFILES
+BBFILES += ${LAYERDIR}/recipes-*/*/*.bb \
+   ${LAYERDIR}/recipes-*/*/*.bbappend
+
+BBFILE_COLLECTIONS += selftest
+BBFILE_PATTERN_selftest = ^${LAYERDIR}/
+BBFILE_PRIORITY_selftest = 5
diff --git a/meta-selftest/recipes-test/aspell/aspell_0.0.0.1.bb 
b/meta-selftest/recipes-test/aspell/aspell_0.0.0.1.bb
new

[OE-core] [PATCH v2 1/2] lib/oeqa: targetcontrol.py: add abstraction for running tests on different targets

2013-11-26 Thread Stefan Stanacar

Add a new module which abstracts the target object used by testimage.bbclass

The purpose of this module is to move the deployment of a target from 
testimage.bbclass,
basically abstracting different implementations of how we setup a target and 
how it runs commands.
It allows to select one implementation or another by setting TEST_TARGET 
(currently to: qemu and simpleremote).

QemuTarget is used to start a qemu instance (as it's currently done in 
testimage.bbclass)
SimpleRemoteTarget is meant for a remote machine (by setting TEST_TARGET_IP) 
that's already up and running
with network and ssh.
Simply put, it opens the door for running the tests on different types of 
targets by adding new classes
(maybe qemu-nfsroot or remote-special etc.). One could also override BaseTarget 
which currently uses
the existing SSHControl module and add a serial implementation.

[ YOCTO #5554 ]

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/targetcontrol.py | 137 +
 1 file changed, 137 insertions(+)
 create mode 100644 meta/lib/oeqa/targetcontrol.py

diff --git a/meta/lib/oeqa/targetcontrol.py b/meta/lib/oeqa/targetcontrol.py
new file mode 100644
index 000..dee38ec
--- /dev/null
+++ b/meta/lib/oeqa/targetcontrol.py
@@ -0,0 +1,137 @@
+# Copyright (C) 2013 Intel Corporation
+#
+# Released under the MIT license (see COPYING.MIT)
+
+# This module is used by testimage.bbclass for setting up and controlling a 
target machine.
+
+import os
+import shutil
+import subprocess
+import bb
+
+from oeqa.utils.sshcontrol import SSHControl
+from oeqa.utils.qemurunner import QemuRunner
+
+
+def get_target_controller(d):
+if d.getVar(TEST_TARGET, True) == qemu:
+return QemuTarget(d)
+elif d.getVar(TEST_TARGET, True) == simpleremote:
+return SimpleRemoteTarget(d)
+else:
+bb.fatal(Please set a valid TEST_TARGET)
+
+
+class BaseTarget(object):
+
+def __init__(self, d):
+self.connection = None
+self.ip = None
+self.server_ip = None
+self.datetime = d.getVar('DATETIME', True)
+self.testdir = d.getVar(TEST_LOG_DIR, True)
+self.pn = d.getVar(PN, True)
+
+def deploy(self):
+
+self.sshlog = os.path.join(self.testdir, ssh_target_log.%s % 
self.datetime)
+sshloglink = os.path.join(self.testdir, ssh_target_log)
+if os.path.islink(sshloglink):
+os.unlink(sshloglink)
+os.symlink(self.sshlog, sshloglink)
+bb.note(SSH log file: %s %  self.sshlog)
+
+def run(self, cmd, timeout=None):
+return self.connection.run(cmd, timeout)
+
+def copy_to(self, localpath, remotepath):
+return self.connection.copy_to(localpath, remotepath)
+
+def copy_from(self, remotepath, localpath):
+return self.connection.copy_from(remotepath, localpath)
+
+
+
+class QemuTarget(BaseTarget):
+
+def __init__(self, d):
+
+super(QemuTarget, self).__init__(d)
+
+self.qemulog = os.path.join(self.testdir, qemu_boot_log.%s % 
self.datetime)
+self.origrootfs = os.path.join(d.getVar(DEPLOY_DIR_IMAGE, True),  
d.getVar(IMAGE_LINK_NAME, True) + '.ext3')
+self.rootfs = os.path.join(self.testdir, d.getVar(IMAGE_LINK_NAME, 
True) + '-testimage.ext3')
+
+self.runner = QemuRunner(machine=d.getVar(MACHINE, True),
+rootfs=self.rootfs,
+tmpdir = d.getVar(TMPDIR, True),
+deploy_dir_image = d.getVar(DEPLOY_DIR_IMAGE, True),
+display = d.getVar(BB_ORIGENV, 
False).getVar(DISPLAY, True),
+logfile = self.qemulog,
+boottime = int(d.getVar(TEST_QEMUBOOT_TIMEOUT, 
True)))
+
+def deploy(self):
+try:
+shutil.copyfile(self.origrootfs, self.rootfs)
+except Exception as e:
+bb.fatal(Error copying rootfs: %s % e)
+
+qemuloglink = os.path.join(self.testdir, qemu_boot_log)
+if os.path.islink(qemuloglink):
+os.unlink(qemuloglink)
+os.symlink(self.qemulog, qemuloglink)
+
+bb.note(rootfs file: %s %  self.rootfs)
+bb.note(Qemu log file: %s % self.qemulog)
+super(QemuTarget, self).deploy()
+
+def start(self, params=None):
+if self.runner.start(params):
+self.ip = self.runner.ip
+self.server_ip = self.runner.server_ip
+self.connection = SSHControl(ip=self.ip, logfile=self.sshlog)
+else:
+raise bb.build.FuncFailed(%s - FAILED to start qemu - check the 
task log and the boot log % self.pn)
+
+def stop(self):
+self.runner.stop()
+self.connection = None
+self.ip = None
+self.server_ip = None
+
+def restart(self, params=None):
+if self.runner.restart(params):
+self.ip = self.runner.ip
+self.server_ip = self.runner.server_ip
+self.connection

[OE-core] [PATCH v2 0/2] Run tests on a remote machine

2013-11-26 Thread Stefan Stanacar

Hello,

These patches add some abstraction on how we control a target, so that we can 
also
run the tests on real hardware.
The backend is controlled by setting TEST_TARGET to qemu or simpleremote. 
(a very simple
remote implementation which does not deploy the image but runs the tests on a 
remote machine
via ssh to a specified IP address).

Cheers,
Stefan

Changed in v2:
  - yay, typos: FunFailed - FuncFailed (there's a gremlin in my computer :))
  - small changes in commit messages
  - one try/except block in targetcontrol.py was too big, scale it down.


The following changes since commit 381cd842caa3db5eabf8973305b6979014d9d0a8:

  linux-firmware: add missing linux-firmware-iwlwifi-7260-7 package (2013-11-25 
16:14:48 +)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib stefans/hwtest3
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=stefans/hwtest3

Stefan Stanacar (2):
  lib/oeqa: targetcontrol.py: add abstraction for running tests on
different targets
  testimage: use the new targetcontrol.py module for running tests

 meta/classes/testimage-auto.bbclass |   3 +-
 meta/classes/testimage.bbclass  | 104 ++-
 meta/lib/oeqa/oetest.py |  20 +++---
 meta/lib/oeqa/runtime/ping.py   |   2 +-
 meta/lib/oeqa/runtime/smart.py  |   4 +-
 meta/lib/oeqa/targetcontrol.py  | 137 
 meta/lib/oeqa/utils/qemurunner.py   |  50 +++--
 meta/lib/oeqa/utils/sshcontrol.py   |  20 ++
 8 files changed, 224 insertions(+), 116 deletions(-)
 create mode 100644 meta/lib/oeqa/targetcontrol.py

-- 
1.8.3.1

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 2/2] testimage: use the new targetcontrol.py module for running tests

2013-11-26 Thread Stefan Stanacar

This patch makes the necessary changes for using the targetcontrol.py module
so that one can run the same tests on a qemu instance or a remote machine
based on the value of TEST_TARGET variable: qemu or simpleremote.
The default value is qemu which starts a qemu instance and it's the
with what we currently have.

With simpleremote, the remote machine must be up with network and ssh
and you need to set TEST_TARGET_IP with the IP address of the remote machine
(it can still be a qemu instance that was manually started).

Basically testimage.bbclass now does something along the lines of:
 - load tests - deploy (prepare) / start target - run tests.
There were a couple of changes necessary for tests and
also some cleanups/renames that were needed to adjust this change. (use
ip everywhere when refering to target and server_ip when refering to host/build 
machine)
Also two unnecessary and unsed methods were dropped from sshcontrol.

[ YOCTO #5554 ]

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/classes/testimage-auto.bbclass |   3 +-
 meta/classes/testimage.bbclass  | 104 +---
 meta/lib/oeqa/oetest.py |  20 +++
 meta/lib/oeqa/runtime/ping.py   |   2 +-
 meta/lib/oeqa/runtime/smart.py  |   4 +-
 meta/lib/oeqa/utils/qemurunner.py   |  50 ++---
 meta/lib/oeqa/utils/sshcontrol.py   |  20 +++
 7 files changed, 87 insertions(+), 116 deletions(-)

diff --git a/meta/classes/testimage-auto.bbclass 
b/meta/classes/testimage-auto.bbclass
index 3d0e289..a5b8f7f 100644
--- a/meta/classes/testimage-auto.bbclass
+++ b/meta/classes/testimage-auto.bbclass
@@ -19,5 +19,4 @@ python do_testimage_auto() {
 testimage_main(d)
 }
 addtask testimage_auto before do_build after do_rootfs
-do_testimage_auto[depends] += qemu-native:do_populate_sysroot
-do_testimage_auto[depends] += qemu-helper-native:do_populate_sysroot
+do_testimage_auto[depends] += ${TESTIMAGEDEPENDS}
diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 5d61c2b..add8009 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -34,13 +34,19 @@ TEST_SUITES ?= ${DEFAULT_TEST_SUITES}
 
 TEST_QEMUBOOT_TIMEOUT ?= 1000
 
+TEST_TARGET ?= qemu
+TEST_TARGET_IP ?= 
+TEST_SERVER_IP ?= 
+
+TESTIMAGEDEPENDS = 
+TESTIMAGEDEPENDS_qemuall = qemu-native:do_populate_sysroot 
qemu-helper-native:do_populate_sysroot
+
 python do_testimage() {
 testimage_main(d)
 }
 addtask testimage
 do_testimage[nostamp] = 1
-do_testimage[depends] += qemu-native:do_populate_sysroot
-do_testimage[depends] += qemu-helper-native:do_populate_sysroot
+do_testimage[depends] += ${TESTIMAGEDEPENDS}
 
 
 def get_tests_list(d):
@@ -83,15 +89,12 @@ def testimage_main(d):
 import unittest
 import os
 import oeqa.runtime
-import re
-import shutil
 import time
-from oeqa.oetest import runTests
-from oeqa.utils.sshcontrol import SSHControl
-from oeqa.utils.qemurunner import QemuRunner
+from oeqa.oetest import loadTests, runTests
+from oeqa.targetcontrol import get_target_controller
 
-testdir = d.getVar(TEST_LOG_DIR, True)
-bb.utils.mkdirhier(testdir)
+pn = d.getVar(PN, True)
+bb.utils.mkdirhier(d.getVar(TEST_LOG_DIR, True))
 
 # tests in TEST_SUITES become required tests
 # they won't be skipped even if they aren't suitable for a image (like 
xorg for minimal)
@@ -99,81 +102,46 @@ def testimage_main(d):
 testslist = get_tests_list(d)
 testsrequired = [t for t in d.getVar(TEST_SUITES, True).split() if t != 
auto]
 
+# the robot dance
+target = get_target_controller(d)
+
 class TestContext:
 def __init__(self):
 self.d = d
 self.testslist = testslist
 self.testsrequired = testsrequired
 self.filesdir = 
os.path.join(os.path.dirname(os.path.abspath(oeqa.runtime.__file__)),files)
+self.target = target
 
 # test context
 tc = TestContext()
 
-# prepare qemu instance
-# and boot each supported fs type
-machine=d.getVar(MACHINE, True)
-#will handle fs type eventually, stick with ext3 for now
-#make a copy of the original rootfs and use that for tests
-origrootfs=os.path.join(d.getVar(DEPLOY_DIR_IMAGE, True),  
d.getVar(IMAGE_LINK_NAME,True) + '.ext3')
-testrootfs=os.path.join(testdir, d.getVar(IMAGE_LINK_NAME, True) + 
'-testimage.ext3')
+# this is a dummy load of tests
+# we are doing that to find compile errors in the tests themselves
+# before booting the image
 try:
-shutil.copyfile(origrootfs, testrootfs)
+loadTests(tc)
 except Exception as e:
-bb.fatal(Error copying rootfs: %s % e)
+bb.fatal(Loading tests failed:\n %s % e)
 
-try:
-boottime = int(d.getVar(TEST_QEMUBOOT_TIMEOUT, True))
-except ValueError:
-boottime = 1000
-
-qemu = QemuRunner(machine=machine, rootfs=testrootfs

[OE-core] [PATCH 2/2] testimage: use the new targetcontrol.py module for running tests

2013-11-25 Thread Stefan Stanacar

This patch makes the necessary changes for using the targetcontrol.py module
so that one can run the same tests on a qemu instance or a remote machine
based on the value of TEST_TARGET variable: qemu or simpleremote.

When using qemu nothing really changes (does what it does currently).
With simpleremote, the remote machine must be up with network and ssh
and you need to set TEST_TARGET_IP with the IP address of the remote machine
(it can still be a qemu instance that was manually started).

Basically testimage.bbclass now does something along the lines of:
 - load tests - deploy (prepare) / start target - run tests.
There were a couple of changes necessary for tests to adjust this change and
also some cleanups/renames that were needed to fit in the picture. (use
ip everywhere when refering to target and server_ip when refering to host/build 
machine)
Also two unnecessary and unsed methods were dropped from sshcontrol.

[ YOCTO #5554 ]

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/classes/testimage-auto.bbclass |   3 +-
 meta/classes/testimage.bbclass  | 104 +---
 meta/lib/oeqa/oetest.py |  20 +++
 meta/lib/oeqa/runtime/ping.py   |   2 +-
 meta/lib/oeqa/runtime/smart.py  |   4 +-
 meta/lib/oeqa/utils/qemurunner.py   |  50 ++---
 meta/lib/oeqa/utils/sshcontrol.py   |  20 +++
 7 files changed, 87 insertions(+), 116 deletions(-)

diff --git a/meta/classes/testimage-auto.bbclass 
b/meta/classes/testimage-auto.bbclass
index 3d0e289..a5b8f7f 100644
--- a/meta/classes/testimage-auto.bbclass
+++ b/meta/classes/testimage-auto.bbclass
@@ -19,5 +19,4 @@ python do_testimage_auto() {
 testimage_main(d)
 }
 addtask testimage_auto before do_build after do_rootfs
-do_testimage_auto[depends] += qemu-native:do_populate_sysroot
-do_testimage_auto[depends] += qemu-helper-native:do_populate_sysroot
+do_testimage_auto[depends] += ${TESTIMAGEDEPENDS}
diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 5d61c2b..add8009 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -34,13 +34,19 @@ TEST_SUITES ?= ${DEFAULT_TEST_SUITES}
 
 TEST_QEMUBOOT_TIMEOUT ?= 1000
 
+TEST_TARGET ?= qemu
+TEST_TARGET_IP ?= 
+TEST_SERVER_IP ?= 
+
+TESTIMAGEDEPENDS = 
+TESTIMAGEDEPENDS_qemuall = qemu-native:do_populate_sysroot 
qemu-helper-native:do_populate_sysroot
+
 python do_testimage() {
 testimage_main(d)
 }
 addtask testimage
 do_testimage[nostamp] = 1
-do_testimage[depends] += qemu-native:do_populate_sysroot
-do_testimage[depends] += qemu-helper-native:do_populate_sysroot
+do_testimage[depends] += ${TESTIMAGEDEPENDS}
 
 
 def get_tests_list(d):
@@ -83,15 +89,12 @@ def testimage_main(d):
 import unittest
 import os
 import oeqa.runtime
-import re
-import shutil
 import time
-from oeqa.oetest import runTests
-from oeqa.utils.sshcontrol import SSHControl
-from oeqa.utils.qemurunner import QemuRunner
+from oeqa.oetest import loadTests, runTests
+from oeqa.targetcontrol import get_target_controller
 
-testdir = d.getVar(TEST_LOG_DIR, True)
-bb.utils.mkdirhier(testdir)
+pn = d.getVar(PN, True)
+bb.utils.mkdirhier(d.getVar(TEST_LOG_DIR, True))
 
 # tests in TEST_SUITES become required tests
 # they won't be skipped even if they aren't suitable for a image (like 
xorg for minimal)
@@ -99,81 +102,46 @@ def testimage_main(d):
 testslist = get_tests_list(d)
 testsrequired = [t for t in d.getVar(TEST_SUITES, True).split() if t != 
auto]
 
+# the robot dance
+target = get_target_controller(d)
+
 class TestContext:
 def __init__(self):
 self.d = d
 self.testslist = testslist
 self.testsrequired = testsrequired
 self.filesdir = 
os.path.join(os.path.dirname(os.path.abspath(oeqa.runtime.__file__)),files)
+self.target = target
 
 # test context
 tc = TestContext()
 
-# prepare qemu instance
-# and boot each supported fs type
-machine=d.getVar(MACHINE, True)
-#will handle fs type eventually, stick with ext3 for now
-#make a copy of the original rootfs and use that for tests
-origrootfs=os.path.join(d.getVar(DEPLOY_DIR_IMAGE, True),  
d.getVar(IMAGE_LINK_NAME,True) + '.ext3')
-testrootfs=os.path.join(testdir, d.getVar(IMAGE_LINK_NAME, True) + 
'-testimage.ext3')
+# this is a dummy load of tests
+# we are doing that to find compile errors in the tests themselves
+# before booting the image
 try:
-shutil.copyfile(origrootfs, testrootfs)
+loadTests(tc)
 except Exception as e:
-bb.fatal(Error copying rootfs: %s % e)
+bb.fatal(Loading tests failed:\n %s % e)
 
-try:
-boottime = int(d.getVar(TEST_QEMUBOOT_TIMEOUT, True))
-except ValueError:
-boottime = 1000
-
-qemu = QemuRunner(machine=machine, rootfs=testrootfs

[OE-core] [PATCH 1/2] lib/oeqa: targetcontrol.py: add abstraction for running tests on different targets

2013-11-25 Thread Stefan Stanacar

Add a new module which abstracts the target object used by testimage.bbclass

The purpose of this module is to move the deployment of a target from 
testimage.bbclass,
basically abstracting different implementations of how we setup a target and 
how it runs commands.
It allows to select one implementation or another by setting TEST_TARGET 
(currently to: qemu and simpleremote).

QemuTarget class does what it's currently done in testimage.bbclass
SimpleRemoteTarget is meant for a remote machine (by setting TEST_TARGET_IP) 
that's already up and running
with network and ssh.
Simply put, it opens the door for running the tests on different types of 
targets by adding new classes
(maybe qemu-nfsroot or remote-special etc.). One could also override BaseTarget 
which currently uses
the existing SSHControl module and add a serial implementation.

[ YOCTO #5554 ]

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/targetcontrol.py | 137 +
 1 file changed, 137 insertions(+)
 create mode 100644 meta/lib/oeqa/targetcontrol.py

diff --git a/meta/lib/oeqa/targetcontrol.py b/meta/lib/oeqa/targetcontrol.py
new file mode 100644
index 000..e1c855b
--- /dev/null
+++ b/meta/lib/oeqa/targetcontrol.py
@@ -0,0 +1,137 @@
+# Copyright (C) 2013 Intel Corporation
+#
+# Released under the MIT license (see COPYING.MIT)
+
+# This module is used by testimage.bbclass for setting up and controlling a 
target machine.
+
+import os
+import shutil
+import subprocess
+import bb
+
+from oeqa.utils.sshcontrol import SSHControl
+from oeqa.utils.qemurunner import QemuRunner
+
+
+def get_target_controller(d):
+if d.getVar(TEST_TARGET, True) == qemu:
+return QemuTarget(d)
+elif d.getVar(TEST_TARGET, True) == simpleremote:
+return SimpleRemoteTarget(d)
+else:
+bb.fatal(Please set a valid TEST_TARGET)
+
+
+class BaseTarget(object):
+
+def __init__(self, d):
+self.connection = None
+self.ip = None
+self.server_ip = None
+self.datetime = d.getVar('DATETIME', True)
+self.testdir = d.getVar(TEST_LOG_DIR, True)
+self.pn = d.getVar(PN, True)
+
+def deploy(self):
+
+self.sshlog = os.path.join(self.testdir, ssh_target_log.%s % 
self.datetime)
+sshloglink = os.path.join(self.testdir, ssh_target_log)
+if os.path.islink(sshloglink):
+os.unlink(sshloglink)
+os.symlink(self.sshlog, sshloglink)
+bb.note(SSH log file: %s %  self.sshlog)
+
+def run(self, cmd, timeout=None):
+return self.connection.run(cmd, timeout)
+
+def copy_to(self, localpath, remotepath):
+return self.connection.copy_to(localpath, remotepath)
+
+def copy_from(self, remotepath, localpath):
+return self.connection.copy_from(remotepath, localpath)
+
+
+
+class QemuTarget(BaseTarget):
+
+def __init__(self, d):
+
+super(QemuTarget, self).__init__(d)
+
+self.qemulog = os.path.join(self.testdir, qemu_boot_log.%s % 
self.datetime)
+self.origrootfs = os.path.join(d.getVar(DEPLOY_DIR_IMAGE, True),  
d.getVar(IMAGE_LINK_NAME, True) + '.ext3')
+self.rootfs = os.path.join(self.testdir, d.getVar(IMAGE_LINK_NAME, 
True) + '-testimage.ext3')
+
+self.runner = QemuRunner(machine=d.getVar(MACHINE, True),
+rootfs=self.rootfs,
+tmpdir = d.getVar(TMPDIR, True),
+deploy_dir_image = d.getVar(DEPLOY_DIR_IMAGE, True),
+display = d.getVar(BB_ORIGENV, 
False).getVar(DISPLAY, True),
+logfile = self.qemulog,
+boottime = int(d.getVar(TEST_QEMUBOOT_TIMEOUT, 
True)))
+
+def deploy(self):
+try:
+shutil.copyfile(self.origrootfs, self.rootfs)
+except Exception as e:
+bb.fatal(Error copying rootfs: %s % e)
+
+qemuloglink = os.path.join(self.testdir, qemu_boot_log)
+if os.path.islink(qemuloglink):
+os.unlink(qemuloglink)
+os.symlink(self.qemulog, qemuloglink)
+
+bb.note(rootfs file: %s %  self.rootfs)
+bb.note(Qemu log file: %s % self.qemulog)
+super(QemuTarget, self).deploy()
+
+def start(self, params=None):
+if self.runner.start(params):
+self.ip = self.runner.ip
+self.server_ip = self.runner.server_ip
+self.connection = SSHControl(ip=self.ip, logfile=self.sshlog)
+else:
+raise bb.build.FunFailed(%s - FAILED to start qemu - check the 
task log and the boot log % self.pn)
+
+def stop(self):
+self.runner.stop()
+self.connection = None
+self.ip = None
+self.server_ip = None
+
+def restart(self, params=None):
+if self.runner.restart(params):
+self.ip = self.runner.ip
+self.server_ip = self.runner.server_ip
+self.connection = SSHControl(ip=self.ip

[OE-core] [PATCH 0/2] Run tests on a remote machine

2013-11-25 Thread Stefan Stanacar

Hello,

These patches add some abstraction on how we control a target, so that we can 
also
run the tests on real hardware.
The backend is controlled by setting TEST_TARGET to qemu or simpleremote. 
(a very simple
remote implementation which does not deploy the image but runs the tests on a 
remote machine
via ssh to a specified IP address).

Cheers,
Stefan


The following changes since commit 381cd842caa3db5eabf8973305b6979014d9d0a8:

  linux-firmware: add missing linux-firmware-iwlwifi-7260-7 package (2013-11-25 
16:14:48 +)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib stefans/hwtest3
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=stefans/hwtest3

Stefan Stanacar (2):
  lib/oeqa: targetcontrol.py: add abstraction for running tests on
different targets
  testimage: use the new targetcontrol.py module for running tests

 meta/classes/testimage-auto.bbclass |   3 +-
 meta/classes/testimage.bbclass  | 104 ++-
 meta/lib/oeqa/oetest.py |  20 +++---
 meta/lib/oeqa/runtime/ping.py   |   2 +-
 meta/lib/oeqa/runtime/smart.py  |   4 +-
 meta/lib/oeqa/targetcontrol.py  | 137 
 meta/lib/oeqa/utils/qemurunner.py   |  50 +++--
 meta/lib/oeqa/utils/sshcontrol.py   |  20 ++
 8 files changed, 224 insertions(+), 116 deletions(-)
 create mode 100644 meta/lib/oeqa/targetcontrol.py

-- 
1.8.3.1

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [RFC PATCH 2/4] meta-selftest: create a new test layer to be used by oe-selftest script

2013-11-19 Thread Stefan Stanacar

From: Corneliu Stoicescu corneliux.stoice...@intel.com

Everything in this layer is meant to be used by tests.

Signed-off-by: Corneliu Stoicescu corneliux.stoice...@intel.com
Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 .gitignore |  1 +
 meta-selftest/COPYING.MIT  | 17 +
 meta-selftest/README   |  2 ++
 meta-selftest/classes/test_events.bbclass  | 16 +
 meta-selftest/conf/layer.conf  | 10 
 .../recipes-test/aspell/aspell_0.0.0.1.bb  | 28 ++
 .../recipes-test/man/man/man-1.5h1-make.patch  | 16 +
 meta-selftest/recipes-test/man/man_1.6g.bbappend   |  2 ++
 .../xcursor-transparent-theme_0.1.1.bbappend   |  3 +++
 9 files changed, 95 insertions(+)
 create mode 100644 meta-selftest/COPYING.MIT
 create mode 100644 meta-selftest/README
 create mode 100644 meta-selftest/classes/test_events.bbclass
 create mode 100644 meta-selftest/conf/layer.conf
 create mode 100644 meta-selftest/recipes-test/aspell/aspell_0.0.0.1.bb
 create mode 100644 meta-selftest/recipes-test/man/man/man-1.5h1-make.patch
 create mode 100644 meta-selftest/recipes-test/man/man_1.6g.bbappend
 create mode 100644 
meta-selftest/recipes-test/xcursor-transparent-theme/xcursor-transparent-theme_0.1.1.bbappend

diff --git a/.gitignore b/.gitignore
index 0171597..b6755fe 100644
--- a/.gitignore
+++ b/.gitignore
@@ -21,3 +21,4 @@ bitbake/doc/manual/pdf/
 bitbake/doc/manual/txt/
 bitbake/doc/manual/xhtml/
 pull-*/
+!meta-selftest
diff --git a/meta-selftest/COPYING.MIT b/meta-selftest/COPYING.MIT
new file mode 100644
index 000..89de354
--- /dev/null
+++ b/meta-selftest/COPYING.MIT
@@ -0,0 +1,17 @@
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the Software), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED AS IS, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.
diff --git a/meta-selftest/README b/meta-selftest/README
new file mode 100644
index 000..e0f803d
--- /dev/null
+++ b/meta-selftest/README
@@ -0,0 +1,2 @@
+This layer is intended as test layer, used by scripts/oe-selftest
+and it's probably a mistake to include it in your builds.
diff --git a/meta-selftest/classes/test_events.bbclass 
b/meta-selftest/classes/test_events.bbclass
new file mode 100644
index 000..35324eb
--- /dev/null
+++ b/meta-selftest/classes/test_events.bbclass
@@ -0,0 +1,16 @@
+python test1_eventhandler() {
+bb.note(Test for bb.event.BuildStarted)
+}
+python test2_eventhandler() {
+bb.note(Test for bb.event.BuildCompleted)
+}
+python test3_eventhandler() {
+bb.note(Test for bb.event.InvalidEvent)
+}
+
+addhandler test1_eventhandler
+test1_eventhandler[eventmask] = bb.event.BuildStarted
+addhandler test2_eventhandler
+test2_eventhandler[eventmask] = bb.event.BuildCompleted
+addhandler test3_eventhandler
+test3_eventhandler[eventmask] = bb.event.InvalidEvent
diff --git a/meta-selftest/conf/layer.conf b/meta-selftest/conf/layer.conf
new file mode 100644
index 000..e3322f6
--- /dev/null
+++ b/meta-selftest/conf/layer.conf
@@ -0,0 +1,10 @@
+# We have a conf and classes directory, add to BBPATH
+BBPATH .= :${LAYERDIR}
+
+# We have recipes-* directories, add to BBFILES
+BBFILES += ${LAYERDIR}/recipes-*/*/*.bb \
+   ${LAYERDIR}/recipes-*/*/*.bbappend
+
+BBFILE_COLLECTIONS += selftest
+BBFILE_PATTERN_selftest = ^${LAYERDIR}/
+BBFILE_PRIORITY_selftest = 6
diff --git a/meta-selftest/recipes-test/aspell/aspell_0.0.0.1.bb 
b/meta-selftest/recipes-test/aspell/aspell_0.0.0.1.bb
new file mode 100644
index 000..79eb06b
--- /dev/null
+++ b/meta-selftest/recipes-test/aspell/aspell_0.0.0.1.bb
@@ -0,0 +1,28 @@
+# This recipe is a copy from the oe-core one.
+# It has a lower and invalid version number in order not to be accidentally 
used by bitbake.
+# It is used for tests that require duplicate recipe files.
+
+DESCRIPTION = GNU Aspell spell-checker
+SECTION = console/utils
+LICENSE=LGPLv2 | LGPLv2.1
+LIC_FILES_CHKSUM = file://COPYING;md5=7fbc338309ac38fefcd64b04bb903e34
+PR = r1
+
+SRC_URI

[OE-core] [RFC PATCH 3/4] scripts/lib/selftest: add a new module for bitbake output tests

2013-11-19 Thread Stefan Stanacar

From: Corneliu Stoicescu corneliux.stoice...@intel.com

Tests for bitbake-layers and expected output for some bitbake options.

Signed-off-by: Corneliu Stoicescu corneliux.stoice...@intel.com
Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 scripts/lib/selftest/tests/__init__.py |  1 +
 scripts/lib/selftest/tests/bboutput.py | 86 ++
 2 files changed, 87 insertions(+)
 create mode 100644 scripts/lib/selftest/tests/bboutput.py

diff --git a/scripts/lib/selftest/tests/__init__.py 
b/scripts/lib/selftest/tests/__init__.py
index e69de29..3acc480 100644
--- a/scripts/lib/selftest/tests/__init__.py
+++ b/scripts/lib/selftest/tests/__init__.py
@@ -0,0 +1 @@
+from selftest.tests.bboutput import *
diff --git a/scripts/lib/selftest/tests/bboutput.py 
b/scripts/lib/selftest/tests/bboutput.py
new file mode 100644
index 000..386572f
--- /dev/null
+++ b/scripts/lib/selftest/tests/bboutput.py
@@ -0,0 +1,86 @@
+import unittest
+import os
+import logging
+import re
+import shutil
+from selftest.base import oeSelfTest
+from selftest.utils.commands import runCmd, bitbake, get_bb_var, get_test_layer
+import selftest.utils.ftools as ftools
+
+class BitbakeTests(oeSelfTest):
+
+def test_run_bitbake_from_1(self):
+os.chdir(os.path.join(self.builddir, 'conf'))
+bitbake('-e')
+
+def test_run_bitbake_from_2(self):
+my_env = os.environ.copy()
+my_env['BBPATH'] = my_env['BUILDDIR']
+os.chdir(os.path.dirname(os.environ['BUILDDIR']))
+bitbake('-e', env=my_env)
+
+def test_event_handler(self):
+self.write_config(INHERIT += \test_events\)
+result = bitbake('m4-native')
+find_build_started = re.search(NOTE: Test for 
bb\.event\.BuildStarted(\n.*)*NOTE: Preparing runqueue, result.output)
+find_build_completed = re.search(Tasks Summary:.*(\n.*)*NOTE: Test 
for bb\.event\.BuildCompleted, result.output)
+self.assertTrue(find_build_started, msg = Match failed in:\n%s  % 
result.output)
+self.assertTrue(find_build_completed, msg = Match failed in:\n%s % 
result.output)
+self.assertFalse('Test for bb.event.InvalidEvent' in result.output)
+
+def test_local_sstate(self):
+bitbake('m4-native -ccleansstate')
+bitbake('m4-native')
+bitbake('m4-native -cclean')
+result = bitbake('m4-native')
+find_setscene = re.search(m4-native.*do_.*_setscene, result.output)
+self.assertTrue(find_setscene)
+
+def test_bitbake_invalid_recipe(self):
+result = bitbake('-b asdf', ignore_status=True)
+self.assertTrue(ERROR: Unable to find any recipe file matching 
'asdf' in result.output)
+
+def test_bitbake_invalid_target(self):
+result = bitbake('asdf', ignore_status=True)
+self.assertTrue(ERROR: Nothing PROVIDES 'asdf' in result.output)
+
+def test_warnings_errors(self):
+result = bitbake('-b asdf', ignore_status=True)
+find_warnings = re.search(Summary: There was [1-9][0-9]* WARNING 
message shown., result.output)
+find_errors = re.search(Summary: There was [1-9][0-9]* ERROR message 
shown., result.output)
+self.assertTrue(find_warnings)
+self.assertTrue(find_errors)
+
+def test_invalid_patch(self):
+inc_file = os.path.join(get_test_layer(), 
'recipes-test/man/test_recipe.inc')
+ftools.write_file(inc_file, 'SRC_URI += file://man-1.5h1-make.patch')
+result = bitbake('man -c patch', ignore_status=True)
+os.remove(inc_file)
+bitbake('-cclean man')
+self.assertTrue(ERROR: Function failed: patch_do_patch in 
result.output)
+
+
+class BitbakeLayers(oeSelfTest):
+
+def test_bitbakelayers_showlayers(self):
+result = runCmd('bitbake-layers show_layers')
+self.assertTrue('meta-selftest' in result.output)
+
+def test_bitbakelayers_showappends(self):
+result = runCmd('bitbake-layers show_appends')
+self.assertTrue('xcursor-transparent-theme_0.1.1.bbappend' in 
result.output, msg='xcursor-transparent-theme_0.1.1.bbappend file was not 
recognised')
+
+def test_bitbakelayers_showoverlayed(self):
+result = runCmd('bitbake-layers show_overlayed')
+self.assertTrue('aspell' in result.output, 
msg='xcursor-transparent-theme_0.1.1.bbappend file was not recognised')
+
+def test_bitbakelayers_flatten(self):
+if os.path.isdir(os.path.join(self.builddir, 'test')):
+shutil.rmtree(os.path.join(self.builddir, 'test'))
+result = runCmd('bitbake-layers flatten test')
+bb_file = os.path.join(self.builddir, 
'test/recipes-graphics/xcursor-transparent-theme/xcursor-transparent-theme_0.1.1.bb')
+self.assertTrue(os.path.isfile(bb_file))
+contents = ftools.read_file(bb_file)
+find_in_contents = re.search(# bbappended from meta-selftest 
#\nTEST_VAR = \\, contents)
+shutil.rmtree(os.path.join(self.builddir, 'test

[OE-core] [RFC PATCH 0/4] Script and layer for running tests

2013-11-19 Thread Stefan Stanacar


Hello,

This series adds an oe-selftest script, some modules and a new layer 
meta-selftest.
which are meant to help in writing tests (using python unittest) for various
bitbake tools/scripts as well as simple output checks or do 
complete builds with different options (with the emphasis that everything 
checked
is done outside of bitbake context, just like a human would do.) For more 
details,
please see YOCTO #4740.

Cheers,
Stefan


The following changes since commit e15893adf9268b2920b24c52d5c2bb777c6f778e:

  bitbake: serv.py: Give pr-server up to 5 seconds to commit data (2013-11-18 
17:19:11 +)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib stefans/testme4
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=stefans/testme4

Alexandru Palalau (1):
  scripts/lib/selftest: add more simple build tests

Corneliu Stoicescu (2):
  meta-selftest: create a new test layer to be used by oe-selftest
script
  scripts/lib/selftest: add a new module for bitbake output tests

Stefan Stanacar (1):
  scripts/oe-selftest: script to run builds as unitests against bitbake
or various scripts

 .gitignore |   1 +
 meta-selftest/COPYING.MIT  |  17 +++
 meta-selftest/README   |   2 +
 meta-selftest/classes/test_events.bbclass  |  16 +++
 meta-selftest/conf/layer.conf  |  10 ++
 .../recipes-test/aspell/aspell_0.0.0.1.bb  |  28 
 .../recipes-test/man/man/man-1.5h1-make.patch  |  16 +++
 meta-selftest/recipes-test/man/man_1.6g.bbappend   |   2 +
 .../xcursor-transparent-theme_0.1.1.bbappend   |   3 +
 scripts/lib/selftest/__init__.py   |   0
 scripts/lib/selftest/base.py   |  54 
 scripts/lib/selftest/tests/__init__.py |   2 +
 scripts/lib/selftest/tests/bboptions.py|  39 ++
 scripts/lib/selftest/tests/bboutput.py |  86 
 scripts/lib/selftest/utils/__init__.py |   0
 scripts/lib/selftest/utils/commands.py | 146 +
 scripts/lib/selftest/utils/ftools.py   |  27 
 scripts/oe-selftest| 130 ++
 18 files changed, 579 insertions(+)
 create mode 100644 meta-selftest/COPYING.MIT
 create mode 100644 meta-selftest/README
 create mode 100644 meta-selftest/classes/test_events.bbclass
 create mode 100644 meta-selftest/conf/layer.conf
 create mode 100644 meta-selftest/recipes-test/aspell/aspell_0.0.0.1.bb
 create mode 100644 meta-selftest/recipes-test/man/man/man-1.5h1-make.patch
 create mode 100644 meta-selftest/recipes-test/man/man_1.6g.bbappend
 create mode 100644 
meta-selftest/recipes-test/xcursor-transparent-theme/xcursor-transparent-theme_0.1.1.bbappend
 create mode 100644 scripts/lib/selftest/__init__.py
 create mode 100644 scripts/lib/selftest/base.py
 create mode 100644 scripts/lib/selftest/tests/__init__.py
 create mode 100644 scripts/lib/selftest/tests/bboptions.py
 create mode 100644 scripts/lib/selftest/tests/bboutput.py
 create mode 100644 scripts/lib/selftest/utils/__init__.py
 create mode 100644 scripts/lib/selftest/utils/commands.py
 create mode 100644 scripts/lib/selftest/utils/ftools.py
 create mode 100755 scripts/oe-selftest

-- 
1.8.3.1

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [RFC PATCH 4/4] scripts/lib/selftest: add more simple build tests

2013-11-19 Thread Stefan Stanacar

From: Alexandru Palalau alexandrux.pala...@intel.com

Tests for RM_OLD_IMAGE and for WARN_QA/ERROR_QA behaviour.

Signed-off-by: Alexandru Palalau alexandrux.pala...@intel.com
Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 scripts/lib/selftest/tests/__init__.py  |  1 +
 scripts/lib/selftest/tests/bboptions.py | 39 +
 2 files changed, 40 insertions(+)
 create mode 100644 scripts/lib/selftest/tests/bboptions.py

diff --git a/scripts/lib/selftest/tests/__init__.py 
b/scripts/lib/selftest/tests/__init__.py
index 3acc480..7826054 100644
--- a/scripts/lib/selftest/tests/__init__.py
+++ b/scripts/lib/selftest/tests/__init__.py
@@ -1 +1,2 @@
 from selftest.tests.bboutput import *
+from selftest.tests.bboptions import *
diff --git a/scripts/lib/selftest/tests/bboptions.py 
b/scripts/lib/selftest/tests/bboptions.py
new file mode 100644
index 000..ad35386
--- /dev/null
+++ b/scripts/lib/selftest/tests/bboptions.py
@@ -0,0 +1,39 @@
+import unittest
+import os
+import logging
+
+from selftest.base import oeSelfTest
+from selftest.utils.commands import runCmd, bitbake, get_bb_var, get_test_layer
+import selftest.utils.ftools as ftools
+
+class OptionsTests(oeSelfTest):
+
+def test_options_warnqa_errorqa_switch(self):
+bitbake(xcursor-transparent-theme -ccleansstate)
+
+if packages-list not in get_bb_var(ERROR_QA):
+self.write_config(ERROR_QA_append = \ packages-list\)
+
+xcursor_path = os.path.join(get_test_layer(), 
recipes-test/xcursor-transparent-theme/test_recipe.inc)
+ftools.write_file(xcursor_path, 'PACKAGES += \${PN}-dbg\')
+
+res = bitbake(xcursor-transparent-theme, ignore_status=True)
+self.assertTrue(ERROR: QA Issue in res.output)
+self.assertEqual(res.status, 1)
+
+self.append_config(ERROR_QA_remove = \packages-list\)
+self.append_config(WARN_QA_append = \ packages-list\)
+bitbake(xcursor-transparent-theme)
+bitbake(xcursor-transparent-theme -ccleansstate)
+os.remove(xcursor_path)
+
+def test_options_rm_old_image(self):
+bitbake(core-image-minimal)
+
+deploydir = get_bb_var(DEPLOY_DIR_IMAGE, target=core-image-minimal)
+imagename = get_bb_var(IMAGE_LINK_NAME, target=core-image-minimal)
+oldimgpath = os.path.realpath(os.path.join(deploydir, imagename + 
.ext3))
+
+self.append_config(RM_OLD_IMAGE = \1\)
+bitbake(core-image-minimal)
+self.assertFalse(os.path.exists(oldimgpath), msg=Old image path still 
exists: %s % oldimgpath)
-- 
1.8.3.1

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [RFC PATCH 1/4] scripts/oe-selftest: script to run builds as unitests against bitbake or various scripts

2013-11-19 Thread Stefan Stanacar

The purpose of oe-selftest is to run unittest modules added in 
scripts/lib/selftest/tests,
which are tests against bitbake tools.
The scripts/lib/selftest/tests path for tests is only temporary.
Tests probably will be moved to meta/lib/oeqa/something once we've worked out 
how that can work.
(the reason to have them there is to allow layers to add their own tests).

Right now the script it's useful for simple tests like:
  - bitbake --someoption, change some metadata, bitbake X, check something 
type scenarios (PR service, error output, etc)
  - or bitbake-layers ... type scripts and yocto-bsp tools.
This commit also adds some helper methods that the tests will use (which are 
currently used by the script itself,
and those also need to move if the tests move).

Also, most of the tests will have a dependency on a meta-selftest layer
which contains specially modified recipes/bbappends/include files for the 
purpose of the tests.

It's a simple matter or sourcing the enviroment, adding the meta-selftest layer
and running: oe-selftest to get some results.

[ YOCTO #4740 ]

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 scripts/lib/selftest/__init__.py   |   0
 scripts/lib/selftest/base.py   |  54 
 scripts/lib/selftest/tests/__init__.py |   0
 scripts/lib/selftest/utils/__init__.py |   0
 scripts/lib/selftest/utils/commands.py | 146 +
 scripts/lib/selftest/utils/ftools.py   |  27 ++
 scripts/oe-selftest| 130 +
 7 files changed, 357 insertions(+)
 create mode 100644 scripts/lib/selftest/__init__.py
 create mode 100644 scripts/lib/selftest/base.py
 create mode 100644 scripts/lib/selftest/tests/__init__.py
 create mode 100644 scripts/lib/selftest/utils/__init__.py
 create mode 100644 scripts/lib/selftest/utils/commands.py
 create mode 100644 scripts/lib/selftest/utils/ftools.py
 create mode 100755 scripts/oe-selftest

diff --git a/scripts/lib/selftest/__init__.py b/scripts/lib/selftest/__init__.py
new file mode 100644
index 000..e69de29
diff --git a/scripts/lib/selftest/base.py b/scripts/lib/selftest/base.py
new file mode 100644
index 000..4d90828
--- /dev/null
+++ b/scripts/lib/selftest/base.py
@@ -0,0 +1,54 @@
+# Copyright (c) 2013 Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+# DESCRIPTION
+# Base class inherited by test classes in scripts/lib/selftest/tests
+
+import unittest
+import os
+import sys
+import logging
+
+import selftest.utils.ftools as ftools
+
+
+class oeSelfTest(unittest.TestCase):
+
+log = logging.getLogger(selftest.base)
+longMessage = True
+
+def __init__(self, methodName=runTest):
+self.builddir = os.environ.get(BUILDDIR)
+self.localconf_path = os.path.join(self.builddir, conf/local.conf)
+self.testinc_path = os.path.join(self.builddir, conf/selftest.inc)
+super(oeSelfTest, self).__init__(methodName)
+
+def setUp(self):
+os.chdir(self.builddir)
+# we don't know what the previous test set in here, we need a fresh 
start
+if os.path.isfile(self.testinc_path):
+os.remove(self.testinc_path)
+
+def write_config(self, data):
+self.log.debug(Writing to: %s\n%s\n % (self.testinc_path, data))
+ftools.write_file(self.testinc_path, data)
+
+def append_config(self, data):
+self.log.debug(Appending to: %s\n%s\n % (self.testinc_path, data))
+ftools.append_file(self.testinc_path, data)
+
+def remove_config(self, data):
+self.log.debug(Removing from: %s\n\%s\n % (self.testinc_path, data))
+ftools.remove_from_file(self.testinc_path, data)
diff --git a/scripts/lib/selftest/tests/__init__.py 
b/scripts/lib/selftest/tests/__init__.py
new file mode 100644
index 000..e69de29
diff --git a/scripts/lib/selftest/utils/__init__.py 
b/scripts/lib/selftest/utils/__init__.py
new file mode 100644
index 000..e69de29
diff --git a/scripts/lib/selftest/utils/commands.py 
b/scripts/lib/selftest/utils/commands.py
new file mode 100644
index 000..c808927
--- /dev/null
+++ b/scripts/lib/selftest/utils/commands.py
@@ -0,0 +1,146 @@
+# Copyright (c) 2013 Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published

[OE-core] [PATCH] lib/oeqa/utils: sshcontrol: make timeout depend on output

2013-11-03 Thread Stefan Stanacar

Instead of running the commands with a fixed timeout,
we should kill the command if there is no output
for timeout seconds.
Also changed some strings/comments.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/utils/sshcontrol.py | 41 ++-
 1 file changed, 27 insertions(+), 14 deletions(-)

diff --git a/meta/lib/oeqa/utils/sshcontrol.py 
b/meta/lib/oeqa/utils/sshcontrol.py
index 1539ff2..07257b8 100644
--- a/meta/lib/oeqa/utils/sshcontrol.py
+++ b/meta/lib/oeqa/utils/sshcontrol.py
@@ -9,6 +9,7 @@
 import subprocess
 import time
 import os
+import select
 
 class SSHControl(object):
 
@@ -50,32 +51,44 @@ class SSHControl(object):
 if self.host:
 sshconn = self._internal_run(cmd)
 else:
-raise Exception(Remote IP hasn't been set: '%s' % actualcmd)
+raise Exception(Remote IP/host hasn't been set, I can't run ssh 
without one.)
 
+# run the command forever
 if timeout == 0:
-self._out = sshconn.communicate()[0]
-self._ret = sshconn.poll()
+output = sshconn.communicate()[0]
 else:
+# use the default timeout
 if timeout is None:
 tdelta = self.timeout
+# use the specified timeout
 else:
 tdelta = timeout
 endtime = self._starttime + tdelta
-while sshconn.poll() is None and time.time()  endtime:
-time.sleep(1)
+output = ''
+eof = False
+while time.time()  endtime and not eof:
+if select.select([sshconn.stdout], [], [], 5)[0] != []:
+data = os.read(sshconn.stdout.fileno(), 1024)
+if not data:
+sshconn.stdout.close()
+eof = True
+else:
+output += data
+endtime = time.time() + tdelta
+
 # process hasn't returned yet
 if sshconn.poll() is None:
-self._ret = 255
 sshconn.terminate()
-sshconn.kill()
-self._out = sshconn.stdout.read()
-sshconn.stdout.close()
-self._out += \n[!!! SSH command timed out after %d seconds 
and it was killed] % tdelta
-else:
-self._out = sshconn.stdout.read()
-self._ret = sshconn.poll()
+time.sleep(3)
+try:
+sshconn.kill()
+except OSError:
+pass
+output += \n[!!! SSH command killed - no output for %d 
seconds. Total running time: %d seconds. % (tdelta, time.time() - 
self._starttime)
+
+self._ret = sshconn.poll()
 # strip the last LF so we can test the output
-self._out = self._out.rstrip()
+self._out = output.rstrip()
 self.log(%s % self._out)
 self.log([SSH command returned after %d seconds]: %s % (time.time() 
- self._starttime, self._ret))
 return (self._ret, self._out)
-- 
1.8.3.1

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] lib/oeqa/runtime: build kernel module on target test

2013-10-18 Thread Stefan Stanacar

Builds a simple Hello World module on target.
Added to the defaults for core-image-sato-sdk.

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/classes/testimage.bbclass|  2 +-
 meta/lib/oeqa/runtime/files/hellomod.c| 19 +++
 meta/lib/oeqa/runtime/files/hellomod_makefile |  8 +++
 meta/lib/oeqa/runtime/kernelmodule.py | 33 +++
 4 files changed, 61 insertions(+), 1 deletion(-)
 create mode 100644 meta/lib/oeqa/runtime/files/hellomod.c
 create mode 100644 meta/lib/oeqa/runtime/files/hellomod_makefile
 create mode 100644 meta/lib/oeqa/runtime/kernelmodule.py

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 2f9c974..5d61c2b 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -28,7 +28,7 @@ TEST_LOG_DIR ?= ${WORKDIR}/testimage
 DEFAULT_TEST_SUITES = ping auto
 DEFAULT_TEST_SUITES_pn-core-image-minimal = ping
 DEFAULT_TEST_SUITES_pn-core-image-sato = ping ssh df connman syslog xorg scp 
vnc date rpm smart dmesg
-DEFAULT_TEST_SUITES_pn-core-image-sato-sdk = ping ssh df connman syslog xorg 
scp vnc date perl ldd gcc rpm smart dmesg
+DEFAULT_TEST_SUITES_pn-core-image-sato-sdk = ping ssh df connman syslog xorg 
scp vnc date perl ldd gcc rpm smart kernelmodule dmesg
 
 TEST_SUITES ?= ${DEFAULT_TEST_SUITES}
 
diff --git a/meta/lib/oeqa/runtime/files/hellomod.c 
b/meta/lib/oeqa/runtime/files/hellomod.c
new file mode 100644
index 000..a383397
--- /dev/null
+++ b/meta/lib/oeqa/runtime/files/hellomod.c
@@ -0,0 +1,19 @@
+#include linux/module.h
+#include linux/kernel.h
+#include linux/init.h
+
+static int __init hello_init(void)
+{
+printk(KERN_INFO Hello world!\n);
+return 0;
+}
+
+static void __exit hello_cleanup(void)
+{
+printk(KERN_INFO Cleaning up hellomod.\n);
+}
+
+module_init(hello_init);
+module_exit(hello_cleanup);
+
+MODULE_LICENSE(GPL);
diff --git a/meta/lib/oeqa/runtime/files/hellomod_makefile 
b/meta/lib/oeqa/runtime/files/hellomod_makefile
new file mode 100644
index 000..b92d5c8
--- /dev/null
+++ b/meta/lib/oeqa/runtime/files/hellomod_makefile
@@ -0,0 +1,8 @@
+obj-m := hellomod.o
+KDIR := /usr/src/kernel
+
+all:
+   $(MAKE) -C $(KDIR) M=$(PWD) modules
+
+clean:
+   $(MAKE) -C $(KDIR) M=$(PWD) clean
diff --git a/meta/lib/oeqa/runtime/kernelmodule.py 
b/meta/lib/oeqa/runtime/kernelmodule.py
new file mode 100644
index 000..cbc5742
--- /dev/null
+++ b/meta/lib/oeqa/runtime/kernelmodule.py
@@ -0,0 +1,33 @@
+import unittest
+import os
+from oeqa.oetest import oeRuntimeTest, skipModule
+from oeqa.utils.decorators import *
+
+def setUpModule():
+if not oeRuntimeTest.hasFeature(tools-sdk):
+skipModule(Image doesn't have tools-sdk in IMAGE_FEATURES)
+
+
+class KernelModuleTest(oeRuntimeTest):
+
+def setUp(self):
+self.target.copy_to(os.path.join(oeRuntimeTest.tc.filesdir, 
hellomod.c), /tmp/hellomod.c)
+self.target.copy_to(os.path.join(oeRuntimeTest.tc.filesdir, 
hellomod_makefile), /tmp/Makefile)
+
+@skipUnlessPassed('test_ssh')
+@skipUnlessPassed('test_gcc_compile')
+def test_kernel_module(self):
+cmds = [
+'cd /usr/src/kernel  make scripts',
+'cd /tmp  make',
+'cd /tmp  insmod hellomod.ko',
+'lsmod | grep hellomod',
+'dmesg | grep Hello',
+'rmmod hellomod', 'dmesg | grep Cleaning up hellomod'
+]
+for cmd in cmds:
+(status, output) = self.target.run(cmd, 900)
+self.assertEqual(status, 0, msg=\n.join([cmd, output]))
+
+def tearDown(self):
+self.target.run('rm -f /tmp/Makefile /tmp/hellomod.c')
-- 
1.8.3.1

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] lib/oeqa/runtime: ping: wait for 5 echo replies

2013-09-25 Thread Stefan Stanacar

Instead of considering that ping test passed after 1 reply,
wait for at least 5 consecutive replies in 60 seconds (which should
be enough time for connman to reconfigure the interface in systemd
images and help with the fake ssh/tests fails.)

Signed-off-by: Stefan Stanacar stefanx.stana...@intel.com
---
 meta/lib/oeqa/runtime/ping.py | 11 +++
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/meta/lib/oeqa/runtime/ping.py b/meta/lib/oeqa/runtime/ping.py
index e163b96..0d028f9 100644
--- a/meta/lib/oeqa/runtime/ping.py
+++ b/meta/lib/oeqa/runtime/ping.py
@@ -8,10 +8,13 @@ class PingTest(oeRuntimeTest):
 
 def test_ping(self):
 output = ''
-status = None
+count = 0
 endtime = time.time() + 60
-while status != 0 and time.time()  endtime:
+while count  5 and time.time()  endtime:
 proc = subprocess.Popen(ping -c 1 %s % oeRuntimeTest.tc.qemu.ip, 
shell=True, stdout=subprocess.PIPE)
 output += proc.communicate()[0]
-status = proc.poll()
-self.assertEqual(status, 0, msg = No echo reply, ping output is:\n%s 
% output)
+if proc.poll() == 0:
+count += 1
+else:
+count = 0
+self.assertEqual(count, 5, msg = Expected 5 consecutive replies, got 
%d.\nping output is:\n%s % (count,output))
-- 
1.8.3.1

___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

1 2 3 >

1 - 100 of 231 matches

Mail list logo