[OE-core] [PATCH v2 3/3] classes/sstate: regenerate sstate when signing enabled

[Joshua Lock via Openembedded-core]

This change ensures that the task signatures changes, and therefore
sstate tasks are rerun, when signing is enabled. This has the
positive outcome that if signing is enabled new signed shared state
objects will be produced, rather than just signing shared state
objects for tasks where no work has been performed yet.

The downside of this change is that enabling/disabling sstate object
signing alters the taskhash and results in rebuilding the world.

Signed-off-by: Joshua Lock 
---
 meta/classes/sstate.bbclass | 20 
 1 file changed, 12 insertions(+), 8 deletions(-)

diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 3342c5ef50..ee029196da 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -659,8 +659,12 @@ def sstate_package(ss, d):
 if d.getVar('SSTATE_SKIP_CREATION') == '1':
 return
 
+sstate_create_package = ['sstate_report_unihash', 'sstate_create_package']
+if d.getVar('SSTATE_SIG_KEY'):
+sstate_create_package.append('sstate_sign_package')
+
 for f in (d.getVar('SSTATECREATEFUNCS') or '').split() + \
- ['sstate_report_unihash', 'sstate_create_package', 
'sstate_sign_package'] + \
+ sstate_create_package + \
  (d.getVar('SSTATEPOSTCREATEFUNCS') or '').split():
 # All hooks should run in SSTATE_BUILDDIR.
 bb.build.exec_func(f, d, (sstatebuild,))
@@ -776,13 +780,13 @@ sstate_create_package () {
 python sstate_sign_package () {
 from oe.gpg_sign import get_signer
 
-if d.getVar('SSTATE_SIG_KEY'):
-signer = get_signer(d, 'local')
-sstate_pkg = d.getVar('SSTATE_PKG')
-if os.path.exists(sstate_pkg + '.sig'):
-os.unlink(sstate_pkg + '.sig')
-signer.detach_sign(sstate_pkg, d.getVar('SSTATE_SIG_KEY', False), None,
-   d.getVar('SSTATE_SIG_PASSPHRASE'), armor=False)
+
+signer = get_signer(d, 'local')
+sstate_pkg = d.getVar('SSTATE_PKG')
+if os.path.exists(sstate_pkg + '.sig'):
+os.unlink(sstate_pkg + '.sig')
+signer.detach_sign(sstate_pkg, d.getVar('SSTATE_SIG_KEY', False), None,
+   d.getVar('SSTATE_SIG_PASSPHRASE'), armor=False)
 }
 
 python sstate_report_unihash() {
-- 
2.21.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 1/3] sstate: fix log message

[Joshua Lock via Openembedded-core]

Referring to the sstate object as a staging package is an artefact of the
code's origins. Switch to referring to an "Sstate package" in order to be more
accurate and consistent with the rest of the file.

Signed-off-by: Joshua Lock 
---
 meta/classes/sstate.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index b604729d84..d8fdcece6a 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -329,7 +329,7 @@ def sstate_installpkg(ss, d):
 pstaging_fetch(sstatefetch, d)
 
 if not os.path.isfile(sstatepkg):
-bb.note("Staging package %s does not exist" % sstatepkg)
+bb.note("Sstate package %s does not exist" % sstatepkg)
 return False
 
 sstate_clean(ss, d)
-- 
2.21.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 2/3] classes/sstate: don't use unsigned sstate when verification enabled

[Joshua Lock via Openembedded-core]

When signature verification of shared state objects is enabled
(SSTATE_VERIFY_SIG) use of an unsigned object, even though it produces a
warning, seems unexpected. Instead skip unsigned objects and force the
non-accelerated task to be run.

Signed-off-by: Joshua Lock 
---
 meta/classes/sstate.bbclass | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index d8fdcece6a..3342c5ef50 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -340,7 +340,8 @@ def sstate_installpkg(ss, d):
 if bb.utils.to_boolean(d.getVar("SSTATE_VERIFY_SIG"), False):
 signer = get_signer(d, 'local')
 if not signer.verify(sstatepkg + '.sig'):
-bb.warn("Cannot verify signature on sstate package %s" % sstatepkg)
+bb.warn("Cannot verify signature on sstate package %s, skipping 
acceleration..." % sstatepkg)
+return False
 
 # Empty sstateinst directory, ensure its clean
 if os.path.exists(sstateinst):
-- 
2.21.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 0/3] Enforce Shared State Signing when optionsset

[Joshua Lock via Openembedded-core]

There are 2 surprising behaviours in the current shared state signing
implementation which this pull request addresses:
1) when signature verification is enabled a failure to verify doesn't prevent
   the shared state object from being used. A warning is printed and the
   unsigned shared state object is used to accelerate the task.
2) when signing is enabled the taskhash doesn't change and any existing,
   unsigned, shared state objects will not be signed. This is particularly
   problematic when combined with 1.

Please review the following changes for suitability for inclusion. If you have
any objections or suggestions for improvement, please respond to the patches. If
you agree with the changes, please provide your Acked-by.

Changes since v1:
* removed spurious Python subclass class and handling of an error condition that
  couldn't occur

The following changes since commit 835f7eac0610325e906591cd81890bebe8627580:

  meta/lib/oeqa: Test for bootimg-biosplusefi Source (2019-07-23 22:26:28 +0100)

are available in the Git repository at:

  https://github.com/joshuagl/poky joshuagl/signed-sstate2
  https://github.com/joshuagl/poky/tree/joshuagl/signed-sstate2

Joshua Lock (3):
  sstate: fix log message
  classes/sstate: don't use unsigned sstate when verification enabled
  classes/sstate: regenerate sstate when signing enabled

 meta/classes/sstate.bbclass | 25 +++--
 1 file changed, 15 insertions(+), 10 deletions(-)

-- 
2.21.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [RFC PATCH 3/3] classes/sstate: regenerate sstate when signing enabled

[Joshua Lock via Openembedded-core]

> On 27 Jul 2019, at 11:12, Richard Purdie  
> wrote:
> 
> On Fri, 2019-07-26 at 11:26 +0000, Joshua Lock via Openembedded-core
> wrote:
>> This change ensures that the task signatures changes, and therefore
>> sstate tasks are rerun, when signing is enabled. This has the
>> positive outcome that if signing is enabled new signed shared state
>> objects will be produced, rather than just signing shared state
>> objects for tasks where no work has been performed yet.
>> 
>> The downside of this change is that enabling/disabling sstate object
>> signing alters the taskhash and results in rebuilding the world.
>> 
>> Signed-off-by: Joshua Lock 
>> ---
>> meta/classes/sstate.bbclass | 10 --
>> meta/lib/oe/gpg_sign.py | 10 ++
>> 2 files changed, 18 insertions(+), 2 deletions(-)
>> 
>> diff --git a/meta/classes/sstate.bbclass
>> b/meta/classes/sstate.bbclass
>> index 3342c5ef50..b060e15053 100644
>> --- a/meta/classes/sstate.bbclass
>> +++ b/meta/classes/sstate.bbclass
>> @@ -659,8 +659,12 @@ def sstate_package(ss, d):
>> if d.getVar('SSTATE_SKIP_CREATION') == '1':
>> return
>> 
>> +sstate_create_package = ['sstate_report_unihash',
>> 'sstate_create_package']
>> +if d.getVar('SSTATE_SIG_KEY'):
>> +sstate_create_package.append('sstate_sign_package')
>> +
>> for f in (d.getVar('SSTATECREATEFUNCS') or '').split() + \
>> - ['sstate_report_unihash', 'sstate_create_package',
>> 'sstate_sign_package'] + \
>> + sstate_create_package + \
>>  (d.getVar('SSTATEPOSTCREATEFUNCS') or '').split():
>> # All hooks should run in SSTATE_BUILDDIR.
>> bb.build.exec_func(f, d, (sstatebuild,))
>> @@ -774,7 +778,7 @@ sstate_create_package () {
>> }
>> 
>> python sstate_sign_package () {
>> -from oe.gpg_sign import get_signer
>> +from oe.gpg_sign import get_signer, SignFailedError
>> 
>> if d.getVar('SSTATE_SIG_KEY'):
>> signer = get_signer(d, 'local')
>> @@ -783,6 +787,8 @@ python sstate_sign_package () {
>> os.unlink(sstate_pkg + '.sig')
>> signer.detach_sign(sstate_pkg, d.getVar('SSTATE_SIG_KEY',
>> False), None,
>>d.getVar('SSTATE_SIG_PASSPHRASE'),
>> armor=False)
>> +else:
>> +raise SignFailedError("Can't sign sstate packages without
>> key, SSTATE_SIG_KEY empty")
>> }
>> 
>> python sstate_report_unihash() {
>> diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
>> index 2fd8c3b1ac..ec5ace0dd0 100644
>> --- a/meta/lib/oe/gpg_sign.py
>> +++ b/meta/lib/oe/gpg_sign.py
>> @@ -123,6 +123,16 @@ class LocalSigner(object):
>> return ret
>> 
>> 
>> +class SignFailedError(bb.build.FuncFailed):
>> +def __init__(self, description, name=None, logfile=None):
>> +self.description = description
>> +self.name = name
>> +self.logfile = logfile
>> +
>> +def __str__(self):
>> +return 'Signing failed: %s' % self.description
>> +
>> +
> 
> Whilst this subclass is quite pythonic, I'm not sure we want to do
> this. At the back of my mind is the feeling that FuncFailed is a bit
> pointless :/.

This is pointless in more ways than one, because above I changed the 
SSTATECREATEFUNCS to only include sstate_sign_package if SSTATE_SIG_KEY is set 
this exception shouldn’t ever be raised.

I’ll send an update series with this class and its usage dropped.

Joshua

> Put differently, what does this buy us that bb.fatal() doesn't?
> 
> We do have a few users of FuncFailed in OECore but not many (about
> 8)...
> 
> Cheers,
> 
> Richard
> 
> 
> 

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [RFC PATCH 1/3] sstate: fix log message

[Joshua Lock via Openembedded-core]

Referring to the sstate object as a staging package is an artefact of the
code's origins. Switch to referring to an "Sstate package" in order to be more
accurate and consistent with the rest of the file.

Signed-off-by: Joshua Lock 
---
 meta/classes/sstate.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index b604729d84..d8fdcece6a 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -329,7 +329,7 @@ def sstate_installpkg(ss, d):
 pstaging_fetch(sstatefetch, d)
 
 if not os.path.isfile(sstatepkg):
-bb.note("Staging package %s does not exist" % sstatepkg)
+bb.note("Sstate package %s does not exist" % sstatepkg)
 return False
 
 sstate_clean(ss, d)
-- 
2.21.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [RFC PATCH 3/3] classes/sstate: regenerate sstate when signing enabled

[Joshua Lock via Openembedded-core]

This change ensures that the task signatures changes, and therefore
sstate tasks are rerun, when signing is enabled. This has the
positive outcome that if signing is enabled new signed shared state
objects will be produced, rather than just signing shared state
objects for tasks where no work has been performed yet.

The downside of this change is that enabling/disabling sstate object
signing alters the taskhash and results in rebuilding the world.

Signed-off-by: Joshua Lock 
---
 meta/classes/sstate.bbclass | 10 --
 meta/lib/oe/gpg_sign.py | 10 ++
 2 files changed, 18 insertions(+), 2 deletions(-)

diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 3342c5ef50..b060e15053 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -659,8 +659,12 @@ def sstate_package(ss, d):
 if d.getVar('SSTATE_SKIP_CREATION') == '1':
 return
 
+sstate_create_package = ['sstate_report_unihash', 'sstate_create_package']
+if d.getVar('SSTATE_SIG_KEY'):
+sstate_create_package.append('sstate_sign_package')
+
 for f in (d.getVar('SSTATECREATEFUNCS') or '').split() + \
- ['sstate_report_unihash', 'sstate_create_package', 
'sstate_sign_package'] + \
+ sstate_create_package + \
  (d.getVar('SSTATEPOSTCREATEFUNCS') or '').split():
 # All hooks should run in SSTATE_BUILDDIR.
 bb.build.exec_func(f, d, (sstatebuild,))
@@ -774,7 +778,7 @@ sstate_create_package () {
 }
 
 python sstate_sign_package () {
-from oe.gpg_sign import get_signer
+from oe.gpg_sign import get_signer, SignFailedError
 
 if d.getVar('SSTATE_SIG_KEY'):
 signer = get_signer(d, 'local')
@@ -783,6 +787,8 @@ python sstate_sign_package () {
 os.unlink(sstate_pkg + '.sig')
 signer.detach_sign(sstate_pkg, d.getVar('SSTATE_SIG_KEY', False), None,
d.getVar('SSTATE_SIG_PASSPHRASE'), armor=False)
+else:
+raise SignFailedError("Can't sign sstate packages without key, 
SSTATE_SIG_KEY empty")
 }
 
 python sstate_report_unihash() {
diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
index 2fd8c3b1ac..ec5ace0dd0 100644
--- a/meta/lib/oe/gpg_sign.py
+++ b/meta/lib/oe/gpg_sign.py
@@ -123,6 +123,16 @@ class LocalSigner(object):
 return ret
 
 
+class SignFailedError(bb.build.FuncFailed):
+def __init__(self, description, name=None, logfile=None):
+self.description = description
+self.name = name
+self.logfile = logfile
+
+def __str__(self):
+return 'Signing failed: %s' % self.description
+
+
 def get_signer(d, backend):
 """Get signer object for the specified backend"""
 # Use local signing by default
-- 
2.21.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [RFC PATCH 2/3] classes/sstate: don't use unsigned sstate when verification enabled

[Joshua Lock via Openembedded-core]

When signature verification of shared state objects is enabled
(SSTATE_VERIFY_SIG) use of an unsigned object, even though it produces a
warning, seems unexpected. Instead skip unsigned objects and force the
non-accelerated task to be run.

Signed-off-by: Joshua Lock 
---
 meta/classes/sstate.bbclass | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index d8fdcece6a..3342c5ef50 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -340,7 +340,8 @@ def sstate_installpkg(ss, d):
 if bb.utils.to_boolean(d.getVar("SSTATE_VERIFY_SIG"), False):
 signer = get_signer(d, 'local')
 if not signer.verify(sstatepkg + '.sig'):
-bb.warn("Cannot verify signature on sstate package %s" % sstatepkg)
+bb.warn("Cannot verify signature on sstate package %s, skipping 
acceleration..." % sstatepkg)
+return False
 
 # Empty sstateinst directory, ensure its clean
 if os.path.exists(sstateinst):
-- 
2.21.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [RFC PATCH 0/3] Enforce Shared State Signing when options set

[Joshua Lock via Openembedded-core]

There are 2 surprising behaviours in the current shared state signing
implementation which this pull request addresses:
1) when signature verification is enabled a failure to verify doesn't prevent
   the shared state object from being used. A warning is printed and the
   unsigned shared state object is used to accelerate the task.
2) when signing is enabled the taskhash doesn't change and any existing,
   unsigned, shared state objects will not be signed. This is particularly
   problematic when combined with 1.

Please review the following changes for suitability for inclusion. If you have
any objections or suggestions for improvement, please respond to the patches. If
you agree with the changes, please provide your Acked-by.

The following changes since commit 835f7eac0610325e906591cd81890bebe8627580:

  meta/lib/oeqa: Test for bootimg-biosplusefi Source (2019-07-23 22:26:28 +0100)

are available in the Git repository at:

  https://github.com/joshuagl/poky joshuagl/signed-sstate
  https://github.com/joshuagl/poky/tree/joshuagl/signed-sstate

Joshua Lock (3):
  sstate: fix log message
  classes/sstate: don't use unsigned sstate when verification enabled
  classes/sstate: regenerate sstate when signing enabled

 meta/classes/sstate.bbclass | 15 +++
 meta/lib/oe/gpg_sign.py | 10 ++
 2 files changed, 21 insertions(+), 4 deletions(-)

-- 
2.21.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] python3-manifest: add entry to package the timeit module

[Joshua Lock]

Signed-off-by: Joshua Lock 
---
 .../python/python3/python3-manifest.json | 12 
 1 file changed, 12 insertions(+)

diff --git a/meta/recipes-devtools/python/python3/python3-manifest.json 
b/meta/recipes-devtools/python/python3/python3-manifest.json
index 0803ac0033..17d41c34b9 100644
--- a/meta/recipes-devtools/python/python3/python3-manifest.json
+++ b/meta/recipes-devtools/python/python3/python3-manifest.json
@@ -1131,6 +1131,18 @@
 "${libdir}/python${PYTHON_MAJMIN}/__pycache__/queue.*.pyc"
 ]
 },
+"timeit": {
+"summary": "Simple way to time small bits of code",
+"rdepends": [
+"core"
+],
+"files": [
+"${libdir}/python${PYTHON_MAJMIN}/timeit.py"
+],
+"cached": [
+"${libdir}/python${PYTHON_MAJMIN}/__pycache__/timeit.*.pyc"
+]
+},
 "tkinter": {
 "summary": "Python Tcl/Tk bindings",
 "rdepends": [
-- 
2.21.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] glib-2.0: Re-add the gettext depend

[Joshua Lock]

On 18/01/18 19:12, Alistair Francis wrote:

Commit "glib-2.0: rationalise build dependencies"
(8b0ca0cb54c91611213556bdb99316d5e7ac5b3b) broke mingw glib builds with
the following error:

This patch fixes the issue by re-adding the gettext dependency.

Signed-off-by: Alistair Francis 
---
  meta/recipes-core/glib-2.0/glib.inc | 1 +
  1 file changed, 1 insertion(+)

diff --git a/meta/recipes-core/glib-2.0/glib.inc 
b/meta/recipes-core/glib-2.0/glib.inc
index 354d95cfe3..e6997ae88f 100644
--- a/meta/recipes-core/glib-2.0/glib.inc
+++ b/meta/recipes-core/glib-2.0/glib.inc
@@ -24,6 +24,7 @@ DEPENDS = "glib-2.0-native \
 virtual/libiconv \
 libffi \
 zlib \
+   gettext \


If this is only required for mingw we should add it in a mingw override, 
something like (untested):


DEPENDS_mingw32 += "gettext"

That way we:
a) know that it's a mingw dependency (and to test that toolchain before 
removing)

b) don't bloat the DEPENDS where gettext isn't a strict requirement.


 ${@bb.utils.contains('PTEST_ENABLED', '1', 'dbus', '', d)}"
  
  PACKAGES += "${PN}-codegen ${PN}-utils"



--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] oeqa/selftest/cases/wic: update OETestID decorator for test_qemu

[Joshua Lock]

Test case ID 1422 is not correct for this issue, the correct test
case ID for test_qemu is 1424.

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/lib/oeqa/selftest/cases/wic.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/selftest/cases/wic.py 
b/meta/lib/oeqa/selftest/cases/wic.py
index ccbc82ed9b4..b84466d9aef 100644
--- a/meta/lib/oeqa/selftest/cases/wic.py
+++ b/meta/lib/oeqa/selftest/cases/wic.py
@@ -623,7 +623,7 @@ part /etc --source rootfs --ondisk mmcblk0 --fstype=ext4 
--exclude-path bin/ --r
 self.assertTrue(os.path.islink(path))
 self.assertTrue(os.path.isfile(os.path.realpath(path)))
 
-@OETestID(1422)
+@OETestID(1424)
 @only_for_arch(['i586', 'i686', 'x86_64'])
 def test_qemu(self):
 """Test wic-image-minimal under qemu"""
-- 
2.14.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] core/loader.py: fix regex to include all available test cases

[Joshua Lock]

On 05/12/17 15:51, leonardo.sandoval.gonza...@linux.intel.com wrote:

From: Leonardo Sandoval <leonardo.sandoval.gonza...@linux.intel.com>

Some test cases (i.e. eSDK.oeSDKExtSelfTest*) does not match with current regex,
fix it to accept these (and previous ones).

Without it, the following runtime exception is observed when executing eSDK
selftest (oe-selftest -r eSDK)

 Traceback (most recent call last):
   File "/openembedded-core/scripts/oe-selftest", line 70, in 
 ret = main()
   File "/openembedded-core/scripts/oe-selftest", line 57, in main
 results = args.func(logger, args)
   File "/openembedded-core/meta/lib/oeqa/selftest/context.py", line 253, 
in run
 rc = self._internal_run(logger, args)
   File "/openembedded-core/meta/lib/oeqa/selftest/context.py", line 205, 
in _internal_run
 self.tc.loadTests(self.module_paths, **self.tc_kwargs['load'])
   File "/openembedded-core/meta/lib/oeqa/core/context.py", line 58, in 
loadTests
 modules_required, filters)
   File "/openembedded-core/meta/lib/oeqa/core/loader.py", line 69, in 
__init__
 self.modules = _built_modules_dict(modules)
   File "/openembedded-core/meta/lib/oeqa/core/loader.py", line 48, in 
_built_modules_dict
 module_name, class_name, test_name = m.groups()
 AttributeError: 'NoneType' object has no attribute 'groups'

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonza...@linux.intel.com>


LGTM and fixes [YOCTO #12385], thanks!

Acked-by: Joshua Lock <joshua.g.l...@intel.com>


---
  meta/lib/oeqa/core/loader.py | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/core/loader.py b/meta/lib/oeqa/core/loader.py
index 975a081ba4..9ab4df0a60 100644
--- a/meta/lib/oeqa/core/loader.py
+++ b/meta/lib/oeqa/core/loader.py
@@ -43,7 +43,7 @@ def _built_modules_dict(modules):
  for module in modules:
  # Assumption: package and module names do not contain upper case
  # characters, whereas class names do
-m = re.match(r'^([^A-Z]+)(?:\.([A-Z][^.]*)(?:\.([^.]+))?)?$', module)
+m = re.match(r'^([a-zA-Z]+)(?:\.([a-zA-Z][^.]*)(?:\.([^.]+))?)?$', 
module)
  
  module_name, class_name, test_name = m.groups()
  


--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 3/3] oeqa/selftest/runtime_test: simplify postinst testing

[Joshua Lock]

On 23/11/2017 00:04, Ross Burton wrote:

Update the packages and file names to reflect the new postinst recipe.

Fix a sh syntax error in the run_serial file exists test which was hidden by a
logic problem in the status code.

Remove the older test_verify_postinst as it's effectively a subset of
test_postinst_rootfs_and_boot, and doesn't work: when booting under systemd the
strings it searches for are not output to the console, but the test still
passes.

Signed-off-by: Ross Burton 
---
  meta/lib/oeqa/selftest/cases/runtime_test.py | 82 +++-
  1 file changed, 19 insertions(+), 63 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/runtime_test.py 
b/meta/lib/oeqa/selftest/cases/runtime_test.py
index 25270b7535b..1c69255b568 100644
--- a/meta/lib/oeqa/selftest/cases/runtime_test.py
+++ b/meta/lib/oeqa/selftest/cases/runtime_test.py
@@ -167,55 +167,6 @@ class TestImage(OESelftestTestCase):
  
  class Postinst(OESelftestTestCase):

  @OETestID(1540)


We're effectively stating that test ID 1540 and test ID 1545 are both 
covered by the same test method, right?


Perhaps we don't need to maintain both test cases in Testopia, as we'll 
only ever see both tests pass or fail?



-def test_verify_postinst(self):
-"""
-Summary: The purpose of this test is to verify the execution order of 
postinst Bugzilla ID: [5319]
-Expected :
-1. Compile a minimal image.
-2. The compiled image will add the created layer with the recipes 
postinst[ abdpt]
-3. Run qemux86
-4. Validate the task execution order
-Author: Francisco Pedraza 
-"""
-features = 'INHERIT += "testimage"\n'
-features += 'CORE_IMAGE_EXTRA_INSTALL += "postinst-at-rootfs \
-postinst-delayed-a \
-postinst-delayed-b \
-postinst-delayed-d \
-postinst-delayed-p \
-postinst-delayed-t \
-"\n'
-self.write_config(features)
-
-bitbake('core-image-minimal -f ')
-
-postinst_list = ['100-postinst-at-rootfs',
- '101-postinst-delayed-a',
- '102-postinst-delayed-b',
- '103-postinst-delayed-d',
- '104-postinst-delayed-p',
- '105-postinst-delayed-t']
-path_workdir = get_bb_var('WORKDIR','core-image-minimal')
-workspacedir = 'testimage/qemu_boot_log'
-workspacedir = os.path.join(path_workdir, workspacedir)
-rexp = re.compile("^Running postinst .*/(?P.*)\.\.\.$")
-with runqemu('core-image-minimal') as qemu:
-with open(workspacedir) as f:
-found = False
-idx = 0
-for line in f.readlines():
-line = line.strip().replace("^M","")
-if not line: # To avoid empty lines
-continue
-m = rexp.search(line)
-if m:
-self.assertEqual(postinst_list[idx], m.group('postinst'), 
"Fail")
-idx = idx+1
-found = True
-elif found:
-self.assertEqual(idx, len(postinst_list), "Not found all 
postinsts")
-break
-
  @OETestID(1545)
  def test_postinst_rootfs_and_boot(self):
  """
@@ -234,16 +185,22 @@ postinst-delayed-t \
  for initialization managers: sysvinit and systemd.
  
  """

-file_rootfs_name = "this-was-created-at-rootfstime"
-fileboot_name = "this-was-created-at-first-boot"
-rootfs_pkg = 'postinst-at-rootfs'
-boot_pkg = 'postinst-delayed-a'
+
+import oe.path
+
+vars = get_bb_vars(("IMAGE_ROOTFS", "sysconfdir"), 
"core-image-minimal")
+rootfs = vars["IMAGE_ROOTFS"]
+self.assertIsNotNone(rootfs)
+sysconfdir = vars["sysconfdir"]
+self.assertIsNotNone(sysconfdir)
+# Need to use oe.path here as sysconfdir starts with /
+hosttestdir = oe.path.join(rootfs, sysconfdir, "postinst-test")
+targettestdir = os.path.join(sysconfdir, "postinst-test")
  
  for init_manager in ("sysvinit", "systemd"):

  for classes in ("package_rpm", "package_deb", "package_ipk"):
  with self.subTest(init_manager=init_manager, 
package_class=classes):
-features = 'MACHINE = "qemux86"\n'
-features += 'CORE_IMAGE_EXTRA_INSTALL += "%s %s "\n'% 
(rootfs_pkg, boot_pkg)
+features = 'CORE_IMAGE_EXTRA_INSTALL = 
"postinst-delayed-b"\n'
  features += 'IMAGE_FEATURES += "package-management 
empty-root-password"\n'


The indentation doesn't look right here?


  features += 'PACKAGE_CLASSES = "%s"\n' % classes
  if init_manager == "systemd":
@@ -255,13 +212,12 @@ 

Re: [OE-core] [PATCH] scripts/runqemu: when qemu fails to start with kvm retry with kvm disabled

[Joshua Lock]

On 03/11/17 15:18, akuster808 wrote:

Joshua,


On 11/03/2017 02:11 AM, Joshua Lock wrote:

On the Yocto Project Autobuilder we occasionally see qemu fail to start,
most frequently on Fedora 25, because kvm fails to allocate memory. See:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=12058

This patch introduces a fallback path where if the inital qemu command fails
and kvm is enabled we retry running qemu with the kvm related options
removed from the qemu command line.


I will backport once this lands in master.
thanks,


I don't think it will make it. It complicates the code for a relatively 
rare edge case.


To avoid this on Yocto Autobuilder we're going to drop Fedora 25 from 
the clusters.


Joshua
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] scripts/runqemu: when qemu fails to start with kvm retry with kvm disabled

[Joshua Lock]

On the Yocto Project Autobuilder we occasionally see qemu fail to start,
most frequently on Fedora 25, because kvm fails to allocate memory. See:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=12058

This patch introduces a fallback path where if the inital qemu command fails
and kvm is enabled we retry running qemu with the kvm related options
removed from the qemu command line.

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 scripts/runqemu | 20 +++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/scripts/runqemu b/scripts/runqemu
index df762709043..b7f8b3c4d55 100755
--- a/scripts/runqemu
+++ b/scripts/runqemu
@@ -1168,7 +1168,25 @@ class BaseConfig(object):
 logger.info('Running %s\n' % cmd)
 process = subprocess.Popen(cmd, shell=True, stderr=subprocess.PIPE)
 if process.wait():
-logger.error("Failed to run qemu: %s", 
process.stderr.read().decode())
+# If starting qemu failed and kvm is enabled we try again, with the
+# kvm related options removed. This is because we have observed
+# failures where kvm is unable to allocate memory (particularly on
+# Fedora 25) but where a run withou kvm may be successful.
+if self.kvm_enabled:
+logger.info("Failed to run qemu: %s\nRetrying without kvm...",
+process.stderr.read().decode())
+nokvm_cmd = cmd
+nokvm_cmd = nokvm_cmd.replace(' -enable-kvm', '')
+nokvm_cmd = nokvm_cmd.replace(',vhost=on', '')
+logger.info('Running %s\n' % nokvm_cmd)
+process = subprocess.Popen(nokvm_cmd, shell=True,
+   stderr=subprocess.PIPE)
+if process.wait():
+logger.error("Failed to run qemu (with kvm disabled): %s",
+ process.stderr.read().decode())
+else:
+logger.error("Failed to run qemu: %s",
+ process.stderr.read().decode())
 
 def cleanup(self):
 if self.cleantap:
-- 
2.13.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2][morty] ruby: fix build of ruby-native with gcc7

[Joshua Lock]

Marshalling is broken when ruby-2.2.x is built with gcc7, backport the
fix in Ruby SVN r57410 to apply to ruby 2.2.5:

https://svn.ruby-lang.org/cgi-bin/viewvc.cgi?view=revision=57410

Fixes [YOCTO #12271]

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---

v2: fixes typo in commit message

 meta/recipes-devtools/ruby/ruby/prevent-gc.patch | 32 
 meta/recipes-devtools/ruby/ruby_2.2.5.bb |  2 ++
 2 files changed, 34 insertions(+)
 create mode 100644 meta/recipes-devtools/ruby/ruby/prevent-gc.patch

diff --git a/meta/recipes-devtools/ruby/ruby/prevent-gc.patch 
b/meta/recipes-devtools/ruby/ruby/prevent-gc.patch
new file mode 100644
index 000..729710f2df1
--- /dev/null
+++ b/meta/recipes-devtools/ruby/ruby/prevent-gc.patch
@@ -0,0 +1,32 @@
+Fix marshalling with gcc7. Based on upstream revision 57410:
+https://svn.ruby-lang.org/cgi-bin/viewvc.cgi?view=revision=57410
+https://github.com/ruby/ruby/commit/7c1b30a602ab109d8d5388d7dfb3c5b180ba24e1
+https://bugs.ruby-lang.org/issues/13150
+
+with the upstream patches intent ported to Ruby 2.2.5
+
+Upstream-Status: Backport
+
+Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
+
+Index: ruby-2.2.5/marshal.c
+===
+--- ruby-2.2.5.orig/marshal.c
 ruby-2.2.5/marshal.c
+@@ -17,7 +17,6 @@
+ #include "ruby/io.h"
+ #include "ruby/st.h"
+ #include "ruby/util.h"
+-
+ #include 
+ #ifdef HAVE_FLOAT_H
+ #include 
+@@ -985,7 +984,7 @@ marshal_dump(int argc, VALUE *argv)
+ VALUE obj, port, a1, a2;
+ int limit = -1;
+ struct dump_arg *arg;
+-VALUE wrapper; /* used to avoid memory leak in case of exception */
++volatile VALUE wrapper; /* used to avoid memory leak in case of exception 
*/
+ 
+ port = Qnil;
+ rb_scan_args(argc, argv, "12", , , );
diff --git a/meta/recipes-devtools/ruby/ruby_2.2.5.bb 
b/meta/recipes-devtools/ruby/ruby_2.2.5.bb
index 5a645821ba0..66ba1d4b844 100644
--- a/meta/recipes-devtools/ruby/ruby_2.2.5.bb
+++ b/meta/recipes-devtools/ruby/ruby_2.2.5.bb
@@ -3,6 +3,8 @@ require ruby.inc
 SRC_URI[md5sum] = "bd8e349d4fb2c75d90817649674f94be"
 SRC_URI[sha256sum] = 
"30c4b31697a4ca4ea0c8db8ad30cf45e6690a0f09687e5d483c933c03ca335e3"
 
+SRC_URI += "file://prevent-gc.patch"
+
 # it's unknown to configure script, but then passed to extconf.rb
 # maybe it's not really needed as we're hardcoding the result with
 # 0001-socket-extconf-hardcode-wide-getaddr-info-test-outco.patch
-- 
2.13.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH][morty] ruby: fix build of ruby-native with gcc7

[Joshua Lock]

Marsalling is broken when ruby-2.2.x is built with gcc7, backport the change
fix  in Ruby SVN r57410 to apply to ruby 2.2.5:

https://svn.ruby-lang.org/cgi-bin/viewvc.cgi?view=revision=57410

Fixes [YOCTO #12271]

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/recipes-devtools/ruby/ruby/prevent-gc.patch | 32 
 meta/recipes-devtools/ruby/ruby_2.2.5.bb |  2 ++
 2 files changed, 34 insertions(+)
 create mode 100644 meta/recipes-devtools/ruby/ruby/prevent-gc.patch

diff --git a/meta/recipes-devtools/ruby/ruby/prevent-gc.patch 
b/meta/recipes-devtools/ruby/ruby/prevent-gc.patch
new file mode 100644
index 000..2eaa955fba2
--- /dev/null
+++ b/meta/recipes-devtools/ruby/ruby/prevent-gc.patch
@@ -0,0 +1,32 @@
+Fix marshaling with gcc7. Based on upstream revision 57410:
+https://svn.ruby-lang.org/cgi-bin/viewvc.cgi?view=revision=57410
+https://github.com/ruby/ruby/commit/7c1b30a602ab109d8d5388d7dfb3c5b180ba24e1
+https://bugs.ruby-lang.org/issues/13150
+
+with the upstream patches intent ported to Ruby 2.2.5
+
+Upstream-Status: Backport
+
+Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
+
+Index: ruby-2.2.5/marshal.c
+===
+--- ruby-2.2.5.orig/marshal.c
 ruby-2.2.5/marshal.c
+@@ -17,7 +17,6 @@
+ #include "ruby/io.h"
+ #include "ruby/st.h"
+ #include "ruby/util.h"
+-
+ #include 
+ #ifdef HAVE_FLOAT_H
+ #include 
+@@ -985,7 +984,7 @@ marshal_dump(int argc, VALUE *argv)
+ VALUE obj, port, a1, a2;
+ int limit = -1;
+ struct dump_arg *arg;
+-VALUE wrapper; /* used to avoid memory leak in case of exception */
++volatile VALUE wrapper; /* used to avoid memory leak in case of exception 
*/
+ 
+ port = Qnil;
+ rb_scan_args(argc, argv, "12", , , );
diff --git a/meta/recipes-devtools/ruby/ruby_2.2.5.bb 
b/meta/recipes-devtools/ruby/ruby_2.2.5.bb
index 5a645821ba0..66ba1d4b844 100644
--- a/meta/recipes-devtools/ruby/ruby_2.2.5.bb
+++ b/meta/recipes-devtools/ruby/ruby_2.2.5.bb
@@ -3,6 +3,8 @@ require ruby.inc
 SRC_URI[md5sum] = "bd8e349d4fb2c75d90817649674f94be"
 SRC_URI[sha256sum] = 
"30c4b31697a4ca4ea0c8db8ad30cf45e6690a0f09687e5d483c933c03ca335e3"
 
+SRC_URI += "file://prevent-gc.patch"
+
 # it's unknown to configure script, but then passed to extconf.rb
 # maybe it's not really needed as we're hardcoding the result with
 # 0001-socket-extconf-hardcode-wide-getaddr-info-test-outco.patch
-- 
2.13.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] devtool: ensure temp directory is created

[Joshua Lock]

On 17/10/17 17:12, Burton, Ross wrote:
On 17 October 2017 at 17:02, Joshua Lock <joshua.g.l...@linux.intel.com 
<mailto:joshua.g.l...@linux.intel.com>> wrote:


FWIW the failure has been observed on multiple host distros (Ubuntu
16.04, Fedora 26, Tumbleweed) and I could replicate it very easily
on my Fedora 26 host with an use of devtool modify -x.


If you can replicate it on demand how about using a tracing tool to 
figure out if a) the mkdir doesn't actually happen or b) something 
deletes the directory.


It seems it's not as easy to reproduce as I thought. I can't reproduce 
it at all today on either of my two systems.


I will attempt to gather some logs next time we see it on the autobuilder.

Joshua
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] devtool: ensure temp directory is created

[Joshua Lock]

On 17/10/17 17:00, Leonardo Sandoval wrote:

On Tue, 17 Oct 2017 16:05:06 +0100
Joshua Lock <joshua.g.l...@intel.com> wrote:

I would be nice that we keep the bugzilla ticket open (and included the tag on 
this patch) and investigate further  this issue. Seems like in some distros, 
the temfile.mkdtemp is not doing what it is supposed thus we have this 
workaround (perhaps strace may lead the real reason)


Agreed. I meant to write more before sending this patch (and copy Paul). 
 It's clearly not the right fix as the tempfile.mkdtemp() call should 
create the directory, but I wanted to share my simple findings (that the 
directory doesn't appear to be reliably created by the mkdtemp call, as 
expected).


FWIW the failure has been observed on multiple host distros (Ubuntu 
16.04, Fedora 26, Tumbleweed) and I could replicate it very easily on my 
Fedora 26 host with an use of devtool modify -x.


Joshua




We often see nightly-oe-selftest on the Yocto Project Autobuilder fail in
test_devtool_modify_native with:

  'devtool modify apt-native -x /tmp/devtoolqahlptvsfq/apt-native' returned
  non-zero exit status 1

and the following traceback:

Traceback (most recent call last):
   File 
"/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/scripts/devtool",
 line 331, in 
 ret = main()
   File 
"/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/scripts/devtool",
 line 318, in main
 ret = args.func(args, config, basepath, workspace)
   File 
"/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/scripts/lib/devtool/standard.py",
 line 714, in modify
 initial_rev = _extract_source(srctree, args.keep_temp, args.branch, False, 
config, basepath, workspace, args.fixed_setup, rd, tinfoil)
   File 
"/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/scripts/lib/devtool/standard.py",
 line 545, in _extract_source
 with open(os.path.join(tempdir, 'initial_rev'), 'r') as f:

Ensure the tempdir is created before we try and create any files there.

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
  scripts/lib/devtool/standard.py | 1 +
  1 file changed, 1 insertion(+)

diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index beea0d4c274..38197251b42 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -494,6 +494,7 @@ def _extract_source(srctree, keep_temp, devbranch, sync, 
config, basepath, works
  tempbasedir = d.getVar('WORKDIR')
  bb.utils.mkdirhier(tempbasedir)
  tempdir = tempfile.mkdtemp(prefix='devtooltmp-', dir=tempbasedir)
+bb.utils.mkdirhier(tempdir)
  try:
  tinfoil.logger.setLevel(logging.WARNING)
  
--

2.13.6

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core




--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] devtool: ensure temp directory is created

[Joshua Lock]

We often see nightly-oe-selftest on the Yocto Project Autobuilder fail in
test_devtool_modify_native with:

 'devtool modify apt-native -x /tmp/devtoolqahlptvsfq/apt-native' returned
 non-zero exit status 1

and the following traceback:

Traceback (most recent call last):
  File 
"/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/scripts/devtool",
 line 331, in 
ret = main()
  File 
"/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/scripts/devtool",
 line 318, in main
ret = args.func(args, config, basepath, workspace)
  File 
"/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/scripts/lib/devtool/standard.py",
 line 714, in modify
initial_rev = _extract_source(srctree, args.keep_temp, args.branch, False, 
config, basepath, workspace, args.fixed_setup, rd, tinfoil)
  File 
"/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/scripts/lib/devtool/standard.py",
 line 545, in _extract_source
with open(os.path.join(tempdir, 'initial_rev'), 'r') as f:

Ensure the tempdir is created before we try and create any files there.

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 scripts/lib/devtool/standard.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index beea0d4c274..38197251b42 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -494,6 +494,7 @@ def _extract_source(srctree, keep_temp, devbranch, sync, 
config, basepath, works
 tempbasedir = d.getVar('WORKDIR')
 bb.utils.mkdirhier(tempbasedir)
 tempdir = tempfile.mkdtemp(prefix='devtooltmp-', dir=tempbasedir)
+bb.utils.mkdirhier(tempdir)
 try:
 tinfoil.logger.setLevel(logging.WARNING)
 
-- 
2.13.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] lib/oeqa/selftest/cases/gotoolchain: add selftest for the Go toolchain

[Joshua Lock]

Add a simple test case to being testing of the Go toolchain:
1) build meta-go-toolchain
2) create a temp directory and install the generated Go toolchain within
3) fetch an archive of the Go Dep tool
4) create an appropriately laid out GOROOT and inflate the dep archive there
5) build the dep command with the SDK's Go toolchain and check it returned
   successfully.

[YOCTO #12152]

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/lib/oeqa/selftest/cases/gotoolchain.py | 67 +
 1 file changed, 67 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/cases/gotoolchain.py

diff --git a/meta/lib/oeqa/selftest/cases/gotoolchain.py 
b/meta/lib/oeqa/selftest/cases/gotoolchain.py
new file mode 100644
index 000..1e23257f4de
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/gotoolchain.py
@@ -0,0 +1,67 @@
+import glob
+import os
+import shutil
+import tempfile
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import runCmd, bitbake, get_bb_vars
+
+
+class oeGoToolchainSelfTest(OESelftestTestCase):
+"""
+Test cases for OE's Go toolchain
+"""
+
+@staticmethod
+def get_sdk_environment(tmpdir_SDKQA):
+pattern = os.path.join(tmpdir_SDKQA, "environment-setup-*")
+# FIXME: this is a very naive implementation
+return glob.glob(pattern)[0]
+
+@staticmethod
+def get_sdk_toolchain():
+bb_vars = get_bb_vars(['SDK_DEPLOY', 'TOOLCHAIN_OUTPUTNAME'],
+  "meta-go-toolchain")
+sdk_deploy = bb_vars['SDK_DEPLOY']
+toolchain_name = bb_vars['TOOLCHAIN_OUTPUTNAME']
+return os.path.join(sdk_deploy, toolchain_name + ".sh")
+
+@classmethod
+def setUpClass(cls):
+super(oeGoToolchainSelfTest, cls).setUpClass()
+cls.tmpdir_SDKQA = tempfile.mkdtemp(prefix='SDKQA')
+cls.go_path = os.path.join(cls.tmpdir_SDKQA, "go")
+# Build the SDK and locate it in DEPLOYDIR
+bitbake("meta-go-toolchain")
+cls.sdk_path = oeGoToolchainSelfTest.get_sdk_toolchain()
+# Install the SDK into the tmpdir
+runCmd("sh %s -y -d \"%s\"" % (cls.sdk_path, cls.tmpdir_SDKQA))
+cls.env_SDK = 
oeGoToolchainSelfTest.get_sdk_environment(cls.tmpdir_SDKQA)
+
+@classmethod
+def tearDownClass(cls):
+shutil.rmtree(cls.tmpdir_SDKQA, ignore_errors=True)
+super(oeGoToolchainSelfTest, cls).tearDownClass()
+
+def run_sdk_go_command(self, gocmd):
+cmd = "cd %s; " % self.tmpdir_SDKQA
+cmd = cmd + ". %s; " % self.env_SDK
+cmd = cmd + "export GOPATH=%s; " % self.go_path
+cmd = cmd + "${CROSS_COMPILE}go %s" % gocmd
+return runCmd(cmd).status
+
+def test_go_dep_build(self):
+proj = "github.com/golang"
+name = "dep"
+ver = "v0.3.1"
+archive = ".tar.gz"
+url = "https://%s/%s/archive/%s%s; % (proj, name, ver, archive)
+
+runCmd("cd %s; wget %s" % (self.tmpdir_SDKQA, url))
+runCmd("cd %s; tar -xf %s" % (self.tmpdir_SDKQA, ver+archive))
+runCmd("mkdir -p %s/src/%s" % (self.go_path, proj))
+runCmd("mv %s/dep-0.3.1 %s/src/%s/%s"
+   % (self.tmpdir_SDKQA, self.go_path, proj, name))
+retv = self.run_sdk_go_command('build  %s/%s/cmd/dep'
+   % (proj, name))
+self.assertEqual(retv, 0,
+ msg="Running go build failed for %s" % name)
-- 
2.13.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] oe-build-perf-report-email.py: add cc and bcc options

[Joshua Lock]

Enable carbon copy and blind carbon copy recipients for the performance
report emails.

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 scripts/contrib/oe-build-perf-report-email.py | 17 +++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/scripts/contrib/oe-build-perf-report-email.py 
b/scripts/contrib/oe-build-perf-report-email.py
index 261ca514e5a..64e85c26add 100755
--- a/scripts/contrib/oe-build-perf-report-email.py
+++ b/scripts/contrib/oe-build-perf-report-email.py
@@ -71,6 +71,10 @@ def parse_args(argv):
 help="Only print errors")
 parser.add_argument('--to', action='append',
 help="Recipients of the email")
+parser.add_argument('--cc', action='append',
+help="Carbon copy recipients of the email")
+parser.add_argument('--bcc', action='append',
+help="Blind carbon copy recipients of the email")
 parser.add_argument('--subject', default="Yocto build perf test report",
 help="Email subject")
 parser.add_argument('--outdir', '-o',
@@ -188,7 +192,7 @@ def scrape_html_report(report, outdir, 
phantomjs_extra_args=None):
 finally:
 shutil.rmtree(tmpdir)
 
-def send_email(text_fn, html_fn, subject, recipients):
+def send_email(text_fn, html_fn, subject, recipients, copy=[], blind_copy=[]):
 """Send email"""
 # Generate email message
 text_msg = html_msg = None
@@ -217,6 +221,10 @@ def send_email(text_fn, html_fn, subject, recipients):
'{}@{}'.format(pw_data.pw_name, socket.getfqdn()))
 msg['From'] = "{} <{}>".format(full_name, email)
 msg['To'] = ', '.join(recipients)
+if copy:
+msg['Cc'] = ', '.join(copy)
+if blind_copy:
+msg['Bcc'] = ', '.join(blind_copy)
 msg['Subject'] = subject
 
 # Send email
@@ -250,7 +258,12 @@ def main(argv=None):
 
 if args.to:
 log.info("Sending email to %s", ', '.join(args.to))
-send_email(args.text, html_report, args.subject, args.to)
+if args.cc:
+log.info("Copying to %s", ', '.join(args.cc))
+if args.bcc:
+log.info("Blind copying to %s", ', '.join(args.bcc))
+send_email(args.text, html_report, args.subject,
+   args.to, args.cc, args.bcc)
 except subprocess.CalledProcessError as err:
 log.error("%s, with output:\n%s", str(err), err.output.decode())
 return 1
-- 
2.13.5

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/2] package_[deb|ipk]: improve multiprocess logic when creating deb/ipk packages

[Joshua Lock]

On 04/09/17 22:35, leonardo.sandoval.gonza...@linux.intel.com wrote:

From: Leonardo Sandoval 

Current implementation does not handle possible exceptions coming from child
processes, the latter responsible for creating packages. With the aim to have 
more
control, use pipes to communicate exceptions and stop package creation in case
of failure.

Helps to debug [YOCTO #12012].

Signed-off-by: Leonardo Sandoval 
---
  meta/classes/package_deb.bbclass | 36 +---
  meta/classes/package_ipk.bbclass | 36 +---
  2 files changed, 66 insertions(+), 6 deletions(-)






diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
index ec90996184..8439cda6dd 100644
--- a/meta/classes/package_ipk.bbclass
+++ b/meta/classes/package_ipk.bbclass
@@ -17,7 +17,29 @@ OPKG_ARGS += "${@['', '--add-exclude ' + ' --add-exclude 
'.join((d.getVar('PACKA
  OPKGLIBDIR = "${localstatedir}/lib"
  
  python do_package_ipk () {

-from multiprocessing import Process
+import multiprocessing
+import traceback
+
+class IPKWritePkgProcess(multiprocessing.Process):
+def __init__(self, *args, **kwargs):
+multiprocessing.Process.__init__(self, *args, **kwargs)
+self._pconn, self._cconn = multiprocessing.Pipe()
+self._exception = None
+
+def run(self):
+try:
+multiprocessing.Process.run(self)
+self._cconn.send(None)
+except Exception as e:
+tb = traceback.format_exc()
+self._cconn.send((e, tb))
+
+@property
+def exception(self):
+if self._pconn.poll():
+self._exception = self._pconn.recv()
+return self._exception
+


Other than the name is IPKWritePkgProcess the same as DebianWritePkgProcess?

If so, could we put a single WritePkgProcess class somewhere it can be 
shared in lib/oe ?


Joshua
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] oeqa/buildperf: Add 'bitbake -m' on sync function to ensure bitbake is unloaded

[Joshua Lock]

On 25/07/2017 20:42, jose.perez.carra...@linux.intel.com wrote:

From: Jose Perez Carranza 

Add 'bitbake -m' to the sync method and ensure all process related to
bitbake are correctly unloaded before doing the different measurements.
Also add a call to sync funtion on Test4 before final measurment of
eSDK deploy dir disk usage.

Signed-off-by: Jose Perez Carranza 
---
  meta/lib/oeqa/buildperf/base.py   | 1 +
  meta/lib/oeqa/buildperf/test_basic.py | 2 ++
  2 files changed, 3 insertions(+)

diff --git a/meta/lib/oeqa/buildperf/base.py b/meta/lib/oeqa/buildperf/base.py
index 6e62b279c1..7b2b4aa2a4 100644
--- a/meta/lib/oeqa/buildperf/base.py
+++ b/meta/lib/oeqa/buildperf/base.py
@@ -485,6 +485,7 @@ class BuildPerfTestCase(unittest.TestCase):
  @staticmethod
  def sync():
  """Sync and drop kernel caches"""
+runCmd2('bitbake -m', ignore_status=True)


Looks like the indentation is out here?


  log.debug("Syncing and dropping kernel caches""")
  KernelDropCaches.drop()
  os.sync()
diff --git a/meta/lib/oeqa/buildperf/test_basic.py 
b/meta/lib/oeqa/buildperf/test_basic.py
index a9e4a5b731..0033ee4815 100644
--- a/meta/lib/oeqa/buildperf/test_basic.py
+++ b/meta/lib/oeqa/buildperf/test_basic.py
@@ -121,5 +121,7 @@ class Test4(BuildPerfTestCase):
  self.sync()
  self.measure_cmd_resources([installer, '-y', '-d', deploy_dir],
 'deploy', 'eSDK deploy')
+#make sure bitbake is unloaded
+#self.sync()


Is this sync() call supposed to be commented out?

Thanks,

Joshua


  self.measure_disk_usage(deploy_dir, 'deploy_dir', 'deploy dir',
  apparent_size=True)


--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/2] x86-base: build only hddimg, rather than iso and hddimg, for x86 machines

[Joshua Lock]

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/conf/machine/include/x86-base.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/conf/machine/include/x86-base.inc 
b/meta/conf/machine/include/x86-base.inc
index bfacdf5..a29641a 100644
--- a/meta/conf/machine/include/x86-base.inc
+++ b/meta/conf/machine/include/x86-base.inc
@@ -10,7 +10,7 @@ MACHINE_FEATURES += "screen keyboard pci usbhost ext2 ext3 
x86 \
 
 MACHINE_EXTRA_RRECOMMENDS += "kernel-modules"
 
-IMAGE_FSTYPES ?= "live"
+IMAGE_FSTYPES ?= "hddimg"
 
 KERNEL_IMAGETYPE ?= "bzImage"
 
-- 
2.9.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/2] image_types: remove Hob convenience variable

[Joshua Lock]

IMAGE_EXTENSION_live is no longer used. It was used by Hob to map the
live image type, but Hob is no longer part of the Bitbake codebase.

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/classes/image_types.bbclass | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
index 92e0f2c..d04bb08 100644
--- a/meta/classes/image_types.bbclass
+++ b/meta/classes/image_types.bbclass
@@ -301,9 +301,6 @@ RUNNABLE_MACHINE_PATTERNS ?= "qemu"
 
 DEPLOYABLE_IMAGE_TYPES ?= "hddimg iso" 
 
-# Use IMAGE_EXTENSION_xxx to map image type 'xxx' with real image file 
extension name(s) for Hob
-IMAGE_EXTENSION_live = "hddimg iso"
-
 # The IMAGE_TYPES_MASKED variable will be used to mask out from the 
IMAGE_FSTYPES,
 # images that will not be built at do_rootfs time: vmdk, vdi, qcow2, 
hdddirect, hddimg, iso, etc.
 IMAGE_TYPES_MASKED ?= ""
-- 
2.9.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] update distro_alias to include unmap packages on other distro

[Joshua Lock]

Hi Shen Joon,

The commit message doesn't match the expected format of

"primary item changed: short summary

Long description"

as documented at:
https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines

If in doubt about how to describe the primary item changed git log is
invaluable, in this case we can see from a `git log
meta/conf/distro/include/distro_alias.inc` that commit messages for
changes to this file use "distro_alias.inc:" to describe the file.



On Tue, 2017-07-11 at 09:30 +0800, Tan Shen Joon wrote:
> From: Shen Joon Tan 

You should update your git config to use a real email address.

Thanks,

Joshua

> Added packages with different names or based on sources that
> generates multiple binaries eg: python-pycurl source generates
> python3-pycurl as well. Currently, some of these packages are not
> displayed as part of other distro due to naming differences on
> the source rather than the binaries as a single source could
> produce multiple binaries with different names.
> 
> Signed-off-by: Shen Joon Tan 
> ---
>  meta/conf/distro/include/distro_alias.inc | 7 +++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/meta/conf/distro/include/distro_alias.inc
> b/meta/conf/distro/include/distro_alias.inc
> index ba47f90..f7c8b4a 100644
> --- a/meta/conf/distro/include/distro_alias.inc
> +++ b/meta/conf/distro/include/distro_alias.inc
> @@ -62,6 +62,7 @@ DISTRO_PN_ALIAS_pn-core-image-testmaster = "OE-
> Core"
>  DISTRO_PN_ALIAS_pn-core-image-testmaster-initramfs = "OE-Core"
>  DISTRO_PN_ALIAS_pn-core-image-weston = "OE-Core"
>  DISTRO_PN_ALIAS_pn-core-image-x11 = "OE-Core"
> +DISTRO_PN_ALIAS_pn-createrepo-c = "Fedora=createrepo_c
> Clear=createrepo_c"
>  DISTRO_PN_ALIAS_pn-cross-localedef = "OSPDT"
>  DISTRO_PN_ALIAS_pn-cryptodev-linux = "OE-Core"
>  DISTRO_PN_ALIAS_pn-cryptodev-module = "OE-Core"
> @@ -128,6 +129,7 @@ DISTRO_PN_ALIAS_pn-gstreamer1.0-plugins-bad =
> "Debian=gstreamer1.0-plugins-bad U
>  DISTRO_PN_ALIAS_pn-gstreamer1.0-plugins-base = "Debian=gstreamer1.0-
> plugins-base Ubuntu=gstreamer1.0-plugins-base"
>  DISTRO_PN_ALIAS_pn-gstreamer1.0-plugins-good = "Debian=gstreamer1.0-
> plugins-good Ubuntu=gstreamer1.0-plugins-bad"
>  DISTRO_PN_ALIAS_pn-gstreamer1.0-rtsp-server = "Ubuntu=gstreamer0.10-
> rtsp Fedora=gstreamer-rtsp"
> +DISTRO_PN_ALIAS_pn-gstreamer1.0-vaapi = "Fedora=gstreamer1-vaapi
> Debian=gstreamer-vaapi Clear=gstreamer-vaapi"
>  DISTRO_PN_ALIAS_pn-gtk+ = "Meego=gtk2 Fedora=gtk2 OpenSuSE=gtk2
> Ubuntu=gtk+2.0 Mandriva=gtk+2.0 Debian=gtk+2.0"
>  DISTRO_PN_ALIAS_pn-gtk+3 = "Ubuntu=gtk+3.0 Debian=gtk+3.0
> Fedora=gtk3"
>  DISTRO_PN_ALIAS_pn-gtk-doc = "Fedora=gtk-doc Ubuntu=gtk-doc"
> @@ -193,6 +195,7 @@ DISTRO_PN_ALIAS_pn-libomxil = "OSPDT upstream=htt
> p://omxil.sourceforge.net/"
>  DISTRO_PN_ALIAS_pn-libowl = "Debian=owl OpenedHand"
>  DISTRO_PN_ALIAS_pn-libpam = "Meego=pam Fedora=pam OpenSuSE=pam
> Ubuntu=pam Mandriva=pam Debian=pam"
>  DISTRO_PN_ALIAS_pn-libpcre = "Mandriva=libpcre0 Fedora=pcre"
> +DISTRO_PN_ALIAS_pn-libpcre2 = "Fedora=pcre2 Debian=pcre2
> Clear=pcre2"
>  DISTRO_PN_ALIAS_pn-libpng12 = "Debian=libpng12-0 Fedora=libpng"
>  DISTRO_PN_ALIAS_pn-libpod-plainer-perl = "OSPDT"
>  DISTRO_PN_ALIAS_pn-libsamplerate0 = "Meego=libsamplerate
> Fedora=libsamplerate OpenSuSE=libsamplerate Ubuntu=libsamplerate
> Mandriva=libsamplerate Debian=libsamplerate"
> @@ -207,6 +210,7 @@ DISTRO_PN_ALIAS_pn-libusb1 = "Debian=libusb-1.0-0 
> Fedora=libusb1"
>  DISTRO_PN_ALIAS_pn-libusb-compat = "OSPDT"
>  DISTRO_PN_ALIAS_pn-libx11 = "Debian=libx11-6 Fedora=libX11
> Ubuntu=libx11-6 OpenSuSE=xorg-x11-libX11"
>  DISTRO_PN_ALIAS_pn-libxcalibrate = "OSPDT upstream=http://cgit.freed
> esktop.org/xorg/lib/libXCalibrate/"
> +DISTRO_PN_ALIAS_pn-libxfont2 = "Fedora=libXfont2 Clear=libXfont2"
>  DISTRO_PN_ALIAS_pn-libxft = "Mandriva=libxft Debian=libxft2
> Ubuntu=libxft2"
>  DISTRO_PN_ALIAS_pn-libxi = "Ubuntu=libxi Fedora=libXi"
>  DISTRO_PN_ALIAS_pn-libxkbcommon = "Fedora=libxkbcommon
> Debian=libxkbcommon"
> @@ -323,7 +327,10 @@ DISTRO_PN_ALIAS_pn-pulseaudio-client-conf-sato =
> "OE-Core"
>  DISTRO_PN_ALIAS_pn-puzzles = "Debian=sgt-puzzles Fedora=puzzles"
>  DISTRO_PN_ALIAS_pn-python3 = "Fedora=python3 Debian=python3.2"
>  DISTRO_PN_ALIAS_pn-python3-distribute = "Debian=python3-setuptools
> Fedora=python3-setuptools"
> +DISTRO_PN_ALIAS_pn-python3-iniparse = "Fedora=python-iniparse
> Debian=python-iniparse"
>  DISTRO_PN_ALIAS_pn-python3-pip = "OpenSuSE=python3-pip
> Debian=python3-pip"
> +DISTRO_PN_ALIAS_pn-python3-pycurl = "Fedora=python-pycurl
> Debian=pycurl"
> +DISTRO_PN_ALIAS_pn-python3-pygpgme = "Fedora=python-pygpgme
> Debian=pygpgme"
>  DISTRO_PN_ALIAS_pn-python3-setuptools = "OpenSuSE=python3-setuptools 
> Debian=python3-setuptools"
>  DISTRO_PN_ALIAS_pn-python-dbus = "Ubuntu=python-dbus Debian=python-
> dbus Mandriva=python-dbus"
>  DISTRO_PN_ALIAS_pn-python-distribute = "Opensuse=python-setuptools
> 

[OE-core] [PATCH 2/2] lib/oe/distro_check: drop Mandriva from create_distro_packages_list()

[Joshua Lock]

Mandriva is no longer maintained, with its last release having been in
2011. It's no longer useful as a yard-stick distro, therefore drop it from
distro_check.create_distro_packages_list()

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/lib/oe/distro_check.py | 8 
 1 file changed, 8 deletions(-)

diff --git a/meta/lib/oe/distro_check.py b/meta/lib/oe/distro_check.py
index 7a727dd..e775c3a 100644
--- a/meta/lib/oe/distro_check.py
+++ b/meta/lib/oe/distro_check.py
@@ -83,13 +83,6 @@ def get_latest_released_opensuse_source_package_list(d):
 package_names |= 
get_source_package_list_from_url("http://download.opensuse.org/update/leap/%s/oss/src/;
 % latest, "updates", d)
 return latest, package_names
 
-def get_latest_released_mandriva_source_package_list(d):
-"Returns list of all the name os packages in the latest mandriva distro"
-latest = 
find_latest_numeric_release("http://distrib-coffee.ipsl.jussieu.fr/pub/linux/MandrivaLinux/official/;,
 d)
-package_names = 
get_source_package_list_from_url("http://distrib-coffee.ipsl.jussieu.fr/pub/linux/MandrivaLinux/official/%s/SRPMS/main/release/;
 % latest, "main", d)
-package_names |= 
get_source_package_list_from_url("http://distrib-coffee.ipsl.jussieu.fr/pub/linux/MandrivaLinux/official/%s/SRPMS/main/updates/;
 % latest, "updates", d)
-return latest, package_names
-
 def get_latest_released_clear_source_package_list(d):
 latest = 
find_latest_numeric_release("https://download.clearlinux.org/releases/;, d)
 package_names = 
get_source_package_list_from_url("https://download.clearlinux.org/releases/%s/clear/source/SRPMS/;
 % latest, "main", d)
@@ -162,7 +155,6 @@ def create_distro_packages_list(distro_check_dir, d):
 ("Ubuntu", 
get_latest_released_ubuntu_source_package_list),
 ("Fedora", 
get_latest_released_fedora_source_package_list),
 ("openSUSE", 
get_latest_released_opensuse_source_package_list),
-("Mandriva", 
get_latest_released_mandriva_source_package_list),
 ("Clear", 
get_latest_released_clear_source_package_list),
)
 
-- 
2.9.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/2] lib/oe/distro_check: update openSUSE to use Leap url formats

[Joshua Lock]

Since openSUSE switched to their dual development model of Leap and
Tumbleweed the urls we need to check for package lists has changed. Update
get_latest_released_opensuse_source_package_list() to use the new Leap
urls.

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/lib/oe/distro_check.py | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/meta/lib/oe/distro_check.py b/meta/lib/oe/distro_check.py
index 37f04ed..7a727dd 100644
--- a/meta/lib/oe/distro_check.py
+++ b/meta/lib/oe/distro_check.py
@@ -77,10 +77,10 @@ def get_latest_released_fedora_source_package_list(d):
 
 def get_latest_released_opensuse_source_package_list(d):
 "Returns list of all the name os packages in the latest opensuse distro"
-latest = 
find_latest_numeric_release("http://download.opensuse.org/source/distribution/",d)
+latest = 
find_latest_numeric_release("http://download.opensuse.org/source/distribution/leap;,
 d)
 
-package_names = 
get_source_package_list_from_url("http://download.opensuse.org/source/distribution/%s/repo/oss/suse/src/;
 % latest, "main", d)
-package_names |= 
get_source_package_list_from_url("http://download.opensuse.org/update/%s/src/; 
% latest, "updates", d)
+package_names = 
get_source_package_list_from_url("http://download.opensuse.org/source/distribution/leap/%s/repo/oss/suse/src/;
 % latest, "main", d)
+package_names |= 
get_source_package_list_from_url("http://download.opensuse.org/update/leap/%s/oss/src/;
 % latest, "updates", d)
 return latest, package_names
 
 def get_latest_released_mandriva_source_package_list(d):
@@ -161,7 +161,7 @@ def create_distro_packages_list(distro_check_dir, d):
 ("Debian", 
get_latest_released_debian_source_package_list),
 ("Ubuntu", 
get_latest_released_ubuntu_source_package_list),
 ("Fedora", 
get_latest_released_fedora_source_package_list),
-("OpenSuSE", 
get_latest_released_opensuse_source_package_list),
+("openSUSE", 
get_latest_released_opensuse_source_package_list),
 ("Mandriva", 
get_latest_released_mandriva_source_package_list),
 ("Clear", 
get_latest_released_clear_source_package_list),
)
-- 
2.9.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 0/2] Multi-threaded RPM support

[Joshua Lock]

On Mon, 2017-06-05 at 15:14 +0300, Alexander Kanavin wrote:
> On 06/04/2017 05:15 PM, Richard Purdie wrote:
> 
> > https://autobuilder.yoctoproject.org/main/builders/nightly-x86/buil
> > ds/1
> > 163/steps/BuildImages/logs/stdio
> > 
> > https://autobuilder.yoctoproject.org/main/builders/nightly-x86-64/b
> > uild
> > s/1203/steps/BuildImages/logs/stdio
> > 
> > https://autobuilder.yoctoproject.org/main/builders/nightly-ipk/buil
> > ds/1
> > 112/steps/BuildImages/logs/stdio
> > 
> > I suspect the same issue causing failures in multiple places.
> 
> The patches as they are now require that the C compiler on the host
> is 
> recent; specifically that it defaults to C11 mode. I'm not sure when
> GCC 
> made the switch, but it works that way with GCC 6.3.0.
> 
> In any case, I'll rework the patches to use openmp, as requested by 
> upstream - this may however cause a different set of compatibility 
> problems with older GCC versions. What are the oldest host compiler 
> versions/distro versions that we still need to support in master?

On the Yocto Autobuilder clusters we have:
* centos7 / gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-11)
* debian8 / gcc (Debian 4.9.2-10) 4.9.2
* debian-testing / gcc (Debian 6.3.0-14) 6.3.0 20170415
* fedora24 / gcc (GCC) 6.3.1 20161221 (Red Hat 6.3.1-1)
* fedora25 / gcc (GCC) 6.3.1 20161221 (Red Hat 6.3.1-1)
* opensuse132 / gcc (SUSE Linux) 4.8.3 20140627 [gcc-4_8-branch
revision 212064]
* opensuse422 / gcc (SUSE Linux) 4.8.5
* ubuntu1604 / gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
* ubuntu1610 / gcc (Ubuntu 6.2.0-5ubuntu12) 6.2.0 20161005

We are aiming to replace openSUSE 13.2 (as it was discontinued in
January of this year) but CentOS 7 has a good long lifetime ahead of
it, therefore supporting it and RHEL 7 will require gcc 4.8.5 support.

Joshua
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [morty][PATCH 1/1] perf: add perf-feature for systemtap

[Joshua Lock]

Based on OE-Core rev: 8aafed5f4757550f2ff688e1b8981e5e8e6e3428 if systemtap
isn't disabled it's possible to race with systemtap headers being present
during autodetection and disappearing during the build:

tests/sdt.c:14:21: fatal error: sys/sdt.h: No such file or directory
 #include 

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/recipes-kernel/perf/perf.bb | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb
index 471023d..03ae446 100644
--- a/meta/recipes-kernel/perf/perf.bb
+++ b/meta/recipes-kernel/perf/perf.bb
@@ -66,6 +66,7 @@ SCRIPTING_DEFINES = 
"${@perf_feature_enabled('perf-scripting', '', 'NO_LIBPERL=1
 TUI_DEFINES = "${@perf_feature_enabled('perf-tui', '', 'NO_NEWT=1',d)}"
 LIBUNWIND_DEFINES = "${@perf_feature_enabled('perf-libunwind', '', 
'NO_LIBUNWIND=1 NO_LIBDW_DWARF_UNWIND=1',d)}"
 LIBNUMA_DEFINES = "${@perf_feature_enabled('perf-libnuma', '', 
'NO_LIBNUMA=1',d)}"
+SYSTEMTAP_DEFINES = "${@perf_feature_enabled('perf-systemtap', '', 'NO_SDT=1', 
d)}"
 
 # The LDFLAGS is required or some old kernels fails due missing
 # symbols and this is preferred than requiring patches to every old
@@ -83,7 +84,7 @@ EXTRA_OEMAKE = '\
 EXTRA_CFLAGS="-ldw" \
 perfexecdir=${libexecdir} \
 NO_GTK2=1 ${TUI_DEFINES} NO_DWARF=1 ${LIBUNWIND_DEFINES} \
-${SCRIPTING_DEFINES} ${LIBNUMA_DEFINES} \
+${SCRIPTING_DEFINES} ${LIBNUMA_DEFINES} ${SYSTEMTAP_DEFINES} \
 '
 
 EXTRA_OEMAKE += "\
-- 
2.9.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] kernel: predefine KBUILD_BUILD_USER and KBUILD_BUILD_HOST

[Joshua Lock]

By exporting KBUILD_BUILD_USER with a pre-defined value we improve the
reproducibility of the kernel and remove the requirement for whoami in the
HOSTTOOLS.
KBUILD_BUILD_HOST also helps improve the reproducibility of the kernel.

For more kernel reproducibility options see:
https://lwn.net/Articles/437864/

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/classes/kernel.bbclass | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 8954b28..7a134d5 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -130,6 +130,8 @@ PACKAGES_DYNAMIC += "^kernel-firmware-.*"
 
 export OS = "${TARGET_OS}"
 export CROSS_COMPILE = "${TARGET_PREFIX}"
+export KBUILD_BUILD_USER = "oe-user"
+export KBUILD_BUILD_HOST = "oe-host"
 
 KERNEL_PRIORITY ?= 
"${@int(d.getVar('PV').split('-')[0].split('+')[0].split('.')[0]) * 1 + \

int(d.getVar('PV').split('-')[0].split('+')[0].split('.')[1]) * 100 + \
-- 
2.9.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [morty] sanity.bbclass: fix logging of an error

[Joshua Lock]

From: Markus Lehtonen <markus.lehto...@linux.intel.com>

Fixes a crash in exception handler. All bb logging functions need an
string instances as arguments.

(From OE-Core master rev: 62b521722ae72bc87d599719b400dab771154fa7)

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
Signed-off-by: Ross Burton <ross.bur...@intel.com>
Signed-off-by: Richard Purdie <richard.pur...@linuxfoundation.org>
Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/classes/sanity.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/sanity.bbclass b/meta/classes/sanity.bbclass
index 7682ffb..9db3f1d 100644
--- a/meta/classes/sanity.bbclass
+++ b/meta/classes/sanity.bbclass
@@ -565,7 +565,7 @@ def sanity_check_conffiles(d):
 try:
 bb.build.exec_func(func, d, pythonexception=True)
 except NotImplementedError as e:
-bb.fatal(e)
+bb.fatal(str(e))
 d.setVar("BB_INVALIDCONF", True)
 
 def sanity_handle_abichanges(status, d):
-- 
2.9.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 1/1] cve-check.bbclass: make warning contain CVE IDs

[Joshua Lock]

On Tue, 2017-05-09 at 17:13 +0800, Chen Qi wrote:
> When warning users about unpatched CVE, we'd better put CVE IDs into
> the warning message, so that it would be more straight forward for
> the
> user to know which CVEs are not patched.
> 
> So instead of:
>   WARNING: gnutls-3.5.9-r0 do_cve_check: Found unpatched CVE, for
> more information check /path/to/workdir/cve/cve.log.
> We should have:
>   WARNING: gnutls-3.5.9-r0 do_cve_check: Found unpatched CVE (CVE-
> 2017-7869), for more information check /path/to/workdir/cve/cve.log.
> 
> Signed-off-by: Chen Qi 
> ---
>  meta/classes/cve-check.bbclass | 11 +++
>  1 file changed, 7 insertions(+), 4 deletions(-)
> 
> diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-
> check.bbclass
> index 0e4294f..496d744 100644
> --- a/meta/classes/cve-check.bbclass
> +++ b/meta/classes/cve-check.bbclass
> @@ -234,7 +234,8 @@ def cve_write_data(d, patched, unpatched,
> cve_data):
>  cve_file = d.getVar("CVE_CHECK_LOCAL_FILE")
>  nvd_link = "https://web.nvd.nist.gov/view/vuln/detail?vulnId=;
>  write_string = ""
> -first_alert = True
> +has_unpatched_cve = False
> +unpatched_cves = []
>  bb.utils.mkdirhier(d.getVar("CVE_CHECK_LOCAL_DIR"))
>  
>  for cve in sorted(cve_data):
> @@ -244,15 +245,17 @@ def cve_write_data(d, patched, unpatched,
> cve_data):
>  if cve in patched:
>  write_string += "CVE STATUS: Patched\n"
>  else:
> +unpatched_cves.append(cve)
>  write_string += "CVE STATUS: Unpatched\n"
> -if first_alert:
> -bb.warn("Found unpatched CVE, for more information
> check %s" % cve_file)
> -first_alert = False
> +has_unpatched_cve = True
>  write_string += "CVE SUMMARY: %s\n" %
> cve_data[cve]["summary"]
>  write_string += "CVSS v2 BASE SCORE: %s\n" %
> cve_data[cve]["score"]
>  write_string += "VECTOR: %s\n" % cve_data[cve]["vector"]
>  write_string += "MORE INFORMATION: %s%s\n\n" % (nvd_link,
> cve)
>  
> +if has_unpatched_cve:

There's no need for the has_unpatched_cve variable, you can just test
whether the unpatched_cves list is empty:

>>> foo = []
>>> bar = [1, 2, 3]
>>> if foo:
...   print("foo")
... 
>>> if bar:
...   print("bar")
... 
bar

Your conditional can just be:

  + if unpatched_cve:
> +bb.warn("Found unpatched CVE (%s), for more information
> check %s" % (" ".join(unpatched_cves),cve_file))
> +
>  with open(cve_file, "w") as f:
>  bb.note("Writing file %s with CVE information" % cve_file)
>  f.write(write_string)
> -- 
> 1.9.1
> 
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 0/4] Reproducible binaries

[Joshua Lock]

On Wed, 2017-04-26 at 19:50 +, Bystricky, Juro wrote:
> > I wasnted to say "passing the seed value" through some separate
> > variable
> > which is easy to override from somewhere else.
> > 
> > e.g.
> > export PRELINK_TIMESTAMP=`git log -1 --pretty=%ct `
> > 
> > in image-prelink.bbclass:prelink_image() is a bit difficult to
> > override.
> > 
> 
> I think it is best to keep all reproducible build related variables
> gathered together, 
> so bitbake.conf seems like a natural place (same place as the
> variable BUILD_REPRODUCIBLE_BINARIES),
> i.e something like:
> 
> PRELINK_TIMESTAMP ??=`git log -1 --pretty=%ct `
> 
> A single "general purpose" fallback timestamp variable (not specific
> to prelinking) would be my
> preference. Unfortunately, the timestamps may need to come in various
> different formats...

One option for a single general purpose timestamp variable is just to
use a static value. Guix[1] just set the value of SOURCE_DATE_EPOCH to
1[2].

Joshua

1. https://www.gnu.org/software/guix/
2. http://git.savannah.gnu.org/cgit/guix.git/tree/guix/build/gnu-build-
system.scm#n42
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] oeqa.utils.metadata: cope with invalid lines in os-release file

[Joshua Lock]

On Wed, 2017-04-26 at 17:39 +0300, Markus Lehtonen wrote:
> Don't crash if every line in /etc/os-release does not adhere to the
> expected "key=val" format. E.g. CentOS 7 has empty lines in the file.
> 
> Signed-off-by: Markus Lehtonen 
> ---
>  meta/lib/oeqa/utils/metadata.py | 6 --
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/meta/lib/oeqa/utils/metadata.py
> b/meta/lib/oeqa/utils/metadata.py
> index cb81155e54..d291ddb960 100644
> --- a/meta/lib/oeqa/utils/metadata.py
> +++ b/meta/lib/oeqa/utils/metadata.py
> @@ -20,8 +20,10 @@ def get_os_release():
>  return None
>  with open(os_release_file) as fobj:
>  for line in fobj:
> -key, value = line.split('=', 1)
> -data[key.strip().lower()] = value.strip().strip('"')
> +split = line.split('=', 1)
> +if len(split) == 2:
> +key, value = split
> +data[key.strip().lower()] = value.strip().strip('"')

We have a function to read os-release in oe.lsb, return_dict_osr().

It handles this situation slightly differently, but the real reason I'm
pointing this out is that it seems like we could factor out a shared
function to be used both here and in oe.lsb ?

Joshua
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] oeqa/sdk/cases/buildcpio: enable use of cached cpio from DL_DIR

[Joshua Lock]

All build project test cases will try to copy the requested source
artefacts from DL_DIR before fetching. This testcase is referencing bzipped
tarball whereas the recipe fetches a gzipped tarball.

Switch to fetching the gzipped tarball in the test case so that we're able
to use a cached tarball from DL_DIR

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/lib/oeqa/sdk/cases/buildcpio.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/sdk/cases/buildcpio.py 
b/meta/lib/oeqa/sdk/cases/buildcpio.py
index 11d629a..333dc7c 100644
--- a/meta/lib/oeqa/sdk/cases/buildcpio.py
+++ b/meta/lib/oeqa/sdk/cases/buildcpio.py
@@ -10,7 +10,7 @@ class BuildCpioTest(OESDKTestCase):
 dl_dir = self.td.get('DL_DIR', None)
 
 self.project = SDKBuildProject(self.tc.sdk_dir + "/cpio/", 
self.tc.sdk_env,
-"https://ftp.gnu.org/gnu/cpio/cpio-2.12.tar.bz2;,
+"https://ftp.gnu.org/gnu/cpio/cpio-2.12.tar.gz;,
 self.tc.sdk_dir, self.td['DATETIME'], dl_dir=dl_dir)
 self.project.download_archive()
 
-- 
2.9.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 0/2] Fixes for building on Fedora 26 (Alpha)

[Joshua Lock]

On Thu, 2017-04-06 at 06:50 -0700, Khem Raj wrote:
> On Thu, Apr 6, 2017 at 6:23 AM, Joshua Lock <joshua.g.l...@intel.com>
> wrote:
> > As Fedora 26 has a new gcc [1] and a new system pkg-config
> > implementation [2] I
> > wanted to do some testing on that host OS so that we might be able
> > to get some
> > fixes in before Pyro/2.3 is released.
> > 
> > The following series were the only changes required to start
> > building (I have
> > tested core-image-minimal, core-image-base and core-image-sato) on
> > Fedora 26.
> 
> gcc patch looks fine to me.

Thanks for the review.

> > 
> > 1. https://fedoraproject.org/wiki/Changes/GCC7
> > 2. https://fedoraproject.org/wiki/Changes/pkgconf_as_system_pkg-con
> > fig_implementation
> 
> I think we should think about making pkgconf as default too for OE in
> next release cycle.

I've filed that as a 2.4 enhancement bug:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=11308

Joshua
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/2] gcc-6.3: backport fix of check for empty string in ubsan.c

[Joshua Lock]

Building gcc-cross-initial with GCC7 on the host fails due to the
comparison of a pointer to an integer in ubsan_use_new_style_p, which
is forbidden by ISO C++:

ubsan.c:1474:23: error: ISO C++ forbids comparison between pointer and
integer [-fpermissive]
   || xloc.file == '\0' || xloc.file[0] == '\xff'

Backport the fix from upstream GCC to enable the build with GCC 7

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/recipes-devtools/gcc/gcc-6.3.inc  |  1 +
 .../gcc/gcc-6.3/ubsan-fix-check-empty-string.patch | 28 ++
 2 files changed, 29 insertions(+)
 create mode 100644 
meta/recipes-devtools/gcc/gcc-6.3/ubsan-fix-check-empty-string.patch

diff --git a/meta/recipes-devtools/gcc/gcc-6.3.inc 
b/meta/recipes-devtools/gcc/gcc-6.3.inc
index da7a083..71d0aff 100644
--- a/meta/recipes-devtools/gcc/gcc-6.3.inc
+++ b/meta/recipes-devtools/gcc/gcc-6.3.inc
@@ -84,6 +84,7 @@ SRC_URI = "\
 "
 BACKPORTS = "\
file://CVE-2016-6131.patch \
+   file://ubsan-fix-check-empty-string.patch \
 "
 SRC_URI[md5sum] = "677a7623c7ef6ab99881bc4e048debb6"
 SRC_URI[sha256sum] = 
"f06ae7f3f790fbf0f018f6d40e844451e6bc3b7bc96e128e63b09825c1f8b29f"
diff --git 
a/meta/recipes-devtools/gcc/gcc-6.3/ubsan-fix-check-empty-string.patch 
b/meta/recipes-devtools/gcc/gcc-6.3/ubsan-fix-check-empty-string.patch
new file mode 100644
index 000..c012719
--- /dev/null
+++ b/meta/recipes-devtools/gcc/gcc-6.3/ubsan-fix-check-empty-string.patch
@@ -0,0 +1,28 @@
+From 8db2cf6353c13f2a84cbe49b689654897906c499 Mon Sep 17 00:00:00 2001
+From: kyukhin <kyukhin@138bc75d-0d04-0410-961f-82ee72b054a4>
+Date: Sat, 3 Sep 2016 10:57:05 +
+Subject: [PATCH] gcc/  * ubsan.c (ubsan_use_new_style_p): Fix check for empty
+ string.
+
+git-svn-id: svn+ssh://gcc.gnu.org/svn/gcc/trunk@239971 
138bc75d-0d04-0410-961f-82ee72b054a4
+
+Upstream-Status: Backport
+Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
+
+---
+ gcc/ubsan.c   | 2 +-
+ 2 files changed, 5 insertions(+), 1 deletion(-)
+
+Index: gcc-6.3.0/gcc/ubsan.c
+===
+--- gcc-6.3.0.orig/gcc/ubsan.c
 gcc-6.3.0/gcc/ubsan.c
+@@ -1471,7 +1471,7 @@ ubsan_use_new_style_p (location_t loc)
+ 
+   expanded_location xloc = expand_location (loc);
+   if (xloc.file == NULL || strncmp (xloc.file, "\1", 2) == 0
+-  || xloc.file == '\0' || xloc.file[0] == '\xff'
++  || xloc.file[0] == '\0' || xloc.file[0] == '\xff'
+   || xloc.file[1] == '\xff')
+ return false;
+ 
-- 
2.9.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/2] elfutils: fix building elfutils-native with GCC7

[Joshua Lock]

Backport a fix from upstream for a -Wformat-truncation=2 warning
and implement a simple fix for a -Wimplicit-fallthrough warning.

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 .../elfutils-0.168/Fix_one_GCC7_warning.patch  | 44 ++
 .../elfutils/elfutils-0.168/fallthrough.patch  | 36 ++
 meta/recipes-devtools/elfutils/elfutils_0.168.bb   |  3 ++
 3 files changed, 83 insertions(+)
 create mode 100644 
meta/recipes-devtools/elfutils/elfutils-0.168/Fix_one_GCC7_warning.patch
 create mode 100644 
meta/recipes-devtools/elfutils/elfutils-0.168/fallthrough.patch

diff --git 
a/meta/recipes-devtools/elfutils/elfutils-0.168/Fix_one_GCC7_warning.patch 
b/meta/recipes-devtools/elfutils/elfutils-0.168/Fix_one_GCC7_warning.patch
new file mode 100644
index 000..d88f4eb
--- /dev/null
+++ b/meta/recipes-devtools/elfutils/elfutils-0.168/Fix_one_GCC7_warning.patch
@@ -0,0 +1,44 @@
+From 93c51144c3f664d4e9709da75a1d0fa00ea0fe95 Mon Sep 17 00:00:00 2001
+From: Mark Wielaard <m...@klomp.org>
+Date: Sun, 12 Feb 2017 21:51:34 +0100
+Subject: [PATCH] libasm: Fix one GCC7 -Wformat-truncation=2 warning.
+
+Make sure that if we have really lots of labels the tempsym doesn't get
+truncated because it is too small to hold the whole name.
+
+This doesn't enable -Wformat-truncation=2 or fix other "issues" pointed
+out by enabling this warning because there are currently some issues
+with it. https://gcc.gnu.org/bugzilla/show_bug.cgi?id=79448
+
+Signed-off-by: Mark Wielaard <m...@klomp.org>
+
+Upstream-Status: Backport 
(https://sourceware.org/git/?p=elfutils.git;a=commit;h=93c51144c3f664d4e9709da75a1d0fa00ea0fe95)
+Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
+
+---
+ libasm/ChangeLog| 6 +-
+ libasm/asm_newsym.c | 6 --
+ 2 files changed, 9 insertions(+), 3 deletions(-)
+
+Index: elfutils-0.168/libasm/asm_newsym.c
+===
+--- elfutils-0.168.orig/libasm/asm_newsym.c
 elfutils-0.168/libasm/asm_newsym.c
+@@ -1,5 +1,5 @@
+ /* Define new symbol for current position in given section.
+-   Copyright (C) 2002, 2005, 2016 Red Hat, Inc.
++   Copyright (C) 2002, 2005, 2016, 2017 Red Hat, Inc.
+This file is part of elfutils.
+Written by Ulrich Drepper <drep...@redhat.com>, 2002.
+ 
+@@ -44,7 +44,9 @@ AsmSym_t *
+ asm_newsym (AsmScn_t *asmscn, const char *name, GElf_Xword size,
+   int type, int binding)
+ {
+-#define TEMPSYMLEN 10
++/* We don't really expect labels with many digits, but in theory it could
++   be 10 digits (plus ".L" and a zero terminator).  */
++#define TEMPSYMLEN 13
+   char tempsym[TEMPSYMLEN];
+   AsmSym_t *result;
+ 
diff --git a/meta/recipes-devtools/elfutils/elfutils-0.168/fallthrough.patch 
b/meta/recipes-devtools/elfutils/elfutils-0.168/fallthrough.patch
new file mode 100644
index 000..b2623f9
--- /dev/null
+++ b/meta/recipes-devtools/elfutils/elfutils-0.168/fallthrough.patch
@@ -0,0 +1,36 @@
+GCC7 adds -Wimplicit-fallthrough to warn when a switch case falls through,
+however this causes warnings (which are promoted to errors) with the elfutils
+patches from Debian for mips and parisc, which use fallthrough's by design.
+
+Explicitly mark the intentional fallthrough switch cases with a comment to
+disable the warnings where the fallthrough behaviour is desired.
+
+Upstream-Status: Pending [debian]
+Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
+
+Index: elfutils-0.168/backends/parisc_retval.c
+===
+--- elfutils-0.168.orig/backends/parisc_retval.c
 elfutils-0.168/backends/parisc_retval.c
+@@ -166,7 +166,7 @@ parisc_return_value_location_ (Dwarf_Die
+ return nloc_intregpair;
+ 
+   /* Else fall through.  */
+-  }
++  } // fallthrough
+ 
+ case DW_TAG_structure_type:
+ case DW_TAG_class_type:
+Index: elfutils-0.168/backends/mips_retval.c
+===
+--- elfutils-0.168.orig/backends/mips_retval.c
 elfutils-0.168/backends/mips_retval.c
+@@ -387,7 +387,7 @@ mips_return_value_location (Dwarf_Die *f
+   else
+ return nloc_intregpair;
+ }
+-}
++} // fallthrough
+ 
+   /* Fallthrough to handle large types */
+ 
diff --git a/meta/recipes-devtools/elfutils/elfutils_0.168.bb 
b/meta/recipes-devtools/elfutils/elfutils_0.168.bb
index 9557933..3b8f2a3 100644
--- a/meta/recipes-devtools/elfutils/elfutils_0.168.bb
+++ b/meta/recipes-devtools/elfutils/elfutils_0.168.bb
@@ -17,6 +17,7 @@ SRC_URI += "\
 file://0001-remove-the-unneed-checking.patch \
 file://0001-fix-a-stack-usage-warning.patch \
 file://aarch64_uio.patch \
+file://Fix_one_GCC7_warning.patch \
 file://shadow.patch \
 "
 
@@ -35,6 +36,8 @@ SRC_URI += "\
 file://debian/hu

[OE-core] [PATCH 0/2] Fixes for building on Fedora 26 (Alpha)

[Joshua Lock]

As Fedora 26 has a new gcc [1] and a new system pkg-config implementation [2] I
wanted to do some testing on that host OS so that we might be able to get some
fixes in before Pyro/2.3 is released.

The following series were the only changes required to start building (I have 
tested core-image-minimal, core-image-base and core-image-sato) on Fedora 26.

1. https://fedoraproject.org/wiki/Changes/GCC7
2. 
https://fedoraproject.org/wiki/Changes/pkgconf_as_system_pkg-config_implementation

The following changes since commit 633ad6c9f436f5d2b6ee1a005b697661a054a394:

  oeqa/runtime/utils/targetbuildproject: use parent classes defaults tmpdir 
(2017-04-06 10:13:39 +0100)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib joshuagl/fedora26
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=joshuagl/fedora26

Joshua Lock (2):
  elfutils: fix building elfutils-native with GCC7
  gcc-6.3: backport fix of check for empty string in ubsan.c

 .../elfutils-0.168/Fix_one_GCC7_warning.patch  | 44 ++
 .../elfutils/elfutils-0.168/fallthrough.patch  | 36 ++
 meta/recipes-devtools/elfutils/elfutils_0.168.bb   |  3 ++
 meta/recipes-devtools/gcc/gcc-6.3.inc  |  1 +
 .../gcc/gcc-6.3/ubsan-fix-check-empty-string.patch | 28 ++
 5 files changed, 112 insertions(+)
 create mode 100644 
meta/recipes-devtools/elfutils/elfutils-0.168/Fix_one_GCC7_warning.patch
 create mode 100644 
meta/recipes-devtools/elfutils/elfutils-0.168/fallthrough.patch
 create mode 100644 
meta/recipes-devtools/gcc/gcc-6.3/ubsan-fix-check-empty-string.patch

-- 
2.9.3
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/3] oeqa/utils/targetbuild: tmp dir improvements

[Joshua Lock]

Don't hard-code /tmp as the tmpdir, instead use WORKDIR as the tmpdir if the
instantiater doesn't specify a value.

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/lib/oeqa/utils/targetbuild.py | 10 +++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/meta/lib/oeqa/utils/targetbuild.py 
b/meta/lib/oeqa/utils/targetbuild.py
index 6f237b5..9249fa2 100644
--- a/meta/lib/oeqa/utils/targetbuild.py
+++ b/meta/lib/oeqa/utils/targetbuild.py
@@ -8,14 +8,19 @@ import os
 import re
 import bb.utils
 import subprocess
+import tempfile
 from abc import ABCMeta, abstractmethod
 
 class BuildProject(metaclass=ABCMeta):
 
-def __init__(self, d, uri, foldername=None, tmpdir="/tmp/"):
+def __init__(self, d, uri, foldername=None, tmpdir=None):
 self.d = d
 self.uri = uri
 self.archive = os.path.basename(uri)
+if not tmpdir:
+tmpdir = self.d.getVar('WORKDIR')
+if not tmpdir:
+tmpdir = tempfile.mkdtemp(prefix='buildproject')
 self.localarchive = os.path.join(tmpdir,self.archive)
 if foldername:
 self.fname = foldername
@@ -24,7 +29,6 @@ class BuildProject(metaclass=ABCMeta):
 
 # Download self.archive to self.localarchive
 def _download_archive(self):
-
 dl_dir = self.d.getVar("DL_DIR")
 if dl_dir and os.path.exists(os.path.join(dl_dir, self.archive)):
 bb.utils.copyfile(os.path.join(dl_dir, self.archive), 
self.localarchive)
@@ -73,7 +77,7 @@ class TargetBuildProject(BuildProject):
 def __init__(self, target, d, uri, foldername=None):
 self.target = target
 self.targetdir = "~/"
-BuildProject.__init__(self, d, uri, foldername, tmpdir="/tmp")
+BuildProject.__init__(self, d, uri, foldername)
 
 def download_archive(self):
 
-- 
2.9.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/3] oeqa/runtime/utils/targetbuildproject: use parent classes defaults tmpdir

[Joshua Lock]

Rather than hard-coding the tmpdir for TargetBuildProject to /tmp allow the
parent's default handling to define an appropriate tmpdir.

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/lib/oeqa/runtime/utils/targetbuildproject.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/meta/lib/oeqa/runtime/utils/targetbuildproject.py 
b/meta/lib/oeqa/runtime/utils/targetbuildproject.py
index 551b0b6..5af55d7 100644
--- a/meta/lib/oeqa/runtime/utils/targetbuildproject.py
+++ b/meta/lib/oeqa/runtime/utils/targetbuildproject.py
@@ -8,8 +8,7 @@ class TargetBuildProject(BuildProject):
 def __init__(self, target, uri, foldername=None, dl_dir=None):
 self.target = target
 self.targetdir = "~/"
-BuildProject.__init__(self, uri, foldername, tmpdir="/tmp",
-dl_dir=dl_dir)
+BuildProject.__init__(self, uri, foldername, dl_dir=dl_dir)
 
 def download_archive(self):
 self._download_archive()
-- 
2.9.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/3] oeqa/utils/buildproject: create a more unique tmp dir

[Joshua Lock]

Rather than hardcoding /tmp as the default tmpdir make a more unique tmpdir
with tempfile.mkdtemp() when the caller doesn't specify a tmpdir value.

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/lib/oeqa/utils/buildproject.py | 7 +--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oeqa/utils/buildproject.py 
b/meta/lib/oeqa/utils/buildproject.py
index fc8879c..487f08b 100644
--- a/meta/lib/oeqa/utils/buildproject.py
+++ b/meta/lib/oeqa/utils/buildproject.py
@@ -8,14 +8,17 @@ import os
 import re
 import subprocess
 import shutil
+import tempfile
 
 from abc import ABCMeta, abstractmethod
 
 class BuildProject(metaclass=ABCMeta):
-def __init__(self, uri, foldername=None, tmpdir="/tmp/", dl_dir=None):
+def __init__(self, uri, foldername=None, tmpdir=None, dl_dir=None):
 self.uri = uri
 self.archive = os.path.basename(uri)
-self.localarchive = os.path.join(tmpdir,self.archive)
+if not tmpdir:
+tmpdir = tempfile.mkdtemp(prefix='buildproject')
+self.localarchive = os.path.join(tmpdir, self.archive)
 self.dl_dir = dl_dir
 if foldername:
 self.fname = foldername
-- 
2.9.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] yocto-uninative: Update to the 1.6 release

[Joshua Lock]

This release includes fixes for Windows/Mingw support.

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/conf/distro/include/yocto-uninative.inc | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/conf/distro/include/yocto-uninative.inc 
b/meta/conf/distro/include/yocto-uninative.inc
index 7f9eaba..7c2151c 100644
--- a/meta/conf/distro/include/yocto-uninative.inc
+++ b/meta/conf/distro/include/yocto-uninative.inc
@@ -6,6 +6,6 @@
 # to the distro running on the build machine.
 #
 
-UNINATIVE_URL ?= "http://downloads.yoctoproject.org/releases/uninative/1.5/;
-UNINATIVE_CHECKSUM[i686] ?= 
"4cda652ba90f98e4c34e4c5a7e9f6364b51d3538228a4ffc24d0168fdf331369"
-UNINATIVE_CHECKSUM[x86_64] ?= 
"22f5a061d077379f5728197bfcc239aa087332293038df48535c7badcb6939fd"
+UNINATIVE_URL ?= "http://downloads.yoctoproject.org/releases/uninative/1.6/;
+UNINATIVE_CHECKSUM[i686] ?= 
"d9daf66785834f888b6f50a1d37a8120815f8380e0e267d311e1dfaf4bb5404e"
+UNINATIVE_CHECKSUM[x86_64] ?= 
"2b4fffa308d9f19e0742a1a404ff42495fb50c165e5ca0458cedca157372691a"
-- 
2.9.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/2] scripts: Add yocto-compat-layer-wrapper

[Joshua Lock]

On Thu, 2017-03-30 at 10:43 -0600, Aníbal Limón wrote:
> 
> On 03/30/2017 10:13 AM, Patrick Ohly wrote:
> > On Thu, 2017-03-30 at 10:03 -0600, Aníbal Limón wrote:
> > > 
> > > On 03/30/2017 12:02 AM, Patrick Ohly wrote:
> > > > On Wed, 2017-03-29 at 15:44 -0600, Aníbal Limón wrote:
> > > > ...
> > > > > +show_help() {
> > > > > + printf "Usage: %s [-o output_log] [-h] LAYER_DIR
> > > > > ...\n" $0
> > > > > +}
> > > > > +
> > > > 
> > > > ...
> > > > > +env_dir=$(mktemp -d -t yocto-compat-)
> > > > > +echo "The environment will be setup at $env_dir"
> > > > > +echo ""
> > > > 
> > > > The directory gets created, but not removed.
> > > 
> > > I didn't remove the temp directory because may be the user wants
> > > to
> > > access the dir after the check.
> > 
> > I think that this should be something that the user explicitly
> > needs to
> > request. As it is now, accumulating directories in /tmp when
> > invoking
> > the script is just unexpected and not the normal behavior of tools
> > creating something in /tmp.
> 
> Ok, i will add an option to don't delete and change the default
> behavior
> to delete after check.
> 
> 
> > 
> > > > > +echo "Cloning oe-core..."
> > > > > +git clone $oe_core_repo $env_dir
> > > > > +if [ $? -ne 0 ]; then
> > > > > + echo "Failed to clone oe-core repository"
> > > > > + exit 1
> > > > > +fi
> > > > > +
> > > > > +echo "Cloning bitbake..."
> > > > > +git clone $bitbake_repo $env_dir/bitbake
> > > > > +if [ $? -ne 0 ]; then
> > > > > + echo "Failed to clone bitbake repository"
> > > > > + exit 1
> > > > > +fi
> > > > 
> > > > Cloning bitbake and OE-core each time the script runs will be
> > > > fairly
> > > > slow. There's also a chicken-and-egg problem: if you don't have
> > > > bitbake,
> > > > where's the script?
> > > > 
> > > > I'd prefer to use an existing checkout of both, just as for the
> > > > layers
> > > > which are to be tested.
> > > 
> > > I choose to clone the oe-core/bitbake to ensure there are a clean
> > > environment, without any previous layer added.
> > 
> > I don't quite get that argument. When using existing bitbake and
> > OE-core
> > directories, how can they have layers added to them? I understand
> > that
> > the existing checkout might contain additional modifications and/or
> > might not match current master, but I consider that a feature. As
> > it
> > stands now, the caller of the script cannot test against Yocto 2.3
> > once
> > it is released, because the script will always check out master.
> > 
> > Control over that can of course also be added to the script, but I
> > just
> > don't see the need for all this additional complexity. Just my 2
> > cents.
> 
> That's the original issue to create this script, if a user tries to
> validate a layer and has modifications in configuration (bblayers,
> local, auto), the check will be contaminated in some how.

Those are all build-directory state, not items which are tracked in the
git repository.

Rather than cloning repositories to ensure clean state your script
could ensure a unique/new build directory by sourcing oe-init-build-env 
with a non-standard build directory, perhaps checking whether the
directory exists first and adding random characters until you get
something new?

Joshua
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v3, 1/2] lib/oe/sdk: Add get_extra_sdk_info to reuse code in buildhistory

[Joshua Lock]

On Tue, 2017-03-28 at 00:15 -0700, Francisco Pedraza wrote:
> This function is going to be used for generating the target and host
> manifest files packages for eSDK.
> [YOCTO #9038]
> 
> Signed-off-by: Francisco Pedraza  com>
> ---
>  meta/classes/buildhistory.bbclass | 16 
>  meta/lib/oe/sdk.py| 16 
>  2 files changed, 20 insertions(+), 12 deletions(-)
> 
> diff --git a/meta/classes/buildhistory.bbclass
> b/meta/classes/buildhistory.bbclass
> index 109b375..ab04db7 100644
> --- a/meta/classes/buildhistory.bbclass
> +++ b/meta/classes/buildhistory.bbclass
> @@ -554,25 +554,17 @@ END
>  python buildhistory_get_extra_sdkinfo() {
>  import operator
>  import math

I don't think we need to import math here anymore?

> +# importing oe.sdk due package information req.

No need for this comment.

> +from oe.sdk import get_extra_sdkinfo
>  
>  if d.getVar('BB_CURRENTTASK') == 'populate_sdk_ext' and \
>  "sdk" in (d.getVar('BUILDHISTORY_FEATURES') or
> "").split():
> -tasksizes = {}
> -filesizes = {}
> -for root, _, files in
> os.walk(d.expand('${SDK_OUTPUT}/${SDKPATH}/sstate-cache')):
> -for fn in files:
> -if fn.endswith('.tgz'):
> -fsize =
> int(math.ceil(float(os.path.getsize(os.path.join(root, fn))) / 1024))
> -task = fn.rsplit(':', 1)[1].split('_',
> 1)[1].split('.')[0]
> -origtotal = tasksizes.get(task, 0)
> -tasksizes[task] = origtotal + fsize
> -filesizes[fn] = fsize
>  with open(d.expand('${BUILDHISTORY_DIR_SDK}/sstate-package-
> sizes.txt'), 'w') as f:
> -filesizes_sorted = sorted(filesizes.items(),
> key=operator.itemgetter(1, 0), reverse=True)
> +filesizes_sorted =
> sorted(extra_info['filesizes'].items(), key=operator.itemgetter(1,
> 0), reverse=True)
>  for fn, size in filesizes_sorted:
>  f.write('%10d KiB %s\n' % (size, fn))
>  with open(d.expand('${BUILDHISTORY_DIR_SDK}/sstate-task-
> sizes.txt'), 'w') as f:
> -tasksizes_sorted = sorted(tasksizes.items(),
> key=operator.itemgetter(1, 0), reverse=True)
> + tasksizes_sorted =
> sorted(extra_info['tasksizes'].items(), key=operator.itemgetter(1,
> 0), reverse=True)

It looks like the indentation changes here?

>  for task, size in tasksizes_sorted:
>  f.write('%10d KiB %s\n' % (size, task))
>  }
> diff --git a/meta/lib/oe/sdk.py b/meta/lib/oe/sdk.py
> index deb823b..d101bf0 100644
> --- a/meta/lib/oe/sdk.py
> +++ b/meta/lib/oe/sdk.py
> @@ -372,5 +372,21 @@ def populate_sdk(d, manifest_dir=None):
>  os.environ.clear()
>  os.environ.update(env_bkp)
>  
> +def get_extra_sdk_info(sstate_dir):
> +import operator

I don't think we need operator here?

A docstring for this function would be nice.

> +import math
> +
> +extra_info = {}
> +extra_info['tasksizes']
> +for root, _, files in os.walk(sstate_dir):
> +for fn in files:
> +if fn.endswith('.tgz'):
> +fsize =
> int(math.ceil(float(os.path.getsize(os.path.join(root, fn))) / 1024))
> +task =
> fn.rsplit(':',1)[1].split('_',1)[1].split(',')[0]
> +origtotal = extra_info['tasksizes'].get(task, 0)
> +extra_info['tasksizes'][task] = origtotal + fsize
> +extra_info['filesizes'][fn] = fsize
> +return extra_info 
> +
>  if __name__ == "__main__":
>  pass
> -- 
> 1.8.3.1
> 
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] oeqa/selftest: don't use cleanall in selftests

[Joshua Lock]

* devtool: oeqa.selftest.devtool.DevtoolTests is taking 4 or more hours on the
  autobuilder because the cleanall call deletes the tarball of the kernel
  source, forcing a large clone which is especially long on an NFS mount.

  The cleanall invocation isn't really required here and switching to clean
  instead should avoid hours of re-cloning the same repository each time we
  run oe-selftest on the autobuilder.
* buildoptions: We only need to ensure the first image isn't an incremental
  image, clean will suffice here.
* runtime-test: no need to clean up the artefacts generated during the test,
  let's leave that to a higher level process rather than using the cleanall
  hammer.

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/lib/oeqa/selftest/buildoptions.py | 2 +-
 meta/lib/oeqa/selftest/devtool.py  | 2 +-
 meta/lib/oeqa/selftest/runtime-test.py | 6 --
 3 files changed, 2 insertions(+), 8 deletions(-)

diff --git a/meta/lib/oeqa/selftest/buildoptions.py 
b/meta/lib/oeqa/selftest/buildoptions.py
index 7ace747..d155f64 100644
--- a/meta/lib/oeqa/selftest/buildoptions.py
+++ b/meta/lib/oeqa/selftest/buildoptions.py
@@ -16,7 +16,7 @@ class ImageOptionsTests(oeSelfTest):
 image_pkgtype = get_bb_var("IMAGE_PKGTYPE")
 if image_pkgtype != 'rpm':
 self.skipTest('Not using RPM as main package format')
-bitbake("-c cleanall core-image-minimal")
+bitbake("-c clean core-image-minimal")
 self.write_config('INC_RPM_IMAGE_GEN = "1"')
 self.append_config('IMAGE_FEATURES += "ssh-server-openssh"')
 bitbake("core-image-minimal")
diff --git a/meta/lib/oeqa/selftest/devtool.py 
b/meta/lib/oeqa/selftest/devtool.py
index a160c83..7f9efe6 100644
--- a/meta/lib/oeqa/selftest/devtool.py
+++ b/meta/lib/oeqa/selftest/devtool.py
@@ -1654,7 +1654,7 @@ class DevtoolTests(DevtoolBase):
 
 tmpconfig = os.path.join(buildir, '.config')
 #Step 3
-bitbake('linux-yocto -c cleanall')
+bitbake('linux-yocto -c clean')
 #Step 4.1
 runCmd('devtool modify virtual/kernel -x %s' % tempdir)
 self.assertTrue(os.path.exists(os.path.join(tempdir, 'Makefile')),
diff --git a/meta/lib/oeqa/selftest/runtime-test.py 
b/meta/lib/oeqa/selftest/runtime-test.py
index ea31c18..e498d04 100644
--- a/meta/lib/oeqa/selftest/runtime-test.py
+++ b/meta/lib/oeqa/selftest/runtime-test.py
@@ -193,8 +193,6 @@ postinst-delayed-t \
present on rootfs dir.
 4. Boot the image created on qemu and verify that the 
file
created by postinst_boot recipe is present on image.
-5. Clean the packages and image created to test with
-   different package managers
 Expected:   The files are successfully created during rootfs and 
boot
 time for 3 different package managers: rpm,ipk,deb and
 for initialization managers: sysvinit and systemd.
@@ -237,7 +235,3 @@ postinst-delayed-t \
 sshargs = '-o UserKnownHostsFile=/dev/null -o 
StrictHostKeyChecking=no'
 result = runCmd('ssh %s root@%s %s' % (sshargs, qemu.ip, 
testcommand))
 self.assertEqual(result.status, 0, 'File %s was not 
created at firts boot'% fileboot_name)
-
-#Step 5
-bitbake(' %s %s -c cleanall' % (rootfs_pkg, boot_pkg))
-bitbake('core-image-minimal -c cleanall')
-- 
2.9.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] meta: replace uses of bb.data.expand(VARNAME, d) with d.expand(VARNAME)

[Joshua Lock]

bb.data.expand(x, d) is deprecated API.

[YOCTO #10678]

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/classes/archiver.bbclass  | 2 +-
 meta/classes/package.bbclass   | 4 ++--
 meta/classes/utility-tasks.bbclass | 2 +-
 meta/lib/oe/patch.py   | 2 +-
 meta/recipes-extended/pam/libpam_1.3.0.bb  | 2 +-
 meta/recipes-multimedia/alsa/alsa-plugins_1.1.1.bb | 2 +-
 6 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/meta/classes/archiver.bbclass b/meta/classes/archiver.bbclass
index ee4790d..2c04557 100644
--- a/meta/classes/archiver.bbclass
+++ b/meta/classes/archiver.bbclass
@@ -365,7 +365,7 @@ python do_ar_recipe () {
 elif include_re.match(line):
 incfile = include_re.match(line).group(1)
 if incfile:
-incfile = bb.data.expand(incfile, d)
+incfile = d.expand(incfile)
 incfile = bb.utils.which(bbpath, incfile)
 if incfile:
 shutil.copy(incfile, outdir)
diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index 871263f..b23458b 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -778,7 +778,7 @@ python fixup_perms () {
 dir = d.getVar(path) or ""
 if dir == "":
 continue
-fs_perms_table[dir] = fs_perms_entry(bb.data.expand("%s 0755 root root 
false - - -" % (dir), d))
+fs_perms_table[dir] = fs_perms_entry(d.expand("%s 0755 root root false 
- - -" % (dir)))
 
 # Now we actually load from the configuration files
 for conf in get_fs_perms_list(d).split():
@@ -1801,7 +1801,7 @@ python package_do_pkgconfig () {
 m = field_re.match(l)
 if m:
 hdr = m.group(1)
-exp = bb.data.expand(m.group(2), pd)
+exp = pd.expand(m.group(2))
 if hdr == 'Requires':
 pkgconfig_needed[pkg] += exp.replace(',', ' 
').split()
 
diff --git a/meta/classes/utility-tasks.bbclass 
b/meta/classes/utility-tasks.bbclass
index da69c3a..587bfd4 100644
--- a/meta/classes/utility-tasks.bbclass
+++ b/meta/classes/utility-tasks.bbclass
@@ -28,7 +28,7 @@ python do_clean() {
 bb.note("Removing " + dir)
 oe.path.remove(dir)
 
-dir = "%s.*" % bb.data.expand(d.getVar('STAMP', False), d)
+dir = "%s.*" % d.getVar('STAMP')
 bb.note("Removing " + dir)
 oe.path.remove(dir)
 
diff --git a/meta/lib/oe/patch.py b/meta/lib/oe/patch.py
index aaa2d3a..f1ab3dd 100644
--- a/meta/lib/oe/patch.py
+++ b/meta/lib/oe/patch.py
@@ -81,7 +81,7 @@ class PatchSet(object):
 patch[param] = PatchSet.defaults[param]
 
 if patch.get("remote"):
-patch["file"] = 
bb.data.expand(bb.fetch2.localpath(patch["remote"], self.d), self.d)
+patch["file"] = self.d.expand(bb.fetch2.localpath(patch["remote"], 
self.d))
 
 patch["filemd5"] = bb.utils.md5_file(patch["file"])
 
diff --git a/meta/recipes-extended/pam/libpam_1.3.0.bb 
b/meta/recipes-extended/pam/libpam_1.3.0.bb
index 67b9d53..df56d27 100644
--- a/meta/recipes-extended/pam/libpam_1.3.0.bb
+++ b/meta/recipes-extended/pam/libpam_1.3.0.bb
@@ -116,7 +116,7 @@ python populate_packages_prepend () {
 d.setVar('RPROVIDES_' + pkg, provides)
 
 mlprefix = d.getVar('MLPREFIX') or ''
-dvar = bb.data.expand('${WORKDIR}/package', d, True)
+dvar = d.expand('${WORKDIR}/package')
 pam_libdir = d.expand('${base_libdir}/security')
 pam_sbindir = d.expand('${sbindir}')
 pam_filterdir = d.expand('${base_libdir}/security/pam_filter')
diff --git a/meta/recipes-multimedia/alsa/alsa-plugins_1.1.1.bb 
b/meta/recipes-multimedia/alsa/alsa-plugins_1.1.1.bb
index 3928fb1..16686a0 100644
--- a/meta/recipes-multimedia/alsa/alsa-plugins_1.1.1.bb
+++ b/meta/recipes-multimedia/alsa/alsa-plugins_1.1.1.bb
@@ -59,7 +59,7 @@ do_install_append() {
 }
 
 python populate_packages_prepend() {
-plugindir = bb.data.expand('${libdir}/alsa-lib/', d)
+plugindir = d.expand('${libdir}/alsa-lib/')
 packages = " ".join(do_split_packages(d, plugindir, 
'^libasound_module_(.*)\.so$', 'libasound-module-%s', 'Alsa plugin for %s', 
extra_depends=''))
 d.setVar("RDEPENDS_alsa-plugins", packages)
 }
-- 
2.9.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/4] meta-selftest: add selftest-ed recipes

[Joshua Lock]

On Thu, 2017-03-09 at 21:05 +0100, Patrick Ohly wrote:
> On Wed, 2017-03-08 at 17:24 +0000, Joshua Lock wrote:
> > The oe-selftest oescripts.TestScripts.test_cleanup_workdir was
> > using
> > gzip and the GPLv2 variant to test cleanup of the workdir. This
> > broke
> > with the removal of GPLv2 recipes from OE-Core.
> > 
> > Instead of relying on recipes in OE-Core remaining static we should
> > ensure that meta-selftest provides recipes required for the tests
> > to pass.
> > To that end we take a copy of the current GPLv2 and GPLv3 variants
> > of ed
> > and include them in meta-selftest as new recipes.
> > We chose ed over gzip as gzip has dependencies which would require
> > additional GPLv2 recipes to be included in meta-selftest.
> 
> Wouldn't the test become faster when using artificial recipes, i.e.
> something which doesn't really need sources? Such an artificial
> recipe
> can use a local file as its "source", for example.

Indeed, that would make the test faster. I took a shorter route for
this series in an attempt to unblock M3, however I realise that the
bitbake targets used in oe-selftest could do with a consistent review.

I've filed an enhancement request to that effect:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=11142

> Or did I misunderstand something and the selftest doesn't really
> build
> ed?

It does build ed, which is an improvement over building gzip but not as
fast as something artificial.

Thanks for the feedback,

Joshua
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/4] lib/oeqa/selftest/oescripts: make test_cleanup_workdir use selftest-ed

[Joshua Lock]

Use a recipe which is bundled in the meta-selftest layer for this test,
rather than relying on OE-Core remaining static (or updating the tests
when OE-Core changes recipes).

Tested with oe-selftest -r oescripts.TestScripts.test_cleanup_workdir

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/lib/oeqa/selftest/oescripts.py | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/meta/lib/oeqa/selftest/oescripts.py 
b/meta/lib/oeqa/selftest/oescripts.py
index 28345dc..baf3b6a 100644
--- a/meta/lib/oeqa/selftest/oescripts.py
+++ b/meta/lib/oeqa/selftest/oescripts.py
@@ -14,10 +14,10 @@ class TestScripts(oeSelfTest):
 
 @testcase(300)
 def test_cleanup_workdir(self):
-path = os.path.dirname(get_bb_var('WORKDIR', 'gzip'))
-old_version_recipe = os.path.join(get_bb_var('COREBASE'), 
'meta/recipes-extended/gzip/gzip_1.3.12.bb')
-old_version = '1.3.12'
-bitbake("-c clean gzip")
+path = os.path.dirname(get_bb_var('WORKDIR', 'selftest-ed'))
+old_version_recipe = os.path.join(get_bb_var('COREBASE'), 
'meta-selftest/recipes-test/selftest-ed/selftest-ed_0.5.bb')
+old_version = '0.5'
+bitbake("-c clean selftest-ed")
 bitbake("-c clean -b %s" % old_version_recipe)
 
 if os.path.exists(path):
@@ -25,7 +25,7 @@ class TestScripts(oeSelfTest):
 else:
 initial_contents = []
 
-bitbake('gzip')
+bitbake('selftest-ed')
 intermediary_contents = os.listdir(path)
 bitbake("-b %s" % old_version_recipe)
 runCmd('cleanup-workdir')
-- 
2.9.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/4] chkconfig-alternatives-native: fix obey_variables

[Joshua Lock]

This postfunc assumes it's run from S, whereas that seems not to always
be the case in practice. Explicitly define the full path of the file
we wish to sed.

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/recipes-extended/chkconfig/chkconfig-alternatives-native_1.3.59.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git 
a/meta/recipes-extended/chkconfig/chkconfig-alternatives-native_1.3.59.bb 
b/meta/recipes-extended/chkconfig/chkconfig-alternatives-native_1.3.59.bb
index c0f7f16..87fa8eb 100644
--- a/meta/recipes-extended/chkconfig/chkconfig-alternatives-native_1.3.59.bb
+++ b/meta/recipes-extended/chkconfig/chkconfig-alternatives-native_1.3.59.bb
@@ -21,7 +21,7 @@ inherit native
 # into the sysroot, as we may use them to construct the rootfs. As such, we
 # only adjust the paths to match the metadata for the target, not native.
 obey_variables () {
-   sed -i 's,ALTERNATIVES_ROOT,OPKG_OFFLINE_ROOT,' alternatives.c
+   sed -i 's,ALTERNATIVES_ROOT,OPKG_OFFLINE_ROOT,' ${S}/alternatives.c
 }
 
 do_compile () {
-- 
2.9.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 4/4] lib/oeqa/selftest/bbtests: update test_non_gplv3 to use selftest-ed

[Joshua Lock]

GPLv2 recipes have been moved to a new layer (meta-gplv2), instead of
readline perform this test on the selftest-ed recipe in meta-selftest
which has gplv2 and gplv3 variants.

Tested with oe-selftest -r bbtests.BitbakeTests.test_non_gplv3

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/lib/oeqa/selftest/bbtests.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/lib/oeqa/selftest/bbtests.py 
b/meta/lib/oeqa/selftest/bbtests.py
index 4d9eec1..c2b79c6 100644
--- a/meta/lib/oeqa/selftest/bbtests.py
+++ b/meta/lib/oeqa/selftest/bbtests.py
@@ -231,11 +231,11 @@ INHERIT_remove = \"report-error\"
 @testcase(1119)
 def test_non_gplv3(self):
 self.write_config('INCOMPATIBLE_LICENSE = "GPLv3"')
-result = bitbake('readline', ignore_status=True)
+result = bitbake('selftest-ed', ignore_status=True)
 self.assertEqual(result.status, 0, "Bitbake failed, exit code %s, 
output %s" % (result.status, result.output))
 lic_dir = get_bb_var('LICENSE_DIRECTORY')
-self.assertFalse(os.path.isfile(os.path.join(lic_dir, 
'readline/generic_GPLv3')))
-self.assertTrue(os.path.isfile(os.path.join(lic_dir, 
'readline/generic_GPLv2')))
+self.assertFalse(os.path.isfile(os.path.join(lic_dir, 
'selftest-ed/generic_GPLv3')))
+self.assertTrue(os.path.isfile(os.path.join(lic_dir, 
'selftest-ed/generic_GPLv2')))
 
 @testcase(1422)
 def test_setscene_only(self):
-- 
2.9.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/4] meta-selftest: add selftest-ed recipes

[Joshua Lock]

The oe-selftest oescripts.TestScripts.test_cleanup_workdir was using
gzip and the GPLv2 variant to test cleanup of the workdir. This broke
with the removal of GPLv2 recipes from OE-Core.

Instead of relying on recipes in OE-Core remaining static we should
ensure that meta-selftest provides recipes required for the tests to pass.
To that end we take a copy of the current GPLv2 and GPLv3 variants of ed
and include them in meta-selftest as new recipes.
We chose ed over gzip as gzip has dependencies which would require
additional GPLv2 recipes to be included in meta-selftest.

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 .../recipes-test/selftest-ed/selftest-ed_0.5.bb| 22 ++
 .../recipes-test/selftest-ed/selftest-ed_1.14.1.bb | 35 ++
 2 files changed, 57 insertions(+)
 create mode 100644 meta-selftest/recipes-test/selftest-ed/selftest-ed_0.5.bb
 create mode 100644 meta-selftest/recipes-test/selftest-ed/selftest-ed_1.14.1.bb

diff --git a/meta-selftest/recipes-test/selftest-ed/selftest-ed_0.5.bb 
b/meta-selftest/recipes-test/selftest-ed/selftest-ed_0.5.bb
new file mode 100644
index 000..8e0d1cd
--- /dev/null
+++ b/meta-selftest/recipes-test/selftest-ed/selftest-ed_0.5.bb
@@ -0,0 +1,22 @@
+SUMMARY = "Line-oriented text editor -- selftest GPLv2 version"
+HOMEPAGE = "http://www.gnu.org/software/ed/;
+SECTION = "base"
+LICENSE = "GPLv2+"
+LIC_FILES_CHKSUM = "file://COPYING;md5=6ddd5335ef96fb858a138230af773710 \
+
file://main.c;beginline=1;endline=17;md5=36d4b85e5ae9028e918d1cc775c2475e"
+
+PR = "r2"
+SRC_URI = "${SAVANNAH_GNU_MIRROR}/ed/ed-${PV}.tar.bz2"
+
+SRC_URI[md5sum] = "4ee21e9dcc9b5b6012c23038734e1632"
+SRC_URI[sha256sum] = 
"edef2bbde0fbf0d88232782a0eded323f483a0519d6fde9a3b1809056fd35f3e"
+
+inherit autotools texinfo
+
+S = "${WORKDIR}/ed-${PV}"
+
+EXTRA_OECONF = "'CC=${CC}' 'CXX=${CXX}' 'CFLAGS=${CFLAGS}' 
'CXXFLAGS=${CXXFLAGS}' 'CPPFLAGS=${CPPFLAGS}' 'LDFLAGS=${LDFLAGS}'"
+
+CONFIGUREOPTS_remove = "--disable-dependency-tracking"
+CONFIGUREOPTS_remove = "--disable-silent-rules"
+EXTRA_OECONF_remove = "--disable-static"
diff --git a/meta-selftest/recipes-test/selftest-ed/selftest-ed_1.14.1.bb 
b/meta-selftest/recipes-test/selftest-ed/selftest-ed_1.14.1.bb
new file mode 100644
index 000..62931c4
--- /dev/null
+++ b/meta-selftest/recipes-test/selftest-ed/selftest-ed_1.14.1.bb
@@ -0,0 +1,35 @@
+SUMMARY = "Line-oriented text editor -- selftest variant"
+HOMEPAGE = "http://www.gnu.org/software/ed/;
+
+LICENSE = "GPLv3+"
+LIC_FILES_CHKSUM = "file://COPYING;md5=0c7051aef9219dc7237f206c5c4179a7 \
+
file://ed.h;endline=20;md5=4e36b7a40e137f42aee718165590d125 \
+
file://main.c;endline=17;md5=c5b8f78f115df187af76868a2aead16a"
+
+SECTION = "base"
+
+# LSB states that ed should be in /bin/
+bindir = "${base_bindir}"
+
+SRC_URI = "${GNU_MIRROR}/ed/ed-${PV}.tar.lz"
+
+SRC_URI[md5sum] = "7f4a54fa7f366479f03654b8af645fd0"
+SRC_URI[sha256sum] = 
"ffb97eb8f2a2b5a71a9b97e3872adce953aa1b8958e04c5b7bf11d556f32552a"
+
+S = "${WORKDIR}/ed-${PV}"
+
+EXTRA_OEMAKE = "-e MAKEFLAGS="
+
+inherit texinfo
+
+do_configure() {
+   ${S}/configure
+}
+
+do_install() {
+   oe_runmake 'DESTDIR=${D}' install
+   # Info dir listing isn't interesting at this point so remove it if it 
exists.
+   if [ -e "${D}${infodir}/dir" ]; then
+   rm -f ${D}${infodir}/dir
+   fi
+}
-- 
2.9.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] recipes: Move out stale GPLv2 versions to a seperate layer

[Joshua Lock]

On Wed, 2017-03-08 at 11:11 +, Richard Purdie wrote:
> On Wed, 2017-03-08 at 09:34 +, Peter Kjellerstedt wrote:
> > 
> > And the layer should be added to the layer index...
> 
> Yes, it should.

I've submitted meta-gplv2 to the layer index.

Joshua
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 3/3] classes/populate_sdk_base: Adds support to generate extensible sdk package manifest

[Joshua Lock]

On Tue, 2017-02-21 at 14:48 -0800, Francisco Pedraza wrote:
> This function writes on host and target manifest files for eSDK.
> [YOCTO #9038]
> 
> Signed-off-by: Francisco Pedraza  com>
> ---
>  meta/classes/populate_sdk_base.bbclass | 26
> +-
>  1 file changed, 25 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/classes/populate_sdk_base.bbclass
> b/meta/classes/populate_sdk_base.bbclass
> index 33665cd..db700d7 100644
> --- a/meta/classes/populate_sdk_base.bbclass
> +++ b/meta/classes/populate_sdk_base.bbclass
> @@ -59,6 +59,9 @@ SDK_TITLE ?= "${@d.getVar('DISTRO_NAME') or
> d.getVar('DISTRO')} SDK"
>  
>  SDK_TARGET_MANIFEST =
> "${SDKDEPLOYDIR}/${TOOLCHAIN_OUTPUTNAME}.target.manifest"
>  SDK_HOST_MANIFEST =
> "${SDKDEPLOYDIR}/${TOOLCHAIN_OUTPUTNAME}.host.manifest"
> +SDK_EXT_TARGET_MANIFEST =
> "${SDK_DEPLOY}/${TOOLCHAINEXT_OUTPUTNAME}.target.manifest"
> +SDK_EXT_HOST_MANIFEST =
> "${SDK_DEPLOY}/${TOOLCHAINEXT_OUTPUTNAME}.host.manifest"
> +
>  python write_target_sdk_manifest () {
>  if d.getVar('BB_CURRENTTASK') == 'populate_sdk':
>  from oe.sdk import sdk_list_installed_packages
> @@ -69,6 +72,17 @@ python write_target_sdk_manifest () {
>  bb.utils.mkdirhier(sdkmanifestdir)
>  with open(d.getVar('SDK_TARGET_MANIFEST'), 'w') as output:
>  output.write(format_pkg_list(pkgs, 'ver'))
> +elif d.getVar('BB_CURRENTTASK') == 'populate_sdk_ext':

Rather than checking which task is running like this couldn't we do the
following logic in the
populate_sdk_ext.bbclass:write_target_sdk_manifest() and similarly move
the above into populate_sdk.bbclass?

Joshua
> +from oe.sdk import get_extra_sdkinfo
> +sstate_dir = d.expand('${SDK_OUTPUT}/${SDKPATH}/sstate-
> cache')
> +extra_info = get_extra_sdkinfo(sstate_dir)
> +target = d.getVar('TARGET_SYS')
> +target_multimach = d.getVar('MULTIMACH_TARGET_SYS')
> +with open(d.getVar('SDK_EXT_TARGET_MANIFEST'), 'w') as f:
> +for fn in extra_info['filesizes']:
> +info = fn.split(':') #added
> +if info[2] in (target, target_multimach):
> +f.write("%s %s %s\n" % (info[1], info[2],
> info[3]))
>  }
>  
>  python write_sdk_test_data() {
> @@ -88,12 +102,22 @@ python write_host_sdk_manifest () {
>  bb.utils.mkdirhier(sdkmanifestdir)
>  with open(d.getVar('SDK_HOST_MANIFEST'), 'w') as output:
>  output.write(format_pkg_list(pkgs, 'ver'))
> +elif d.getVar('BB_CURRENTTASK') == 'populate_sdk_ext':
> +from oe.sdk import get_extra_sdkinfo
> +sstate_dir = d.expand('${SDK_OUTPUT}/${SDKPATH}/sstate-
> cache')
> +extra_info = get_extra_sdkinfo(sstate_dir)
> +host = d.getVar('BUILD_SYS')
> +with open(d.getVar('SDK_EXT_HOST_MANIFEST'), 'w') as f:
> +for fn in extra_info['filesizes']:
> +info = fn.split(':') #added
> +if info[2] == host:
> +f.write("%s %s %s\n" % (info[1], info[2],
> info[3]))
>  }
>  
>  POPULATE_SDK_POST_TARGET_COMMAND_append = "
> write_target_sdk_manifest ; write_sdk_test_data ; "
>  POPULATE_SDK_POST_HOST_COMMAND_append = " write_host_sdk_manifest; "
>  SDK_PACKAGING_COMMAND = "${@'${SDK_PACKAGING_FUNC};' if
> '${SDK_PACKAGING_FUNC}' else ''}"
> -SDK_POSTPROCESS_COMMAND = " create_sdk_files; check_sdk_sysroots;
> tar_sdk; ${SDK_PACKAGING_COMMAND} "
> +SDK_POSTPROCESS_COMMAND = " create_sdk_files; check_sdk_sysroots;
> tar_sdk; write_target_sdk_manifest; write_host_sdk_manifest;
> ${SDK_PACKAGING_COMMAND} "
>  
>  def populate_sdk_common(d):
>  from oe.sdk import populate_sdk
> -- 
> 1.8.3.1
> 
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] layer.conf: bump version for change in eSDK selftest behaviour

[Joshua Lock]

Since 59a99cd8 "oeqa/sdk: Updates sanity tests for minimal eSDK"
we can now run oe-selftests against a minimal eSDK.

We need to increase the layer version so that we can detect this
change in behaviour in our automated testing.

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/conf/layer.conf | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/conf/layer.conf b/meta/conf/layer.conf
index 24b4df0..87c235f 100644
--- a/meta/conf/layer.conf
+++ b/meta/conf/layer.conf
@@ -9,7 +9,7 @@ BBFILE_PRIORITY_core = "5"
 
 # This should only be incremented on significant changes that will
 # cause compatibility issues with other layers
-LAYERVERSION_core = "9"
+LAYERVERSION_core = "10"
 
 BBLAYERS_LAYERINDEX_NAME_core = "openembedded-core"
 
-- 
2.9.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] zlib: update SRC_URI to fix fetching

[Joshua Lock]

On Fri, 2017-01-13 at 17:53 +0200, Alexander Kanavin wrote:
> On 01/13/2017 05:36 PM, Joshua Lock wrote:
> 
> > Running checkpkg on the autobuilders won't really help as the
> > autobuilders rely on the bitbake invocation returning a non-zero
> > exit
> > code to determine whether to mark the build step as failed, and
> > that's
> > not the case when checkpkg doesn't find an update version.
> > 
> > If we regularly run checkpkg on the autobuilders how should we
> > detect
> > that a SRC_URI change has caused the upstream version check to
> > fail?
> 
> - run bitbake -c checkpkg world
> - inspect tmp/log/checkpkg.csv for lines with 'UNKNOWN' upstream
> status, 
> make a list of recipes that have it
> - compare that list against a stored list of exceptions (currently
> it 
> would have about 32 entries), if the lists don't match exactly, the 
> upstream version check has failed.
> 
> All of this can be wrapped in poky/scripts/upstream-check-all
> perhaps.

Thanks, I've filed a bug to track implementing this feature:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=11031

Joshua
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 1/3] go: Add recipes for golang compilers and tools

[Joshua Lock]

On Tue, 2017-02-07 at 17:35 +, Burton, Ross wrote:
> On 7 February 2017 at 04:24, Khem Raj  wrote:
> > This is converging the recipes for go from
> > 
> > meta-virtualization and oe-meta-go
> > 
> 
> I'm still of the opinion that this should be in meta-go, not oe-
> core...

FWIW I disagree. Go is a popular systems language and we're
increasingly supporting a wide-array of systems with OE (from iot to
ivi and beyond). 

Go is already in wide enough use and duplicated in enough layers that
we would be doing folks a favour by including it in OE-Core and our
build/test cycle.

Joshua-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] pseudo: update to 1.8.2

[Joshua Lock]

Update to the newly minted 1.8.2, dropping several patches we'd
backported since the last release.

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 .../files/0001-Don-t-send-SIGUSR1-to-init.patch|  48 -
 ...t-diagnostics-during-startup-for-pseudo-d.patch |  54 --
 .../files/0002-Use-correct-file-descriptor.patch   |  53 --
 ...nameat-parallel-to-previous-fix-to-rename.patch |  64 ---
 .../pseudo/files/Fix-xattr-performance.patch   | 117 -
 .../pseudo/files/More-correctly-fix-xattrs.patch   |  37 ---
 meta/recipes-devtools/pseudo/pseudo_1.8.1.bb   |  17 ---
 meta/recipes-devtools/pseudo/pseudo_1.8.2.bb   |  11 ++
 meta/recipes-devtools/pseudo/pseudo_git.bb |   4 +-
 9 files changed, 13 insertions(+), 392 deletions(-)
 delete mode 100644 
meta/recipes-devtools/pseudo/files/0001-Don-t-send-SIGUSR1-to-init.patch
 delete mode 100644 
meta/recipes-devtools/pseudo/files/0001-Quiet-diagnostics-during-startup-for-pseudo-d.patch
 delete mode 100644 
meta/recipes-devtools/pseudo/files/0002-Use-correct-file-descriptor.patch
 delete mode 100644 
meta/recipes-devtools/pseudo/files/0003-Fix-renameat-parallel-to-previous-fix-to-rename.patch
 delete mode 100644 
meta/recipes-devtools/pseudo/files/Fix-xattr-performance.patch
 delete mode 100644 
meta/recipes-devtools/pseudo/files/More-correctly-fix-xattrs.patch
 delete mode 100644 meta/recipes-devtools/pseudo/pseudo_1.8.1.bb
 create mode 100644 meta/recipes-devtools/pseudo/pseudo_1.8.2.bb

diff --git 
a/meta/recipes-devtools/pseudo/files/0001-Don-t-send-SIGUSR1-to-init.patch 
b/meta/recipes-devtools/pseudo/files/0001-Don-t-send-SIGUSR1-to-init.patch
deleted file mode 100644
index 6c694ce..000
--- a/meta/recipes-devtools/pseudo/files/0001-Don-t-send-SIGUSR1-to-init.patch
+++ /dev/null
@@ -1,48 +0,0 @@
-From befc6dbd6469d428c9e0830dbe51bdf7ac39d9ae Mon Sep 17 00:00:00 2001
-From: Seebs <se...@seebs.net>
-Date: Thu, 22 Sep 2016 14:35:04 -0500
-Subject: [PATCH] Don't send SIGUSR1 to init.
-
-If the parent exits due to child process being slow, getppid() will return
-1, and we'll send SIGUSR1 to init, which can break things like dumbinit
-which aren't adequately protected against non-root processes sending them
-signals.
-
-Signed-off-by: Seebs <se...@seebs.net>
-
-Upstream-Status: Backport (commit befc6dbd6469d428c9e0830dbe51bdf7ac39d9ae)
-
-[YOCTO #10324]
-
-This resolves an issue where a docker container running builds would die
-due to it's 'mini init' being signaled by pseudo.
-
-Signed-off-by: Mark Hatle <mark.ha...@windriver.com>

- pseudo_server.c | 11 ---
-
-diff --git a/pseudo_server.c b/pseudo_server.c
-index 65102dd..8731d20 100644
 a/pseudo_server.c
-+++ b/pseudo_server.c
-@@ -358,9 +358,14 @@ pseudo_server_start(int daemonize) {
-   signal(SIGTERM, quit_now);
-   /* tell parent process to stop waiting */
-   if (daemonize) {
--  pseudo_diag("Setup complete, sending SIGUSR1 to pid %d.\n",
--  getppid());
--  kill(getppid(), SIGUSR1);
-+  pid_t ppid = getppid();
-+  if (ppid == 1) {
-+  pseudo_diag("Setup complete, but parent is init, not 
sending SIGUSR1.\n");
-+  } else {
-+  pseudo_diag("Setup complete, sending SIGUSR1 to pid 
%d.\n",
-+  ppid);
-+  kill(ppid, SIGUSR1);
-+  }
-   }
-   pseudo_server_loop();
-   return 0;
--- 
-2.5.5
-
diff --git 
a/meta/recipes-devtools/pseudo/files/0001-Quiet-diagnostics-during-startup-for-pseudo-d.patch
 
b/meta/recipes-devtools/pseudo/files/0001-Quiet-diagnostics-during-startup-for-pseudo-d.patch
deleted file mode 100644
index d4b9f6a..000
--- 
a/meta/recipes-devtools/pseudo/files/0001-Quiet-diagnostics-during-startup-for-pseudo-d.patch
+++ /dev/null
@@ -1,54 +0,0 @@
-From eead8a505245a292c43f070c0e836cdfeb7bd7bd Mon Sep 17 00:00:00 2001
-From: Seebs <se...@seebs.net>
-Date: Wed, 28 Sep 2016 17:05:17 -0500
-Subject: [PATCH 1/2] Quiet diagnostics during startup for pseudo -d
-
-When the client spawns a pseudo server, it starts out sending diagnostics
-to stderr. This can be spammy in some cases with races during startup;
-everything resolves, but we get scary-looking diagnostics. So shove
-those into a log file.
-
-Signed-off-by: Seebs <se...@seebs.net>
-
-Upstream-Status: Backport
-Signed-off-by: Robert Yang <liezhi.y...@windriver.com>

- ChangeLog.txt   | 5 +
- pseudo_server.c | 4 +++-
- 2 files changed, 8 insertions(+), 1 deletion(-)
-
-diff --git a/ChangeLog.txt b/ChangeLog.txt
-index d6359ca..4cc24de 100644
 a/ChangeLog.txt
-+++ b/ChangeLog.txt
-@@ -1,3 +1,8 @@
-+2016-09-28:
-+  * (seebs) Send errors to log when daemonizing, but do that a lot
-+sooner to prevent startup messages which can show up spuriously

[OE-core] [PATCH] toolchain-scripts: remove CCACHE_PATH from environment script

[Joshua Lock]

CCACHE_PATH was added to the toolchain-scripts in commit
1d31ddb856a80ba9da1a64ed5d701dc0f7351ef7 without a detailed
explanation as to why. This commit removes that environment
variable as it's causing failures since the introduction of
host_gcc_version() to oe.utils in
d36f41e56586080ee833027879c119edf3e0, as used by
uninative to determine NATIVELSBSTRING.

This causes errors when the host has ccache available in in PATH
- host_gcc_version() fails because ccache has been told to only
look for the compiler in CCACHE_PATH and can't find gcc in order
to check its version. Toolchain compilers aren't detected due to
the binaries being prefixed with the triplet of the target.

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/classes/toolchain-scripts.bbclass | 2 --
 1 file changed, 2 deletions(-)

diff --git a/meta/classes/toolchain-scripts.bbclass 
b/meta/classes/toolchain-scripts.bbclass
index 7b68d20..260ece9 100644
--- a/meta/classes/toolchain-scripts.bbclass
+++ b/meta/classes/toolchain-scripts.bbclass
@@ -31,7 +31,6 @@ toolchain_create_sdk_env_script () {

EXTRAPATH="$EXTRAPATH:$sdkpathnative$bindir/${TARGET_ARCH}${TARGET_VENDOR}-$i"
done
echo "export 
PATH=$sdkpathnative$bindir:$sdkpathnative$sbindir:$sdkpathnative$base_bindir:$sdkpathnative$base_sbindir:$sdkpathnative$bindir/../${HOST_SYS}/bin:$sdkpathnative$bindir/${TARGET_SYS}"$EXTRAPATH':$PATH'
 >> $script
-   echo "export 
CCACHE_PATH=$sdkpathnative$bindir:$sdkpathnative$bindir/../${HOST_SYS}/bin:$sdkpathnative$bindir/${TARGET_SYS}"$EXTRAPATH':$CCACHE_PATH'
 >> $script
echo 'export PKG_CONFIG_SYSROOT_DIR=$SDKTARGETSYSROOT' >> $script
echo 'export 
PKG_CONFIG_PATH=$SDKTARGETSYSROOT'"$libdir"'/pkgconfig:$SDKTARGETSYSROOT'"$prefix"'/share/pkgconfig'
 >> $script
echo 'export 
CONFIG_SITE=${SDKPATH}/site-config-'"${multimach_target_sys}" >> $script
@@ -50,7 +49,6 @@ toolchain_create_tree_env_script () {
rm -f $script
touch $script
echo 'export PATH=${STAGING_DIR_NATIVE}/usr/bin:${PATH}' >> $script
-   echo 'export CCACHE_PATH=${STAGING_DIR_NATIVE}/usr/bin:${CCACHE_PATH}' 
>> $script
echo 'export PKG_CONFIG_SYSROOT_DIR=${PKG_CONFIG_SYSROOT_DIR}' >> 
$script
echo 'export PKG_CONFIG_PATH=${PKG_CONFIG_PATH}' >> $script
echo 'export CONFIG_SITE="${@siteinfo_get_files(d)}"' >> $script
-- 
2.9.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] zlib: update SRC_URI to fix fetching

[Joshua Lock]

On Fri, 2017-01-13 at 16:18 +0200, Alexander Kanavin wrote:
> On 01/13/2017 03:51 PM, Lock, Joshua G wrote:
> 
> > Noted, thanks. Do we maintain a list of things we'd like people to
> > check, and how to do it, when making updates to recipes?
> 
> I don't think there's such a list, perhaps we should make one and
> place 
> it in patch guidelines wiki.

I think we should, yes please.

> > checkpkg isn't available by default (had to add distrodata to
> > INHERITS)
> > and it took me a while to realise the results are written to a file
> > I
> > have to check. If we expect people to run these checks before
> > submitting updates we'd best make it as easy as possible to know
> > what
> > the checks are and how to run them.
> 
> I think this particular check should be automated and run in
> package_qa, 
> if it's okay to access the network in that step. Is it?

I don't think we should be accessing the network during package_qa.

> Alternatively, if checkuri is regularly run somewhere, then checkpkg 
> should be as well.

We run checkuri on the autobuilders and checkpkg is used by the recipe
report tool, though that's run less frequently. 

Running checkpkg on the autobuilders won't really help as the
autobuilders rely on the bitbake invocation returning a non-zero exit
code to determine whether to mark the build step as failed, and that's
not the case when checkpkg doesn't find an update version.

If we regularly run checkpkg on the autobuilders how should we detect
that a SRC_URI change has caused the upstream version check to fail?

Joshua
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] zlib: update SRC_URI to fix fetching

[Joshua Lock]

> On Fri, 2017-01-13 at 15:00 +0200, Alexander Kanavin wrote:
> On 01/09/2017 02:56 PM, Alexander Kanavin wrote:
> > On 01/05/2017 06:34 PM, Joshua Lock wrote:
> > > Upstream have removed the file from zlib.net as a new version has
> > > been released, switch to fetching from the official sourceforge
> > > mirror.
> > > 
> > > [YOCTO #10879]
> > 
> > If a new version has been released, you should also update to that
> > version. Also, does upstream version check (-c checkpkg) still work
> > with
> > the new SRC_URI?
> 
> Actually it doesn't, accordingly we have no way of knowing if zlib
> has a 
> new version upstream. Please do check these things when you update
> SRC_URI.

Noted, thanks. Do we maintain a list of things we'd like people to
check, and how to do it, when making updates to recipes?

checkpkg isn't available by default (had to add distrodata to INHERITS)
and it took me a while to realise the results are written to a file I
have to check. If we expect people to run these checks before
submitting updates we'd best make it as easy as possible to know what
the checks are and how to run them.

The only reference I can find to checkpkg on the OE and YP wikis is
here:

https://wiki.yoctoproject.org/wiki/How_do_I#Q:_How_do_I_ensure_that_I_a
m_using_the_latest_upstream_version_of_the_package.3F

Joshua
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 0/1] Fido backport pull request

[Joshua Lock]

Backport babeltrace fix as requested by Jonathan Rajotte Julien:
http://lists.openembedded.org/pipermail/openembedded-core/2017-January/130849.html

The following changes since commit f9c35dfc6601901664dbef2cba3a9b96bef96e64:

  systemd: re-enable mount propagation for udevd (2016-12-05 17:20:46 +)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib joshuagl/fido-next
  
http://git.openembedded.org/openembedded-core-contrib/log/?h=joshuagl/fido-next

Nathan Lynch (1):
  as-needed.inc: add babeltrace exception

 meta/conf/distro/include/as-needed.inc | 1 +
 1 file changed, 1 insertion(+)

-- 
2.9.3
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] zlib: update SRC_URI to fix fetching

[Joshua Lock]

Upstream have removed the file from zlib.net as a new version has
been released, switch to fetching from the official sourceforge
mirror.

[YOCTO #10879]

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/recipes-core/zlib/zlib_1.2.8.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-core/zlib/zlib_1.2.8.bb 
b/meta/recipes-core/zlib/zlib_1.2.8.bb
index 9470adb..913c703 100644
--- a/meta/recipes-core/zlib/zlib_1.2.8.bb
+++ b/meta/recipes-core/zlib/zlib_1.2.8.bb
@@ -6,7 +6,7 @@ SECTION = "libs"
 LICENSE = "Zlib"
 LIC_FILES_CHKSUM = 
"file://zlib.h;beginline=4;endline=23;md5=fde612df1e5933c428b73844a0c494fd"
 
-SRC_URI = "http://www.zlib.net/${BPN}-${PV}.tar.xz \
+SRC_URI = "${SOURCEFORGE_MIRROR}/libpng/${BPN}/${PV}/${BPN}-${PV}.tar.xz \
file://remove.ldconfig.call.patch \
file://Makefile-runtests.patch \
file://ldflags-tests.patch \
-- 
2.9.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/4] scripts: remove True option to getVar calls

[Joshua Lock]

getVar() now defaults to expanding by default, thus remove the True
option from getVar() calls with a regex search and replace.

Search made with the following regex: getVar ?\(( ?[^,()]*), True\)

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 scripts/contrib/list-packageconfig-flags.py  |  6 +--
 scripts/contrib/verify-homepage.py   |  2 +-
 scripts/devtool  |  2 +-
 scripts/lib/devtool/__init__.py  |  8 ++--
 scripts/lib/devtool/build_image.py   | 10 ++--
 scripts/lib/devtool/deploy.py|  2 +-
 scripts/lib/devtool/package.py   |  2 +-
 scripts/lib/devtool/runqemu.py   |  4 +-
 scripts/lib/devtool/sdk.py   |  8 ++--
 scripts/lib/devtool/search.py|  4 +-
 scripts/lib/devtool/standard.py  | 58 
 scripts/lib/devtool/upgrade.py   | 20 
 scripts/lib/devtool/utilcmds.py  | 18 
 scripts/lib/recipetool/append.py | 22 -
 scripts/lib/recipetool/create.py | 24 +-
 scripts/lib/recipetool/create_buildsys_python.py |  4 +-
 scripts/lib/recipetool/create_kernel.py  |  4 +-
 scripts/lib/recipetool/create_npm.py |  6 +--
 scripts/lib/recipetool/newappend.py  |  2 +-
 scripts/lib/scriptutils.py   |  2 +-
 scripts/oe-pkgdata-util  |  2 +-
 scripts/recipetool   |  2 +-
 22 files changed, 106 insertions(+), 106 deletions(-)

diff --git a/scripts/contrib/list-packageconfig-flags.py 
b/scripts/contrib/list-packageconfig-flags.py
index 389fb97..7ce7186 100755
--- a/scripts/contrib/list-packageconfig-flags.py
+++ b/scripts/contrib/list-packageconfig-flags.py
@@ -76,7 +76,7 @@ def collect_pkgs(data_dict):
 for fn in data_dict:
 pkgconfigflags = data_dict[fn].getVarFlags("PACKAGECONFIG")
 pkgconfigflags.pop('doc', None)
-pkgname = data_dict[fn].getVar("P", True)
+pkgname = data_dict[fn].getVar("P")
 pkg_dict[pkgname] = sorted(pkgconfigflags.keys())
 
 return pkg_dict
@@ -124,9 +124,9 @@ def display_all(data_dict):
 ''' Display all pkgs and PACKAGECONFIG information '''
 print(str("").ljust(50, '='))
 for fn in data_dict:
-print('%s' % data_dict[fn].getVar("P", True))
+print('%s' % data_dict[fn].getVar("P"))
 print(fn)
-packageconfig = data_dict[fn].getVar("PACKAGECONFIG", True) or ''
+packageconfig = data_dict[fn].getVar("PACKAGECONFIG") or ''
 if packageconfig.strip() == '':
 packageconfig = 'None'
 print('PACKAGECONFIG %s' % packageconfig)
diff --git a/scripts/contrib/verify-homepage.py 
b/scripts/contrib/verify-homepage.py
index d39dd1d..76f1749 100755
--- a/scripts/contrib/verify-homepage.py
+++ b/scripts/contrib/verify-homepage.py
@@ -44,7 +44,7 @@ def verifyHomepage(bbhandler):
 if realfn in checked:
 continue
 data = bbhandler.parse_recipe_file(realfn)
-homepage = data.getVar("HOMEPAGE", True)
+homepage = data.getVar("HOMEPAGE")
 if homepage:
 try:
 urllib.request.urlopen(homepage, timeout=5)
diff --git a/scripts/devtool b/scripts/devtool
index 578db5a..0866be9 100755
--- a/scripts/devtool
+++ b/scripts/devtool
@@ -291,7 +291,7 @@ def main():
 try:
 tinfoil = setup_tinfoil(config_only=True, basepath=basepath)
 try:
-global_args.bbpath = tinfoil.config_data.getVar('BBPATH', True)
+global_args.bbpath = tinfoil.config_data.getVar('BBPATH')
 finally:
 tinfoil.shutdown()
 except bb.BBHandledException:
diff --git a/scripts/lib/devtool/__init__.py b/scripts/lib/devtool/__init__.py
index 99c5534..fd2f042 100644
--- a/scripts/lib/devtool/__init__.py
+++ b/scripts/lib/devtool/__init__.py
@@ -87,13 +87,13 @@ def exec_watch(cmd, **options):
 def exec_fakeroot(d, cmd, **kwargs):
 """Run a command under fakeroot (pseudo, in fact) so that it picks up the 
appropriate file permissions"""
 # Grab the command and check it actually exists
-fakerootcmd = d.getVar('FAKEROOTCMD', True)
+fakerootcmd = d.getVar('FAKEROOTCMD')
 if not os.path.exists(fakerootcmd):
 logger.error('pseudo executable %s could not be found - have you run a 
build yet? pseudo-native should install this and if you have run any build then 
that should have been built')
 return 2
 # Set up the appropriate environment
 newenv = dict(os.environ)
-fakerootenv = d.getVar('FAKEROOTENV', True)
+fakerootenv = d.getVar('FAKEROOTENV')
 

[OE-core] [PATCH 3/4] meta: remove True option to getVarFlag calls

[Joshua Lock]

getVarFlag() now defaults to expanding by default, thus remove the
True option from getVarFlag() calls with a regex search and
replace.

Search made with the following regex:
getVarFlag ?\(( ?[^,()]*, ?[^,()]*), True\)

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/classes/archiver.bbclass  | 28 ++--
 meta/classes/blacklist.bbclass |  2 +-
 meta/classes/compress_doc.bbclass  | 14 +++---
 meta/classes/devshell.bbclass  |  4 ++--
 meta/classes/externalsrc.bbclass   |  2 +-
 meta/classes/image-buildinfo.bbclass   |  2 +-
 meta/classes/image.bbclass |  4 ++--
 meta/classes/license.bbclass   | 16 
 meta/classes/package.bbclass   |  4 ++--
 meta/classes/package_rpm.bbclass   |  6 +++---
 meta/classes/package_tar.bbclass   |  2 +-
 meta/classes/packagefeed-stability.bbclass |  2 +-
 meta/classes/patch.bbclass |  2 +-
 meta/classes/populate_sdk_ext.bbclass  |  4 ++--
 meta/classes/rootfs_ipk.bbclass|  2 +-
 meta/classes/rootfs_rpm.bbclass|  2 +-
 meta/classes/sstate.bbclass| 14 +++---
 meta/classes/testexport.bbclass|  2 +-
 meta/classes/typecheck.bbclass |  2 +-
 meta/classes/update-alternatives.bbclass   | 14 +++---
 meta/classes/utility-tasks.bbclass |  6 +++---
 meta/lib/oe/data.py|  2 +-
 meta/lib/oe/packagegroup.py|  4 ++--
 meta/lib/oe/sstatesig.py   |  2 +-
 24 files changed, 71 insertions(+), 71 deletions(-)

diff --git a/meta/classes/archiver.bbclass b/meta/classes/archiver.bbclass
index 5295977..ee4790d 100644
--- a/meta/classes/archiver.bbclass
+++ b/meta/classes/archiver.bbclass
@@ -73,9 +73,9 @@ python () {
 bb.debug(1, 'archiver: %s is excluded, covered by gcc-source' % pn)
 return
 
-ar_src = d.getVarFlag('ARCHIVER_MODE', 'src', True)
-ar_dumpdata = d.getVarFlag('ARCHIVER_MODE', 'dumpdata', True)
-ar_recipe = d.getVarFlag('ARCHIVER_MODE', 'recipe', True)
+ar_src = d.getVarFlag('ARCHIVER_MODE', 'src')
+ar_dumpdata = d.getVarFlag('ARCHIVER_MODE', 'dumpdata')
+ar_recipe = d.getVarFlag('ARCHIVER_MODE', 'recipe')
 
 if ar_src == "original":
 d.appendVarFlag('do_deploy_archives', 'depends', ' %s:do_ar_original' 
% pn)
@@ -104,7 +104,7 @@ python () {
 d.appendVarFlag('do_deploy_archives', 'depends', ' %s:do_ar_recipe' % 
pn)
 
 # Output the srpm package
-ar_srpm = d.getVarFlag('ARCHIVER_MODE', 'srpm', True)
+ar_srpm = d.getVarFlag('ARCHIVER_MODE', 'srpm')
 if ar_srpm == "1":
 if d.getVar('PACKAGES') != '' and d.getVar('IMAGE_PKGTYPE') == 'rpm':
 d.appendVarFlag('do_deploy_archives', 'depends', ' 
%s:do_package_write_rpm' % pn)
@@ -127,7 +127,7 @@ python do_ar_original() {
 
 import shutil, tempfile
 
-if d.getVarFlag('ARCHIVER_MODE', 'src', True) != "original":
+if d.getVarFlag('ARCHIVER_MODE', 'src') != "original":
 return
 
 ar_outdir = d.getVar('ARCHIVER_OUTDIR')
@@ -191,7 +191,7 @@ python do_ar_original() {
 
 python do_ar_patched() {
 
-if d.getVarFlag('ARCHIVER_MODE', 'src', True) != 'patched':
+if d.getVarFlag('ARCHIVER_MODE', 'src') != 'patched':
 return
 
 # Get the ARCHIVER_OUTDIR before we reset the WORKDIR
@@ -206,7 +206,7 @@ python do_ar_configured() {
 import shutil
 
 ar_outdir = d.getVar('ARCHIVER_OUTDIR')
-if d.getVarFlag('ARCHIVER_MODE', 'src', True) == 'configured':
+if d.getVarFlag('ARCHIVER_MODE', 'src') == 'configured':
 bb.note('Archiving the configured source...')
 pn = d.getVar('PN')
 # "gcc-source-${PV}" recipes don't have "do_configure"
@@ -226,12 +226,12 @@ python do_ar_configured() {
 bb.build.exec_func('do_kernel_configme', d)
 if bb.data.inherits_class('cmake', d):
 bb.build.exec_func('do_generate_toolchain_file', d)
-prefuncs = d.getVarFlag('do_configure', 'prefuncs', True)
+prefuncs = d.getVarFlag('do_configure', 'prefuncs')
 for func in (prefuncs or '').split():
 if func != "sysroot_cleansstate":
 bb.build.exec_func(func, d)
 bb.build.exec_func('do_configure', d)
-postfuncs = d.getVarFlag('do_configure', 'postfuncs', True)
+postfuncs = d.getVarFlag('do_configure', 'postfuncs')
 for func in (postfuncs or '').split():
 if func != "do_qa_configure":
 bb.build.exec_func(func, d)
@@ -279,7 +279,7 @@ def create_diff_gz(d, src_orig, src, ar_outdir):
 # exclude.
 src_patched = src + '.patched'
 oe.path.copyhardlinktree(src, src_patched)
-for i in d.getVarFlag('ARCHIVER_M

[OE-core] [PATCH 0/4] Remove True option to getVar[Flag] calls

[Joshua Lock]

getVar() and getVarFlag() now default to expanding by default, thus remove the
True option from calls to both with a regex search and replace.

(Note: this series is relative to ross/mut3)

The following changes since commit d17879fdbc46e2cf10f724d3fbcf1b09d4e1bc7b:

  uninative: rebuild uninative for gcc 4.8 and 4.9 (2016-12-14 09:44:15 +)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib joshuagl/deprecations
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=joshuagl/deprecations

Joshua Lock (4):
  meta: remove True option to getVar calls
  scripts: remove True option to getVar calls
  meta: remove True option to getVarFlag calls
  scripts: remove True option to getVarFlag calls

 meta/classes/allarch.bbclass   |   4 +-
 meta/classes/archiver.bbclass  | 102 
 meta/classes/autotools.bbclass |  12 +-
 meta/classes/base.bbclass  | 126 -
 meta/classes/binconfig.bbclass |   2 +-
 meta/classes/blacklist.bbclass |   6 +-
 meta/classes/bugzilla.bbclass  |  24 +-
 meta/classes/buildhistory.bbclass  |  62 ++---
 meta/classes/buildstats-summary.bbclass|   2 +-
 meta/classes/buildstats.bbclass|  14 +-
 meta/classes/ccache.bbclass|   2 +-
 meta/classes/chrpath.bbclass   |   4 +-
 meta/classes/cmake.bbclass |  16 +-
 meta/classes/cml1.bbclass  |   4 +-
 meta/classes/compress_doc.bbclass  |  40 +--
 meta/classes/copyleft_compliance.bbclass   |  10 +-
 meta/classes/copyleft_filter.bbclass   |  10 +-
 meta/classes/cross-canadian.bbclass|  34 +--
 meta/classes/crosssdk.bbclass  |   2 +-
 meta/classes/cve-check.bbclass |  46 ++--
 meta/classes/debian.bbclass|  24 +-
 meta/classes/devshell.bbclass  |  10 +-
 meta/classes/distro_features_check.bbclass |   8 +-
 meta/classes/distrodata.bbclass| 132 +-
 meta/classes/distutils-base.bbclass|   2 +-
 meta/classes/distutils3-base.bbclass   |   2 +-
 meta/classes/externalsrc.bbclass   |  40 +--
 meta/classes/extrausers.bbclass|   2 +-
 meta/classes/fontcache.bbclass |  14 +-
 meta/classes/fs-uuid.bbclass   |   2 +-
 meta/classes/gconf.bbclass |  12 +-
 meta/classes/gettext.bbclass   |   8 +-
 meta/classes/gio-module-cache.bbclass  |  10 +-
 meta/classes/grub-efi.bbclass  |  22 +-
 meta/classes/gsettings.bbclass |  10 +-
 meta/classes/gtk-icon-cache.bbclass|  14 +-
 meta/classes/gtk-immodules-cache.bbclass   |  10 +-
 meta/classes/icecc.bbclass |   4 +-
 meta/classes/image-buildinfo.bbclass   |  12 +-
 meta/classes/image-live.bbclass|   8 +-
 meta/classes/image-vm.bbclass  |  12 +-
 meta/classes/image.bbclass | 100 +++
 meta/classes/image_types.bbclass   |  24 +-
 meta/classes/insane.bbclass| 194 +++---
 meta/classes/kernel-arch.bbclass   |   6 +-
 meta/classes/kernel-fitimage.bbclass   |  14 +-
 meta/classes/kernel-grub.bbclass   |   2 +-
 meta/classes/kernel-module-split.bbclass   |  38 +--
 meta/classes/kernel-uimage.bbclass |   8 +-
 meta/classes/kernel-yocto.bbclass  |   2 +-
 meta/classes/kernel.bbclass|  40 +--
 meta/classes/libc-common.bbclass   |  10 +-
 meta/classes/libc-package.bbclass  |  60 ++---
 meta/classes/license.bbclass   | 112 
 meta/classes/live-vm-common.bbclass|   8 +-
 meta/classes/metadata_scm.bbclass  |   2 +-
 meta/classes/migrate_localcount.bbclass|  12 +-
 meta/classes/mime.bbclass  |  12 +-
 meta/classes/module.bbclass|   2 +-
 meta/classes/multilib.bbclass  |  32 +--
 meta/classes/multilib_global.bbclass   |  22 +-
 meta/classes/native.bbclass|  10 +-
 meta/classes/nativesdk.bbclass |   8 +-
 meta/classes/npm.bbclass   |   4 +-
 meta/classes/oelint.bbclass|   4 +-
 meta/classes/package.bbclass   | 286 ++---
 meta/classes/package_deb.bbclass   |  54 ++--
 meta/classes/package_ipk.bbclass   |  60 ++---
 meta/classes

[OE-core] [PATCH 4/4] scripts: remove True option to getVarFlag calls

[Joshua Lock]

getVarFlag() now defaults to expanding by default, thus remove the
True option from getVarFlag() calls with a regex search and
replace.

Search made with the following regex:
getVarFlag ?\(( ?[^,()]*, ?[^,()]*), True\)

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 scripts/lib/recipetool/append.py | 6 +++---
 scripts/verify-bashisms  | 2 +-
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/scripts/lib/recipetool/append.py b/scripts/lib/recipetool/append.py
index def4f90..69c8bb7 100644
--- a/scripts/lib/recipetool/append.py
+++ b/scripts/lib/recipetool/append.py
@@ -121,7 +121,7 @@ def determine_file_source(targetpath, rd):
 logger.debug('source path: %s' % srcpath)
 if not srcpath.startswith('/'):
 # Handle non-absolute path
-srcpath = os.path.abspath(os.path.join(rd.getVarFlag('do_install', 
'dirs', True).split()[-1], srcpath))
+srcpath = os.path.abspath(os.path.join(rd.getVarFlag('do_install', 
'dirs').split()[-1], srcpath))
 if srcpath.startswith(workdir):
 # OK, now we have the source file name, look for it in SRC_URI
 workdirfile = os.path.relpath(srcpath, workdir)
@@ -191,10 +191,10 @@ def get_source_path(cmdelements):
 def get_func_deps(func, d):
 """Find the function dependencies of a shell function"""
 deps = bb.codeparser.ShellParser(func, logger).parse_shell(d.getVar(func))
-deps |= set((d.getVarFlag(func, "vardeps", True) or "").split())
+deps |= set((d.getVarFlag(func, "vardeps") or "").split())
 funcdeps = []
 for dep in deps:
-if d.getVarFlag(dep, 'func', True):
+if d.getVarFlag(dep, 'func'):
 funcdeps.append(dep)
 return funcdeps
 
diff --git a/scripts/verify-bashisms b/scripts/verify-bashisms
index 0741e18..a8f761d 100755
--- a/scripts/verify-bashisms
+++ b/scripts/verify-bashisms
@@ -97,7 +97,7 @@ if __name__=='__main__':
 result = []
 data = tinfoil.parse_recipe_file(fn)
 for key in data.keys():
-if data.getVarFlag(key, "func", True) and not data.getVarFlag(key, 
"python", True):
+if data.getVarFlag(key, "func") and not data.getVarFlag(key, 
"python"):
 script = data.getVar(key, False)
 if not script: continue
 #print ("%s:%s" % (fn, key))
-- 
2.9.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/2] meta: remove True option to getVarFlag calls

[Joshua Lock]

getVarFlag() now defaults to expanding by default, thus remove the
True option from getVarFlag() calls with a regex search and
replace.

Search made with the following regex:
getVarFlag ?\(( ?[^,()]*, ?[^,()]*), True\)

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/classes/archiver.bbclass  | 28 ++--
 meta/classes/blacklist.bbclass |  2 +-
 meta/classes/compress_doc.bbclass  | 14 +++---
 meta/classes/devshell.bbclass  |  4 ++--
 meta/classes/externalsrc.bbclass   |  2 +-
 meta/classes/image-buildinfo.bbclass   |  2 +-
 meta/classes/image.bbclass |  4 ++--
 meta/classes/license.bbclass   | 16 
 meta/classes/package.bbclass   |  4 ++--
 meta/classes/package_rpm.bbclass   |  6 +++---
 meta/classes/package_tar.bbclass   |  2 +-
 meta/classes/packagefeed-stability.bbclass |  2 +-
 meta/classes/populate_sdk_ext.bbclass  |  4 ++--
 meta/classes/rootfs_ipk.bbclass|  2 +-
 meta/classes/rootfs_rpm.bbclass|  2 +-
 meta/classes/sstate.bbclass| 14 +++---
 meta/classes/testexport.bbclass|  2 +-
 meta/classes/typecheck.bbclass |  2 +-
 meta/classes/update-alternatives.bbclass   | 14 +++---
 meta/classes/utility-tasks.bbclass |  6 +++---
 meta/lib/oe/data.py|  2 +-
 meta/lib/oe/packagegroup.py|  4 ++--
 meta/lib/oe/sstatesig.py   |  2 +-
 scripts/lib/recipetool/append.py   |  6 +++---
 scripts/verify-bashisms|  2 +-
 25 files changed, 74 insertions(+), 74 deletions(-)

diff --git a/meta/classes/archiver.bbclass b/meta/classes/archiver.bbclass
index 30215ae..3e3c382 100644
--- a/meta/classes/archiver.bbclass
+++ b/meta/classes/archiver.bbclass
@@ -73,9 +73,9 @@ python () {
 bb.debug(1, 'archiver: %s is excluded, covered by gcc-source' % pn)
 return
 
-ar_src = d.getVarFlag('ARCHIVER_MODE', 'src', True)
-ar_dumpdata = d.getVarFlag('ARCHIVER_MODE', 'dumpdata', True)
-ar_recipe = d.getVarFlag('ARCHIVER_MODE', 'recipe', True)
+ar_src = d.getVarFlag('ARCHIVER_MODE', 'src')
+ar_dumpdata = d.getVarFlag('ARCHIVER_MODE', 'dumpdata')
+ar_recipe = d.getVarFlag('ARCHIVER_MODE', 'recipe')
 
 if ar_src == "original":
 d.appendVarFlag('do_deploy_archives', 'depends', ' %s:do_ar_original' 
% pn)
@@ -104,7 +104,7 @@ python () {
 d.appendVarFlag('do_deploy_archives', 'depends', ' %s:do_ar_recipe' % 
pn)
 
 # Output the srpm package
-ar_srpm = d.getVarFlag('ARCHIVER_MODE', 'srpm', True)
+ar_srpm = d.getVarFlag('ARCHIVER_MODE', 'srpm')
 if ar_srpm == "1":
 if d.getVar('PACKAGES') != '' and d.getVar('IMAGE_PKGTYPE') == 'rpm':
 d.appendVarFlag('do_deploy_archives', 'depends', ' 
%s:do_package_write_rpm' % pn)
@@ -127,7 +127,7 @@ python do_ar_original() {
 
 import shutil, tarfile, tempfile
 
-if d.getVarFlag('ARCHIVER_MODE', 'src', True) != "original":
+if d.getVarFlag('ARCHIVER_MODE', 'src') != "original":
 return
 
 ar_outdir = d.getVar('ARCHIVER_OUTDIR')
@@ -191,7 +191,7 @@ python do_ar_original() {
 
 python do_ar_patched() {
 
-if d.getVarFlag('ARCHIVER_MODE', 'src', True) != 'patched':
+if d.getVarFlag('ARCHIVER_MODE', 'src') != 'patched':
 return
 
 # Get the ARCHIVER_OUTDIR before we reset the WORKDIR
@@ -206,7 +206,7 @@ python do_ar_configured() {
 import shutil
 
 ar_outdir = d.getVar('ARCHIVER_OUTDIR')
-if d.getVarFlag('ARCHIVER_MODE', 'src', True) == 'configured':
+if d.getVarFlag('ARCHIVER_MODE', 'src') == 'configured':
 bb.note('Archiving the configured source...')
 pn = d.getVar('PN')
 # "gcc-source-${PV}" recipes don't have "do_configure"
@@ -226,12 +226,12 @@ python do_ar_configured() {
 bb.build.exec_func('do_kernel_configme', d)
 if bb.data.inherits_class('cmake', d):
 bb.build.exec_func('do_generate_toolchain_file', d)
-prefuncs = d.getVarFlag('do_configure', 'prefuncs', True)
+prefuncs = d.getVarFlag('do_configure', 'prefuncs')
 for func in (prefuncs or '').split():
 if func != "sysroot_cleansstate":
 bb.build.exec_func(func, d)
 bb.build.exec_func('do_configure', d)
-postfuncs = d.getVarFlag('do_configure', 'postfuncs', True)
+postfuncs = d.getVarFlag('do_configure', 'postfuncs')
 for func in (postfuncs or '').split():
 if func != "do_qa_configure":
 bb.build.exec_func(func, d)
@@ -283,7 +283,7 @@ def create_diff_gz(d, src_orig, src, ar_outdir):
 # exclude.
 src_patched = src + '.patched'
 oe.path.copyha

[OE-core] [PATCH] lib/oe/lsb: better handle missing fields

[Joshua Lock]

Some rolling release distros, such as Arch Linux, don't include a
VERSION_ID field in their os-release file.

Change release_dict_osr() to better handle this optional field
being absent.

Further improve the resilience of the release_dict_*() methods by
always returning a dict and using dict.get() in distro_identifier()
to supply a default, empty string, value when then key is missing.

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/lib/oe/lsb.py | 13 +
 1 file changed, 5 insertions(+), 8 deletions(-)

diff --git a/meta/lib/oe/lsb.py b/meta/lib/oe/lsb.py
index 5a795a1..3a945e0 100644
--- a/meta/lib/oe/lsb.py
+++ b/meta/lib/oe/lsb.py
@@ -15,9 +15,6 @@ def release_dict_osr():
 if key == 'VERSION_ID':
 data['DISTRIB_RELEASE'] = val.strip('"')
 
-if len(data.keys()) != 2:
-return None
-
 return data
 
 def release_dict_lsb():
@@ -27,7 +24,7 @@ def release_dict_lsb():
 try:
 output, err = bb.process.run(['lsb_release', '-ir'], stderr=PIPE)
 except bb.process.CmdError as exc:
-return None
+return {}
 
 lsb_map = { 'Distributor ID': 'DISTRIB_ID',
 'Release': 'DISTRIB_RELEASE'}
@@ -51,7 +48,7 @@ def release_dict_lsb():
 
 def release_dict_file():
 """ Try to gather release information manually when other methods fail """
-data = None
+data = {}
 try:
 if os.path.exists('/etc/lsb-release'):
 data = {}
@@ -78,7 +75,7 @@ def release_dict_file():
 break
 
 except IOError:
-return None
+return {}
 return data
 
 def distro_identifier(adjust_hook=None):
@@ -96,8 +93,8 @@ def distro_identifier(adjust_hook=None):
 if not distro_data:
 distro_data = release_dict_file()
 
-distro_id = distro_data['DISTRIB_ID']
-release = distro_data['DISTRIB_RELEASE']
+distro_id = distro_data.get('DISTRIB_ID', '')
+release = distro_data.get('DISTRIB_RELEASE', '')
 
 if adjust_hook:
 distro_id, release = adjust_hook(distro_id, release)
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] python3: Build and package precompiled modules

[Joshua Lock]

On Tue, 2016-10-25 at 19:20 +0200, Dominic Sacré wrote:
> Remove the patch that was applied in the python3 and python3-native
> recipes to skip compilation of python modules.
> 
> Modify generate-manifest-3.5.py to match '__pycache__' directories in
> FILES_*.
> This is necessary because Python3 puts .pyc files in '__pycache__'
> subdirectories one level below the corresponding .py files, whereas
> in
> Python2 they used to be right next to the sources.
> 
> This change significantly reduces the startup overhead of Python3
> scripts. For example, on a Cortex-A9, "python3 -c pass" took 0.40s
> before, and 0.19s after.
> 
> Signed-off-by: Dominic Sacré <dominic.sa...@gmx.de>

This looks good, thanks. I've had a similar change queued locally and
testing for [YOCTO #10315].

Acked-by: Joshua Lock <joshua.g.l...@intel.com>

> ---
>  .../python/python-3.5-manifest.inc | 130 ++-
> --
>  .../python/python3-native_3.5.2.bb |   1 -
>  .../python3/020-dont-compile-python-files.patch|  48 
>  meta/recipes-devtools/python/python3_3.5.2.bb  |   1 -
>  scripts/contrib/python/generate-manifest-3.5.py|  10 ++
>  5 files changed, 75 insertions(+), 115 deletions(-)
>  delete mode 100644 meta/recipes-devtools/python/python3/020-dont-
> compile-python-files.patch
> 
> diff --git a/meta/recipes-devtools/python/python-3.5-manifest.inc
> b/meta/recipes-devtools/python/python-3.5-manifest.inc
> index 3046114..851f12e 100644
> --- a/meta/recipes-devtools/python/python-3.5-manifest.inc
> +++ b/meta/recipes-devtools/python/python-3.5-manifest.inc
> @@ -11,59 +11,59 @@ PACKAGES="${PN}-dbg ${PN}-2to3 ${PN}-argparse
> ${PN}-asyncio ${PN}-audio ${PN}-co
>  
>  SUMMARY_${PN}-2to3="Python automated Python 2 to 3 code translator"
>  RDEPENDS_${PN}-2to3="${PN}-core"
> -FILES_${PN}-2to3="${libdir}/python3.5/lib2to3 "
> +FILES_${PN}-2to3="${libdir}/python3.5/lib2to3
> ${libdir}/python3.5/lib2to3/__pycache__ "
>  
>  SUMMARY_${PN}-argparse="Python command line argument parser"
>  RDEPENDS_${PN}-argparse="${PN}-core ${PN}-codecs ${PN}-textutils"
> -FILES_${PN}-argparse="${libdir}/python3.5/argparse.* "
> +FILES_${PN}-argparse="${libdir}/python3.5/argparse.*
> ${libdir}/python3.5/__pycache__/argparse.* "
>  
>  SUMMARY_${PN}-asyncio="Python Asynchronous I/O, event loop,
> coroutines and tasks"
>  RDEPENDS_${PN}-asyncio="${PN}-core"
> -FILES_${PN}-asyncio="${libdir}/python3.5/asyncio "
> +FILES_${PN}-asyncio="${libdir}/python3.5/asyncio
> ${libdir}/python3.5/asyncio/__pycache__ "
>  
>  SUMMARY_${PN}-audio="Python Audio Handling"
>  RDEPENDS_${PN}-audio="${PN}-core"
> -FILES_${PN}-audio="${libdir}/python3.5/wave.*
> ${libdir}/python3.5/chunk.* ${libdir}/python3.5/sndhdr.*
> ${libdir}/python3.5/lib-dynload/ossaudiodev.*.so
> ${libdir}/python3.5/lib-dynload/audioop.*.so
> ${libdir}/python3.5/audiodev.* ${libdir}/python3.5/sunaudio.*
> ${libdir}/python3.5/sunau.* ${libdir}/python3.5/toaiff.* "
> +FILES_${PN}-audio="${libdir}/python3.5/wave.*
> ${libdir}/python3.5/__pycache__/wave.* ${libdir}/python3.5/chunk.*
> ${libdir}/python3.5/__pycache__/chunk.* ${libdir}/python3.5/sndhdr.*
> ${libdir}/python3.5/__pycache__/sndhdr.* ${libdir}/python3.5/lib-
> dynload/ossaudiodev.*.so ${libdir}/python3.5/lib-
> dynload/__pycache__/ossaudiodev.*.so ${libdir}/python3.5/lib-
> dynload/audioop.*.so ${libdir}/python3.5/lib-
> dynload/__pycache__/audioop.*.so ${libdir}/python3.5/audiodev.*
> ${libdir}/python3.5/__pycache__/audiodev.*
> ${libdir}/python3.5/sunaudio.*
> ${libdir}/python3.5/__pycache__/sunaudio.*
> ${libdir}/python3.5/sunau.* ${libdir}/python3.5/__pycache__/sunau.*
> ${libdir}/python3.5/toaiff.* ${libdir}/python3.5/__pycache__/toaiff.*
> "
>  
>  SUMMARY_${PN}-codecs="Python codecs, encodings & i18n support"
>  RDEPENDS_${PN}-codecs="${PN}-core ${PN}-lang"
> -FILES_${PN}-codecs="${libdir}/python3.5/codecs.*
> ${libdir}/python3.5/encodings ${libdir}/python3.5/gettext.*
> ${libdir}/python3.5/locale.* ${libdir}/python3.5/lib-
> dynload/_locale.*.so ${libdir}/python3.5/lib-dynload/_codecs*
> ${libdir}/python3.5/lib-dynload/_multibytecodec.*.so
> ${libdir}/python3.5/lib-dynload/unicodedata.*.so
> ${libdir}/python3.5/stringprep.* ${libdir}/python3.5/xdrlib.* "
> +FILES_${PN}-codecs="${libdir}/python3.5/codecs.*
> ${libdir}/python3.5/__pycache__/codecs.*
> ${libdir}/python3.5/encodings
> ${libdir}/python3.5/encodings/__pycache__
> ${libdir}/python3.5/gettext.*
> ${libdir}/python3.5/__pycache__/gettex

Re: [OE-core] krogoth 2.1.2 status

[Joshua Lock]

On Wed, 2016-11-09 at 12:30 +, Richard Purdie wrote:
> I ran a krogoth 2.1.2 build. It did have some issues:
> 
> * one runtime sanity test had a timeout issue
> * some urls are stale (perpetual problem)
> * musl runtime testing fails (never tested on krogoth previously)
> * no-x11 runtime testing had a failure (never tested on krogoth
> previously)

We can update the autobuilder not to run these tests for krogoth.
I've filed a bug for musl:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=10634

for no-x11 there's a single failure, it might be worthwhile backporting
fixes to make that work?

Regards,

Joshua
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 1/4] lib/oe/path: remove duplicate import

[Joshua Lock]

There's no need to import glob inside copyhardlinktree() as it's
already imported for the entire path module.

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/lib/oe/path.py | 1 -
 1 file changed, 1 deletion(-)

diff --git a/meta/lib/oe/path.py b/meta/lib/oe/path.py
index 06a5af2..f73fd4a 100644
--- a/meta/lib/oe/path.py
+++ b/meta/lib/oe/path.py
@@ -81,7 +81,6 @@ def copyhardlinktree(src, dst):
 subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
 source = ''
 if os.path.isdir(src):
-import glob
 if len(glob.glob('%s/.??*' % src)) > 0:
 source = '%s/.??* ' % src
 source = source + '%s/*' % src
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 4/4] lib/oe/lsb: attempt to ensure consistent distro id regardless of source

[Joshua Lock]

The LSB Distributor ID and os-release NAME differ for most of the
distributions tested by the Yocto Project (CentOS, Debian, Fedora,
openSUSE and Ubuntu) however for all but openSUSE the os-release ID
matches the LSB Distributor ID when both are lowered before
comparison.

Therefore, in order to improve the consistency of identification of
a distribution, switch to using the os-release ID and converting
the ID value to lowercase.

Table showing comparison of LSB Distributor ID to os-release fields NAME
and ID for current Yocto Project supported host distributions:

Distribution | Version | Distributor ID   | NAME | ID   |
-
CentOS   | 7   | CentOS   | CentOS Linux | centos   |
Debian   | 8   | Debian   | Debian GNU/Linux | debian   |
Fedora   | 23  | Fedora   | Fedora   | fedora   |
Fedora   | 24  | Fedora   | Fedora   | fedora   |
openSUSE | 13.2| openSUSE project | openSUSE | opensuse |
openSUSE | 42.1| SUSE LINUX   | openSUSE Leap| opensuse |
Ubuntu   | 14.04   | Ubuntu   | Ubuntu   | ubuntu   |
Ubuntu   | 16.04   | Ubuntu   | Ubuntu   | ubuntu   |

[YOCTO #10591]

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/lib/oe/lsb.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oe/lsb.py b/meta/lib/oe/lsb.py
index 8018c7b..5a795a1 100644
--- a/meta/lib/oe/lsb.py
+++ b/meta/lib/oe/lsb.py
@@ -10,7 +10,7 @@ def release_dict_osr():
 key, val = line.rstrip().split('=', 1)
 except ValueError:
 continue
-if key == 'NAME':
+if key == 'ID':
 data['DISTRIB_ID'] = val.strip('"')
 if key == 'VERSION_ID':
 data['DISTRIB_RELEASE'] = val.strip('"')
@@ -107,7 +107,7 @@ def distro_identifier(adjust_hook=None):
 distro_id = re.sub(r'\W', '', distro_id)
 
 if release:
-id_str = '{0}-{1}'.format(distro_id, release)
+id_str = '{0}-{1}'.format(distro_id.lower(), release)
 else:
 id_str = distro_id
 return id_str.replace(' ','-').replace('/','-')
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 2/4] lib/oe/lsb: make the release dict keys consistent regardless of source

[Joshua Lock]

Rather than have the distro_identifier method look for different keys in
the dict depending on the source ensure that each function for retrieving
release data uses the same key names in the returned dict.

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/lib/oe/lsb.py | 36 +---
 1 file changed, 21 insertions(+), 15 deletions(-)

diff --git a/meta/lib/oe/lsb.py b/meta/lib/oe/lsb.py
index e0bdfba..0bb7686 100644
--- a/meta/lib/oe/lsb.py
+++ b/meta/lib/oe/lsb.py
@@ -1,5 +1,5 @@
-def release_dict():
-"""Return the output of lsb_release -ir as a dictionary"""
+def release_dict_lsb():
+""" Return the output of lsb_release -ir as a dictionary """
 from subprocess import PIPE
 
 try:
@@ -7,19 +7,28 @@ def release_dict():
 except bb.process.CmdError as exc:
 return None
 
+lsb_map = { 'Distributor ID': 'DISTRIB_ID',
+'Release': 'DISTRIB_RELEASE'}
+lsb_keys = lsb_map.keys()
+
 data = {}
 for line in output.splitlines():
-if line.startswith("-e"): line = line[3:]
+if line.startswith("-e"):
+line = line[3:]
 try:
 key, value = line.split(":\t", 1)
 except ValueError:
 continue
-else:
-data[key] = value
+if key in lsb_keys:
+data[lsb_map[key]] = value
+
+if len(data.keys()) != 2:
+return None
+
 return data
 
 def release_dict_file():
-""" Try to gather LSB release information manually when lsb_release tool 
is unavailable """
+""" Try to gather release information manually when other methods fail """
 data = None
 try:
 if os.path.exists('/etc/lsb-release'):
@@ -64,15 +73,12 @@ def distro_identifier(adjust_hook=None):
 
 import re
 
-lsb_data = release_dict()
-if lsb_data:
-distro_id, release = lsb_data['Distributor ID'], lsb_data['Release']
-else:
-lsb_data_file = release_dict_file()
-if lsb_data_file:
-distro_id, release = lsb_data_file['DISTRIB_ID'], 
lsb_data_file.get('DISTRIB_RELEASE', None)
-else:
-distro_id, release = None, None
+distro_data = release_dict_lsb()
+if not distro_data:
+distro_data = release_dict_file()
+
+distro_id = distro_data['DISTRIB_ID']
+release = distro_data['DISTRIB_RELEASE']
 
 if adjust_hook:
 distro_id, release = adjust_hook(distro_id, release)
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 3/4] lib/oe/lsb: prefer /etc/os-release for distribution data

[Joshua Lock]

os-release(5) is an increasingly standard source of operating system
identification and more likely to be present on modern OS deployments, i.e.
many container variants of common distros include os-release and not the
lsb_release tool.

Therefore we should favour parsing /etc/os-release in distro_identifier(),
try lsb_release when that fails and finally fall back on various distro
specific sources of OS identification.

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/lib/oe/lsb.py | 37 -
 1 file changed, 28 insertions(+), 9 deletions(-)

diff --git a/meta/lib/oe/lsb.py b/meta/lib/oe/lsb.py
index 0bb7686..8018c7b 100644
--- a/meta/lib/oe/lsb.py
+++ b/meta/lib/oe/lsb.py
@@ -1,3 +1,25 @@
+def release_dict_osr():
+""" Populate a dict with pertinent values from /etc/os-release """
+if not os.path.exists('/etc/os-release'):
+return None
+
+data = {}
+with open('/etc/os-release') as f:
+for line in f:
+try:
+key, val = line.rstrip().split('=', 1)
+except ValueError:
+continue
+if key == 'NAME':
+data['DISTRIB_ID'] = val.strip('"')
+if key == 'VERSION_ID':
+data['DISTRIB_RELEASE'] = val.strip('"')
+
+if len(data.keys()) != 2:
+return None
+
+return data
+
 def release_dict_lsb():
 """ Return the output of lsb_release -ir as a dictionary """
 from subprocess import PIPE
@@ -46,14 +68,6 @@ def release_dict_file():
 if match:
 data['DISTRIB_ID'] = match.group(1)
 data['DISTRIB_RELEASE'] = match.group(2)
-elif os.path.exists('/etc/os-release'):
-data = {}
-with open('/etc/os-release') as f:
-for line in f:
-if line.startswith('NAME='):
-data['DISTRIB_ID'] = line[5:].rstrip().strip('"')
-if line.startswith('VERSION_ID='):
-data['DISTRIB_RELEASE'] = line[11:].rstrip().strip('"')
 elif os.path.exists('/etc/SuSE-release'):
 data = {}
 data['DISTRIB_ID'] = 'SUSE LINUX'
@@ -73,7 +87,12 @@ def distro_identifier(adjust_hook=None):
 
 import re
 
-distro_data = release_dict_lsb()
+# Try /etc/os-release first, then the output of `lsb_release -ir` and
+# finally fall back on parsing various release files in order to determine
+# host distro name and version.
+distro_data = release_dict_osr()
+if not distro_data:
+distro_data = release_dict_lsb()
 if not distro_data:
 distro_data = release_dict_file()
 
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 0/4] Make oe.lsb.distro_identifier() more consistent

[Joshua Lock]

The oe.lsb.distro_identifier() method call will return different identification
information depending on the source which is found to provide that information.

This series attempts to address this in two ways:
1) preferring os-release(5) as the source of distribution identification. this
  increasingly common standard mechanism is available on each of the build host
  distributions we commonly test on.
2) converting the distribution identifier to lower case before including it in
  the distro_identifier return value. This ensures that, for most of the tested
  distros, the identifier returned via the LSB code paths matches that returned
  by the os-release code paths.

Changes since v1:
* improve release_dict_osr() in patch 3 to handle empty lines and values with 
surrounding quotation marks in /etc/os-release, as used in CentOS


The following changes since commit 9303d8055c45a0f6af295d70a6f6a8b9d8d8a7c9:

  devtool: add "rename" subcommand (2016-11-07 11:04:17 +)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib joshuagl/liboe
  
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=joshuagl/liboe

Joshua Lock (4):
  lib/oe/path: remove duplicate import
  lib/oe/lsb: make the release dict keys consistent regardless of source
  lib/oe/lsb: prefer /etc/os-release for distribution data
  lib/oe/lsb: attempt to ensure consistent distro id regardless of
source

 meta/lib/oe/lsb.py  | 73 +++--
 meta/lib/oe/path.py |  1 -
 2 files changed, 49 insertions(+), 25 deletions(-)

-- 
2.7.4
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/4] lib/oe/path: remove duplicate import

[Joshua Lock]

There's no need to import glob inside copyhardlinktree() as it's
already imported for the entire path module.

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/lib/oe/path.py | 1 -
 1 file changed, 1 deletion(-)

diff --git a/meta/lib/oe/path.py b/meta/lib/oe/path.py
index 06a5af2..f73fd4a 100644
--- a/meta/lib/oe/path.py
+++ b/meta/lib/oe/path.py
@@ -81,7 +81,6 @@ def copyhardlinktree(src, dst):
 subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
 source = ''
 if os.path.isdir(src):
-import glob
 if len(glob.glob('%s/.??*' % src)) > 0:
 source = '%s/.??* ' % src
 source = source + '%s/*' % src
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/4] lib/oe/lsb: prefer /etc/os-release for distribution data

[Joshua Lock]

os-release(5) is an increasingly standard source of operating system
identification and more likely to be present on modern OS deployments, i.e.
many container variants of common distros include os-release and not the
lsb_release tool.

Therefore we should favour parsing /etc/os-release in distro_identifier(),
try lsb_release when that fails and finally fall back on various distro
specific sources of OS identification.

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/lib/oe/lsb.py | 34 +-
 1 file changed, 25 insertions(+), 9 deletions(-)

diff --git a/meta/lib/oe/lsb.py b/meta/lib/oe/lsb.py
index 0bb7686..bdc893f 100644
--- a/meta/lib/oe/lsb.py
+++ b/meta/lib/oe/lsb.py
@@ -1,3 +1,22 @@
+def release_dict_osr():
+""" Populate a dict with pertinent values from /etc/os-release """
+if not os.path.exists('/etc/os-release'):
+return None
+
+data = {}
+with open('/etc/os-release') as f:
+for line in f:
+key, val = line.rstrip().split('=', 1)
+if key == 'NAME':
+data['DISTRIB_ID'] = val
+if key == 'VERSION_ID':
+data['DISTRIB_RELEASE'] = val
+
+if len(data.keys()) != 2:
+return None
+
+return data
+
 def release_dict_lsb():
 """ Return the output of lsb_release -ir as a dictionary """
 from subprocess import PIPE
@@ -46,14 +65,6 @@ def release_dict_file():
 if match:
 data['DISTRIB_ID'] = match.group(1)
 data['DISTRIB_RELEASE'] = match.group(2)
-elif os.path.exists('/etc/os-release'):
-data = {}
-with open('/etc/os-release') as f:
-for line in f:
-if line.startswith('NAME='):
-data['DISTRIB_ID'] = line[5:].rstrip().strip('"')
-if line.startswith('VERSION_ID='):
-data['DISTRIB_RELEASE'] = line[11:].rstrip().strip('"')
 elif os.path.exists('/etc/SuSE-release'):
 data = {}
 data['DISTRIB_ID'] = 'SUSE LINUX'
@@ -73,7 +84,12 @@ def distro_identifier(adjust_hook=None):
 
 import re
 
-distro_data = release_dict_lsb()
+# Try /etc/os-release first, then the output of `lsb_release -ir` and
+# finally fall back on parsing various release files in order to determine
+# host distro name and version.
+distro_data = release_dict_osr()
+if not distro_data:
+distro_data = release_dict_lsb()
 if not distro_data:
 distro_data = release_dict_file()
 
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/4] lib/oe/lsb: make the release dict keys consistent regardless of source

[Joshua Lock]

Rather than have the distro_identifier method look for different keys in
the dict depending on the source ensure that each function for retrieving
release data uses the same key names in the returned dict.

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/lib/oe/lsb.py | 36 +---
 1 file changed, 21 insertions(+), 15 deletions(-)

diff --git a/meta/lib/oe/lsb.py b/meta/lib/oe/lsb.py
index e0bdfba..0bb7686 100644
--- a/meta/lib/oe/lsb.py
+++ b/meta/lib/oe/lsb.py
@@ -1,5 +1,5 @@
-def release_dict():
-"""Return the output of lsb_release -ir as a dictionary"""
+def release_dict_lsb():
+""" Return the output of lsb_release -ir as a dictionary """
 from subprocess import PIPE
 
 try:
@@ -7,19 +7,28 @@ def release_dict():
 except bb.process.CmdError as exc:
 return None
 
+lsb_map = { 'Distributor ID': 'DISTRIB_ID',
+'Release': 'DISTRIB_RELEASE'}
+lsb_keys = lsb_map.keys()
+
 data = {}
 for line in output.splitlines():
-if line.startswith("-e"): line = line[3:]
+if line.startswith("-e"):
+line = line[3:]
 try:
 key, value = line.split(":\t", 1)
 except ValueError:
 continue
-else:
-data[key] = value
+if key in lsb_keys:
+data[lsb_map[key]] = value
+
+if len(data.keys()) != 2:
+return None
+
 return data
 
 def release_dict_file():
-""" Try to gather LSB release information manually when lsb_release tool 
is unavailable """
+""" Try to gather release information manually when other methods fail """
 data = None
 try:
 if os.path.exists('/etc/lsb-release'):
@@ -64,15 +73,12 @@ def distro_identifier(adjust_hook=None):
 
 import re
 
-lsb_data = release_dict()
-if lsb_data:
-distro_id, release = lsb_data['Distributor ID'], lsb_data['Release']
-else:
-lsb_data_file = release_dict_file()
-if lsb_data_file:
-distro_id, release = lsb_data_file['DISTRIB_ID'], 
lsb_data_file.get('DISTRIB_RELEASE', None)
-else:
-distro_id, release = None, None
+distro_data = release_dict_lsb()
+if not distro_data:
+distro_data = release_dict_file()
+
+distro_id = distro_data['DISTRIB_ID']
+release = distro_data['DISTRIB_RELEASE']
 
 if adjust_hook:
 distro_id, release = adjust_hook(distro_id, release)
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 0/4] Make oe.lsb.distro_identifier() more consistent

[Joshua Lock]

The oe.lsb.distro_identifier() method call will return different identification
information depending on the source which is found to provide that information.

This series attempts to address this in two ways:
1) preferring os-release(5) as the source of distribution identification. this
  increasingly common standard mechanism is available on each of the build host
  distributions we commonly test on.
2) converting the distribution identifier to lower case before including it in
  the distro_identifier return value. This ensures that, for most of the tested
  distros, the identifier returned via the LSB code paths matches that returned
  by the os-release code paths.

The following changes since commit 98c6ebf1e05158c689e01b785d32757847cdb10c:

  oeqa/selftest/kernel.py: Add new file destined for kernel related tests 
(2016-11-01 10:05:40 +)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib joshuagl/liboe
  
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=joshuagl/liboe

Joshua Lock (4):
  lib/oe/path: remove duplicate import
  lib/oe/lsb: make the release dict keys consistent regardless of source
  lib/oe/lsb: prefer /etc/os-release for distribution data
  lib/oe/lsb: attempt to ensure consistent distro id regardless of
source

 meta/lib/oe/lsb.py  | 70 +++--
 meta/lib/oe/path.py |  1 -
 2 files changed, 46 insertions(+), 25 deletions(-)

-- 
2.7.4
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 4/4] lib/oe/lsb: attempt to ensure consistent distro id regardless of source

[Joshua Lock]

The LSB Distributor ID and os-release NAME differ for most of the
distributions tested by the Yocto Project (CentOS, Debian, Fedora,
openSUSE and Ubuntu) however for all but openSUSE the os-release ID
matches the LSB Distributor ID when both are lowered before
comparison.

Therefore, in order to improve the consistency of identification of
a distribution, switch to using the os-release ID and converting
the ID value to lowercase.

Table showing comparison of LSB Distributor ID to os-release fields NAME
and ID for current Yocto Project supported host distributions:

Distribution | Version | Distributor ID   | NAME | ID   |
-
CentOS   | 7   | CentOS   | CentOS Linux | centos   |
Debian   | 8   | Debian   | Debian GNU/Linux | debian   |
Fedora   | 23  | Fedora   | Fedora   | fedora   |
Fedora   | 24  | Fedora   | Fedora   | fedora   |
openSUSE | 13.2| openSUSE project | openSUSE | opensuse |
openSUSE | 42.1| SUSE LINUX   | openSUSE Leap| opensuse |
Ubuntu   | 14.04   | Ubuntu   | Ubuntu   | ubuntu   |
Ubuntu   | 16.04   | Ubuntu   | Ubuntu   | ubuntu   |

[YOCTO #10591]

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/lib/oe/lsb.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oe/lsb.py b/meta/lib/oe/lsb.py
index bdc893f..cd43731 100644
--- a/meta/lib/oe/lsb.py
+++ b/meta/lib/oe/lsb.py
@@ -7,7 +7,7 @@ def release_dict_osr():
 with open('/etc/os-release') as f:
 for line in f:
 key, val = line.rstrip().split('=', 1)
-if key == 'NAME':
+if key == 'ID':
 data['DISTRIB_ID'] = val
 if key == 'VERSION_ID':
 data['DISTRIB_RELEASE'] = val
@@ -104,7 +104,7 @@ def distro_identifier(adjust_hook=None):
 distro_id = re.sub(r'\W', '', distro_id)
 
 if release:
-id_str = '{0}-{1}'.format(distro_id, release)
+id_str = '{0}-{1}'.format(distro_id.lower(), release)
 else:
 id_str = distro_id
 return id_str.replace(' ','-').replace('/','-')
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 7/9] selftest: skip runtime-test's test_testimage_install on poky-tiny distro

[Joshua Lock]

On Mon, 2016-10-17 at 11:02 -0500, Leonardo Sandoval wrote:
> 
> On 10/17/2016 02:09 AM, Joshua Lock wrote:
> > 
> > On Fri, 2016-10-14 at 11:40 -0500,
> > leonardo.sandoval.gonza...@linux.intel.com wrote:
> > > 
> > > From: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.c
> > > om>
> > > 
> > > Distro poky-tiny cannot build the core-image-full-cmdline image,
> > > so
> > > skip the
> > > test if this is the case.
> > > 
> > > [YOCTO #8525]
> > Wouldn't it be better to use a different image, most likely with
> > extra
> > packages installed, and maybe a different list of TEST_SUITES for
> > poky-
> > tiny so that we're still testing this functionality on poky-tiny?
> 
> Yes, that is a good idea. Which image and the accompanied tests do
> you 
> suggest?

I'd be inclined to suggest core-image-minimal with extra packages
installed to enable the same TEST_SUITES, but I can't recall exactly
what's being tested here. Would that combination make sense?

Thanks,

Joshua

> 
> > 
> > Thanks,
> > 
> > Joshua
> > 
> > > 
> > > Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linu
> > > x.in
> > > tel.com>
> > > ---
> > >   meta/lib/oeqa/selftest/runtime-test.py | 2 ++
> > >   1 file changed, 2 insertions(+)
> > > 
> > > diff --git a/meta/lib/oeqa/selftest/runtime-test.py
> > > b/meta/lib/oeqa/selftest/runtime-test.py
> > > index c2d5b45..e17c261 100644
> > > --- a/meta/lib/oeqa/selftest/runtime-test.py
> > > +++ b/meta/lib/oeqa/selftest/runtime-test.py
> > > @@ -94,6 +94,8 @@ class TestImage(oeSelfTest):
> > >   Product: oe-core
> > >   Author: Mariano Lopez <mariano.lo...@intel.com>
> > >   """
> > > +if self.distro == 'poky-tiny':
> > > +self.skipTest('core-image-full-cmdline not buildable
> > > for
> > > poky-tiny')
> > >   
> > >   features = 'INHERIT += "testimage"\n'
> > >   features += 'TEST_SUITES = "ping ssh selftest"\n'
> > > -- 
> > > 2.1.4
> > > 
> 
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 4/9] selftest: skip some devtool tests when distro is poky-tiny due to missing libx11

[Joshua Lock]

On Mon, 2016-10-17 at 10:19 -0500, Leonardo Sandoval wrote:
> 
> 
> 
> 
> 
> 
> On 10/17/2016 06:42 AM, Burton, Ross
>   wrote:
> 
> 
> 
> >   
> > 
> > 
> > > >   On 14 October 2016 at 17:40, 
> > wrote:
> > 
> > >      
> > >  def test_devtool_add_fetch_git(self):
> > > 
> > > -        # Fetch source
> > > 
> > > +        if self.distro == 'poky-tiny':
> > > 
> > > +            self.skipTest('libmatchbox2 is not
> > > > > > buildable with poky-tiny because it requires a
libx11
> > > provider')
> > > 
> > > > > >          tempdir =
tempfile.mkdtemp(prefix='devtoolqa')
> > > 
> > >          self.track_for_cleanup(tempdir)
> > > 
> > > > > >          url =
'git://git.yoctoproject.org/libmatchbox'
> > > 
> > > 
> >   
> > 
> >   
> > 
> >   Changing the test to use a recipe that doesn't require a
> > > >   distro feature that many distros disable seems like a
more
> >   sustainable fix.
> > 
> > 
> > 
> > 
> > 
> >   
> > 
> Which recipe do you think would be more generic?

Something in the minimal image which doesn't rely on DISTRO_FEATURES
which aren't in the default distro.

Thanks,

Joshua
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/9] selftest: skip two bbtest tests depending on distro

[Joshua Lock]

On Mon, 2016-10-17 at 11:04 -0500, Leonardo Sandoval wrote:
> 
> On 10/17/2016 02:08 AM, Joshua Lock wrote:
> > 
> > On Fri, 2016-10-14 at 11:40 -0500,
> > leonardo.sandoval.gonza...@linux.intel.com wrote:
> > > 
> > > From: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.c
> > > om>
> > > 
> > > There are two bbtests that just makes sense on certain distros:
> > > test_bitbake_g cannot be done on poky-tiny and test_non_gplv3
> > > just makes sense on poky and poky-tiny. Skip these when
> > > necessary under the latter conditions.
> > > 
> > > [YOCTO #8525]
> > > 
> > > Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linu
> > > x.in
> > > tel.com>
> > > ---
> > >   meta/lib/oeqa/selftest/bbtests.py | 7 +++
> > >   1 file changed, 7 insertions(+)
> > > 
> > > diff --git a/meta/lib/oeqa/selftest/bbtests.py
> > > b/meta/lib/oeqa/selftest/bbtests.py
> > > index baae1e0..9b11cd9 100644
> > > --- a/meta/lib/oeqa/selftest/bbtests.py
> > > +++ b/meta/lib/oeqa/selftest/bbtests.py
> > > @@ -114,6 +114,9 @@ class BitbakeTests(oeSelfTest):
> > >   
> > >   @testcase(167)
> > >   def test_bitbake_g(self):
> > > +if self.distro == 'poky-tiny':
> > > +self.skipTest('core-image-full-cmdline is not
> > > buildable
> > > with poky-tiny')
> > > +
> > This test is still valid for poky-tiny, it's just that the image
> > the
> > test builds isn't buildable with poky-tiny, right?
> > 
> > I think it would be better to change the test to a different,
> > smaller,
> > image that can also be built for poky-tiny (core-image-minimal?).
> > We'd also need to change the target the test checks for the
> > presence of
> > in pn-buildlist.
> Good idea. Which image and pn-buildlist package check do you suggest?

It doesn't really matter and will depend on which image you decide to
build for the test. 

The goal of the test appears to be to validate the output of `bitbake
-e`. Hopefully we can find an image and recipe combination which is
buildable across distros but I don't know what that would be off the
top of my head.

Regards,

Joshua

> > 
> > Regards,
> > 
> > Joshua
> > 
> > > 
> > >   result = bitbake('-g core-image-full-cmdline')
> > >   for f in ['pn-buildlist', 'pn-depends.dot', 'package-
> > > depends.dot', 'task-depends.dot']:
> > >   self.addCleanup(os.remove, f)
> > > @@ -229,6 +232,10 @@ INHERIT_remove = \"report-error\"
> > >   
> > >   @testcase(1119)
> > >   def test_non_gplv3(self):
> > > +supported_distros = ['poky', 'poky-tiny']
> > > +if not self.distro in supported_distros:
> > > +self.skipTest('Test considers only %s distros' %
> > > ','.join(supported_distros))
> > > +
> > >   data = 'INCOMPATIBLE_LICENSE = "GPLv3"'
> > >   conf = os.path.join(self.builddir, 'conf/local.conf')
> > >   ftools.append_file(conf ,data)
> > > -- 
> > > 2.1.4
> > > 
> 
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 7/9] selftest: skip runtime-test's test_testimage_install on poky-tiny distro

[Joshua Lock]

On Fri, 2016-10-14 at 11:40 -0500,
leonardo.sandoval.gonza...@linux.intel.com wrote:
> From: Leonardo Sandoval 
> 
> Distro poky-tiny cannot build the core-image-full-cmdline image, so
> skip the
> test if this is the case.
> 
> [YOCTO #8525]

Wouldn't it be better to use a different image, most likely with extra
packages installed, and maybe a different list of TEST_SUITES for poky-
tiny so that we're still testing this functionality on poky-tiny?

Thanks,

Joshua

> 
> Signed-off-by: Leonardo Sandoval  tel.com>
> ---
>  meta/lib/oeqa/selftest/runtime-test.py | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/meta/lib/oeqa/selftest/runtime-test.py
> b/meta/lib/oeqa/selftest/runtime-test.py
> index c2d5b45..e17c261 100644
> --- a/meta/lib/oeqa/selftest/runtime-test.py
> +++ b/meta/lib/oeqa/selftest/runtime-test.py
> @@ -94,6 +94,8 @@ class TestImage(oeSelfTest):
>  Product: oe-core
>  Author: Mariano Lopez 
>  """
> +if self.distro == 'poky-tiny':
> +self.skipTest('core-image-full-cmdline not buildable for
> poky-tiny')
>  
>  features = 'INHERIT += "testimage"\n'
>  features += 'TEST_SUITES = "ping ssh selftest"\n'
> -- 
> 2.1.4
> 
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/9] selftest: skip two bbtest tests depending on distro

[Joshua Lock]

On Fri, 2016-10-14 at 11:40 -0500,
leonardo.sandoval.gonza...@linux.intel.com wrote:
> From: Leonardo Sandoval 
> 
> There are two bbtests that just makes sense on certain distros:
> test_bitbake_g cannot be done on poky-tiny and test_non_gplv3
> just makes sense on poky and poky-tiny. Skip these when
> necessary under the latter conditions.
> 
> [YOCTO #8525]
> 
> Signed-off-by: Leonardo Sandoval  tel.com>
> ---
>  meta/lib/oeqa/selftest/bbtests.py | 7 +++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/meta/lib/oeqa/selftest/bbtests.py
> b/meta/lib/oeqa/selftest/bbtests.py
> index baae1e0..9b11cd9 100644
> --- a/meta/lib/oeqa/selftest/bbtests.py
> +++ b/meta/lib/oeqa/selftest/bbtests.py
> @@ -114,6 +114,9 @@ class BitbakeTests(oeSelfTest):
>  
>  @testcase(167)
>  def test_bitbake_g(self):
> +if self.distro == 'poky-tiny':
> +self.skipTest('core-image-full-cmdline is not buildable
> with poky-tiny')
> +

This test is still valid for poky-tiny, it's just that the image the
test builds isn't buildable with poky-tiny, right?

I think it would be better to change the test to a different, smaller,
image that can also be built for poky-tiny (core-image-minimal?). 
We'd also need to change the target the test checks for the presence of
in pn-buildlist.

Regards,

Joshua

>  result = bitbake('-g core-image-full-cmdline')
>  for f in ['pn-buildlist', 'pn-depends.dot', 'package-
> depends.dot', 'task-depends.dot']:
>  self.addCleanup(os.remove, f)
> @@ -229,6 +232,10 @@ INHERIT_remove = \"report-error\"
>  
>  @testcase(1119)
>  def test_non_gplv3(self):
> +supported_distros = ['poky', 'poky-tiny']
> +if not self.distro in supported_distros:
> +self.skipTest('Test considers only %s distros' %
> ','.join(supported_distros))
> +
>  data = 'INCOMPATIBLE_LICENSE = "GPLv3"'
>  conf = os.path.join(self.builddir, 'conf/local.conf')
>  ftools.append_file(conf ,data)
> -- 
> 2.1.4
> 
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/2] local.conf.sample.extended: remove RM_OLD_IMAGE

[Joshua Lock]

RM_OLD_IMAGE has been removed so don't include this variable in
sample local.conf

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/conf/local.conf.sample.extended | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/meta/conf/local.conf.sample.extended 
b/meta/conf/local.conf.sample.extended
index da4820d..57f656a 100644
--- a/meta/conf/local.conf.sample.extended
+++ b/meta/conf/local.conf.sample.extended
@@ -266,9 +266,6 @@
 #COPYLEFT_RECIPE_TYPES = 'target'
 #
 
-# Remove the old image before the new one generated to save disk space
-#RM_OLD_IMAGE = "1"
-
 #
 # GCC/LD FLAGS to enable more secure code generation
 #
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/2] Remove RM_OLD_IMAGE, it's no longer useful

[Joshua Lock]

Since the move to put image deployment under sstate control in
d54339d4b1a7e884de636f6325ca60409ebd95ff old images are automatically
removed before a new image is deployed (the default behaviour of the
sstate logic).

RM_OLD_IMAGE is therefore no longer required to provide this
behaviour, remove the variable and its users.

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta-selftest/recipes-test/images/wic-image-minimal.bb |  1 -
 meta/classes/cve-check.bbclass |  2 --
 meta/classes/image-vm.bbclass  |  8 
 meta/classes/image.bbclass |  5 +
 meta/classes/qemuboot.bbclass  |  3 ---
 meta/classes/rootfs-postcommands.bbclass   |  3 ---
 meta/conf/documentation.conf   |  1 -
 meta/lib/oeqa/selftest/buildoptions.py | 16 
 8 files changed, 1 insertion(+), 38 deletions(-)

diff --git a/meta-selftest/recipes-test/images/wic-image-minimal.bb 
b/meta-selftest/recipes-test/images/wic-image-minimal.bb
index 58bf5a5..9e93b8e 100644
--- a/meta-selftest/recipes-test/images/wic-image-minimal.bb
+++ b/meta-selftest/recipes-test/images/wic-image-minimal.bb
@@ -5,7 +5,6 @@ SRC_URI = "file://${FILE_DIRNAME}/${BPN}.wks"
 IMAGE_INSTALL = "packagegroup-core-boot ${ROOTFS_PKGMANAGE_BOOTSTRAP}"
 
 IMAGE_FSTYPES = "wic"
-RM_OLD_IMAGE = "1"
 
 DEPENDS = "syslinux syslinux-native dosfstools-native mtools-native 
gptfdisk-native"
 
diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index 8251ca7..1425a40 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -92,8 +92,6 @@ python cve_check_write_rootfs_manifest () {
 manifest_link = os.path.join(deploy_dir, "%s.cve" % link_name)
 # If we already have another manifest, update symlinks
 if os.path.exists(os.path.realpath(manifest_link)):
-if d.getVar('RM_OLD_IMAGE', True) == "1":
-os.remove(os.path.realpath(manifest_link))
 os.remove(manifest_link)
 os.symlink(os.path.basename(manifest_name), manifest_link)
 bb.plain("Image CVE report stored in: %s" % manifest_name)
diff --git a/meta/classes/image-vm.bbclass b/meta/classes/image-vm.bbclass
index 32c6550..2f35d6b 100644
--- a/meta/classes/image-vm.bbclass
+++ b/meta/classes/image-vm.bbclass
@@ -106,10 +106,6 @@ build_boot_dd() {
 
cd ${IMGDEPLOYDIR}
 
-   if [ "${RM_OLD_IMAGE}" = "1" ] && [ -L 
${IMGDEPLOYDIR}/${IMAGE_LINK_NAME}.hdddirect ]; then
-   rm -f $(readlink -f 
${IMGDEPLOYDIR}/${IMAGE_LINK_NAME}.hdddirect)
-   fi
-
ln -sf ${IMAGE_NAME}.hdddirect 
${IMGDEPLOYDIR}/${IMAGE_LINK_NAME}.hdddirect
 } 
 
@@ -147,10 +143,6 @@ run_qemu_img (){
 type="$1"
 qemu-img convert -O $type ${IMGDEPLOYDIR}/${IMAGE_LINK_NAME}.hdddirect 
${IMGDEPLOYDIR}/${IMAGE_NAME}.$type
 
-if [ "${RM_OLD_IMAGE}" = "1" ] && [ -L 
${IMGDEPLOYDIR}/${IMAGE_LINK_NAME}.$type ]; then
-rm -f $(readlink -f ${IMGDEPLOYDIR}/${IMAGE_LINK_NAME}.$type)
-fi
-
 ln -sf ${IMAGE_NAME}.$type ${IMGDEPLOYDIR}/${IMAGE_LINK_NAME}.$type
 }
 create_vmdk_image () {
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 915500a..6111f6d 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -117,7 +117,7 @@ python () {
 def rootfs_variables(d):
 from oe.rootfs import variable_depends
 variables = 
['IMAGE_DEVICE_TABLE','IMAGE_DEVICE_TABLES','BUILD_IMAGES_FROM_FEEDS','IMAGE_TYPES_MASKED','IMAGE_ROOTFS_ALIGNMENT','IMAGE_OVERHEAD_FACTOR','IMAGE_ROOTFS_SIZE','IMAGE_ROOTFS_EXTRA_SPACE',
- 
'IMAGE_ROOTFS_MAXSIZE','IMAGE_NAME','IMAGE_LINK_NAME','IMAGE_MANIFEST','DEPLOY_DIR_IMAGE','RM_OLD_IMAGE','IMAGE_FSTYPES','IMAGE_INSTALL_COMPLEMENTARY','IMAGE_LINGUAS',
+ 
'IMAGE_ROOTFS_MAXSIZE','IMAGE_NAME','IMAGE_LINK_NAME','IMAGE_MANIFEST','DEPLOY_DIR_IMAGE','IMAGE_FSTYPES','IMAGE_INSTALL_COMPLEMENTARY','IMAGE_LINGUAS',
  
'MULTILIBRE_ALLOW_REP','MULTILIB_TEMP_ROOTFS','MULTILIB_VARIANTS','MULTILIBS','ALL_MULTILIB_PACKAGE_ARCHS','MULTILIB_GLOBAL_VARIANTS','BAD_RECOMMENDATIONS','NO_RECOMMENDATIONS',
  
'PACKAGE_ARCHS','PACKAGE_CLASSES','TARGET_VENDOR','TARGET_ARCH','TARGET_OS','OVERRIDES','BBEXTENDVARIANT','FEED_DEPLOYDIR_BASE_URI','INTERCEPT_DIR','USE_DEVFS',
  'CONVERSIONTYPES', 'IMAGE_GEN_DEBUGFS', 'ROOTFS_RO_UNNEEDED', 
'IMGDEPLOYDIR', 'PACKAGE_EXCLUDE_COMPLEMENTARY']
@@ -588,9 +588,6 @@ python create_symlinks() {
 if os.path.exists(os.path.join(deploy_dir, src)):
 bb.note("Creating symlink: %s -> %s" % (dst, src))
 if os.path.islink(dst):
- 

Re: [OE-core] rpm: Recover RPM4 to OE-core

[Joshua Lock]

On Fri, 2016-10-07 at 12:19 +0900, Fan Xin wrote:
> Hi Joshua,
> 
> Thanks for your info.
> 
> Actually, our origin motivation is that we found smartpm is not 
> maintained any more. We maintain smartpm for our distro at
> https://github.com/ubinux/smart2

Indeed, the lack of maintenance for Smart is one of the reasons we're
looking to switch away from it.

> Then we realize smartpm should switch to python3. In order to solve
> this 
> issue, there are four solutions as follows.
> 
> [1] smart/RPM5
> [2] smart/RPM4
> [3] dnf/RPM5
> [4] dnf/RPM4
> 
> I confirm that RPM5 have some bugs with python3 and I am not
> familiar 
> with RPM. So I think the easy way to solve this issue is [2],
> recover 
> RPM4 and solve the bugs in smartpm.
> 
> > 
> > 
> > It's also worth pointing out that we're strongly considering
> > dropping
> > SMART in the next (2.3) development cycle[4].
> > 
> Considering YP would like to drop smart in 2.3, I wonder YP would
> use 
> [3]dnf/RPM5 or [4]dnf/RPM4 ?

There are two main reasons we want to switch away from Smart. Firstly
we made a concerted effort to switch to Python3 in the 2.2 cycle —
Smart is one of the few recipes in OE-Core which still pulls in
Python2, maybe the only one we use by default (for RPM using distros).

Secondly Smart is unmaintained and has several known issues related to
its abstraction over package backends.

So far as I'm aware no decision has been made as to what we'll replace
Smart with yet. I believe the first step will be to investigate RPM5 &
dnf.

Regards,

Joshua
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] pseudo: backport a patch to fix renameat()

[Joshua Lock]

renameat calls under pseudo were losing extended attributes.
Backport the fix for this from pseudo upstream.

[YOCTO '10349]

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 ...nameat-parallel-to-previous-fix-to-rename.patch | 64 ++
 meta/recipes-devtools/pseudo/pseudo_1.8.1.bb   |  1 +
 2 files changed, 65 insertions(+)
 create mode 100644 
meta/recipes-devtools/pseudo/files/0003-Fix-renameat-parallel-to-previous-fix-to-rename.patch

diff --git 
a/meta/recipes-devtools/pseudo/files/0003-Fix-renameat-parallel-to-previous-fix-to-rename.patch
 
b/meta/recipes-devtools/pseudo/files/0003-Fix-renameat-parallel-to-previous-fix-to-rename.patch
new file mode 100644
index 000..739c03e
--- /dev/null
+++ 
b/meta/recipes-devtools/pseudo/files/0003-Fix-renameat-parallel-to-previous-fix-to-rename.patch
@@ -0,0 +1,64 @@
+From d9ab3a0acc94151048498b1ea4d69e7707df1526 Mon Sep 17 00:00:00 2001
+From: Seebs <se...@seebs.net>
+Date: Fri, 30 Sep 2016 10:56:35 -0500
+Subject: [PATCH 3/3] Fix renameat (parallel to previous fix to rename)
+
+There was a bug in rename(), which was duplicated when renameat() was
+implemented, and which got fixed two years ago for rename(), but no
+one ever uses renameat() so it didn't get fixed there. Thanks
+to Anton Gerasimov <an...@advancedtelematic.com> for the bug report
+and patch.
+
+Signed-off-by: Seebs <se...@seebs.net>
+
+Upstream-Status: Backport
+Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
+
+---
+ ChangeLog.txt  | 4 
+ ports/unix/guts/renameat.c | 7 ++-
+ 2 files changed, 10 insertions(+), 1 deletion(-)
+
+diff --git a/ChangeLog.txt b/ChangeLog.txt
+index 65b9759..ca04cc0 100644
+--- a/ChangeLog.txt
 b/ChangeLog.txt
+@@ -1,3 +1,7 @@
++2016-09-30:
++  * (seebs) Fix rename at, matching fix from ee00f63d for rename. Bug
++and fix provided by Anton Gerasimov <an...@advancedtelematic.com>.
++
+ 2016-09-28:
+   * (seebs) Send errors to log when daemonizing, but do that a lot
+ sooner to prevent startup messages which can show up spuriously
+diff --git a/ports/unix/guts/renameat.c b/ports/unix/guts/renameat.c
+index ade0509..d5e36fa 100644
+--- a/ports/unix/guts/renameat.c
 b/ports/unix/guts/renameat.c
+@@ -11,6 +11,7 @@
+   int oldrc, newrc;
+   int save_errno;
+   int old_db_entry = 0;
++  int may_unlinked = 0;
+ 
+   pseudo_debug(PDBGF_FILE, "renameat: %d,%s->%d,%s\n",
+   olddirfd, oldpath ? oldpath : "",
+@@ -44,10 +45,14 @@
+   /* as with unlink, we have to mark that the file may get deleted */
+   msg = pseudo_client_op(OP_MAY_UNLINK, 0, -1, newdirfd, newpath, newrc ? 
NULL : );
+   if (msg && msg->result == RESULT_SUCCEED)
++  may_unlinked = 1;
++  msg = pseudo_client_op(OP_STAT, 0, -1, olddirfd, oldpath, oldrc ? NULL 
: );
++  if (msg && msg->result == RESULT_SUCCEED)
+   old_db_entry = 1;
++
+   rc = real_renameat(olddirfd, oldpath, newdirfd, newpath);
+   save_errno = errno;
+-  if (old_db_entry) {
++  if (may_unlinked) {
+   if (rc == -1) {
+   /* since we failed, that wasn't really unlinked -- put
+* it back.
+-- 
+2.7.4
+
diff --git a/meta/recipes-devtools/pseudo/pseudo_1.8.1.bb 
b/meta/recipes-devtools/pseudo/pseudo_1.8.1.bb
index 67eb298..fb70034 100644
--- a/meta/recipes-devtools/pseudo/pseudo_1.8.1.bb
+++ b/meta/recipes-devtools/pseudo/pseudo_1.8.1.bb
@@ -9,6 +9,7 @@ SRC_URI = 
"http://downloads.yoctoproject.org/releases/pseudo/${BPN}-${PV}.tar.bz
file://0001-Don-t-send-SIGUSR1-to-init.patch \
file://0001-Quiet-diagnostics-during-startup-for-pseudo-d.patch \
file://0002-Use-correct-file-descriptor.patch \
+   file://0003-Fix-renameat-parallel-to-previous-fix-to-rename.patch \
"
 
 SRC_URI[md5sum] = "ee38e4fb62ff88ad067b1a5a3825bac7"
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] rpm: Recover RPM4 to OE-core

[Joshua Lock]

Hi Fan,

On Thu, 2016-10-06 at 20:18 +0900, Fan Xin wrote:
> By the way, could anyone give more detail info about the bugs in RPM

We removed rpm4 early this year[1] because rather than have people be
surprised that it didn't work we decided to drop support for rpm4 until
such time as the project had sufficient resources to properly maintain
it.

There were at least two issues[2][3] reported that indicated rpm4
support was incomplete, though I believe they were mostly related to
Smart's interactions with RPM 4 rather than any issue in RPM 4 itself.

It's also worth pointing out that we're strongly considering dropping
SMART in the next (2.3) development cycle[4].

Do you have a reason for wanting rpm4 beyond the proven Python3
support?

Regards,

Joshua

1. http://lists.openembedded.org/pipermail/openembedded-core/2016-March
/118849.html
2. https://bugzilla.yoctoproject.org/show_bug.cgi?id=8968
3. https://bugzilla.yoctoproject.org/show_bug.cgi?id=8969
4. https://bugzilla.yoctoproject.org/show_bug.cgi?id=9675
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] testimage: disable build tests for qemumips and qemumips64

[Joshua Lock]

It's not uncommon for qemumips[64] builds on the Yocto Project
autobuilder to fail during Sanity Tests after a very long timeout
period. This is due to the MIPS emulation in QEMU being slow and
some of the build tests taking a very long time on MIPS machines.

This patch works around this slowness by disabling the more
complex build tests for QEMU MIPS machines.

[YOCTO #10340]

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/classes/testimage.bbclass | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 5ddbecb..6b6781d 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -58,6 +58,12 @@ DEFAULT_TEST_SUITES_pn-meta-toolchain = "auto"
 # aarch64 has no graphics
 DEFAULT_TEST_SUITES_remove_aarch64 = "xorg"
 
+# qemumips is quite slow and has reached the timeout limit several times on 
the YP build cluster,
+# mitigate this by removing build tests for qemumips machines.
+MIPSREMOVE ??= "buildcvs buildiptables buildgalculator"
+DEFAULT_TEST_SUITES_remove_qemumips = "${MIPSREMOVE}"
+DEFAULT_TEST_SUITES_remove_qemumips64 = "${MIPSREMOVE}"
+
 TEST_SUITES ?= "${DEFAULT_TEST_SUITES}"
 
 TEST_QEMUBOOT_TIMEOUT ?= "1000"
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] multilib_header: avoid sstate checksum issues for -nativesdk recipes

[Joshua Lock]

On Fri, 2016-09-30 at 08:29 -0700, akuster808 wrote:
> 
> On 09/30/2016 02:09 AM, Joshua Lock wrote:
> > 
> > Much as with -native recipes, as addressed in commit
> > b15730caf0d4c40271796887505507f2501958bb, arch specific variables
> > like MIPSPKGSFX_ABI were affecting -nativesdk sstate checksums for
> > recipes like nativesdk-glibc-initial.
> Since commit
> 
> b15730caf0d4c40271796887505507f2501958bb is in Jethro and Krogoth, do
> both of those branch need this fix?

This patch improves sstate object reuse for nativesdk recipes when
switching between a mips and non-mips MACHINE. 
It's no doubt useful to have in krogoth and jethro too.

Joshua

> - armin
> 
> > 
> > 
> > Disable multilib_header for nativesdk as we don't use multilibs in
> > this scenario.
> > 
> > [YOCTO #10320]
> > 
> > Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
> > ---
> >   meta/classes/multilib_header.bbclass | 5 +
> >   1 file changed, 5 insertions(+)
> > 
> > diff --git a/meta/classes/multilib_header.bbclass
> > b/meta/classes/multilib_header.bbclass
> > index 5ee0a2d..304c28e 100644
> > --- a/meta/classes/multilib_header.bbclass
> > +++ b/meta/classes/multilib_header.bbclass
> > @@ -52,3 +52,8 @@ oe_multilib_header() {
> >   oe_multilib_header_class-native () {
> >     return
> >   }
> > +
> > +# Nor do we need multilib headers for nativesdk builds.
> > +oe_multilib_header_class-nativesdk () {
> > +   return
> > +}
> 
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] multilib_header: avoid sstate checksum issues for -nativesdk recipes

[Joshua Lock]

Much as with -native recipes, as addressed in commit
b15730caf0d4c40271796887505507f2501958bb, arch specific variables
like MIPSPKGSFX_ABI were affecting -nativesdk sstate checksums for
recipes like nativesdk-glibc-initial.

Disable multilib_header for nativesdk as we don't use multilibs in
this scenario.

[YOCTO #10320]

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/classes/multilib_header.bbclass | 5 +
 1 file changed, 5 insertions(+)

diff --git a/meta/classes/multilib_header.bbclass 
b/meta/classes/multilib_header.bbclass
index 5ee0a2d..304c28e 100644
--- a/meta/classes/multilib_header.bbclass
+++ b/meta/classes/multilib_header.bbclass
@@ -52,3 +52,8 @@ oe_multilib_header() {
 oe_multilib_header_class-native () {
return
 }
+
+# Nor do we need multilib headers for nativesdk builds.
+oe_multilib_header_class-nativesdk () {
+   return
+}
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH RFC] scripts/runqemu: Be more loose when guessing QB_SYSTEM_NAME variable

[Joshua Lock]

On Tue, 2016-09-27 at 11:27 -0700, California Sullivan wrote:
> Following the instructions to reproduce YOCTO #10026 I found that
> runqemu would fail to start because QB_SYSTEM_NAME is null. This
> patch
> makes the guessing algorithm more loose allowing it to correctly
> guess
> the variable from the MACHINE name.

Out of interest can you help me understand how/when the current code
fails?

> 
> It is still a good assumption that if it contains "qemux86-64" for
> example, that the system is x86-64, so this is unlikely to cause any
> new issues while preserving some peoples' old workflows.
> 
> Signed-off-by: California Sullivan 
> ---
>  scripts/runqemu | 20 ++--
>  1 file changed, 10 insertions(+), 10 deletions(-)
> 
> diff --git a/scripts/runqemu b/scripts/runqemu
> index 45bcad7..b06d09e 100755
> --- a/scripts/runqemu
> +++ b/scripts/runqemu
> @@ -870,20 +870,20 @@ class BaseConfig(object):
>  if not mach:
>  return None
>  
> -if mach == 'qemuarm':
> -qbsys = 'arm'
> -elif mach == 'qemuarm64':
> +elif 'qemuarm64' in mach:

This elif should be below the "if 'qemuarm' in mach" below which begins
the control flow statements for determining the value to assign to the
qbsys variable.

>  qbsys = 'aarch64'
> -elif mach == 'qemux86':
> -qbsys = 'i386'
> -elif mach == 'qemux86-64':
> +if 'qemuarm' in mach:
> +qbsys = 'arm'
> +elif 'qemux86-64' in mach:
>  qbsys = 'x86_64'
> -elif mach == 'qemuppc':
> +elif 'qemux86' in mach:
> +qbsys = 'i386'
> +elif 'qemuppc' in mach:
>  qbsys = 'ppc'
> -elif mach == 'qemumips':
> -qbsys = 'mips'
> -elif mach == 'qemumips64':
> +elif 'qemumips64' in mach:
>  qbsys = 'mips64'
> +elif 'qemumips' in mach:
> +qbsys = 'mips'
>  
>  return 'qemu-system-%s' % qbsys
>  
> -- 
> 2.5.5
> 
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/2] buildtools-tarball: improve stamp independence

[Joshua Lock]

buildtools-tarball shouldn't be regenerated when MACHINE changes,
nor should variants for other SDKMACHINE be removed from the deploy
directory when SDKMACHINE changes.

Remove target architecture dependencies so that deploy artefacts
can overlap.

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/recipes-core/meta/buildtools-tarball.bb | 15 +--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-core/meta/buildtools-tarball.bb 
b/meta/recipes-core/meta/buildtools-tarball.bb
index 2e738e6..da18c09 100644
--- a/meta/recipes-core/meta/buildtools-tarball.bb
+++ b/meta/recipes-core/meta/buildtools-tarball.bb
@@ -28,10 +28,13 @@ TOOLCHAIN_HOST_TASK ?= "\
 
 MULTIMACH_TARGET_SYS = "${SDK_ARCH}-nativesdk${SDK_VENDOR}-${SDK_OS}"
 PACKAGE_ARCH = "${SDK_ARCH}"
+PACKAGE_ARCHS = ""
+TARGET_ARCH = "none"
+TARGET_OS = "none"
 
 SDK_PACKAGE_ARCHS += "buildtools-dummy-${SDKPKGSUFFIX}"
 
-TOOLCHAIN_OUTPUTNAME ?= 
"${SDK_NAME}-buildtools-nativesdk-standalone-${DISTRO_VERSION}"
+TOOLCHAIN_OUTPUTNAME ?= 
"${SDK_ARCH}-buildtools-nativesdk-standalone-${DISTRO_VERSION}"
 
 SDK_TITLE = "Build tools"
 
@@ -41,10 +44,18 @@ EXCLUDE_FROM_WORLD = "1"
 
 inherit meta
 inherit populate_sdk
-inherit toolchain-scripts
+inherit toolchain-scripts-base
+inherit nopackages
+
+deltask install
+deltask package
+deltask packagedata
+deltask populate_sysroot
 
 do_populate_sdk[stamp-extra-info] = "${SDKMACHINE}"
 
+REAL_MULTIMACH_TARGET_SYS = "none"
+
 create_sdk_files_append () {
rm -f ${SDK_OUTPUT}/${SDKPATH}/site-config-*
rm -f ${SDK_OUTPUT}/${SDKPATH}/environment-setup-*
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/2] toolchain-scripts-base: add base class for toolchain_create_sdk_version

[Joshua Lock]

We use toolchain_create_sdk_version() in buildtools-tarball but
don't want the extra classes toolchain-scripts pulls in, therefore
split out a separate base class for this function which both
toolchain-scripts and the buildtools-tarball can inherit.

Signed-off-by: Joshua Lock <joshua.g.l...@intel.com>
---
 meta/classes/toolchain-scripts-base.bbclass | 11 +++
 meta/classes/toolchain-scripts.bbclass  | 14 +-
 2 files changed, 12 insertions(+), 13 deletions(-)
 create mode 100644 meta/classes/toolchain-scripts-base.bbclass

diff --git a/meta/classes/toolchain-scripts-base.bbclass 
b/meta/classes/toolchain-scripts-base.bbclass
new file mode 100644
index 000..2489b9d
--- /dev/null
+++ b/meta/classes/toolchain-scripts-base.bbclass
@@ -0,0 +1,11 @@
+#This function create a version information file
+toolchain_create_sdk_version () {
+   local versionfile=$1
+   rm -f $versionfile
+   touch $versionfile
+   echo 'Distro: ${DISTRO}' >> $versionfile
+   echo 'Distro Version: ${DISTRO_VERSION}' >> $versionfile
+   echo 'Metadata Revision: ${METADATA_REVISION}' >> $versionfile
+   echo 'Timestamp: ${DATETIME}' >> $versionfile
+}
+toolchain_create_sdk_version[vardepsexclude] = "DATETIME"
diff --git a/meta/classes/toolchain-scripts.bbclass 
b/meta/classes/toolchain-scripts.bbclass
index 997ff88..0e11f2d 100644
--- a/meta/classes/toolchain-scripts.bbclass
+++ b/meta/classes/toolchain-scripts.bbclass
@@ -1,4 +1,4 @@
-inherit siteinfo kernel-arch
+inherit toolchain-scripts-base siteinfo kernel-arch
 
 # We want to be able to change the value of MULTIMACH_TARGET_SYS, because it
 # doesn't always match our expectations... but we default to the stock value
@@ -136,18 +136,6 @@ toolchain_create_sdk_siteconfig () {
 # The immediate expansion above can result in unwanted path dependencies here
 toolchain_create_sdk_siteconfig[vardepsexclude] = 
"TOOLCHAIN_CONFIGSITE_SYSROOTCACHE"
 
-#This function create a version information file
-toolchain_create_sdk_version () {
-   local versionfile=$1
-   rm -f $versionfile
-   touch $versionfile
-   echo 'Distro: ${DISTRO}' >> $versionfile
-   echo 'Distro Version: ${DISTRO_VERSION}' >> $versionfile
-   echo 'Metadata Revision: ${METADATA_REVISION}' >> $versionfile
-   echo 'Timestamp: ${DATETIME}' >> $versionfile
-}
-toolchain_create_sdk_version[vardepsexclude] = "DATETIME"
-
 python __anonymous () {
 import oe.classextend
 deps = ""
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core