Re: [OE-core] [PATCH RFC 1/1] kernel.bbclass: Add kernel_version_sanity_check function

[Bruce Ashfield]

On 2016-08-24 5:56 PM, Burton, Ross wrote:

Bruce, can you have a look at this?


Wow. 8 days ago, and I admit that I didn't notice it go past.

Yes, I'll have a look at this during the day on Thursday.

Bruce



Ross

On 17 August 2016 at 06:25, California Sullivan
> wrote:

The kernel being built should match what the recipe claims it is
building. This function ensures that happens for anyone using
LINUX_VERSION as they should. As it will likely break outside kernels
not using LINUX_VERSION, only enable the function for linux-yocto for
now.

Signed-off-by: California Sullivan >
---
I'm not absolutely sure this is the correct path to solve the issue.
This patch relies on LINUX_VERSION being set which isn't a guarantee.
There is probably a more general solution that I'm not thinking of.





--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCHv4 0/2] Add initial capability to check CVEs for recipes

[mariano . lopez]

From: Mariano Lopez 

This series add the cve-check-tool recipe, a tool used to identify
potentially vulnerable software through version matching. It will
check if a vulnerability has been addressed by a patch.

Also add the new cve-check class that will add a task for all recipes
to check for CVEs using cve-check-tool. This tool can be used by recipe,
mage (will generate an image report in deploy dir), and with "world"
and "universe"

To run it just inherit the class and enter:

bitbake -c cve_check 

Changes in v2:

- Set the explicit dependency of cve-check-tool-native:do_populate_cve_db
  in the cve-check class
- Squashed patch 2 into patch 1

Changes in v3:

- Support for python3
- Add check for BB_NO_NETWORK

Changes in v4:

- Support for GCC 6 adding -Wno-error=pedantic to CFLAGS
- Support for musl thanks to patch from Khem
- Tweaks when building the the CVE database
- Fix license

The following changes since commit a81b326933d15f08e06780f92d8dc0d4efb3cd23:

  combo-layer: python3: fix UnicodeDecodeError (2016-08-24 13:58:28 +0100)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib mariano/bug7515v5
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=mariano/bug7515v5

Mariano Lopez (2):
  cve-check-tool: Add recipe
  cve-check.bbclass: Add class

 meta/classes/cve-check.bbclass | 267 +
 .../cve-check-tool/cve-check-tool_5.6.4.bb |  59 +
 .../check-for-malloc_trim-before-using-it.patch|  51 
 3 files changed, 377 insertions(+)
 create mode 100644 meta/classes/cve-check.bbclass
 create mode 100644 meta/recipes-devtools/cve-check-tool/cve-check-tool_5.6.4.bb
 create mode 100644 
meta/recipes-devtools/cve-check-tool/files/check-for-malloc_trim-before-using-it.patch

-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCHv4 1/2] cve-check-tool: Add recipe

[mariano . lopez]

From: Mariano Lopez 

cve-check-tool is a program for public CVEs checking.
This tool also seek to determine if a vulnerability has
been addressed by a patch.

The recipe also includes the do_populate_cve_db task
that will populate the database used by the tool.

[YOCTO #7515]

Signed-off-by: Mariano Lopez 
---
 .../cve-check-tool/cve-check-tool_5.6.4.bb | 59 ++
 .../check-for-malloc_trim-before-using-it.patch| 51 +++
 2 files changed, 110 insertions(+)
 create mode 100644 meta/recipes-devtools/cve-check-tool/cve-check-tool_5.6.4.bb
 create mode 100644 
meta/recipes-devtools/cve-check-tool/files/check-for-malloc_trim-before-using-it.patch

diff --git a/meta/recipes-devtools/cve-check-tool/cve-check-tool_5.6.4.bb 
b/meta/recipes-devtools/cve-check-tool/cve-check-tool_5.6.4.bb
new file mode 100644
index 000..56a3b99
--- /dev/null
+++ b/meta/recipes-devtools/cve-check-tool/cve-check-tool_5.6.4.bb
@@ -0,0 +1,59 @@
+SUMMARY = "cve-check-tool"
+DESCRIPTION = "cve-check-tool is a tool for checking known (public) CVEs.\
+The tool will identify potentially vunlnerable software packages within Linux 
distributions through version matching."
+HOMEPAGE = "https://github.com/ikeydoherty/cve-check-tool;
+SECTION = "Development/Tools"
+LICENSE = "GPL-2.0+"
+LIC_FILES_CHKSUM = "file://LICENSE;md5=e8c1458438ead3c34974bc0be3a03ed6"
+
+SRC_URI = 
"https://github.com/ikeydoherty/${BPN}/releases/download/v${PV}/${BP}.tar.xz \
+   file://check-for-malloc_trim-before-using-it.patch \
+  "
+
+SRC_URI[md5sum] = "c5f4247140fc9be3bf41491d31a34155"
+SRC_URI[sha256sum] = 
"b8f283be718af8d31232ac1bfc10a0378fb95849af39168f8acf501e6a5b"
+
+DEPENDS = "libcheck glib-2.0 json-glib curl libxml2 sqlite3 openssl 
ca-certificates"
+
+RDEPENDS_${PN} = "ca-certificates"
+
+inherit pkgconfig autotools
+
+EXTRA_OECONF = "--disable-coverage"
+CFLAGS += "-Wno-error=pedantic"
+
+python do_populate_cve_db () {
+import subprocess
+import time
+
+if d.getVar("BB_NO_NETWORK", True) == "1":
+bb.error("BB_NO_NETWORK is set; Can't update cve-check-tool database, "
+  "CVEs won't be checked")
+return
+
+bb.utils.export_proxies(d)
+# In case we don't inherit cve-check class, use default values defined in 
the class.
+cve_dir = d.getVar("CVE_CHECK_DB_DIR", True) or 
d.expand("${DL_DIR}/CVE_CHECK")
+cve_file = d.getVar("CVE_CHECK_TMP_FILE", True) or 
d.expand("${TMPDIR}/cve_check")
+cve_cmd = "cve-check-update"
+cmd = [cve_cmd, "-d", cve_dir]
+bb.debug(1, "Updating cve-check-tool database located in %s" % cve_dir)
+try:
+output = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
+bb.debug(2, "Command '%s' returned:\n%s" % ("\n".join(cmd), output))
+time_utc = time.gmtime(time.time())
+time_format = "%Y-%m-%d %H:%M:%S"
+with open(cve_file, "w") as f:
+f.write("CVE database was updated on %s UTC\n\n"
+% time.strftime(time_format, time_utc))
+
+except subprocess.CalledProcessError as e:
+bb.warn("Error in executing cve-check-update: %s (output %s)" % (e, 
e.output))
+if bb.data.inherits_class('cve-check', d):
+bb.warn("Failed to update cve-check-tool database, CVEs won't be 
checked")
+}
+
+addtask populate_cve_db after do_populate_sysroot
+do_populate_cve_db[nostamp] = "1"
+
+BBCLASSEXTEND = "native nativesdk"
diff --git 
a/meta/recipes-devtools/cve-check-tool/files/check-for-malloc_trim-before-using-it.patch
 
b/meta/recipes-devtools/cve-check-tool/files/check-for-malloc_trim-before-using-it.patch
new file mode 100644
index 000..0774ad9
--- /dev/null
+++ 
b/meta/recipes-devtools/cve-check-tool/files/check-for-malloc_trim-before-using-it.patch
@@ -0,0 +1,51 @@
+From ce64633b9733e962b8d8482244301f614d8b5845 Mon Sep 17 00:00:00 2001
+From: Khem Raj 
+Date: Mon, 22 Aug 2016 22:54:24 -0700
+Subject: [PATCH] Check for malloc_trim before using it
+
+malloc_trim is gnu specific and not all libc
+implement it, threfore write a configure check
+to poke for it first and use the define to
+guard its use.
+
+Helps in compiling on musl based systems
+
+Signed-off-by: Khem Raj 
+---
+Upstream-Status: Submitted 
[https://github.com/ikeydoherty/cve-check-tool/pull/48]
+ configure.ac | 2 ++
+ src/core.c   | 4 ++--
+ 2 files changed, 4 insertions(+), 2 deletions(-)
+
+diff --git a/configure.ac b/configure.ac
+index d3b66ce..79c3542 100644
+--- a/configure.ac
 b/configure.ac
+@@ -19,6 +19,8 @@ m4_define([json_required_version], [0.16.0])
+ m4_define([openssl_required_version],[1.0.0])
+ # TODO: Set minimum sqlite
+ 
++AC_CHECK_FUNCS_ONCE(malloc_trim)
++
+ PKG_CHECK_MODULES(CVE_CHECK_TOOL,
+  [
+   glib-2.0 >= glib_required_version,
+diff --git a/src/core.c b/src/core.c
+index 

[OE-core] [PATCHv4 2/2] cve-check.bbclass: Add class

[mariano . lopez]

From: Mariano Lopez 

This class adds a new task for all the recipes to use
cve-check-tool in order to look for public CVEs affecting
the packages generated.

It is possible to use this class when building an image,
building a recipe, or using the "world" or "universe" cases.

In order to use this class it must be inherited and it will
add the task automatically to every recipe.

[YOCTO #7515]

Co-authored by Ross Burton & Mariano Lopez

Signed-off-by: Mariano Lopez 
---
 meta/classes/cve-check.bbclass | 267 +
 1 file changed, 267 insertions(+)
 create mode 100644 meta/classes/cve-check.bbclass

diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
new file mode 100644
index 000..8251ca7
--- /dev/null
+++ b/meta/classes/cve-check.bbclass
@@ -0,0 +1,267 @@
+# This class is used to check recipes against public CVEs.
+#
+# In order to use this class just inherit the class in the
+# local.conf file and it will add the cve_check task for
+# every recipe. The task can be used per recipe, per image,
+# or using the special cases "world" and "universe". The
+# cve_check task will print a warning for every unpatched
+# CVE found and generate a file in the recipe WORKDIR/cve
+# directory. If an image is build it will generate a report
+# in DEPLOY_DIR_IMAGE for all the packages used.
+#
+# Example:
+#   bitbake -c cve_check openssl
+#   bitbake core-image-sato
+#   bitbake -k -c cve_check universe
+#
+# DISCLAIMER
+#
+# This class/tool is meant to be used as support and not
+# the only method to check against CVEs. Running this tool
+# doesn't guarantee your packages are free of CVEs.
+
+CVE_CHECK_DB_DIR ?= "${DL_DIR}/CVE_CHECK"
+CVE_CHECK_DB_FILE ?= "${CVE_CHECK_DB_DIR}/nvd.db"
+
+CVE_CHECK_LOCAL_DIR ?= "${WORKDIR}/cve"
+CVE_CHECK_LOCAL_FILE ?= "${CVE_CHECK_LOCAL_DIR}/cve.log"
+CVE_CHECK_TMP_FILE ?= "${TMPDIR}/cve_check"
+
+CVE_CHECK_DIR ??= "${DEPLOY_DIR}/cve"
+CVE_CHECK_MANIFEST ?= 
"${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.cve"
+CVE_CHECK_COPY_FILES ??= "1"
+CVE_CHECK_CREATE_MANIFEST ??= "1"
+
+# Whitelist for packages (PN)
+CVE_CHECK_PN_WHITELIST = "\
+glibc-locale \
+"
+
+# Whitelist for CVE and version of package
+CVE_CHECK_CVE_WHITELIST = "{\
+'CVE-2014-2524': ('6.3',), \
+}"
+
+python do_cve_check () {
+"""
+Check recipe for patched and unpatched CVEs
+"""
+
+if os.path.exists(d.getVar("CVE_CHECK_TMP_FILE", True)):
+patched_cves = get_patches_cves(d)
+patched, unpatched = check_cves(d, patched_cves)
+if patched or unpatched:
+cve_data = get_cve_info(d, patched + unpatched)
+cve_write_data(d, patched, unpatched, cve_data)
+else:
+bb.note("Failed to update CVE database, skipping CVE check")
+}
+
+addtask cve_check after do_unpack before do_build
+do_cve_check[depends] = "cve-check-tool-native:do_populate_cve_db"
+do_cve_check[nostamp] = "1"
+
+python cve_check_cleanup () {
+"""
+Delete the file used to gather all the CVE information.
+"""
+
+bb.utils.remove(e.data.getVar("CVE_CHECK_TMP_FILE", True))
+}
+
+addhandler cve_check_cleanup
+cve_check_cleanup[eventmask] = "bb.cooker.CookerExit"
+
+python cve_check_write_rootfs_manifest () {
+"""
+Create CVE manifest when building an image
+"""
+
+import shutil
+
+if os.path.exists(d.getVar("CVE_CHECK_TMP_FILE", True)):
+bb.note("Writing rootfs CVE manifest")
+deploy_dir = d.getVar("DEPLOY_DIR_IMAGE", True)
+link_name = d.getVar("IMAGE_LINK_NAME", True)
+manifest_name = d.getVar("CVE_CHECK_MANIFEST", True)
+cve_tmp_file = d.getVar("CVE_CHECK_TMP_FILE", True)
+
+shutil.copyfile(cve_tmp_file, manifest_name)
+
+if manifest_name and os.path.exists(manifest_name):
+manifest_link = os.path.join(deploy_dir, "%s.cve" % link_name)
+# If we already have another manifest, update symlinks
+if os.path.exists(os.path.realpath(manifest_link)):
+if d.getVar('RM_OLD_IMAGE', True) == "1":
+os.remove(os.path.realpath(manifest_link))
+os.remove(manifest_link)
+os.symlink(os.path.basename(manifest_name), manifest_link)
+bb.plain("Image CVE report stored in: %s" % manifest_name)
+}
+
+ROOTFS_POSTPROCESS_COMMAND_prepend = "${@'cve_check_write_rootfs_manifest; ' 
if d.getVar('CVE_CHECK_CREATE_MANIFEST', True) == '1' else ''}"
+
+def get_patches_cves(d):
+"""
+Get patches that solve CVEs using the "CVE: " tag.
+"""
+
+import re
+
+pn = d.getVar("PN", True)
+cve_match = re.compile("CVE:( CVE\-\d{4}\-\d+)+")
+patched_cves = set()
+bb.debug(2, "Looking for patches that solves CVEs for %s" % pn)
+for url in src_patches(d):
+patch_file = bb.fetch.decodeurl(url)[2]
+with open(patch_file, "r", encoding="utf-8") as f:
+   

Re: [OE-core] [PATCH 0/2] Toggle bind configure options

[Kang Kai]

On 2016年08月25日 04:10, Burton, Ross wrote:


On 24 August 2016 at 02:46, Kang Kai > wrote:


I didn't see Wangxin's work to upgrade bind. So please comment the
patches, and I will rebase them when the upgrade patch is merged.


Looks good to me!


Thanks.

--Kai



Ross



--
Regards,
Neil | Kai Kang

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 1/1] bash: fix interpreter of ptest script

[Kang Kai]

On 2016年08月25日 04:17, Burton, Ross wrote:


On 24 August 2016 at 10:26, > wrote:


The interpreter of bash ptest script tests/getopts5.sub is
/local/bin/bash which is not suitable for oe. Use /bin/bash instead.


Presumably this test is executed under bash explicitly, so is this 
really worth fixing?


I sent the patch to bash maillist and it is not accepted too.
So please ignore it. Thanks.


--Kai



Ross



--
Regards,
Neil | Kai Kang

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH RFC 1/1] kernel.bbclass: Add kernel_version_sanity_check function

[Burton, Ross]

Bruce, can you have a look at this?

Ross

On 17 August 2016 at 06:25, California Sullivan <
california.l.sulli...@intel.com> wrote:

> The kernel being built should match what the recipe claims it is
> building. This function ensures that happens for anyone using
> LINUX_VERSION as they should. As it will likely break outside kernels
> not using LINUX_VERSION, only enable the function for linux-yocto for
> now.
>
> Signed-off-by: California Sullivan 
> ---
> I'm not absolutely sure this is the correct path to solve the issue.
> This patch relies on LINUX_VERSION being set which isn't a guarantee.
> There is probably a more general solution that I'm not thinking of.
>
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [oe-core][PATCH 1/1] useradd: do not delete users and groups during configure

[Burton, Ross]

On 24 August 2016 at 16:29, Randy MacLeod 
wrote:

> I don't see this in oe-core/master[-next] yet.
>

It's in mut so assuming it doesn't break anything may be in master next
week.

Ross
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] linux-firmware: set a preferred provider for brcmfmac-sdio.bin

[Ross Burton]

This recipe packages six alternatives to brcmfmac-sdio.bin but as they all have
equal priority there is no determinism on what provider will be used if they are
all installed.

Arbitrarily select 4339 to be the highest priority.

Signed-off-by: Ross Burton 
---
 meta/recipes-kernel/linux-firmware/linux-firmware_git.bb | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-kernel/linux-firmware/linux-firmware_git.bb 
b/meta/recipes-kernel/linux-firmware/linux-firmware_git.bb
index 948dfa8..ed0c6ec 100644
--- a/meta/recipes-kernel/linux-firmware/linux-firmware_git.bb
+++ b/meta/recipes-kernel/linux-firmware/linux-firmware_git.bb
@@ -433,6 +433,7 @@ ALTERNATIVE_linux-firmware-bcm4330 = "brcmfmac-sdio.bin"
 ALTERNATIVE_TARGET_linux-firmware-bcm4330[brcmfmac-sdio.bin] = 
"/lib/firmware/brcm/brcmfmac4330-sdio.bin"
 ALTERNATIVE_linux-firmware-bcm4339 = "brcmfmac-sdio.bin"
 ALTERNATIVE_TARGET_linux-firmware-bcm4339[brcmfmac-sdio.bin] = 
"/lib/firmware/brcm/brcmfmac4339-sdio.bin"
+ALTERNATIVE_PRIORITY_linux-firmware-bcm4339[brcmfmac-sdio.bin] = "20"
 
 RDEPENDS_${PN}-bcm4329 += "${PN}-broadcom-license"
 RDEPENDS_${PN}-bcm4330 += "${PN}-broadcom-license"
-- 
2.8.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] Broadcom firmware

[Burton, Ross]

Every time I do an image build this is in the buildhistory:

  /lib/firmware/brcm/brcmfmac-sdio.bin changed symlink target from
/lib/firmware/brcm/brcmfmac4339-sdio.bin to
/lib/firmware/brcm/brcmfmac4329-sdio.bin

Because linux-firmware has alternatives for brcmfmac-sdio.bin but none of
the choices is preferred.  If I made the 4339 file preferred by bumping its
priority, would anyone object?

Ross
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v3] bind: 9.10.3-P3 -> 9.11.0b2

[Burton, Ross]

On 22 August 2016 at 10:52, Wang Xin 
wrote:

> +EXTRA_OECONF = " ${ENABLE_IPV6} --with-python=python
> --with-randomdev=/dev/random --disable-threads \
> @@ -81,7 +76,7 @@ do_install_append() {
> install -d "${D}${sysconfdir}/init.d"
> install -m 644 ${S}/conf/* "${D}${sysconfdir}/bind/"
> install -m 755 "${S}/init.d" "${D}${sysconfdir}/init.d/bind"
> -   sed -i -e '1s,#!.*python,#! /usr/bin/python3,'
> ${D}${sbindir}/dnssec-coverage ${D}${sbindir}/dnssec-checkds
> +   sed -i -e '1s,#!.*python,#! /usr/bin/python3,'
> ${D}${sbindir}/dnssec-coverage ${D}${sbindir}/dnssec-checkds
> ${D}${sbindir}/dnssec-keymgr
>

Would passing --with-python=python3 mean you can remove the sed entirely?

Ross
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] kernel.bbclass: include signing keys when copying files required for module builds

[Burton, Ross]

On 12 August 2016 at 15:10, Waldo Mattias 
wrote:

> The absence of certs/signing_key.* in $kerneldir made signing of
> out-of-tree kernel modules fail (silently). Add copying of these files
> during the shared_workdir task.
>

This doesn't apply to current master, can you rebase and resend?

Ross
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2] oeqa/selftest Adds eSDK test cases to devtool verification.

[Francisco Pedraza]

The covered functions are, install libraries headers and image generation
binary feeds.

Signed-off-by: Francisco Pedraza 
---
 meta/lib/oeqa/selftest/eSDK.py | 103 +
 1 file changed, 103 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/eSDK.py

diff --git a/meta/lib/oeqa/selftest/eSDK.py b/meta/lib/oeqa/selftest/eSDK.py
new file mode 100644
index 000..9d5c680
--- /dev/null
+++ b/meta/lib/oeqa/selftest/eSDK.py
@@ -0,0 +1,103 @@
+import unittest
+import tempfile
+import shutil
+import os
+import glob
+import logging
+import subprocess
+import oeqa.utils.ftools as ftools
+from oeqa.utils.decorators import testcase 
+from oeqa.selftest.base import oeSelfTest
+from oeqa.utils.commands import runCmd, bitbake, get_bb_var
+from oeqa.utils.httpserver import HTTPService
+
+class oeSDKExtSelfTest(oeSelfTest):
+"""
+# Bugzilla Test Plan: 6033
+# This code is planned to be part of the automation for eSDK containig
+# Install libraries and headers, image generation binary feeds.
+"""
+
+@staticmethod
+def get_esdk_environment(env_eSDK, tmpdir_eSDKQA):
+# XXX: at this time use the first env need to investigate
+# what environment load oe-selftest, i586, x86_64
+pattern = os.path.join(tmpdir_eSDKQA, 'environment-setup-*')
+return glob.glob(pattern)[0]
+
+@staticmethod
+def run_esdk_cmd(env_eSDK, tmpdir_eSDKQA, cmd, postconfig=None, **options):
+if postconfig:
+esdk_conf_file = os.path.join(tmpdir_eSDKQA, 'conf', 'local.conf')
+with open(esdk_conf_file, 'a+') as f:
+f.write(postconfig)
+if not options:
+options = {}
+if not 'shell' in options:
+options['shell'] = True
+
+runCmd("cd %s; . %s; %s" % (tmpdir_eSDKQA, env_eSDK, cmd), **options)
+
+@staticmethod
+def generate_eSDK(image):
+pn_task = '%s -c populate_sdk_ext' % image
+bitbake(pn_task)
+
+@staticmethod
+def get_eSDK_toolchain(image):
+pn_task = '%s -c populate_sdk_ext' % image
+
+sdk_deploy = get_bb_var('SDK_DEPLOY', pn_task)
+toolchain_name = get_bb_var('TOOLCHAINEXT_OUTPUTNAME', pn_task)
+return os.path.join(sdk_deploy, toolchain_name + '.sh')
+
+
+@classmethod
+def setUpClass(cls):
+# Start to serve sstate dir
+sstate_dir = os.path.join(os.environ['BUILDDIR'], 'sstate-cache')
+cls.http_service = HTTPService(sstate_dir)
+cls.http_service.start()
+
+http_url = "127.0.0.1:%d" % cls.http_service.port
+ 
+image = 'core-image-minimal'
+
+cls.tmpdir_eSDKQA = tempfile.mkdtemp(prefix='eSDKQA')
+oeSDKExtSelfTest.generate_eSDK(image)
+
+# Install eSDK
+ext_sdk_path = oeSDKExtSelfTest.get_eSDK_toolchain(image)
+runCmd("%s -y -d \"%s\"" % (ext_sdk_path, cls.tmpdir_eSDKQA))
+
+cls.env_eSDK = oeSDKExtSelfTest.get_esdk_environment('', 
cls.tmpdir_eSDKQA)
+
+# Configure eSDK to use sstate mirror from poky
+sstate_config="""
+SDK_LOCAL_CONF_WHITELIST = "SSTATE_MIRRORS"
+SSTATE_MIRRORS =  "file://.* http://%s/PATH;
+""" % http_url
+with open(os.path.join(cls.tmpdir_eSDKQA, 'conf', 'local.conf'), 'a+') 
as f:
+f.write(sstate_config)
+
+  
+@classmethod
+def tearDownClass(cls):
+shutil.rmtree(cls.tmpdir_eSDKQA)
+cls.http_service.stop()
+
+@testcase (1471)
+def test_install_libraries_headers(self):
+pn_sstate = 'bc'
+bitbake(pn_sstate)
+cmd = "devtool sdk-install %s " % pn_sstate
+oeSDKExtSelfTest.run_esdk_cmd(self.env_eSDK, self.tmpdir_eSDKQA, cmd)
+
+@testcase(1472)
+def test_image_generation_binary_feeds(self):
+image = 'core-image-minimal'
+cmd = "devtool build-image %s" % image
+oeSDKExtSelfTest.run_esdk_cmd(self.env_eSDK, self.tmpdir_eSDKQA, cmd)
+
+if __name__ == '__main__':
+unittest.main()
-- 
2.5.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2] /oeqa/sdkext Adds verification for devtool on eSDK.

[Francisco Pedraza]

The covered funcions are, build make, build esdk package, build cmake
extend autotools recipe creation, kernel module,
node.js installation and recipe creation.

Signed-off-by: Francisco Pedraza 
---
 meta/lib/oeqa/sdkext/devtool.py| 90 --
 .../oeqa/sdkext/files/myapp_cmake/CMakeLists.txt   | 11 +++
 meta/lib/oeqa/sdkext/files/myapp_cmake/myapp.c |  9 +++
 3 files changed, 103 insertions(+), 7 deletions(-)
 create mode 100644 meta/lib/oeqa/sdkext/files/myapp_cmake/CMakeLists.txt
 create mode 100644 meta/lib/oeqa/sdkext/files/myapp_cmake/myapp.c

diff --git a/meta/lib/oeqa/sdkext/devtool.py b/meta/lib/oeqa/sdkext/devtool.py
index c5bb310..ba12799 100644
--- a/meta/lib/oeqa/sdkext/devtool.py
+++ b/meta/lib/oeqa/sdkext/devtool.py
@@ -1,32 +1,108 @@
 import shutil
-
+import subprocess
+import urllib.request
 from oeqa.oetest import oeSDKExtTest
 from oeqa.utils.decorators import *
 
 class DevtoolTest(oeSDKExtTest):
-
 @classmethod
 def setUpClass(self):
 self.myapp_src = os.path.join(self.tc.sdkextfilesdir, "myapp")
 self.myapp_dst = os.path.join(self.tc.sdktestdir, "myapp")
 shutil.copytree(self.myapp_src, self.myapp_dst)
 
+self.myapp_cmake_src = os.path.join(self.tc.sdkextfilesdir, 
"myapp_cmake")
+self.myapp_cmake_dst = os.path.join(self.tc.sdktestdir, "myapp_cmake")
+shutil.copytree(self.myapp_cmake_src, self.myapp_cmake_dst)
+
+def _test_devtool_build(self, directory):
+self._run('devtool add myapp %s' % directory)
+try:
+self._run('devtool build myapp')
+except Exception as e:
+print(e.output)
+self._run('devtool reset myapp')
+raise e
+self._run('devtool reset myapp')
+
+def _test_devtool_build_package(self, directory):
+self._run('devtool add myapp %s' % directory)
+try:
+self._run('devtool package myapp')
+except Exception as e:
+print(e.output)
+self._run('devtool reset myapp')
+raise e
+self._run('devtool reset myapp')
+
 def test_devtool_location(self):
 output = self._run('which devtool')
 self.assertEqual(output.startswith(self.tc.sdktestdir), True, \
 msg="Seems that devtool isn't the eSDK one: %s" % output)
-
+
 @skipUnlessPassed('test_devtool_location')
 def test_devtool_add_reset(self):
 self._run('devtool add myapp %s' % self.myapp_dst)
 self._run('devtool reset myapp')
+
+@testcase(1473)
+@skipUnlessPassed('test_devtool_location')
+def test_devtool_build_make(self):
+self._test_devtool_build(self.myapp_dst)
+
+@testcase(1474)
+@skipUnlessPassed('test_devtool_location')
+def test_devtool_build_esdk_package(self):
+self._test_devtool_build_package(self.myapp_dst)
 
+@testcase(1479)
 @skipUnlessPassed('test_devtool_location')
-def test_devtool_build(self):
-self._run('devtool add myapp %s' % self.myapp_dst)
-self._run('devtool build myapp')
-self._run('devtool reset myapp')
+def test_devtool_build_cmake(self):
+self._test_devtool_build(self.myapp_cmake_dst)
+
+@testcase(1482)
+@skipUnlessPassed('test_devtool_location')
+def test_extend_autotools_recipe_creation(self):
+req = 'https://github.com/rdfa/librdfa'
+recipe = "bbexample"
+self._run('devtool add %s %s' % (recipe, req) )
+try:
+self._run('devtool build %s' % recipe)
+except Exception as e:
+print(e.output)
+self._run('devtool reset %s' % recipe)
+raise e
+self._run('devtool reset %s' % recipe)
+
+@testcase(1484)
+@skipUnlessPassed('test_devtool_location')
+def test_devtool_kernelmodule(self):
+docfile = 'https://github.com/umlaeute/v4l2loopback.git'
+recipe = 'v4l2loopback-driver'
+self._run('devtool add %s %s' % (recipe, docfile) )
+try:
+self._run('devtool build %s' % recipe)
+except Exception as e:
+print(e.output)
+self._run('devtool reset %s' % recipe)
+raise e
+self._run('devtool reset %s' % recipe)
+
+@testcase(1478)
+@skipUnlessPassed('test_devtool_location')
+def test_recipes_for_nodejs(self):
+package_nodejs = "npm://registry.npmjs.org;name=forever;version=0.15.1"
+self._run('devtool add %s ' % package_nodejs)
+try:
+self._run('devtool build %s ' % package_nodejs)
+except Exception as e:
+print(e.output)
+self._run('devtool reset %s' % package_nodejs)
+raise e
+self._run('devtool reset %s '% package_nodejs)
+
 
 @classmethod
 def tearDownClass(self):
 shutil.rmtree(self.myapp_dst)
+shutil.rmtree(self.myapp_cmake_dst)
diff --git 

Re: [OE-core] [PATCH 1/1] bash: fix interpreter of ptest script

[Burton, Ross]

On 24 August 2016 at 10:26,  wrote:

> The interpreter of bash ptest script tests/getopts5.sub is
> /local/bin/bash which is not suitable for oe. Use /bin/bash instead.
>

Presumably this test is executed under bash explicitly, so is this really
worth fixing?

Ross
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 0/2] Toggle bind configure options

[Burton, Ross]

On 24 August 2016 at 02:46, Kang Kai  wrote:

> I didn't see Wangxin's work to upgrade bind. So please comment the
> patches, and I will rebase them when the upgrade patch is merged.
>

Looks good to me!

Ross
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 1/8] [autotools] export CCLD_FOR_BUILD

[Burton, Ross]

On 24 August 2016 at 12:28, Jérémy Rosen  wrote:

> ok, so CC_FOR_BUILD seems to be a common convention and CCLD_FOR_BUILD
> seems only used by our patch for pcre. Fixing it to use CC_FOR_BUILD for
> linking is trivial
>
> but the patch is marked as submitted upstream, so Fahad... what is the
> status ? how can we do that ?
>
> I can also set CCLD_FOR_BUILD in ther .bb file instead of
> autotools.bbclass, that's trivial... the question is whether we should fix
> the patch or not...
>

CC_FOR_BUILD and CPP_FOR_BUILD are de facto standards by virtue of being
part of autoconf-archive:
https://www.gnu.org/software/autoconf-archive/ax_prog_cc_for_build.html

Looking at the patch wouldn't it be possible to avoid the entire .c -> .o
-> binary process and just go from .c -> binary via CC_FOR_BUILD?

Ross
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] weston startup problem

[Tom Hochstein]

Hi All,

The weston systemd startup implementation is not working correctly and I need 
some help fixing it.

With systemd and pam on the image, the expected behavior is that 
XDG_RUNTIME_DIR would already be set when weston is launched from 
weston.service. Turns out XDG_RUNTIME_DIR is not set, and weston.service sets 
it itself. This is already a problem, and the problem would be worse except 
that weston happens to use a different folder. If weston.service is modified to 
use the same folder name, the folder ends up getting replaced later by pam.

>From looking at the weston-launch code and its pam handling, I thought to try 
>the weston-launch option '--user=root' to force the creation of the pam 
>session. This does seem to help, and I was able to get it mostly working by 
>modifying the openvt args in weston-start, removing -e and adding -w:

openvt -v -s -w -- weston-launch --user=root -- --modules=xwayland.so 
--log=/var/log/weston.log

The problem I have now is that keyboard input goes to both weston and the 
desktop. I tried with and without the -s option, and there is no difference.

I'm stuck at this point and would appreciate some help.

Tom
<>-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] linux-firmware: package carl9170 separately

[Khem Raj]

> On Aug 24, 2016, at 5:22 AM, Ioan-Adrian Ratiu  wrote:
> 
> carl9170 is an atheros-based firmware, the succesor of ar9170 which is
> deprecated and its driver (ar9170) was removed from the kernel tree.
> carl9170 comes both as a binary blob and also with free sources; the
> sources are deleted in linux-firmware.bb to avoid depeding on bash/etc.
> 
> Package the carl9170 binary firmware separately because it is needed
> by various usb dongles and installing whole linux-firmware is overkill.

while you are at it. Can you also scrutinize the license needs for this 
firmware ?
should it require packaging a license file explicitly into rootfs or some such

> 
> Also we should keep ar9170 as is because OOT drivers might still use it.
> 
> Signed-off-by: Ioan-Adrian Ratiu 
> ---
> meta/recipes-kernel/linux-firmware/linux-firmware_git.bb | 6 +-
> 1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/recipes-kernel/linux-firmware/linux-firmware_git.bb 
> b/meta/recipes-kernel/linux-firmware/linux-firmware_git.bb
> index 948dfa8..1657445 100644
> --- a/meta/recipes-kernel/linux-firmware/linux-firmware_git.bb
> +++ b/meta/recipes-kernel/linux-firmware/linux-firmware_git.bb
> @@ -226,7 +226,7 @@ PACKAGES =+ "${PN}-ralink-license ${PN}-ralink \
>  ${PN}-vt6656-license ${PN}-vt6656 \
>  ${PN}-rtl-license ${PN}-rtl8192cu ${PN}-rtl8192ce 
> ${PN}-rtl8192su \
>  ${PN}-broadcom-license ${PN}-bcm4329 ${PN}-bcm4330 ${PN}-bcm4334 
> ${PN}-bcm43340 ${PN}-bcm4339 ${PN}-bcm4354 \
> - ${PN}-atheros-license ${PN}-ar9170 ${PN}-ath6k ${PN}-ath9k \
> + ${PN}-atheros-license ${PN}-ar9170 ${PN}-carl9170 ${PN}-ath6k 
> ${PN}-ath9k \
>  ${PN}-ar3k-license  ${PN}-ar3k  ${PN}-ath10k-license  
> ${PN}-ath10k  \
>  \
>  ${PN}-iwlwifi-license ${PN}-iwlwifi \
> @@ -255,6 +255,9 @@ FILES_${PN}-atheros-license = 
> "/lib/firmware/LICENCE.atheros_firmware"
> FILES_${PN}-ar9170 = " \
>   /lib/firmware/ar9170*.fw \
> "
> +FILES_${PN}-carl9170 = " \
> +  /lib/firmware/carl9170*.fw \
> +"
> FILES_${PN}-ath6k = " \
>   /lib/firmware/ath6k \
> "
> @@ -266,6 +269,7 @@ FILES_${PN}-ath9k = " \
> "
> 
> RDEPENDS_${PN}-ar9170 += "${PN}-atheros-license"
> +RDEPENDS_${PN}-carl9170 += "${PN}-atheros-license"
> RDEPENDS_${PN}-ath6k += "${PN}-atheros-license"
> RDEPENDS_${PN}-ath9k += "${PN}-atheros-license"
> 
> --
> 2.9.3
> 
> --
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core



signature.asc
Description: Message signed with OpenPGP using GPGMail
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [oe-core][PATCH 1/1] useradd: do not delete users and groups during configure

[Randy MacLeod]

On 2016-08-16 02:16 PM, Joe Slater wrote:

If two recipes both create the same users and groups, the
second recipe can delete items created by the first causing
things like "chown" to fail for the first recipe.

Signed-off-by: Joe Slater 
---
 meta/classes/useradd.bbclass |2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/useradd.bbclass b/meta/classes/useradd.bbclass
index 8d51fb5..6df9822 100644
--- a/meta/classes/useradd.bbclass
+++ b/meta/classes/useradd.bbclass
@@ -129,7 +129,7 @@ useradd_sysroot_sstate () {

 userdel_sysroot_sstate () {
 if test "x${STAGING_DIR_TARGET}" != "x"; then
-if [ "${BB_CURRENTTASK}" = "configure" -o "${BB_CURRENTTASK}" = "clean" ]; 
then
+if [ "${BB_CURRENTTASK}" = "clean" ]; then
 export PSEUDO="${FAKEROOTENV} 
PSEUDO_LOCALSTATEDIR=${STAGING_DIR_TARGET}${localstatedir}/pseudo 
${STAGING_DIR_NATIVE}${bindir_native}/pseudo"
 OPT="--root ${STAGING_DIR_TARGET}"





I don't see this in oe-core/master[-next] yet.

Is there a better solution or should the recipes
that create the user (ftp in this case), be modified some how?

--
# Randy MacLeod. SMTS, Linux, Wind River
Direct: 613.963.1350 | 350 Terry Fox Drive, Suite 200, Ottawa, ON, 
Canada, K2K 2W5

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 3/8] [rpm] force a configure before make distclean

[Mark Hatle]

On 8/24/16 9:17 AM, Jérémy Rosen wrote:
> 
> 
> On 24/08/2016 16:13, Mark Hatle wrote:
>>
>> Make often has a rule looking at the time stamp of specific files and 
>> re-running
>> specific autotools if they are 'out of date'.  That COULD be the cause of the
>> errant call to gcc?
> that would make sense...
>>
>> I have not observed the above occurring -- but the key thing is that syck 
>> needs
>> to be cleaned of the generated files.  We don't want to run any host commands
>> other then clean it.
>>
>> If you have to replace it with something else, you will need to identify all 
>> of
>> the files that need to be cleaned and manually remove them avoiding the
>> distclean call.
>>
>> Easiest way to do that extract the source for syck, back it up, run the 
>> make
>> distclean.. compare the difference and use that list as a replacement 
>> 'cleanup'
>> command.
> 
> That wouldn't be very robust to future change in syck, i'm afraid... but 
> i'll have a look.

There is nothing robust about this change for future work.  Future versions will
have to do the work in a different way -- and hopefully won't have pre-built
crap slipped into the download that we have to cleanup.

So I have no concern about future proofing this, as I know the next integration
-will- be different in this case.

--Mark

>>
>> --Mark
>>
>>
> 

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 3/8] [rpm] force a configure before make distclean

[Jérémy Rosen]

On 24/08/2016 16:13, Mark Hatle wrote:


Make often has a rule looking at the time stamp of specific files and re-running
specific autotools if they are 'out of date'.  That COULD be the cause of the
errant call to gcc?

that would make sense...


I have not observed the above occurring -- but the key thing is that syck needs
to be cleaned of the generated files.  We don't want to run any host commands
other then clean it.

If you have to replace it with something else, you will need to identify all of
the files that need to be cleaned and manually remove them avoiding the
distclean call.

Easiest way to do that extract the source for syck, back it up, run the make
distclean.. compare the difference and use that list as a replacement 'cleanup'
command.


That wouldn't be very robust to future change in syck, i'm afraid... but 
i'll have a look.


--Mark




--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 3/8] [rpm] force a configure before make distclean

[Mark Hatle]

On 8/24/16 2:44 AM, Jérémy Rosen wrote:
> 
> 
> On 23/08/2016 19:16, Mark Hatle wrote:
>> On 8/23/16 11:54 AM, Burton, Ross wrote:
>>> On 23 August 2016 at 08:25, Jérémy Rosen >> > wrote:
>>>
>>>  -   ( cd ${S}/syck ; set +e ; rm -- -l* ; make distclean ) || :
>>>  +   ( cd ${S}/syck ; set +e ; rm -- -l* ; ./configure ; make 
>>> distclean )
>>>  || :
>>>
>>>
>>> Running configure without all the options it needs to actually work just so 
>>> that
>>> distclean works in a subbuild that shouldn't be breaking on rebuilds 
>>> because we
>>> have out of tree builds scares me a little.
>>>
>>> Mark, you added this line, can you remember what it was for?
>> The syck version can be preconfigured, and if it is -- it's wrong.
>>
>> The 'set +e' is specifically in there to ALLOW it to fail.
>>
>> If it is configured, we much remove the junk and clean it.. if it's not been
>> configured, we ignore the make distclean failure.
>>
>> So I'm not sure why it is necessary to make the change.  'Failure is 
>> permitted',
>> and make distclean shouldn't be using any 'BUILD_CC' values from prior 
>> builds...
>> since it's only wiping everything that MAY have been previously built and 
>> leaked
>> into the release archive.
> I havn't dived deeply on the whole chain of event, I have to admit that 
> autotools scares me...
> 
> This particular line does work as expected. the set +e does indeed allow 
> the make distclean to fail.
> 
> The problem is that if that distclean is not failing because nothing 
> needs to be done it is failing because it can't find gcc. The junk isn't 
> clean and the build of rpm will fail later complaining that the syck 
> subdir is
> already configured.
> 
> This is probably why you run the make distclean in the first place.
> 
> The need for gcc is weird... the error message I get is that no working 
> version of gcc could be found, but the trace itself looks more like a 
> call to configure (as if make distclean did some configuration check)
> 
> Again, I have not dug deeply enough in here to completely understand 
> what's going on. I'd gladly take any clues you might have for a better fix.

Make often has a rule looking at the time stamp of specific files and re-running
specific autotools if they are 'out of date'.  That COULD be the cause of the
errant call to gcc?

I have not observed the above occurring -- but the key thing is that syck needs
to be cleaned of the generated files.  We don't want to run any host commands
other then clean it.

If you have to replace it with something else, you will need to identify all of
the files that need to be cleaned and manually remove them avoiding the
distclean call.

Easiest way to do that extract the source for syck, back it up, run the make
distclean.. compare the difference and use that list as a replacement 'cleanup'
command.

--Mark

>>
>> --Mark
>>
>>> Ross
>>
> 

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/6] linux-yocto/4.1: bump to v4.1.29

[Bruce Ashfield]

On Wed, Aug 24, 2016 at 7:25 AM, Richard Purdie <
richard.pur...@linuxfoundation.org> wrote:

> On Fri, 2016-08-19 at 10:57 -0400, Bruce Ashfield wrote:
> > On 2016-08-18 11:20 AM, Richard Purdie wrote:
> > > On Thu, 2016-08-18 at 11:16 -0400, Bruce Ashfield wrote:
> > > > On 2016-08-18 11:15 AM, Richard Purdie wrote:
> > > > > On Mon, 2016-08-15 at 14:26 -0400, Bruce Ashfield wrote:
> > > > > > Integrating the korg 4.1.29 -stable release
> > > > > >
> > > > > > Signed-off-by: Bruce Ashfield 
> > > > > > ---
> > > > > >  meta/recipes-kernel/linux/linux-yocto-rt_4.1.bb   |  6 +++--
> > > > > > -
> > > > > >  meta/recipes-kernel/linux/linux-yocto-tiny_4.1.bb |  6 +++--
> > > > > > -
> > > > > >  meta/recipes-kernel/linux/linux-yocto_4.1.bb  | 20
> > > > > > ++---
> > > > > > ---
> > > > > >  3 files changed, 16 insertions(+), 16 deletions(-)
> > > > >
> > > > > FWIW since the update before this one, we've seen a lot of
> > > > > instability
> > > > > in ppc:
> > > > >
> > > > > https://autobuilder.yoctoproject.org/main/builders/nightly-ppc-
> > > > > lsb/
> > > > > builds/881/steps/Running%20Sanity%20Tests/logs/stdio
> > > > >
> > > > > basically the network seems very flaky. Doesn't happen every
> > > > > time
> > > > > but
> > > > > we are seeing a rising number of these kinds of 'random'
> > > > > failures.
> > > > > Are
> > > > > there any ppc changes or fixes which look like they're
> > > > > responsible,
> > > > > I'm
> > > > > pretty sure its something 4.1 kernel related...
> > > > >
> > > > > The update also triggered a new 'error' for amd_nb in the error
> > > > > log
> > > > > parsing which I just whitelisted, probably from some change in
> > > > > the
> > > > > configuration options enabled.
> > > >
> > > > I have a 4.1.30 update queued locally, and I'll double check to
> > > > see
> > > > what may have changed in the configuration that could explain any
> > > > new warnings or instability.
> > >
> > > Just to be clear, the amb_nd change was on qemux86* on 4.1
> > >
> > > https://autobuilder.yoctoproject.org/main/builders/nightly-x86-64-l
> > > sb/builds/911/steps/Running%20Sanity%20Tests/logs/stdio
> > >
> > > fixed with:
> > >
> > > http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=26b0657b2f7875
> > > 1e60df6e695d5bc0dd703a5bc2
> >
> > I had a look at this, and I don't have a root cause for why this is
> > just popping up now. I'm trying to track down an older build, so I
> > can
> > compare the .configs.
> >
> > The kernel code, and Kconfig files themselves haven't changed in some
> > time, and the symbol in question AMD_NB is default 'y' if the
> > dependencies are met. We include cfg/amd.scc in all of the qemux86*
> > builds, so it gets enabled (as it should have before).
> >
> > I could force that option off, but the whitelist is an appropriate
> > fix,
> > since this is an interaction between qemu and the kernel, whereas the
> > files that include amd.scc are targeted at a range of platforms.
> >
> > >
> > > The ppc issue is a separate one.
> >
> > I've been blasting away on my 4.1.30 qemuppc build, and I'm not
> > seeing
> > anything yet, but I will keep trying.
>
> Its still going on even with the recent updates:
>
> https://autobuilder.yoctoproject.org/main/builders/nightly-ppc-lsb/
> builds/889/steps/Running%20Sanity%20Tests/logs/stdio
>
> Seems the network works intermittently, dropping connections...
>

Hmm.

I've had a qemuppc up and running for 3 days now, doing transfers of files
from my server, and yet my network hasn't dropped.
I'll continue trying to figure this out.

Bruce


>
> Cheers,
>
> Richard
> --
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end"
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] linux-firmware: package carl9170 separately

[Ioan-Adrian Ratiu]

carl9170 is an atheros-based firmware, the succesor of ar9170 which is
deprecated and its driver (ar9170) was removed from the kernel tree.
carl9170 comes both as a binary blob and also with free sources; the
sources are deleted in linux-firmware.bb to avoid depeding on bash/etc.

Package the carl9170 binary firmware separately because it is needed
by various usb dongles and installing whole linux-firmware is overkill.

Also we should keep ar9170 as is because OOT drivers might still use it.

Signed-off-by: Ioan-Adrian Ratiu 
---
 meta/recipes-kernel/linux-firmware/linux-firmware_git.bb | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-kernel/linux-firmware/linux-firmware_git.bb 
b/meta/recipes-kernel/linux-firmware/linux-firmware_git.bb
index 948dfa8..1657445 100644
--- a/meta/recipes-kernel/linux-firmware/linux-firmware_git.bb
+++ b/meta/recipes-kernel/linux-firmware/linux-firmware_git.bb
@@ -226,7 +226,7 @@ PACKAGES =+ "${PN}-ralink-license ${PN}-ralink \
  ${PN}-vt6656-license ${PN}-vt6656 \
  ${PN}-rtl-license ${PN}-rtl8192cu ${PN}-rtl8192ce ${PN}-rtl8192su 
\
  ${PN}-broadcom-license ${PN}-bcm4329 ${PN}-bcm4330 ${PN}-bcm4334 
${PN}-bcm43340 ${PN}-bcm4339 ${PN}-bcm4354 \
- ${PN}-atheros-license ${PN}-ar9170 ${PN}-ath6k ${PN}-ath9k \
+ ${PN}-atheros-license ${PN}-ar9170 ${PN}-carl9170 ${PN}-ath6k 
${PN}-ath9k \
  ${PN}-ar3k-license  ${PN}-ar3k  ${PN}-ath10k-license  
${PN}-ath10k  \
  \
  ${PN}-iwlwifi-license ${PN}-iwlwifi \
@@ -255,6 +255,9 @@ FILES_${PN}-atheros-license = 
"/lib/firmware/LICENCE.atheros_firmware"
 FILES_${PN}-ar9170 = " \
   /lib/firmware/ar9170*.fw \
 "
+FILES_${PN}-carl9170 = " \
+  /lib/firmware/carl9170*.fw \
+"
 FILES_${PN}-ath6k = " \
   /lib/firmware/ath6k \
 "
@@ -266,6 +269,7 @@ FILES_${PN}-ath9k = " \
 "
 
 RDEPENDS_${PN}-ar9170 += "${PN}-atheros-license"
+RDEPENDS_${PN}-carl9170 += "${PN}-atheros-license"
 RDEPENDS_${PN}-ath6k += "${PN}-atheros-license"
 RDEPENDS_${PN}-ath9k += "${PN}-atheros-license"
 
-- 
2.9.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] gitpkgv: Fix $GITPKV for a single named git source

[Clemens Lang]

Hi,

Clemens Lang  wrote:

> Shameless bump. I haven’t heard any feedback on this other than this “looks
> good” comment, but it isn’t merged. Is there a problem with the patch?

My apologies, it seems this patch should have gone to openembedded-devel  
rather than this list. I’ll repost it over there.

-- 
Clemens Lang • Development Specialist
BMW Car IT GmbH • Lise-Meitner-Str. 14 • 89081 Ulm • http://bmw-carit.com
-
BMW Car IT GmbH
Geschäftsführer: Michael Würtenberger und Alexis Trolin
Sitz und Registergericht: München HRB 134810
-
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] inherit allarch and use RDEPENDS

[Mike Looijmans]

On 24-08-16 09:18, Richard Purdie wrote:> On Wed, 2016-08-24 at 08:35 +0200, Mike Looijmans wrote:>> First time I've heard about this limitation, I don't understand>> what's this about either. I've been telling people to put their arch>> or machine specific code into a seperate package and RDEPEND on it>> from an allarch package that holds the Python code that calls into>> that. We haven't seen any problems with that. So what is the bad thing that would happen if we don't set the>> SIGGEN_EXCLUDE... variables in the layer.conf file?>> If you have an allarch recipe X and you depend on a tune or machine> specific package from it, what you'll see is that when you switch> MACHINE (with a different tune), the recipe X will rebuild each time.So it's just inefficient, but does not break things?> This is obviously suboptimal however its not the end of the world. It> is problematic if you're using PR server.Which brings me to the next question: What would go wrong then?(PR server is somewhat mandatory today)> The reason it rebuilds is that its checksum has a dependency on> something which is tune or machine specific, you change MACHINE, the> checksum changes and the allarch recipe's checksum changes.>> If we remove the dependency from the task checksum, the problem isn't> there and it doesn't rebuild, however there are some circumstances> where this could be an issue, e.g. if a dependency name was renamed by> debian.bbclass for example.>> The mechanism for removing such a dependency is the one I've mentioned> in layer.conf.>> You can see if you have a problem with your layers by running:>> oe-selftest -r sstatetests.SStateTests.test_sstate_allarch_samesigs>> Does that make the issue any clearer?A bit. But I don't understand what I have to fill in where. Example:mike@mikebuntu:~/.../recipes-spike/spike$ cat spikeall.bbLICENSE="CLOSED"inherit allarchRDEPENDS_${PN} = "spikemach"do_install() {   install -d ${D}${bindir}   echo 'cat ${sysconfdir}/hello' > ${S}/${PN}.sh   install -m 755 ${S}/${PN}.sh ${D}${bindir}}mike@mikebuntu:~/.../recipes-spike/spike$ cat spikemach.bbLICENSE="CLOSED"PACKAGE_ARCH="${MACHINE_ARCH}"PV="2"do_install() {   install -d ${D}${sysconfdir}   echo "${MACHINE}" > ${D}${sysconfdir}/hello}I run "bitbake spikeall" and both packages get built.I run "bitbake spikeall" and nothing builds (which is good)Change the PV in spikemach.bb, and run "bitbake spikeall". Both spikemach and spikeall are built (which is the problem we're talking about then, spikeall needn't build again).I add the following line to the layer.conf in that layer:SIGGEN_EXCLUDERECIPES_ABISAFE += "spikeall"No change, spikeall still gets rebuilt along with spikemach.The other way around maybe?SIGGEN_EXCLUDERECIPES_ABISAFE += "spikemach"Now it appears to work. Changing spikemach no longer causes spikeall to rebuild.I have no clue as to why this would be needed, but I'll take your word for it.Why can't we just do SIGGEN_EXCLUDERECIPES_ABISAFE="1" in the spikemach.bb recipe? That's much easier to maintain and understand...
 
Kind regards,
 
Mike Looijmans
System Expert
 





  
  

  TOPIC
  Products

   

   
  

  Materiaalweg
  4

   

   
  

  5681
  RJ Best

  T:

  +31 (0) 499 33
  69 69
  

  Postbus
  440

  E:

  mike.looijm...@topicproducts.com
  

  5680 AK
  Best

  W:

  www.topicproducts.com
  
The Netherlands



 Please consider the environment before printing this
e-mailTopic zoekt gedreven (embedded) software specialisten!



-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 1/8] [autotools] export CCLD_FOR_BUILD

[Jérémy Rosen]

(Adding original patch author to mail)


On 24/08/2016 13:05, Jérémy Rosen wrote:

[...]


+export CCLD_FOR_BUILD = "${BUILD_CCLD}”
if its used by just one recipe then its not worth abstracting it out. 
Perhaps

fixing
https://github.com/openembedded/openembedded-core/blob/master/meta/recipes-support/libpcre/libpcre/pcre-cross.patch 


is another option ?
I'll have a look, I have been looking around and i'm not even sure if 
CC_FOR_BUILD is an autoconf thing... CCLD seems to be unique to that 
recipe. i'll look at the patch


ok, so CC_FOR_BUILD seems to be a common convention and CCLD_FOR_BUILD 
seems only used by our patch for pcre. Fixing it to use CC_FOR_BUILD for 
linking is trivial


but the patch is marked as submitted upstream, so Fahad... what is the 
status ? how can we do that ?


I can also set CCLD_FOR_BUILD in ther .bb file instead of 
autotools.bbclass, that's trivial... the question is whether we should 
fix the patch or not...



+
def append_libtool_sysroot(d):
 # Only supply libtool sysroot option for non-native packages
 if not bb.data.inherits_class('native', d):
--
git-series 0.8.9
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core




--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/6] linux-yocto/4.1: bump to v4.1.29

[Richard Purdie]

On Fri, 2016-08-19 at 10:57 -0400, Bruce Ashfield wrote:
> On 2016-08-18 11:20 AM, Richard Purdie wrote:
> > On Thu, 2016-08-18 at 11:16 -0400, Bruce Ashfield wrote:
> > > On 2016-08-18 11:15 AM, Richard Purdie wrote:
> > > > On Mon, 2016-08-15 at 14:26 -0400, Bruce Ashfield wrote:
> > > > > Integrating the korg 4.1.29 -stable release
> > > > > 
> > > > > Signed-off-by: Bruce Ashfield 
> > > > > ---
> > > > >  meta/recipes-kernel/linux/linux-yocto-rt_4.1.bb   |  6 +++--
> > > > > -
> > > > >  meta/recipes-kernel/linux/linux-yocto-tiny_4.1.bb |  6 +++--
> > > > > -
> > > > >  meta/recipes-kernel/linux/linux-yocto_4.1.bb  | 20
> > > > > ++---
> > > > > ---
> > > > >  3 files changed, 16 insertions(+), 16 deletions(-)
> > > > 
> > > > FWIW since the update before this one, we've seen a lot of
> > > > instability
> > > > in ppc:
> > > > 
> > > > https://autobuilder.yoctoproject.org/main/builders/nightly-ppc-
> > > > lsb/
> > > > builds/881/steps/Running%20Sanity%20Tests/logs/stdio
> > > > 
> > > > basically the network seems very flaky. Doesn't happen every
> > > > time
> > > > but
> > > > we are seeing a rising number of these kinds of 'random'
> > > > failures.
> > > > Are
> > > > there any ppc changes or fixes which look like they're
> > > > responsible,
> > > > I'm
> > > > pretty sure its something 4.1 kernel related...
> > > > 
> > > > The update also triggered a new 'error' for amd_nb in the error
> > > > log
> > > > parsing which I just whitelisted, probably from some change in
> > > > the
> > > > configuration options enabled.
> > > 
> > > I have a 4.1.30 update queued locally, and I'll double check to
> > > see
> > > what may have changed in the configuration that could explain any
> > > new warnings or instability.
> > 
> > Just to be clear, the amb_nd change was on qemux86* on 4.1
> > 
> > https://autobuilder.yoctoproject.org/main/builders/nightly-x86-64-l
> > sb/builds/911/steps/Running%20Sanity%20Tests/logs/stdio
> > 
> > fixed with:
> > 
> > http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=26b0657b2f7875
> > 1e60df6e695d5bc0dd703a5bc2
> 
> I had a look at this, and I don't have a root cause for why this is
> just popping up now. I'm trying to track down an older build, so I
> can
> compare the .configs.
> 
> The kernel code, and Kconfig files themselves haven't changed in some
> time, and the symbol in question AMD_NB is default 'y' if the
> dependencies are met. We include cfg/amd.scc in all of the qemux86*
> builds, so it gets enabled (as it should have before).
> 
> I could force that option off, but the whitelist is an appropriate
> fix,
> since this is an interaction between qemu and the kernel, whereas the
> files that include amd.scc are targeted at a range of platforms.
> 
> > 
> > The ppc issue is a separate one.
> 
> I've been blasting away on my 4.1.30 qemuppc build, and I'm not
> seeing
> anything yet, but I will keep trying.

Its still going on even with the recent updates:

https://autobuilder.yoctoproject.org/main/builders/nightly-ppc-lsb/builds/889/steps/Running%20Sanity%20Tests/logs/stdio

Seems the network works intermittently, dropping connections...

Cheers,

Richard
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 1/8] [autotools] export CCLD_FOR_BUILD

[Jérémy Rosen]

On 24/08/2016 02:50, Khem Raj wrote:

On Aug 23, 2016, at 12:25 AM, Jérémy Rosen  wrote:

The autotools class already export most *_FOR_BUILD from bitbake's
BUILD_* variables. Somehow the variable CCLD_FOR_BUILD was missing

At least libpcre uses that variable and will use gcc instead of BUILD_CCLD
to link binaries

Signed-off-by: Jérémy Rosen 
---
meta/classes/autotools.bbclass | 2 ++
1 file changed, 2 insertions(+), 0 deletions(-)

diff --git a/meta/classes/autotools.bbclass b/meta/classes/autotools.bbclass
index 076899c..9041021 100644
--- a/meta/classes/autotools.bbclass
+++ b/meta/classes/autotools.bbclass
@@ -49,6 +49,8 @@ export CXXFLAGS_FOR_BUILD="${BUILD_CXXFLAGS}"
export LD_FOR_BUILD = "${BUILD_LD}"
export LDFLAGS_FOR_BUILD = "${BUILD_LDFLAGS}"

+export CCLD_FOR_BUILD = "${BUILD_CCLD}”

if its used by just one recipe then its not worth abstracting it out. Perhaps
fixing
https://github.com/openembedded/openembedded-core/blob/master/meta/recipes-support/libpcre/libpcre/pcre-cross.patch
is another option ?
I'll have a look, I have been looking around and i'm not even sure if 
CC_FOR_BUILD is an autoconf thing... CCLD seems to be unique to that 
recipe. i'll look at the patch



+
def append_libtool_sysroot(d):
 # Only supply libtool sysroot option for non-native packages
 if not bb.data.inherits_class('native', d):
--
git-series 0.8.9
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] gitpkgv: Fix $GITPKV for a single named git source

[Clemens Lang]

Hi everybody,

Khem Raj  wrote:

>> On Aug 11, 2016, at 3:04 AM, Clemens Lang   
>> wrote:
>>
>> Recipes that fetch from a single git source, assign a name to this
>> source and do not set SRCREV_FORMAT to this name will always get
>> a GITPKGV value of "default", which causes version-going-backwards QA
>> errors.
>>
>> Fix this by automatically determining a suitable SRCREV_FORMAT from the
>> SRC_URI if none is set explicitly. This code does not run for multiple
>> git sources, because bitbake's fetcher enforces setting SRCREV_FORMAT
>> when multiple version-controlled sources are used.
>
> looks good.
>
>> Signed-off-by: Clemens Lang 
>> ---
>> meta-oe/classes/gitpkgv.bbclass | 9 -
>> 1 file changed, 8 insertions(+), 1 deletion(-)
>>
>> diff --git a/meta-oe/classes/gitpkgv.bbclass  
>> b/meta-oe/classes/gitpkgv.bbclass
>> index 1cba00c..f73304a 100644
>> --- a/meta-oe/classes/gitpkgv.bbclass
>> +++ b/meta-oe/classes/gitpkgv.bbclass
>> @@ -61,7 +61,14 @@ def get_git_pkgv(d, use_tags):
>> #
>> format = d.getVar('SRCREV_FORMAT', True)
>> if not format:
>> -format = 'default'
>> +names = []
>> +for url in ud.values():
>> +if url.type == 'git' or url.type == 'gitsm':
>> +names.extend(url.revisions.keys())
>> +if len(names) > 0:
>> +format = '_'.join(names)
>> +else:
>> +format = 'default'
>>
>> found = False
>> for url in ud.values():
>> —
>> 2.8.1

Shameless bump. I haven’t heard any feedback on this other than this “looks  
good” comment, but it isn’t merged. Is there a problem with the patch?

-- 
Clemens Lang • Development Specialist
BMW Car IT GmbH • Lise-Meitner-Str. 14 • 89081 Ulm • http://bmw-carit.com
-
BMW Car IT GmbH
Geschäftsführer: Michael Würtenberger und Alexis Trolin
Sitz und Registergericht: München HRB 134810
-
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] combo-layer: python3: fix UnicodeDecodeError

[Ed Bartosh]

check_patch function opens patch file in text mode. This causes
python3 to throw exception when calling readline():
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa7 in position
: invalid start byte

Opening file in binary mode and using binary type instead of strings
should fix this.

Signed-off-by: Ed Bartosh 
---
 scripts/combo-layer | 16 
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/scripts/combo-layer b/scripts/combo-layer
index 8f57ba5..b90bfc8 100755
--- a/scripts/combo-layer
+++ b/scripts/combo-layer
@@ -482,32 +482,32 @@ def check_repo_clean(repodir):
 sys.exit(1)
 
 def check_patch(patchfile):
-f = open(patchfile)
+f = open(patchfile, 'rb')
 ln = f.readline()
 of = None
 in_patch = False
 beyond_msg = False
-pre_buf = ''
+pre_buf = b''
 while ln:
 if not beyond_msg:
-if ln == '---\n':
+if ln == b'---\n':
 if not of:
 break
 in_patch = False
 beyond_msg = True
-elif ln.startswith('--- '):
+elif ln.startswith(b'--- '):
 # We have a diff in the commit message
 in_patch = True
 if not of:
 print('WARNING: %s contains a diff in its commit message, 
indenting to avoid failure during apply' % patchfile)
-of = open(patchfile + '.tmp', 'w')
+of = open(patchfile + '.tmp', 'wb')
 of.write(pre_buf)
-pre_buf = ''
-elif in_patch and not ln[0] in '+-@ \n\r':
+pre_buf = b''
+elif in_patch and not ln[0] in b'+-@ \n\r':
 in_patch = False
 if of:
 if in_patch:
-of.write(' ' + ln)
+of.write(b' ' + ln)
 else:
 of.write(ln)
 else:
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 0/1] bash: fix interpreter of ptest script

[kai.kang]

From: Kai Kang 

The following changes since commit 94646f4828c2aa11a89996763edaa2e9d57106a9:

  oeqa/buildiptables: Switch from netfilter.org to yoctoproject.org mirror 
(2016-08-23 17:59:43 +0100)

are available in the git repository at:

  git://git.pokylinux.org/poky-contrib kangkai/bash
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=kangkai/bash

Kai Kang (1):
  bash: fix interpreter of ptest script

 meta/recipes-extended/bash/bash.inc | 2 ++
 1 file changed, 2 insertions(+)

-- 
2.9.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/1] bash: fix interpreter of ptest script

[kai.kang]

From: Kai Kang 

The interpreter of bash ptest script tests/getopts5.sub is
/local/bin/bash which is not suitable for oe. Use /bin/bash instead.

Signed-off-by: Kai Kang 
---
 meta/recipes-extended/bash/bash.inc | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/recipes-extended/bash/bash.inc 
b/meta/recipes-extended/bash/bash.inc
index 1d08526..d299a4a 100644
--- a/meta/recipes-extended/bash/bash.inc
+++ b/meta/recipes-extended/bash/bash.inc
@@ -54,6 +54,8 @@ do_install_ptest () {
cp ${B}/Makefile ${D}${PTEST_PATH}
 sed -i -e 's/^Makefile/_Makefile/' -e 
"s,--sysroot=${STAGING_DIR_TARGET},,g" \
-e "s,${S},,g" -e "s,${B},,g" -e "s,${STAGING_DIR_NATIVE},,g" 
${D}${PTEST_PATH}/Makefile
+
+   sed -i -e 's:/local\(/bin/bash\):\1:' 
${D}${PTEST_PATH}/tests/getopts5.sub
 }
 
 pkg_postinst_${PN} () {
-- 
2.9.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] groff: correct the location path for awk

[Zhenbo Gao]

awk is located at /usr/bin/, but not /bin/

Signed-off-by: Zhenbo Gao 
---
 meta/recipes-extended/groff/groff_1.22.3.bb | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/meta/recipes-extended/groff/groff_1.22.3.bb 
b/meta/recipes-extended/groff/groff_1.22.3.bb
index 4bffa81..af5acbe 100644
--- a/meta/recipes-extended/groff/groff_1.22.3.bb
+++ b/meta/recipes-extended/groff/groff_1.22.3.bb
@@ -56,6 +56,12 @@ do_install_append() {
if [ -e ${D}${libdir}/charset.alias ]; then
rm -rf ${D}${libdir}/charset.alias
fi
+
+   # awk is located at /usr/bin/, not /bin/
+   SPECIAL_AWK=`find ${D} -name special.awk`
+   if [ -f ${SPECIAL_AWK} ]; then
+   sed -i -e 's:#!.*awk:#! ${USRBINPATH}/awk:' ${SPECIAL_AWK}
+   fi
 }
 
 do_install_append_class-native() {
-- 
1.9.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 7/8] [sanity] fix hardcoded references to gcc

[Jérémy Rosen]

On 23/08/2016 17:40, Jérémy Rosen wrote:




On 23/08/2016 17:11, Burton, Ross wrote:


On 23 August 2016 at 15:53, Burton, Ross > wrote:


I think this is because BUILD_CC by default contains whitespace,
and bb.utils.which looks for "gcc ".  I added a strip() to
check_app_exists and it passed, so can you please verify that
your series doesn't regress when BUILD_CC isn't overridden?


Patch sent to fix check_app_exists().  It currently just strips 
whitespace and doesn't handle someone passing "foo --bar" to 
check_app_exists but this is still a step in the right direction.


will do, but rebuilding from scratch is always long... so not right 
away :)


and... done. I do indeed need your patch to get this working, but it 
works correctly.
since I need to get a patch upstream in glibc V2 of this serie won't be 
here right away. Hopefully your patch will be accepted at that point 
(I'll check that before sending V2)

Ross







-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 0/9] oe-build-perf-test: use Python unittest framework

[Markus Lehtonen]

Patch #8, i.e. the patch suppressing logger output was too aggressive in v1 of
this patchset. We want to get most of the logger output to stderr, after all.
In v2 the logger output to stderr is only disabled during the time the tests
are being run.

Other patches are not changed (except for rebase on top of the latest master).


The following changes since commit a92c196449c516fe51786d429078bbb1213bb029:

  Allow for simultaneous do_rootfs tasks with rpm (2016-08-11 13:36:06 +0100)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib 
marquiz/buildperf/unittest
  
http://git.openembedded.org/openembedded-core-contrib/log/?h=marquiz/buildperf/unittest


Markus Lehtonen (9):
  oeqa.buildperf: rename module containing basic tests
  oeqa.buildperf: derive BuildPerfTestCase class from unitest.TestCase
  oeqa.buildperf: add BuildPerfTestLoader class
  oeqa.buildperf: add BuildPerfTestResult class
  oeqa.buildperf: convert test cases to unittest
  oe-build-perf-test: use new unittest based framework
  oeqa.buildperf: introduce runCmd2()
  oe-build-perf-test: suppress logger output when tests are being run
  oeqa.buildperf: be more verbose about failed commands

 meta/lib/oeqa/buildperf/__init__.py|  10 +-
 meta/lib/oeqa/buildperf/base.py| 245 +
 meta/lib/oeqa/buildperf/basic_tests.py | 133 --
 meta/lib/oeqa/buildperf/test_basic.py  | 121 
 scripts/oe-build-perf-test |  65 ++---
 5 files changed, 307 insertions(+), 267 deletions(-)
 delete mode 100644 meta/lib/oeqa/buildperf/basic_tests.py
 create mode 100644 meta/lib/oeqa/buildperf/test_basic.py

-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 5/8] [linux-libc-headers] allow overriding compiler name in KConfig

[Jérémy Rosen]

[...]


If this is broken, it needs to be fixed upstream.


I agree. now a days we have more than gcc available as system C 
compiler perhaps your patch might be upstream worthy.


As stated elsewhere, I now have a proper way to override the variables 
without patching the build system. V2 of this patch serie will get rid 
of that patch entirely and deal with it purely in the recipe

--
"Thou shalt not follow the NULL pointer, for chaos and madness await 
thee at its end"

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org 


http://lists.openembedded.org/mailman/listinfo/openembedded-core




-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 3/8] [rpm] force a configure before make distclean

[Jérémy Rosen]

On 23/08/2016 19:16, Mark Hatle wrote:

On 8/23/16 11:54 AM, Burton, Ross wrote:

On 23 August 2016 at 08:25, Jérémy Rosen > wrote:

 -   ( cd ${S}/syck ; set +e ; rm -- -l* ; make distclean ) || :
 +   ( cd ${S}/syck ; set +e ; rm -- -l* ; ./configure ; make distclean 
)
 || :


Running configure without all the options it needs to actually work just so that
distclean works in a subbuild that shouldn't be breaking on rebuilds because we
have out of tree builds scares me a little.

Mark, you added this line, can you remember what it was for?

The syck version can be preconfigured, and if it is -- it's wrong.

The 'set +e' is specifically in there to ALLOW it to fail.

If it is configured, we much remove the junk and clean it.. if it's not been
configured, we ignore the make distclean failure.

So I'm not sure why it is necessary to make the change.  'Failure is permitted',
and make distclean shouldn't be using any 'BUILD_CC' values from prior builds...
since it's only wiping everything that MAY have been previously built and leaked
into the release archive.
I havn't dived deeply on the whole chain of event, I have to admit that 
autotools scares me...


This particular line does work as expected. the set +e does indeed allow 
the make distclean to fail.


The problem is that if that distclean is not failing because nothing 
needs to be done it is failing because it can't find gcc. The junk isn't 
clean and the build of rpm will fail later complaining that the syck 
subdir is

already configured.

This is probably why you run the make distclean in the first place.

The need for gcc is weird... the error message I get is that no working 
version of gcc could be found, but the trace itself looks more like a 
call to configure (as if make distclean did some configuration check)


Again, I have not dug deeply enough in here to completely understand 
what's going on. I'd gladly take any clues you might have for a better fix.




--Mark


Ross




--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] inherit allarch and use RDEPENDS

[Richard Purdie]

On Wed, 2016-08-24 at 08:35 +0200, Mike Looijmans wrote:
> First time I've heard about this limitation, I don't understand 
> what's this about either. I've been telling people to put their arch 
> or machine specific code into a seperate package and RDEPEND on it 
> from an allarch package that holds the Python code that calls into
> that.
> 
> We haven't seen any problems with that.
> 
> So what is the bad thing that would happen if we don't set the 
> SIGGEN_EXCLUDE... variables in the layer.conf file?

If you have an allarch recipe X and you depend on a tune or machine
specific package from it, what you'll see is that when you switch
MACHINE (with a different tune), the recipe X will rebuild each time.
This is obviously suboptimal however its not the end of the world. It
is problematic if you're using PR server.

The reason it rebuilds is that its checksum has a dependency on
something which is tune or machine specific, you change MACHINE, the
checksum changes and the allarch recipe's checksum changes.

If we remove the dependency from the task checksum, the problem isn't
there and it doesn't rebuild, however there are some circumstances
where this could be an issue, e.g. if a dependency name was renamed by
debian.bbclass for example.

The mechanism for removing such a dependency is the one I've mentioned
in layer.conf.

You can see if you have a problem with your layers by running:

oe-selftest -r sstatetests.SStateTests.test_sstate_allarch_samesigs

Does that make the issue any clearer?

Related bugs:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=7298
https://bugzilla.yoctoproject.org/show_bug.cgi?id=8078
https://bugzilla.yoctoproject.org/show_bug.cgi?id=7724

Cheers,

Richard


-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 19/19] build-perf-test-wrapper.sh: make workdir configurable

[Markus Lehtonen]

New command line argument '-w' may be used to specify work dir other
than the default /build-perf-test.

Signed-off-by: Markus Lehtonen 
---
 scripts/contrib/build-perf-test-wrapper.sh | 12 +---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/scripts/contrib/build-perf-test-wrapper.sh 
b/scripts/contrib/build-perf-test-wrapper.sh
index ef2a46e..efc2b79 100755
--- a/scripts/contrib/build-perf-test-wrapper.sh
+++ b/scripts/contrib/build-perf-test-wrapper.sh
@@ -29,6 +29,7 @@ Optional arguments:
 disable tarball archiving
   -c COMMITISH  test (checkout) this commit
   -C GIT_REPO   commit results into Git
+  -w WORK_DIR   work dir for this script
 EOF
 }
 
@@ -37,7 +38,7 @@ EOF
 archive_dir=~/perf-results/archives
 commitish=""
 results_repo=""
-while getopts "ha:c:C:" opt; do
+while getopts "ha:c:C:w:" opt; do
 case $opt in
 h)  usage
 exit 0
@@ -48,13 +49,14 @@ while getopts "ha:c:C:" opt; do
 ;;
 C)  results_repo=`realpath "$OPTARG"`
 ;;
+w)  base_dir=`realpath "$OPTARG"`
+;;
 *)  usage
 exit 1
 ;;
 esac
 done
 
-
 echo "Running on `uname -n`"
 if ! git_topdir=$(git rev-parse --show-toplevel); then
 echo "The current working dir doesn't seem to be a git clone. Please 
cd there before running `basename $0`"
@@ -76,9 +78,13 @@ if [ -n "$commitish" ]; then
 fi
 
 # Setup build environment
+if [ -z "$base_dir" ]; then
+base_dir="$git_topdir/build-perf-test"
+fi
+echo "Using working dir $base_dir"
+
 timestamp=`date "+%Y%m%d%H%M%S"`
 git_rev=$(git rev-parse --short HEAD)  || exit 1
-base_dir="$git_topdir/build-perf-test"
 build_dir="$base_dir/build-$git_rev-$timestamp"
 results_dir="$base_dir/results-$git_rev-$timestamp"
 globalres_log="$base_dir/globalres.log"
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 16/19] build-perf-test-wrapper.sh: parse args with getopts

[Markus Lehtonen]

Use getopts for parsing the command line. This changes the usage so that
if a commit (to-be-tested) is defined it must be given by using '-c',
instead of a positional argument.

Signed-off-by: Markus Lehtonen 
---
 scripts/contrib/build-perf-test-wrapper.sh | 30 +++---
 1 file changed, 23 insertions(+), 7 deletions(-)

diff --git a/scripts/contrib/build-perf-test-wrapper.sh 
b/scripts/contrib/build-perf-test-wrapper.sh
index e8e8021..8eb4fdb 100755
--- a/scripts/contrib/build-perf-test-wrapper.sh
+++ b/scripts/contrib/build-perf-test-wrapper.sh
@@ -20,17 +20,33 @@
 
 script=`basename $0`
 usage () {
-echo "Usage: $script [COMMITISH]"
+cat << EOF
+Usage: $script [-h] [-c COMMITISH] [-C GIT_REPO]
+
+Optional arguments:
+  -hshow this help and exit.
+  -c COMMITISH  test (checkout) this commit
+EOF
 }
 
-if [ $# -gt 1 ]; then
-usage
-exit 1
-fi
-commitish=$1
 
-echo "Running on `uname -n`"
+# Parse command line arguments
+commitish=""
+while getopts "hc:" opt; do
+case $opt in
+h)  usage
+exit 0
+;;
+c)  commitish=$OPTARG
+;;
+*)  usage
+exit 1
+;;
+esac
+done
+
 
+echo "Running on `uname -n`"
 if ! git_topdir=$(git rev-parse --show-toplevel); then
 echo "The current working dir doesn't seem to be a git clone. Please 
cd there before running `basename $0`"
 exit 1
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 14/19] oeqa.buildperf: add git commit count to result data

[Markus Lehtonen]

This number represents the number of commits since the beginning of git
history until the tested revision. This helps e.g. in ordering results.

Signed-off-by: Markus Lehtonen 
---
 meta/lib/oeqa/buildperf/base.py | 18 +++---
 1 file changed, 11 insertions(+), 7 deletions(-)

diff --git a/meta/lib/oeqa/buildperf/base.py b/meta/lib/oeqa/buildperf/base.py
index faa30c7..f29c167 100644
--- a/meta/lib/oeqa/buildperf/base.py
+++ b/meta/lib/oeqa/buildperf/base.py
@@ -96,30 +96,34 @@ class BuildPerfTestResult(unittest.TextTestResult):
 self.repo = GitRepo('.')
 except GitError:
 self.repo = None
-self.git_commit, self.git_branch = self.get_git_revision()
+self.git_commit, self.git_commit_count, self.git_branch = \
+self.get_git_revision()
 self.hostname = socket.gethostname()
 self.start_time = self.elapsed_time = None
 self.successes = []
-log.info("Using Git branch:commit %s:%s", self.git_branch,
- self.git_commit)
+log.info("Using Git branch:commit %s:%s (%s)", self.git_branch,
+ self.git_commit, self.git_commit_count)
 
 def get_git_revision(self):
 """Get git branch and commit under testing"""
 commit = os.getenv('OE_BUILDPERFTEST_GIT_COMMIT')
+commit_cnt = os.getenv('OE_BUILDPERFTEST_GIT_COMMIT_COUNT')
 branch = os.getenv('OE_BUILDPERFTEST_GIT_BRANCH')
-if not self.repo and (not commit or not branch):
+if not self.repo and (not commit or not commit_cnt or not branch):
 log.info("The current working directory doesn't seem to be a Git "
  "repository clone. You can specify branch and commit "
- "displayed in test results with 
OE_BUILDPERFTEST_GIT_BRANCH "
- "and OE_BUILDPERFTEST_GIT_COMMIT environment variables")
+ "displayed in test results with 
OE_BUILDPERFTEST_GIT_BRANCH, "
+ "OE_BUILDPERFTEST_GIT_COMMIT and "
+ "OE_BUILDPERFTEST_GIT_COMMIT_COUNT environment variables")
 else:
 if not commit:
 commit = self.repo.rev_parse('HEAD^0')
+commit_cnt = self.repo.run_cmd(['rev-list', '--count', 
'HEAD^0'])
 if not branch:
 branch = self.repo.get_current_branch()
 if not branch:
 log.debug('Currently on detached HEAD')
-return str(commit), str(branch)
+return str(commit), str(commit_cnt), str(branch)
 
 def addSuccess(self, test):
 """Record results from successful tests"""
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 11/19] oe-build-perf-test: pre-check Git repo when using --commit-results

[Markus Lehtonen]

Do a pre-check on the path that is specified with --commit-results
before running any tests. The script will create and/or initialize a
fresh Git repository if the given directory does not exist or if it is
an empty directory. It fails if it finds a non-empty directory that is
not a Git repository.

Signed-off-by: Markus Lehtonen 
---
 scripts/oe-build-perf-test | 27 +++
 1 file changed, 27 insertions(+)

diff --git a/scripts/oe-build-perf-test b/scripts/oe-build-perf-test
index 390e4c9..437611c 100755
--- a/scripts/oe-build-perf-test
+++ b/scripts/oe-build-perf-test
@@ -31,6 +31,7 @@ import oeqa.buildperf
 from oeqa.buildperf import (BuildPerfTestLoader, BuildPerfTestResult,
 BuildPerfTestRunner, KernelDropCaches)
 from oeqa.utils.commands import runCmd
+from oeqa.utils.git import GitRepo, GitError
 
 
 # Set-up logging
@@ -68,7 +69,30 @@ def pre_run_sanity_check():
 if ret.status:
 log.error("bitbake command not found")
 return False
+return True
 
+def init_git_repo(path):
+"""Check/create Git repository where to store results"""
+path = os.path.abspath(path)
+if os.path.isfile(path):
+log.error("Invalid Git repo %s: path exists but is not a directory", 
path)
+return False
+if not os.path.isdir(path):
+try:
+os.mkdir(path)
+except (FileNotFoundError, PermissionError) as err:
+log.error("Failed to mkdir %s: %s", path, err)
+return False
+if not os.listdir(path):
+log.info("Initializing a new Git repo at %s", path)
+GitRepo.init(path)
+try:
+GitRepo(path, is_topdir=True)
+except GitError:
+log.error("No Git repository but a non-empty directory found at %s.\n"
+  "Please specify a Git repository, an empty directory or "
+  "a non-existing directory", path)
+return False
 return True
 
 
@@ -137,6 +161,9 @@ def main(argv=None):
 
 if not pre_run_sanity_check():
 return 1
+if args.commit_results:
+if not init_git_repo(args.commit_results):
+return 1
 
 # Check our capability to drop caches and ask pass if needed
 KernelDropCaches.check()
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 13/19] oe-build-perf-test: new {tag_num} keyword for --commit-results-tag

[Markus Lehtonen]

This makes it possible to create numbered tags, where the "basename" of
the tag is the same and the only difference is an (automatically)
increasing index number. This is useful if you do multiple test runs on
the same commit. For example, using:
--commit-results-tag {tester_host}/{git_commit}/{tag_num}

would give you tags something like:
myhost/decb3119dffd3fd38b800bebc1e510f9217a152e/0
myhost/decb3119dffd3fd38b800bebc1e510f9217a152e/1
...

The default tag format is updated to use this new keyword in order to
prevent unintentional tag name clashes.

Signed-off-by: Markus Lehtonen 
---
 meta/lib/oeqa/buildperf/base.py | 19 +++
 scripts/oe-build-perf-test  |  2 +-
 2 files changed, 16 insertions(+), 5 deletions(-)

diff --git a/meta/lib/oeqa/buildperf/base.py b/meta/lib/oeqa/buildperf/base.py
index a3cd3f3..faa30c7 100644
--- a/meta/lib/oeqa/buildperf/base.py
+++ b/meta/lib/oeqa/buildperf/base.py
@@ -226,10 +226,21 @@ class BuildPerfTestResult(unittest.TextTestResult):
 
 # Create (annotated) tag
 if tag:
-# Replace keywords
-tag = tag.format(git_branch=self.git_branch,
- git_commit=self.git_commit,
- tester_host=self.hostname)
+# Find tags matching the pattern
+tag_keywords = dict(git_branch=self.git_branch,
+git_commit=self.git_commit,
+tester_host=self.hostname,
+tag_num='[0-9]{1,5}')
+tag_re = re.compile(tag.format(**tag_keywords) + '$')
+tag_keywords['tag_num'] = 0
+for existing_tag in repo.run_cmd('tag').splitlines():
+if tag_re.match(existing_tag):
+tag_keywords['tag_num'] += 1
+
+tag = tag.format(**tag_keywords)
+msg = "Test run #{} of {}:{}\n".format(tag_keywords['tag_num'],
+   self.git_branch,
+   self.git_commit)
 repo.run_cmd(['tag', '-a', '-m', msg, tag, commit])
 
 finally:
diff --git a/scripts/oe-build-perf-test b/scripts/oe-build-perf-test
index 1ed5bdb..cd27584 100755
--- a/scripts/oe-build-perf-test
+++ b/scripts/oe-build-perf-test
@@ -140,7 +140,7 @@ def parse_args(argv):
 default="{git_branch}",
 help="Commit results to branch BRANCH.")
 parser.add_argument('--commit-results-tag', metavar='TAG',
-default="{git_branch}/{git_commit}",
+default="{git_branch}/{git_commit}/{tag_num}",
 help="Tag results commit with TAG.")
 
 return parser.parse_args(argv)
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 12/19] oe-build-perf-test: tag results committed to Git

[Markus Lehtonen]

Create a Git tag when committing results to a Git repository. This patch
also implements --commit-results-tag command line option for controlling
the tag name. The value
is a format string where the following fields may be used:
- {git_branch} - target branch being tested
- {git_commit} - target commit being tested
- {tester_host} - hostname of the tester machine

Tagging can be disabled by giving an empty string to
--commit-results-tag. The option has no effect if --commit-results is
not defined.

Signed-off-by: Markus Lehtonen 
---
 meta/lib/oeqa/buildperf/base.py | 11 ++-
 scripts/oe-build-perf-test  |  6 +-
 2 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oeqa/buildperf/base.py b/meta/lib/oeqa/buildperf/base.py
index 119e6ed..a3cd3f3 100644
--- a/meta/lib/oeqa/buildperf/base.py
+++ b/meta/lib/oeqa/buildperf/base.py
@@ -188,7 +188,7 @@ class BuildPerfTestResult(unittest.TextTestResult):
 fobj.write(','.join(values) + '\n')
 
 
-def git_commit_results(self, repo_path, branch=None):
+def git_commit_results(self, repo_path, branch=None, tag=None):
 """Commit results into a Git repository"""
 repo = GitRepo(repo_path, is_topdir=True)
 if not branch:
@@ -223,6 +223,15 @@ class BuildPerfTestResult(unittest.TextTestResult):
 if repo.get_current_branch() == branch:
 log.info("Updating %s HEAD to latest commit", repo_path)
 repo.run_cmd('reset --hard')
+
+# Create (annotated) tag
+if tag:
+# Replace keywords
+tag = tag.format(git_branch=self.git_branch,
+ git_commit=self.git_commit,
+ tester_host=self.hostname)
+repo.run_cmd(['tag', '-a', '-m', msg, tag, commit])
+
 finally:
 if os.path.exists(tmp_index):
 os.unlink(tmp_index)
diff --git a/scripts/oe-build-perf-test b/scripts/oe-build-perf-test
index 437611c..1ed5bdb 100755
--- a/scripts/oe-build-perf-test
+++ b/scripts/oe-build-perf-test
@@ -139,6 +139,9 @@ def parse_args(argv):
 parser.add_argument('--commit-results-branch', metavar='BRANCH',
 default="{git_branch}",
 help="Commit results to branch BRANCH.")
+parser.add_argument('--commit-results-tag', metavar='TAG',
+default="{git_branch}/{git_commit}",
+help="Tag results commit with TAG.")
 
 return parser.parse_args(argv)
 
@@ -193,7 +196,8 @@ def main(argv=None):
 result.update_globalres_file(args.globalres_file)
 if args.commit_results:
 result.git_commit_results(args.commit_results,
-  args.commit_results_branch)
+  args.commit_results_branch,
+  args.commit_results_tag)
 return 0
 
 return 1
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 09/19] oe-build-perf-test: implement --commit-results-branch

[Markus Lehtonen]

A new command line option for defining the branch where results are
commited. The value is actually a format string accepting two field
names:
- {git_branch} expands to the name of the target branch being tested
- {tester_host} expands to the hostname of the tester machine

The option has no effect if --commit-results is not used.

Signed-off-by: Markus Lehtonen 
---
 meta/lib/oeqa/buildperf/base.py | 5 +
 scripts/oe-build-perf-test  | 6 +-
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/buildperf/base.py b/meta/lib/oeqa/buildperf/base.py
index 8f7d88c..119e6ed 100644
--- a/meta/lib/oeqa/buildperf/base.py
+++ b/meta/lib/oeqa/buildperf/base.py
@@ -193,6 +193,11 @@ class BuildPerfTestResult(unittest.TextTestResult):
 repo = GitRepo(repo_path, is_topdir=True)
 if not branch:
 branch = self.git_branch
+else:
+# Replace keywords
+branch = branch.format(git_branch=self.git_branch,
+   tester_host=self.hostname)
+
 log.info("Committing test results into %s %s", repo_path, branch)
 tmp_index = os.path.join(repo_path, '.git', 'index.oe-build-perf')
 try:
diff --git a/scripts/oe-build-perf-test b/scripts/oe-build-perf-test
index d6ea5ce..390e4c9 100755
--- a/scripts/oe-build-perf-test
+++ b/scripts/oe-build-perf-test
@@ -112,6 +112,9 @@ def parse_args(argv):
 parser.add_argument('--commit-results', metavar='GIT_DIR',
 type=os.path.abspath,
 help="Commit result data to a (local) git repository")
+parser.add_argument('--commit-results-branch', metavar='BRANCH',
+default="{git_branch}",
+help="Commit results to branch BRANCH.")
 
 return parser.parse_args(argv)
 
@@ -162,7 +165,8 @@ def main(argv=None):
 if args.globalres_file:
 result.update_globalres_file(args.globalres_file)
 if args.commit_results:
-result.git_commit_results(args.commit_results)
+result.git_commit_results(args.commit_results,
+  args.commit_results_branch)
 return 0
 
 return 1
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 06/19] oeqa.utils.git.GitRepo: new arg to require topdir

[Markus Lehtonen]

Add a new 'is_topdir' argument to the GitRepo init method which
validates that the given path is the top directory of a Git repository.
Without this argument GitRepo also accepts subdirectories of a Git
repository (in which case GitRepo will point to the parent directory
that is the top directory of this repository) which may have undesired
in some cases.

Signed-off-by: Markus Lehtonen 
---
 meta/lib/oeqa/utils/git.py | 9 +++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oeqa/utils/git.py b/meta/lib/oeqa/utils/git.py
index 0fc8112..ca84680 100644
--- a/meta/lib/oeqa/utils/git.py
+++ b/meta/lib/oeqa/utils/git.py
@@ -4,6 +4,8 @@
 # Released under the MIT license (see COPYING.MIT)
 #
 """Git repository interactions"""
+import os
+
 from oeqa.utils.commands import runCmd
 
 
@@ -13,9 +15,12 @@ class GitError(Exception):
 
 class GitRepo(object):
 """Class representing a Git repository clone"""
-def __init__(self, cwd):
+def __init__(self, path, is_topdir=False):
 self.top_dir = self._run_git_cmd_at(['rev-parse', '--show-toplevel'],
-cwd)
+path)
+realpath = os.path.realpath(path)
+if is_topdir and realpath != self.top_dir:
+raise GitError("{} is not a Git top directory".format(realpath))
 
 @staticmethod
 def _run_git_cmd_at(git_args, cwd, **kwargs):
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 05/19] oeqa.utils.git: implement GitRepo.get_current_branch()

[Markus Lehtonen]

Signed-off-by: Markus Lehtonen 
---
 meta/lib/oeqa/buildperf/base.py | 7 ++-
 meta/lib/oeqa/utils/git.py  | 8 
 2 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/meta/lib/oeqa/buildperf/base.py b/meta/lib/oeqa/buildperf/base.py
index 6a8d9fe..adc3da3 100644
--- a/meta/lib/oeqa/buildperf/base.py
+++ b/meta/lib/oeqa/buildperf/base.py
@@ -116,12 +116,9 @@ class BuildPerfTestResult(unittest.TextTestResult):
 if not rev:
 rev = self.repo.rev_parse('HEAD')
 if not branch:
-try:
-# Strip 11 chars, i.e. 'refs/heads' from the beginning
-branch = self.repo.run_cmd(['symbolic-ref', 'HEAD'])[11:]
-except GitError:
+branch = self.repo.get_current_branch()
+if not branch:
 log.debug('Currently on detached HEAD')
-branch = None
 return str(rev), str(branch)
 
 def addSuccess(self, test):
diff --git a/meta/lib/oeqa/utils/git.py b/meta/lib/oeqa/utils/git.py
index 6474654..0fc8112 100644
--- a/meta/lib/oeqa/utils/git.py
+++ b/meta/lib/oeqa/utils/git.py
@@ -46,4 +46,12 @@ class GitRepo(object):
 # Revision does not exist
 return None
 
+def get_current_branch(self):
+"""Get current branch"""
+try:
+# Strip 11 chars, i.e. 'refs/heads' from the beginning
+return self.run_cmd(['symbolic-ref', 'HEAD'])[11:]
+except GitError:
+return None
+
 
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 17/19] build-perf-test-wrapper.sh: allow saving results in Git

[Markus Lehtonen]

Add new command line argument '-C' that allows saving results in a Git
repository.

Signed-off-by: Markus Lehtonen 
---
 scripts/contrib/build-perf-test-wrapper.sh | 14 +++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/scripts/contrib/build-perf-test-wrapper.sh 
b/scripts/contrib/build-perf-test-wrapper.sh
index 8eb4fdb..2fb4b11 100755
--- a/scripts/contrib/build-perf-test-wrapper.sh
+++ b/scripts/contrib/build-perf-test-wrapper.sh
@@ -26,19 +26,23 @@ Usage: $script [-h] [-c COMMITISH] [-C GIT_REPO]
 Optional arguments:
   -hshow this help and exit.
   -c COMMITISH  test (checkout) this commit
+  -C GIT_REPO   commit results into Git
 EOF
 }
 
 
 # Parse command line arguments
 commitish=""
-while getopts "hc:" opt; do
+results_repo=""
+while getopts "hc:C:" opt; do
 case $opt in
 h)  usage
 exit 0
 ;;
 c)  commitish=$OPTARG
 ;;
+C)  results_repo=`realpath "$OPTARG"`
+;;
 *)  usage
 exit 1
 ;;
@@ -73,13 +77,14 @@ base_dir="$git_topdir/build-perf-test"
 build_dir="$base_dir/build-$git_rev-$timestamp"
 results_dir="$base_dir/results-$git_rev-$timestamp"
 globalres_log="$base_dir/globalres.log"
+machine="qemux86"
 
 mkdir -p "$base_dir"
 source ./oe-init-build-env $build_dir >/dev/null || exit 1
 
 # Additional config
 auto_conf="$build_dir/conf/auto.conf"
-echo 'MACHINE = "qemux86"' > "$auto_conf"
+echo "MACHINE = \"$machine\"" > "$auto_conf"
 echo 'BB_NUMBER_THREADS = "8"' >> "$auto_conf"
 echo 'PARALLEL_MAKE = "-j 8"' >> "$auto_conf"
 echo "DL_DIR = \"$base_dir/downloads\"" >> "$auto_conf"
@@ -93,7 +98,10 @@ fi
 # Run actual test script
 if ! oe-build-perf-test --out-dir "$results_dir" \
 --globalres-file "$globalres_log" \
---lock-file "$base_dir/oe-build-perf.lock"; then
+--lock-file "$base_dir/oe-build-perf.lock" \
+--commit-results "$results_repo" \
+--commit-results-branch 
"{tester_host}/{git_branch}/$machine" \
+--commit-results-tag 
"{tester_host}/{git_branch}/$machine/{git_commit_count}-g{git_commit}/{tag_num}";
 then
 echo "oe-build-perf-test script failed!"
 exit 1
 fi
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 18/19] build-perf-test-wrapper.sh: make archive dir configurable

[Markus Lehtonen]

Add new command line argument '-a' that can be used to define the
directory where results (tarballs) are archived. Giving an empty string
disables archiving which makes sense if you store results in Git.

Signed-off-by: Markus Lehtonen 
---
 scripts/contrib/build-perf-test-wrapper.sh | 22 ++
 1 file changed, 14 insertions(+), 8 deletions(-)

diff --git a/scripts/contrib/build-perf-test-wrapper.sh 
b/scripts/contrib/build-perf-test-wrapper.sh
index 2fb4b11..ef2a46e 100755
--- a/scripts/contrib/build-perf-test-wrapper.sh
+++ b/scripts/contrib/build-perf-test-wrapper.sh
@@ -25,6 +25,8 @@ Usage: $script [-h] [-c COMMITISH] [-C GIT_REPO]
 
 Optional arguments:
   -hshow this help and exit.
+  -a ARCHIVE_DIRarchive results tarball here, give an empty string to
+disable tarball archiving
   -c COMMITISH  test (checkout) this commit
   -C GIT_REPO   commit results into Git
 EOF
@@ -32,13 +34,16 @@ EOF
 
 
 # Parse command line arguments
+archive_dir=~/perf-results/archives
 commitish=""
 results_repo=""
-while getopts "hc:C:" opt; do
+while getopts "ha:c:C:" opt; do
 case $opt in
 h)  usage
 exit 0
 ;;
+a)  archive_dir=`realpath "$OPTARG"`
+;;
 c)  commitish=$OPTARG
 ;;
 C)  results_repo=`realpath "$OPTARG"`
@@ -112,13 +117,14 @@ echo -ne "\n"
 
 cat "$globalres_log"
 
-echo -ne "\n\n-\n"
-echo "Archiving results dir..."
-archive_dir=~/perf-results/archives
-mkdir -p "$archive_dir"
-results_basename=`basename "$results_dir"`
-results_dirname=`dirname "$results_dir"`
-tar -czf "$archive_dir/`uname -n`-${results_basename}.tar.gz" -C 
"$results_dirname" "$results_basename"
+if [ -n "$archive_dir" ]; then
+echo -ne "\n\n-\n"
+echo "Archiving results in $archive_dir"
+mkdir -p "$archive_dir"
+results_basename=`basename "$results_dir"`
+results_dirname=`dirname "$results_dir"`
+tar -czf "$archive_dir/`uname -n`-${results_basename}.tar.gz" -C 
"$results_dirname" "$results_basename"
+fi
 
 rm -rf "$build_dir"
 rm -rf "$results_dir"
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 10/19] oeqa.utils.git: implement init() method

[Markus Lehtonen]

Method for doing 'git init'.

Signed-off-by: Markus Lehtonen 
---
 meta/lib/oeqa/utils/git.py | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/meta/lib/oeqa/utils/git.py b/meta/lib/oeqa/utils/git.py
index ca84680..ae85d27 100644
--- a/meta/lib/oeqa/utils/git.py
+++ b/meta/lib/oeqa/utils/git.py
@@ -35,6 +35,12 @@ class GitRepo(object):
 cmd_str, ret.status, ret.output))
 return ret.output.strip()
 
+@staticmethod
+def init(path):
+"""Initialize a new Git repository"""
+GitRepo._run_git_cmd_at('init', cwd=path)
+return GitRepo(path, is_topdir=True)
+
 def run_cmd(self, git_args, env_update=None):
 """Run Git command"""
 env = None
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 15/19] oe-build-perf-test: add {git_commit_count} keyword for --commit-results-tag

[Markus Lehtonen]

Makes it possible to create easily sortable tags. Also, the default tag
format is updated to use the new keyword.

Signed-off-by: Markus Lehtonen 
---
 meta/lib/oeqa/buildperf/base.py | 1 +
 scripts/oe-build-perf-test  | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/buildperf/base.py b/meta/lib/oeqa/buildperf/base.py
index f29c167..97be58f 100644
--- a/meta/lib/oeqa/buildperf/base.py
+++ b/meta/lib/oeqa/buildperf/base.py
@@ -233,6 +233,7 @@ class BuildPerfTestResult(unittest.TextTestResult):
 # Find tags matching the pattern
 tag_keywords = dict(git_branch=self.git_branch,
 git_commit=self.git_commit,
+git_commit_count=self.git_commit_count,
 tester_host=self.hostname,
 tag_num='[0-9]{1,5}')
 tag_re = re.compile(tag.format(**tag_keywords) + '$')
diff --git a/scripts/oe-build-perf-test b/scripts/oe-build-perf-test
index cd27584..35a4839 100755
--- a/scripts/oe-build-perf-test
+++ b/scripts/oe-build-perf-test
@@ -140,7 +140,7 @@ def parse_args(argv):
 default="{git_branch}",
 help="Commit results to branch BRANCH.")
 parser.add_argument('--commit-results-tag', metavar='TAG',
-default="{git_branch}/{git_commit}/{tag_num}",
+
default="{git_branch}/{git_commit_count}-g{git_commit}/{tag_num}",
 help="Tag results commit with TAG.")
 
 return parser.parse_args(argv)
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 07/19] oeqa.buildperf: use term commit instead of revision

[Markus Lehtonen]

This is basically a internal change, at this point. Term 'commit' better
represents the data we actually have. Term 'revision' is more vague and
could be understood to point to a tag object, for example.

Signed-off-by: Markus Lehtonen 
---
 meta/lib/oeqa/buildperf/base.py | 30 +++---
 1 file changed, 15 insertions(+), 15 deletions(-)

diff --git a/meta/lib/oeqa/buildperf/base.py b/meta/lib/oeqa/buildperf/base.py
index adc3da3..0e77aae 100644
--- a/meta/lib/oeqa/buildperf/base.py
+++ b/meta/lib/oeqa/buildperf/base.py
@@ -96,30 +96,30 @@ class BuildPerfTestResult(unittest.TextTestResult):
 self.repo = GitRepo('.')
 except GitError:
 self.repo = None
-self.git_revision, self.git_branch = self.get_git_revision()
+self.git_commit, self.git_branch = self.get_git_revision()
 self.hostname = socket.gethostname()
 self.start_time = self.elapsed_time = None
 self.successes = []
-log.info("Using Git branch:revision %s:%s", self.git_branch,
- self.git_revision)
+log.info("Using Git branch:commit %s:%s", self.git_branch,
+ self.git_commit)
 
 def get_git_revision(self):
-"""Get git branch and revision under testing"""
-rev = os.getenv('OE_BUILDPERFTEST_GIT_REVISION')
+"""Get git branch and commit under testing"""
+commit = os.getenv('OE_BUILDPERFTEST_GIT_COMMIT')
 branch = os.getenv('OE_BUILDPERFTEST_GIT_BRANCH')
-if not self.repo and (not rev or not branch):
+if not self.repo and (not commit or not branch):
 log.info("The current working directory doesn't seem to be a Git "
- "repository clone. You can specify branch and revision "
- "used in test results with OE_BUILDPERFTEST_GIT_REVISION "
- "and OE_BUILDPERFTEST_GIT_BRANCH environment variables")
+ "repository clone. You can specify branch and commit "
+ "displayed in test results with 
OE_BUILDPERFTEST_GIT_BRANCH "
+ "and OE_BUILDPERFTEST_GIT_COMMIT environment variables")
 else:
-if not rev:
-rev = self.repo.rev_parse('HEAD')
+if not commit:
+commit = self.repo.rev_parse('HEAD^0')
 if not branch:
 branch = self.repo.get_current_branch()
 if not branch:
 log.debug('Currently on detached HEAD')
-return str(rev), str(branch)
+return str(commit), str(branch)
 
 def addSuccess(self, test):
 """Record results from successful tests"""
@@ -165,9 +165,9 @@ class BuildPerfTestResult(unittest.TextTestResult):
   'test4': ((7, 1), (10, 2))}
 
 if self.repo:
-git_tag_rev = self.repo.run_cmd(['describe', self.git_revision])
+git_tag_rev = self.repo.run_cmd(['describe', self.git_commit])
 else:
-git_tag_rev = self.git_revision
+git_tag_rev = self.git_commit
 
 values = ['0'] * 12
 for status, (test, msg) in self.all_results():
@@ -183,7 +183,7 @@ class BuildPerfTestResult(unittest.TextTestResult):
 with open(filename, 'a') as fobj:
 fobj.write('{},{}:{},{},'.format(self.hostname,
  self.git_branch,
- self.git_revision,
+ self.git_commit,
  git_tag_rev))
 fobj.write(','.join(values) + '\n')
 
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 08/19] oe-build-perf-test: support committing results data to Git

[Markus Lehtonen]

Implement a new command line option '--commit-results' which commits the
test results data into a Git repository. The given path must be an
existing initialized local Git repository.

Signed-off-by: Markus Lehtonen 
---
 meta/lib/oeqa/buildperf/base.py | 35 +++
 scripts/oe-build-perf-test  |  5 +
 2 files changed, 40 insertions(+)

diff --git a/meta/lib/oeqa/buildperf/base.py b/meta/lib/oeqa/buildperf/base.py
index 0e77aae..8f7d88c 100644
--- a/meta/lib/oeqa/buildperf/base.py
+++ b/meta/lib/oeqa/buildperf/base.py
@@ -188,6 +188,41 @@ class BuildPerfTestResult(unittest.TextTestResult):
 fobj.write(','.join(values) + '\n')
 
 
+def git_commit_results(self, repo_path, branch=None):
+"""Commit results into a Git repository"""
+repo = GitRepo(repo_path, is_topdir=True)
+if not branch:
+branch = self.git_branch
+log.info("Committing test results into %s %s", repo_path, branch)
+tmp_index = os.path.join(repo_path, '.git', 'index.oe-build-perf')
+try:
+# Create new commit object from the new results
+env_update = {'GIT_INDEX_FILE': tmp_index,
+  'GIT_WORK_TREE': self.out_dir}
+repo.run_cmd('add .', env_update)
+tree = repo.run_cmd('write-tree', env_update)
+parent = repo.rev_parse(branch)
+msg = "Results of {}:{}\n".format(self.git_branch, self.git_commit)
+git_cmd = ['commit-tree', tree, '-m', msg]
+if parent:
+git_cmd += ['-p', parent]
+commit = repo.run_cmd(git_cmd, env_update)
+
+# Update branch head
+git_cmd = ['update-ref', 'refs/heads/' + branch, commit]
+if parent:
+git_cmd.append(parent)
+repo.run_cmd(git_cmd)
+
+# Update current HEAD, if we're on branch 'branch'
+if repo.get_current_branch() == branch:
+log.info("Updating %s HEAD to latest commit", repo_path)
+repo.run_cmd('reset --hard')
+finally:
+if os.path.exists(tmp_index):
+os.unlink(tmp_index)
+
+
 class BuildPerfTestCase(unittest.TestCase):
 """Base class for build performance tests"""
 SYSRES = 'sysres'
diff --git a/scripts/oe-build-perf-test b/scripts/oe-build-perf-test
index 21759c6..d6ea5ce 100755
--- a/scripts/oe-build-perf-test
+++ b/scripts/oe-build-perf-test
@@ -109,6 +109,9 @@ def parse_args(argv):
 help="Output directory for test results")
 parser.add_argument('--run-tests', nargs='+', metavar='TEST',
 help="List of tests to run")
+parser.add_argument('--commit-results', metavar='GIT_DIR',
+type=os.path.abspath,
+help="Commit result data to a (local) git repository")
 
 return parser.parse_args(argv)
 
@@ -158,6 +161,8 @@ def main(argv=None):
 if result.wasSuccessful():
 if args.globalres_file:
 result.update_globalres_file(args.globalres_file)
+if args.commit_results:
+result.git_commit_results(args.commit_results)
 return 0
 
 return 1
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 01/19] oe-build-perf-test: implement --run-tests option

[Markus Lehtonen]

Makes it possible to run only a subset of tests.

NOTE: The tests currently have (unwritten) dependencies on each other so
use this option with care. Mainly for debugging.

Signed-off-by: Markus Lehtonen 
---
 scripts/oe-build-perf-test | 7 ++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/scripts/oe-build-perf-test b/scripts/oe-build-perf-test
index 88af40a..808531e 100755
--- a/scripts/oe-build-perf-test
+++ b/scripts/oe-build-perf-test
@@ -105,6 +105,8 @@ def parse_args(argv):
 help="Lock file to use")
 parser.add_argument('-o', '--out-dir', default='results-{date}',
 help="Output directory for test results")
+parser.add_argument('--run-tests', nargs='+', metavar='TEST',
+help="List of tests to run")
 
 return parser.parse_args(argv)
 
@@ -133,7 +135,10 @@ def main(argv=None):
 
 # Load build perf tests
 loader = BuildPerfTestLoader()
-suite = loader.discover(start_dir=os.path.dirname(oeqa.buildperf.__file__))
+if args.run_tests:
+suite = loader.loadTestsFromNames(args.run_tests, oeqa.buildperf)
+else:
+suite = loader.loadTestsFromModule(oeqa.buildperf)
 
 archive_build_conf(out_dir)
 runner = BuildPerfTestRunner(out_dir, verbosity=2)
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 00/19] oe-build-pef-test: support saving results in a Git repo

[Markus Lehtonen]

This patchset makes it possible to store build performance test results in a
Git repository. The support is written directly in oe-build-perf-test script
which hopefully makes it easier use. The new feature is disabled by default,
and, it must be enabled with --commit-results option. Further command line
options allow customization of the branch and tag names to use for the results.

[YOCTO #9624]


The following changes since commit 25f6af8895d5f5c6dcedde0a21285d63522769c8:

  oeqa/buildiptables: Switch from netfilter.org to yoctoproject.org mirror 
(2016-08-23 17:59:20 +0100)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib marquiz/buildperf/git
  
http://git.openembedded.org/openembedded-core-contrib/log/?h=marquiz/buildperf/git


Markus Lehtonen (19):
  oe-build-perf-test: implement --run-tests option
  oe-build-perf-test: use absolute paths in cmdline args
  oeqa.utils.git: support git commands with updated env
  oeqa.utils.git: introduce GitRepo.rev_parse()
  oeqa.utils.git: implement GitRepo.get_current_branch()
  oeqa.utils.git.GitRepo: new arg to require topdir
  oeqa.buildperf: use term commit instead of revision
  oe-build-perf-test: support committing results data to Git
  oe-build-perf-test: implement --commit-results-branch
  oeqa.utils.git: implement init() method
  oe-build-perf-test: pre-check Git repo when using --commit-results
  oe-build-perf-test: tag results committed to Git
  oe-build-perf-test: new {tag_num} keyword for --commit-results-tag
  oeqa.buildperf: add git commit count to result data
  oe-build-perf-test: add {git_commit_count} keyword for
--commit-results-tag
  build-perf-test-wrapper.sh: parse args with getopts
  build-perf-test-wrapper.sh: allow saving results in Git
  build-perf-test-wrapper.sh: make archive dir configurable
  build-perf-test-wrapper.sh: make workdir configurable

 meta/lib/oeqa/buildperf/base.py| 102 +++--
 meta/lib/oeqa/utils/git.py |  38 +--
 scripts/contrib/build-perf-test-wrapper.sh |  70 +++-
 scripts/oe-build-perf-test |  51 ++-
 4 files changed, 218 insertions(+), 43 deletions(-)

-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 04/19] oeqa.utils.git: introduce GitRepo.rev_parse()

[Markus Lehtonen]

Signed-off-by: Markus Lehtonen 
---
 meta/lib/oeqa/buildperf/base.py | 2 +-
 meta/lib/oeqa/utils/git.py  | 7 +++
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/buildperf/base.py b/meta/lib/oeqa/buildperf/base.py
index 30b8e47..6a8d9fe 100644
--- a/meta/lib/oeqa/buildperf/base.py
+++ b/meta/lib/oeqa/buildperf/base.py
@@ -114,7 +114,7 @@ class BuildPerfTestResult(unittest.TextTestResult):
  "and OE_BUILDPERFTEST_GIT_BRANCH environment variables")
 else:
 if not rev:
-rev = self.repo.run_cmd(['rev-parse', 'HEAD'])
+rev = self.repo.rev_parse('HEAD')
 if not branch:
 try:
 # Strip 11 chars, i.e. 'refs/heads' from the beginning
diff --git a/meta/lib/oeqa/utils/git.py b/meta/lib/oeqa/utils/git.py
index 6a2987f..6474654 100644
--- a/meta/lib/oeqa/utils/git.py
+++ b/meta/lib/oeqa/utils/git.py
@@ -38,5 +38,12 @@ class GitRepo(object):
 env.update(env_update)
 return self._run_git_cmd_at(git_args, self.top_dir, env=env)
 
+def rev_parse(self, revision):
+"""Do git rev-parse"""
+try:
+return self.run_cmd(['rev-parse', revision])
+except GitError:
+# Revision does not exist
+return None
 
 
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 03/19] oeqa.utils.git: support git commands with updated env

[Markus Lehtonen]

Extend GitRepo.run_cmd so that the caller may redefine and/or define
additional environment variables that will be used when the git command
is run.

Signed-off-by: Markus Lehtonen 
---
 meta/lib/oeqa/utils/git.py | 8 ++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oeqa/utils/git.py b/meta/lib/oeqa/utils/git.py
index a4c6741..6a2987f 100644
--- a/meta/lib/oeqa/utils/git.py
+++ b/meta/lib/oeqa/utils/git.py
@@ -30,9 +30,13 @@ class GitRepo(object):
 cmd_str, ret.status, ret.output))
 return ret.output.strip()
 
-def run_cmd(self, git_args):
+def run_cmd(self, git_args, env_update=None):
 """Run Git command"""
-return self._run_git_cmd_at(git_args, self.top_dir)
+env = None
+if env_update:
+env = os.environ.copy()
+env.update(env_update)
+return self._run_git_cmd_at(git_args, self.top_dir, env=env)
 
 
 
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 02/19] oe-build-perf-test: use absolute paths in cmdline args

[Markus Lehtonen]

This is safer as the current working directory may change.

Signed-off-by: Markus Lehtonen 
---
 scripts/oe-build-perf-test | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/scripts/oe-build-perf-test b/scripts/oe-build-perf-test
index 808531e..21759c6 100755
--- a/scripts/oe-build-perf-test
+++ b/scripts/oe-build-perf-test
@@ -99,11 +99,13 @@ def parse_args(argv):
 parser.add_argument('-D', '--debug', action='store_true',
 help='Enable debug level logging')
 parser.add_argument('--globalres-file',
+type=os.path.abspath,
 help="Append results to 'globalres' csv file")
 parser.add_argument('--lock-file', default='./oe-build-perf.lock',
-metavar='FILENAME',
+metavar='FILENAME', type=os.path.abspath,
 help="Lock file to use")
 parser.add_argument('-o', '--out-dir', default='results-{date}',
+type=os.path.abspath,
 help="Output directory for test results")
 parser.add_argument('--run-tests', nargs='+', metavar='TEST',
 help="List of tests to run")
-- 
2.6.6

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] perl: Correct perl path for ptest

[zhe.he]

From: He Zhe 

Substitute /usr/local with ${bindir}

Signed-off-by: He Zhe 
---
 meta/recipes-devtools/perl/perl-ptest.inc | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-devtools/perl/perl-ptest.inc 
b/meta/recipes-devtools/perl/perl-ptest.inc
index bf6fde6..d136c5c 100644
--- a/meta/recipes-devtools/perl/perl-ptest.inc
+++ b/meta/recipes-devtools/perl/perl-ptest.inc
@@ -5,6 +5,7 @@ SRC_URI += "file://run-ptest \
 
 do_install_ptest () {
mkdir -p ${D}${PTEST_PATH}
+   sed -e "s:\/usr\/local:${bindir}:g" -i cpan/version/t/*
sed -e "s:\/opt:\/usr:" -i Porting/add-package.pl
sed -e "s:\/local\/gnu\/:\/:" -i hints/cxux.sh
tar -c --exclude=\*.o --exclude=libperl.so --exclude=Makefile 
--exclude=makefile --exclude=hostperl \
-- 
2.8.3

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] nasm: 2.11.08 -> 2.12.02

[Dai Caiyun]

Upgrade nasm from 2.11.08 to 2.12.02.

Signed-off-by: Dai Caiyun 
---
 meta/recipes-devtools/nasm/{nasm_2.11.08.bb => nasm_2.12.02.bb} | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
 rename meta/recipes-devtools/nasm/{nasm_2.11.08.bb => nasm_2.12.02.bb} (81%)

diff --git a/meta/recipes-devtools/nasm/nasm_2.11.08.bb 
b/meta/recipes-devtools/nasm/nasm_2.12.02.bb
similarity index 81%
rename from meta/recipes-devtools/nasm/nasm_2.11.08.bb
rename to meta/recipes-devtools/nasm/nasm_2.12.02.bb
index e0724e6..e2e7158 100644
--- a/meta/recipes-devtools/nasm/nasm_2.11.08.bb
+++ b/meta/recipes-devtools/nasm/nasm_2.12.02.bb
@@ -7,8 +7,8 @@ COMPATIBLE_HOST = '(x86_64|i.86).*-(linux|freebsd.*)'
 
 SRC_URI = "http://www.nasm.us/pub/nasm/releasebuilds/${PV}/nasm-${PV}.tar.bz2 "
 
-SRC_URI[md5sum] = "7aae5cb8e03fac48029c82a7470ab066"
-SRC_URI[sha256sum] = 
"9da3a0291a0bdc06305b7ba194f1e2c2b55ae6f11210b4af43729868149d5445"
+SRC_URI[md5sum] = "d15843c3fb7db39af80571ee27ec6fad"
+SRC_URI[sha256sum] = 
"00b0891c678c065446ca59bcee64719d0096d54d6886e6e472aeee2e170ae324"
 
 inherit autotools-brokensep
 
-- 
2.7.4



-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] lsbinitscripts: 9.64 -> 9.68

[Wang Xin]

Upgrade lsbinitscripts from 9.64 to 9.68.

Signed-off-by: Wang Xin 
---
 .../0001-functions-avoid-exit-1-which-causes-init-scripts-to-.patch | 5 +++--
 .../lsb/{lsbinitscripts_9.64.bb => lsbinitscripts_9.68.bb}  | 6 +++---
 2 files changed, 6 insertions(+), 5 deletions(-)
 rename meta/recipes-extended/lsb/{lsbinitscripts_9.64.bb => 
lsbinitscripts_9.68.bb} (82%)

diff --git 
a/meta/recipes-extended/lsb/lsbinitscripts/0001-functions-avoid-exit-1-which-causes-init-scripts-to-.patch
 
b/meta/recipes-extended/lsb/lsbinitscripts/0001-functions-avoid-exit-1-which-causes-init-scripts-to-.patch
index 0809c9b..17c9002 100644
--- 
a/meta/recipes-extended/lsb/lsbinitscripts/0001-functions-avoid-exit-1-which-causes-init-scripts-to-.patch
+++ 
b/meta/recipes-extended/lsb/lsbinitscripts/0001-functions-avoid-exit-1-which-causes-init-scripts-to-.patch
@@ -11,13 +11,14 @@ diff --git a/rc.d/init.d/functions b/rc.d/init.d/functions
 index 6850046..11223f7 100644
 --- a/rc.d/init.d/functions
 +++ b/rc.d/init.d/functions
-@@ -597,5 +597,5 @@ if [ "$_use_systemctl" = "1" ]; then
+@@ -597,6 +597,6 @@ if [ "$_use_systemctl" = "1" ]; then
fi
  fi
  
 -strstr "$(cat /proc/cmdline)" "rc.debug" && set -x
 +strstr "$(cat /proc/cmdline)" "rc.debug" && set -x || true
- 
+ return 0
+
 -- 
 2.1.0
 
diff --git a/meta/recipes-extended/lsb/lsbinitscripts_9.64.bb 
b/meta/recipes-extended/lsb/lsbinitscripts_9.68.bb
similarity index 82%
rename from meta/recipes-extended/lsb/lsbinitscripts_9.64.bb
rename to meta/recipes-extended/lsb/lsbinitscripts_9.68.bb
index 7273050..0c08fff 100644
--- a/meta/recipes-extended/lsb/lsbinitscripts_9.64.bb
+++ b/meta/recipes-extended/lsb/lsbinitscripts_9.68.bb
@@ -8,13 +8,13 @@ RDEPENDS_${PN} += "util-linux"
 LIC_FILES_CHKSUM = "file://COPYING;md5=ebf4e8b49780ab187d51bd26aaa022c6"
 
 S="${WORKDIR}/initscripts-${PV}"
-SRC_URI = 
"http://pkgs.fedoraproject.org/repo/pkgs/initscripts/initscripts-${PV}.tar.bz2/9cce2ae1009750e84be37c09a028757e/initscripts-${PV}.tar.bz2
 \
+SRC_URI = 
"http://pkgs.fedoraproject.org/repo/pkgs/initscripts/initscripts-${PV}.tar.bz2/6a51a5af38e01445f53989ed0727c3e1/initscripts-${PV}.tar.bz2
 \
file://functions.patch \

file://0001-functions-avoid-exit-1-which-causes-init-scripts-to-.patch \
   " 
 
-SRC_URI[md5sum] = "9cce2ae1009750e84be37c09a028757e"
-SRC_URI[sha256sum] = 
"48b59ce8157cfc58bbd4b1dfa58ad1087245761ae11c2033b66ae3864ea7e1cf"
+SRC_URI[md5sum] = "6a51a5af38e01445f53989ed0727c3e1"
+SRC_URI[sha256sum] = 
"2a1c6e9dbaa37a676518f4803b501e107c058bb14ef7a8db24c52b77fbcba531"
 
 inherit update-alternatives
 
-- 
2.7.4



-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] inherit allarch and use RDEPENDS

[Mike Looijmans]

On 23-08-16 17:32, Richard Purdie wrote:> On Tue, 2016-08-23 at 15:15 +, Peter Kjellerstedt wrote:>> This is the first I heard about limits to recipes that inherit>> allarch>> and also use RDEPENDS. So after I read the above, I went and read the>> documentation for the allarch class, which unfortunately did not make>> my understanding much clearer. Am I to understand that a recipe that inherits allarch cannot have>> runtime dependencies on packages that are built differently per>> architecture or MACHINE? If so, what can it have runtime dependencies>> on? Only other allarch recipes? What are the design limitations>> behind this and is there anything that could be done to change the>> situation?>> It can depend on them, only if you add it to the list of dependencies> in layer.conf, either SIGGEN_EXCLUDERECIPES_ABISAFE or>   SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS. The implication of that is that if> one of these recipes changes, the dependency will not be rebuilt, so> you have to be careful about things like these packages getting> renamed.>> Its basically a choice, we can have allarch packages and accept we> can't always rebuild them through the sstate hash changes, or we don't> have allarch packages at all (or have packages that can't have non> -allarch dependencies).First time I've heard about this limitation, I don't understand what's this about either. I've been telling people to put their arch or machine specific code into a seperate package and RDEPEND on it from an allarch package that holds the Python code that calls into that.We haven't seen any problems with that.So what is the bad thing that would happen if we don't set the SIGGEN_EXCLUDE... variables in the layer.conf file?>> As an example, say that I have a recipe that only installs a static>> script, so inheriting allarch is a natural thing to do. However, for>> this script to work it must call a binary built by another recipe so>> it of course RDEPENDS on that other package. Are you saying this is>> wrong? Because that sounds odd to me as it severely limits the>> usefulness of the allarch class.>> You can do this if and only if you list it in layer.conf.>>> Another example would be a recipe that installs a static Perl script.>> Can it not inherit allarch while also have a runtime dependency on>> perl? If the above is true, why are there no QA tests or similar that catch>> these kind of problems?>> We do have sstate hash tests in oe-selftest which would highlight such> a problem.>> There are various open bugs complaining about this situation however> I've yet to come up with a way which solves the problem perfectly, much> as I'd like to.I'm totally in the dark as to what should go where now.Given the situation above, a recipe called "static-perl-script.bb" that requires the "perl" interpreter,  what should we put into layer.conf then?
 
Kind regards,
 
Mike Looijmans
System Expert
 





  
  

  TOPIC
  Products

   

   
  

  Materiaalweg
  4

   

   
  

  5681
  RJ Best

  T:

  +31 (0) 499 33
  69 69
  

  Postbus
  440

  E:

  mike.looijm...@topicproducts.com
  

  5680 AK
  Best

  W:

  www.topicproducts.com
  
The Netherlands



 Please consider the environment before printing this
e-mailTopic zoekt gedreven (embedded) software specialisten!



-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core