[OE-core] [PATCH 0/2] Bugfixes to package signing

[Markus Lehtonen]

This patchset contains two small bugfixes to rpm package and package feed
signing.

Markus Lehtonen (2):
  oe/gpg_sign: fix incorrect variable name
  sign_package_feed.bbclass: fix task dependencies

 meta/classes/sign_package_feed.bbclass | 1 +
 meta/lib/oe/gpg_sign.py| 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

-- 
2.6.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/2] sign_package_feed.bbclass: fix task dependencies

[Markus Lehtonen]

This dependency was already added to sign_rpm.bbclass. However, the same dep 
needs to be
added to sign_package_feed.bbclass, too, to cover the case where rpm
signing is disabled but package feed signing is enabled.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/classes/sign_package_feed.bbclass | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/classes/sign_package_feed.bbclass 
b/meta/classes/sign_package_feed.bbclass
index d5df8af..63ca02f 100644
--- a/meta/classes/sign_package_feed.bbclass
+++ b/meta/classes/sign_package_feed.bbclass
@@ -35,3 +35,4 @@ python () {
 }
 
 do_package_index[depends] += "signing-keys:do_export_public_keys"
+do_rootfs[depends] += "signing-keys:do_export_public_keys"
-- 
2.6.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/2] oe/gpg_sign: fix incorrect variable name

[Markus Lehtonen]

Prevents crash in signing if GPG_PATH is defined.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/lib/oe/gpg_sign.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
index 55abad8..821787e 100644
--- a/meta/lib/oe/gpg_sign.py
+++ b/meta/lib/oe/gpg_sign.py
@@ -58,7 +58,7 @@ class LocalSigner(object):
   "--passphrase-file '%s' -u '%s' " % \
   (self.gpg_bin, self.passphrase_file, self.keyid)
 if self.gpg_path:
-gpg_cmd += "--homedir %s " % self.gpg_path
+cmd += "--homedir %s " % self.gpg_path
 cmd += input_file
 status, output = oe.utils.getstatusoutput(cmd)
 if status:
-- 
2.6.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/3] oe/gpg_sign: check for python-pexpect when using local signing

[Markus Lehtonen]

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/lib/oe/gpg_sign.py | 10 +++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
index 8832ea9..ea35564 100644
--- a/meta/lib/oe/gpg_sign.py
+++ b/meta/lib/oe/gpg_sign.py
@@ -17,13 +17,17 @@ class LocalSigner(object):
 @classmethod
 def check_sanity(cls, d, keyid, passphrase_file):
 """(Pre-)check the sanity of a configuration"""
-msg = ""
+msgs = []
+try:
+import pexpect
+except ImportError:
+msgs.append("Please install python-pexpect that is needed by 
lcocal gpg signing.")
 missing_vars = ['%(keyid)s'] if not keyid else []
 if not passphrase_file:
 missing_vars.append('%(passphrase_file)s')
 if missing_vars:
-msg += "You need to define " + ' and '.join(missing_vars) + " in 
the config."
-return msg
+msgs.append("You need to define " + ' and '.join(missing_vars) + " 
in the config.")
+return ' '.join(msgs)
 
 def export_pubkey(self, output_file):
 """Export GPG public key to a file"""
-- 
2.6.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 0/3] signing: enhance sanity checking

[Markus Lehtonen]

This patchset contains that should make sanity checking of rpm and package feed
signing more sane.


The following changes since commit 11a6227759515da433230eb44eca1a4cb2ac3b14:

  e2fsprogs: Ensure we use the right mke2fs.conf when restoring from sstate 
(2016-02-05 11:16:20 +)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib marquiz/rpmsign

for you to fetch changes up to 643834ad064be34e2ad4218b436420cd5a1bc520:

  package signing: do sanity checking in an event handler (2016-02-05 15:32:29 
+0200)


Markus Lehtonen (3):
  package signing: do actual sanity checking in the signer class
  oe/gpg_sign: check for python-pexpect when using local signing
  package signing: do sanity checking in an event handler

 meta/classes/sign_package_feed.bbclass | 22 +-
 meta/classes/sign_rpm.bbclass  | 22 +-
 meta/lib/oe/gpg_sign.py| 26 ++
 3 files changed, 56 insertions(+), 14 deletions(-)

-- 
2.6.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/3] package signing: do actual sanity checking in the signer class

[Markus Lehtonen]

The configuration needed for different signing backends may vary
(although we currently support only one backend). Thus, do the actual
sanity checking of the configuration there.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/classes/sign_package_feed.bbclass | 14 ++
 meta/classes/sign_rpm.bbclass  | 14 ++
 meta/lib/oe/gpg_sign.py| 22 ++
 3 files changed, 38 insertions(+), 12 deletions(-)

diff --git a/meta/classes/sign_package_feed.bbclass 
b/meta/classes/sign_package_feed.bbclass
index d5df8af..3f6ff2d 100644
--- a/meta/classes/sign_package_feed.bbclass
+++ b/meta/classes/sign_package_feed.bbclass
@@ -23,10 +23,16 @@ PACKAGE_FEED_GPG_BACKEND ?= 'local'
 
 
 python () {
-# Check sanity of configuration
-for var in ('PACKAGE_FEED_GPG_NAME', 'PACKAGE_FEED_GPG_PASSPHRASE_FILE'):
-if not d.getVar(var, True):
-raise_sanity_error("You need to define %s in the config" % var, d)
+# Check sanity of config
+from oe.gpg_sign import get_signer_class
+signer = get_signer_class(d.getVar('PACKAGE_FEED_GPG_BACKEND', True))
+err_msg = signer.check_sanity(d,
+  d.getVar('PACKAGE_FEED_GPG_NAME', True),
+  d.getVar('PACKAGE_FEED_GPG_PASSPHRASE_FILE', 
True))
+if err_msg:
+raise_sanity_error(err_msg %{'keyid': 'PACKAGE_FEED_GPG_NAME',
+ 'passphrase_file': 
'PACKAGE_FEED_GPG_PASSPHRASE_FILE'},
+   d)
 
 # Set expected location of the public key
 d.setVar('PACKAGE_FEED_GPG_PUBKEY',
diff --git a/meta/classes/sign_rpm.bbclass b/meta/classes/sign_rpm.bbclass
index 8bcabee..79dc517 100644
--- a/meta/classes/sign_rpm.bbclass
+++ b/meta/classes/sign_rpm.bbclass
@@ -22,10 +22,16 @@ RPM_GPG_BACKEND ?= 'local'
 
 
 python () {
-# Check configuration
-for var in ('RPM_GPG_NAME', 'RPM_GPG_PASSPHRASE_FILE'):
-if not d.getVar(var, True):
-raise_sanity_error("You need to define %s in the config" % var, d)
+# Check sanity of config
+from oe.gpg_sign import get_signer_class
+signer = get_signer_class(d.getVar('RPM_GPG_BACKEND', True))
+err_msg = signer.check_sanity(d,
+  d.getVar('RPM_GPG_NAME', True),
+  d.getVar('RPM_GPG_PASSPHRASE_FILE', True))
+if err_msg:
+raise_sanity_error(err_msg %{'keyid': 'RPM_GPG_NAME',
+ 'passphrase_file': 
'RPM_GPG_PASSPHRASE_FILE'},
+   d)
 
 # Set the expected location of the public key
 d.setVar('RPM_GPG_PUBKEY', os.path.join(d.getVar('STAGING_ETCDIR_NATIVE', 
False),
diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
index 55abad8..8832ea9 100644
--- a/meta/lib/oe/gpg_sign.py
+++ b/meta/lib/oe/gpg_sign.py
@@ -14,6 +14,17 @@ class LocalSigner(object):
 self.gpg_path = d.getVar('GPG_PATH', True)
 self.rpm_bin = bb.utils.which(os.getenv('PATH'), "rpm")
 
+@classmethod
+def check_sanity(cls, d, keyid, passphrase_file):
+"""(Pre-)check the sanity of a configuration"""
+msg = ""
+missing_vars = ['%(keyid)s'] if not keyid else []
+if not passphrase_file:
+missing_vars.append('%(passphrase_file)s')
+if missing_vars:
+msg += "You need to define " + ' and '.join(missing_vars) + " in 
the config."
+return msg
+
 def export_pubkey(self, output_file):
 """Export GPG public key to a file"""
 cmd = '%s --batch --yes --export --armor -o %s ' % \
@@ -66,11 +77,14 @@ class LocalSigner(object):
   (input_file, output))
 
 
-def get_signer(d, backend, keyid, passphrase_file):
-"""Get signer object for the specified backend"""
-# Use local signing by default
+def get_signer_class(backend):
+"""Get signer class for the specified backend"""
 if backend == 'local':
-return LocalSigner(d, keyid, passphrase_file)
+return LocalSigner
 else:
 bb.fatal("Unsupported signing backend '%s'" % backend)
 
+
+def get_signer(d, backend, keyid, passphrase_file):
+"""Get signer object for the specified backend"""
+return get_signer_class(backend)(keyid. passphrase_file)
-- 
2.6.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/3] package signing: do sanity checking in an event handler

[Markus Lehtonen]

This way, one does not get a ton of identical error messages. But, only
one error message before all the recipes are parsed.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/classes/sign_package_feed.bbclass | 26 --
 meta/classes/sign_rpm.bbclass  | 26 --
 2 files changed, 32 insertions(+), 20 deletions(-)

diff --git a/meta/classes/sign_package_feed.bbclass 
b/meta/classes/sign_package_feed.bbclass
index 3f6ff2d..5170562 100644
--- a/meta/classes/sign_package_feed.bbclass
+++ b/meta/classes/sign_package_feed.bbclass
@@ -22,18 +22,24 @@ PACKAGE_FEED_SIGN = '1'
 PACKAGE_FEED_GPG_BACKEND ?= 'local'
 
 
-python () {
+addhandler sign_package_feed_eventhandler
+sign_package_feed_eventhandler[eventmask] = "bb.event.SanityCheck"
+python sign_package_feed_eventhandler() {
 # Check sanity of config
-from oe.gpg_sign import get_signer_class
-signer = get_signer_class(d.getVar('PACKAGE_FEED_GPG_BACKEND', True))
-err_msg = signer.check_sanity(d,
-  d.getVar('PACKAGE_FEED_GPG_NAME', True),
-  d.getVar('PACKAGE_FEED_GPG_PASSPHRASE_FILE', 
True))
-if err_msg:
-raise_sanity_error(err_msg %{'keyid': 'PACKAGE_FEED_GPG_NAME',
- 'passphrase_file': 
'PACKAGE_FEED_GPG_PASSPHRASE_FILE'},
-   d)
+if bb.event.getName(e) == "SanityCheck":
+from oe.gpg_sign import get_signer_class
+d = e.data
+signer = get_signer_class(d.getVar('PACKAGE_FEED_GPG_BACKEND', True))
+err_msg = signer.check_sanity(d,
+  d.getVar('PACKAGE_FEED_GPG_NAME', True),
+  
d.getVar('PACKAGE_FEED_GPG_PASSPHRASE_FILE', True))
+if err_msg:
+raise_sanity_error(err_msg %{'keyid': 'PACKAGE_FEED_GPG_NAME',
+ 'passphrase_file': 
'PACKAGE_FEED_GPG_PASSPHRASE_FILE'},
+   d)
+}
 
+python () {
 # Set expected location of the public key
 d.setVar('PACKAGE_FEED_GPG_PUBKEY',
  os.path.join(d.getVar('STAGING_ETCDIR_NATIVE', False),
diff --git a/meta/classes/sign_rpm.bbclass b/meta/classes/sign_rpm.bbclass
index 79dc517..47a8378 100644
--- a/meta/classes/sign_rpm.bbclass
+++ b/meta/classes/sign_rpm.bbclass
@@ -21,18 +21,24 @@ RPM_SIGN_PACKAGES='1'
 RPM_GPG_BACKEND ?= 'local'
 
 
-python () {
+addhandler sign_rpm_eventhandler
+sign_rpm_eventhandler[eventmask] = "bb.event.SanityCheck"
+python sign_rpm_eventhandler() {
 # Check sanity of config
-from oe.gpg_sign import get_signer_class
-signer = get_signer_class(d.getVar('RPM_GPG_BACKEND', True))
-err_msg = signer.check_sanity(d,
-  d.getVar('RPM_GPG_NAME', True),
-  d.getVar('RPM_GPG_PASSPHRASE_FILE', True))
-if err_msg:
-raise_sanity_error(err_msg %{'keyid': 'RPM_GPG_NAME',
- 'passphrase_file': 
'RPM_GPG_PASSPHRASE_FILE'},
-   d)
+if bb.event.getName(e) == "SanityCheck":
+from oe.gpg_sign import get_signer_class
+d = e.data
+signer = get_signer_class(d.getVar('RPM_GPG_BACKEND', True))
+err_msg = signer.check_sanity(d,
+  d.getVar('RPM_GPG_NAME', True),
+  d.getVar('RPM_GPG_PASSPHRASE_FILE', 
True))
+if err_msg:
+raise_sanity_error(err_msg %{'keyid': 'RPM_GPG_NAME',
+ 'passphrase_file': 
'RPM_GPG_PASSPHRASE_FILE'},
+   d)
+}
 
+python () {
 # Set the expected location of the public key
 d.setVar('RPM_GPG_PUBKEY', os.path.join(d.getVar('STAGING_ETCDIR_NATIVE', 
False),
 'RPM-GPG-PUBKEY'))
-- 
2.6.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] ncurses: use closing curly brackets

[Markus Lehtonen]

This patch fixes the usage of curly brackets inside python expression in
ncurses recipe. This patch should be applied together with
http://lists.openembedded.org/pipermail/bitbake-devel/2016-February/007007.html
which changes python expression parsing in bitbake.

No similar construct was found in any other recipe in git.openembedded.org
repositories or the layer repositories in git.yoctoproject.org (master
branches).


Markus Lehtonen (1):
  ncurses: use closing curly brackets in FILES_${PN}-tools variable

 meta/recipes-core/ncurses/ncurses.inc | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

-- 
2.6.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] ncurses: use closing curly brackets in FILES_${PN}-tools variable

[Markus Lehtonen]

This patch removes a workaround (needed for bitbake python parser) where
closing curly brackets were replaced by ascii code '\x7d'.

This commit requires a bitbake version with the
"data_smart: simple bracket matching inside python expressions" patch
applied.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/recipes-core/ncurses/ncurses.inc | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-core/ncurses/ncurses.inc 
b/meta/recipes-core/ncurses/ncurses.inc
index fbe74d5..35b8b94 100644
--- a/meta/recipes-core/ncurses/ncurses.inc
+++ b/meta/recipes-core/ncurses/ncurses.inc
@@ -284,8 +284,8 @@ FILES_${PN}-tools = "\
   ${bindir}/infotocap \
   ${bindir}/captoinfo \
   ${bindir}/infocmp \
-  ${bindir}/clear${@['', '.${BPN\x7d']['${CLASSOVERRIDE}' == 'class-target']} \
-  ${bindir}/reset${@['', '.${BPN\x7d']['${CLASSOVERRIDE}' == 'class-target']} \
+  ${bindir}/clear${@['', '.${BPN}']['${CLASSOVERRIDE}' == 'class-target']} \
+  ${bindir}/reset${@['', '.${BPN}']['${CLASSOVERRIDE}' == 'class-target']} \
   ${bindir}/tack \
   ${bindir}/tabs \
 "
-- 
2.6.2

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/3] externalsrc: use shared CONFIGURESTAMPFILE if B=S

[Markus Lehtonen]

External source tree is used as the builddir even for different
architectures in case ${B}=${S}. This may cause problems if MACHINE is
changed as do_configure is not being re-run (which would run make
clean). This patches changes externalsrc to use a common (per-recipe)
CONFIGURESTAMPFILE under 'work-shared' if ${B}=${S}. In addition,
do_configure will depend on changes of this stamp file. As a result,
do_configure is re-run and the build dir is cleaned correctly if a
rebuild for different MACHINE is done.

[YOCTO #8950]

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/classes/externalsrc.bbclass | 11 +++
 1 file changed, 11 insertions(+)

diff --git a/meta/classes/externalsrc.bbclass b/meta/classes/externalsrc.bbclass
index f7ed66d..be0fe55 100644
--- a/meta/classes/externalsrc.bbclass
+++ b/meta/classes/externalsrc.bbclass
@@ -85,6 +85,17 @@ python () {
 
 # Ensure compilation happens every time
 d.setVarFlag('do_compile', 'nostamp', '1')
+
+# If B=S the same builddir is used even for different architectures.
+# Thus, use a shared CONFIGURESTAMPFILE so that change of do_configure
+# task hash is correctly detected if e.g. MACHINE changes. In addition,
+# do_configure needs to depend on the stamp file so that the task is
+# re-run when the stamp was changed since the last run on this
+# architecture.
+if d.getVar('S', True) == d.getVar('B', True):
+configstamp = 
'${TMPDIR}/work-shared/${PN}/${EXTENDPE}${PV}-${PR}/configure.sstate'
+d.setVar('CONFIGURESTAMPFILE', configstamp)
+d.setVarFlag('do_configure', 'file-checksums', configstamp + 
':True')
 }
 
 python externalsrc_compile_prefunc() {
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/3] Make sure that the directory for CONFIGURESTAMPFILE exists

[Markus Lehtonen]

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/classes/autotools.bbclass | 1 +
 meta/classes/base.bbclass  | 1 +
 2 files changed, 2 insertions(+)

diff --git a/meta/classes/autotools.bbclass b/meta/classes/autotools.bbclass
index 7bf510b..7f60c2a 100644
--- a/meta/classes/autotools.bbclass
+++ b/meta/classes/autotools.bbclass
@@ -122,6 +122,7 @@ autotools_preconfigure() {
 
 autotools_postconfigure(){
if [ -n "${CONFIGURESTAMPFILE}" ]; then
+   mkdir -p `dirname ${CONFIGURESTAMPFILE}`
echo ${BB_TASKHASH} > ${CONFIGURESTAMPFILE}
fi
 }
diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index 5fc9271..0eb1870 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -292,6 +292,7 @@ base_do_configure() {
fi
fi
if [ -n "${CONFIGURESTAMPFILE}" ]; then
+   mkdir -p `dirname ${CONFIGURESTAMPFILE}`
echo ${BB_TASKHASH} > ${CONFIGURESTAMPFILE}
fi
 }
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 0/3] correctly clean ${B] when externalsrc and {B}=${S}

[Markus Lehtonen]

This patchset aims in fixing a build failure when B=S and externalsrc is used.
The build failure was caused by build artefacts from a previous build not being
cleaned up when MACHINE was changed.

[YOCTO #8950]

The following changes since commit 2a6e061712cfe9cb4738806a0c351a64e0d30144:

  cmake: update to 3.4.2 (2016-01-26 22:48:57 +)

are available in the git repository at:

  contrib-git marquiz/externalsrc

for you to fetch changes up to 58bb2b3bd3195af2715e8dd594326dfa73fc72e9:

  externalsrc: use shared CONFIGURESTAMPFILE if B=S (2016-01-29 16:53:19 +0200)


Markus Lehtonen (3):
  autotools.bbclass: use oe_runmake instead of ${MAKE}
  Make sure that the directory for CONFIGURESTAMPFILE exists
  externalsrc: use shared CONFIGURESTAMPFILE if B=S

 meta/classes/autotools.bbclass   |  4 ++--
 meta/classes/base.bbclass|  1 +
 meta/classes/externalsrc.bbclass | 11 +++
 3 files changed, 14 insertions(+), 2 deletions(-)

-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/3] autotools.bbclass: use oe_runmake instead of ${MAKE}

[Markus Lehtonen]

Use oe_runmake like in base.bbclass so that EXTRA_OEMAKE will be
respected.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/classes/autotools.bbclass | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/meta/classes/autotools.bbclass b/meta/classes/autotools.bbclass
index 1400b44..7bf510b 100644
--- a/meta/classes/autotools.bbclass
+++ b/meta/classes/autotools.bbclass
@@ -112,8 +112,7 @@ autotools_preconfigure() {
# regenerate them even if CFLAGS/LDFLAGS are 
different
cd ${S}
if [ "${CLEANBROKEN}" != "1" -a \( -e Makefile 
-o -e makefile -o -e GNUmakefile \) ]; then
-   echo "Running \"${MAKE} clean\" in ${S}"
-   ${MAKE} clean
+   oe_runmake clean
fi
find ${S} -ignore_readdir_race -name \*.la 
-delete
fi
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] autotools.bbclass: use oe_runmake instead of ${MAKE}

[Markus Lehtonen]

Use oe_runmake like in base.bbclass so that EXTRA_OEMAKE will be
respected.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/classes/autotools.bbclass | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/meta/classes/autotools.bbclass b/meta/classes/autotools.bbclass
index 1400b44..7bf510b 100644
--- a/meta/classes/autotools.bbclass
+++ b/meta/classes/autotools.bbclass
@@ -112,8 +112,7 @@ autotools_preconfigure() {
# regenerate them even if CFLAGS/LDFLAGS are 
different
cd ${S}
if [ "${CLEANBROKEN}" != "1" -a \( -e Makefile 
-o -e makefile -o -e GNUmakefile \) ]; then
-   echo "Running \"${MAKE} clean\" in ${S}"
-   ${MAKE} clean
+   oe_runmake clean
fi
find ${S} -ignore_readdir_race -name \*.la 
-delete
fi
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] buildhistory: fix the check for existence of a git repo

[Markus Lehtonen]

Previously, in order to determine the existence of an already
initialized Git repository we checked if a directory named '.git' was
present in the buildhistory dir. However, e.g. in the case of git
submodules '.git' may also be a regular file referencing some other
location which was causing unwanted behavior. This patch changes
buildhistory.bbclass to check for any file named '.git' which fixes
these problems.

[YOCTO #8911]

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/classes/buildhistory.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/buildhistory.bbclass 
b/meta/classes/buildhistory.bbclass
index 9f17442..3c4647a 100644
--- a/meta/classes/buildhistory.bbclass
+++ b/meta/classes/buildhistory.bbclass
@@ -684,7 +684,7 @@ END
 
( cd ${BUILDHISTORY_DIR}/
# Initialise the repo if necessary
-   if [ ! -d .git ] ; then
+   if [ ! -e .git ] ; then
git init -q
else
git tag -f build-minus-3 build-minus-2 > /dev/null 2>&1 
|| true
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/2] externalsrc.bbclas: remove nostamp from do_configure

[Markus Lehtonen]

Be a bit more intelligent than mindlessly re-compiling every time.
Instead of using 'nostamp' flag for do_compile add the whole source tree
as 'file-checksums' flag. This way, do_compile is only re-run if
something in the source tree content changes. Hidden files and
directories in the source tree root are ignored by the glob currently
used. This has the advantage of automatically ignoring .git directory,
for example.

This does not work perfectly, though, as many packages are built under
${S} which effectively changes the source tree causing some unwanted
re-compilations.  However, if do_compile of the recipe does not produce
new/different artefacts on every run (as commonly is and should be the
case) the re-compilation loop stops. Thus, you should usually see only
one re-compilation (if any) after which the source tree is "stabilized"
and no more re-compilations happen.

During the first bitbake run preparing of the task runqueue may take
much longer because all the files in the source tree are hashed.
Subsequent builds are not significantly slower because (most) file
hashes are found from the cache.

[YOCTO #8853]

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/classes/externalsrc.bbclass | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/meta/classes/externalsrc.bbclass b/meta/classes/externalsrc.bbclass
index f7ed66d..fe4963d 100644
--- a/meta/classes/externalsrc.bbclass
+++ b/meta/classes/externalsrc.bbclass
@@ -83,8 +83,7 @@ python () {
 
 d.prependVarFlag('do_compile', 'prefuncs', 
"externalsrc_compile_prefunc ")
 
-# Ensure compilation happens every time
-d.setVarFlag('do_compile', 'nostamp', '1')
+d.setVarFlag('do_compile', 'file-checksums', externalsrc + '/*:True')
 }
 
 python externalsrc_compile_prefunc() {
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 0/2] Improve externalsrc task dependency tracking

[Markus Lehtonen]

This patchset improves task hashing of do_compile when externalsrc is enabled.
Previously, it was simply a 'nostamp' task that was always being (re-)run. This
patchset changes externalsrc to utilize the file-checksum dependency feature of
tasks to really track changes in the source tree, and thus, aims in preventing
unneeded (re-)compilations.

[YOCTO #8853]


The following changes since commit fc4209baa098caebf9c4cb75f9a6f2e85f4c:

  Revert "xz: Allow to work with ASSUME_PROVIDED xz-native" (2016-01-25 
10:08:25 +)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib marquiz/devtool/fixes


Markus Lehtonen (2):
  externalsrc.bbclas: remove nostamp from do_configure
  devtool: create-workspace: define separate cache for task file
checksums

 meta/classes/externalsrc.bbclass | 3 +--
 scripts/devtool  | 1 +
 2 files changed, 2 insertions(+), 2 deletions(-)

-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/2] devtool: create-workspace: define separate cache for task file checksums

[Markus Lehtonen]

Define BB_HASH_CHECKSUM_CACHE_FILE variable in the workspace layer
config so that a specific cache is used for storing the task file
dependency checksums. The file checksum cache can grow quite large after
hashing of all file in externalsrc trees was enabled.  This patch
prevents polluting/growing the fetcher local file checksum cache. Also,
this new devtool-specific cache is not used after the workspace layer is
disabled.

[YOCTO #8853]

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 scripts/devtool | 1 +
 1 file changed, 1 insertion(+)

diff --git a/scripts/devtool b/scripts/devtool
index 2d57da0..e44b87e9 100755
--- a/scripts/devtool
+++ b/scripts/devtool
@@ -152,6 +152,7 @@ def _create_workspace(workspacedir, config, basepath):
 f.write('BBFILE_PATTERN_workspacelayer = "^$' + '{LAYERDIR}/"\n')
 f.write('BBFILE_PATTERN_IGNORE_EMPTY_workspacelayer = "1"\n')
 f.write('BBFILE_PRIORITY_workspacelayer = "99"\n')
+f.write('BB_HASH_CHECKSUM_CACHE_FILE = 
"filedep_checksum_cache.dat"\n')
 # Add a README file
 with open(os.path.join(workspacedir, 'README'), 'w') as f:
 f.write('This layer was created by the OpenEmbedded devtool 
utility in order to\n')
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2] New lib module for handling GPG signing

[Markus Lehtonen]

Add a new Python module (oe.gpg_sign) for handling GPG signing
operations, i.e. currently package and package feed signing. The purpose
is to be able to more easily support various signing backends and to be
able to centralise signing functionality into one place (e.g.  package
signing and sstate signing). Currently, only local signing with gpg is
implemented.

[YOCTO #8755]

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/classes/sign_package_feed.bbclass |  6 +++
 meta/classes/sign_rpm.bbclass  | 47 +
 meta/lib/oe/gpg_sign.py| 76 ++
 meta/lib/oe/package_manager.py | 31 +-
 meta/recipes-core/meta/signing-keys.bb | 26 ++--
 5 files changed, 116 insertions(+), 70 deletions(-)
 create mode 100644 meta/lib/oe/gpg_sign.py

diff --git a/meta/classes/sign_package_feed.bbclass 
b/meta/classes/sign_package_feed.bbclass
index d89bc0b..d5df8af 100644
--- a/meta/classes/sign_package_feed.bbclass
+++ b/meta/classes/sign_package_feed.bbclass
@@ -6,6 +6,10 @@
 #   Path to a file containing the passphrase of the signing key.
 # PACKAGE_FEED_GPG_NAME
 #   Name of the key to sign with. May be key id or key name.
+# PACKAGE_FEED_GPG_BACKEND
+#   Optional variable for specifying the backend to use for signing.
+#   Currently the only available option is 'local', i.e. local signing
+#   on the build host.
 # GPG_BIN
 #   Optional variable for specifying the gpg binary/wrapper to use for
 #   signing.
@@ -15,6 +19,8 @@
 inherit sanity
 
 PACKAGE_FEED_SIGN = '1'
+PACKAGE_FEED_GPG_BACKEND ?= 'local'
+
 
 python () {
 # Check sanity of configuration
diff --git a/meta/classes/sign_rpm.bbclass b/meta/classes/sign_rpm.bbclass
index 7906b64..8bcabee 100644
--- a/meta/classes/sign_rpm.bbclass
+++ b/meta/classes/sign_rpm.bbclass
@@ -5,6 +5,10 @@
 #   Path to a file containing the passphrase of the signing key.
 # RPM_GPG_NAME
 #   Name of the key to sign with. May be key id or key name.
+# RPM_GPG_BACKEND
+#   Optional variable for specifying the backend to use for signing.
+#   Currently the only available option is 'local', i.e. local signing
+#   on the build host.
 # GPG_BIN
 #   Optional variable for specifying the gpg binary/wrapper to use for
 #   signing.
@@ -14,6 +18,7 @@
 inherit sanity
 
 RPM_SIGN_PACKAGES='1'
+RPM_GPG_BACKEND ?= 'local'
 
 
 python () {
@@ -27,47 +32,17 @@ python () {
 'RPM-GPG-PUBKEY'))
 }
 
-
-def rpmsign_wrapper(d, files, passphrase, gpg_name=None):
-import pexpect
-
-# Find the correct rpm binary
-rpm_bin_path = d.getVar('STAGING_BINDIR_NATIVE', True) + '/rpm'
-cmd = rpm_bin_path + " --addsign --define '_gpg_name %s' " % gpg_name
-if d.getVar('GPG_BIN', True):
-cmd += "--define '%%__gpg %s' " % d.getVar('GPG_BIN', True)
-if d.getVar('GPG_PATH', True):
-cmd += "--define '_gpg_path %s' " % d.getVar('GPG_PATH', True)
-cmd += ' '.join(files)
-
-# Need to use pexpect for feeding the passphrase
-proc = pexpect.spawn(cmd)
-try:
-proc.expect_exact('Enter pass phrase:', timeout=15)
-proc.sendline(passphrase)
-proc.expect(pexpect.EOF, timeout=900)
-proc.close()
-except pexpect.TIMEOUT as err:
-bb.warn('rpmsign timeout: %s' % err)
-proc.terminate()
-else:
-if os.WEXITSTATUS(proc.status) or not os.WIFEXITED(proc.status):
-bb.warn('rpmsign failed: %s' % proc.before.strip())
-return proc.exitstatus
-
-
 python sign_rpm () {
 import glob
+from oe.gpg_sign import get_signer
 
-with open(d.getVar("RPM_GPG_PASSPHRASE_FILE", True)) as fobj:
-rpm_gpg_passphrase = fobj.readlines()[0].rstrip('\n')
-
-rpm_gpg_name = (d.getVar("RPM_GPG_NAME", True) or "")
-
+signer = get_signer(d,
+d.getVar('RPM_GPG_BACKEND', True),
+d.getVar('RPM_GPG_NAME', True),
+d.getVar('RPM_GPG_PASSPHRASE_FILE', True))
 rpms = glob.glob(d.getVar('RPM_PKGWRITEDIR', True) + '/*')
 
-if rpmsign_wrapper(d, rpms, rpm_gpg_passphrase, rpm_gpg_name) != 0:
-raise bb.build.FuncFailed("RPM signing failed")
+signer.sign_rpms(rpms)
 }
 
 do_package_index[depends] += "signing-keys:do_export_public_keys"
diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
new file mode 100644
index 000..55abad8
--- /dev/null
+++ b/meta/lib/oe/gpg_sign.py
@@ -0,0 +1,76 @@
+"""Helper module for GPG signing"""
+import os
+
+import bb
+import oe.utils
+
+class LocalSigner(object):
+"""Class for handling local (on the build host) signing"""
+def __init__(self, d, key

[OE-core] [PATCH v2] New lib module for signing

[Markus Lehtonen]

Re-sending this patch as a standalone refactoring / improvement. Getting this
merged would make it easier work separately on different parts of signing, e.g.
sstate signing, remote rpm signing, ipk and dpkg signing.

No code changes since the previous version of the patch. Only commit message is
slightly modified.

Markus Lehtonen (1):
  New lib module for handling GPG signing

 meta/classes/sign_package_feed.bbclass |  6 +++
 meta/classes/sign_rpm.bbclass  | 47 +
 meta/lib/oe/gpg_sign.py| 76 ++
 meta/lib/oe/package_manager.py | 31 +-
 meta/recipes-core/meta/signing-keys.bb | 26 ++--
 5 files changed, 116 insertions(+), 70 deletions(-)
 create mode 100644 meta/lib/oe/gpg_sign.py

-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 3/3] oe.gpg_sign: support obs-signd

[Markus Lehtonen]

Hi Mark,



(CC'd the mailing list which was accidentally dropped from my previous email)

On 21/01/16 17:21, "Mark Hatle" <mark.ha...@windriver.com> wrote:

>On 1/21/16 5:20 AM, Markus Lehtonen wrote:
>> On Wed, 2016-01-13 at 12:28 +0200, Markus Lehtonen wrote:
>>> On Tue, 2016-01-12 at 18:24 +0200, Markus Lehtonen wrote:
>>>> Hi Mark,
>>>>
>>>> Thank you for your review! Comments below.
>>>>
>>>> On Mon, 2016-01-11 at 10:33 -0600, Mark Hatle wrote:
>> 
>> [...SNIP...]
>> 
>>>>>
>>>>> Why are you removing existing signatures?  I believe for many cases this 
>>>>> is
>>>>> actually incorrect.
>>>>>
>>>>> RPM (5) has the ability to have an endless number of signatures within a 
>>>>> given
>>>>> package.  The package SHOULD included the internal non-repudiable 
>>>>> signature...
>>>>>
>>>>> (to refresh memory) all RPM 5 packages include an internal non-repudiable
>>>>> signature.  Think of this as an extended md5sum, sha256sum, etc.  It 
>>>>> doesn't
>>>>> change that a package is 'authentic' in any way (often the purpose of 
>>>>> signatures
>>>>> like what this code is doing), but instead keeps a high reliability way 
>>>>> to sign
>>>>> and verify the package is signed properly.
>>>>>
>>>>> This is used for validation if the system doing the install does not have 
>>>>> the
>>>>> public key that the package was signed with.
>>>>>
>>>>> ... as well as one or more repudiable signatures that can be used to 
>>>>> verify that
>>>>> it's "authentic" in some way.  A system could very easily have OSV, OEM, 
>>>>> and ISV
>>>>> keys install on them.  You can program RPM in such a way that it will 
>>>>> refused to
>>>>> install packages with unknown authentication keys or the non-repudiable 
>>>>> key as well.
>>>>>
>>>>> So, I believe running delsign is wrong.  If the obs-signd can't handle 
>>>>> ADDING
>>>>> signatures to packages, then I'd say it is broken and should be fixed in 
>>>>> some
>>>>> way -- or at least the signature deletion code should be optional.
>>>>
>>>> Yes, unfortunately this is currently the limitation of obs-signd. It
>>>> refuses to sign if there are signatures present in the rpm package.
>>>> Using --delsign is "unfortunate" consequence of this and that should've
>>>> probably been described in a comment. Making signature deletion a
>>>> configurable setting is hopefully a decent resolution for now. I will
>>>> send a new version of the patchset later.
>>>
>>> Backing up a bit here. I did some quick testing and it seems that RPM5
>>> does not support multiple signatures (anymore?). Doing --addsign seems
>>> to overwrite the existing signatures similarly to --resign. Support for
>>> multiple signatures were removed from RPM4 years ago.
>>>
>>> In this light, doing --delsign should be ok. What do you think?
>> 
>> Hi Mark. Do you have any comments to the above? I'd like to get this
>> patchset out of my hands :)
>
>RPM5 does have multiple signatures, but only allows one of each of the three
>types to be installed.  The delsign shouldn't be used as it might remove the
>wrong signature.

AFAIU, rpm only allows one signature so be present. The file format allows 
that, but, the rpm tool does not (anymore). For example, rpm --addsign will 
remove an existing DSA signature when adding an RSA signature. The SHA1 / MD5 
digests are not touched by --delsign.


>(Three types are DSA/RSA, ECDSA, and simple SHA256 or similar.)

I didn't know that rpm(5?) supports ECDSA signatures.


>But making the --delsign optional I think is the best approach.  (It would be
>better to move it to the obs-sign script itself -- but I can live with doing it
>on the OE side since people are trying to use their owns systems.)

I still believe that making it optional is just worthless and complicates 
things because doing rpm --addsign has exactly the same effect.


>The alternative would be to not call the script 'obs-sign', but instead call an
>arbitrarily named (and defined in a bitbake variable) script.. Then THAT script
>can do the del and call the obs-sign.)

Hmm, I probably don't like this idea that much. This user-written script would 
need to be a bit more complex as a it needs to support multiple operations 
(signrpm, detach sign, export pubkey). Of course, I could write a default 
script and put it under scripts/ but somehow feels more complex than needed.


Thanks,
  Markus


-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/2] build-perf-test.sh: add eSDK testing

[Markus Lehtonen]

Add simple initial eSDK test. Currently, only download size and
installation time of eSDK is measured. The eSDK to be tested is
generated from the same image that the other tests are run for. This
patch will add two new fields to the global results log and that needs
to be taken into account when examining the results.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 scripts/contrib/build-perf-test.sh | 24 
 1 file changed, 24 insertions(+)

diff --git a/scripts/contrib/build-perf-test.sh 
b/scripts/contrib/build-perf-test.sh
index 105b54b..f1f44d3 100755
--- a/scripts/contrib/build-perf-test.sh
+++ b/scripts/contrib/build-perf-test.sh
@@ -353,6 +353,29 @@ test3 () {
 bbtime -p
 }
 
+#
+# Test 4 - eSDK
+# Measure: eSDK size and installation time
+test4 () {
+log "Running Test 4: eSDK size and installation time"
+bbnotime $IMAGE -c do_populate_sdk_ext
+
+esdk_installer=(tmp/deploy/sdk/*-toolchain-ext-*.sh)
+
+if [ ${#esdk_installer[*]} -eq 1 ]; then
+s=$((`stat -c %s "$esdk_installer"` / 1024))
+SIZES[(( size_count++ ))]="$s"
+log "Download SIZE of eSDK is: $s kB"
+
+do_sync
+time_cmd "$esdk_installer" -y -d "tmp/esdk-deploy"
+else
+log "ERROR: other than one sdk found (${esdk_installer[*]}), reporting 
size and time as 0."
+SIZES[(( size_count++ ))]="0"
+TIMES[(( time_count++ ))]="0"
+fi
+
+}
 
 
 # RUN!
@@ -362,6 +385,7 @@ test1_p2
 test1_p3
 test2
 test3
+test4
 
 # if we got til here write to global results
 write_results
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/2] build-perf-test.sh: more generic timing function

[Markus Lehtonen]

Make it possible to time also other than bitbake commands. The name of
the log file is changed from bitbake.log to commands.log.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 scripts/contrib/build-perf-test.sh | 17 ++---
 1 file changed, 10 insertions(+), 7 deletions(-)

diff --git a/scripts/contrib/build-perf-test.sh 
b/scripts/contrib/build-perf-test.sh
index cdd7885..105b54b 100755
--- a/scripts/contrib/build-perf-test.sh
+++ b/scripts/contrib/build-perf-test.sh
@@ -128,7 +128,7 @@ rev=$(git rev-parse --short HEAD)  || exit 1
 OUTDIR="$clonedir/build-perf-test/results-$rev-`date "+%Y%m%d%H%M%S"`"
 BUILDDIR="$OUTDIR/build"
 resultsfile="$OUTDIR/results.log"
-bboutput="$OUTDIR/bitbake.log"
+cmdoutput="$OUTDIR/commands.log"
 myoutput="$OUTDIR/output.log"
 globalres="$clonedir/build-perf-test/globalres.log"
 
@@ -180,14 +180,13 @@ time_count=0
 declare -a SIZES
 size_count=0
 
-bbtime () {
-local arg="$@"
-log "   Timing: bitbake ${arg}"
+time_cmd () {
+log "   Timing: $*"
 
 if [ $verbose -eq 0 ]; then 
-/usr/bin/time -v -o $resultsfile bitbake ${arg} >> $bboutput
+/usr/bin/time -v -o $resultsfile "$@" >> $cmdoutput
 else
-/usr/bin/time -v -o $resultsfile bitbake ${arg}
+/usr/bin/time -v -o $resultsfile "$@"
 fi
 ret=$?
 if [ $ret -eq 0 ]; then
@@ -206,12 +205,16 @@ bbtime () {
 log "More stats can be found in ${resultsfile}.${i}"
 }
 
+bbtime () {
+time_cmd bitbake "$@"
+}
+
 #we don't time bitbake here
 bbnotime () {
 local arg="$@"
 log "   Running: bitbake ${arg}"
 if [ $verbose -eq 0 ]; then
-bitbake ${arg} >> $bboutput
+bitbake ${arg} >> $cmdoutput
 else
 bitbake ${arg}
 fi
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] oeqa/selftest/signing: use temporary rpmdb

[Markus Lehtonen]

Use temporary rpmdb when importing gpg public key and checking rpm
signature. This patch should fix a problem where test_signing_packages()
sometimes fails with
"pmdb: BDB1540 configured environment flags incompatible with existing
 environment"

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/lib/oeqa/selftest/signing.py | 12 ++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oeqa/selftest/signing.py 
b/meta/lib/oeqa/selftest/signing.py
index c33662b..c402e37 100644
--- a/meta/lib/oeqa/selftest/signing.py
+++ b/meta/lib/oeqa/selftest/signing.py
@@ -3,6 +3,8 @@ from oeqa.utils.commands import runCmd, bitbake, get_bb_var
 import os
 import glob
 import re
+import shutil
+import tempfile
 from oeqa.utils.decorators import testcase
 
 
@@ -69,11 +71,17 @@ class Signing(oeSelfTest):
 
 pkg_deploy = os.path.join(deploy_dir_rpm, package_arch, '.'.join((pf, 
package_arch, 'rpm')))
 
-runCmd('%s/rpm --import %s%s' % (staging_bindir_native, self.gpg_dir, 
self.pub_key_name))
+# Use a temporary rpmdb
+rpmdb = tempfile.mkdtemp(prefix='oeqa-rpmdb')
 
-ret = runCmd('%s/rpm --checksig %s' % (staging_bindir_native, 
pkg_deploy))
+runCmd('%s/rpm --define "_dbpath %s" --import %s%s' %
+   (staging_bindir_native, rpmdb, self.gpg_dir, self.pub_key_name))
+
+ret = runCmd('%s/rpm --define "_dbpath %s" --checksig %s' %
+ (staging_bindir_native, rpmdb, pkg_deploy))
 # tmp/deploy/rpm/i586/ed-1.9-r0.i586.rpm: rsa sha1 md5 OK
 self.assertIn('rsa sha1 md5 OK', ret.output, 'Package signed 
incorrectly.')
+shutil.rmtree(rpmdb)
 
 @testcase(1382)
 def test_signing_sstate_archive(self):
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 3/3] oe.gpg_sign: support obs-signd

[Markus Lehtonen]

Hi,

On Tue, 2016-01-12 at 18:24 +0200, Markus Lehtonen wrote:
> Hi Mark,
> 
> Thank you for your review! Comments below.
> 
> On Mon, 2016-01-11 at 10:33 -0600, Mark Hatle wrote:
> > On 1/11/16 10:13 AM, Markus Lehtonen wrote:
> > > Implement support for remote signing using obs-signd. It is now possible
> > > to sign both RPM packages and package feeds with this method. The user
> > > just needs to set RPM_GPG_BACKEND and/or PACKAGE_FEED_GPG_BACKEND
> > > variables to 'obssign' in the bitbake config. Of course, in addition,
> > > one needs to setup the signing server and the configure the 'sign'
> > > client command on the build host. The *_PASSPHRASE_FILE settings are not
> > > used when the obssign backend is enabled.
> > > 
> > > [YOCTO #8755]
> > > 
> > > Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
> > > ---
> > >  meta/classes/sign_package_feed.bbclass |  5 +++-
> > >  meta/classes/sign_rpm.bbclass  |  5 +++-
> > >  meta/lib/oe/gpg_sign.py| 48 
> > > ++
> > >  3 files changed, 56 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/meta/classes/sign_package_feed.bbclass 
> > > b/meta/classes/sign_package_feed.bbclass
> > > index d5df8af..953fa85 100644
> > > --- a/meta/classes/sign_package_feed.bbclass
> > > +++ b/meta/classes/sign_package_feed.bbclass
> > > @@ -24,7 +24,10 @@ PACKAGE_FEED_GPG_BACKEND ?= 'local'
> > >  
> > >  python () {
> > >  # Check sanity of configuration
> > > -for var in ('PACKAGE_FEED_GPG_NAME', 
> > > 'PACKAGE_FEED_GPG_PASSPHRASE_FILE'):
> > > +required = ['PACKAGE_FEED_GPG_NAME']
> > > +if d.getVar('PACKAGE_FEED_GPG_BACKEND', True) != 'obssign':
> > > +required.append('PACKAGE_FEED_GPG_PASSPHRASE_FILE')
> > > +for var in required:
> > >  if not d.getVar(var, True):
> > >  raise_sanity_error("You need to define %s in the config" % 
> > > var, d)
> > >  
> > > diff --git a/meta/classes/sign_rpm.bbclass b/meta/classes/sign_rpm.bbclass
> > > index 8bcabee..8be1c35 100644
> > > --- a/meta/classes/sign_rpm.bbclass
> > > +++ b/meta/classes/sign_rpm.bbclass
> > > @@ -23,7 +23,10 @@ RPM_GPG_BACKEND ?= 'local'
> > >  
> > >  python () {
> > >  # Check configuration
> > > -for var in ('RPM_GPG_NAME', 'RPM_GPG_PASSPHRASE_FILE'):
> > > +required = ['RPM_GPG_NAME']
> > > +if d.getVar('RPM_GPG_BACKEND', True) != 'obssign':
> > > +required.append('RPM_GPG_PASSPHRASE_FILE')
> > > +for var in required:
> > >  if not d.getVar(var, True):
> > >  raise_sanity_error("You need to define %s in the config" % 
> > > var, d)
> > >  
> > > diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
> > > index 55abad8..d8ab816 100644
> > > --- a/meta/lib/oe/gpg_sign.py
> > > +++ b/meta/lib/oe/gpg_sign.py
> > > @@ -66,11 +66,59 @@ class LocalSigner(object):
> > >(input_file, output))
> > >  
> > >  
> > > +class ObsSigner(object):
> > > +"""Class for handling signing with obs-signd"""
> > > +def __init__(self, keyid):
> > > +self.keyid = keyid
> > > +self.rpm_bin = bb.utils.which(os.getenv('PATH'), "rpm")
> > > +
> > > +def export_pubkey(self, output_file):
> > > +"""Export GPG public key to a file"""
> > > +cmd = "sign -u '%s' -p" % self.keyid
> > > +status, output = oe.utils.getstatusoutput(cmd)
> > > +if status:
> > > +raise bb.build.FuncFailed('Failed to export gpg public key 
> > > (%s): %s' %
> > > +  (self.keyid, output))
> > > +with open(output_file, 'w') as fobj:
> > > +fobj.write(output)
> > > +fobj.write('\n')
> > > +
> > > +def sign_rpms(self, files):
> > > +"""Sign RPM files"""
> > > +import pexpect
> > > +
> > > +# Remove existing signatures
> > > +cmd = "%s --delsign %s" % (self.rpm_bin, ' '.join(files))
> > 
> > Why are you removing existing signatures?  

Re: [OE-core] [PATCH 3/3] oe.gpg_sign: support obs-signd

[Markus Lehtonen]

Hi Mark,

Thank you for your review! Comments below.

On Mon, 2016-01-11 at 10:33 -0600, Mark Hatle wrote:
> On 1/11/16 10:13 AM, Markus Lehtonen wrote:
> > Implement support for remote signing using obs-signd. It is now possible
> > to sign both RPM packages and package feeds with this method. The user
> > just needs to set RPM_GPG_BACKEND and/or PACKAGE_FEED_GPG_BACKEND
> > variables to 'obssign' in the bitbake config. Of course, in addition,
> > one needs to setup the signing server and the configure the 'sign'
> > client command on the build host. The *_PASSPHRASE_FILE settings are not
> > used when the obssign backend is enabled.
> > 
> > [YOCTO #8755]
> > 
> > Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
> > ---
> >  meta/classes/sign_package_feed.bbclass |  5 +++-
> >  meta/classes/sign_rpm.bbclass  |  5 +++-
> >  meta/lib/oe/gpg_sign.py| 48 
> > ++
> >  3 files changed, 56 insertions(+), 2 deletions(-)
> > 
> > diff --git a/meta/classes/sign_package_feed.bbclass 
> > b/meta/classes/sign_package_feed.bbclass
> > index d5df8af..953fa85 100644
> > --- a/meta/classes/sign_package_feed.bbclass
> > +++ b/meta/classes/sign_package_feed.bbclass
> > @@ -24,7 +24,10 @@ PACKAGE_FEED_GPG_BACKEND ?= 'local'
> >  
> >  python () {
> >  # Check sanity of configuration
> > -for var in ('PACKAGE_FEED_GPG_NAME', 
> > 'PACKAGE_FEED_GPG_PASSPHRASE_FILE'):
> > +required = ['PACKAGE_FEED_GPG_NAME']
> > +if d.getVar('PACKAGE_FEED_GPG_BACKEND', True) != 'obssign':
> > +required.append('PACKAGE_FEED_GPG_PASSPHRASE_FILE')
> > +for var in required:
> >  if not d.getVar(var, True):
> >  raise_sanity_error("You need to define %s in the config" % 
> > var, d)
> >  
> > diff --git a/meta/classes/sign_rpm.bbclass b/meta/classes/sign_rpm.bbclass
> > index 8bcabee..8be1c35 100644
> > --- a/meta/classes/sign_rpm.bbclass
> > +++ b/meta/classes/sign_rpm.bbclass
> > @@ -23,7 +23,10 @@ RPM_GPG_BACKEND ?= 'local'
> >  
> >  python () {
> >  # Check configuration
> > -for var in ('RPM_GPG_NAME', 'RPM_GPG_PASSPHRASE_FILE'):
> > +required = ['RPM_GPG_NAME']
> > +if d.getVar('RPM_GPG_BACKEND', True) != 'obssign':
> > +required.append('RPM_GPG_PASSPHRASE_FILE')
> > +for var in required:
> >  if not d.getVar(var, True):
> >  raise_sanity_error("You need to define %s in the config" % 
> > var, d)
> >  
> > diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
> > index 55abad8..d8ab816 100644
> > --- a/meta/lib/oe/gpg_sign.py
> > +++ b/meta/lib/oe/gpg_sign.py
> > @@ -66,11 +66,59 @@ class LocalSigner(object):
> >(input_file, output))
> >  
> >  
> > +class ObsSigner(object):
> > +"""Class for handling signing with obs-signd"""
> > +def __init__(self, keyid):
> > +self.keyid = keyid
> > +self.rpm_bin = bb.utils.which(os.getenv('PATH'), "rpm")
> > +
> > +def export_pubkey(self, output_file):
> > +"""Export GPG public key to a file"""
> > +cmd = "sign -u '%s' -p" % self.keyid
> > +status, output = oe.utils.getstatusoutput(cmd)
> > +if status:
> > +raise bb.build.FuncFailed('Failed to export gpg public key 
> > (%s): %s' %
> > +  (self.keyid, output))
> > +with open(output_file, 'w') as fobj:
> > +fobj.write(output)
> > +fobj.write('\n')
> > +
> > +def sign_rpms(self, files):
> > +"""Sign RPM files"""
> > +import pexpect
> > +
> > +# Remove existing signatures
> > +cmd = "%s --delsign %s" % (self.rpm_bin, ' '.join(files))
> 
> Why are you removing existing signatures?  I believe for many cases this is
> actually incorrect.
> 
> RPM (5) has the ability to have an endless number of signatures within a given
> package.  The package SHOULD included the internal non-repudiable signature...
> 
> (to refresh memory) all RPM 5 packages include an internal non-repudiable
> signature.  Think of this as an extended md5sum, sha256sum, etc.  It doesn't
> change that a package is 'authentic' in any way (often the purpose of 
> signatures
> like what this code is doing), but instead

[OE-core] [PATCH 3/3] oe.gpg_sign: support obs-signd

[Markus Lehtonen]

Implement support for remote signing using obs-signd. It is now possible
to sign both RPM packages and package feeds with this method. The user
just needs to set RPM_GPG_BACKEND and/or PACKAGE_FEED_GPG_BACKEND
variables to 'obssign' in the bitbake config. Of course, in addition,
one needs to setup the signing server and the configure the 'sign'
client command on the build host. The *_PASSPHRASE_FILE settings are not
used when the obssign backend is enabled.

[YOCTO #8755]

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/classes/sign_package_feed.bbclass |  5 +++-
 meta/classes/sign_rpm.bbclass  |  5 +++-
 meta/lib/oe/gpg_sign.py| 48 ++
 3 files changed, 56 insertions(+), 2 deletions(-)

diff --git a/meta/classes/sign_package_feed.bbclass 
b/meta/classes/sign_package_feed.bbclass
index d5df8af..953fa85 100644
--- a/meta/classes/sign_package_feed.bbclass
+++ b/meta/classes/sign_package_feed.bbclass
@@ -24,7 +24,10 @@ PACKAGE_FEED_GPG_BACKEND ?= 'local'
 
 python () {
 # Check sanity of configuration
-for var in ('PACKAGE_FEED_GPG_NAME', 'PACKAGE_FEED_GPG_PASSPHRASE_FILE'):
+required = ['PACKAGE_FEED_GPG_NAME']
+if d.getVar('PACKAGE_FEED_GPG_BACKEND', True) != 'obssign':
+required.append('PACKAGE_FEED_GPG_PASSPHRASE_FILE')
+for var in required:
 if not d.getVar(var, True):
 raise_sanity_error("You need to define %s in the config" % var, d)
 
diff --git a/meta/classes/sign_rpm.bbclass b/meta/classes/sign_rpm.bbclass
index 8bcabee..8be1c35 100644
--- a/meta/classes/sign_rpm.bbclass
+++ b/meta/classes/sign_rpm.bbclass
@@ -23,7 +23,10 @@ RPM_GPG_BACKEND ?= 'local'
 
 python () {
 # Check configuration
-for var in ('RPM_GPG_NAME', 'RPM_GPG_PASSPHRASE_FILE'):
+required = ['RPM_GPG_NAME']
+if d.getVar('RPM_GPG_BACKEND', True) != 'obssign':
+required.append('RPM_GPG_PASSPHRASE_FILE')
+for var in required:
 if not d.getVar(var, True):
 raise_sanity_error("You need to define %s in the config" % var, d)
 
diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
index 55abad8..d8ab816 100644
--- a/meta/lib/oe/gpg_sign.py
+++ b/meta/lib/oe/gpg_sign.py
@@ -66,11 +66,59 @@ class LocalSigner(object):
   (input_file, output))
 
 
+class ObsSigner(object):
+"""Class for handling signing with obs-signd"""
+def __init__(self, keyid):
+self.keyid = keyid
+self.rpm_bin = bb.utils.which(os.getenv('PATH'), "rpm")
+
+def export_pubkey(self, output_file):
+"""Export GPG public key to a file"""
+cmd = "sign -u '%s' -p" % self.keyid
+status, output = oe.utils.getstatusoutput(cmd)
+if status:
+raise bb.build.FuncFailed('Failed to export gpg public key (%s): 
%s' %
+  (self.keyid, output))
+with open(output_file, 'w') as fobj:
+fobj.write(output)
+fobj.write('\n')
+
+def sign_rpms(self, files):
+"""Sign RPM files"""
+import pexpect
+
+# Remove existing signatures
+cmd = "%s --delsign %s" % (self.rpm_bin, ' '.join(files))
+status, output = oe.utils.getstatusoutput(cmd)
+if status:
+raise bb.build.FuncFailed("Failed to remove RPM signatures: %s" %
+  output)
+# Sign packages
+cmd = "sign -u '%s' -r %s" % (self.keyid, ' '.join(files))
+status, output = oe.utils.getstatusoutput(cmd)
+if status:
+raise bb.build.FuncFailed("Failed to sign RPM packages: %s" %
+  output)
+
+def detach_sign(self, input_file):
+"""Create a detached signature of a file"""
+cmd = "sign -u '%s' -d %s" % (self.keyid, input_file)
+status, output = oe.utils.getstatusoutput(cmd)
+if status:
+raise bb.build.FuncFailed("Failed to create signature for '%s': 
%s" %
+  (input_file, output))
+
+
 def get_signer(d, backend, keyid, passphrase_file):
 """Get signer object for the specified backend"""
 # Use local signing by default
 if backend == 'local':
 return LocalSigner(d, keyid, passphrase_file)
+elif backend == 'obssign':
+if passphrase_file:
+bb.note("GPG passphrase file setting not used when 'obssign' "
+"backend is used.")
+return ObsSigner(keyid)
 else:
 bb.fatal("Unsupported signing backend '%s'" % backend)
 
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/3] sign_rpm.bbclass: fix task dependencies

[Markus Lehtonen]

do_rootfs task needs to depend on signing-keys:do_export_public_keys.
The rpm signing public key needs to be present in order to prevent a
crash because it is imported into the rootfs rpmdb before rootfs
creation starts.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/classes/sign_rpm.bbclass | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/classes/sign_rpm.bbclass b/meta/classes/sign_rpm.bbclass
index bc916a7..7906b64 100644
--- a/meta/classes/sign_rpm.bbclass
+++ b/meta/classes/sign_rpm.bbclass
@@ -71,3 +71,4 @@ python sign_rpm () {
 }
 
 do_package_index[depends] += "signing-keys:do_export_public_keys"
+do_rootfs[depends] += "signing-keys:do_export_public_keys"
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 0/3] Support remote RPM signing

[Markus Lehtonen]

This patchset enables remote signing of RPM packages and package feeds using
the obs-signd signing server from openSUSE.
https://github.com/openSUSE/obs-sign
https://en.opensuse.org/openSUSE:Build_Service_Signer

Other remote signing methods should be easy to add, later.

The first patch of the series is a generic task dependency bugfix for rpm
signing.


The following changes since commit 95fced137a46dc98863fe5af7be5cbce708602f2:

  udev-extraconf: introduce multiple blacklist files for more complex setups 
(2016-01-05 17:55:05 +)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib marquiz/rpmsign

for you to fetch changes up to 3ac8c3e5ab0dd6cab1438efd4484e0e313e55d8d:

  oe.gpg_sign: support obs-signd (2016-01-11 18:00:19 +0200)

Markus Lehtonen (3):
  sign_rpm.bbclass: fix task dependencies
  New lib module for handling GPG signing
  oe.gpg_sign: support obs-signd

 meta/classes/sign_package_feed.bbclass |  11 ++-
 meta/classes/sign_rpm.bbclass  |  53 +-
 meta/lib/oe/gpg_sign.py| 124 +
 meta/lib/oe/package_manager.py |  31 +++--
 meta/recipes-core/meta/signing-keys.bb |  26 ---
 5 files changed, 173 insertions(+), 72 deletions(-)
 create mode 100644 meta/lib/oe/gpg_sign.py

-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/3] New lib module for handling GPG signing

[Markus Lehtonen]

Add a new Python module (oe.gpg_sign) for handling GPG signing
operations, i.e. currently package and package feed signing. The
purpose is to be able to more easily support various signing backends.
Currently, only local on-the-build-host signing is implemented.

[YOCTO #8755]

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/classes/sign_package_feed.bbclass |  6 +++
 meta/classes/sign_rpm.bbclass  | 47 +
 meta/lib/oe/gpg_sign.py| 76 ++
 meta/lib/oe/package_manager.py | 31 +-
 meta/recipes-core/meta/signing-keys.bb | 26 ++--
 5 files changed, 116 insertions(+), 70 deletions(-)
 create mode 100644 meta/lib/oe/gpg_sign.py

diff --git a/meta/classes/sign_package_feed.bbclass 
b/meta/classes/sign_package_feed.bbclass
index d89bc0b..d5df8af 100644
--- a/meta/classes/sign_package_feed.bbclass
+++ b/meta/classes/sign_package_feed.bbclass
@@ -6,6 +6,10 @@
 #   Path to a file containing the passphrase of the signing key.
 # PACKAGE_FEED_GPG_NAME
 #   Name of the key to sign with. May be key id or key name.
+# PACKAGE_FEED_GPG_BACKEND
+#   Optional variable for specifying the backend to use for signing.
+#   Currently the only available option is 'local', i.e. local signing
+#   on the build host.
 # GPG_BIN
 #   Optional variable for specifying the gpg binary/wrapper to use for
 #   signing.
@@ -15,6 +19,8 @@
 inherit sanity
 
 PACKAGE_FEED_SIGN = '1'
+PACKAGE_FEED_GPG_BACKEND ?= 'local'
+
 
 python () {
 # Check sanity of configuration
diff --git a/meta/classes/sign_rpm.bbclass b/meta/classes/sign_rpm.bbclass
index 7906b64..8bcabee 100644
--- a/meta/classes/sign_rpm.bbclass
+++ b/meta/classes/sign_rpm.bbclass
@@ -5,6 +5,10 @@
 #   Path to a file containing the passphrase of the signing key.
 # RPM_GPG_NAME
 #   Name of the key to sign with. May be key id or key name.
+# RPM_GPG_BACKEND
+#   Optional variable for specifying the backend to use for signing.
+#   Currently the only available option is 'local', i.e. local signing
+#   on the build host.
 # GPG_BIN
 #   Optional variable for specifying the gpg binary/wrapper to use for
 #   signing.
@@ -14,6 +18,7 @@
 inherit sanity
 
 RPM_SIGN_PACKAGES='1'
+RPM_GPG_BACKEND ?= 'local'
 
 
 python () {
@@ -27,47 +32,17 @@ python () {
 'RPM-GPG-PUBKEY'))
 }
 
-
-def rpmsign_wrapper(d, files, passphrase, gpg_name=None):
-import pexpect
-
-# Find the correct rpm binary
-rpm_bin_path = d.getVar('STAGING_BINDIR_NATIVE', True) + '/rpm'
-cmd = rpm_bin_path + " --addsign --define '_gpg_name %s' " % gpg_name
-if d.getVar('GPG_BIN', True):
-cmd += "--define '%%__gpg %s' " % d.getVar('GPG_BIN', True)
-if d.getVar('GPG_PATH', True):
-cmd += "--define '_gpg_path %s' " % d.getVar('GPG_PATH', True)
-cmd += ' '.join(files)
-
-# Need to use pexpect for feeding the passphrase
-proc = pexpect.spawn(cmd)
-try:
-proc.expect_exact('Enter pass phrase:', timeout=15)
-proc.sendline(passphrase)
-proc.expect(pexpect.EOF, timeout=900)
-proc.close()
-except pexpect.TIMEOUT as err:
-bb.warn('rpmsign timeout: %s' % err)
-proc.terminate()
-else:
-if os.WEXITSTATUS(proc.status) or not os.WIFEXITED(proc.status):
-bb.warn('rpmsign failed: %s' % proc.before.strip())
-return proc.exitstatus
-
-
 python sign_rpm () {
 import glob
+from oe.gpg_sign import get_signer
 
-with open(d.getVar("RPM_GPG_PASSPHRASE_FILE", True)) as fobj:
-rpm_gpg_passphrase = fobj.readlines()[0].rstrip('\n')
-
-rpm_gpg_name = (d.getVar("RPM_GPG_NAME", True) or "")
-
+signer = get_signer(d,
+d.getVar('RPM_GPG_BACKEND', True),
+d.getVar('RPM_GPG_NAME', True),
+d.getVar('RPM_GPG_PASSPHRASE_FILE', True))
 rpms = glob.glob(d.getVar('RPM_PKGWRITEDIR', True) + '/*')
 
-if rpmsign_wrapper(d, rpms, rpm_gpg_passphrase, rpm_gpg_name) != 0:
-raise bb.build.FuncFailed("RPM signing failed")
+signer.sign_rpms(rpms)
 }
 
 do_package_index[depends] += "signing-keys:do_export_public_keys"
diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
new file mode 100644
index 000..55abad8
--- /dev/null
+++ b/meta/lib/oe/gpg_sign.py
@@ -0,0 +1,76 @@
+"""Helper module for GPG signing"""
+import os
+
+import bb
+import oe.utils
+
+class LocalSigner(object):
+"""Class for handling local (on the build host) signing"""
+def __init__(self, d, keyid, passphrase_file):
+self.keyid = keyid
+self.passphrase_file = passphrase_file
+self.gpg_b

[OE-core] [PATCH 5/5] devtool: update-recipe: create kernel config fragment

[Markus Lehtonen]

Create kernel config fragment if the user makes modifications to
.config. User may change .config e.g. by directly editing it or by
running the 'do_menuconfig' bitbake task which will copy the modified
.config back to the source tree. Devtool generates one monolithic
fragment by simply doing a diff between .config and .config.orig files
in the source directory. If either of these files is missing, the config
fragment is not gerenrated or updated. The output is a file,
'devtool-fragment.cfg' that gets added to SRC_URI in the recipe (as well
as copied into the 'oe-local-files' directory if that is present in the
source tree).

This patch also changes the devtool 'extract' command to create the
.config.orig file at the source tree creation time.

[YOCTO #6658]

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 scripts/lib/devtool/standard.py | 50 -
 1 file changed, 49 insertions(+), 1 deletion(-)

diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index f817671..aa9414b 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -20,6 +20,7 @@ import os
 import sys
 import re
 import shutil
+import subprocess
 import tempfile
 import logging
 import argparse
@@ -474,6 +475,7 @@ def _extract_source(srctree, keep_temp, devbranch, sync, d):
 if kconfig:
 # Store kernel config in srctree
 shutil.copy2(kconfig, srcsubdir)
+shutil.copy2(kconfig, os.path.join(srcsubdir, '.config.orig'))
 
 
 tempdir_localdir = os.path.join(tempdir, 'oe-local-files')
@@ -804,6 +806,30 @@ def _export_patches(srctree, rd, start_rev, destdir):
 return (updated, added, existing_patches)
 
 
+def _create_kconfig_diff(srctree, rd, outfile):
+"""Create a kernel config fragment"""
+# Only update config fragment if both config files exist
+orig_config = os.path.join(srctree, '.config.orig')
+new_config = os.path.join(srctree, '.config')
+if os.path.exists(orig_config) and os.path.exists(new_config):
+cmd = ['diff', '--new-line-format=%L', '--old-line-format=',
+   '--unchanged-line-format=', orig_config, new_config]
+pipe = subprocess.Popen(cmd, stdout=subprocess.PIPE,
+stderr=subprocess.PIPE)
+stdout, stderr = pipe.communicate()
+if pipe.returncode == 1:
+with open(outfile, 'w') as fobj:
+fobj.write(stdout)
+elif pipe.returncode == 0:
+if os.path.exists(outfile):
+# Remove fragment file in case of empty diff
+os.unlink(outfile)
+else:
+raise bb.process.ExecutionError(cmd, pipe.returncode, stdout, 
stderr)
+return True
+return False
+
+
 def _export_local_files(srctree, rd, destdir):
 """Copy local files from srctree to given location.
Returns three-tuple of dicts:
@@ -824,6 +850,7 @@ def _export_local_files(srctree, rd, destdir):
 updated = OrderedDict()
 added = OrderedDict()
 removed = OrderedDict()
+local_files_dir = os.path.join(srctree, 'oe-local-files')
 git_files = _git_ls_tree(srctree)
 if 'oe-local-files' in git_files:
 # If tracked by Git, take the files from srctree HEAD. First get
@@ -834,11 +861,32 @@ def _export_local_files(srctree, rd, destdir):
 env=dict(os.environ, GIT_WORK_TREE=destdir,
  GIT_INDEX_FILE=tmp_index))
 new_set = _git_ls_tree(srctree, tree, True).keys()
-elif os.path.isdir(os.path.join(srctree, 'oe-local-files')):
+elif os.path.isdir(local_files_dir):
 # If not tracked by Git, just copy from working copy
 new_set = _ls_tree(os.path.join(srctree, 'oe-local-files'))
 bb.process.run(['cp', '-ax',
 os.path.join(srctree, 'oe-local-files', '.'), destdir])
+else:
+new_set = []
+
+# Special handling for kernel config
+if bb.data.inherits_class('kernel-yocto', rd):
+fragment_fn = 'devtool-fragment.cfg'
+fragment_path = os.path.join(destdir, fragment_fn)
+if _create_kconfig_diff(srctree, rd, fragment_path):
+if os.path.exists(fragment_path):
+if fragment_fn not in new_set:
+new_set.append(fragment_fn)
+# Copy fragment to local-files
+if os.path.isdir(local_files_dir):
+shutil.copy2(fragment_path, local_files_dir)
+else:
+if fragment_fn in new_set:
+new_set.remove(fragment_fn)
+# Remove fragment from local-files
+if os.path.exists(os.path.join(local_files_dir, fragment_fn)):
+os.unlink(os.path.join(local_files_dir, fragment_fn))
+
 if new_set is not None:
 for fname in new_set:

[OE-core] [PATCH 0/5] devtool: create kernel config fragment

[Markus Lehtonen]

This patchset implements an initial support for creating kernel config
fragments on 'devtool update-recipe'. The feature relies on having an updated
.config file and an initial base config (.config.orig) in the source tree.
Devtool does a diff operation between the two and creates/upates a config
fragment which is added to SRC_URI of the recipe.

The first patch in the series is a bugfix and should be straightforward to
merge.

[YOCTO #6658]


The following changes since commit 6f822a9fd185f479ef86c584b6d91a51b3a24e44:

  meta: more removals of redunant FILES_${PN}-dbg (2015-12-16 12:11:26 +)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib marquiz/devtool/kernel

for you to fetch changes up to 2adc35e3129be74afe28dc3087ea44665a80221c:

  devtool: update-recipe: create kernel config fragment (2015-12-17 14:34:15 
+0200)


Markus Lehtonen (5):
  devtool: extract: use the correct datastore for builddir
  kernel.bbclass: copy .config instead of moving
  devtool: extract: cleanup srctree
  cml1.bbclass: copy .config to S if externalsr is in use
  devtool: update-recipe: create kernel config fragment

 meta/classes/cml1.bbclass   |  5 
 meta/classes/kernel.bbclass |  2 +-
 scripts/lib/devtool/standard.py | 65 +++--
 3 files changed, 68 insertions(+), 4 deletions(-)

-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/5] devtool: extract: cleanup srctree

[Markus Lehtonen]

Some bitbake tasks, notably do_kernel_metadata et al. dirty the
sourcetree. Run git clean in order to get rid of the confusing and
possibly outdated extra files.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 scripts/lib/devtool/standard.py | 13 -
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index 43fce11..f817671 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -458,10 +458,21 @@ def _extract_source(srctree, keep_temp, devbranch, sync, 
d):
 bb.process.run('git checkout patches', cwd=srcsubdir)
 
 if bb.data.inherits_class('kernel-yocto', d):
-# Store generate and store kernel config
 logger.info('Generating kernel config')
 task_executor.exec_func('do_configure', False)
 kconfig = os.path.join(crd.getVar('B', True), '.config')
+else:
+kconfig = None
+
+# Clean source tree in case it has been dirtied by some bitbake task
+stdout, _ = bb.process.run('git status --porcelain --ignored',
+   cwd=srcsubdir)
+if stdout:
+logger.info('Source tree is dirty, cleaning up')
+bb.process.run('git clean -fdx', cwd=srcsubdir)
+
+if kconfig:
+# Store kernel config in srctree
 shutil.copy2(kconfig, srcsubdir)
 
 
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/5] devtool: extract: use the correct datastore for builddir

[Markus Lehtonen]

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 scripts/lib/devtool/standard.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index a5e81f3..43fce11 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -461,7 +461,7 @@ def _extract_source(srctree, keep_temp, devbranch, sync, d):
 # Store generate and store kernel config
 logger.info('Generating kernel config')
 task_executor.exec_func('do_configure', False)
-kconfig = os.path.join(d.getVar('B', True), '.config')
+kconfig = os.path.join(crd.getVar('B', True), '.config')
 shutil.copy2(kconfig, srcsubdir)
 
 
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/5] kernel.bbclass: copy .config instead of moving

[Markus Lehtonen]

Copy kernel .config from ${S} to ${B}, instead of moving it. This
prevents mangling the source tree, which is undesirable e.g. when
externalsrc is used.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/classes/kernel.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 4ce1611..7de9f20 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -338,7 +338,7 @@ kernel_do_configure() {
touch ${B}/.scmversion ${S}/.scmversion
 
if [ "${S}" != "${B}" ] && [ -f "${S}/.config" ] && [ ! -f 
"${B}/.config" ]; then
-   mv "${S}/.config" "${B}/.config"
+   cp "${S}/.config" "${B}/.config"
fi
 
# Copy defconfig to .config if .config does not exist. This allows
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 4/5] cml1.bbclass: copy .config to S if externalsr is in use

[Markus Lehtonen]

This makes it easier to track changes made to config.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/classes/cml1.bbclass | 5 +
 1 file changed, 5 insertions(+)

diff --git a/meta/classes/cml1.bbclass b/meta/classes/cml1.bbclass
index 95cf584..a95a2bf 100644
--- a/meta/classes/cml1.bbclass
+++ b/meta/classes/cml1.bbclass
@@ -28,6 +28,11 @@ python do_menuconfig() {
 
 oe_terminal("${SHELL} -c \"make ${KCONFIG_CONFIG_COMMAND}; if [ \$? -ne 0 
]; then echo 'Command failed.'; printf 'Press any key to continue... '; read r; 
fi\"", '${PN} Configuration', d)
 
+# Copy .config back to source tree if externalsrc is in use
+if (d.getVar('EXTERNALSRC', True) and
+os.path.exists(os.path.join(d.getVar('S', True), '.config'))):
+shutil.copy2('.config', os.path.join(d.getVar('S', True), '.config'))
+
 # FIXME this check can be removed when the minimum bitbake version has 
been bumped
 if hasattr(bb.build, 'write_taint'):
 try:
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/5] kernel.bbclass: copy .config instead of moving

[Markus Lehtonen]

Hi Richard,



On 18/12/15 14:22, "Richard Purdie" <richard.pur...@linuxfoundation.org> wrote:

>On Fri, 2015-12-18 at 10:39 +0200, Markus Lehtonen wrote:
>> Copy kernel .config from ${S} to ${B}, instead of moving it. This
>> prevents mangling the source tree, which is undesirable e.g. when
>> externalsrc is used.
>> 
>> Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
>> ---
>>  meta/classes/kernel.bbclass | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>> 
>> diff --git a/meta/classes/kernel.bbclass
>> b/meta/classes/kernel.bbclass
>> index 4ce1611..7de9f20 100644
>> --- a/meta/classes/kernel.bbclass
>> +++ b/meta/classes/kernel.bbclass
>> @@ -338,7 +338,7 @@ kernel_do_configure() {
>>  touch ${B}/.scmversion ${S}/.scmversion
>>  
>>  if [ "${S}" != "${B}" ] && [ -f "${S}/.config" ] && [ ! -f
>> "${B}/.config" ]; then
>> -mv "${S}/.config" "${B}/.config"
>> +cp "${S}/.config" "${B}/.config"
>>  fi
>>  
>>  # Copy defconfig to .config if .config does not exist. This
>> allows
>
>I'm not sure about this, doesn't this trigger the kernel to see ${S} as
>being 'dirty' and cause other issues when you try and do out of tree
>builds with it?

That shouldn't be a problem as the kernel .gitignore ignores .config (or '.*' 
to be more specific). There are other tasks that make changes to the kernel 
source tree, as well, like do_kernel_metadata.


>It also means we have two copies of "config" around which can end up
>being different and confuse users no end :(.

Yes, I must agree. What do you think if ${B}/.config would be a symlink to 
${S}/.config? I.e.
+   ln -s "${S}/.config" "${B}/.config"




Thanks,
  Markus



-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/5] kernel.bbclass: copy .config instead of moving

[Markus Lehtonen]

On 18/12/15 16:18, "Richard Purdie" <richard.pur...@linuxfoundation.org> wrote:

>On Fri, 2015-12-18 at 14:39 +0200, Markus Lehtonen wrote:
>> On 18/12/15 14:22, "Richard Purdie" <
>> richard.pur...@linuxfoundation.org> wrote:
>> > On Fri, 2015-12-18 at 10:39 +0200, Markus Lehtonen wrote: 
>> > >  if [ "${S}" != "${B}" ] && [ -f "${S}/.config" ] && [ ! 
>> > > -f
>> > > "${B}/.config" ]; then
>> > > -mv "${S}/.config" "${B}/.config"
>> > > +cp "${S}/.config" "${B}/.config"
>> > >  fi
>> > >  
>> > >  # Copy defconfig to .config if .config does not exist.
>> > > This
>> > > allows
>> > 
>> > I'm not sure about this, doesn't this trigger the kernel to see
>> > ${S} as
>> > being 'dirty' and cause other issues when you try and do out of
>> > tree
>> > builds with it?
>> 
>> That shouldn't be a problem as the kernel .gitignore ignores .config
>> (or '.*' to be more specific). There are other tasks that make
>> changes to the kernel source tree, as well, like do_kernel_metadata.
>> 
>> 
>> > It also means we have two copies of "config" around which can end
>> > up
>> > being different and confuse users no end :(.
>> 
>> Yes, I must agree. What do you think if ${B}/.config would be a
>> symlink to ${S}/.config? I.e.
>> +   ln -s "${S}/.config" "${B}/.config"
>> 
>
>I think I'd prefer we move the file over the ${B} and then symlink from
>${S} since that way if its modified, its mostly likely to be done from
>${B} at least by the automated code?

Hmm, what I suggested is not good (at least on its own) because menuconfig 
moves .config to .config.old.

Symlinking from ${S} to ${B} is also problematic for the intended devtool use 
case because removing/cleaning ${B} also removes our .config. Back to the 
drawing board, I guess...


Thanks,
  Markus



-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/2] devtool: extract: copy kernel config to srctree

[Markus Lehtonen]

This makes the correct kernel config to be used when building kernel
from srctree (extrernalsrc). If no kernel config is present in the
builddir 'do_configure' task copies .config from the srctree.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 scripts/lib/devtool/standard.py | 8 
 1 file changed, 8 insertions(+)

diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index 68d6eb9..a4f3da4 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -457,6 +457,14 @@ def _extract_source(srctree, keep_temp, devbranch, sync, 
d):
 if haspatches:
 bb.process.run('git checkout patches', cwd=srcsubdir)
 
+if bb.data.inherits_class('kernel-yocto', d):
+# Store generate and store kernel config
+logger.info('Generating kernel config')
+task_executor.exec_func('do_configure', False)
+kconfig = os.path.join(d.getVar('B', True), '.config')
+shutil.copy2(kconfig, srcsubdir)
+
+
 tempdir_localdir = os.path.join(tempdir, 'oe-local-files')
 srctree_localdir = os.path.join(srctree, 'oe-local-files')
 
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 0/2] devtool: kernel config fixes

[Markus Lehtonen]

Two patches changing the kernel config handling (of kernel packages). Now the
srctree is expected to have .config which will be used as the initial kernel
config when building.


The following changes since commit 687e5ef86361a16d6c411386939d4ba96a5909ea:

  libsdl: remove redundant configure_tweak patch (2015-12-01 21:31:04 +)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib marquiz/devtool/kernel

for you to fetch changes up to 0ccea2b62e4d28f3989a72f773e0cbf023739721:

  devtool: extract: update SRCTREECOVEREDTASKS for kernel (2015-12-03 15:35:42 
+0200)


Markus Lehtonen (2):
  devtool: extract: copy kernel config to srctree
  devtool: extract: update SRCTREECOVEREDTASKS for kernel

 scripts/lib/devtool/standard.py | 11 ++-
 1 file changed, 10 insertions(+), 1 deletion(-)

-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/2] devtool: extract: update SRCTREECOVEREDTASKS for kernel

[Markus Lehtonen]

Add 'do_kernel_configme' and 'do_kernel_configcheck' to
SRCTREECOVEREDTASKS of kernel packages. These tasks should not be run
because kernel meta in the srctree is not necessarily up-to-date or
even present which causes build failures and/or invalid kernel config.
Especially so because 'do_patch' which is a dependency of
'do_kernel_configme' is not being run.

We now store .config in the srctree and 'do_configure' task is able to
run successfully.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 scripts/lib/devtool/standard.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index a4f3da4..a5e81f3 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -650,7 +650,8 @@ def modify(args, config, basepath, workspace):
 f.write('EXTERNALSRC_BUILD_pn-%s = "%s"\n' % (pn, srctree))
 
 if bb.data.inherits_class('kernel', rd):
-f.write('SRCTREECOVEREDTASKS = "do_validate_branches 
do_kernel_checkout do_fetch do_unpack do_patch"\n')
+f.write('SRCTREECOVEREDTASKS = "do_validate_branches 
do_kernel_checkout '
+'do_fetch do_unpack do_patch do_kernel_configme 
do_kernel_configcheck"\n')
 if initial_rev:
 f.write('\n# initial_rev: %s\n' % initial_rev)
 for commit in commits:
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] oeqa/selftest/signing: New test for Signing packages in the package feeds.

[Markus Lehtonen]

Hi,

On Fri, 2015-10-16 at 18:16 +0300, Daniel Istrate wrote:
> [YOCTO # 8134] This test verifies features introduced in bug 8134.
> 
> It requires as resources the files from meta-selftest/files/signing:
> For 'gpg --gen-key' the used input was:
> key: RSA
> key-size: 2048
> key-valid: 0
> realname: testuser
> email: testu...@email.com
> comment: nocomment
> passphrase: test123
> 
> Depends on: 
> http://lists.openembedded.org/pipermail/openembedded-core/2015-October/111550.html
> 
> Signed-off-by: Daniel Istrate 
> ---
>  meta-selftest/files/signing/key.pub |  30 
>  meta-selftest/files/signing/key.secret  |  59 
> 
>  meta-selftest/files/signing/pubring.gpg | Bin 0 -> 1204 bytes
>  meta-selftest/files/signing/secret.txt  |   1 +
>  meta-selftest/files/signing/secring.gpg | Bin 0 -> 2582 bytes
>  meta-selftest/files/signing/trustdb.gpg | Bin 0 -> 40 bytes
>  meta/lib/oeqa/selftest/signing.py   |  51 +++
>  7 files changed, 141 insertions(+)
>  create mode 100644 meta-selftest/files/signing/key.pub
>  create mode 100644 meta-selftest/files/signing/key.secret
>  create mode 100644 meta-selftest/files/signing/pubring.gpg
>  create mode 100644 meta-selftest/files/signing/secret.txt
>  create mode 100644 meta-selftest/files/signing/secring.gpg
>  create mode 100644 meta-selftest/files/signing/trustdb.gpg
>  create mode 100644 meta/lib/oeqa/selftest/signing.py
> 
> diff --git a/meta-selftest/files/signing/key.pub 
> b/meta-selftest/files/signing/key.pub
> new file mode 100644
> index 000..e197bb3
> --- /dev/null
> +++ b/meta-selftest/files/signing/key.pub
> @@ -0,0 +1,30 @@
> +-BEGIN PGP PUBLIC KEY BLOCK-
> +Version: GnuPG v1
> +
> +mQENBFYeMycBCADISkEj+u+3SkGbmC4b09StA3Fk4J8bKZrTTpQqUhOH4QFIQpso
> +q96Q907h/ABAgB+IV0SGIeN866E7BqToqoXZ74X6EoyXWdndaMaFZSj+oNqqg6Gi
> +hVsuGNpvRyyXSCYW8w9H2lFx09UufFrUxoSeP2iVdJJaUAmb8e00PCwkYrS2BZEa
> +tO2VgllbaqczldmlUGnkIZt8YUSQSI/xZBDYUvbcZYBaOnDH1SDQl26f+bgyeIyS
> +TW5TZb96o4tMfiifgPoqAapAxQLahG0WtjF/n1yNV5wUNQYsEQf6/h6W2rHGsCP5
> +6FVFnr/ZPVam9iHUxL4lvJSI8dEH37s9GmarABEBAAG0LXRlc3R1c2VyIChub2Nv
> +bW1lbnQpIDx0ZXN0dXNlckB0ZXN0ZW1haWwuY29tPokBOAQTAQIAIgUCVh4zJwIb
> +AwYLCQgHAwIGFQgCCQoLBBYCAwECHgECF4AACgkQezExa11krVLM2wf/fW1C8DPx
> +tZEyl6iPXFjNotslo+t2TL6jPefC22KmbokJCtCnxcopBjQRuhUSNDTkXkUdVagy
> +TaaYILV8XGajTmcVGQTaKeh+j6TM6CBGApQB5KhHvZCyvNBrGcNyuiex0Sm/rIhS
> +fZre6ptZM/026W2kLwwJESXzHJEqCoFmU6aSOUCVyiDgMfcNw6c4NmEoqZtLdnxU
> +B7Nac98o933AIvaaQMGtKIOcyOM7P/dyv8eMc38z2ew5bEB8E9aSdg5koXb3zIt5
> +IKea631k4INAsFFyLMQNSmmKV7RK0miF5b4hGyekrYZRtiic5+dq5aWnVka4hBfi
> +x31euxwQE87gQLkBDQRWHjMnAQgAt7C9QCFPWzLGQuQ/YaQub+8s2lYNQnmfwDHm
> +5PuON+Wj/f5GyQhHKsbdUAPZ7GsjFIQnva7xNYYF/IvpC+0saB5NLMkBzjfIsg92
> +6MkadAKlOR2o9gKlF59mulsJmJqNFTXiRcVXvpUnU8WB9ECmm321XfYHhk+4EMay
> +H3OUZ0k6dEmvrWBTKNTR7M0z6j/jW+8J3vP3L9k1H+OV0EZwAKXfbh1lN4H467jY
> +3gA7FU1WDmA06HphoSaFUEGTuXGtrRP0eksCUj3BtVygXnyQb379dISDOWcs/9Ke
> +v3KMrZWgDnA4pH1eQpjycBhwKOCHYyhSSVOwCS3DGkaaklmQZwARAQABiQEfBBgB
> +AgAJBQJWHjMnAhsMAAoJEHsxMWtdZK1SoPsIAKadG/tvS5COCyF8FuriL89Ysfov
> +kMRKeb9hsMDbKX2lm3UtoS5ErmpkEUO/SbazQYm6/vYc8noQquqhkIdCljIvpWDv
> +17tXEFfTGA493dlTTEWFt5bvzbQN6OhBu3904lAE4JGtlOOa9OKDeguwXbneLOyl
> +dnlj2f7rw05cB9t/RDu7T11dTI39BMTUUm1lpWxYJk41o59b9g+fpJZkiIAJwnN3
> +MwM1u9/AWfTqjNRgMAO5dIYceceTwGogujG+xz93flt+NjQhILG0T9jd0DFBgIAX
> +Zq4PzX5aFDKjGoFaOOZ6r+kppBLH/HN6okMGIcfqaPPdnJI1MXFQvFzUNpo=
> +=2cSJ
> +-END PGP PUBLIC KEY BLOCK-
> diff --git a/meta-selftest/files/signing/key.secret 
> b/meta-selftest/files/signing/key.secret
> new file mode 100644
> index 000..70ef829
> --- /dev/null
> +++ b/meta-selftest/files/signing/key.secret
> @@ -0,0 +1,59 @@
> +-BEGIN PGP PRIVATE KEY BLOCK-
> +Version: GnuPG v1
> +
> +lQO+BFYeLjIBCADxa6HxI7YMC4fedDBB2IvQHXF7fc8JnXtDPCJFbRT4JgBvVzqy
> +9QRRGfL9+OOr6oKM3cXBUNFWz4UXpC5K3OIcBTy4n0X2YqUrF4jLNZvEZB0+Qpxi
> +PGQERacD5pPALZDlMPOulfVaq3up7qiMR2gXuQjggPIKmIlQGo5yr2KBNAbcXykh
> +1DI12qrwsaaXiruFyKCJItzFGlu6B0PqCE0NQOkY/wO+kUSiBP5aQH/WM5We17Wb
> +Lxl7MLwicheSLQix+YOftFYacs8zBIlkdoVnrwDkJLSwjqHw/i+03LTznr+i3Vp9
> +mWRQFI+rcEI8XcLFxOemTYZcCQC+ppZA0F3VABEBAAH+AwMCggofrCu0WR9gR6VS
> +8/XQ3+yKFwp03/4dds0sYaS5GqIvWnKYOjKlClFDkdtvwKEV/0fvcfeTLMSCSVt3
> +RqM+HnDQeCG4Ml+EkTlumUEUJcx03wFqDLpZDu2Ka/NpieYZTLvkUdl/SvUWoTDx
> +4XAeZGe82BMSUIfa0VDP+7xhsOl/YFqq25Ra/ykiiPWJdKZz75f90gjmX60MmIt/
> +egJHx/ec7VaehvVPJ4HgY1dVokfW+WErsZmDP+Ei/zwcdzMIaeXsHJ8FSOqfeejG
> +u+hCADUUfta/IwdR7wVxvibJ1qqJSa+pf8slxeRjpfp+V6l5G+edfrtmOVkM7HaN
> +uonCdErAT6n+/l4ce/BuG76GtA232KWNGDJseyhfx011CttkPVEq8adGLA7iiTLC
> +IHBP58t8CNCRlzOn3IRpRuKkam+yg+vxe7ujaupMUtkBZmECBQa7oSoAGTcetqf3
> +nq7N9D3CD7KJffoX+M/0Ye6Ptpc/1Szoea+Yl4u4upVdpie0DhD/o9k8pNT0MGdK
> +GdMwcgp2XSUpkatCEYD8tg0l8suxdXl4fbtLCi4RvKdU0ZhH6CFQ0IR3D6xtURBR
> +c0+bYPN3Vb+ynmXxwaUsYVvj7gkkfJbx0y592WpAAZqkfllDsmEaxyNd9SdBagld
> +KKpgDoV1Cmd7g0rrZJi83Nm5i2F5M1HCt/A91Gh0sx4N0BjnFolC7hCYXKoLBLPv
> 

[OE-core] [PATCH 5/5] package signing: automatically export public keys

[Markus Lehtonen]

Automatically export public key(s) of the signing key(s) from the gpg
keyring. Adds a new simple recipe that does the actual task of exporting
the keys.  This patch makes the RPM_GPG_PUBKEY and PACKAGE_FEED_GPG
PUBKEY settings obsolete.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/classes/sign_package_feed.bbclass |  7 +
 meta/classes/sign_rpm.bbclass  |  9 --
 meta/recipes-core/meta/signing-keys.bb | 45 ++
 meta/recipes-core/os-release/os-release.bb |  1 +
 4 files changed, 59 insertions(+), 3 deletions(-)
 create mode 100644 meta/recipes-core/meta/signing-keys.bb

diff --git a/meta/classes/sign_package_feed.bbclass 
b/meta/classes/sign_package_feed.bbclass
index 8877d90..4263810 100644
--- a/meta/classes/sign_package_feed.bbclass
+++ b/meta/classes/sign_package_feed.bbclass
@@ -21,4 +21,11 @@ python () {
 for var in ('PACKAGE_FEED_GPG_NAME', 'PACKAGE_FEED_GPG_PASSPHRASE_FILE'):
 if not d.getVar(var, True):
 raise_sanity_error("You need to define %s in the config" % var, d)
+
+# Set expected location of the public key
+d.setVar('PACKAGE_FEED_GPG_PUBKEY',
+ os.path.join(d.getVar('STAGING_ETCDIR_NATIVE'),
+   'PACKAGE-FEED-GPG-PUBKEY'))
 }
+
+do_package_index[depends] += "signing-keys:do_export_public_keys"
diff --git a/meta/classes/sign_rpm.bbclass b/meta/classes/sign_rpm.bbclass
index 4da1763..f0c3dc9 100644
--- a/meta/classes/sign_rpm.bbclass
+++ b/meta/classes/sign_rpm.bbclass
@@ -5,9 +5,6 @@
 #   Path to a file containing the passphrase of the signing key.
 # RPM_GPG_NAME
 #   Name of the key to sign with. May be key id or key name.
-# RPM_GPG_PUBKEY
-#   Path to a file containing the public key (in "armor" format)
-#   corresponding the signing key.
 # GPG_BIN
 #   Optional variable for specifying the gpg binary/wrapper to use for
 #   signing.
@@ -24,6 +21,10 @@ python () {
 for var in ('RPM_GPG_NAME', 'RPM_GPG_PASSPHRASE_FILE'):
 if not d.getVar(var, True):
 raise_sanity_error("You need to define %s in the config" % var, d)
+
+# Set the expected location of the public key
+d.setVar('RPM_GPG_PUBKEY', os.path.join(d.getVar('STAGING_ETCDIR_NATIVE'),
+'RPM-GPG-PUBKEY'))
 }
 
 
@@ -68,3 +69,5 @@ python sign_rpm () {
 if rpmsign_wrapper(d, rpms, rpm_gpg_passphrase, rpm_gpg_name) != 0:
 raise bb.build.FuncFailed("RPM signing failed")
 }
+
+do_package_index[depends] += "signing-keys:do_export_public_keys"
diff --git a/meta/recipes-core/meta/signing-keys.bb 
b/meta/recipes-core/meta/signing-keys.bb
new file mode 100644
index 000..cc401f3
--- /dev/null
+++ b/meta/recipes-core/meta/signing-keys.bb
@@ -0,0 +1,45 @@
+# Copyright (C) 2015 Intel Corporation
+# Released under the MIT license (see COPYING.MIT for the terms)
+
+DESCRIPTION = "Make public keys of the signing keys available"
+LICENSE = "MIT"
+PACKAGES = ""
+
+do_fetch[noexec] = "1"
+do_unpack[noexec] = "1"
+do_patch[noexec] = "1"
+do_configure[noexec] = "1"
+do_compile[noexec] = "1"
+do_install[noexec] = "1"
+do_package[noexec] = "1"
+do_packagedata[noexec] = "1"
+do_package_write_ipk[noexec] = "1"
+do_package_write_rpm[noexec] = "1"
+do_package_write_deb[noexec] = "1"
+do_populate_sysroot[noexec] = "1"
+
+EXCLUDE_FROM_WORLD = "1"
+
+def export_gpg_pubkey(d, keyid, path):
+import bb
+gpg_bin = d.getVar('GPG_BIN', True) or \
+  bb.utils.which(os.getenv('PATH'), "gpg")
+cmd = '%s --batch --yes --export --armor -o %s %s' % \
+  (gpg_bin, path, keyid)
+status, output = oe.utils.getstatusoutput(cmd)
+if status:
+raise bb.build.FuncFailed('Failed to export gpg public key (%s): %s' %
+  (keyid, output))
+
+python do_export_public_keys () {
+if d.getVar("RPM_SIGN_PACKAGES", True):
+# Export public key of the rpm signing key
+export_gpg_pubkey(d, d.getVar("RPM_GPG_NAME", True),
+  d.getVar('RPM_GPG_PUBKEY', True))
+
+if d.getVar('PACKAGE_FEED_SIGN', True) == '1':
+# Export public key of the feed signing key
+export_gpg_pubkey(d, d.getVar("PACKAGE_FEED_GPG_NAME", True),
+  d.getVar('PACKAGE_FEED_GPG_PUBKEY', True))
+}
+addtask do_export_public_keys before do_build
diff --git a/meta/recipes-core/os-release/os-release.bb 
b/meta/recipes-core/os-release/os-release.bb
index db82760..c690b82 100644
--- a/meta/recipes-core/os-release/os-release.bb
+++ b/meta/recipes-core/os-release/os-release.bb
@@ -37,6 +37,7 @@ python do_compile () {

[OE-core] [PATCH 2/5] sign_rpm.bbclass: make RPM_GPG_NAME a mandatory setting

[Markus Lehtonen]

Simplifies the configuration. Makes way for the removal of
RPM_GPG_PUBKEY setting and possible future implementation of a separate
signing server support. Also, moves the configuration sanity checking
into a separate function.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/classes/sign_rpm.bbclass | 30 +-
 1 file changed, 9 insertions(+), 21 deletions(-)

diff --git a/meta/classes/sign_rpm.bbclass b/meta/classes/sign_rpm.bbclass
index 39f877a..4da1763 100644
--- a/meta/classes/sign_rpm.bbclass
+++ b/meta/classes/sign_rpm.bbclass
@@ -4,8 +4,7 @@
 # RPM_GPG_PASSPHRASE_FILE
 #   Path to a file containing the passphrase of the signing key.
 # RPM_GPG_NAME
-#   Name of the key to sign with. Alternatively you can define
-#   %_gpg_name macro in your ~/.oerpmmacros file.
+#   Name of the key to sign with. May be key id or key name.
 # RPM_GPG_PUBKEY
 #   Path to a file containing the public key (in "armor" format)
 #   corresponding the signing key.
@@ -20,9 +19,11 @@ inherit sanity
 RPM_SIGN_PACKAGES='1'
 
 
-_check_gpg_name () {
-macrodef=`rpm -E '%_gpg_name'`
-[ "$macrodef" == "%_gpg_name" ] && return 1 || return 0
+python () {
+# Check configuration
+for var in ('RPM_GPG_NAME', 'RPM_GPG_PASSPHRASE_FILE'):
+if not d.getVar(var, True):
+raise_sanity_error("You need to define %s in the config" % var, d)
 }
 
 
@@ -31,16 +32,7 @@ def rpmsign_wrapper(d, files, passphrase, gpg_name=None):
 
 # Find the correct rpm binary
 rpm_bin_path = d.getVar('STAGING_BINDIR_NATIVE', True) + '/rpm'
-cmd = rpm_bin_path + " --addsign "
-if gpg_name:
-cmd += "--define '%%_gpg_name %s' " % gpg_name
-else:
-try:
-bb.build.exec_func('_check_gpg_name', d)
-except bb.build.FuncFailed:
-raise_sanity_error("You need to define RPM_GPG_NAME in bitbake "
-   "config or the %_gpg_name RPM macro defined "
-   "(e.g. in  ~/.oerpmmacros", d)
+cmd = rpm_bin_path + " --addsign --define '_gpg_name %s' " % gpg_name
 if d.getVar('GPG_BIN', True):
 cmd += "--define '%%__gpg %s' " % d.getVar('GPG_BIN', True)
 if d.getVar('GPG_PATH', True):
@@ -66,12 +58,8 @@ def rpmsign_wrapper(d, files, passphrase, gpg_name=None):
 python sign_rpm () {
 import glob
 
-rpm_gpg_pass_file = (d.getVar("RPM_GPG_PASSPHRASE_FILE", True) or "")
-if rpm_gpg_pass_file:
-with open(rpm_gpg_pass_file) as fobj:
-rpm_gpg_passphrase = fobj.readlines()[0].rstrip('\n')
-else:
-raise_sanity_error("You need to define RPM_GPG_PASSPHRASE_FILE in the 
config", d)
+with open(d.getVar("RPM_GPG_PASSPHRASE_FILE", True)) as fobj:
+rpm_gpg_passphrase = fobj.readlines()[0].rstrip('\n')
 
 rpm_gpg_name = (d.getVar("RPM_GPG_NAME", True) or "")
 
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/5] Add new bbclass for package feed signing

[Markus Lehtonen]

After this change signed package feeds should be enabled by adding
INERIT += "sign_package_feed"
instead of definining PACKAGE_FEED_SIGN="1".

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/classes/sign_package_feed.bbclass | 24 
 1 file changed, 24 insertions(+)
 create mode 100644 meta/classes/sign_package_feed.bbclass

diff --git a/meta/classes/sign_package_feed.bbclass 
b/meta/classes/sign_package_feed.bbclass
new file mode 100644
index 000..8877d90
--- /dev/null
+++ b/meta/classes/sign_package_feed.bbclass
@@ -0,0 +1,24 @@
+# Class for signing package feeds
+#
+# Related configuration variables that will be used after this class is
+# iherited:
+# PACKAGE_FEED_PASSPHRASE_FILE
+#   Path to a file containing the passphrase of the signing key.
+# PACKAGE_FEED_GPG_NAME
+#   Name of the key to sign with. May be key id or key name.
+# GPG_BIN
+#   Optional variable for specifying the gpg binary/wrapper to use for
+#   signing.
+# GPG_PATH
+#   Optional variable for specifying the gnupg "home" directory:
+#
+inherit sanity
+
+PACKAGE_FEED_SIGN = '1'
+
+python () {
+# Check sanity of configuration
+for var in ('PACKAGE_FEED_GPG_NAME', 'PACKAGE_FEED_GPG_PASSPHRASE_FILE'):
+if not d.getVar(var, True):
+raise_sanity_error("You need to define %s in the config" % var, d)
+}
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 5/5] Automatically export public keys used for package(feed) signing

[Markus Lehtonen]

Automatically export public key(s) from the gpg keyring. Adds a new
simple recipe that does the actual task of exporting the keys.  This
patch makes the RPM_GPG_PUBKEY and PACKAGE_FEED_GPG PUBKEY settings
obsolete.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/classes/sign_package_feed.bbclass |  7 +
 meta/classes/sign_rpm.bbclass  |  9 --
 meta/recipes-core/meta/signing-keys.bb | 45 ++
 meta/recipes-core/os-release/os-release.bb |  1 +
 4 files changed, 59 insertions(+), 3 deletions(-)
 create mode 100644 meta/recipes-core/meta/signing-keys.bb

diff --git a/meta/classes/sign_package_feed.bbclass 
b/meta/classes/sign_package_feed.bbclass
index 8877d90..4263810 100644
--- a/meta/classes/sign_package_feed.bbclass
+++ b/meta/classes/sign_package_feed.bbclass
@@ -21,4 +21,11 @@ python () {
 for var in ('PACKAGE_FEED_GPG_NAME', 'PACKAGE_FEED_GPG_PASSPHRASE_FILE'):
 if not d.getVar(var, True):
 raise_sanity_error("You need to define %s in the config" % var, d)
+
+# Set expected location of the public key
+d.setVar('PACKAGE_FEED_GPG_PUBKEY',
+ os.path.join(d.getVar('STAGING_ETCDIR_NATIVE'),
+   'PACKAGE-FEED-GPG-PUBKEY'))
 }
+
+do_package_index[depends] += "signing-keys:do_export_public_keys"
diff --git a/meta/classes/sign_rpm.bbclass b/meta/classes/sign_rpm.bbclass
index 4da1763..f0c3dc9 100644
--- a/meta/classes/sign_rpm.bbclass
+++ b/meta/classes/sign_rpm.bbclass
@@ -5,9 +5,6 @@
 #   Path to a file containing the passphrase of the signing key.
 # RPM_GPG_NAME
 #   Name of the key to sign with. May be key id or key name.
-# RPM_GPG_PUBKEY
-#   Path to a file containing the public key (in "armor" format)
-#   corresponding the signing key.
 # GPG_BIN
 #   Optional variable for specifying the gpg binary/wrapper to use for
 #   signing.
@@ -24,6 +21,10 @@ python () {
 for var in ('RPM_GPG_NAME', 'RPM_GPG_PASSPHRASE_FILE'):
 if not d.getVar(var, True):
 raise_sanity_error("You need to define %s in the config" % var, d)
+
+# Set the expected location of the public key
+d.setVar('RPM_GPG_PUBKEY', os.path.join(d.getVar('STAGING_ETCDIR_NATIVE'),
+'RPM-GPG-PUBKEY'))
 }
 
 
@@ -68,3 +69,5 @@ python sign_rpm () {
 if rpmsign_wrapper(d, rpms, rpm_gpg_passphrase, rpm_gpg_name) != 0:
 raise bb.build.FuncFailed("RPM signing failed")
 }
+
+do_package_index[depends] += "signing-keys:do_export_public_keys"
diff --git a/meta/recipes-core/meta/signing-keys.bb 
b/meta/recipes-core/meta/signing-keys.bb
new file mode 100644
index 000..cc401f3
--- /dev/null
+++ b/meta/recipes-core/meta/signing-keys.bb
@@ -0,0 +1,45 @@
+# Copyright (C) 2015 Intel Corporation
+# Released under the MIT license (see COPYING.MIT for the terms)
+
+DESCRIPTION = "Make public keys of the signing keys available"
+LICENSE = "MIT"
+PACKAGES = ""
+
+do_fetch[noexec] = "1"
+do_unpack[noexec] = "1"
+do_patch[noexec] = "1"
+do_configure[noexec] = "1"
+do_compile[noexec] = "1"
+do_install[noexec] = "1"
+do_package[noexec] = "1"
+do_packagedata[noexec] = "1"
+do_package_write_ipk[noexec] = "1"
+do_package_write_rpm[noexec] = "1"
+do_package_write_deb[noexec] = "1"
+do_populate_sysroot[noexec] = "1"
+
+EXCLUDE_FROM_WORLD = "1"
+
+def export_gpg_pubkey(d, keyid, path):
+import bb
+gpg_bin = d.getVar('GPG_BIN', True) or \
+  bb.utils.which(os.getenv('PATH'), "gpg")
+cmd = '%s --batch --yes --export --armor -o %s %s' % \
+  (gpg_bin, path, keyid)
+status, output = oe.utils.getstatusoutput(cmd)
+if status:
+raise bb.build.FuncFailed('Failed to export gpg public key (%s): %s' %
+  (keyid, output))
+
+python do_export_public_keys () {
+if d.getVar("RPM_SIGN_PACKAGES", True):
+# Export public key of the rpm signing key
+export_gpg_pubkey(d, d.getVar("RPM_GPG_NAME", True),
+  d.getVar('RPM_GPG_PUBKEY', True))
+
+if d.getVar('PACKAGE_FEED_SIGN', True) == '1':
+# Export public key of the feed signing key
+export_gpg_pubkey(d, d.getVar("PACKAGE_FEED_GPG_NAME", True),
+  d.getVar('PACKAGE_FEED_GPG_PUBKEY', True))
+}
+addtask do_export_public_keys before do_build
diff --git a/meta/recipes-core/os-release/os-release.bb 
b/meta/recipes-core/os-release/os-release.bb
index db82760..c690b82 100644
--- a/meta/recipes-core/os-release/os-release.bb
+++ b/meta/recipes-core/os-release/os-release.bb
@@ -37,6 +37,7 @@ python do_compile () {

[OE-core] [PATCH 1/5] sign_rpm.bbclass: be more verbose in case of error

[Markus Lehtonen]

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/classes/sign_rpm.bbclass | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/meta/classes/sign_rpm.bbclass b/meta/classes/sign_rpm.bbclass
index 23aea42..39f877a 100644
--- a/meta/classes/sign_rpm.bbclass
+++ b/meta/classes/sign_rpm.bbclass
@@ -55,8 +55,11 @@ def rpmsign_wrapper(d, files, passphrase, gpg_name=None):
 proc.expect(pexpect.EOF, timeout=900)
 proc.close()
 except pexpect.TIMEOUT as err:
-bb.debug('rpmsign timeout: %s' % err)
+bb.warn('rpmsign timeout: %s' % err)
 proc.terminate()
+else:
+if os.WEXITSTATUS(proc.status) or not os.WIFEXITED(proc.status):
+bb.warn('rpmsign failed: %s' % proc.before.strip())
 return proc.exitstatus
 
 
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 0/5] Rework rpm signing

[Markus Lehtonen]

Additionl patches on top of my previous "Implement GPG_PATH variable" patchset.

Slightly rework the rpm and package feed signing feature. The user-visible
changes are that the RPM_GPG_PUBKEY and PACKAGE_FEED_GPG PUBKEY configuration
variables are not needed anymore, and, package feed signing should now be
enabled using INERIT += "sign_package_feed".

The following changes since commit 27ce8876e4b4b19fd26c2f70ccc2acc2c1bc5d40:

  package_manager: support GPG_PATH variable (2015-10-16 14:03:46 +0300)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib marquiz/rpmsign
  http://git.openembedded.org/openembedded-core-contrib/log/?h=marquiz/rpmsign

Markus Lehtonen (5):
  sign_rpm.bbclass: be more verbose in case of error
  sign_rpm.bbclass: make RPM_GPG_NAME a mandatory setting
  Add new bbclass for package feed signing
  package_manager: fail if signed feeds are enabled for ipk or dpkg
  package signing: automatically export public keys

 meta/classes/sign_package_feed.bbclass | 31 
 meta/classes/sign_rpm.bbclass  | 44 +
 meta/lib/oe/package_manager.py |  4 +++
 meta/recipes-core/meta/signing-keys.bb | 45 ++
 meta/recipes-core/os-release/os-release.bb |  1 +
 5 files changed, 100 insertions(+), 25 deletions(-)
 create mode 100644 meta/classes/sign_package_feed.bbclass
 create mode 100644 meta/recipes-core/meta/signing-keys.bb

-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 4/5] package_manager: fail if signed feeds are enabled for ipk or dpkg

[Markus Lehtonen]

Signed package feeds are not yet implemented for these package formats.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/lib/oe/package_manager.py | 4 
 1 file changed, 4 insertions(+)

diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index 5c2130b..964fddc 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -203,6 +203,8 @@ class OpkgIndexer(Indexer):
 result = oe.utils.multiprocess_exec(index_cmds, create_index)
 if result:
 bb.fatal('%s' % ('\n'.join(result)))
+if self.d.getVar('PACKAGE_FEED_SIGN', True) == '1':
+raise NotImplementedError('Package feed signing not implementd for 
ipk')
 
 
 
@@ -278,6 +280,8 @@ class DpkgIndexer(Indexer):
 result = oe.utils.multiprocess_exec(index_cmds, create_index)
 if result:
 bb.fatal('%s' % ('\n'.join(result)))
+if self.d.getVar('PACKAGE_FEED_SIGN', True) == '1':
+raise NotImplementedError('Package feed signing not implementd for 
dpkg')
 
 
 
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 0/2] Implement GPG_PATH variable

[Markus Lehtonen]

This optional setting makes it possible to specify a non-default gpg home
directory in the bitbake configuration.

[YOCTO #8134]

The following changes since commit e42d8eff9eed7d1454b4f331d96dcee6dea232df:

  bash: Disable custom memory allocator (2015-10-12 14:35:42 +0100)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib marquiz/rpmsign
  http://git.openembedded.org/openembedded-core-contrib/log/?h=marquiz/rpmsign

Markus Lehtonen (2):
  sign_rpm.bbclass: introduce GPG_PATH variable
  package_manager: support GPG_PATH variable

 meta/classes/sign_rpm.bbclass  | 4 
 meta/lib/oe/package_manager.py | 7 +--
 2 files changed, 9 insertions(+), 2 deletions(-)

-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/2] sign_rpm.bbclass: introduce GPG_PATH variable

[Markus Lehtonen]

This bitbake configuration variable can be used to define the gpg home
directory.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/classes/sign_rpm.bbclass | 4 
 1 file changed, 4 insertions(+)

diff --git a/meta/classes/sign_rpm.bbclass b/meta/classes/sign_rpm.bbclass
index 0aa4cd8..23aea42 100644
--- a/meta/classes/sign_rpm.bbclass
+++ b/meta/classes/sign_rpm.bbclass
@@ -12,6 +12,8 @@
 # GPG_BIN
 #   Optional variable for specifying the gpg binary/wrapper to use for
 #   signing.
+# GPG_PATH
+#   Optional variable for specifying the gnupg "home" directory:
 #
 inherit sanity
 
@@ -41,6 +43,8 @@ def rpmsign_wrapper(d, files, passphrase, gpg_name=None):
"(e.g. in  ~/.oerpmmacros", d)
 if d.getVar('GPG_BIN', True):
 cmd += "--define '%%__gpg %s' " % d.getVar('GPG_BIN', True)
+if d.getVar('GPG_PATH', True):
+cmd += "--define '_gpg_path %s' " % d.getVar('GPG_PATH', True)
 cmd += ' '.join(files)
 
 # Need to use pexpect for feeding the passphrase
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/2] package_manager: support GPG_PATH variable

[Markus Lehtonen]

If defined, use GPG_PATH as the gpg home directory when signing package
feeds. This setting is only used by package_manager if package feed
singning has been enabled, i.e.  PACKAGE_FEED_SIGN="1".

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/lib/oe/package_manager.py | 7 +--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index c34e436..5c2130b 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -133,8 +133,11 @@ class RpmIndexer(Indexer):
 if pkgfeed_gpg_name:
 repomd_file = os.path.join(arch_dir, 'repodata', 'repomd.xml')
 gpg_cmd = "%s --detach-sign --armor --batch --no-tty --yes " \
-  "--passphrase-file '%s' -u '%s' %s" % (gpg_bin,
-  pkgfeed_gpg_pass, pkgfeed_gpg_name, repomd_file)
+  "--passphrase-file '%s' -u '%s' " % \
+  (gpg_bin, pkgfeed_gpg_pass, pkgfeed_gpg_name)
+if self.d.getVar('GPG_PATH', True):
+gpg_cmd += "--homedir %s " % self.d.getVar('GPG_PATH', 
True)
+gpg_cmd += repomd_file
 repo_sign_cmds.append(gpg_cmd)
 
 rpm_dirs_found = True
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2] os-release: fix do_compile() when RPM signing is enabled

[Markus Lehtonen]

do_compile() task failed when RPM signing was in use.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/recipes-core/os-release/os-release.bb | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-core/os-release/os-release.bb 
b/meta/recipes-core/os-release/os-release.bb
index cc431d2..db82760 100644
--- a/meta/recipes-core/os-release/os-release.bb
+++ b/meta/recipes-core/os-release/os-release.bb
@@ -32,8 +32,8 @@ python do_compile () {
 f.write('{0}={1}\n'.format(field, value))
 if d.getVar('RPM_SIGN_PACKAGES', True) == '1':
 rpm_gpg_pubkey = d.getVar('RPM_GPG_PUBKEY', True)
-os.mkdir('${B}/rpm-gpg')
-distro_version = self.d.getVar('DISTRO_VERSION', True) or "oe.0"
+bb.utils.mkdirhier('${B}/rpm-gpg')
+distro_version = d.getVar('DISTRO_VERSION', True) or "oe.0"
 shutil.copy2(rpm_gpg_pubkey, d.expand('${B}/rpm-gpg/RPM-GPG-KEY-%s' % 
distro_version))
 }
 do_compile[vardeps] += "${OS_RELEASE_FIELDS}"
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] os-release: fix do_compile() when RPM signing is enabled

[Markus Lehtonen]

do_compile() task failed when RPM signing was in use.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/recipes-core/os-release/os-release.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-core/os-release/os-release.bb 
b/meta/recipes-core/os-release/os-release.bb
index cc431d2..c99ea3e 100644
--- a/meta/recipes-core/os-release/os-release.bb
+++ b/meta/recipes-core/os-release/os-release.bb
@@ -33,7 +33,7 @@ python do_compile () {
 if d.getVar('RPM_SIGN_PACKAGES', True) == '1':
 rpm_gpg_pubkey = d.getVar('RPM_GPG_PUBKEY', True)
 os.mkdir('${B}/rpm-gpg')
-distro_version = self.d.getVar('DISTRO_VERSION', True) or "oe.0"
+distro_version = d.getVar('DISTRO_VERSION', True) or "oe.0"
 shutil.copy2(rpm_gpg_pubkey, d.expand('${B}/rpm-gpg/RPM-GPG-KEY-%s' % 
distro_version))
 }
 do_compile[vardeps] += "${OS_RELEASE_FIELDS}"
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] devtool: modify: use correct local files directory name

[Markus Lehtonen]

The name of the directory for local source files under srctree is
'oe-local-files', not 'local-files'. Fixes a bug that slipped through
in b7ab82485e4514e07ab8a76e554da27ddc92e6c0.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 scripts/lib/devtool/standard.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index b455a22..8676e42 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -562,7 +562,7 @@ def modify(args, config, basepath, workspace):
 # Local files can be modified/tracked in separate subdir under srctree
 # Mostly useful for packages with S != WORKDIR
 f.write('FILESPATH_prepend := "%s:"\n' %
-os.path.join(srctree, 'local-files'))
+os.path.join(srctree, 'oe-local-files'))
 
 f.write('\ninherit externalsrc\n')
 f.write('# NOTE: We use pn- overrides here to avoid affecting multiple 
variants in the case where the recipe uses BBCLASSEXTEND\n')
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v3 10/10] devtool: modify: make bitbake use local files from srctree

[Markus Lehtonen]

Hi,

On Mon, 2015-09-28 at 14:48 +0100, Paul Eggleton wrote:
> Hi Markus,
> 
> On Thursday 24 September 2015 14:53:07 Markus Lehtonen wrote:
> > This change makes it possible to have local files (non-remote SRC_URI
> > files, i.e. files that are located in the "recipe space") under the
> > srctree even if S!=WORKDIR. The files must be placed under the
> > 'local-files' subdirectory.
> > 
> > Complements the previous patch that imports local files into srctree.
> > 
> > [YOCTO #7602]
> > 
> > Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
> > ---
> >  scripts/lib/devtool/standard.py | 9 +++--
> >  1 file changed, 7 insertions(+), 2 deletions(-)
> > 
> > diff --git a/scripts/lib/devtool/standard.py
> > b/scripts/lib/devtool/standard.py index 6b85c8c..78b0d27 100644
> > --- a/scripts/lib/devtool/standard.py
> > +++ b/scripts/lib/devtool/standard.py
> > @@ -558,8 +558,13 @@ def modify(args, config, basepath, workspace):
> >  if not os.path.exists(appendpath):
> >  os.makedirs(appendpath)
> >  with open(appendfile, 'w') as f:
> > -f.write('FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"\n\n')
> > -f.write('inherit externalsrc\n')
> > +f.write('FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"\n')
> > +# Local files can be modified/tracked in separate subdir under
> > srctree +# Mostly useful for packages with S != WORKDIR
> > +f.write('FILESPATH_prepend := "%s:"\n' %
> > +os.path.join(srctree, 'local-files'))
> 
> Shouldn't this directory be named "oe-local-files"?

Argh, sorry, I had missed this email earlier. Yes, it should be
'oe-local-files'. An updated patch with a fix is available at:

http://cgit.openembedded.org/openembedded-core-contrib/log/?h=marquiz/devtool/localfiles



Thanks,
  Markus

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v3 10/10] devtool: modify: make bitbake use local files from srctree

[Markus Lehtonen]

Hi,

On Wed, 2015-09-30 at 10:21 +0100, Paul Eggleton wrote:
> On Wednesday 30 September 2015 12:01:13 Markus Lehtonen wrote:
> > Hi,
> > 
> > On Mon, 2015-09-28 at 14:48 +0100, Paul Eggleton wrote:
> > > Hi Markus,
> > > 
> > > On Thursday 24 September 2015 14:53:07 Markus Lehtonen wrote:
> > > > This change makes it possible to have local files (non-remote SRC_URI
> > > > files, i.e. files that are located in the "recipe space") under the
> > > > srctree even if S!=WORKDIR. The files must be placed under the
> > > > 'local-files' subdirectory.
> > > > 
> > > > Complements the previous patch that imports local files into srctree.
> > > > 
> > > > [YOCTO #7602]
> > > > 
> > > > Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
> > > > ---
> > > > 
> > > >  scripts/lib/devtool/standard.py | 9 +++--
> > > >  1 file changed, 7 insertions(+), 2 deletions(-)
> > > > 
> > > > diff --git a/scripts/lib/devtool/standard.py
> > > > b/scripts/lib/devtool/standard.py index 6b85c8c..78b0d27 100644
> > > > --- a/scripts/lib/devtool/standard.py
> > > > +++ b/scripts/lib/devtool/standard.py
> > > > 
> > > > @@ -558,8 +558,13 @@ def modify(args, config, basepath, workspace):
> > > >  if not os.path.exists(appendpath):
> > > >  os.makedirs(appendpath)
> > > >  
> > > >  with open(appendfile, 'w') as f:
> > > > -f.write('FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"\n\n')
> > > > -f.write('inherit externalsrc\n')
> > > > +f.write('FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"\n')
> > > > +# Local files can be modified/tracked in separate subdir under
> > > > srctree +# Mostly useful for packages with S != WORKDIR
> > > > +f.write('FILESPATH_prepend := "%s:"\n' %
> > > > +os.path.join(srctree, 'local-files'))
> > > 
> > > Shouldn't this directory be named "oe-local-files"?
> > 
> > Argh, sorry, I had missed this email earlier. Yes, it should be
> > 'oe-local-files'. An updated patch with a fix is available at:
> > 
> > http://cgit.openembedded.org/openembedded-core-contrib/log/?h=marquiz/devtoo
> > l/localfiles
> 
> OK, great, but the commit message still has "local-files".

Oh my, you're right. Yet another version of the patch in the same
location:
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=marquiz/devtool/localfiles


Thank you for your patience,
  Markus



-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] devtool: update-recipe: enable var history tracking

[Markus Lehtonen]

Enable variable history tracking so that the variables are updated in
the correct file - i.e. in the file they are already defined.

[YOCTO #7715]

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 scripts/lib/devtool/__init__.py | 4 ++--
 scripts/lib/devtool/standard.py | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/scripts/lib/devtool/__init__.py b/scripts/lib/devtool/__init__.py
index 7b1ab11..844aa12 100644
--- a/scripts/lib/devtool/__init__.py
+++ b/scripts/lib/devtool/__init__.py
@@ -96,7 +96,7 @@ def exec_fakeroot(d, cmd, **kwargs):
 newenv[splitval[0]] = splitval[1]
 return subprocess.call("%s %s" % (fakerootcmd, cmd), env=newenv, **kwargs)
 
-def setup_tinfoil(config_only=False, basepath=None):
+def setup_tinfoil(config_only=False, basepath=None, tracking=False):
 """Initialize tinfoil api from bitbake"""
 import scriptpath
 orig_cwd = os.path.abspath(os.curdir)
@@ -108,7 +108,7 @@ def setup_tinfoil(config_only=False, basepath=None):
 sys.exit(1)
 
 import bb.tinfoil
-tinfoil = bb.tinfoil.Tinfoil()
+tinfoil = bb.tinfoil.Tinfoil(tracking=tracking)
 tinfoil.prepare(config_only)
 tinfoil.logger.setLevel(logger.getEffectiveLevel())
 os.chdir(orig_cwd)
diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index 1dcf7cd..686c9d9 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -797,7 +797,7 @@ def update_recipe(args, config, basepath, workspace):
 raise DevtoolError('conf/layer.conf not found in bbappend '
'destination layer "%s"' % args.append)
 
-tinfoil = setup_tinfoil(basepath=basepath)
+tinfoil = setup_tinfoil(basepath=basepath, tracking=True)
 
 rd = parse_recipe(config, tinfoil, args.recipename, True)
 if not rd:
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v3 03/10] oe-selftest: devtool: add method for checking workspace dir

[Markus Lehtonen]

On Tue, 2015-09-29 at 13:57 +0300, Markus Lehtonen wrote:
> Hi,
> 
> 
> On Mon, 2015-09-28 at 15:25 -0500, Leonardo Sandoval wrote:
> > 
> > On 09/24/2015 06:53 AM, Markus Lehtonen wrote:
> > > In order to remove some code duplication.
> > >
> > > Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
> > > ---
> > >   meta/lib/oeqa/selftest/devtool.py | 63 
> > > +++
> > >   1 file changed, 24 insertions(+), 39 deletions(-)
> > >
> > > diff --git a/meta/lib/oeqa/selftest/devtool.py 
> > > b/meta/lib/oeqa/selftest/devtool.py
> > > index 3a8168c..b8b872c 100644
> > > --- a/meta/lib/oeqa/selftest/devtool.py
> > > +++ b/meta/lib/oeqa/selftest/devtool.py
> > > @@ -84,11 +84,18 @@ class DevtoolBase(oeSelfTest):
> > >
> > >   class DevtoolTests(DevtoolBase):
> > >
> > > +def _get_workspace_dir(self):
> > > +"""Get workspace directory"""
> > > +workspacedir = os.path.join(self.builddir, 'workspace')
> > > +self.assertTrue(not os.path.exists(workspacedir),
> > > +'This test cannot be run with a workspace 
> > > directory '
> > > +'under the build directory')
> > > +return workspacedir
> > > +
> > >   @testcase(1158)
> > >   def test_create_workspace(self):
> > >   # Check preconditions
> > > -workspacedir = os.path.join(self.builddir, 'workspace')
> > > -self.assertTrue(not os.path.exists(workspacedir), 'This test 
> > > cannot be run with a workspace directory under the build directory')
> > > +workspacedir = self._get_workspace_dir()
> > 
> > If all tests are using workspacedir, I believe it make sense to have a 
> > setUp method and setting workspacedir there:
> > 
> > .
> >  def setUp(self):
> > self.workspacedir = # the _get_workspace_dir body code goes here
> > .
> > .
> 
> Good point! Yes, I think this check is in every single test case so a
> setup() method is nicer.

An updated patchset is available at:
  git://git.openembedded.org/openembedded-core-contrib
marquiz/devtool/localfiles

http://cgit.openembedded.org/openembedded-core-contrib/log/?h=marquiz/devtool/localfiles



Thanks,
  Markus



> 
> > >   result = runCmd('bitbake-layers show-layers')
> > >   self.assertTrue('/workspace' not in result.output, 'This test 
> > > cannot be run with a workspace layer in bblayers.conf')
> > >   # Try creating a workspace layer with a specific path
> > > @@ -109,9 +116,7 @@ class DevtoolTests(DevtoolBase):
> > >
> > >   @testcase(1159)
> > >   def test_devtool_add(self):
> > > -# Check preconditions
> > > -workspacedir = os.path.join(self.builddir, 'workspace')
> > > -self.assertTrue(not os.path.exists(workspacedir), 'This test 
> > > cannot be run with a workspace directory under the build directory')
> > > +workspacedir = self._get_workspace_dir()
> > >   # Fetch source
> > >   tempdir = tempfile.mkdtemp(prefix='devtoolqa')
> > >   self.track_for_cleanup(tempdir)
> > > @@ -144,9 +149,7 @@ class DevtoolTests(DevtoolBase):
> > >
> > >   @testcase(1162)
> > >   def test_devtool_add_library(self):
> > > -# Check preconditions
> > > -workspacedir = os.path.join(self.builddir, 'workspace')
> > > -self.assertTrue(not os.path.exists(workspacedir), 'This test 
> > > cannot be run with a workspace directory under the build directory')
> > > +workspacedir = self._get_workspace_dir()
> > >   # We don't have the ability to pick up this dependency 
> > > automatically yet...
> > >   bitbake('libusb1')
> > >   # Fetch source
> > > @@ -185,9 +188,7 @@ class DevtoolTests(DevtoolBase):
> > >
> > >   @testcase(1160)
> > >   def test_devtool_add_fetch(self):
> > > -# Check preconditions
> > > -workspacedir = os.path.join(self.builddir, 'workspace')
> > > -self.assertTrue(not os.path.exists(workspacedir), 'This test 
> > > cannot be run with a workspace directory under the build directory')
> > > +workspacedir = self._get_workspace_dir()
> > >   # Fetch source
> > >   temp

Re: [OE-core] [PATCH v3 03/10] oe-selftest: devtool: add method for checking workspace dir

[Markus Lehtonen]

Hi,


On Mon, 2015-09-28 at 15:25 -0500, Leonardo Sandoval wrote:
> 
> On 09/24/2015 06:53 AM, Markus Lehtonen wrote:
> > In order to remove some code duplication.
> >
> > Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
> > ---
> >   meta/lib/oeqa/selftest/devtool.py | 63 
> > +++
> >   1 file changed, 24 insertions(+), 39 deletions(-)
> >
> > diff --git a/meta/lib/oeqa/selftest/devtool.py 
> > b/meta/lib/oeqa/selftest/devtool.py
> > index 3a8168c..b8b872c 100644
> > --- a/meta/lib/oeqa/selftest/devtool.py
> > +++ b/meta/lib/oeqa/selftest/devtool.py
> > @@ -84,11 +84,18 @@ class DevtoolBase(oeSelfTest):
> >
> >   class DevtoolTests(DevtoolBase):
> >
> > +def _get_workspace_dir(self):
> > +"""Get workspace directory"""
> > +workspacedir = os.path.join(self.builddir, 'workspace')
> > +self.assertTrue(not os.path.exists(workspacedir),
> > +'This test cannot be run with a workspace 
> > directory '
> > +'under the build directory')
> > +return workspacedir
> > +
> >   @testcase(1158)
> >   def test_create_workspace(self):
> >   # Check preconditions
> > -workspacedir = os.path.join(self.builddir, 'workspace')
> > -self.assertTrue(not os.path.exists(workspacedir), 'This test 
> > cannot be run with a workspace directory under the build directory')
> > +workspacedir = self._get_workspace_dir()
> 
> If all tests are using workspacedir, I believe it make sense to have a 
> setUp method and setting workspacedir there:
> 
> .
>  def setUp(self):
>   self.workspacedir = # the _get_workspace_dir body code goes here
> .
> .

Good point! Yes, I think this check is in every single test case so a
setup() method is nicer.


Thanks,
   Markus




> >   result = runCmd('bitbake-layers show-layers')
> >   self.assertTrue('/workspace' not in result.output, 'This test 
> > cannot be run with a workspace layer in bblayers.conf')
> >   # Try creating a workspace layer with a specific path
> > @@ -109,9 +116,7 @@ class DevtoolTests(DevtoolBase):
> >
> >   @testcase(1159)
> >   def test_devtool_add(self):
> > -# Check preconditions
> > -workspacedir = os.path.join(self.builddir, 'workspace')
> > -self.assertTrue(not os.path.exists(workspacedir), 'This test 
> > cannot be run with a workspace directory under the build directory')
> > +workspacedir = self._get_workspace_dir()
> >   # Fetch source
> >   tempdir = tempfile.mkdtemp(prefix='devtoolqa')
> >   self.track_for_cleanup(tempdir)
> > @@ -144,9 +149,7 @@ class DevtoolTests(DevtoolBase):
> >
> >   @testcase(1162)
> >   def test_devtool_add_library(self):
> > -# Check preconditions
> > -workspacedir = os.path.join(self.builddir, 'workspace')
> > -self.assertTrue(not os.path.exists(workspacedir), 'This test 
> > cannot be run with a workspace directory under the build directory')
> > +workspacedir = self._get_workspace_dir()
> >   # We don't have the ability to pick up this dependency 
> > automatically yet...
> >   bitbake('libusb1')
> >   # Fetch source
> > @@ -185,9 +188,7 @@ class DevtoolTests(DevtoolBase):
> >
> >   @testcase(1160)
> >   def test_devtool_add_fetch(self):
> > -# Check preconditions
> > -workspacedir = os.path.join(self.builddir, 'workspace')
> > -self.assertTrue(not os.path.exists(workspacedir), 'This test 
> > cannot be run with a workspace directory under the build directory')
> > +workspacedir = self._get_workspace_dir()
> >   # Fetch source
> >   tempdir = tempfile.mkdtemp(prefix='devtoolqa')
> >   self.track_for_cleanup(tempdir)
> > @@ -232,9 +233,7 @@ class DevtoolTests(DevtoolBase):
> >
> >   @testcase(1161)
> >   def test_devtool_add_fetch_git(self):
> > -# Check preconditions
> > -workspacedir = os.path.join(self.builddir, 'workspace')
> > -self.assertTrue(not os.path.exists(workspacedir), 'This test 
> > cannot be run with a workspace directory under the build directory')
> > +workspacedir = self._get_workspace_dir()
> >   # Fetch source
> >   tempdir = tempfile.mkdtemp(prefix='devtoolqa')
> >   self.track_for_c

[OE-core] [PATCH v3 07/10] devtool: update_recipe: refactor patch generation

[Markus Lehtonen]

Implement new function that handles patch file generation. The new
function also does the discovery of new, updated and deleted patches.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 scripts/lib/devtool/standard.py | 119 
 1 file changed, 72 insertions(+), 47 deletions(-)

diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index 1154030..7c8e447 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -25,6 +25,7 @@ import logging
 import argparse
 import scriptutils
 import errno
+from collections import OrderedDict
 from devtool import exec_build_env_command, setup_tinfoil, 
check_workspace_recipe, use_external_build, setup_git_repo, DevtoolError
 from devtool import parse_recipe
 
@@ -590,11 +591,55 @@ def _remove_patch_files(args, patches, destpath):
 if ose.errno != errno.ENOTEMPTY:
 raise
 
+
+def _export_patches(srctree, rd, start_rev, destdir):
+"""Export patches from srctree to given location.
+   Returns three-tuple of dicts:
+ 1. updated - patches that already exist in SRCURI
+ 2. added - new patches that don't exist in SRCURI
+ 3  removed - patches that exist in SRCURI but not in exported patches
+  In each dict the key is the 'basepath' of the URI and value is the
+  absolute path to the existing file in recipe space (if any).
+"""
+import oe.recipeutils
+from oe.patch import GitApplyTree
+updated = OrderedDict()
+added = OrderedDict()
+seqpatch_re = re.compile('^([0-9]{4}-)?(.+)')
+
+existing_patches = dict((os.path.basename(path), path) for path in
+oe.recipeutils.get_recipe_patches(rd))
+
+# Generate patches from Git
+GitApplyTree.extractPatches(srctree, start_rev, destdir)
+
+new_patches = sorted(os.listdir(destdir))
+for new_patch in new_patches:
+# Strip numbering from patch names. If it's a git sequence named patch,
+# the numbers might not match up since we are starting from a different
+# revision This does assume that people are using unique shortlog
+# values, but they ought to be anyway...
+new_basename = seqpatch_re.match(new_patch).group(2)
+found = False
+for old_patch in existing_patches:
+old_basename = seqpatch_re.match(old_patch).group(2)
+if new_basename == old_basename:
+updated[new_patch] = existing_patches.pop(old_patch)
+found = True
+# Rename patch files
+if new_patch != old_patch:
+os.rename(os.path.join(destdir, new_patch),
+  os.path.join(destdir, old_patch))
+break
+if not found:
+added[new_patch] = None
+return (updated, added, existing_patches)
+
+
 def _update_recipe_srcrev(args, srctree, rd, config_data):
 """Implement the 'srcrev' mode of update-recipe"""
 import bb
 import oe.recipeutils
-from oe.patch import GitApplyTree
 
 recipefile = rd.getVar('FILE', True)
 logger.info('Updating SRCREV in recipe %s' % os.path.basename(recipefile))
@@ -621,12 +666,10 @@ def _update_recipe_srcrev(args, srctree, rd, config_data):
 old_srcrev = (rd.getVar('SRCREV', False) or '')
 tempdir = tempfile.mkdtemp(prefix='devtool')
 try:
-GitApplyTree.extractPatches(srctree, old_srcrev, tempdir)
-newpatches = os.listdir(tempdir)
-for patch in existing_patches:
-patchfile = os.path.basename(patch)
-if patchfile in newpatches:
-removepatches.append(patch)
+upd_p, new_p, del_p = _export_patches(srctree, rd, old_srcrev,
+  tempdir)
+# Remove "overlapping" patches
+removepatches = upd_p.values()
 finally:
 shutil.rmtree(tempdir)
 
@@ -654,7 +697,6 @@ def _update_recipe_patch(args, config, srctree, rd, 
config_data):
 """Implement the 'patch' mode of update-recipe"""
 import bb
 import oe.recipeutils
-from oe.patch import GitApplyTree
 
 recipefile = rd.getVar('FILE', True)
 append = os.path.join(config.workspace_path, 'appends', '%s.bbappend' %
@@ -677,40 +719,27 @@ def _update_recipe_patch(args, config, srctree, rd, 
config_data):
 # Get all patches from source tree and check if any should be removed
 tempdir = tempfile.mkdtemp(prefix='devtool')
 try:
-GitApplyTree.extractPatches(srctree, initial_rev, tempdir)
-# Strip numbering from patch names. If it's a git sequence named
-# patch, the numbers might not match up since we are starting from
-# a diffe

[OE-core] [PATCH v3 01/10] recipeutils: implement get_recipe_local_files()

[Markus Lehtonen]

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/lib/oe/recipeutils.py | 16 
 1 file changed, 16 insertions(+)

diff --git a/meta/lib/oe/recipeutils.py b/meta/lib/oe/recipeutils.py
index 35b88d3..56056db 100644
--- a/meta/lib/oe/recipeutils.py
+++ b/meta/lib/oe/recipeutils.py
@@ -336,6 +336,22 @@ def copy_recipe_files(d, tgt_dir, whole_dir=False, 
download=True):
 return remotes
 
 
+def get_recipe_local_files(d, patches=False):
+"""Get a list of local files in SRC_URI within a recipe."""
+uris = (d.getVar('SRC_URI', True) or "").split()
+fetch = bb.fetch2.Fetch(uris, d)
+ret = {}
+for uri in uris:
+if fetch.ud[uri].type == 'file':
+if (not patches and
+bb.utils.exec_flat_python_func('patch_path', uri, fetch, 
'')):
+continue
+# Skip files that are referenced by absolute path
+if not os.path.isabs(fetch.ud[uri].basepath):
+ret[fetch.ud[uri].basepath] = fetch.localpath(uri)
+return ret
+
+
 def get_recipe_patches(d):
 """Get a list of the patches included in SRC_URI within a recipe."""
 patchfiles = []
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v3 03/10] oe-selftest: devtool: add method for checking workspace dir

[Markus Lehtonen]

In order to remove some code duplication.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/lib/oeqa/selftest/devtool.py | 63 +++
 1 file changed, 24 insertions(+), 39 deletions(-)

diff --git a/meta/lib/oeqa/selftest/devtool.py 
b/meta/lib/oeqa/selftest/devtool.py
index 3a8168c..b8b872c 100644
--- a/meta/lib/oeqa/selftest/devtool.py
+++ b/meta/lib/oeqa/selftest/devtool.py
@@ -84,11 +84,18 @@ class DevtoolBase(oeSelfTest):
 
 class DevtoolTests(DevtoolBase):
 
+def _get_workspace_dir(self):
+"""Get workspace directory"""
+workspacedir = os.path.join(self.builddir, 'workspace')
+self.assertTrue(not os.path.exists(workspacedir),
+'This test cannot be run with a workspace directory '
+'under the build directory')
+return workspacedir
+
 @testcase(1158)
 def test_create_workspace(self):
 # Check preconditions
-workspacedir = os.path.join(self.builddir, 'workspace')
-self.assertTrue(not os.path.exists(workspacedir), 'This test cannot be 
run with a workspace directory under the build directory')
+workspacedir = self._get_workspace_dir()
 result = runCmd('bitbake-layers show-layers')
 self.assertTrue('/workspace' not in result.output, 'This test cannot 
be run with a workspace layer in bblayers.conf')
 # Try creating a workspace layer with a specific path
@@ -109,9 +116,7 @@ class DevtoolTests(DevtoolBase):
 
 @testcase(1159)
 def test_devtool_add(self):
-# Check preconditions
-workspacedir = os.path.join(self.builddir, 'workspace')
-self.assertTrue(not os.path.exists(workspacedir), 'This test cannot be 
run with a workspace directory under the build directory')
+workspacedir = self._get_workspace_dir()
 # Fetch source
 tempdir = tempfile.mkdtemp(prefix='devtoolqa')
 self.track_for_cleanup(tempdir)
@@ -144,9 +149,7 @@ class DevtoolTests(DevtoolBase):
 
 @testcase(1162)
 def test_devtool_add_library(self):
-# Check preconditions
-workspacedir = os.path.join(self.builddir, 'workspace')
-self.assertTrue(not os.path.exists(workspacedir), 'This test cannot be 
run with a workspace directory under the build directory')
+workspacedir = self._get_workspace_dir()
 # We don't have the ability to pick up this dependency automatically 
yet...
 bitbake('libusb1')
 # Fetch source
@@ -185,9 +188,7 @@ class DevtoolTests(DevtoolBase):
 
 @testcase(1160)
 def test_devtool_add_fetch(self):
-# Check preconditions
-workspacedir = os.path.join(self.builddir, 'workspace')
-self.assertTrue(not os.path.exists(workspacedir), 'This test cannot be 
run with a workspace directory under the build directory')
+workspacedir = self._get_workspace_dir()
 # Fetch source
 tempdir = tempfile.mkdtemp(prefix='devtoolqa')
 self.track_for_cleanup(tempdir)
@@ -232,9 +233,7 @@ class DevtoolTests(DevtoolBase):
 
 @testcase(1161)
 def test_devtool_add_fetch_git(self):
-# Check preconditions
-workspacedir = os.path.join(self.builddir, 'workspace')
-self.assertTrue(not os.path.exists(workspacedir), 'This test cannot be 
run with a workspace directory under the build directory')
+workspacedir = self._get_workspace_dir()
 # Fetch source
 tempdir = tempfile.mkdtemp(prefix='devtoolqa')
 self.track_for_cleanup(tempdir)
@@ -284,9 +283,7 @@ class DevtoolTests(DevtoolBase):
 
 @testcase(1164)
 def test_devtool_modify(self):
-# Check preconditions
-workspacedir = os.path.join(self.builddir, 'workspace')
-self.assertTrue(not os.path.exists(workspacedir), 'This test cannot be 
run with a workspace directory under the build directory')
+workspacedir = self._get_workspace_dir()
 # Clean up anything in the workdir/sysroot/sstate cache
 bitbake('mdadm -c cleansstate')
 # Try modifying a recipe
@@ -336,9 +333,7 @@ class DevtoolTests(DevtoolBase):
 
 @testcase(1166)
 def test_devtool_modify_invalid(self):
-# Check preconditions
-workspacedir = os.path.join(self.builddir, 'workspace')
-self.assertTrue(not os.path.exists(workspacedir), 'This test cannot be 
run with a workspace directory under the build directory')
+workspacedir = self._get_workspace_dir()
 # Try modifying some recipes
 tempdir = tempfile.mkdtemp(prefix='devtoolqa')
 self.track_for_cleanup(tempdir)
@@ -400,8 +395,7 @@ class DevtoolTests(DevtoolBase):
 @testcase(1165)
 def test_devtool_modify_git(self):
 # Check preconditions
-workspacedir = os.path.join(self.builddir, 'workspace')
-self.assertTrue(not os.path.exists(workspacedir), 'This test cannot be 

[OE-core] [PATCH v3 10/10] devtool: modify: make bitbake use local files from srctree

[Markus Lehtonen]

This change makes it possible to have local files (non-remote SRC_URI
files, i.e. files that are located in the "recipe space") under the
srctree even if S!=WORKDIR. The files must be placed under the
'local-files' subdirectory.

Complements the previous patch that imports local files into srctree.

[YOCTO #7602]

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 scripts/lib/devtool/standard.py | 9 +++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index 6b85c8c..78b0d27 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -558,8 +558,13 @@ def modify(args, config, basepath, workspace):
 if not os.path.exists(appendpath):
 os.makedirs(appendpath)
 with open(appendfile, 'w') as f:
-f.write('FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"\n\n')
-f.write('inherit externalsrc\n')
+f.write('FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"\n')
+# Local files can be modified/tracked in separate subdir under srctree
+# Mostly useful for packages with S != WORKDIR
+f.write('FILESPATH_prepend := "%s:"\n' %
+os.path.join(srctree, 'local-files'))
+
+f.write('\ninherit externalsrc\n')
 f.write('# NOTE: We use pn- overrides here to avoid affecting multiple 
variants in the case where the recipe uses BBCLASSEXTEND\n')
 f.write('EXTERNALSRC_pn-%s = "%s"\n' % (args.recipename, srctree))
 
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v3 04/10] oe-selftest: devtool: add method for checking srctree repo

[Markus Lehtonen]

Removes some code duplication.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/lib/oeqa/selftest/devtool.py | 49 ++-
 1 file changed, 18 insertions(+), 31 deletions(-)

diff --git a/meta/lib/oeqa/selftest/devtool.py 
b/meta/lib/oeqa/selftest/devtool.py
index b8b872c..f459a6d 100644
--- a/meta/lib/oeqa/selftest/devtool.py
+++ b/meta/lib/oeqa/selftest/devtool.py
@@ -92,6 +92,17 @@ class DevtoolTests(DevtoolBase):
 'under the build directory')
 return workspacedir
 
+def _check_src_repo(self, repo_dir):
+"""Check srctree git repository"""
+self.assertTrue(os.path.isdir(os.path.join(repo_dir, '.git')),
+'git repository for external source tree not found')
+result = runCmd('git status --porcelain', cwd=repo_dir)
+self.assertEqual(result.output.strip(), "",
+ 'Created git repo is not clean')
+result = runCmd('git symbolic-ref HEAD', cwd=repo_dir)
+self.assertEqual(result.output.strip(), "refs/heads/devtool",
+ 'Wrong branch in git repo')
+
 @testcase(1158)
 def test_create_workspace(self):
 # Check preconditions
@@ -294,7 +305,6 @@ class DevtoolTests(DevtoolBase):
 self.add_command_to_tearDown('bitbake -c clean mdadm')
 result = runCmd('devtool modify mdadm -x %s' % tempdir)
 self.assertTrue(os.path.exists(os.path.join(tempdir, 'Makefile')), 
'Extracted source could not be found')
-self.assertTrue(os.path.isdir(os.path.join(tempdir, '.git')), 'git 
repository for external source tree not found')
 self.assertTrue(os.path.exists(os.path.join(workspacedir, 'conf', 
'layer.conf')), 'Workspace directory not created')
 matches = glob.glob(os.path.join(workspacedir, 'appends', 
'mdadm_*.bbappend'))
 self.assertTrue(matches, 'bbappend not created %s' % result.output)
@@ -303,10 +313,7 @@ class DevtoolTests(DevtoolBase):
 self.assertIn('mdadm', result.output)
 self.assertIn(tempdir, result.output)
 # Check git repo
-result = runCmd('git status --porcelain', cwd=tempdir)
-self.assertEqual(result.output.strip(), "", 'Created git repo is not 
clean')
-result = runCmd('git symbolic-ref HEAD', cwd=tempdir)
-self.assertEqual(result.output.strip(), "refs/heads/devtool", 'Wrong 
branch in git repo')
+self._check_src_repo(tempdir)
 # Try building
 bitbake('mdadm')
 # Try making (minor) modifications to the source
@@ -409,7 +416,6 @@ class DevtoolTests(DevtoolBase):
 self.add_command_to_tearDown('bitbake -c clean %s' % testrecipe)
 result = runCmd('devtool modify %s -x %s' % (testrecipe, tempdir))
 self.assertTrue(os.path.exists(os.path.join(tempdir, 'Makefile')), 
'Extracted source could not be found')
-self.assertTrue(os.path.isdir(os.path.join(tempdir, '.git')), 'git 
repository for external source tree not found')
 self.assertTrue(os.path.exists(os.path.join(workspacedir, 'conf', 
'layer.conf')), 'Workspace directory not created. devtool output: %s' % 
result.output)
 matches = glob.glob(os.path.join(workspacedir, 'appends', 
'mkelfimage_*.bbappend'))
 self.assertTrue(matches, 'bbappend not created')
@@ -418,10 +424,7 @@ class DevtoolTests(DevtoolBase):
 self.assertIn(testrecipe, result.output)
 self.assertIn(tempdir, result.output)
 # Check git repo
-result = runCmd('git status --porcelain', cwd=tempdir)
-self.assertEqual(result.output.strip(), "", 'Created git repo is not 
clean')
-result = runCmd('git symbolic-ref HEAD', cwd=tempdir)
-self.assertEqual(result.output.strip(), "refs/heads/devtool", 'Wrong 
branch in git repo')
+self._check_src_repo(tempdir)
 # Try building
 bitbake(testrecipe)
 
@@ -475,11 +478,7 @@ class DevtoolTests(DevtoolBase):
 # (don't bother with cleaning the recipe on teardown, we won't be 
building it)
 result = runCmd('devtool modify %s -x %s' % (testrecipe, tempdir))
 # Check git repo
-self.assertTrue(os.path.isdir(os.path.join(tempdir, '.git')), 'git 
repository for external source tree not found')
-result = runCmd('git status --porcelain', cwd=tempdir)
-self.assertEqual(result.output.strip(), "", 'Created git repo is not 
clean')
-result = runCmd('git symbolic-ref HEAD', cwd=tempdir)
-self.assertEqual(result.output.strip(), "refs/heads/devtool", 'Wrong 
branch in git repo')
+self._check_src_repo(tempdir)
 # Add a couple of commits
 # FIXME: this only tests adding, need to also test update and remove
 result = runCmd('echo "Additional line" >> README', cwd=te

[OE-core] [PATCH v3 06/10] devtool: update-recipe: add new patches in correct order

[Markus Lehtonen]

When adding multiple new patches append them to SRC_URI in correct order
so that they apply correctly.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 scripts/lib/devtool/standard.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index 96b271c..1154030 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -701,7 +701,7 @@ def _update_recipe_patch(args, config, srctree, rd, 
config_data):
 updatepatches = False
 updaterecipe = False
 destpath = None
-newpatches = os.listdir(tempdir)
+newpatches = sorted(os.listdir(tempdir))
 if args.append:
 patchfiles = {}
 for patch in existing_patches:
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v3 00/10] devtool: improve handling of local source files

[Markus Lehtonen]

Third iteration of the patchset to improve handling of local source files.
Functional changes after v2:
- name of the subdirectory containing local sources in srctree is now named
  'oe-local-files'
- oe-local-files directory is not automatically committed into srctree when
  doing extract operation
- devtool update-recipe now handles adding and deleting local source files


The following changes since commit 2ad7308ee7166641eff99f3b9fe6794de143f6bc:

  oeqa/utils/qemurunner.py: Remove duplicate message on LoggingThread start 
(2015-09-22 18:13:02 +0100)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib 
marquiz/devtool/localfiles
  
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=marquiz/devtool/localfiles
for you to fetch changes up to 986784d368c297d3b3fc04e40dd922adcca00505:

  devtool: modify: make bitbake use local files from srctree (2015-09-24 
14:29:31 +0300)


Markus Lehtonen (10):
  recipeutils: implement get_recipe_local_files()
  oe.patch.GitApplyTree: add paths argument to extractPatches
  oe-selftest: devtool: add method for checking workspace dir
  oe-selftest: devtool: add method for checking srctree repo
  oe-selftest: devtool: add method for checking repo status
  devtool: update-recipe: add new patches in correct order
  devtool: update_recipe: refactor patch generation
  devtool: file mover function that creates target dir
  devtool: better support for local source files
  devtool: modify: make bitbake use local files from srctree

 meta/lib/oe/patch.py  |   5 +-
 meta/lib/oe/recipeutils.py|  16 ++
 meta/lib/oeqa/selftest/devtool.py | 272 +---
 scripts/lib/devtool/__init__.py   |  10 +-
 scripts/lib/devtool/standard.py   | 422 ++
 5 files changed, 468 insertions(+), 257 deletions(-)

-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v3 09/10] devtool: better support for local source files

[Markus Lehtonen]

* extract: Copy all local source files (i.e.  non-compressed/non-arcived
  SRC_URI files that have file:// URI prefix) - excluding patches - to
  the srctree repository. The files will be placed in a subdirectory
  called 'oe-local-files'. The oe-local-files directory is not committed
  to the Git repository, but, marked to be ignored by a .gitignore file.
  The developer can manually add and commit the files to Git if the
  changes to them need to be tracked.

  Before this patch, local source files (were copied (and committed) to
  the srctree repository only in some special cases (basically when
  S=WORKDIR) when doing devtool-extract. For most of the packages local
  files were not copied at all.

* update-recipe: This patch causes the local files to be 'synced' from
  the srctree (i.e. from the 'oe-local-files' subdirectory) to the
  layer.  Being 'synced' means that in addition to copying modified
  files over the original sources, devtool will also handle removing and
  adding local source files and updating the recipe accordingly.  We
  don't want to create patches against the local source files but rather
  update them directly.  Thus, 'oe-local-file' directory is ignored in
  patch generation when doing update-recipe, even if committed to Git.
  This functionality is only enabled if the 'oe-local-files' directory
  is present in srctree.

[YOCTO #7602]

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/lib/oeqa/selftest/devtool.py |  73 +
 scripts/lib/devtool/__init__.py   |  10 +-
 scripts/lib/devtool/standard.py   | 314 ++
 3 files changed, 299 insertions(+), 98 deletions(-)

diff --git a/meta/lib/oeqa/selftest/devtool.py 
b/meta/lib/oeqa/selftest/devtool.py
index a893ed3..59f0fae 100644
--- a/meta/lib/oeqa/selftest/devtool.py
+++ b/meta/lib/oeqa/selftest/devtool.py
@@ -738,6 +738,79 @@ class DevtoolTests(DevtoolBase):
 self.assertEqual(expectedlines, f.readlines())
 # Deleting isn't expected to work under these circumstances
 
+@testcase(1173)
+def test_devtool_update_recipe_local_files(self):
+"""Check that local source files are copied over instead of patched"""
+workspacedir = self._get_workspace_dir()
+testrecipe = 'makedevs'
+recipefile = get_bb_var('FILE', testrecipe)
+# Setup srctree for modifying the recipe
+tempdir = tempfile.mkdtemp(prefix='devtoolqa')
+self.track_for_cleanup(tempdir)
+self.track_for_cleanup(workspacedir)
+self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+# (don't bother with cleaning the recipe on teardown, we won't be
+# building it)
+result = runCmd('devtool modify %s -x %s' % (testrecipe, tempdir))
+# Check git repo
+self._check_src_repo(tempdir)
+# Edit / commit local source
+runCmd('echo "/* Foobar */" >> oe-local-files/makedevs.c', cwd=tempdir)
+runCmd('echo "Foo" > oe-local-files/new-local', cwd=tempdir)
+runCmd('echo "Bar" > new-file', cwd=tempdir)
+runCmd('git add new-file', cwd=tempdir)
+runCmd('git commit -m "Add new file"', cwd=tempdir)
+self.add_command_to_tearDown('cd %s; git clean -fd .; git checkout .' %
+ os.path.dirname(recipefile))
+runCmd('devtool update-recipe %s' % testrecipe)
+expected_status = [(' M', '.*/%s$' % os.path.basename(recipefile)),
+   (' M', '.*/makedevs/makedevs.c$'),
+   ('??', '.*/makedevs/new-local$'),
+   ('??', '.*/makedevs/0001-Add-new-file.patch$')]
+self._check_repo_status(os.path.dirname(recipefile), expected_status)
+
+@testcase(1174)
+def test_devtool_update_recipe_local_files_2(self):
+"""Check local source files support when oe-local-files is in Git"""
+workspacedir = self._get_workspace_dir()
+testrecipe = 'lzo'
+recipefile = get_bb_var('FILE', testrecipe)
+# Setup srctree for modifying the recipe
+tempdir = tempfile.mkdtemp(prefix='devtoolqa')
+self.track_for_cleanup(tempdir)
+self.track_for_cleanup(workspacedir)
+self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+result = runCmd('devtool modify %s -x %s' % (testrecipe, tempdir))
+# Check git repo
+self._check_src_repo(tempdir)
+# Add oe-local-files to Git
+runCmd('rm oe-local-files/.gitignore', cwd=tempdir)
+runCmd('git add oe-local-files', cwd=tempdir)
+runCmd('git commit -m "Add local sources"', cwd=tempdir)
+# Edit / commit local sources
+runCmd('echo "# Foobar" >> oe-local-files/acinclude.m4', cwd=tempdir)
+runCmd('git comm

[OE-core] [PATCH v3 08/10] devtool: file mover function that creates target dir

[Markus Lehtonen]

Helper function for replacing a pattern like:
  target_dir = os.path.dirname(target)
  bb.utils.mkdirhier(target_dir)
  shutil.move(source, target)

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 scripts/lib/devtool/standard.py | 16 +++-
 1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index 7c8e447..efa6fd1 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -174,6 +174,13 @@ def _check_compatible_recipe(pn, d):
"from working. You will need to disable this "
"first." % pn)
 
+def _move_file(src, dst):
+"""Move a file. Creates all the directory components of destination 
path."""
+dst_d = os.path.dirname(dst)
+if dst_d:
+bb.utils.mkdirhier(dst_d)
+shutil.move(src, dst)
+
 def _ls_tree(directory):
 """Recursive listing of files in a directory"""
 ret = []
@@ -330,9 +337,8 @@ def _extract_source(srctree, keep_temp, devbranch, d):
 crd.setVar('S', srcsubdir)
 # Move source files to S
 for path in src_files:
-tgt_dir = os.path.join(srcsubdir, os.path.dirname(path))
-bb.utils.mkdirhier(tgt_dir)
-shutil.move(os.path.join(workdir, path), tgt_dir)
+_move_file(os.path.join(workdir, path),
+   os.path.join(srcsubdir, path))
 elif os.path.dirname(srcsubdir) != workdir:
 # Handle if S is set to a subdirectory of the source
 srcsubdir = os.path.join(workdir, os.path.relpath(srcsubdir, 
workdir).split(os.sep)[0])
@@ -893,8 +899,8 @@ def reset(args, config, basepath, workspace):
 for root, dirs, files in os.walk(origdir):
 for fn in files:
 logger.warn('Preserving %s in %s' % (fn, preservepath))
-bb.utils.mkdirhier(preservepath)
-shutil.move(os.path.join(origdir, fn), 
os.path.join(preservepath, fn))
+_move_file(os.path.join(origdir, fn),
+   os.path.join(preservepath, fn))
 for dn in dirs:
 os.rmdir(os.path.join(root, dn))
 os.rmdir(origdir)
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v3 02/10] oe.patch.GitApplyTree: add paths argument to extractPatches

[Markus Lehtonen]

Makes it possible to define which paths are included in the patches.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/lib/oe/patch.py | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oe/patch.py b/meta/lib/oe/patch.py
index 7441214..2bf501e 100644
--- a/meta/lib/oe/patch.py
+++ b/meta/lib/oe/patch.py
@@ -337,12 +337,15 @@ class GitApplyTree(PatchTree):
 return (tmpfile, cmd)
 
 @staticmethod
-def extractPatches(tree, startcommit, outdir):
+def extractPatches(tree, startcommit, outdir, paths=None):
 import tempfile
 import shutil
 tempdir = tempfile.mkdtemp(prefix='oepatch')
 try:
 shellcmd = ["git", "format-patch", startcommit, "-o", tempdir]
+if paths:
+shellcmd.append('--')
+shellcmd.extend(paths)
 out = runcmd(["sh", "-c", " ".join(shellcmd)], tree)
 if out:
 for srcfile in out.split():
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v3 05/10] oe-selftest: devtool: add method for checking repo status

[Markus Lehtonen]

New method for checking the status of the working tree of a repository.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/lib/oeqa/selftest/devtool.py | 87 ---
 1 file changed, 35 insertions(+), 52 deletions(-)

diff --git a/meta/lib/oeqa/selftest/devtool.py 
b/meta/lib/oeqa/selftest/devtool.py
index f459a6d..a893ed3 100644
--- a/meta/lib/oeqa/selftest/devtool.py
+++ b/meta/lib/oeqa/selftest/devtool.py
@@ -103,6 +103,23 @@ class DevtoolTests(DevtoolBase):
 self.assertEqual(result.output.strip(), "refs/heads/devtool",
  'Wrong branch in git repo')
 
+def _check_repo_status(self, repo_dir, expected_status):
+"""Check the worktree status of a repository"""
+result = runCmd('git status . --porcelain',
+cwd=repo_dir)
+for line in result.output.splitlines():
+for ind, (f_status, fn_re) in enumerate(expected_status):
+if re.match(fn_re, line[3:]):
+if f_status != line[:2]:
+self.fail('Unexpected status in line: %s' % line)
+expected_status.pop(ind)
+break
+else:
+self.fail('Unexpected modified file in line: %s' % line)
+if expected_status:
+self.fail('Missing file changes: %s' % expected_status)
+
+
 @testcase(1158)
 def test_create_workspace(self):
 # Check preconditions
@@ -468,8 +485,7 @@ class DevtoolTests(DevtoolBase):
 recipefile = get_bb_var('FILE', testrecipe)
 src_uri = get_bb_var('SRC_URI', testrecipe)
 self.assertNotIn('git://', src_uri, 'This test expects the %s recipe 
to NOT be a git recipe' % testrecipe)
-result = runCmd('git status . --porcelain', 
cwd=os.path.dirname(recipefile))
-self.assertEqual(result.output.strip(), "", '%s recipe is not clean' % 
testrecipe)
+self._check_repo_status(os.path.dirname(recipefile), [])
 # First, modify a recipe
 tempdir = tempfile.mkdtemp(prefix='devtoolqa')
 self.track_for_cleanup(tempdir)
@@ -488,19 +504,10 @@ class DevtoolTests(DevtoolBase):
 result = runCmd('git commit -m "Add a new file"', cwd=tempdir)
 self.add_command_to_tearDown('cd %s; rm %s/*.patch; git checkout %s 
%s' % (os.path.dirname(recipefile), testrecipe, testrecipe, 
os.path.basename(recipefile)))
 result = runCmd('devtool update-recipe %s' % testrecipe)
-result = runCmd('git status . --porcelain', 
cwd=os.path.dirname(recipefile))
-self.assertNotEqual(result.output.strip(), "", '%s recipe should be 
modified' % testrecipe)
-status = result.output.splitlines()
-self.assertEqual(len(status), 3, 'Less/more files modified than 
expected. Entire status:\n%s' % result.output)
-for line in status:
-if line.endswith('0001-Change-the-README.patch'):
-self.assertEqual(line[:3], '?? ', 'Unexpected status in line: 
%s' % line)
-elif line.endswith('0002-Add-a-new-file.patch'):
-self.assertEqual(line[:3], '?? ', 'Unexpected status in line: 
%s' % line)
-elif re.search('%s_[^_]*.bb$' % testrecipe, line):
-self.assertEqual(line[:3], ' M ', 'Unexpected status in line: 
%s' % line)
-else:
-raise AssertionError('Unexpected modified file in status: %s' 
% line)
+expected_status = [(' M', '.*/%s$' % os.path.basename(recipefile)),
+   ('??', '.*/0001-Change-the-README.patch$'),
+   ('??', '.*/0002-Add-a-new-file.patch$')]
+self._check_repo_status(os.path.dirname(recipefile), expected_status)
 
 @testcase(1172)
 def test_devtool_update_recipe_git(self):
@@ -515,8 +522,7 @@ class DevtoolTests(DevtoolBase):
 if entry.startswith('file://') and entry.endswith('.patch'):
 patches.append(entry[7:].split(';')[0])
 self.assertGreater(len(patches), 0, 'The %s recipe does not appear to 
contain any patches, so this test will not be effective' % testrecipe)
-result = runCmd('git status . --porcelain', 
cwd=os.path.dirname(recipefile))
-self.assertEqual(result.output.strip(), "", '%s recipe is not clean' % 
testrecipe)
+self._check_repo_status(os.path.dirname(recipefile), [])
 # First, modify a recipe
 tempdir = tempfile.mkdtemp(prefix='devtoolqa')
 self.track_for_cleanup(tempdir)
@@ -535,19 +541,10 @@ class DevtoolTests(DevtoolBase):
 result = runCmd('git commit -m "Add a new file"', cwd=tempdir)
 self.add_command_to_tearDown('cd %s; rm -rf %s; git checkout %s %s' % 
(os.path.dirname(recipefile), testrecipe, testrecipe, 
os.path.basename(recipefile)))
 result = runCmd('devtool upd

[OE-core] [PATCH] devtool: upgrade: use shutil.move instead of os.rename

[Markus Lehtonen]

Rename fails over filesystem boundaries.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 scripts/lib/devtool/upgrade.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/lib/devtool/upgrade.py b/scripts/lib/devtool/upgrade.py
index e74e795..18e010c 100644
--- a/scripts/lib/devtool/upgrade.py
+++ b/scripts/lib/devtool/upgrade.py
@@ -53,7 +53,7 @@ def _copy_source_code(orig, dest):
 dest_dir = os.path.join(dest, os.path.dirname(path))
 bb.utils.mkdirhier(dest_dir)
 dest_path = os.path.join(dest, path)
-os.rename(os.path.join(orig, path), dest_path)
+shutil.move(os.path.join(orig, path), dest_path)
 
 def _get_checksums(rf):
 import re
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] devtool: second fix for running from a different directory

[Markus Lehtonen]

Do not change change current working directory permanently, but, only
for the duration of tinfoil initialization instead. The previous fix
caused very unintuitive behavior where using relative paths were solved
with respect to the builddir instead of the current working directory.
E.g. calling "devtool extract zlib ./zlib" would always create create
srctree in ${TOPDIR}/zlib, independent of the users cwd.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 scripts/devtool| 5 +
 scripts/lib/devtool/__init__.py| 6 +-
 scripts/lib/devtool/build-image.py | 6 +++---
 scripts/lib/devtool/deploy.py  | 2 +-
 scripts/lib/devtool/package.py | 2 +-
 scripts/lib/devtool/standard.py| 8 
 scripts/lib/devtool/upgrade.py | 2 +-
 7 files changed, 16 insertions(+), 15 deletions(-)

diff --git a/scripts/devtool b/scripts/devtool
index 87df951..e4d9db3 100755
--- a/scripts/devtool
+++ b/scripts/devtool
@@ -221,9 +221,6 @@ def main():
 if not config.read():
 return -1
 
-# We need to be in this directory or we won't be able to initialise tinfoil
-os.chdir(basepath)
-
 bitbake_subdir = config.get('General', 'bitbake_subdir', '')
 if bitbake_subdir:
 # Normally set for use within the SDK
@@ -244,7 +241,7 @@ def main():
 scriptutils.logger_setup_color(logger, global_args.color)
 
 if global_args.bbpath is None:
-tinfoil = setup_tinfoil(config_only=True)
+tinfoil = setup_tinfoil(config_only=True, basepath=basepath)
 global_args.bbpath = tinfoil.config_data.getVar('BBPATH', True)
 else:
 tinfoil = None
diff --git a/scripts/lib/devtool/__init__.py b/scripts/lib/devtool/__init__.py
index 404d3e6..37745fd 100644
--- a/scripts/lib/devtool/__init__.py
+++ b/scripts/lib/devtool/__init__.py
@@ -96,9 +96,12 @@ def exec_fakeroot(d, cmd, **kwargs):
 newenv[splitval[0]] = splitval[1]
 return subprocess.call("%s %s" % (fakerootcmd, cmd), env=newenv, **kwargs)
 
-def setup_tinfoil(config_only=False):
+def setup_tinfoil(config_only=False, basepath=None):
 """Initialize tinfoil api from bitbake"""
 import scriptpath
+orig_cwd = os.path.abspath(os.curdir)
+if basepath:
+os.chdir(basepath)
 bitbakepath = scriptpath.add_bitbake_lib_path()
 if not bitbakepath:
 logger.error("Unable to find bitbake by searching parent directory of 
this script or PATH")
@@ -108,6 +111,7 @@ def setup_tinfoil(config_only=False):
 tinfoil = bb.tinfoil.Tinfoil()
 tinfoil.prepare(config_only)
 tinfoil.logger.setLevel(logger.getEffectiveLevel())
+os.chdir(orig_cwd)
 return tinfoil
 
 def get_recipe_file(cooker, pn):
diff --git a/scripts/lib/devtool/build-image.py 
b/scripts/lib/devtool/build-image.py
index 2c01428..f1a4017 100644
--- a/scripts/lib/devtool/build-image.py
+++ b/scripts/lib/devtool/build-image.py
@@ -25,10 +25,10 @@ from devtool import exec_build_env_command, setup_tinfoil, 
parse_recipe
 
 logger = logging.getLogger('devtool')
 
-def _get_recipes(workspace, config):
+def _get_recipes(workspace, config, basepath):
 """Get list of target recipes from the workspace."""
 result = []
-tinfoil = setup_tinfoil()
+tinfoil = setup_tinfoil(basepath=basepath)
 for recipe in workspace:
 data = parse_recipe(config, tinfoil, recipe, True)
 if 'class-target' in data.getVar('OVERRIDES', True).split(':'):
@@ -51,7 +51,7 @@ def build_image(args, config, basepath, workspace):
 if os.path.isfile(appendfile):
 os.unlink(appendfile)
 
-recipes = _get_recipes(workspace, config)
+recipes = _get_recipes(workspace, config, basepath)
 if recipes:
 with open(appendfile, 'w') as afile:
 # include selected recipes into the image
diff --git a/scripts/lib/devtool/deploy.py b/scripts/lib/devtool/deploy.py
index fa93adf..5bed72b 100644
--- a/scripts/lib/devtool/deploy.py
+++ b/scripts/lib/devtool/deploy.py
@@ -41,7 +41,7 @@ def deploy(args, config, basepath, workspace):
 deploy_dir = os.path.join(basepath, 'target_deploy', args.target)
 deploy_file = os.path.join(deploy_dir, args.recipename + '.list')
 
-tinfoil = setup_tinfoil()
+tinfoil = setup_tinfoil(basepath=basepath)
 try:
 rd = oe.recipeutils.parse_recipe_simple(tinfoil.cooker, 
args.recipename, tinfoil.config_data)
 except Exception as e:
diff --git a/scripts/lib/devtool/package.py b/scripts/lib/devtool/package.py
index 3a7a36b..f3ab809 100644
--- a/scripts/lib/devtool/package.py
+++ b/scripts/lib/devtool/package.py
@@ -36,7 +36,7 @@ def package(args, config, basepath, workspace):
 
 image_pkgtype = config.get('Package', 'image_pkgtype', '')
 if not image_pkgtype:
-tinfoil = setup_tinfoil()
+tinfoil = setup_tinfoil(basepath=basepath)
 try:
 tinfoil.prepare

Re: [OE-core] [PATCH] rpm: search for gpg if gpg2 is not found

[Markus Lehtonen]

Hi Mark,

On Tue, 2015-09-15 at 08:58 -0500, Mark Hatle wrote:
> On 9/15/15 8:05 AM, Markus Lehtonen wrote:
> > Some (host) systems only have a binary named 'gpg' (e.g. Fedora) while
> > some only have 'gpg2' (Ubuntu) and others have both of them (openSUSE).
> > Currently the behavior of rpm-native with regards to GnuPG depends on
> > the host platform: rpm(-native) is configured to use GnuPG binary of the
> > host system if 'gpg2' is found in $PATH. Otherwise, rpm(-native) will
> > default to using '%{_bindir}/gpg2' which will be pointing to a sysroot
> > binary which usually does not exist.
> > 
> > This patch changes rpm to look for both 'gpg' and 'gpg2' when searching
> > for the GnuPG binary in PATH. This makes possible to create signed RPM
> > packages on different host platforms, using the GnuPG binary of the
> > host, without the need to explicitly define the gpg binary in bitbake
> > configuration (via GPG_BIN variable).
> > 
> > [YOCTO #8134]
> 
> The only concern I have with this change is that it may affect both native and
> target RPM.  Please verify that the target RPM settings are still correct.

The target rpm is not affected. Autotools in bitbake environment will
not find host system binaries.



> FYI, the value isn't used for anything but the initial setup of some RPM macro
> scripts.  Typically I tell uses that they are responsible for providing the
> proper ~/.oerpmmacros file in order to instruct RPM where some of these types 
> of
> tools are present.  My file for instance:
> 
> %__gpg gpg2
> %_gpg_name Test RPM Signing Key

Yes, this is possible, as well as usage of the GPG_BIN configuration
variable. I just find it nicer for the user if signing works without
similarly on all host platforms (without the need for these settings on
some hosts, like Ubuntu).


Thanks,
  Markus


> > Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
> > ---
> >  .../configure.ac-check-for-both-gpg2-and-gpg.patch | 29 
> > ++
> >  meta/recipes-devtools/rpm/rpm_5.4.14.bb|  1 +
> >  2 files changed, 30 insertions(+)
> >  create mode 100644 
> > meta/recipes-devtools/rpm/rpm/configure.ac-check-for-both-gpg2-and-gpg.patch
> > 
> > diff --git 
> > a/meta/recipes-devtools/rpm/rpm/configure.ac-check-for-both-gpg2-and-gpg.patch
> >  
> > b/meta/recipes-devtools/rpm/rpm/configure.ac-check-for-both-gpg2-and-gpg.patch
> > new file mode 100644
> > index 000..f5db167
> > --- /dev/null
> > +++ 
> > b/meta/recipes-devtools/rpm/rpm/configure.ac-check-for-both-gpg2-and-gpg.patch
> > @@ -0,0 +1,29 @@
> > +configure.ac: search for both gpg2 and gpg
> > +
> > +On some platforms the GnuPG binary is named 'gpg2' whereas others have 
> > 'gpp'.
> > +This patch increases compatibility by searching for 'gpg' in addition to
> > +'gpg2'.
> > +
> > +Upstream-Status: Pending
> > +
> > +Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
> > +---
> > + configure.ac | 2 +-
> > + 1 file changed, 1 insertion(+), 1 deletion(-)
> > +
> > +diff --git a/configure.ac b/configure.ac
> > +index 6746b4c..f6922ae 100644
> > +--- a/configure.ac
> >  b/configure.ac
> > +@@ -562,7 +562,7 @@ AC_PATH_PROG(__DIFF, diff, /bin/diff, $MYPATH)
> > + AC_PATH_PROG(__DITTO, ditto, %{_bindir}/ditto, $MYPATH)
> > + AC_PATH_PROG(__FILE, file, %{_bindir}/file, $MYPATH)
> > + AC_PATH_PROG(__GIT, git, %{_bindir}/git, $MYPATH)
> > +-AC_PATH_PROG(__GPG, gpg2, %{_bindir}/gpg2, $MYPATH)
> > ++AC_PATH_PROGS(__GPG, [gpg2 gpg], %{_bindir}/gpg2, $MYPATH)
> > + AC_PATH_PROG(__GSR, gsr, %{_bindir}/gsr, $MYPATH)
> > + AC_PATH_PROG(__GST_INSPECT, gst-inspect-0.10, 
> > %{_bindir}/gst-inspect-0.10, $MYPATH)
> > + AC_PATH_PROG(__GZIP, gzip, /bin/gzip, $MYPATH)
> > +-- 
> > +2.1.4
> > +
> > diff --git a/meta/recipes-devtools/rpm/rpm_5.4.14.bb 
> > b/meta/recipes-devtools/rpm/rpm_5.4.14.bb
> > index 1f9a4bd..b450c6f 100644
> > --- a/meta/recipes-devtools/rpm/rpm_5.4.14.bb
> > +++ b/meta/recipes-devtools/rpm/rpm_5.4.14.bb
> > @@ -98,6 +98,7 @@ SRC_URI = 
> > "http://www.rpm5.org/files/rpm/rpm-5.4/rpm-5.4.14-0.20131024.src.rpm;e
> >file://rpm-check-rootpath-reasonableness.patch \
> >file://rpm-macros.in-disable-external-key-server.patch \
> >file://rpm-opendb-before-verifyscript-to-avoid-null-point.patch \
> > +  file://configure.ac-check-for-both-gpg2-and-gpg.patch \
> >   "
> >  
> >  # Uncomment the following line to enable platform score debugging
> > 
> 



-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] rpm: search for gpg if gpg2 is not found

[Markus Lehtonen]

Hi Gary,

On Tue, 2015-09-15 at 07:11 -0600, Gary Thomas wrote:
> On 2015-09-15 07:05, Markus Lehtonen wrote:
> > Some (host) systems only have a binary named 'gpg' (e.g. Fedora) while
> > some only have 'gpg2' (Ubuntu) and others have both of them (openSUSE).
> > Currently the behavior of rpm-native with regards to GnuPG depends on
> > the host platform: rpm(-native) is configured to use GnuPG binary of the
> > host system if 'gpg2' is found in $PATH. Otherwise, rpm(-native) will
> > default to using '%{_bindir}/gpg2' which will be pointing to a sysroot
> > binary which usually does not exist.
> >
> > This patch changes rpm to look for both 'gpg' and 'gpg2' when searching
> > for the GnuPG binary in PATH. This makes possible to create signed RPM
> > packages on different host platforms, using the GnuPG binary of the
> > host, without the need to explicitly define the gpg binary in bitbake
> > configuration (via GPG_BIN variable).
> >
> > [YOCTO #8134]
> >
> > Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
> > ---
> >   .../configure.ac-check-for-both-gpg2-and-gpg.patch | 29 
> > ++
> >   meta/recipes-devtools/rpm/rpm_5.4.14.bb|  1 +
> >   2 files changed, 30 insertions(+)
> >   create mode 100644 
> > meta/recipes-devtools/rpm/rpm/configure.ac-check-for-both-gpg2-and-gpg.patch
> >
> > diff --git 
> > a/meta/recipes-devtools/rpm/rpm/configure.ac-check-for-both-gpg2-and-gpg.patch
> >  
> > b/meta/recipes-devtools/rpm/rpm/configure.ac-check-for-both-gpg2-and-gpg.patch
> > new file mode 100644
> > index 000..f5db167
> > --- /dev/null
> > +++ 
> > b/meta/recipes-devtools/rpm/rpm/configure.ac-check-for-both-gpg2-and-gpg.patch
> > @@ -0,0 +1,29 @@
> > +configure.ac: search for both gpg2 and gpg
> > +
> > +On some platforms the GnuPG binary is named 'gpg2' whereas others have 
> > 'gpp'.
> 
> ^^^
> Typo?

Oh, indeed. Well spotted, thanks! An updated patch with the typo fixed
is found here:
git://git.openembedded.org/openembedded-core-contrib marquiz/rpmsign
http://cgit.openembedded.org/openembedded-core-contrib/commit/?h=marquiz/rpmsign=b1d86be082e3cdb9dc1f3885e7b18c56a6094cc3


Cheers,
  Markus



> 
> > +This patch increases compatibility by searching for 'gpg' in addition to
> > +'gpg2'.
> > +
> > +Upstream-Status: Pending
> > +
> > +Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
> > +---
> > + configure.ac | 2 +-
> > + 1 file changed, 1 insertion(+), 1 deletion(-)
> > +
> > +diff --git a/configure.ac b/configure.ac
> > +index 6746b4c..f6922ae 100644
> > +--- a/configure.ac
> >  b/configure.ac
> > +@@ -562,7 +562,7 @@ AC_PATH_PROG(__DIFF, diff, /bin/diff, $MYPATH)
> > + AC_PATH_PROG(__DITTO, ditto, %{_bindir}/ditto, $MYPATH)
> > + AC_PATH_PROG(__FILE, file, %{_bindir}/file, $MYPATH)
> > + AC_PATH_PROG(__GIT, git, %{_bindir}/git, $MYPATH)
> > +-AC_PATH_PROG(__GPG, gpg2, %{_bindir}/gpg2, $MYPATH)
> > ++AC_PATH_PROGS(__GPG, [gpg2 gpg], %{_bindir}/gpg2, $MYPATH)
> > + AC_PATH_PROG(__GSR, gsr, %{_bindir}/gsr, $MYPATH)
> > + AC_PATH_PROG(__GST_INSPECT, gst-inspect-0.10, 
> > %{_bindir}/gst-inspect-0.10, $MYPATH)
> > + AC_PATH_PROG(__GZIP, gzip, /bin/gzip, $MYPATH)
> > +--
> > +2.1.4
> > +
> > diff --git a/meta/recipes-devtools/rpm/rpm_5.4.14.bb 
> > b/meta/recipes-devtools/rpm/rpm_5.4.14.bb
> > index 1f9a4bd..b450c6f 100644
> > --- a/meta/recipes-devtools/rpm/rpm_5.4.14.bb
> > +++ b/meta/recipes-devtools/rpm/rpm_5.4.14.bb
> > @@ -98,6 +98,7 @@ SRC_URI = 
> > "http://www.rpm5.org/files/rpm/rpm-5.4/rpm-5.4.14-0.20131024.src.rpm;e
> >file://rpm-check-rootpath-reasonableness.patch \
> >file://rpm-macros.in-disable-external-key-server.patch \
> >file://rpm-opendb-before-verifyscript-to-avoid-null-point.patch \
> > +  file://configure.ac-check-for-both-gpg2-and-gpg.patch \
> >   "
> >
> >   # Uncomment the following line to enable platform score debugging
> >
> 
> -- 
> 
> Gary Thomas |  Consulting for the
> MLB Associates  |Embedded world
> 



-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] rpm: search for gpg if gpg2 is not found

[Markus Lehtonen]

Some (host) systems only have a binary named 'gpg' (e.g. Fedora) while
some only have 'gpg2' (Ubuntu) and others have both of them (openSUSE).
Currently the behavior of rpm-native with regards to GnuPG depends on
the host platform: rpm(-native) is configured to use GnuPG binary of the
host system if 'gpg2' is found in $PATH. Otherwise, rpm(-native) will
default to using '%{_bindir}/gpg2' which will be pointing to a sysroot
binary which usually does not exist.

This patch changes rpm to look for both 'gpg' and 'gpg2' when searching
for the GnuPG binary in PATH. This makes possible to create signed RPM
packages on different host platforms, using the GnuPG binary of the
host, without the need to explicitly define the gpg binary in bitbake
configuration (via GPG_BIN variable).

[YOCTO #8134]

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 .../configure.ac-check-for-both-gpg2-and-gpg.patch | 29 ++
 meta/recipes-devtools/rpm/rpm_5.4.14.bb|  1 +
 2 files changed, 30 insertions(+)
 create mode 100644 
meta/recipes-devtools/rpm/rpm/configure.ac-check-for-both-gpg2-and-gpg.patch

diff --git 
a/meta/recipes-devtools/rpm/rpm/configure.ac-check-for-both-gpg2-and-gpg.patch 
b/meta/recipes-devtools/rpm/rpm/configure.ac-check-for-both-gpg2-and-gpg.patch
new file mode 100644
index 000..f5db167
--- /dev/null
+++ 
b/meta/recipes-devtools/rpm/rpm/configure.ac-check-for-both-gpg2-and-gpg.patch
@@ -0,0 +1,29 @@
+configure.ac: search for both gpg2 and gpg
+
+On some platforms the GnuPG binary is named 'gpg2' whereas others have 'gpp'.
+This patch increases compatibility by searching for 'gpg' in addition to
+'gpg2'.
+
+Upstream-Status: Pending
+
+Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
+---
+ configure.ac | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/configure.ac b/configure.ac
+index 6746b4c..f6922ae 100644
+--- a/configure.ac
 b/configure.ac
+@@ -562,7 +562,7 @@ AC_PATH_PROG(__DIFF, diff, /bin/diff, $MYPATH)
+ AC_PATH_PROG(__DITTO, ditto, %{_bindir}/ditto, $MYPATH)
+ AC_PATH_PROG(__FILE, file, %{_bindir}/file, $MYPATH)
+ AC_PATH_PROG(__GIT, git, %{_bindir}/git, $MYPATH)
+-AC_PATH_PROG(__GPG, gpg2, %{_bindir}/gpg2, $MYPATH)
++AC_PATH_PROGS(__GPG, [gpg2 gpg], %{_bindir}/gpg2, $MYPATH)
+ AC_PATH_PROG(__GSR, gsr, %{_bindir}/gsr, $MYPATH)
+ AC_PATH_PROG(__GST_INSPECT, gst-inspect-0.10, %{_bindir}/gst-inspect-0.10, 
$MYPATH)
+ AC_PATH_PROG(__GZIP, gzip, /bin/gzip, $MYPATH)
+-- 
+2.1.4
+
diff --git a/meta/recipes-devtools/rpm/rpm_5.4.14.bb 
b/meta/recipes-devtools/rpm/rpm_5.4.14.bb
index 1f9a4bd..b450c6f 100644
--- a/meta/recipes-devtools/rpm/rpm_5.4.14.bb
+++ b/meta/recipes-devtools/rpm/rpm_5.4.14.bb
@@ -98,6 +98,7 @@ SRC_URI = 
"http://www.rpm5.org/files/rpm/rpm-5.4/rpm-5.4.14-0.20131024.src.rpm;e
   file://rpm-check-rootpath-reasonableness.patch \
   file://rpm-macros.in-disable-external-key-server.patch \
   file://rpm-opendb-before-verifyscript-to-avoid-null-point.patch \
+  file://configure.ac-check-for-both-gpg2-and-gpg.patch \
  "
 
 # Uncomment the following line to enable platform score debugging
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] devtool: update-recipe: get srcuri parameters with decodeurl()

[Markus Lehtonen]

Use already existing bb.fetch.decodeurl() for getting the parameters for
a URI. This is more fault tolerant and maintainable.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 scripts/lib/devtool/standard.py | 7 ++-
 1 file changed, 2 insertions(+), 5 deletions(-)

diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index 4702491..20a7fe2 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -766,11 +766,8 @@ def _guess_recipe_update_mode(srctree, rdata):
 # Just use the first URI for now
 uri = git_uris[0]
 # Check remote branch
-upstr_branch = 'master'
-for paramdef in uri.split(';')[1:]:
-name, value = paramdef.split('=', 1)
-if name == 'branch':
-upstr_branch = value
+params = bb.fetch.decodeurl(uri)[5]
+upstr_branch = params['branch'] if 'branch' in params else 'master'
 # Check if current branch HEAD is found in upstream branch
 stdout, _ = bb.process.run('git rev-parse HEAD', cwd=srctree)
 head_rev = stdout.rstrip()
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 0/4] devtool: better support for kernel

[Markus Lehtonen]

This patchset contains few patches to improve the devtool support for kernel
recipes. With these patches it is possible build a modified kernel and also
build other packages (e.g. external modules) against it. The first three
patches are actually a bit more generic patches that make externalsrc behave
better with kernel recipes.

The following changes since commit 8402958cd2cb87b8283c8ee4e2d08e1a6717d67a:

  pseudo_1.7.3.bb: New version of pseudo (2015-09-06 15:24:28 +0100)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib marquiz/devtool/kernel
  
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=marquiz/devtool/kernel

for you to fetch changes up to a7aeb51b63040a2783d60c45d9249daa47c2fde3:

  devtool: modify: enable do_shared_workdir for kernel (2015-09-08 13:29:39 
+0300)


Markus Lehtonen (4):
  extrernalsrc.bbclass: treat kernel meta like local source
  kernel-yocto.bbclass: do_kernel_metadata depends on do_unpack
  kernel.bbclass: do not mv/link sources when externalsrc enabled
  devtool: modify: enable do_shared_workdir for kernel

 meta/classes/externalsrc.bbclass  | 12 
 meta/classes/kernel-yocto.bbclass |  2 +-
 meta/classes/kernel.bbclass   | 10 +++---
 scripts/lib/devtool/standard.py   |  2 +-
 4 files changed, 17 insertions(+), 9 deletions(-)

-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 4/4] devtool: modify: enable do_shared_workdir for kernel

[Markus Lehtonen]

Do not put 'do_shared_workdir' into SRCTREECOVEREDTASKS when creating
bbappend for kernel packages. This will allow building packages that
depend on the shared build artifacts of kernel.

[YOCTO #6658]

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 scripts/lib/devtool/standard.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index e59fb5e..ee00c6d 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -493,7 +493,7 @@ def modify(args, config, basepath, workspace):
 f.write('EXTERNALSRC_BUILD_pn-%s = "%s"\n' % (args.recipename, 
srctree))
 
 if bb.data.inherits_class('kernel', rd):
-f.write('SRCTREECOVEREDTASKS = "do_validate_branches 
do_kernel_checkout do_shared_workdir do_fetch do_unpack"\n')
+f.write('SRCTREECOVEREDTASKS = "do_validate_branches 
do_kernel_checkout do_fetch do_unpack"\n')
 if initial_rev:
 f.write('\n# initial_rev: %s\n' % initial_rev)
 for commit in commits:
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] externalsrc.bbclass: better filtering of cleandirs

[Markus Lehtonen]

We really do not want our (external) source tree to be removed. There
might be multiple values in the 'cleandirs' varflag pointing to our
source tree - causing it to be wiped out. This patch improves the
filtering of 'cleandirs' by examining the expanded values inside it. Any
(expanded) values pointing to our source tree get removed.

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/classes/externalsrc.bbclass | 18 +++---
 1 file changed, 7 insertions(+), 11 deletions(-)

diff --git a/meta/classes/externalsrc.bbclass b/meta/classes/externalsrc.bbclass
index 8f7f479..499688b 100644
--- a/meta/classes/externalsrc.bbclass
+++ b/meta/classes/externalsrc.bbclass
@@ -58,18 +58,14 @@ python () {
 d.appendVarFlag(task, "lockfiles", " ${S}/singletask.lock")
 
 # We do not want our source to be wiped out, ever (kernel.bbclass 
does this for do_clean)
-cleandirs = d.getVarFlag(task, 'cleandirs', False)
-if cleandirs:
-cleandirs = cleandirs.split()
-setvalue = False
-if '${S}' in cleandirs:
-cleandirs.remove('${S}')
+cleandirs = (d.getVarFlag(task, 'cleandirs', False) or '').split()
+setvalue = False
+for cleandir in cleandirs[:]:
+if d.expand(cleandir) == externalsrc:
+cleandirs.remove(cleandir)
 setvalue = True
-if externalsrcbuild == externalsrc and '${B}' in cleandirs:
-cleandirs.remove('${B}')
-setvalue = True
-if setvalue:
-d.setVarFlag(task, 'cleandirs', ' '.join(cleandirs))
+if setvalue:
+d.setVarFlag(task, 'cleandirs', ' '.join(cleandirs))
 
 fetch_tasks = ['do_fetch', 'do_unpack']
 # If we deltask do_patch, there's no dependency to ensure do_unpack 
gets run, so add one
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/4] kernel-yocto.bbclass: do_kernel_metadata depends on do_unpack

[Markus Lehtonen]

Make sure that 'do_unpack' is executed before 'do_kernel_metadata'.
Enabling externalsrc for kernel disables 'do_validate_branches' task
which caused 'do_kernel_metadata' to fail as the dependency chain to
'do_unpack' got broken.

[YOCTO #6658]

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/classes/kernel-yocto.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/kernel-yocto.bbclass 
b/meta/classes/kernel-yocto.bbclass
index 231e08d..325f94c 100644
--- a/meta/classes/kernel-yocto.bbclass
+++ b/meta/classes/kernel-yocto.bbclass
@@ -228,7 +228,7 @@ do_kernel_checkout() {
 do_kernel_checkout[dirs] = "${S}"
 
 addtask kernel_checkout before do_kernel_metadata after do_unpack
-addtask kernel_metadata after do_validate_branches before do_patch
+addtask kernel_metadata after do_validate_branches do_unpack before do_patch
 do_kernel_metadata[depends] = "kern-tools-native:do_populate_sysroot"
 
 do_kernel_configme[dirs] += "${S} ${B}"
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/4] extrernalsrc.bbclass: treat kernel meta like local source

[Markus Lehtonen]

Kernel metadata repository needs to be fetched/unpacked into the work
directory in order to build the kernel. Sources pointing to a remote
location are skipped by externalsrc by default which caused kernel build
to fail (because of remote kernel-meta was not made availeble). This
patch will make kernel-meta always available.

[YOCTO #6658]

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/classes/externalsrc.bbclass | 12 
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/meta/classes/externalsrc.bbclass b/meta/classes/externalsrc.bbclass
index 8f7f479..37c6320 100644
--- a/meta/classes/externalsrc.bbclass
+++ b/meta/classes/externalsrc.bbclass
@@ -36,11 +36,15 @@ python () {
 else:
 d.setVar('B', '${WORKDIR}/${BPN}-${PV}/')
 
-srcuri = (d.getVar('SRC_URI', True) or '').split()
 local_srcuri = []
-for uri in srcuri:
-if uri.startswith('file://'):
-local_srcuri.append(uri)
+fetch = bb.fetch2.Fetch((d.getVar('SRC_URI', True) or '').split(), d)
+for url in fetch.urls:
+url_data = fetch.ud[url]
+parm = url_data.parm
+if (url_data.type == 'file' or
+'type' in parm and parm['type'] == 'kmeta'):
+local_srcuri.append(url)
+
 d.setVar('SRC_URI', ' '.join(local_srcuri))
 
 if '{SRCPV}' in d.getVar('PV', False):
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/4] kernel.bbclass: do not mv/link sources when externalsrc enabled

[Markus Lehtonen]

If externalsrc is enabled the 'do_unpack' task is run if the recipe has
some local source files. In the case of kernel recipe this caused the
(externalsrc) source tree to be moved/symlinked. This patch prevents the
behaviour, making sure the source tree is not moved around when
externalsrc is enabled. Instead of moving the source tree,
STAGING_KERNEL_DIR will be a symlink to it.

[YOCTO #6658]

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 meta/classes/kernel.bbclass | 10 +++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 919293e..dfbdfd2 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -68,9 +68,13 @@ base_do_unpack_append () {
 if s != kernsrc:
 bb.utils.mkdirhier(kernsrc)
 bb.utils.remove(kernsrc, recurse=True)
-import subprocess
-subprocess.call(d.expand("mv ${S} ${STAGING_KERNEL_DIR}"), shell=True)
-os.symlink(kernsrc, s)
+if d.getVar("EXTERNALSRC", True):
+# With EXTERNALSRC S will not be wiped so we can symlink to it
+os.symlink(s, kernsrc)
+else:
+import shutil
+shutil.move(s, kernsrc)
+os.symlink(kernsrc, s)
 }
 
 inherit kernel-arch deploy
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] devtool: update-recipe: better 'auto' mode

[Markus Lehtonen]

Enhance the logic behind the 'auto' mode a bit by only updating the
SRCREV if the changes are already found upstream. The logic is simple:
update SRCREV only if the current local HEAD commit is found in the
remote branch (i.e. 'origin/'). Otherwise resort to
patching.

[YOCTO #7907]

Signed-off-by: Markus Lehtonen <markus.lehto...@linux.intel.com>
---
 scripts/lib/devtool/standard.py | 34 +++---
 1 file changed, 27 insertions(+), 7 deletions(-)

diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index de7afd9..6f3bb8e 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -725,6 +725,31 @@ def _update_recipe_patch(args, config, srctree, rd, 
config_data):
 
 _remove_patch_files(args, removepatches, destpath)
 
+def _guess_recipe_update_mode(srctree, rdata):
+"""Guess the recipe update mode to use"""
+src_uri = (rdata.getVar('SRC_URI', False) or '').split()
+git_uris = [uri for uri in src_uri if uri.startswith('git://')]
+if not git_uris:
+return 'patch'
+# Just use the first URI for now
+uri = git_uris[0]
+# Check remote branch
+upstr_branch = 'master'
+for paramdef in uri.split(';')[1:]:
+name, value = paramdef.split('=', 1)
+if name == 'branch':
+upstr_branch = value
+# Check if current branch HEAD is found in upstream branch
+stdout, _ = bb.process.run('git rev-parse HEAD', cwd=srctree)
+head_rev = stdout.rstrip()
+stdout, _ = bb.process.run('git branch -r --contains %s' % head_rev,
+   cwd=srctree)
+remote_brs = [branch.strip() for branch in stdout.splitlines()]
+if 'origin/' + upstr_branch in remote_brs:
+return 'srcrev'
+
+return 'patch'
+
 def update_recipe(args, config, basepath, workspace):
 """Entry point for the devtool 'update-recipe' subcommand"""
 if not args.recipename in workspace:
@@ -745,17 +770,12 @@ def update_recipe(args, config, basepath, workspace):
 if not rd:
 return 1
 
-orig_src_uri = rd.getVar('SRC_URI', False) or ''
+srctree = workspace[args.recipename]['srctree']
 if args.mode == 'auto':
-if 'git://' in orig_src_uri:
-mode = 'srcrev'
-else:
-mode = 'patch'
+mode = _guess_recipe_update_mode(srctree, rd)
 else:
 mode = args.mode
 
-srctree = workspace[args.recipename]['srctree']
-
 if mode == 'srcrev':
 _update_recipe_srcrev(args, srctree, rd, tinfoil.config_data)
 elif mode == 'patch':
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2 0/4] Sign packages in RPM feeds

[Markus Lehtonen]

Hi,

On 27/08/15 12:31, Markus Lehtonen
openembedded-core-boun...@lists.openembedded.org on behalf of
markus.lehto...@linux.intel.com wrote:

Second iteration of my patchset. I tried to address the issues pointed
out by
Mark:
1. The gpg key is not imported to the (temporary) rpm databases used by
   createrepo. Instead, createrepo is patched to ignore signature
   verification altogether.
2. There is a new optional config variable GPG_BIN which can be used to
   define the gpg binary used for signing.
3. The filename of the public keys (published with the package feed and
   depoyed into the target rootfs as part of os-release package) is now
   postfixed with -${DISTRO_VERSION}.

[YOCTO #8134]

*** BLURB HERE ***

Markus Lehtonen (4):
  createrepo: disable RPM signature validation
  package_rpm: support signing of rpm packages
  os-release: add the public package-signing key
  package_manager: support for signed RPM package feeds

 meta/classes/package_rpm.bbclass   |  5 ++
 meta/classes/sign_rpm.bbclass  | 60
++
 meta/lib/oe/package_manager.py | 40 +++
 meta/recipes-core/os-release/os-release.bb | 11 
 ...dumpMetadata-disable-signature-validation.patch | 31 +++
 .../createrepo/createrepo_0.4.11.bb| 17 +++---
 6 files changed, 156 insertions(+), 8 deletions(-)
 create mode 100644 meta/classes/sign_rpm.bbclass
 create mode 100644
meta/recipes-support/createrepo/createrepo/dumpMetadata-disable-signature-
validation.patch

Please use a slightly updated version of the patchset found here:
  git://git.openembedded.org/openembedded-core-contrib marquiz/rpmsign
  
http://git.openembedded.org/openembedded-core-contrib/log/?h=marquiz/rpmsig
n


I noticed some typos in the commit messages of my v2 patchset (GPG_CMD vs.
GPG_BIN). Also, I added a comment header to the sign_rpm.bbclass file.


Thanks,
  Markus


-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 3/3] package_manager: support for signed RPM package feeds

[Markus Lehtonen]

Hi,

On 27/08/15 15:03, Mark Hatle mark.ha...@windriver.com wrote:

On 8/26/15 11:27 PM, Markus Lehtonen wrote:
 Hi Mark,
 
 On 26/08/15 18:10, Mark Hatle mark.ha...@windriver.com wrote:
 
 On 8/26/15 6:18 AM, Markus Lehtonen wrote:
 This change makes it possible to create GPG signed RPM package feeds -
 i.e. package feed with GPG signed metadata (repodata). All deployed
RPM
 repositories will be signed and the GPG public key is copied to the
rpm
 deployment directory.

 In order to enable the new feature one needs to define four variables
in
 bitbake configuration.
 1. 'PACKAGE_FEED_SIGN = 1' enabling the feature
 2. 'PACKAGE_FEED_GPG_NAME = key_id' defining the GPG key to use
for
signing
 3. 'PACKAGE_FEED_GPG_PASSPHRASE_FILE = path_to_file' pointing to a
file containing the passphrase for the secret signing key
 4. 'PACKAGE_FEED_GPG_PUBKEY = path_to_pubkey' pointing to the
corresponding public key (in armor format)

 [YOCTO #8134]

 Signed-off-by: Markus Lehtonen markus.lehto...@linux.intel.com
 ---
  meta/lib/oe/package_manager.py | 24 ++--
  1 file changed, 22 insertions(+), 2 deletions(-)

 diff --git a/meta/lib/oe/package_manager.py
 b/meta/lib/oe/package_manager.py
 index 753b3eb..5d7ef54 100644
 --- a/meta/lib/oe/package_manager.py
 +++ b/meta/lib/oe/package_manager.py
 @@ -113,8 +113,15 @@ class RpmIndexer(Indexer):
  rpm_pubkey = self.d.getVar('RPM_GPG_PUBKEY', True)
  else:
  rpm_pubkey = None
 +if self.d.getVar('PACKAGE_FEED_SIGN', True) == '1':
 +pkgfeed_gpg_name = self.d.getVar('PACKAGE_FEED_GPG_NAME',
 True)
 +pkgfeed_gpg_pass =
 self.d.getVar('PACKAGE_FEED_GPG_PASSPHRASE_FILE', True)
 +else:
 +pkgfeed_gpg_name = None
 +pkgfeed_gpg_pass = None
  
  index_cmds = []
 +repo_sign_cmds = []
  key_import_cmds = []
  rpm_dirs_found = False
  for arch in archs:
 @@ -126,10 +133,16 @@ class RpmIndexer(Indexer):
  continue
  
  if rpm_pubkey:
 -key_import_cmds.append(%s --define '_dbpath %s'
 --import %s %
 +key_import_cmds.append(%s --dbpath '%s' --import
%s %
 (rpm_bin, dbpath, rpm_pubkey))
  index_cmds.append(%s --dbpath %s --update -q %s % \
   (rpm_createrepo, dbpath, arch_dir))
 +if pkgfeed_gpg_name:
 +repomd_file = os.path.join(arch_dir, 'repodata',
 'repomd.xml')
 +gpg_cmd = gpg2 --detach-sign --armor --batch
--no-tty
 --yes  \
 +  --passphrase-file '%s' -u '%s' %s % \
 +  (pkgfeed_gpg_pass, pkgfeed_gpg_name,
 repomd_file)
 +repo_sign_cmds.append(gpg_cmd)

 I've had problems in the past hard coding 'gpg' or 'gpg2' as the name
to
 use.

 Can we get this to be dynamic.. even if it's a system level define for
 what
 GPG/PGP program to use?
 
 OK, I can introduce a new variable for defining this.
 
 
 Also I'd forgotten about it until there.  RPM has a similar variable to
 define
 the GPG program to use.  So using that variable (_signature) and
 defaulting to
 the same item would be a good idea.
 
 I think this is not feasible as we're actually using the host's gpg(2)
 here and rpm might not even be available.

Sorry I listed the wrong variable..  What I was referring to was the gpg
program.  See below..

What I'm asking for is similar to the above of replacing:

gpg_cmd = gpg2 --detach-sign --armor --batch --no-tty --yes 

with something like:

gpg_cmd = d.getVar(GPG, True) + --detach-sign --armor --batch --no-tty
--yes 

In the sections where you setup the RPM macros you would define signature
in the
same way:

(patch 1/3)

if gpg_name:
cmd += --define '%%_gpg_name %s'  % gpg_name

cmd += --define '__gpg %s' --define '%%_gpg_name %s'  % (d.getVar(GPG,
True), gpg_name)

I got the point and did something along these lines in my v2 patchset.
Although the variable name I used was GPG_BIN.

Thanks for your comments,
   Markus




--Mark

 
 Thanks,
Markus
 
 
 
 (One such reason to do this is to write a wrapper that uses an
alternative
 keychain for these keys)

  
  rpm_dirs_found = True
  
 @@ -145,10 +158,17 @@ class RpmIndexer(Indexer):
  result = oe.utils.multiprocess_exec(index_cmds, create_index)
  if result:
  bb.fatal('%s' % ('\n'.join(result)))
 -# Copy pubkey to repo
 +# Sign repomd
 +result = oe.utils.multiprocess_exec(repo_sign_cmds,
 create_index)
 +if result:
 +bb.fatal('%s' % ('\n'.join(result)))
 +# Copy pubkey(s) to repo
  if self.d.getVar('RPM_SIGN_PACKAGES', True) == '1':
  shutil.copy2(self.d.getVar('RPM_GPG_PUBKEY', True),
   os.path.join(self.deploy_dir,
 'RPM-GPG-KEY-oe'))
 +if self.d.getVar('PACKAGE_FEED_SIGN

Re: [OE-core] [PATCH 3/3] devtool: run kernel dependencies

[Markus Lehtonen]

Hi,

On 27/08/15 15:45, Paul Eggleton paul.eggle...@linux.intel.com wrote:

Hi Markus,

On Thursday 27 August 2015 14:49:50 Markus Lehtonen wrote:
 The kernel package needs kern-tools-native in order for it's
 do_kernel_metadata. Thus, devtool extract for kernel in a pristine
 environment fails. With the current bb.tinfoil implementation it is not
 possible to run arbitrary bitbake commands - e.g. run
 bitbake kern-tools-native -c populate_sysroot in our case. This patch
 implements an ugly workaround for that problem, basically by hardcoding
 this dependency and running the required bitbake task(s) before tinfoil
 is initialized.
 
 [YOCTO #6658]
 
 Signed-off-by: Markus Lehtonen markus.lehto...@linux.intel.com
 ---
  scripts/lib/devtool/standard.py | 19 ++-
  1 file changed, 18 insertions(+), 1 deletion(-)
 
 diff --git a/scripts/lib/devtool/standard.py
 b/scripts/lib/devtool/standard.py index 3725d87..bf24e32 100644
 --- a/scripts/lib/devtool/standard.py
 +++ b/scripts/lib/devtool/standard.py
 @@ -197,6 +197,8 @@ def extract(args, config, basepath, workspace):
  Entry point for the devtool 'extract' subcommand
  import bb
 
 +_check_extract_deps(config, basepath, args.recipename)
 +
  tinfoil = setup_tinfoil()
 
  rd = _parse_recipe(config, tinfoil, args.recipename, True)
 @@ -237,6 +239,20 @@ class BbTaskExecutor(object):
  self.executed.append(func)
 
 
 +def _check_extract_deps(config, basepath, recipename):
 +HACK: Ugly workaround for making sure that requirements are met
when
 +   trying to extract a package
 +tinfoil = setup_tinfoil()
 +rd = _parse_recipe(config, tinfoil, recipename, True)
 +if bb.data.inherits_class('kernel-yocto', rd):
 +tinfoil.shutdown()

Hmm, surely you need to call shutdown() in the non-kernel case as well?
Otherwise won't the lock still be in place when this function returns and
then 
setup_tinfoil() is called a second time?

Argh, I was a bit too hasty with this one. You can find a new version of
this patch attached as well as in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib
marquiz/devtool/kernel
  
http://git.openembedded.org/openembedded-core-contrib/log/?h=marquiz/devtoo
l/kernel



Thanks,
  Markus



0001-devtool-run-kernel-dependencies.patch
Description: Binary data
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/3] devtool: run kernel dependencies

[Markus Lehtonen]

The kernel package needs kern-tools-native in order for it's
do_kernel_metadata. Thus, devtool extract for kernel in a pristine
environment fails. With the current bb.tinfoil implementation it is not
possible to run arbitrary bitbake commands - e.g. run
bitbake kern-tools-native -c populate_sysroot in our case. This patch
implements an ugly workaround for that problem, basically by hardcoding
this dependency and running the required bitbake task(s) before tinfoil
is initialized.

[YOCTO #6658]

Signed-off-by: Markus Lehtonen markus.lehto...@linux.intel.com
---
 scripts/lib/devtool/standard.py | 19 ++-
 1 file changed, 18 insertions(+), 1 deletion(-)

diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index 3725d87..bf24e32 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -197,6 +197,8 @@ def extract(args, config, basepath, workspace):
 Entry point for the devtool 'extract' subcommand
 import bb
 
+_check_extract_deps(config, basepath, args.recipename)
+
 tinfoil = setup_tinfoil()
 
 rd = _parse_recipe(config, tinfoil, args.recipename, True)
@@ -237,6 +239,20 @@ class BbTaskExecutor(object):
 self.executed.append(func)
 
 
+def _check_extract_deps(config, basepath, recipename):
+HACK: Ugly workaround for making sure that requirements are met when
+   trying to extract a package
+tinfoil = setup_tinfoil()
+rd = _parse_recipe(config, tinfoil, recipename, True)
+if bb.data.inherits_class('kernel-yocto', rd):
+tinfoil.shutdown()
+try:
+stdout, _ = exec_build_env_command(config.init_path, basepath,
+   'bitbake kern-tools-native')
+except bb.process.ExecutionError as err:
+raise DevtoolError(Failed to build kern-tools-native:\n%s %
+   err.stdout)
+
 def _extract_source(srctree, keep_temp, devbranch, d):
 Extract sources of a recipe
 import bb.event
@@ -422,7 +438,8 @@ def modify(args, config, basepath, workspace):
 raise DevtoolError(directory %s does not exist or not a directory 
(specify -x to extract source from recipe) %
args.srctree)
-
+if args.extract:
+_check_extract_deps(config, basepath, args.recipename)
 tinfoil = setup_tinfoil()
 
 rd = _parse_recipe(config, tinfoil, args.recipename, True)
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 0/3] devtool: improvements to kernel support

[Markus Lehtonen]

This patchset contains few patches to slightly improve the support for kernel
packages in devtool. The last of which is an ugly hack - to be removed after a
future tinfoil rewrite when running arbitrary bitbake task functions becomes
possible.

Markus Lehtonen (3):
  devtool: make required tasks be run in kernel build
  devtool: extract: correct initial rev for kernel packages
  devtool: run kernel dependencies

 scripts/lib/devtool/standard.py | 54 +++--
 1 file changed, 36 insertions(+), 18 deletions(-)

-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/3] devtool: make required tasks be run in kernel build

[Markus Lehtonen]

Set SRCTREECOVEREDTASKS appropriately in the workspace .bbappend file
for kernel recipes. This tries to ensure that all needed tasks (esp.
configure and patch) are run when building the kernel - tasks which
would normally be disabled by externalsrc.bbclass.

[YOCTO #6658]

Signed-off-by: Markus Lehtonen markus.lehto...@linux.intel.com
---
 scripts/lib/devtool/standard.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index e1c5584..00d0b2c 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -500,6 +500,8 @@ def modify(args, config, basepath, workspace):
 if b_is_s:
 f.write('EXTERNALSRC_BUILD_pn-%s = %s\n' % (args.recipename, 
srctree))
 
+if bb.data.inherits_class('kernel', rd):
+f.write('SRCTREECOVEREDTASKS = do_validate_branches 
do_kernel_checkout do_shared_workdir do_fetch do_unpack\n')
 if initial_rev:
 f.write('\n# initial_rev: %s\n' % initial_rev)
 for commit in commits:
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/3] devtool: extract: correct initial rev for kernel packages

[Markus Lehtonen]

Change handling of kernel packages so that the initial rev is parsed
correctly. Also, the devtool-specific git tags (devtool-base and
devtoo-patched) are now generated for kernel packages as well.

[YOCTO #6658]

Signed-off-by: Markus Lehtonen markus.lehto...@linux.intel.com
---
 scripts/lib/devtool/standard.py | 33 -
 1 file changed, 16 insertions(+), 17 deletions(-)

diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index 00d0b2c..3725d87 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -300,6 +300,10 @@ def _extract_source(srctree, keep_temp, devbranch, d):
 task_executor.exec_func('do_fetch', False)
 logger.info('Unpacking...')
 task_executor.exec_func('do_unpack', False)
+if bb.data.inherits_class('kernel-yocto', d):
+# Extra step for kernel to populate the source directory
+logger.info('Doing kernel checkout...')
+task_executor.exec_func('do_kernel_checkout', False)
 srcsubdir = crd.getVar('S', True)
 if srcsubdir == workdir:
 # Find non-patch sources that were unpacked to srctree directory
@@ -329,26 +333,21 @@ def _extract_source(srctree, keep_temp, devbranch, d):
 else:
 os.rmdir(patchdir)
 
-if bb.data.inherits_class('kernel-yocto', d):
-(stdout, _) = bb.process.run('git --git-dir=%s rev-parse HEAD' % 
crd.expand('${WORKDIR}/git'), cwd=srcsubdir)
-initial_rev = stdout.rstrip()
-else:
-if not os.listdir(srcsubdir):
-raise DevtoolError(no source unpacked to S, perhaps the %s 
-   recipe doesn't use any source? % pn)
+if not os.listdir(srcsubdir):
+raise DevtoolError(no source unpacked to S, perhaps the %s 
+   recipe doesn't use any source? % pn)
 
-if not os.path.exists(os.path.join(srcsubdir, '.git')):
-bb.process.run('git init', cwd=srcsubdir)
-bb.process.run('git add .', cwd=srcsubdir)
-bb.process.run('git commit -q -m Initial commit from upstream 
at version %s' % crd.getVar('PV', True), cwd=srcsubdir)
+if not os.path.exists(os.path.join(srcsubdir, '.git')):
+bb.process.run('git init', cwd=srcsubdir)
+bb.process.run('git add .', cwd=srcsubdir)
+bb.process.run('git commit -q -m Initial commit from upstream at 
version %s' % crd.getVar('PV', True), cwd=srcsubdir)
 
-(stdout, _) = bb.process.run('git rev-parse HEAD', cwd=srcsubdir)
-initial_rev = stdout.rstrip()
-
-bb.process.run('git checkout -b %s' % devbranch, cwd=srcsubdir)
-bb.process.run('git tag -f devtool-base', cwd=srcsubdir)
+(stdout, _) = bb.process.run('git rev-parse HEAD', cwd=srcsubdir)
+initial_rev = stdout.rstrip()
 
-crd.setVar('PATCHTOOL', 'git')
+bb.process.run('git checkout -b %s' % devbranch, cwd=srcsubdir)
+bb.process.run('git tag -f devtool-base', cwd=srcsubdir)
+crd.setVar('PATCHTOOL', 'git')
 
 logger.info('Patching...')
 task_executor.exec_func('do_patch', False)
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 2/4] package_rpm: support signing of rpm packages

[Markus Lehtonen]

This patch adds a new bbclass for generating rpm packages that are
signed with a user defined key. The packages are signed as part of the
package_write_rpm task.

In order to enable the feature you need to
1. 'INHERIT +=  sign_rpm' in bitbake config (e.g. local or
   distro)
2. Create a file that contains the passphrase to your gpg secret key
3. 'RPM_GPG_PASSPHRASE_FILE = path_to_file in bitbake config,
   pointing to the passphrase file created in 2.
4. Define GPG key name to use by either defining
   'RPM_GPG_NAME = key_id in bitbake config OR by defining
   %_gpg_name key_id in your ~/.oerpmmacros file
5. 'RPM_GPG_PUBKEY = path_to_pubkey in bitbake config pointing to
   the public key (in armor format)

The user may optionally define GPG_CMD variable in the bitbake
configuration in order to specify a specific gpg binary/wrapper to use.

The sign_rpm.bbclass implements a simple scenario of locally signing the
packages. It could be replaced by a more advanced class that would
utilize a separate signing server for signing the packages, for example.

[YOCTO #8134]

Signed-off-by: Markus Lehtonen markus.lehto...@linux.intel.com
---
 meta/classes/package_rpm.bbclass |  5 
 meta/classes/sign_rpm.bbclass| 60 
 meta/lib/oe/package_manager.py   | 17 
 3 files changed, 82 insertions(+)
 create mode 100644 meta/classes/sign_rpm.bbclass

diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
index 8fd0685..3e933ef 100644
--- a/meta/classes/package_rpm.bbclass
+++ b/meta/classes/package_rpm.bbclass
@@ -695,6 +695,8 @@ python do_package_rpm () {
 else:
 d.setVar('PACKAGE_ARCH_EXTEND', package_arch)
 pkgwritedir = d.expand('${PKGWRITEDIRRPM}/${PACKAGE_ARCH_EXTEND}')
+d.setVar('RPM_PKGWRITEDIR', pkgwritedir)
+bb.debug(1, 'PKGWRITEDIR: %s' % d.getVar('RPM_PKGWRITEDIR', True))
 pkgarch = d.expand('${PACKAGE_ARCH_EXTEND}${HOST_VENDOR}-${HOST_OS}')
 magicfile = 
d.expand('${STAGING_DIR_NATIVE}${datadir_native}/misc/magic.mgc')
 bb.utils.mkdirhier(pkgwritedir)
@@ -730,6 +732,9 @@ python do_package_rpm () {
 d.setVar('BUILDSPEC', cmd + \n)
 d.setVarFlag('BUILDSPEC', 'func', '1')
 bb.build.exec_func('BUILDSPEC', d)
+
+if d.getVar('RPM_SIGN_PACKAGES', True) == '1':
+bb.build.exec_func(sign_rpm, d)
 }
 
 python () {
diff --git a/meta/classes/sign_rpm.bbclass b/meta/classes/sign_rpm.bbclass
new file mode 100644
index 000..552af68
--- /dev/null
+++ b/meta/classes/sign_rpm.bbclass
@@ -0,0 +1,60 @@
+inherit sanity
+
+RPM_SIGN_PACKAGES='1'
+
+
+_check_gpg_name () {
+macrodef=`rpm -E '%_gpg_name'`
+[ $macrodef == %_gpg_name ]  return 1 || return 0
+}
+
+
+def rpmsign_wrapper(d, files, passphrase, gpg_name=None):
+import pexpect
+
+# Find the correct rpm binary
+rpm_bin_path = d.getVar('STAGING_BINDIR_NATIVE', True) + '/rpm'
+cmd = rpm_bin_path +  --addsign 
+if gpg_name:
+cmd += --define '%%_gpg_name %s'  % gpg_name
+else:
+try:
+bb.build.exec_func('_check_gpg_name', d)
+except bb.build.FuncFailed:
+raise_sanity_error(You need to define RPM_GPG_NAME in bitbake 
+   config or the %_gpg_name RPM macro defined 
+   (e.g. in  ~/.oerpmmacros, d)
+if d.getVar('GPG_BIN', True):
+cmd += --define '%%__gpg %s'  % d.getVar('GPG_BIN', True)
+cmd += ' '.join(files)
+
+# Need to use pexpect for feeding the passphrase
+proc = pexpect.spawn(cmd)
+try:
+proc.expect_exact('Enter pass phrase:', timeout=15)
+proc.sendline(passphrase)
+proc.expect(pexpect.EOF, timeout=900)
+proc.close()
+except pexpect.TIMEOUT as err:
+bb.debug('rpmsign timeout: %s' % err)
+proc.terminate()
+return proc.exitstatus
+
+
+python sign_rpm () {
+import glob
+
+rpm_gpg_pass_file = (d.getVar(RPM_GPG_PASSPHRASE_FILE, True) or )
+if rpm_gpg_pass_file:
+with open(rpm_gpg_pass_file) as fobj:
+rpm_gpg_passphrase = fobj.readlines()[0].rstrip('\n')
+else:
+raise_sanity_error(You need to define RPM_GPG_PASSPHRASE_FILE in the 
config, d)
+
+rpm_gpg_name = (d.getVar(RPM_GPG_NAME, True) or )
+
+rpms = glob.glob(d.getVar('RPM_PKGWRITEDIR', True) + '/*')
+
+if rpmsign_wrapper(d, rpms, rpm_gpg_passphrase, rpm_gpg_name) != 0:
+raise bb.build.FuncFailed(RPM signing failed)
+}
diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index 2ab1d78..936887c 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -108,6 +108,7 @@ class RpmIndexer(Indexer):
 archs = archs.union(set(sdk_pkg_archs))
 
 rpm_createrepo = bb.utils.which(os.getenv('PATH'), createrepo)
+
 index_cmds = []
 rpm_dirs_found = False
 for arch in archs:
@@ -127,9 +128,16 @@ class RpmIndexer(Indexer

[OE-core] [PATCH v2 4/4] package_manager: support for signed RPM package feeds

[Markus Lehtonen]

This change makes it possible to create GPG signed RPM package feeds -
i.e. package feed with GPG signed metadata (repodata). All deployed RPM
repositories will be signed and the GPG public key is copied to the rpm
deployment directory.

In order to enable the new feature one needs to define four variables in
bitbake configuration.
1. 'PACKAGE_FEED_SIGN = 1' enabling the feature
2. 'PACKAGE_FEED_GPG_NAME = key_id' defining the GPG key to use for
   signing
3. 'PACKAGE_FEED_GPG_PASSPHRASE_FILE = path_to_file' pointing to a
   file containing the passphrase for the secret signing key
4. 'PACKAGE_FEED_GPG_PUBKEY = path_to_pubkey' pointing to the
   corresponding public key (in armor format)
The user may define GPG_CMD in the bitbake configuration in order to
specify a specific the gpg binary/wrapper to use for signing.

[YOCTO #8134]

Signed-off-by: Markus Lehtonen markus.lehto...@linux.intel.com
---
 meta/lib/oe/package_manager.py | 25 -
 1 file changed, 24 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index 936887c..83c119a 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -108,8 +108,17 @@ class RpmIndexer(Indexer):
 archs = archs.union(set(sdk_pkg_archs))
 
 rpm_createrepo = bb.utils.which(os.getenv('PATH'), createrepo)
+if self.d.getVar('PACKAGE_FEED_SIGN', True) == '1':
+pkgfeed_gpg_name = self.d.getVar('PACKAGE_FEED_GPG_NAME', True)
+pkgfeed_gpg_pass = 
self.d.getVar('PACKAGE_FEED_GPG_PASSPHRASE_FILE', True)
+else:
+pkgfeed_gpg_name = None
+pkgfeed_gpg_pass = None
+gpg_bin = self.d.getVar('GPG_BIN', True) or \
+  bb.utils.which(os.getenv('PATH'), gpg)
 
 index_cmds = []
+repo_sign_cmds = []
 rpm_dirs_found = False
 for arch in archs:
 dbpath = os.path.join(self.d.getVar('WORKDIR', True), 'rpmdb', 
arch)
@@ -121,6 +130,12 @@ class RpmIndexer(Indexer):
 
 index_cmds.append(%s --dbpath %s --update -q %s % \
  (rpm_createrepo, dbpath, arch_dir))
+if pkgfeed_gpg_name:
+repomd_file = os.path.join(arch_dir, 'repodata', 'repomd.xml')
+gpg_cmd = %s --detach-sign --armor --batch --no-tty --yes  \
+  --passphrase-file '%s' -u '%s' %s % (gpg_bin,
+  pkgfeed_gpg_pass, pkgfeed_gpg_name, repomd_file)
+repo_sign_cmds.append(gpg_cmd)
 
 rpm_dirs_found = True
 
@@ -132,12 +147,20 @@ class RpmIndexer(Indexer):
 result = oe.utils.multiprocess_exec(index_cmds, create_index)
 if result:
 bb.fatal('%s' % ('\n'.join(result)))
-# Copy pubkey to repo
+# Sign repomd
+result = oe.utils.multiprocess_exec(repo_sign_cmds, create_index)
+if result:
+bb.fatal('%s' % ('\n'.join(result)))
+# Copy pubkey(s) to repo
 distro_version = self.d.getVar('DISTRO_VERSION', True) or oe.0
 if self.d.getVar('RPM_SIGN_PACKAGES', True) == '1':
 shutil.copy2(self.d.getVar('RPM_GPG_PUBKEY', True),
  os.path.join(self.deploy_dir,
   'RPM-GPG-KEY-%s' % distro_version))
+if self.d.getVar('PACKAGE_FEED_SIGN', True) == '1':
+shutil.copy2(self.d.getVar('PACKAGE_FEED_GPG_PUBKEY', True),
+ os.path.join(self.deploy_dir,
+  'REPODATA-GPG-KEY-%s' % distro_version))
 
 
 class OpkgIndexer(Indexer):
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 0/4] Sign packages in RPM feeds

[Markus Lehtonen]

Second iteration of my patchset. I tried to address the issues pointed out by
Mark:
1. The gpg key is not imported to the (temporary) rpm databases used by
   createrepo. Instead, createrepo is patched to ignore signature
   verification altogether.
2. There is a new optional config variable GPG_BIN which can be used to
   define the gpg binary used for signing.
3. The filename of the public keys (published with the package feed and
   depoyed into the target rootfs as part of os-release package) is now
   postfixed with -${DISTRO_VERSION}.

[YOCTO #8134]

*** BLURB HERE ***

Markus Lehtonen (4):
  createrepo: disable RPM signature validation
  package_rpm: support signing of rpm packages
  os-release: add the public package-signing key
  package_manager: support for signed RPM package feeds

 meta/classes/package_rpm.bbclass   |  5 ++
 meta/classes/sign_rpm.bbclass  | 60 ++
 meta/lib/oe/package_manager.py | 40 +++
 meta/recipes-core/os-release/os-release.bb | 11 
 ...dumpMetadata-disable-signature-validation.patch | 31 +++
 .../createrepo/createrepo_0.4.11.bb| 17 +++---
 6 files changed, 156 insertions(+), 8 deletions(-)
 create mode 100644 meta/classes/sign_rpm.bbclass
 create mode 100644 
meta/recipes-support/createrepo/createrepo/dumpMetadata-disable-signature-validation.patch

-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 1/4] createrepo: disable RPM signature validation

[Markus Lehtonen]

Disable RPM signature validation so that it is possible to create
package feeds of signed RPM packages without importing the public part
of the signing key into the RPM database. In any case, the signatures
are validated when the packages in the feed are used (e.g. in image
generation of manually installing packages from the feed).

The original idea idea of this patch is from Mark Hatle
mark.ha...@windriver.com.

[YOCTO #8134]

Signed-off-by: Markus Lehtonen markus.lehto...@linux.intel.com
---
 ...dumpMetadata-disable-signature-validation.patch | 31 ++
 .../createrepo/createrepo_0.4.11.bb| 17 ++--
 2 files changed, 40 insertions(+), 8 deletions(-)
 create mode 100644 
meta/recipes-support/createrepo/createrepo/dumpMetadata-disable-signature-validation.patch

diff --git 
a/meta/recipes-support/createrepo/createrepo/dumpMetadata-disable-signature-validation.patch
 
b/meta/recipes-support/createrepo/createrepo/dumpMetadata-disable-signature-validation.patch
new file mode 100644
index 000..905531f
--- /dev/null
+++ 
b/meta/recipes-support/createrepo/createrepo/dumpMetadata-disable-signature-validation.patch
@@ -0,0 +1,31 @@
+dumpMetadata: disable signature validation
+
+Makes it possible to work on RPM repositories that contain signed packages
+without the need of importing the public part of the signing key into the RPM
+database.
+
+Upstream-Status: Pending
+
+Signed-off-by: Markus Lehtonen markus.lehto...@linux.intel.com
+---
+ dumpMetadata.py | 4 +++-
+ 1 file changed, 3 insertions(+), 1 deletion(-)
+
+diff --git a/dumpMetadata.py b/dumpMetadata.py
+index e40e8ac..ca6a82d 100644
+--- a/dumpMetadata.py
 b/dumpMetadata.py
+@@ -92,7 +92,9 @@ def returnHdr(ts, package):
+ fdno = package # let's assume this is an fdno and go with it :)
+ except OSError:
+ raise MDError, Error opening file
+-ts.setVSFlags((rpm.RPMVSF_NOMD5|rpm.RPMVSF_NEEDPAYLOAD))
++ts.setVSFlags((rpm.RPMVSF_NOMD5|rpm.RPMVSF_NEEDPAYLOAD|
++   rpm.RPMVSF_NODSA|rpm.RPMVSF_NORSA|
++   rpm.RPMVSF_NODSAHEADER|rpm.RPMVSF_NORSAHEADER))
+ try:
+ hdr = ts.hdrFromFdno(fdno)
+ except rpm.error:
+-- 
+2.1.4
+
diff --git a/meta/recipes-support/createrepo/createrepo_0.4.11.bb 
b/meta/recipes-support/createrepo/createrepo_0.4.11.bb
index adc193e..debbaec 100644
--- a/meta/recipes-support/createrepo/createrepo_0.4.11.bb
+++ b/meta/recipes-support/createrepo/createrepo_0.4.11.bb
@@ -9,14 +9,15 @@ RDEPENDS_${PN}_class-target = libxml2-python
 
 PR = r9
 
-SRC_URI= http://createrepo.baseurl.org/download/${BP}.tar.gz \
-  file://fix-native-install.patch \
-  file://python-scripts-should-use-interpreter-from-env.patch \
- file://createrepo-rpm549.patch \
- file://recommends.patch \
- file://createrepo-dbpath.patch \
- file://rpm-createsolvedb.py \
- 
+SRC_URI = http://createrepo.baseurl.org/download/${BP}.tar.gz \
+   file://fix-native-install.patch \
+   file://python-scripts-should-use-interpreter-from-env.patch \
+   file://createrepo-rpm549.patch \
+   file://recommends.patch \
+   file://createrepo-dbpath.patch \
+   file://dumpMetadata-disable-signature-validation.patch \
+   file://rpm-createsolvedb.py \
+   
 
 SRC_URI[md5sum] = 3e9ccf4abcffe3f49af078c83611eda2
 SRC_URI[sha256sum] = 
a73ae11a0dcde8bde36d900bc3f7f8f1083ba752c70a5c61b72d1e1e7608f21b
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 3/4] os-release: add the public package-signing key

[Markus Lehtonen]

Adds the public package-signing key into this package. It will be
installed under /etc/pki/rpm-gpg if the RPM signing feature is used. The
key file is not currently directly used by anything in the target
system. It is merely there for possible later use.

[YOCTO #8134]

Signed-off-by: Markus Lehtonen markus.lehto...@linux.intel.com
---
 meta/recipes-core/os-release/os-release.bb | 11 +++
 1 file changed, 11 insertions(+)

diff --git a/meta/recipes-core/os-release/os-release.bb 
b/meta/recipes-core/os-release/os-release.bb
index 87fea6f..f24882a 100644
--- a/meta/recipes-core/os-release/os-release.bb
+++ b/meta/recipes-core/os-release/os-release.bb
@@ -23,15 +23,26 @@ PRETTY_NAME = ${DISTRO_NAME} ${VERSION}
 BUILD_ID ?= ${DATETIME}
 
 python do_compile () {
+import shutil
 with open(d.expand('${B}/os-release'), 'w') as f:
 for field in d.getVar('OS_RELEASE_FIELDS', True).split():
 value = d.getVar(field, True)
 if value:
 f.write('{0}={1}\n'.format(field, value))
+if d.getVar('RPM_SIGN_PACKAGES', True) == '1':
+rpm_gpg_pubkey = d.getVar('RPM_GPG_PUBKEY', True)
+os.mkdir('${B}/rpm-gpg')
+distro_version = self.d.getVar('DISTRO_VERSION', True) or oe.0
+shutil.copy2(rpm_gpg_pubkey, d.expand('${B}/rpm-gpg/RPM-GPG-KEY-%s' % 
distro_version))
 }
 do_compile[vardeps] += ${OS_RELEASE_FIELDS}
 
 do_install () {
 install -d ${D}${sysconfdir}
 install -m 0644 os-release ${D}${sysconfdir}/
+
+if [ -d rpm-gpg ]; then
+install -d ${D}${sysconfdir}/pki
+cp -r rpm-gpg ${D}${sysconfdir}/pki/
+fi
 }
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/3] os-release: add the public package-signing key

[Markus Lehtonen]

Adds the public package-signing key into this package. It will be
installed under /etc/pki/rpm-gpg if the RPM signing feature is used. The
key file is not currently directly used by anything in the target
system. It is merely there for possible later use.

[YOCTO #8134]

Signed-off-by: Markus Lehtonen markus.lehto...@linux.intel.com
---
 meta/recipes-core/os-release/os-release.bb | 9 +
 1 file changed, 9 insertions(+)

diff --git a/meta/recipes-core/os-release/os-release.bb 
b/meta/recipes-core/os-release/os-release.bb
index 87fea6f..542cf56 100644
--- a/meta/recipes-core/os-release/os-release.bb
+++ b/meta/recipes-core/os-release/os-release.bb
@@ -23,15 +23,24 @@ PRETTY_NAME = ${DISTRO_NAME} ${VERSION}
 BUILD_ID ?= ${DATETIME}
 
 python do_compile () {
+import shutil
 with open(d.expand('${B}/os-release'), 'w') as f:
 for field in d.getVar('OS_RELEASE_FIELDS', True).split():
 value = d.getVar(field, True)
 if value:
 f.write('{0}={1}\n'.format(field, value))
+if d.getVar('RPM_SIGN_PACKAGES', True) == '1':
+rpm_gpg_pubkey = d.getVar('RPM_GPG_PUBKEY', True)
+shutil.copy2(rpm_gpg_pubkey, d.expand('${B}/RPM-GPG-KEY-default'))
 }
 do_compile[vardeps] += ${OS_RELEASE_FIELDS}
 
 do_install () {
 install -d ${D}${sysconfdir}
 install -m 0644 os-release ${D}${sysconfdir}/
+
+if [ -f RPM-GPG-KEY-default ]; then
+install -d ${D}${sysconfdir}/pki/rpm-gpg
+install -m 0644 RPM-GPG-KEY-oe ${D}${sysconfdir}/pki/rpm-gpg/
+fi
 }
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/3] package_rpm: support signing of rpm packages

[Markus Lehtonen]

This patch adds a new bbclass for generating rpm packages that are
signed with a user defined key. The packages are signed as part of the
package_write_rpm task.

In order to enable the feature you need to
1. 'INHERIT +=  sign_rpm' in bitbake config (e.g. local or
   distro)
2. Create a file that contains the passphrase to your gpg secret key
3. 'RPM_GPG_PASSPHRASE_FILE = path_to_file in bitbake config,
   pointing to the passphrase file created in 2.
4. Define GPG key name to use by either defining
   'RPM_GPG_NAME = key_id in bitbake config OR by defining
   %_gpg_name key_id in your ~/.oerpmmacros file
5. 'RPM_GPG_PUBKEY = path_to_pubkey in bitbake config pointing to
   the public key (in armor format)

The sign_rpm.bbclass implements a simple scenario of locally signing the
packages. It could be replaced by a more advanced class that would
utilize a separate signing server for signing the packages, for example.

[YOCTO #8134]

Signed-off-by: Markus Lehtonen markus.lehto...@linux.intel.com
---
 meta/classes/package_rpm.bbclass |  5 
 meta/classes/sign_rpm.bbclass| 58 
 meta/lib/oe/package_manager.py   | 28 +++
 3 files changed, 91 insertions(+)
 create mode 100644 meta/classes/sign_rpm.bbclass

diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
index 8fd0685..3e933ef 100644
--- a/meta/classes/package_rpm.bbclass
+++ b/meta/classes/package_rpm.bbclass
@@ -695,6 +695,8 @@ python do_package_rpm () {
 else:
 d.setVar('PACKAGE_ARCH_EXTEND', package_arch)
 pkgwritedir = d.expand('${PKGWRITEDIRRPM}/${PACKAGE_ARCH_EXTEND}')
+d.setVar('RPM_PKGWRITEDIR', pkgwritedir)
+bb.debug(1, 'PKGWRITEDIR: %s' % d.getVar('RPM_PKGWRITEDIR', True))
 pkgarch = d.expand('${PACKAGE_ARCH_EXTEND}${HOST_VENDOR}-${HOST_OS}')
 magicfile = 
d.expand('${STAGING_DIR_NATIVE}${datadir_native}/misc/magic.mgc')
 bb.utils.mkdirhier(pkgwritedir)
@@ -730,6 +732,9 @@ python do_package_rpm () {
 d.setVar('BUILDSPEC', cmd + \n)
 d.setVarFlag('BUILDSPEC', 'func', '1')
 bb.build.exec_func('BUILDSPEC', d)
+
+if d.getVar('RPM_SIGN_PACKAGES', True) == '1':
+bb.build.exec_func(sign_rpm, d)
 }
 
 python () {
diff --git a/meta/classes/sign_rpm.bbclass b/meta/classes/sign_rpm.bbclass
new file mode 100644
index 000..ddf6c3b
--- /dev/null
+++ b/meta/classes/sign_rpm.bbclass
@@ -0,0 +1,58 @@
+inherit sanity
+
+RPM_SIGN_PACKAGES='1'
+
+
+_check_gpg_name () {
+macrodef=`rpm -E '%_gpg_name'`
+[ $macrodef == %_gpg_name ]  return 1 || return 0
+}
+
+
+def rpmsign_wrapper(d, files, passphrase, gpg_name=None):
+import pexpect
+
+# Find the correct rpm binary
+rpm_bin_path = d.getVar('STAGING_BINDIR_NATIVE', True) + '/rpm'
+cmd = rpm_bin_path +  --addsign 
+if gpg_name:
+cmd += --define '%%_gpg_name %s'  % gpg_name
+else:
+try:
+bb.build.exec_func('_check_gpg_name', d)
+except bb.build.FuncFailed:
+raise_sanity_error(You need to define RPM_GPG_NAME in bitbake 
+   config or the %_gpg_name RPM macro defined 
+   (e.g. in  ~/.oerpmmacros, d)
+cmd += ' '.join(files)
+
+# Need to use pexpect for feeding the passphrase
+proc = pexpect.spawn(cmd)
+try:
+proc.expect_exact('Enter pass phrase:', timeout=15)
+proc.sendline(passphrase)
+proc.expect(pexpect.EOF, timeout=900)
+proc.close()
+except pexpect.TIMEOUT as err:
+bb.debug('rpmsign timeout: %s' % err)
+proc.terminate()
+return proc.exitstatus
+
+
+python sign_rpm () {
+import glob
+
+rpm_gpg_pass_file = (d.getVar(RPM_GPG_PASSPHRASE_FILE, True) or )
+if rpm_gpg_pass_file:
+with open(rpm_gpg_pass_file) as fobj:
+rpm_gpg_passphrase = fobj.readlines()[0].rstrip('\n')
+else:
+raise_sanity_error(You need to define RPM_GPG_PASSPHRASE_FILE in the 
config, d)
+
+rpm_gpg_name = (d.getVar(RPM_GPG_NAME, True) or )
+
+rpms = glob.glob(d.getVar('RPM_PKGWRITEDIR', True) + '/*')
+
+if rpmsign_wrapper(d, rpms, rpm_gpg_passphrase, rpm_gpg_name) != 0:
+raise bb.build.FuncFailed(RPM signing failed)
+}
diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index 2ab1d78..753b3eb 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -108,7 +108,14 @@ class RpmIndexer(Indexer):
 archs = archs.union(set(sdk_pkg_archs))
 
 rpm_createrepo = bb.utils.which(os.getenv('PATH'), createrepo)
+rpm_bin = bb.utils.which(os.getenv('PATH'), rpm)
+if self.d.getVar('RPM_SIGN_PACKAGES', True) == '1':
+rpm_pubkey = self.d.getVar('RPM_GPG_PUBKEY', True)
+else:
+rpm_pubkey = None
+
 index_cmds = []
+key_import_cmds = []
 rpm_dirs_found = False
 for arch in archs:
 dbpath = os.path.join

[OE-core] [PATCH 0/3] Sign packages in RPM feeds

[Markus Lehtonen]

Implement simple scheme of signing RPM packages and RPM package feeds locally
in the builder host. RPM package signing is implemented in a new bbclass. This
could be extended/replaced to enable more sophisticated schemes like using a
signing server.

[YOCTO #8134]

Markus Lehtonen (3):
  package_rpm: support signing of rpm packages
  os-release: add the public package-signing key
  package_manager: support for signed RPM package feeds

 meta/classes/package_rpm.bbclass   |  5 +++
 meta/classes/sign_rpm.bbclass  | 58 ++
 meta/lib/oe/package_manager.py | 48 +
 meta/recipes-core/os-release/os-release.bb |  9 +
 4 files changed, 120 insertions(+)
 create mode 100644 meta/classes/sign_rpm.bbclass

-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/3] package_manager: support for signed RPM package feeds

[Markus Lehtonen]

This change makes it possible to create GPG signed RPM package feeds -
i.e. package feed with GPG signed metadata (repodata). All deployed RPM
repositories will be signed and the GPG public key is copied to the rpm
deployment directory.

In order to enable the new feature one needs to define four variables in
bitbake configuration.
1. 'PACKAGE_FEED_SIGN = 1' enabling the feature
2. 'PACKAGE_FEED_GPG_NAME = key_id' defining the GPG key to use for
   signing
3. 'PACKAGE_FEED_GPG_PASSPHRASE_FILE = path_to_file' pointing to a
   file containing the passphrase for the secret signing key
4. 'PACKAGE_FEED_GPG_PUBKEY = path_to_pubkey' pointing to the
   corresponding public key (in armor format)

[YOCTO #8134]

Signed-off-by: Markus Lehtonen markus.lehto...@linux.intel.com
---
 meta/lib/oe/package_manager.py | 24 ++--
 1 file changed, 22 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index 753b3eb..5d7ef54 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -113,8 +113,15 @@ class RpmIndexer(Indexer):
 rpm_pubkey = self.d.getVar('RPM_GPG_PUBKEY', True)
 else:
 rpm_pubkey = None
+if self.d.getVar('PACKAGE_FEED_SIGN', True) == '1':
+pkgfeed_gpg_name = self.d.getVar('PACKAGE_FEED_GPG_NAME', True)
+pkgfeed_gpg_pass = 
self.d.getVar('PACKAGE_FEED_GPG_PASSPHRASE_FILE', True)
+else:
+pkgfeed_gpg_name = None
+pkgfeed_gpg_pass = None
 
 index_cmds = []
+repo_sign_cmds = []
 key_import_cmds = []
 rpm_dirs_found = False
 for arch in archs:
@@ -126,10 +133,16 @@ class RpmIndexer(Indexer):
 continue
 
 if rpm_pubkey:
-key_import_cmds.append(%s --define '_dbpath %s' --import %s %
+key_import_cmds.append(%s --dbpath '%s' --import %s %
(rpm_bin, dbpath, rpm_pubkey))
 index_cmds.append(%s --dbpath %s --update -q %s % \
  (rpm_createrepo, dbpath, arch_dir))
+if pkgfeed_gpg_name:
+repomd_file = os.path.join(arch_dir, 'repodata', 'repomd.xml')
+gpg_cmd = gpg2 --detach-sign --armor --batch --no-tty --yes  
\
+  --passphrase-file '%s' -u '%s' %s % \
+  (pkgfeed_gpg_pass, pkgfeed_gpg_name, repomd_file)
+repo_sign_cmds.append(gpg_cmd)
 
 rpm_dirs_found = True
 
@@ -145,10 +158,17 @@ class RpmIndexer(Indexer):
 result = oe.utils.multiprocess_exec(index_cmds, create_index)
 if result:
 bb.fatal('%s' % ('\n'.join(result)))
-# Copy pubkey to repo
+# Sign repomd
+result = oe.utils.multiprocess_exec(repo_sign_cmds, create_index)
+if result:
+bb.fatal('%s' % ('\n'.join(result)))
+# Copy pubkey(s) to repo
 if self.d.getVar('RPM_SIGN_PACKAGES', True) == '1':
 shutil.copy2(self.d.getVar('RPM_GPG_PUBKEY', True),
  os.path.join(self.deploy_dir, 'RPM-GPG-KEY-oe'))
+if self.d.getVar('PACKAGE_FEED_SIGN', True) == '1':
+shutil.copy2(self.d.getVar('PACKAGE_FEED_GPG_PUBKEY', True),
+ os.path.join(self.deploy_dir, 'REPODATA-GPG-KEY'))
 
 
 class OpkgIndexer(Indexer):
-- 
2.1.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core