date:20170516

Re: [OE-core] [morty] [PATCH] rpm: Fix the Bug of SRPM String error

2017-05-16 Thread Zheng, Ruoqin

Ping

Zheng Ruoqin
Nanjing Fujitsu Nanda Software Tech. Co., Ltd.(FNST)
ADDR.: No.6 Wenzhu Road, Software Avenue,
   Nanjing, 210012, China
MAIL : zhengrq.f...@cn.fujistu.com

-Original Message-
From: Zheng, Ruoqin/郑 若钦 
Sent: Monday, May 08, 2017 6:17 PM
To: openembedded-core@lists.openembedded.org
Cc: Zheng, Ruoqin/郑 若钦 
Subject: [OE-core] [morty] [PATCH] rpm: Fix the Bug of SRPM String error

Add a patch 0001-Fix-the-Bug-of-SRPM-String-error.patch to fix SRPM bug

When use bitbake to build a SRPM package, some sections of SRPM can't be 
displayed normally.
For example:
$ rpm -qpi zlib-1.2.8-r0.src.rpm
warning: zlib-1.2.8-r0.src.rpm: Header V4 DSA/SHA1 Signature, key ID e64962a6: 
NOKEY
Name: zlib
Version : 1.2.8
Release : r0
Architecture: i586
Install Date: (not installed)
Group   : ïº©
Size: 633007
License : Zlib
Signature   : DSA/SHA1, 2017å¹´04æœˆ22æ—¥ 00æ™‚38åˆ†03ç§’, Key ID 
bc5e0520e64962a6
Source RPM  : (none)
Build Date  : 2017å¹´04æœˆ22æ—¥ 00æ™‚38åˆ†03ç§’
Build Host  : force
Relocations : (not relocatable)
Packager: Poky 
URL : http://zlib.net/
Summary : Â¨Â¯
Description :
ïº©

Signed-off-by: Zheng Ruoqin 
---
 .../0001-Fix-the-Bug-of-SRPM-String-error.patch| 162 +
 meta/recipes-devtools/rpm/rpm_5.4.16.bb|   1 +
 2 files changed, 163 insertions(+)
 create mode 100644 
meta/recipes-devtools/rpm/rpm/0001-Fix-the-Bug-of-SRPM-String-error.patch

diff --git 
a/meta/recipes-devtools/rpm/rpm/0001-Fix-the-Bug-of-SRPM-String-error.patch 
b/meta/recipes-devtools/rpm/rpm/0001-Fix-the-Bug-of-SRPM-String-error.patch
new file mode 100644
index 000..85c48ac
--- /dev/null
+++ b/meta/recipes-devtools/rpm/rpm/0001-Fix-the-Bug-of-SRPM-String-erro
+++ r.patch
@@ -0,0 +1,162 @@
+Subject: [PATCH] Fix the Bug of SRPM String error
+
+Upstream-Status: Backport from rpm 5.4.16 $ rpm -qpi 
+zlib-1.2.8-r0.src.rpm
+warning: zlib-1.2.8-r0.src.rpm: Header V4 DSA/SHA1 Signature, key ID e64962a6: 
NOKEY
+Name: zlib
+Version : 1.2.8
+Release : r0
+Architecture: i586
+Install Date: (not installed)
+Group   : ïº©
+Size: 633007
+License : Zlib
+Signature   : DSA/SHA1, 2017å¹´04æœˆ22æ—¥ 00æ™‚38åˆ†03ç§’, Key ID 
bc5e0520e64962a6
+Source RPM  : (none)
+Build Date  : 2017å¹´04æœˆ22æ—¥ 00æ™‚38åˆ†03ç§’
+Build Host  : force
+Relocations : (not relocatable)
+Packager: Poky 
+URL : http://zlib.net/
+Summary : Â¨Â¯
+Description :
+ïº©
+
+Signed-off-by: Zheng Ruoqin 
+---
+ rpmdb/header.c  |  8 
+ rpmdb/rpmtag.h  |  1 +
+ rpmdb/tagname.c | 61 
+-
+ 3 files changed, 48 insertions(+), 22 deletions(-)
+
+diff --git a/rpmdb/header.c b/rpmdb/header.c index 5d6c6cf..5a80df0 
+100644
+--- a/rpmdb/header.c
 b/rpmdb/header.c
+@@ -32,7 +32,7 @@ GENfree(HeaderIterator)
+ #endif/* __cplusplus */
+ 
+ #if defined(SUPPORT_IMPLICIT_TAG_DATA_TYPES)
+-extern void tagTypeValidate(HE_t he)
++extern void tagTypeValidate(HE_t he, unsigned int flags)
+   /*@*/;
+ #endif
+ 
+@@ -2281,7 +2281,7 @@ int headerGet(Header h, HE_t he, unsigned int 
+flags)  /*@-modfilesys@*/
+ /* XXX verify that explicit and implicit types are identical. */
+ if (rc)
+-  tagTypeValidate(he);
++tagTypeValidate(he, flags);
+ /*@=modfilesys@*/
+ #endif
+ 
+@@ -2304,7 +2304,7 @@ int headerPut(Header h, HE_t he, /*@unused@*/ 
+unsigned int flags)  #if defined(SUPPORT_IMPLICIT_TAG_DATA_TYPES)
+ /*@-modfilesys@*/
+ /* XXX verify that explicit and implicit types are identical. */
+-tagTypeValidate(he);
++tagTypeValidate(he, flags);
+ /*@=modfilesys@*/
+ #endif
+ 
+@@ -2329,7 +2329,7 @@ int headerMod(Header h, HE_t he, /*@unused@*/ 
+unsigned int flags)  #if defined(SUPPORT_IMPLICIT_TAG_DATA_TYPES)
+ /*@-modfilesys@*/
+ /* XXX verify that explicit and implicit types are identical. */
+-tagTypeValidate(he);
++tagTypeValidate(he, flags);
+ /*@=modfilesys@*/
+ #endif
+ 
+diff --git a/rpmdb/rpmtag.h b/rpmdb/rpmtag.h index ffce678..8f6a2bf 
+100644
+--- a/rpmdb/rpmtag.h
 b/rpmdb/rpmtag.h
+@@ -868,6 +868,7 @@ int headerGet(Header h, HE_t he, unsigned int flags)
+   /*@modifies he, internalState @*/;
+ #define   HEADERGET_NOEXTENSION   (1 << 0) /*!< Extension search 
disabler. */
+ #define   HEADERGET_NOI18NSTRING  (1 << 1) /*!< Return i18n strings as 
argv. */
++#define HEADERGET_SIGHEADER (1 << 2) /*!< Override signature tag data. */
+ 
+ /** \ingroup header
+  * Add or append tag container to header.
+diff --git a/rpmdb/tagname.c b/rpmdb/tagname.c index cfd1459..374ad77 
+100644
+--- a/rpmdb/tagname.c
 b/rpmdb/tagname.c
+@@ -515,28 +515,53 @@ tagStore_t tagStoreFree(tagStore_t dbiTags, 
+size_t dbiNTags)
+  * Validate that implicit and explicit types are identical.
+  * @param

Re: [OE-core] [krogoth] [PATCH] rpm: Fix the Bug of SRPM String error

2017-05-16 Thread Zheng, Ruoqin

Ping

Zheng Ruoqin
Nanjing Fujitsu Nanda Software Tech. Co., Ltd.(FNST)
ADDR.: No.6 Wenzhu Road, Software Avenue,
   Nanjing, 210012, China
MAIL : zhengrq.f...@cn.fujistu.com


-Original Message-
From: Zheng, Ruoqin/郑 若钦 
Sent: Monday, May 08, 2017 6:16 PM
To: openembedded-core@lists.openembedded.org
Cc: Zheng, Ruoqin/郑 若钦 
Subject: [OE-core] [krogoth] [PATCH] rpm: Fix the Bug of SRPM String error

Add a patch 0001-Fix-the-Bug-of-SRPM-String-error.patch to fix SRPM bug

When use bitbake to build a SRPM package, some sections of SRPM can't be 
displayed normally.
For example:
$ rpm -qpi zlib-1.2.8-r0.src.rpm
warning: zlib-1.2.8-r0.src.rpm: Header V4 DSA/SHA1 Signature, key ID e64962a6: 
NOKEY
Name: zlib
Version : 1.2.8
Release : r0
Architecture: i586
Install Date: (not installed)
Group   : ïº©
Size: 633007
License : Zlib
Signature   : DSA/SHA1, 2017å¹´04æœˆ22æ—¥ 00æ™‚38åˆ†03ç§’, Key ID 
bc5e0520e64962a6
Source RPM  : (none)
Build Date  : 2017å¹´04æœˆ22æ—¥ 00æ™‚38åˆ†03ç§’
Build Host  : force
Relocations : (not relocatable)
Packager: Poky 
URL : http://zlib.net/
Summary : Â¨Â¯
Description :
ïº©


Signed-off-by: Zheng Ruoqin 
---
 .../0001-Fix-the-Bug-of-SRPM-String-error.patch| 162 +
 meta/recipes-devtools/rpm/rpm_5.4.16.bb|   1 +
 2 files changed, 163 insertions(+)
 create mode 100644 
meta/recipes-devtools/rpm/rpm/0001-Fix-the-Bug-of-SRPM-String-error.patch

diff --git 
a/meta/recipes-devtools/rpm/rpm/0001-Fix-the-Bug-of-SRPM-String-error.patch 
b/meta/recipes-devtools/rpm/rpm/0001-Fix-the-Bug-of-SRPM-String-error.patch
new file mode 100644
index 000..85c48ac
--- /dev/null
+++ b/meta/recipes-devtools/rpm/rpm/0001-Fix-the-Bug-of-SRPM-String-erro
+++ r.patch
@@ -0,0 +1,162 @@
+Subject: [PATCH] Fix the Bug of SRPM String error
+
+Upstream-Status: Backport from rpm 5.4.16 $ rpm -qpi 
+zlib-1.2.8-r0.src.rpm
+warning: zlib-1.2.8-r0.src.rpm: Header V4 DSA/SHA1 Signature, key ID e64962a6: 
NOKEY
+Name: zlib
+Version : 1.2.8
+Release : r0
+Architecture: i586
+Install Date: (not installed)
+Group   : ïº©
+Size: 633007
+License : Zlib
+Signature   : DSA/SHA1, 2017å¹´04æœˆ22æ—¥ 00æ™‚38åˆ†03ç§’, Key ID 
bc5e0520e64962a6
+Source RPM  : (none)
+Build Date  : 2017å¹´04æœˆ22æ—¥ 00æ™‚38åˆ†03ç§’
+Build Host  : force
+Relocations : (not relocatable)
+Packager: Poky 
+URL : http://zlib.net/
+Summary : Â¨Â¯
+Description :
+ïº©
+
+Signed-off-by: Zheng Ruoqin 
+---
+ rpmdb/header.c  |  8 
+ rpmdb/rpmtag.h  |  1 +
+ rpmdb/tagname.c | 61 
+-
+ 3 files changed, 48 insertions(+), 22 deletions(-)
+
+diff --git a/rpmdb/header.c b/rpmdb/header.c index 5d6c6cf..5a80df0 
+100644
+--- a/rpmdb/header.c
 b/rpmdb/header.c
+@@ -32,7 +32,7 @@ GENfree(HeaderIterator)
+ #endif/* __cplusplus */
+ 
+ #if defined(SUPPORT_IMPLICIT_TAG_DATA_TYPES)
+-extern void tagTypeValidate(HE_t he)
++extern void tagTypeValidate(HE_t he, unsigned int flags)
+   /*@*/;
+ #endif
+ 
+@@ -2281,7 +2281,7 @@ int headerGet(Header h, HE_t he, unsigned int 
+flags)  /*@-modfilesys@*/
+ /* XXX verify that explicit and implicit types are identical. */
+ if (rc)
+-  tagTypeValidate(he);
++tagTypeValidate(he, flags);
+ /*@=modfilesys@*/
+ #endif
+ 
+@@ -2304,7 +2304,7 @@ int headerPut(Header h, HE_t he, /*@unused@*/ 
+unsigned int flags)  #if defined(SUPPORT_IMPLICIT_TAG_DATA_TYPES)
+ /*@-modfilesys@*/
+ /* XXX verify that explicit and implicit types are identical. */
+-tagTypeValidate(he);
++tagTypeValidate(he, flags);
+ /*@=modfilesys@*/
+ #endif
+ 
+@@ -2329,7 +2329,7 @@ int headerMod(Header h, HE_t he, /*@unused@*/ 
+unsigned int flags)  #if defined(SUPPORT_IMPLICIT_TAG_DATA_TYPES)
+ /*@-modfilesys@*/
+ /* XXX verify that explicit and implicit types are identical. */
+-tagTypeValidate(he);
++tagTypeValidate(he, flags);
+ /*@=modfilesys@*/
+ #endif
+ 
+diff --git a/rpmdb/rpmtag.h b/rpmdb/rpmtag.h index ffce678..8f6a2bf 
+100644
+--- a/rpmdb/rpmtag.h
 b/rpmdb/rpmtag.h
+@@ -868,6 +868,7 @@ int headerGet(Header h, HE_t he, unsigned int flags)
+   /*@modifies he, internalState @*/;
+ #define   HEADERGET_NOEXTENSION   (1 << 0) /*!< Extension search 
disabler. */
+ #define   HEADERGET_NOI18NSTRING  (1 << 1) /*!< Return i18n strings as 
argv. */
++#define HEADERGET_SIGHEADER (1 << 2) /*!< Override signature tag data. */
+ 
+ /** \ingroup header
+  * Add or append tag container to header.
+diff --git a/rpmdb/tagname.c b/rpmdb/tagname.c index cfd1459..374ad77 
+100644
+--- a/rpmdb/tagname.c
 b/rpmdb/tagname.c
+@@ -515,28 +515,53 @@ tagStore_t tagStoreFree(tagStore_t dbiTags, 
+size_t dbiNTags)
+  * Validate that implicit and explicit types are identical.
+  *

Re: [OE-core] [jethro] [PATCH] rpm: Fix the Bug of SRPM String error

2017-05-16 Thread Zheng, Ruoqin

Ping

Zheng Ruoqin
Nanjing Fujitsu Nanda Software Tech. Co., Ltd.(FNST)
ADDR.: No.6 Wenzhu Road, Software Avenue,
   Nanjing, 210012, China
MAIL : zhengrq.f...@cn.fujistu.com


-Original Message-
From: Zheng, Ruoqin/郑 若钦 
Sent: Monday, May 08, 2017 6:16 PM
To: openembedded-core@lists.openembedded.org
Cc: Zheng, Ruoqin/郑 若钦 
Subject: [OE-core] [jethro] [PATCH] rpm: Fix the Bug of SRPM String error

Add a patch 0001-Fix-the-Bug-of-SRPM-String-error.patch to fix SRPM bug

When use bitbake to build a SRPM package, some sections of SRPM can't be 
displayed normally.
For example:
$ rpm -qpi zlib-1.2.8-r0.src.rpm
warning: zlib-1.2.8-r0.src.rpm: Header V4 DSA/SHA1 Signature, key ID e64962a6: 
NOKEY
Name: zlib
Version : 1.2.8
Release : r0
Architecture: i586
Install Date: (not installed)
Group   : ïº©
Size: 633007
License : Zlib
Signature   : DSA/SHA1, 2017å¹´04æœˆ22æ—¥ 00æ™‚38åˆ†03ç§’, Key ID 
bc5e0520e64962a6
Source RPM  : (none)
Build Date  : 2017å¹´04æœˆ22æ—¥ 00æ™‚38åˆ†03ç§’
Build Host  : force
Relocations : (not relocatable)
Packager: Poky 
URL : http://zlib.net/
Summary : Â¨Â¯
Description :
ïº©

Signed-off-by: Zheng Ruoqin 
---
 .../0001-Fix-the-Bug-of-SRPM-String-error.patch| 162 +
 meta/recipes-devtools/rpm/rpm_5.4.14.bb|   1 +
 2 files changed, 163 insertions(+)
 create mode 100644 
meta/recipes-devtools/rpm/rpm/0001-Fix-the-Bug-of-SRPM-String-error.patch

diff --git 
a/meta/recipes-devtools/rpm/rpm/0001-Fix-the-Bug-of-SRPM-String-error.patch 
b/meta/recipes-devtools/rpm/rpm/0001-Fix-the-Bug-of-SRPM-String-error.patch
new file mode 100644
index 000..f85e333
--- /dev/null
+++ b/meta/recipes-devtools/rpm/rpm/0001-Fix-the-Bug-of-SRPM-String-erro
+++ r.patch
@@ -0,0 +1,162 @@
+Subject: [PATCH] Fix the Bug of SRPM String error
+
+Upstream-Status: Backport from rpm 5.4.16 $ rpm -qpi 
+zlib-1.2.8-r0.src.rpm
+warning: zlib-1.2.8-r0.src.rpm: Header V4 DSA/SHA1 Signature, key ID e64962a6: 
NOKEY
+Name: zlib
+Version : 1.2.8
+Release : r0
+Architecture: i586
+Install Date: (not installed)
+Group   : Ø¯
+Size: 633007
+License : Zlib
+Signature   : DSA/SHA1, 2017å¹´04æœˆ22æ—¥ 00æ™‚38åˆ†03ç§’, Key ID 
bc5e0520e64962a6
+Source RPM  : (none)
+Build Date  : 2017å¹´04æœˆ22æ—¥ 00æ™‚38åˆ†03ç§’
+Build Host  : force
+Relocations : (not relocatable)
+Packager: Poky 
+URL : http://zlib.net/
+Summary : Â¨Â¯
+Description :
+Ø¯
+
+Signed-off-by: Zheng Ruoqin 
+---
+ rpmdb/header.c  |  8 
+ rpmdb/rpmtag.h  |  1 +
+ rpmdb/tagname.c | 61 
+-
+ 3 files changed, 48 insertions(+), 22 deletions(-)
+
+diff --git a/rpmdb/header.c b/rpmdb/header.c index 2f72a05..0e1d613 
+100644
+--- a/rpmdb/header.c
 b/rpmdb/header.c
+@@ -32,7 +32,7 @@ GENfree(HeaderIterator)
+ #endif/* __cplusplus */
+ 
+ #if defined(SUPPORT_IMPLICIT_TAG_DATA_TYPES)
+-extern void tagTypeValidate(HE_t he)
++extern void tagTypeValidate(HE_t he, unsigned int flags)
+   /*@*/;
+ #endif
+ 
+@@ -2273,7 +2273,7 @@ int headerGet(Header h, HE_t he, unsigned int 
+flags)  /*@-modfilesys@*/
+ /* XXX verify that explicit and implicit types are identical. */
+ if (rc)
+-  tagTypeValidate(he);
++   tagTypeValidate(he, flags);
+ /*@=modfilesys@*/
+ #endif
+ 
+@@ -2296,7 +2296,7 @@ int headerPut(Header h, HE_t he, /*@unused@*/ 
+unsigned int flags)  #if defined(SUPPORT_IMPLICIT_TAG_DATA_TYPES)
+ /*@-modfilesys@*/
+ /* XXX verify that explicit and implicit types are identical. */
+-tagTypeValidate(he);
++tagTypeValidate(he, flags);
+ /*@=modfilesys@*/
+ #endif
+ 
+@@ -2321,7 +2321,7 @@ int headerMod(Header h, HE_t he, /*@unused@*/ 
+unsigned int flags)  #if defined(SUPPORT_IMPLICIT_TAG_DATA_TYPES)
+ /*@-modfilesys@*/
+ /* XXX verify that explicit and implicit types are identical. */
+-tagTypeValidate(he);
++tagTypeValidate(he, flags);
+ /*@=modfilesys@*/
+ #endif
+ 
+diff --git a/rpmdb/rpmtag.h b/rpmdb/rpmtag.h index b4ad445..e5c895d 
+100644
+--- a/rpmdb/rpmtag.h
 b/rpmdb/rpmtag.h
+@@ -772,6 +772,7 @@ int headerGet(Header h, HE_t he, unsigned int flags)
+   /*@modifies he, internalState @*/;
+ #define   HEADERGET_NOEXTENSION   (1 << 0) /*!< Extension search 
disabler. */
+ #define   HEADERGET_NOI18NSTRING  (1 << 1) /*!< Return i18n strings as 
argv. */
++#define   HEADERGET_SIGHEADER (1 << 2) /*!< Override signature tag 
data. */
+ 
+ /** \ingroup header
+  * Add or append tag container to header.
+diff --git a/rpmdb/tagname.c b/rpmdb/tagname.c index c30a3c2..44fda1c 
+100644
+--- a/rpmdb/tagname.c
 b/rpmdb/tagname.c
+@@ -504,28 +504,53 @@ tagStore_t tagStoreFree(tagStore_t dbiTags, 
+size_t dbiNTags)
+  * Validate that implicit and explicit types are identical.
+  *

[OE-core] [[PATCH v2 1/3] kernel-fitimage: Support ext2.gz filesystems

2017-05-16 Thread Chunrong Guo

From: Chunrong Guo 

Signed-off-by: Chunrong Guo 
---
 meta/classes/kernel-fitimage.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/kernel-fitimage.bbclass 
b/meta/classes/kernel-fitimage.bbclass
index 2630b47..f40fbf3 100644
--- a/meta/classes/kernel-fitimage.bbclass
+++ b/meta/classes/kernel-fitimage.bbclass
@@ -369,7 +369,7 @@ fitimage_assemble() {
#
if [ "x${ramdiskcount}" = "x1" ] ; then
# Find and use the first initramfs image archive type we find
-   for img in cpio.lz4 cpio.lzo cpio.lzma cpio.xz cpio.gz cpio; do
+   for img in cpio.lz4 cpio.lzo cpio.lzma cpio.xz cpio.gz ext2.gz 
cpio; do

initramfs_path="${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE_NAME}.${img}"
echo "Using $initramfs_path"
if [ -e "${initramfs_path}" ]; then
-- 
1.9.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [[PATCH v2 3/3] kernel-fitimage: fix KERNEL_DEVICETREE includes subdirectory issue

2017-05-16 Thread Chunrong Guo

From: Chunrong Guo 

  * For example:
  KERNEL_DEVICETREE ?= "freescale/fsl-ls1046a-rdb.dtb"

  ${DTB}= "freescale/fsl-ls1046a-rdb.dtb"

  but only fsl-ls1046a-rdb.dtb  should be used in fit-image.its

Signed-off-by: Chunrong Guo 
---
 meta/classes/kernel-fitimage.bbclass | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/classes/kernel-fitimage.bbclass 
b/meta/classes/kernel-fitimage.bbclass
index 9fa836f..08644ea 100644
--- a/meta/classes/kernel-fitimage.bbclass
+++ b/meta/classes/kernel-fitimage.bbclass
@@ -347,6 +347,7 @@ fitimage_assemble() {
DTB=`basename ${DTB} | sed 's,\.dts$,.dtb,g'`
fi
DTB_PATH="arch/${ARCH}/boot/dts/${DTB}"
+DTB=`basename ${DTB}`
if [ ! -e "${DTB_PATH}" ]; then
DTB_PATH="arch/${ARCH}/boot/${DTB}"
fi
-- 
1.9.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [[PATCH v2 2/3] kernel-fitimage: set replacementtype to vmlinux

2017-05-16 Thread Chunrong Guo

From: Chunrong Guo 

Signed-off-by: Chunrong Guo 
---
 meta/classes/kernel-fitimage.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/kernel-fitimage.bbclass 
b/meta/classes/kernel-fitimage.bbclass
index f40fbf3..9fa836f 100644
--- a/meta/classes/kernel-fitimage.bbclass
+++ b/meta/classes/kernel-fitimage.bbclass
@@ -10,7 +10,7 @@ python __anonymous () {
 if d.getVar("UBOOT_ARCH") == "x86":
 replacementtype = "bzImage"
 else:
-replacementtype = "zImage"
+replacementtype = "vmlinux"
 
# Override KERNEL_IMAGETYPE_FOR_MAKE variable, which is internal
# to kernel.bbclass . We have to override it, since we pack zImage
-- 
1.9.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] linux-yocto: Use ??= to set LINUX_VERSION_EXTENSION

2017-05-16 Thread swee . aun . khor

From: sweeaun 

Use ??= to set LINUX_VERSION_EXTENSION default value.

Signed-off-by: sweeaun 
---
 meta/recipes-kernel/linux/linux-yocto.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-kernel/linux/linux-yocto.inc 
b/meta/recipes-kernel/linux/linux-yocto.inc
index 637506a..1119f22 100644
--- a/meta/recipes-kernel/linux/linux-yocto.inc
+++ b/meta/recipes-kernel/linux/linux-yocto.inc
@@ -48,7 +48,7 @@ KCONF_AUDIT_LEVEL ?= "1"
 KCONF_BSP_AUDIT_LEVEL ?= "0"
 KMETA_AUDIT ?= "yes"
 
-LINUX_VERSION_EXTENSION ?= "-yocto-${LINUX_KERNEL_TYPE}"
+LINUX_VERSION_EXTENSION ??= "-yocto-${LINUX_KERNEL_TYPE}"
 
 # Pick up shared functions
 inherit kernel
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [for-krogoth] Fix pseudo issue in some hosts

2017-05-16 Thread akuster808


Otavio,


On 05/16/2017 03:23 AM, Otavio Salvador wrote:

Hello guys,

I've been applying following change, on krogoth, and using without problems:

This is already in the last pull request.
http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=akuster/krogoth-next=b5765cc6e0fe6b64b1d39265feea67e6ee664d55



commit db2cf6ec905e2a8870d1adb6c069c08c36055b01 (HEAD -> krogoth)
Author: Richard Purdie 
Date:   Wed May 18 19:28:29 2016 +0100

 pseudo: Work around issues with glibc 2.24

 There are issues with a change made to RTLD_NEXT behaviour in glibc 2.24
 and that change was also backported to older glibc versions in some distros
 like Fedora 23. This adds a workaround whilst the pseudo maintainer fixes
 various issues properly.

 Signed-off-by: Richard Purdie 
 (cherry picked from commit 21c38a091c4a1917f62a942c4751b0fd11dce340)

It would be great if this could be queued together of the pending
merges, for next updates.


thanks,
Armin




--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [morty][PATCH] sstate.bbclass: update .siginfo atime

2017-05-16 Thread akuster808




On 05/16/2017 02:50 PM, Denys Dmytriyenko wrote:

On Mon, Apr 24, 2017 at 09:08:55AM -0700, akuster808 wrote:

On 04/21/2017 04:18 PM, Denys Dmytriyenko wrote:

Ping. It's been over a month now...

its in my stagging since 4/1.  The AB has been failing with my
current changes.

Is morty now dead?


No. why do you say that? I just sent a pull request over the weekend for 
both Krogoth and Morty. Krogoth is the one heading to community 
supported now that Pyro is out.


this changes is in the pull request
http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/meta/classes/sstate.bbclass?h=akuster/morty-next=838580431a8923bf38820f10963aca4af51f6f97

- armin






On Mon, Mar 20, 2017 at 01:38:20PM -0400, Denys Dmytriyenko wrote:

From: Ed Bartosh 

.siginfo files are not being accessed from local or NFS-mounted
sstate mirrors when sstate package is installed, so their atime
is not updated. If sstate mirror is cleaned based on access time,
they get deleted, even though they are still being used.

Updated atime of .siginfo symlinks with 'touch -a'. This command
dereferences symlinks pointing to the local mirror and updates
atime of the .siginfo file on the mirror.

[YOCTO #10857]

Signed-off-by: Ed Bartosh 
Signed-off-by: Ross Burton 
Signed-off-by: Denys Dmytriyenko 
---
  meta/classes/sstate.bbclass | 2 ++
  1 file changed, 2 insertions(+)

diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 8643f3d..4fdfcc8 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -724,6 +724,8 @@ python sstate_sign_package () {
  #
  sstate_unpack_package () {
tar -xvzf ${SSTATE_PKG}
+   # update .siginfo atime on local/NFS mirror
+   [ -h ${SSTATE_PKG}.siginfo ] && touch -a ${SSTATE_PKG}.siginfo
# Use "! -w ||" to return true for read only files
[ ! -w ${SSTATE_PKG} ] || touch --no-dereference ${SSTATE_PKG}
[ ! -w ${SSTATE_PKG}.sig ] || [ ! -e ${SSTATE_PKG}.sig ] || touch 
--no-dereference ${SSTATE_PKG}.sig
--
2.7.4

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [morty][PATCH] sstate.bbclass: update .siginfo atime

2017-05-16 Thread Denys Dmytriyenko

On Mon, Apr 24, 2017 at 09:08:55AM -0700, akuster808 wrote:
> On 04/21/2017 04:18 PM, Denys Dmytriyenko wrote:
> >Ping. It's been over a month now...
> its in my stagging since 4/1.  The AB has been failing with my
> current changes.

Is morty now dead?


> >On Mon, Mar 20, 2017 at 01:38:20PM -0400, Denys Dmytriyenko wrote:
> >>From: Ed Bartosh 
> >>
> >>.siginfo files are not being accessed from local or NFS-mounted
> >>sstate mirrors when sstate package is installed, so their atime
> >>is not updated. If sstate mirror is cleaned based on access time,
> >>they get deleted, even though they are still being used.
> >>
> >>Updated atime of .siginfo symlinks with 'touch -a'. This command
> >>dereferences symlinks pointing to the local mirror and updates
> >>atime of the .siginfo file on the mirror.
> >>
> >>[YOCTO #10857]
> >>
> >>Signed-off-by: Ed Bartosh 
> >>Signed-off-by: Ross Burton 
> >>Signed-off-by: Denys Dmytriyenko 
> >>---
> >>  meta/classes/sstate.bbclass | 2 ++
> >>  1 file changed, 2 insertions(+)
> >>
> >>diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
> >>index 8643f3d..4fdfcc8 100644
> >>--- a/meta/classes/sstate.bbclass
> >>+++ b/meta/classes/sstate.bbclass
> >>@@ -724,6 +724,8 @@ python sstate_sign_package () {
> >>  #
> >>  sstate_unpack_package () {
> >>tar -xvzf ${SSTATE_PKG}
> >>+   # update .siginfo atime on local/NFS mirror
> >>+   [ -h ${SSTATE_PKG}.siginfo ] && touch -a ${SSTATE_PKG}.siginfo
> >># Use "! -w ||" to return true for read only files
> >>[ ! -w ${SSTATE_PKG} ] || touch --no-dereference ${SSTATE_PKG}
> >>[ ! -w ${SSTATE_PKG}.sig ] || [ ! -e ${SSTATE_PKG}.sig ] || touch 
> >> --no-dereference ${SSTATE_PKG}.sig
> >>-- 
> >>2.7.4
> >>
> >>-- 
> >>___
> >>Openembedded-core mailing list
> >>Openembedded-core@lists.openembedded.org
> >>http://lists.openembedded.org/mailman/listinfo/openembedded-core
> 
> -- 
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] tcmode-default: use SDK_SYS suffix instead of SDK_ARCH for binutils-crosssdk

2017-05-16 Thread Denys Dmytriyenko

Ping

On Tue, May 09, 2017 at 05:58:56PM -0400, Denys Dmytriyenko wrote:
> From: Denys Dmytriyenko 
> 
> Commit d2eb70e809d482c493922f23aef4409cfd82 has changed suffixes for
> all -crosssdk packages from SDK_ARCH to SDK_SYS, but missed one line with
> binutils-crosssdk. This change fixes that omission.
> 
> Signed-off-by: Denys Dmytriyenko 
> ---
>  meta/conf/distro/include/tcmode-default.inc | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/conf/distro/include/tcmode-default.inc 
> b/meta/conf/distro/include/tcmode-default.inc
> index 3db16e8..bdc70cf 100644
> --- a/meta/conf/distro/include/tcmode-default.inc
> +++ b/meta/conf/distro/include/tcmode-default.inc
> @@ -48,7 +48,7 @@ PREFERRED_VERSION_nativesdk-libgcc-initial ?= 
> "${SDKGCCVERSION}"
>  PREFERRED_VERSION_binutils ?= "${BINUVERSION}"
>  PREFERRED_VERSION_binutils-native ?= "${BINUVERSION}"
>  PREFERRED_VERSION_binutils-cross-${TARGET_ARCH} ?= "${BINUVERSION}"
> -PREFERRED_VERSION_binutils-crosssdk-${SDK_ARCH} ?= "${BINUVERSION}"
> +PREFERRED_VERSION_binutils-crosssdk-${SDK_SYS} ?= "${BINUVERSION}"
>  PREFERRED_VERSION_binutils-cross-canadian-${TRANSLATED_TARGET_ARCH} ?= 
> "${BINUVERSION}"
>  PREFERRED_VERSION_gdb ?= "${GDBVERSION}"
>  PREFERRED_VERSION_gdb-cross-${TARGET_ARCH} ?= "${GDBVERSION}"
> -- 
> 2.7.4
> 
> -- 
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] python3-nose: rename ${bindir}/nosetests into ${bindir}/nosetests3

2017-05-16 Thread Denys Dmytriyenko

Ping

On Tue, May 09, 2017 at 03:31:54PM -0400, Denys Dmytriyenko wrote:
> From: Denys Dmytriyenko 
> 
> This resolves a conflict when both python-nose and python3-nose are pulled
> into an image and try to install ${bindir}/nosetests binary.
> 
> This matches with how other distros are solving this problem, e.g. Debian:
> https://packages.debian.org/jessie/all/python3-nose/filelist
> 
> Also, other packages like python3-setuptools are already doing the same with
> their binaries.
> 
> Signed-off-by: Denys Dmytriyenko 
> ---
>  meta/recipes-devtools/python/python3-nose_1.3.7.bb | 4 
>  1 file changed, 4 insertions(+)
> 
> diff --git a/meta/recipes-devtools/python/python3-nose_1.3.7.bb 
> b/meta/recipes-devtools/python/python3-nose_1.3.7.bb
> index 99bba44..1e2ff74 100644
> --- a/meta/recipes-devtools/python/python3-nose_1.3.7.bb
> +++ b/meta/recipes-devtools/python/python3-nose_1.3.7.bb
> @@ -17,6 +17,10 @@ S = "${WORKDIR}/nose-${PV}"
>  
>  inherit setuptools3
>  
> +do_install_append() {
> +mv ${D}${bindir}/nosetests ${D}${bindir}/nosetests3
> +}
> +
>  RDEPENDS_${PN}_class-target = "\
>python3-unittest \
>"
> -- 
> 2.7.4
> 
> -- 
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] eudev: opkg not automatically upgrading from udev to eudev

2017-05-16 Thread Bryan Evenson

I have an image based on core-image-minimal that uses sysvinit for init and 
opkg for package management.  I am doing some upgrade testing and I see an 
issue with eudev.  I am testing upgrading an image that was built under dizzy 
to my latest image built under morty.  I'm first testing that all the correct 
packages and new dependencies are correctly being identified for upgrade by 
running "opkg update; opkg upgrade --download-only" from the command line and 
comparing what is in the opkg cache directory with what was available.  All 
packages except eudev and udev-cache are being downloaded for upgrade.

I ran the opkg upgrade again with the -V4 option for additional debug output  
Here's what I saw in one section regarding udev:

pkg_hash_fetch_best_installation_candidate: Best installation candidate for 
udev:
pkg_hash_fetch_best_installation_candidate: apkg=udev nprovides=2.
pkg_hash_fetch_best_installation_candidate: Adding eudev to providers.
pkg_hash_fetch_best_installation_candidate: Adding udev to providers.
pkg_hash_fetch_best_installation_candidate: eudev arch=armv5e arch_priority=41 
version=3.2.
pkg_hash_fetch_best_installation_candidate: udev arch=arm926ejste 
arch_priority=51 version=182.
pkg_hash_fetch_best_installation_candidate: Candidate: udev 182.
pkg_hash_fetch_best_installation_candidate: 2 matching pkgs for apkg=udev:
pkg_hash_fetch_best_installation_candidate: eudev 3.2 armv5e
pkg_hash_fetch_best_installation_candidate: udev 182 arm926ejste
opkg_upgrade_pkg: Comparing visible versions of pkg udev:
182-r9.9 is installed
182-r9.9 is available
0 was comparison result
opkg_upgrade_pkg: Package udev (182-r9.9) installed in root is up to date.

I'm assuming the problem is that udev is version 182 and eudev is version 3.2 
and that opkg doesn't think udev needs to be upgraded.  Is there an RDEPENDS 
that needs to be added somewhere so opkg sees that it needs to upgrade udev to 
eudev?

Thanks,
Bryan
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [RFC][PATCH 0/6] development vs. production builds

2017-05-16 Thread Alexander Kanavin


On 05/16/2017 05:25 PM, Patrick Ohly wrote:


It is done like that already, it's just not very consistent from what I
can see. For example, core-image-sato-dev.bb:
=
require core-image-sato.bb

DESCRIPTION = "Image with Sato for development work. It includes
everything \
within core-image-sato plus a native toolchain, application development
and \
testing libraries, profiling and debug symbols."

IMAGE_FEATURES += "dev-pkgs"
=


That's different. Here an image recipe specifies what it is meant to
*contain*, not how it is meant to *behave*.


You can totally specify behavioral IMAGE_FEATURES in image recipes. The 
distinction is not formally specified or enforced anywhere, as far as I 
know.



One would need core-image-sato-dev-production.bb (no debug-tweaks,
dev-pkgs), core-image-sato-dev-debug.bb (debug-tweaks, dev-pkgs),
core-image-sato-production.bb (no debug-tweaks, no dev-pkgs),
core-image-sato-debug.bb (debug-tweaks, no dev-pkgs).

Allowing EXTRA_IMAGE_FEATURES in local.conf.sample avoids that.


Consider that the suffix in image recipe name is essentially same as 
your IMAGE_MODE idea. It specifies who is the target audience for the 
image. Then you can avoid combinatorial explosion, if you define who the 
image is meant for:


someimage-developers.bb (debug tweaks, dev-pkgs)
someimage-endusers.bb (no debug tweaks, no dev-pkgs)


I'm not a big fan of placing INHERIT into local.conf either, by the way.
I believe in functional programming principles, and this goes directly
against them.


It makes sense to me when the functionality is orthogonal to the
content, like enabling buildhistory.


Unfortunately, there is no way to enforce orthogonality. The door is 
wide open for awful hacks here.


Alex

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 0/2] Allow oe-pkgdata-util package-info to display more info

2017-05-16 Thread Peter Kjellerstedt

> -Original Message-
> From: Richard Purdie [mailto:richard.pur...@linuxfoundation.org]
> Sent: den 16 maj 2017 15:15
> To: Peter Kjellerstedt ; openembedded-
> c...@lists.openembedded.org
> Subject: Re: [OE-core] [PATCH 0/2] Allow oe-pkgdata-util package-info
> to display more info
> 
> On Tue, 2017-05-16 at 12:56 +0200, Peter Kjellerstedt wrote:
> > After a build in our autobuilder, we use `oe-pkgdata-util package-
> > info
> > -f package.manifest` to store a file with information about each
> > installed package. This is typically used to compare builds later
> > on. Since not every difference to a package is explained by its
> > version, we have found it beneficial to also include the SRC_URI in
> > the generated file.
> >
> > This patch set adds SRC_URI to the pkgdata that is stored for each
> > package, and adds a new option to oe-pkgdata-util package-info,
> > -e , that can be used to display extra variables from the
> > pkgdata.
> 
> I'm going to say no to this.
> 
> The reason is that pkgdata is not really about collecting up all build
> information. If you want to know how two different builds differ, you'd
> use the sigdata files. If I take this patch, more will follow where you
> find some new difference you want to track and there are other
> mechanisms I'd suggest (buildhistory and siginfo for starters). I don't
> want to turn the pkgdata files into something they're not.
> 
> Cheers,
> 
> Richard

As you may have expected, that was not the response I had hoped for.
However, I will see if we can come to a working solution that you can 
accept.

Since I hope that the change to oe-pkgdata-util is acceptable to you, 
I will focus on the change in package.bbclass.

Regarding siginfo: I assume it has the information in there somewhere. 
However, I have been working with OE for five years now, and it is still 
basically a black hole to me. I have no idea how to do anything useful 
with it.

Regarding buildhistory: AFAICT there is nothing in there about neither 
recipe names nor SRC_URI. I guess that can be added, though, if really 
needed.

On the other hand, we have a tool, oe-pkgdata-util, that provides a 
simple interface to access the package information, and can produce a 
simple information file (oe-pkgdata-util package-info) with one line 
per package. This file is simple enough that I can give it to our 
maintenance team and they can look at it to see if there are any 
differences they need to know about. An additional benefit here is that 
we can run oe-pkgdata-util from within a bitbake task to generate the 
file together with the other artefacts we produce for a release.

That said, I can understand that you do not want to add information to 
the pkgdata that is not really needed to build. However, would you accept 
a way to add to this data, e.g., by specifying a BitBake variable such as 
EXTRA_PKGDATA_VARS with the names of the extra variables that should be 
stored in pkgdata? That way it would not affect anyone unless they 
actually need this extra data, and I would not have the burden of carrying 
a backported version of package.bbclass forever in our layers with all the 
extra maintenance that incurs.

//Peter

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 5/9] gstreamer1.0-plugins-ugly: upgrade to version 1.12.0

2017-05-16 Thread Maxin B. John

Hi,

On Tue, May 16, 2017 at 01:26:15PM +0200, Carlos Rafael Giani wrote:
> * libmad plugin was removed in 1.12.0, since mad is GPLed, unmaintained,
>   and both gst-libav & the mpg123 plugin are fully functional alternatives
> 
> Signed-off-by: Carlos Rafael Giani 
> ---
>  meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-ugly.inc   | 1 -
>  ...1.0-plugins-ugly_1.10.4.bb => gstreamer1.0-plugins-ugly_1.12.0.bb} | 4 
> ++--
>  2 files changed, 2 insertions(+), 3 deletions(-)
>  rename 
> meta/recipes-multimedia/gstreamer/{gstreamer1.0-plugins-ugly_1.10.4.bb => 
> gstreamer1.0-plugins-ugly_1.12.0.bb} (76%)
> 
> diff --git a/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-ugly.inc 
> b/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-ugly.inc
> index 708ad7a..60aa968 100644
> --- a/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-ugly.inc
> +++ b/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-ugly.inc
> @@ -18,7 +18,6 @@ PACKAGECONFIG[amrwb]= 
> "--enable-amrwb,--disable-amrwb,opencore-amr"
>  PACKAGECONFIG[cdio] = "--enable-cdio,--disable-cdio,libcdio"
>  PACKAGECONFIG[dvdread]  = "--enable-dvdread,--disable-dvdread,libdvdread"
>  PACKAGECONFIG[lame] = "--enable-lame,--disable-lame,lame"
> -PACKAGECONFIG[mad]  = "--enable-mad,--disable-mad,libmad"
>  PACKAGECONFIG[mpeg2dec] = "--enable-mpeg2dec,--disable-mpeg2dec,mpeg2dec"
>  PACKAGECONFIG[mpg123]   = "--enable-mpg123,--disable-mpg123,mpg123"
>  PACKAGECONFIG[x264] = "--enable-x264,--disable-x264,x264"
> diff --git 
> a/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-ugly_1.10.4.bb 
> b/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-ugly_1.12.0.bb
> similarity index 76%
> rename from 
> meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-ugly_1.10.4.bb
> rename to 
> meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-ugly_1.12.0.bb
> index 92a2caa..3bf9a9c 100644
> --- a/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-ugly_1.10.4.bb
> +++ b/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-ugly_1.12.0.bb
> @@ -7,7 +7,7 @@ SRC_URI = " \
>  
> http://gstreamer.freedesktop.org/src/gst-plugins-ugly/gst-plugins-ugly-${PV}.tar.xz
>  \
>  file://0001-introspection.m4-prefix-pkgconfig-paths-with-PKG_CON.patch \
>  "
> -SRC_URI[md5sum] = "c68d0509c9980b0b70a4b836ff73fff1"
> -SRC_URI[sha256sum] = 
> "6386c77ca8459cba431ed0b63da780c7062c7cc48055d222024d8eaf198ffa59"
> +SRC_URI[md5sum] = "30e815834819a4890eb1ef6ef67cf68d"
> +SRC_URI[sha256sum] = 
> "5e68ba5046e83ee87b17d7a13931e6091466fd771e1338c5b929ee0944d40ad6"
>  
>  S = "${WORKDIR}/gst-plugins-ugly-${PV}"

For gstreamer1.0-plugins-ugly, we can remove the build dependency on 
"libid3tag" since it
is not needed anymore.

Best Regards,
Maxin
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [RFC][PATCH 0/6] development vs. production builds

2017-05-16 Thread Patrick Ohly

On Tue, 2017-05-16 at 17:02 +0300, Alexander Kanavin wrote:
> On 05/16/2017 04:47 PM, Patrick Ohly wrote:
> 
> > Then why is not already done like that in practice? Is it just because
> > OE-core and Poky set such a bad precedence with teaching developers to
> > add EXTRA_IMAGE_FEATURES ?= "debug-tweaks" to make the images usable,
> > and then that approach gets copied?
> 
> It is done like that already, it's just not very consistent from what I 
> can see. For example, core-image-sato-dev.bb:
> =
> require core-image-sato.bb
> 
> DESCRIPTION = "Image with Sato for development work. It includes 
> everything \
> within core-image-sato plus a native toolchain, application development 
> and \
> testing libraries, profiling and debug symbols."
> 
> IMAGE_FEATURES += "dev-pkgs"
> =

That's different. Here an image recipe specifies what it is meant to
*contain*, not how it is meant to *behave*.

One would need core-image-sato-dev-production.bb (no debug-tweaks,
dev-pkgs), core-image-sato-dev-debug.bb (debug-tweaks, dev-pkgs),
core-image-sato-production.bb (no debug-tweaks, no dev-pkgs),
core-image-sato-debug.bb (debug-tweaks, no dev-pkgs).

Allowing EXTRA_IMAGE_FEATURES in local.conf.sample avoids that.

> I'm not a big fan of placing INHERIT into local.conf either, by the way. 
> I believe in functional programming principles, and this goes directly 
> against them.

It makes sense to me when the functionality is orthogonal to the
content, like enabling buildhistory.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2 00/20] support profile-optimized build for Python

2017-05-16 Thread Markus Lehtonen

On 16/05/2017, 16.45, "Richard Purdie"  
wrote:

On Tue, 2017-05-16 at 16:18 +0300, Markus Lehtonen wrote:
> This patchset makes it possible to make a PGO (profile-guided-
> optimization) build of python. This version of the patchset is almost
> identical to v1 submitted back in February, with these changes:
> - rebased on top of latest oe-core master - exclude profile data for
> Modules/posixmodule of Python 2.7 as it was not working correctly
> 
> [YOCTO #9338]
> 

I'm wondering if recipe specific sysroots might have made this problem
a bit easier?

It should now be possible to build the two packages with conflicting
files and as long as you exclude the recipe from the shlibs code and
set the RPROVIDES correctly, it should all work without as many
invasive changes?

Of course I haven't tested that...

I'll try that out. That, accompanied with dropping Python 2.x support (thanks 
Alexander) would make the patchset significantly simpler.

Cheers,
   MArkus

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2 00/20] support profile-optimized build for Python

2017-05-16 Thread Alexander Kanavin


On 05/16/2017 05:07 PM, Markus Lehtonen wrote:


You're probably right. There's not much use in churning
soon-to-be-deprecated Python 2.x. I will submit a new patchset for
Python 3.x only. Python 2.x could be added later if deprecation
wouldn't happen for some reason.


The deprecation is not a single point in time, it's an ongoing process. 
A lot of work was done last spring, and some more work has been done 
this spring. The situation with what still needs py 2.x will be 
revisited again in a year's time. I just want to avoid regressions on 
this front :)



Alex
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2 00/20] support profile-optimized build for Python

2017-05-16 Thread Markus Lehtonen

On 16/05/2017, 16.23, "Alexander Kanavin"  
wrote:

On 05/16/2017 04:18 PM, Markus Lehtonen wrote:
> This patchset makes it possible to make a PGO 
(profile-guided-optimization)
> build of python. This version of the patchset is almost identical to v1
> submitted back in February, with these changes:
> - rebased on top of latest oe-core master
> - exclude profile data for Modules/posixmodule of Python 2.7 as it was not
>   working correctly

How about doing this only for Python 3.x? We'd like to stick to our 
resolution of deprecating 2.x, which means not introducing any new 
dependencies or images that need it.

You're probably right. There's not much use in churning soon-to-be-deprecated 
Python 2.x. I will submit a new patchset for Python 3.x only. Python 2.x could 
be added later if deprecation wouldn't happen for some reason.

Also, python-pgo-image commits should probably be all squashed into one 
(with comments within files replacing the commit messages).

Point taken.

Thanks,
  Markus

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [RFC][PATCH 0/6] development vs. production builds

2017-05-16 Thread Alexander Kanavin


On 05/16/2017 04:47 PM, Patrick Ohly wrote:


Then why is not already done like that in practice? Is it just because
OE-core and Poky set such a bad precedence with teaching developers to
add EXTRA_IMAGE_FEATURES ?= "debug-tweaks" to make the images usable,
and then that approach gets copied?


It is done like that already, it's just not very consistent from what I 
can see. For example, core-image-sato-dev.bb:

=
require core-image-sato.bb

DESCRIPTION = "Image with Sato for development work. It includes 
everything \
within core-image-sato plus a native toolchain, application development 
and \

testing libraries, profiling and debug symbols."

IMAGE_FEATURES += "dev-pkgs"
=


I think everyone agrees that removing "debug-tweaks" would be a good
idea. But completely removing the global (sic!) EXTRA_IMAGE_FEATURES in
local.conf.sample would go even further, and I am not sure how the
reactions to that would be. I suspect there are people who find it
useful to have just one image recipe that gets build in different
configurations (dangerous and not so dangerous).


I'm not sure either, but I think that's not actually a bad idea - 
dropping support for it altogether. :)


I'm not a big fan of placing INHERIT into local.conf either, by the way. 
I believe in functional programming principles, and this goes directly 
against them.


Alex
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [RFC][PATCH 0/6] development vs. production builds

2017-05-16 Thread Patrick Ohly

On Tue, 2017-05-16 at 14:49 +0300, Alexander Kanavin wrote:
> On 05/16/2017 11:21 AM, Patrick Ohly wrote:
> 
> >> While the "development/production" switch may be great for some projects,
> >> it'll make things only more complicated for others while gaining nothing 
> >> above
> >> what we have now.
> >
> > What about the approach I outlined in my reponse to Richard, where we
> > just introduce the IMAGE_MODE mechanism in OE-core without defining
> > specific modes? Would you find that useful?
> 
> Please no. Global variables are a headache, and the less we have of them 
> the better.
>
> Here in particularly, what is wrong with defining:
> 
> patricks-awesome-image-developers.bb
> (containing IMAGE_FEATURE = "no-password-for-anything")
> 
> patricks-awesome-image-production.bb
> (containing IMAGE_FEATURE = "serious-security")
> 
> and the common bits can go to patricks-awesome-image.inc.

Then why is not already done like that in practice? Is it just because
OE-core and Poky set such a bad precedence with teaching developers to
add EXTRA_IMAGE_FEATURES ?= "debug-tweaks" to make the images usable,
and then that approach gets copied?

I think everyone agrees that removing "debug-tweaks" would be a good
idea. But completely removing the global (sic!) EXTRA_IMAGE_FEATURES in
local.conf.sample would go even further, and I am not sure how the
reactions to that would be. I suspect there are people who find it
useful to have just one image recipe that gets build in different
configurations (dangerous and not so dangerous).

Feel free to prepare and propose a patch that implements the idea above
for OE-core; I personally won't, I've had enough negative feedback for
this week already ;-}

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2 00/20] support profile-optimized build for Python

2017-05-16 Thread Richard Purdie

On Tue, 2017-05-16 at 16:18 +0300, Markus Lehtonen wrote:
> This patchset makes it possible to make a PGO (profile-guided-
> optimization) build of python. This version of the patchset is almost
> identical to v1 submitted back in February, with these changes:
> - rebased on top of latest oe-core master - exclude profile data for
> Modules/posixmodule of Python 2.7 as it was not working correctly
> 
> [YOCTO #9338]
> 

I'm wondering if recipe specific sysroots might have made this problem
a bit easier?

It should now be possible to build the two packages with conflicting
files and as long as you exclude the recipe from the shlibs code and
set the RPROVIDES correctly, it should all work without as many
invasive changes?

Of course I haven't tested that...

Cheers,

Richard
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2 00/20] support profile-optimized build for Python

2017-05-16 Thread Alexander Kanavin


On 05/16/2017 04:18 PM, Markus Lehtonen wrote:

This patchset makes it possible to make a PGO (profile-guided-optimization)
build of python. This version of the patchset is almost identical to v1
submitted back in February, with these changes:
- rebased on top of latest oe-core master
- exclude profile data for Modules/posixmodule of Python 2.7 as it was not
  working correctly


How about doing this only for Python 3.x? We'd like to stick to our 
resolution of deprecating 2.x, which means not introducing any new 
dependencies or images that need it.


Also, python-pgo-image commits should probably be all squashed into one 
(with comments within files replacing the commit messages).


Alex

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] scripts: add pkgdataui

2017-05-16 Thread Ross Burton

pkgdataui is a Python 3/GObject Introspection/GTK+ 3 tool to browse the pkgdata
database at your leisure.  By being graphical it is easier to explore and can
follow links between packages.

This is very much a work in progress, so be gentle and patches are welcome.

Signed-off-by: Ross Burton 
---
 scripts/pkgdataui.glade | 335 
 scripts/pkgdataui.py| 241 ++
 2 files changed, 576 insertions(+)
 create mode 100644 scripts/pkgdataui.glade
 create mode 100755 scripts/pkgdataui.py

diff --git a/scripts/pkgdataui.glade b/scripts/pkgdataui.glade
new file mode 100644
index 000..04e987b
--- /dev/null
+++ b/scripts/pkgdataui.glade
@@ -0,0 +1,335 @@
+
+
+
+  
+  
+
+  
+  
+  
+  
+
+  
+  
+
+  
+  
+  
+  
+
+  
+  
+
+  
+  
+  
+  
+
+  
+  
+
+  
+  
+
+  
+  
+False
+Package Data Browser
+accessories-dictionary
+True
+
+  
+True
+False
+4
+4
+4
+4
+vertical
+4
+
+  
+False
+pkgdata_store
+1
+
+  
+  
+0
+  
+
+  
+  
+False
+True
+0
+  
+
+
+  
+True
+True
+200
+True
+
+  
+True
+True
+in
+100
+
+  
+True
+True
+recipe_store
+0
+True
+False
+
+  
+
+
+  
+fixed
+Recipe
+
+  
+  
+0
+  
+
+  
+
+  
+
+  
+  
+False
+True
+  
+
+
+  
+True
+True
+200
+True
+
+  
+True
+True
+in
+100
+
+  
+True
+True
+package_store
+0
+False
+
+  
+
+
+  
+True
+autosize
+Package
+0
+
+  
+  
+0
+  
+
+  
+
+
+  
+True
+autosize
+Size
+1
+
+  
+
+  
+
+  
+
+  
+  
+False
+True
+  
+
+
+  
+True
+False
+4
+vertical
+4
+
+  
+True
+False
+0
+label
+  
+  
+False
+True
+0
+  
+
+
+  
+True
+False
+0
+depends_label
+True
+False
+  
+  
+False
+True
+1
+  
+
+
+  
+True
+False
+0
+recs_label
+True
+

[OE-core] [PATCH v2 13/20] python3: add python-profile-opt3 recipe

2017-05-16 Thread Markus Lehtonen

Similar to python-profile-opt for Python 2.7, this is a special version
of Python3 used for profile guided optimization. It is used for
generating profile data on the target device. Libpython3 is renamed to
libpython-profile-opt3 in order to prevent soname clash with "normal"
libpython3.

[YOCTO #9338]

Signed-off-by: Markus Lehtonen 
---
 meta/recipes-devtools/python/python-profile-opt3   |   1 +
 .../python/python-profile-opt3_3.5.2.bb|  15 +++
 ...efile-add-install_generate_profile-target.patch |  25 +
 ...name-libpython3-to-libpython-profile-opt3.patch | 111 +
 meta/recipes-devtools/python/python3_3.5.2.bb  |  46 +
 5 files changed, 178 insertions(+), 20 deletions(-)
 create mode 12 meta/recipes-devtools/python/python-profile-opt3
 create mode 100644 meta/recipes-devtools/python/python-profile-opt3_3.5.2.bb
 create mode 100644 
meta/recipes-devtools/python/python3/Makefile-add-install_generate_profile-target.patch
 create mode 100644 
meta/recipes-devtools/python/python3/rename-libpython3-to-libpython-profile-opt3.patch

diff --git a/meta/recipes-devtools/python/python-profile-opt3 
b/meta/recipes-devtools/python/python-profile-opt3
new file mode 12
index 00..b8a0adbbb9
--- /dev/null
+++ b/meta/recipes-devtools/python/python-profile-opt3
@@ -0,0 +1 @@
+python3
\ No newline at end of file
diff --git a/meta/recipes-devtools/python/python-profile-opt3_3.5.2.bb 
b/meta/recipes-devtools/python/python-profile-opt3_3.5.2.bb
new file mode 100644
index 00..e85807f9f9
--- /dev/null
+++ b/meta/recipes-devtools/python/python-profile-opt3_3.5.2.bb
@@ -0,0 +1,15 @@
+require python3_${PV}.bb
+
+SRC_URI += "file://rename-libpython3-to-libpython-profile-opt3.patch \
+file://Makefile-add-install_generate_profile-target.patch \
+   "
+
+# Use special prefix in order to prevent clash with the normal python3 package
+STAGING_INCDIR_DEFAULT = "${STAGING_DIR_HOST}/usr/include"
+STAGING_LIBDIR_DEFAULT = "${STAGING_DIR_HOST}/usr/${baselib}"
+TARGET_CFLAGS += "-I${STAGING_INCDIR_DEFAULT}"
+TARGET_CPPFLAGS += "-I${STAGING_INCDIR_DEFAULT}"
+prefix = "/opt"
+exec_prefix = "/opt"
+
+PYTHON3_MAKE_TARGET = "build_all_generate_profile"
diff --git 
a/meta/recipes-devtools/python/python3/Makefile-add-install_generate_profile-target.patch
 
b/meta/recipes-devtools/python/python3/Makefile-add-install_generate_profile-target.patch
new file mode 100644
index 00..7aee7fb799
--- /dev/null
+++ 
b/meta/recipes-devtools/python/python3/Makefile-add-install_generate_profile-target.patch
@@ -0,0 +1,25 @@
+Makefile: add install_generate_profile target
+
+Making it possible to install the binaries with profile generation
+enabled. We need this as linking is also performed at the install stage.
+
+Upstream-Status: Inappropriate [oe-specific]
+
+Signed-off-by: Markus Lehtonen 
+
+diff --git a/Makefile.pre.in b/Makefile.pre.in
+index 2b87614..826155b 100644
+--- a/Makefile.pre.in
 b/Makefile.pre.in
+@@ -1046,6 +1046,8 @@ QUICKTESTOPTS=   $(TESTOPTS) -x test_subprocess test_io 
test_lib2to3 \
+ quicktest:all platform
+   $(TESTRUNNER) $(QUICKTESTOPTS)
+ 
++install_generate_profile:
++  $(MAKE) install LDFLAGS="$(LDFLAGS) $(PGO_PROF_GEN_FLAG) @LTOFLAGS@" 
LIBS="$(LIBS)"
+ 
+ install: @FRAMEWORKINSTALLFIRST@ commoninstall bininstall maninstall 
@FRAMEWORKINSTALLLAST@
+   if test "x$(ENSUREPIP)" != "xno"  ; then \
+-- 
+2.6.6
+
diff --git 
a/meta/recipes-devtools/python/python3/rename-libpython3-to-libpython-profile-opt3.patch
 
b/meta/recipes-devtools/python/python3/rename-libpython3-to-libpython-profile-opt3.patch
new file mode 100644
index 00..4ba553107f
--- /dev/null
+++ 
b/meta/recipes-devtools/python/python3/rename-libpython3-to-libpython-profile-opt3.patch
@@ -0,0 +1,111 @@
+rename libpython3 to libpython-profile-opt3
+
+Upstream-Status: Inappropriate [oe-specific]
+
+Signed-off-by: Markus Lehtonen 
+
+diff --git a/Lib/distutils/command/build_ext.py 
b/Lib/distutils/command/build_ext.py
+index f03a4e3..4c65aff 100644
+--- a/Lib/distutils/command/build_ext.py
 b/Lib/distutils/command/build_ext.py
+@@ -746,7 +746,7 @@ class build_ext(Command):
+ else:
+ from distutils import sysconfig
+ if sysconfig.get_config_var('Py_ENABLE_SHARED'):
+-pythonlib = 'python{}.{}{}'.format(
++pythonlib = 'python-profile-opt{}.{}{}'.format(
+ sys.hexversion >> 24, (sys.hexversion >> 16) & 0xff,
+ sysconfig.get_config_var('ABIFLAGS'))
+ return ext.libraries + [pythonlib]
+diff --git a/Makefile.pre.in b/Makefile.pre.in
+index bace00e..2b87614 100644
+--- a/Makefile.pre.in
 b/Makefile.pre.in
+@@ -622,7 +622,7 @@ $(LIBRARY): $(LIBRARY_OBJS)
+   $(AR) $(ARFLAGS) $@ $(MODOBJS)
+   $(RANLIB) $@
+

[OE-core] [PATCH v2 12/20] python3: fix depends of python-tests

2017-05-16 Thread Markus Lehtonen

Similar to an earlier fix for Python 2.7. Make the tests subpackage
depend on all modules as test.regrtest uses most (if not all) of them.

Some tests also depend on libgcc so add that as a runtime dependency as
well.

[YOCTO #9338]

Signed-off-by: Markus Lehtonen 
---
 meta/recipes-devtools/python/python-3.5-manifest.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/python/python-3.5-manifest.inc 
b/meta/recipes-devtools/python/python-3.5-manifest.inc
index 1e20f00c64..3f29b53a73 100644
--- a/meta/recipes-devtools/python/python-3.5-manifest.inc
+++ b/meta/recipes-devtools/python/python-3.5-manifest.inc
@@ -241,7 +241,7 @@ RDEPENDS_${PN}-terminal="${PN}-core ${PN}-io"
 FILES_${PN}-terminal="${libdir}/python3.5/pty.* 
${libdir}/python3.5/__pycache__/pty.* ${libdir}/python3.5/tty.* 
${libdir}/python3.5/__pycache__/tty.* "
 
 SUMMARY_${PN}-tests="Python tests"
-RDEPENDS_${PN}-tests="${PN}-core"
+RDEPENDS_${PN}-tests="${PN}-core ${PN}-modules libgcc"
 FILES_${PN}-tests="${libdir}/python3.5/test 
${libdir}/python3.5/test/__pycache__ "
 
 SUMMARY_${PN}-textutils="Python option parsing, text wrapping and CSV support"
-- 
2.12.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 10/20] python-pgo-image: enable sstate for do_profile

2017-05-16 Thread Markus Lehtonen

There are caveats however, basically because of the circular dependency
between python-pgo-image and python (python is a build dep of
python-pgo-image, and at the same time, depends on the profile data
generated by python-pgo-image). In practice the sstate data is only used
when no profile data is yet present. Existing profile data (i.e.
non-empty PYTHON_PROFILE_DIR) changes signature hash of python which, in
turn, changes signature of python-pgo-image and causes a sstate
signature mismatch and rebuild of python-pgo-image. Thus, in order to
utilize sstate data the user should run "bitbake python-pgo-image -c clean"
(or otherwise wipe pre-existing profile data) before running
"bitbake python-pgo-image -c profile".

[YOCTO #9338]

Signed-off-by: Markus Lehtonen 
---
 meta/recipes-devtools/images/python-pgo-image.bb | 15 +--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/images/python-pgo-image.bb 
b/meta/recipes-devtools/images/python-pgo-image.bb
index 33aa9a1fec..05252a1a46 100644
--- a/meta/recipes-devtools/images/python-pgo-image.bb
+++ b/meta/recipes-devtools/images/python-pgo-image.bb
@@ -9,6 +9,7 @@ LICENSE = "MIT"
 inherit core-image
 
 PYTHON_PROFILE_DIR ?= "${TMPDIR}/work-shared/${MACHINE}/python/pgo-data"
+PROFILE_DATA_WORKDIR = "${WORKDIR}/profile-data"
 #PYTHON_PROFILE_TASK_DEFAULT = "-m test.regrtest --pgo -w -x test_asyncore 
test_gdb test_multiprocessing test_subprocess"
 ## Exclude tests that are segfaulting on qemux86 target
 #PYTHON_PROFILE_TASK_DEFAULT += "test_bytes test_str test_string test_tuple 
test_unicode test_userstring test_xmlrpc"
@@ -48,7 +49,7 @@ python do_profile() {
 bb.fatal("Failed to archive profile data on target: %s" % output)
 
 # Retrieve and unpack profile data
-profile_dir = d.getVar("PYTHON_PROFILE_DIR", True)
+profile_dir = d.getVar("PROFILE_DATA_WORKDIR", True)
 target.copy_from('/home/root/pgo-data.tgz', profile_dir)
 
 profile_tarball = os.path.join(profile_dir, 'pgo-data.tgz')
@@ -68,4 +69,14 @@ python do_profile() {
 
 addtask profile after do_build
 do_profile[depends] += "qemu-native:do_populate_sysroot 
qemu-helper-native:do_populate_sysroot"
-do_profile[cleandirs] = "${PYTHON_PROFILE_DIR}"
+do_profile[cleandirs] = "${PROFILE_DATA_WORKDIR}"
+
+
+python do_profile_setscene () {
+sstate_setscene(d)
+}
+
+SSTATETASKS += "do_profile"
+do_profile[sstate-inputdirs] = "${PROFILE_DATA_WORKDIR}"
+do_profile[sstate-outputdirs] = "${PYTHON_PROFILE_DIR}"
+addtask do_profile_setscene
-- 
2.12.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 19/20] python3: add python3-tools subpackage

2017-05-16 Thread Markus Lehtonen

[YOCTO #9338]

Signed-off-by: Markus Lehtonen 
---
 meta/recipes-devtools/python/python-3.5-manifest.inc | 8 +---
 meta/recipes-devtools/python/python3_3.5.2.bb| 5 +
 2 files changed, 10 insertions(+), 3 deletions(-)

diff --git a/meta/recipes-devtools/python/python-3.5-manifest.inc 
b/meta/recipes-devtools/python/python-3.5-manifest.inc
index 3f29b53a73..8e40aec9c9 100644
--- a/meta/recipes-devtools/python/python-3.5-manifest.inc
+++ b/meta/recipes-devtools/python/python-3.5-manifest.inc
@@ -4,9 +4,9 @@
 
  
 
-PROVIDES+="${PN}-2to3 ${PN}-argparse ${PN}-asyncio ${PN}-audio ${PN}-codecs 
${PN}-compile ${PN}-compression ${PN}-core ${PN}-crypt ${PN}-ctypes 
${PN}-curses ${PN}-datetime ${PN}-db ${PN}-debugger ${PN}-dev ${PN}-difflib 
${PN}-distutils ${PN}-distutils-staticdev ${PN}-doctest ${PN}-email ${PN}-enum 
${PN}-fcntl ${PN}-gdbm ${PN}-html ${PN}-idle ${PN}-image ${PN}-importlib 
${PN}-io ${PN}-json ${PN}-lang ${PN}-logging ${PN}-mailbox ${PN}-math 
${PN}-mime ${PN}-mmap ${PN}-multiprocessing ${PN}-netclient ${PN}-netserver 
${PN}-numbers ${PN}-pickle ${PN}-pkgutil ${PN}-pprint ${PN}-profile ${PN}-pydoc 
${PN}-re ${PN}-readline ${PN}-reprlib ${PN}-resource ${PN}-selectors 
${PN}-shell ${PN}-signal ${PN}-smtpd ${PN}-sqlite3 ${PN}-sqlite3-tests 
${PN}-stringold ${PN}-subprocess ${PN}-syslog ${PN}-terminal ${PN}-tests 
${PN}-textutils ${PN}-threading ${PN}-tkinter ${PN}-typing ${PN}-unittest 
${PN}-unixadmin ${PN}-xml ${PN}-xmlrpc "
+PROVIDES+="${PN}-2to3 ${PN}-argparse ${PN}-asyncio ${PN}-audio ${PN}-codecs 
${PN}-compile ${PN}-compression ${PN}-core ${PN}-crypt ${PN}-ctypes 
${PN}-curses ${PN}-datetime ${PN}-db ${PN}-debugger ${PN}-dev ${PN}-difflib 
${PN}-distutils ${PN}-distutils-staticdev ${PN}-doctest ${PN}-email ${PN}-enum 
${PN}-fcntl ${PN}-gdbm ${PN}-html ${PN}-idle ${PN}-image ${PN}-importlib 
${PN}-io ${PN}-json ${PN}-lang ${PN}-logging ${PN}-mailbox ${PN}-math 
${PN}-mime ${PN}-mmap ${PN}-multiprocessing ${PN}-netclient ${PN}-netserver 
${PN}-numbers ${PN}-pickle ${PN}-pkgutil ${PN}-pprint ${PN}-profile ${PN}-pydoc 
${PN}-re ${PN}-readline ${PN}-reprlib ${PN}-resource ${PN}-selectors 
${PN}-shell ${PN}-signal ${PN}-smtpd ${PN}-sqlite3 ${PN}-sqlite3-tests 
${PN}-stringold ${PN}-subprocess ${PN}-syslog ${PN}-terminal ${PN}-tests 
${PN}-textutils ${PN}-threading ${PN}-tkinter ${PN}-typing ${PN}-unittest 
${PN}-unixadmin ${PN}-xml ${PN}-xmlrpc ${PN}-tools "
 
-PACKAGES="${PN}-dbg ${PN}-2to3 ${PN}-argparse ${PN}-asyncio ${PN}-audio 
${PN}-codecs ${PN}-compile ${PN}-compression ${PN}-core ${PN}-crypt 
${PN}-ctypes ${PN}-curses ${PN}-datetime ${PN}-db ${PN}-debugger ${PN}-dev 
${PN}-difflib ${PN}-distutils-staticdev ${PN}-distutils ${PN}-doctest 
${PN}-email ${PN}-enum ${PN}-fcntl ${PN}-gdbm ${PN}-html ${PN}-idle ${PN}-image 
${PN}-importlib ${PN}-io ${PN}-json ${PN}-lang ${PN}-logging ${PN}-mailbox 
${PN}-math ${PN}-mime ${PN}-mmap ${PN}-multiprocessing ${PN}-netclient 
${PN}-netserver ${PN}-numbers ${PN}-pickle ${PN}-pkgutil ${PN}-pprint 
${PN}-profile ${PN}-pydoc ${PN}-re ${PN}-readline ${PN}-reprlib ${PN}-resource 
${PN}-selectors ${PN}-shell ${PN}-signal ${PN}-smtpd ${PN}-sqlite3 
${PN}-sqlite3-tests ${PN}-stringold ${PN}-subprocess ${PN}-syslog 
${PN}-terminal ${PN}-tests ${PN}-textutils ${PN}-threading ${PN}-tkinter 
${PN}-typing ${PN}-unittest ${PN}-unixadmin ${PN}-xml ${PN}-xmlrpc 
${PN}-modules"
+PACKAGES="${PN}-dbg ${PN}-2to3 ${PN}-argparse ${PN}-asyncio ${PN}-audio 
${PN}-codecs ${PN}-compile ${PN}-compression ${PN}-core ${PN}-crypt 
${PN}-ctypes ${PN}-curses ${PN}-datetime ${PN}-db ${PN}-debugger ${PN}-dev 
${PN}-difflib ${PN}-distutils-staticdev ${PN}-distutils ${PN}-doctest 
${PN}-email ${PN}-enum ${PN}-fcntl ${PN}-gdbm ${PN}-html ${PN}-idle ${PN}-image 
${PN}-importlib ${PN}-io ${PN}-json ${PN}-lang ${PN}-logging ${PN}-mailbox 
${PN}-math ${PN}-mime ${PN}-mmap ${PN}-multiprocessing ${PN}-netclient 
${PN}-netserver ${PN}-numbers ${PN}-pickle ${PN}-pkgutil ${PN}-pprint 
${PN}-profile ${PN}-pydoc ${PN}-re ${PN}-readline ${PN}-reprlib ${PN}-resource 
${PN}-selectors ${PN}-shell ${PN}-signal ${PN}-smtpd ${PN}-sqlite3 
${PN}-sqlite3-tests ${PN}-stringold ${PN}-subprocess ${PN}-syslog 
${PN}-terminal ${PN}-tests ${PN}-textutils ${PN}-threading ${PN}-tkinter 
${PN}-typing ${PN}-unittest ${PN}-unixadmin ${PN}-xml ${PN}-xmlrpc 
${PN}-modules ${PN}-tools"
 
 SUMMARY_${PN}-2to3="Python automated Python 2 to 3 code translator"
 RDEPENDS_${PN}-2to3="${PN}-core"
@@ -280,4 +280,6 @@ SUMMARY_${PN}-modules="All Python modules"
 RDEPENDS_${PN}-modules="${PN}-2to3 ${PN}-argparse ${PN}-asyncio ${PN}-audio 
${PN}-codecs ${PN}-compile ${PN}-compression ${PN}-core ${PN}-crypt 
${PN}-ctypes ${PN}-curses ${PN}-datetime ${PN}-db ${PN}-debugger ${PN}-difflib 
${PN}-distutils ${PN}-doctest ${PN}-email ${PN}-enum ${PN}-fcntl ${PN}-gdbm 
${PN}-html ${PN}-idle ${PN}-image ${PN}-importlib ${PN}-io ${PN}-json 
${PN}-lang ${PN}-logging ${PN}-mailbox ${PN}-math ${PN}-mime

[OE-core] [PATCH v2 16/20] python3: support profile optimized build

2017-05-16 Thread Markus Lehtonen

Make it possible to do profile-optimized build. The implementation is
virtually identical to how we do it in Python 2.7. Basically, you need
to define PYTHON3_PROFILE_OPT = "1" in your local.conf and have profile
data available in the location pointed to by PYTHON3_PROFILE_DIR.
Profile data can be obtainen e.g. by running bitbake python-pgo-image -c
profile3.

[YOCTO #9338]

Signed-off-by: Markus Lehtonen 
---
 meta/recipes-devtools/python/python3_3.5.2.bb | 20 
 1 file changed, 20 insertions(+)

diff --git a/meta/recipes-devtools/python/python3_3.5.2.bb 
b/meta/recipes-devtools/python/python3_3.5.2.bb
index 18d683e4c7..7a3d2b367f 100644
--- a/meta/recipes-devtools/python/python3_3.5.2.bb
+++ b/meta/recipes-devtools/python/python3_3.5.2.bb
@@ -78,6 +78,17 @@ export CROSSPYTHONPATH = 
"${STAGING_LIBDIR_NATIVE}/python${PYTHON_MAJMIN}/lib-dy
 # No ctypes option for python 3
 PYTHONLSBOPTS = ""
 
+# Automatic profile guided optimization
+PYTHON3_MAKE_TARGET ?= "${@'build_all_use_profile' if 
d.getVar('PYTHON3_PROFILE_OPT', True) == '1' else ''}"
+PYTHON3_PROFILE_DIR ?= "${@'${TMPDIR}/work-shared/${MACHINE}/python3/pgo-data' 
if d.getVar('PYTHON3_PROFILE_OPT', True) == '1' else ''}"
+python () {
+if (d.getVar('PYTHON3_PROFILE_OPT', True) == '1' and
+d.getVar('PYTHON3_MAKE_TARGET', True) == 'build_all_use_profile'):
+profile_dir = d.getVar('PYTHON3_PROFILE_DIR', True)
+bb.utils.mkdirhier(profile_dir)
+d.setVarFlag('do_compile', 'file-checksums', '%s:True' % profile_dir)
+}
+
 do_configure_append() {
rm -f ${S}/Makefile.orig
autoreconf -Wcross --verbose --install --force --exclude=autopoint 
../Python-${PV}/Modules/_ctypes/libffi
@@ -117,6 +128,10 @@ do_compile() {
 if [ "${PYTHON3_MAKE_TARGET}" = "build_all_generate_profile" ]; then
 # This is only used in PGO profiling by python-profile-opt package
 export EXTRA_CFLAGS="-fprofile-dir=./python3-pgo-profiles/"
+elif [ -n "${PYTHON3_PROFILE_DIR}" ]; then
+export EXTRA_CFLAGS="-fprofile-dir=${PYTHON3_PROFILE_DIR}"
+# Remove non-optimized build artefacts
+oe_runmake clean
 fi
 
oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/python3-native/pgen \
@@ -137,6 +152,11 @@ do_install() {
install -d ${D}${libdir}/pkgconfig
install -d ${D}${libdir}/python${PYTHON_MAJMIN}/config
 
+# This only has effect if we build with -fprofile-use, e.g. when make
+# target is build_all_use_profile
+if [ -n "${PYTHON3_PROFILE_DIR}" ]; then
+export EXTRA_CFLAGS="-fprofile-dir=${PYTHON3_PROFILE_DIR}"
+fi
# rerun the build once again with original makefile this time
# run install in a separate step to avoid compile/install race
oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/python3-native/pgen \
-- 
2.12.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 15/20] python3: remove two setup.py cross-compile hacks

2017-05-16 Thread Markus Lehtonen

Remove two unneeded hacks. The first hack ("setup.py: no host headers
libs" patch) is not needed because we use cross-compiler (e.g.
i586-oe-linux-gcc) which has not been configured with any host system
include or library directories, and thus, we don't get any host system
directories when running "gcc -E -v".

The second hack becomes useless after the first hack has been removed
and we get the standard include and lib directories normally from gcc.

[YOCTO #9338]

Signed-off-by: Markus Lehtonen 
---
 .../python3-setup.py-no-host-headers-libs.patch| 33 --
 .../setup.py-find-libraries-in-staging-dirs.patch  | 30 
 meta/recipes-devtools/python/python3_3.5.2.bb  |  2 --
 3 files changed, 65 deletions(-)
 delete mode 100644 
meta/recipes-devtools/python/python3/python3-setup.py-no-host-headers-libs.patch
 delete mode 100644 
meta/recipes-devtools/python/python3/setup.py-find-libraries-in-staging-dirs.patch

diff --git 
a/meta/recipes-devtools/python/python3/python3-setup.py-no-host-headers-libs.patch
 
b/meta/recipes-devtools/python/python3/python3-setup.py-no-host-headers-libs.patch
deleted file mode 100644
index 2bc8b8c460..00
--- 
a/meta/recipes-devtools/python/python3/python3-setup.py-no-host-headers-libs.patch
+++ /dev/null
@@ -1,33 +0,0 @@
-From 43238e1ac13e32984d015c92a5841f3de1fe1d15 Mon Sep 17 00:00:00 2001
-From: Jackie Huang 
-Date: Tue, 18 Nov 2014 00:07:07 -0500
-Subject: [PATCH] setup.py: no host headers libs
-
-When we are cross-compiling, setup.py should never look in /usr
-or /usr/local to find headers or libraries.
-
-Upstream-Status: Inappropriate [Cross compile specific]
-
-Signed-off-by: Jackie Huang 

- setup.py | 3 ---
- 1 file changed, 3 deletions(-)
-
-diff --git a/setup.py b/setup.py
-index f020b28..e8339cd 100644
 a/setup.py
-+++ b/setup.py
-@@ -444,10 +444,7 @@ class PyBuildExt(build_ext):
- if not cross_compiling:
- add_dir_to_list(self.compiler.library_dirs, 
os.path.join('/usr/local', sys.lib))
- add_dir_to_list(self.compiler.include_dirs, '/usr/local/include')
--# only change this for cross builds for 3.3, issues on Mageia
--if cross_compiling:
- self.add_gcc_paths()
--if not cross_compiling:
- self.add_multiarch_paths()
- 
- # Add paths specified in the environment variables LDFLAGS and
--- 
-2.0.0
-
diff --git 
a/meta/recipes-devtools/python/python3/setup.py-find-libraries-in-staging-dirs.patch
 
b/meta/recipes-devtools/python/python3/setup.py-find-libraries-in-staging-dirs.patch
deleted file mode 100644
index f26bd38b02..00
--- 
a/meta/recipes-devtools/python/python3/setup.py-find-libraries-in-staging-dirs.patch
+++ /dev/null
@@ -1,30 +0,0 @@
-Patch setup.py so that the detect_modules() function looks for required
-libraries and headers in STAGING_LIBDIR / STAGING_INCDIR.
-
-Without this patch, several extension modules are not built, even though
-their dependencies are present in the compiler's search paths.
-The result is the following warning, and ultimately incomplete packages:
-
-| The necessary bits to build these optional modules were not found:
-| _bz2  _curses_panel _dbm
-| _gdbm _lzma _sqlite3
-| nis   readline  zlib
-| To find the necessary bits, look in setup.py in detect_modules() for the 
module's name.
-
-Upstream-Status:  Inappropriate [Cross compile specific]
-
-Signed-off-by: Dominic Sacré 
-
-Index: Python-3.4.3/setup.py
-===
 Python-3.4.3.orig/setup.py
-+++ Python-3.4.3/setup.py
-@@ -521,6 +521,8 @@ class PyBuildExt(build_ext):
- else:
- lib_dirs = self.compiler.library_dirs[:]
- inc_dirs = self.compiler.include_dirs[:]
-+lib_dirs.append(os.environ.get('STAGING_LIBDIR'))
-+inc_dirs.append(os.environ.get('STAGING_INCDIR'))
- exts = []
- missing = []
- 
diff --git a/meta/recipes-devtools/python/python3_3.5.2.bb 
b/meta/recipes-devtools/python/python3_3.5.2.bb
index 042c2b0339..18d683e4c7 100644
--- a/meta/recipes-devtools/python/python3_3.5.2.bb
+++ b/meta/recipes-devtools/python/python3_3.5.2.bb
@@ -30,10 +30,8 @@ SRC_URI += "\
 file://unixccompiler.patch \
 file://avoid-ncursesw-include-path.patch \
 file://python3-use-CROSSPYTHONPATH-for-PYTHON_FOR_BUILD.patch \
-file://python3-setup.py-no-host-headers-libs.patch \
 file://sysconfig.py-add-_PYTHON_PROJECT_SRC.patch \
 file://setup.py-check-cross_compiling-when-get-FLAGS.patch \
-file://setup.py-find-libraries-in-staging-dirs.patch \
 file://configure.ac-fix-LIBPL.patch \
 file://python3-fix-CVE-2016-1000110.patch

[OE-core] [PATCH v2 09/20] python-pgo-image: switch python default profile task to pybench

2017-05-16 Thread Markus Lehtonen

Switch to pybench in order to reduce the time taken by profiling.

[YOCTO #9338]

Signed-off-by: Markus Lehtonen 
---
 meta/recipes-devtools/images/python-pgo-image.bb | 18 ++
 1 file changed, 10 insertions(+), 8 deletions(-)

diff --git a/meta/recipes-devtools/images/python-pgo-image.bb 
b/meta/recipes-devtools/images/python-pgo-image.bb
index 426d6acb45..33aa9a1fec 100644
--- a/meta/recipes-devtools/images/python-pgo-image.bb
+++ b/meta/recipes-devtools/images/python-pgo-image.bb
@@ -1,20 +1,22 @@
 SUMMARY = "Minimal image for doing Python profiling (for PGO)"
 
 IMAGE_FEATURES += "ssh-server-dropbear"
-IMAGE_INSTALL = "packagegroup-core-boot python-profile-opt 
python-profile-opt-tests"
+IMAGE_INSTALL = "packagegroup-core-boot"
+IMAGE_INSTALL += "python-profile-opt python-profile-opt-tests 
python-profile-opt-tools"
 
 LICENSE = "MIT"
 
 inherit core-image
 
 PYTHON_PROFILE_DIR ?= "${TMPDIR}/work-shared/${MACHINE}/python/pgo-data"
-PYTHON_PROFILE_TASK_DEFAULT = "-m test.regrtest --pgo -w -x test_asyncore 
test_gdb test_multiprocessing test_subprocess"
-# Exclude tests that are segfaulting on qemux86 target
-PYTHON_PROFILE_TASK_DEFAULT += "test_bytes test_str test_string test_tuple 
test_unicode test_userstring test_xmlrpc"
-# Exclude tests that are failing on qemux86
-PYTHON_PROFILE_TASK_DEFAULT += "test_StringIO test_builtin test_calendar 
test_cmath test_ctypes test_distutils test_exceptions test_getargs test_gzip 
test_json test_math test_shutil test_socket test_sqlite test_sysconfig 
test_traceback test_warnings"
-# Exclude tests that are taking very long on qemux86
-PYTHON_PROFILE_TASK_DEFAULT += "test_io test_lib2to3 test_itertools"
+#PYTHON_PROFILE_TASK_DEFAULT = "-m test.regrtest --pgo -w -x test_asyncore 
test_gdb test_multiprocessing test_subprocess"
+## Exclude tests that are segfaulting on qemux86 target
+#PYTHON_PROFILE_TASK_DEFAULT += "test_bytes test_str test_string test_tuple 
test_unicode test_userstring test_xmlrpc"
+## Exclude tests that are failing on qemux86
+#PYTHON_PROFILE_TASK_DEFAULT += "test_StringIO test_builtin test_calendar 
test_cmath test_ctypes test_distutils test_exceptions test_getargs test_gzip 
test_json test_math test_shutil test_socket test_sqlite test_sysconfig 
test_traceback test_warnings"
+## Exclude tests that are taking very long on qemux86
+#PYTHON_PROFILE_TASK_DEFAULT += "test_io test_lib2to3 test_itertools"
+PYTHON_PROFILE_TASK_DEFAULT = 
"/opt/share/doc/python-profile-opt/Tools/pybench/pybench.py -n 2 --with-gc 
--with-syscheck"
 PYTHON_PROFILE_TASK ?= "${PYTHON_PROFILE_TASK_DEFAULT}"
 
 # We need these because we're utilizing the runtime test helpers from oeqa
-- 
2.12.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 20/20] python-pgo-image: change python3 profile target to pybench

2017-05-16 Thread Markus Lehtonen

Running test.regrtest was still taking ridiculously long, i.e. around 4
hours in qemux86 on my desktop machine. Change to pybench which "only"
takes around 55 minutes.

[YOCTO #9338]

Signed-off-by: Markus Lehtonen 
---
 meta/recipes-devtools/images/python-pgo-image.bb | 11 ++-
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/meta/recipes-devtools/images/python-pgo-image.bb 
b/meta/recipes-devtools/images/python-pgo-image.bb
index e16cd52486..dc98e0c839 100644
--- a/meta/recipes-devtools/images/python-pgo-image.bb
+++ b/meta/recipes-devtools/images/python-pgo-image.bb
@@ -3,7 +3,7 @@ SUMMARY = "Minimal image for doing Python profiling (for PGO)"
 IMAGE_FEATURES += "ssh-server-dropbear"
 IMAGE_INSTALL = "packagegroup-core-boot"
 IMAGE_INSTALL += "python-profile-opt python-profile-opt-tests 
python-profile-opt-tools"
-IMAGE_INSTALL += "python-profile-opt3 python-profile-opt3-tests"
+IMAGE_INSTALL += "python-profile-opt3 python-profile-opt3-tests 
python-profile-opt3-tools"
 
 LICENSE = "MIT"
 
@@ -22,12 +22,13 @@ PYTHON_PROFILE_TASK_DEFAULT = 
"/opt/share/doc/python-profile-opt/Tools/pybench/p
 PYTHON_PROFILE_TASK ?= "${PYTHON_PROFILE_TASK_DEFAULT}"
 
 PYTHON3_PROFILE_DIR ?= "${TMPDIR}/work-shared/${MACHINE}/python3/pgo-data"
-PYTHON3_PROFILE_TASK_DEFAULT = "-m test.regrtest --pgo -w -x test_asyncore 
test_gdb test_multiprocessing_fork test_multiprocessing_forkserver 
test_multiprocessing_main_handling test_multiprocessing_spawn test_subprocess"
+#PYTHON3_PROFILE_TASK_DEFAULT = "-m test.regrtest --pgo -w -x test_asyncore 
test_gdb test_multiprocessing_fork test_multiprocessing_forkserver 
test_multiprocessing_main_handling test_multiprocessing_spawn test_subprocess"
 # Exclude tests that are failing on qemux86
-PYTHON3_PROFILE_TASK_DEFAULT += "test_builtin test_cmath 
test_concurrent_futures test_difflib test_distutils test_float test_format 
test_math test_optparse test_shutil test_statistics test_types test_unicode"
+#PYTHON3_PROFILE_TASK_DEFAULT += "test_builtin test_cmath 
test_concurrent_futures test_difflib test_distutils test_float test_format 
test_math test_optparse test_shutil test_statistics test_types test_unicode"
 # Exclude tests that are taking very long on qemux86
-PYTHON3_PROFILE_TASK_DEFAULT += "test_lib2to3 test_buffer test_pickle test_io 
test_threading test_asyncio test_urllib2_localnet test_itertools test_tuple 
test_trace test_tarfile test_unicodedata test_decimal"
-PYTHON3_PROFILE_TASK_DEFAULT += "test_long test_zipfile test_deque test_descr 
test_email test_venv test_bytes test_compileall test_ast test_multibytecodec"
+#PYTHON3_PROFILE_TASK_DEFAULT += "test_lib2to3 test_buffer test_pickle test_io 
test_threading test_asyncio test_urllib2_localnet test_itertools test_tuple 
test_trace test_tarfile test_unicodedata test_decimal"
+#PYTHON3_PROFILE_TASK_DEFAULT += "test_long test_zipfile test_deque test_descr 
test_email test_venv test_bytes test_compileall test_ast test_multibytecodec"
+PYTHON3_PROFILE_TASK_DEFAULT = 
"/opt/share/doc/python-profile-opt3/Tools/pybench/pybench.py -n 2 --with-gc 
--with-syscheck"
 
 PYTHON3_PROFILE_TASK ?= "${PYTHON3_PROFILE_TASK_DEFAULT}"
 
-- 
2.12.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 11/20] python3-native: support profile optimized build

2017-05-16 Thread Markus Lehtonen

This feature is virtually identical to that of python-native. Profile
guided optimization is enabled by defining
PYTHON3_NATIVE_PROFILE_OPT = "1"
in local.conf. In addition, the profile task may be defined with
PYTHON3_NATIVE_PROFILE_TASK.

[YOCTO #9338]

Signed-off-by: Markus Lehtonen 
---
 meta/recipes-devtools/python/python3-native_3.5.2.bb | 9 +
 meta/recipes-devtools/python/python3/000-cross-compile.patch | 9 -
 2 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/meta/recipes-devtools/python/python3-native_3.5.2.bb 
b/meta/recipes-devtools/python/python3-native_3.5.2.bb
index edcf2244f5..1d99e56054 100644
--- a/meta/recipes-devtools/python/python3-native_3.5.2.bb
+++ b/meta/recipes-devtools/python/python3-native_3.5.2.bb
@@ -56,10 +56,19 @@ EXTRA_OEMAKE = '\
 # No ctypes option for python 3
 PYTHONLSBOPTS = ""
 
+PYTHON3_NATIVE_MAKE_TARGET ?= "${@'profile-opt' if 
d.getVar('PYTHON3_NATIVE_PROFILE_OPT', True) == '1' else ''}"
+
 do_configure_append() {
autoreconf --verbose --install --force --exclude=autopoint 
../Python-${PV}/Modules/_ctypes/libffi
 }
 
+do_compile() {
+if [ -n "${PYTHON3_NATIVE_PROFILE_TASK}" ]; then
+sed -i -e 
's,^PROFILE_TASK=.*,PROFILE_TASK=${PYTHON3_NATIVE_PROFILE_TASK},g' Makefile
+fi
+oe_runmake ${PYTHON3_NATIVE_MAKE_TARGET}
+}
+
 do_install() {
install -d ${D}${libdir}/pkgconfig
oe_runmake 'DESTDIR=${D}' install
diff --git a/meta/recipes-devtools/python/python3/000-cross-compile.patch 
b/meta/recipes-devtools/python/python3/000-cross-compile.patch
index 2d822218f4..d6d7820ee8 100644
--- a/meta/recipes-devtools/python/python3/000-cross-compile.patch
+++ b/meta/recipes-devtools/python/python3/000-cross-compile.patch
@@ -29,15 +29,6 @@ Index: Python-3.5.2/Makefile.pre.in
  
  PSRCS=\
Parser/acceler.c \
-@@ -509,7 +511,7 @@ build_all_generate_profile:
- 
- run_profile_task:
-   : # FIXME: can't run for a cross build
--  $(LLVM_PROF_FILE) $(RUNSHARED) ./$(BUILDPYTHON) $(PROFILE_TASK) || true
-+  $(LLVM_PROF_FILE) $(RUNSHARED) $(HOSTPYTHON) $(PROFILE_TASK) || true
- 
- build_all_merge_profile:
-   $(LLVM_PROF_MERGER)
 @@ -792,7 +794,7 @@ $(GRAMMAR_H): $(GRAMMAR_INPUT) $(PGEN)
@$(MKDIR_P) Include
# Avoid copying the file onto itself for an in-tree build
-- 
2.12.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 17/20] python3: fix profile-optimized build of modules

2017-05-16 Thread Markus Lehtonen

Without this the pgo-related compiler flags are not used in
cross-builds.

[YOCTO #9338]

Signed-off-by: Markus Lehtonen 
---
 ...-CFLAGS-for-extensions-when-cross-compili.patch | 56 ++
 meta/recipes-devtools/python/python3_3.5.2.bb  |  1 +
 2 files changed, 57 insertions(+)
 create mode 100644 
meta/recipes-devtools/python/python3/Use-correct-CFLAGS-for-extensions-when-cross-compili.patch

diff --git 
a/meta/recipes-devtools/python/python3/Use-correct-CFLAGS-for-extensions-when-cross-compili.patch
 
b/meta/recipes-devtools/python/python3/Use-correct-CFLAGS-for-extensions-when-cross-compili.patch
new file mode 100644
index 00..bf239c6261
--- /dev/null
+++ 
b/meta/recipes-devtools/python/python3/Use-correct-CFLAGS-for-extensions-when-cross-compili.patch
@@ -0,0 +1,56 @@
+From 0fd8b986888ddf1995b503edf46ac827e83114d9 Mon Sep 17 00:00:00 2001
+From: Markus Lehtonen 
+Date: Wed, 23 Nov 2016 16:08:04 +0200
+Subject: [PATCH] Use correct CFLAGS for extensions when cross-compiling
+
+Take PY_CFLAGS_NODIST into account, like in native build. This is needed
+in order to to profile-optimized build. Also, pass EXTRA_CFLAGS to
+profile-optimized build.
+
+Upstream-Status: Pending
+
+Signed-off-by: Markus Lehtonen 
+---
+ Makefile.pre.in | 4 ++--
+ setup.py| 3 ++-
+ 2 files changed, 4 insertions(+), 3 deletions(-)
+
+diff --git a/Makefile.pre.in b/Makefile.pre.in
+index bace00e..9635aa4 100644
+--- a/Makefile.pre.in
 b/Makefile.pre.in
+@@ -509,7 +509,7 @@ profile-opt:
+   $(MAKE) profile-removal
+ 
+ build_all_generate_profile:
+-  $(MAKE) all CFLAGS_NODIST="$(CFLAGS) $(PGO_PROF_GEN_FLAG) @LTOFLAGS@" 
LDFLAGS="$(LDFLAGS) $(PGO_PROF_GEN_FLAG) @LTOFLAGS@" LIBS="$(LIBS)"
++  $(MAKE) all CFLAGS_NODIST="$(CFLAGS) $(EXTRA_CFLAGS) 
$(PGO_PROF_GEN_FLAG) @LTOFLAGS@" LDFLAGS="$(LDFLAGS) $(PGO_PROF_GEN_FLAG) 
@LTOFLAGS@" LIBS="$(LIBS)"
+ 
+ run_profile_task:
+   : # FIXME: can't run for a cross build
+@@ -519,7 +519,7 @@ build_all_merge_profile:
+   $(LLVM_PROF_MERGER)
+ 
+ build_all_use_profile:
+-  $(MAKE) all CFLAGS_NODIST="$(CFLAGS) $(PGO_PROF_USE_FLAG) @LTOFLAGS@" 
LDFLAGS="$(LDFLAGS) @LTOFLAGS@"
++  $(MAKE) all CFLAGS_NODIST="$(CFLAGS) $(EXTRA_CFLAGS) 
$(PGO_PROF_USE_FLAG) @LTOFLAGS@" LDFLAGS="$(LDFLAGS) @LTOFLAGS@"
+ 
+ # Compile and run with gcov
+ .PHONY=coverage coverage-lcov coverage-report
+diff --git a/setup.py b/setup.py
+index c1ce87e..72d37cf 100644
+--- a/setup.py
 b/setup.py
+@@ -271,7 +271,8 @@ class PyBuildExt(build_ext):
+ # compilers
+ if compiler is not None:
+ if cross_compiling:
+-(ccshared,cflags) = (os.environ.get('CCSHARED') or '', 
os.environ.get('CFLAGS') or '')
++(ccshared,cflags) = (os.environ.get('CCSHARED') or '',
++ (os.environ.get('CFLAGS') or '') + ' ' + 
sysconfig.get_config_var('PY_CFLAGS_NODIST'))
+ else:
+ (ccshared,cflags) = 
sysconfig.get_config_vars('CCSHARED','CFLAGS')
+ args['compiler_so'] = compiler + ' ' + ccshared + ' ' + cflags
+-- 
+2.6.6
+
diff --git a/meta/recipes-devtools/python/python3_3.5.2.bb 
b/meta/recipes-devtools/python/python3_3.5.2.bb
index 7a3d2b367f..7853c69e09 100644
--- a/meta/recipes-devtools/python/python3_3.5.2.bb
+++ b/meta/recipes-devtools/python/python3_3.5.2.bb
@@ -35,6 +35,7 @@ SRC_URI += "\
 file://configure.ac-fix-LIBPL.patch \
 file://python3-fix-CVE-2016-1000110.patch \
 file://upstream-random-fixes.patch \
+file://Use-correct-CFLAGS-for-extensions-when-cross-compili.patch \
"
 SRC_URI[md5sum] = "8906efbacfcdc7c3c9198aeefafd159e"
 SRC_URI[sha256sum] = 
"0010f56100b9b74259ebcd5d4b295a32324b58b517403a10d1a2aa7cb22bca40"
-- 
2.12.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 18/20] python-pgo-image: exclude tests from the python3 profile target

2017-05-16 Thread Markus Lehtonen

Exlude failing tests (on qemux86 target) which would cause the profile
task to fail.

Also, disable some very long running tests. Running in qemux86 target on
my i7-3770K:
  test_lib2to3: 1604.5s
  test_buffer: 1562.8s
  test_pickle: 1527.0s
  test_io: 1517.8s
  test_threading: 1384.1s
  test_asyncio: 1380.9s
  test_urllib2_localnet: 1276.8s
  test_itertools: 944.7s
  test_tuple: 847.7s
  test_trace: 756.7s
  test_tarfile: 716.7s
  test_unicodedata: 709.9s
  test_decimal: 615.1s
  test_buffer: 1562.8s
  test_pickle: 1527.0s
  test_io: 1517.8s
  test_threading: 1384.1s
  test_asyncio: 1380.9s
  test_urllib2_localnet: 1276.8s
  test_itertools: 944.7s
  test_tuple: 847.7s
  test_trace: 756.7s
  test_tarfile: 716.7s
  test_unicodedata: 709.9s
  test_decimal: 615.1s

[YOCTO #9338]

Signed-off-by: Markus Lehtonen 
---
 meta/recipes-devtools/images/python-pgo-image.bb | 5 +
 1 file changed, 5 insertions(+)

diff --git a/meta/recipes-devtools/images/python-pgo-image.bb 
b/meta/recipes-devtools/images/python-pgo-image.bb
index 285e403d32..e16cd52486 100644
--- a/meta/recipes-devtools/images/python-pgo-image.bb
+++ b/meta/recipes-devtools/images/python-pgo-image.bb
@@ -23,6 +23,11 @@ PYTHON_PROFILE_TASK ?= "${PYTHON_PROFILE_TASK_DEFAULT}"
 
 PYTHON3_PROFILE_DIR ?= "${TMPDIR}/work-shared/${MACHINE}/python3/pgo-data"
 PYTHON3_PROFILE_TASK_DEFAULT = "-m test.regrtest --pgo -w -x test_asyncore 
test_gdb test_multiprocessing_fork test_multiprocessing_forkserver 
test_multiprocessing_main_handling test_multiprocessing_spawn test_subprocess"
+# Exclude tests that are failing on qemux86
+PYTHON3_PROFILE_TASK_DEFAULT += "test_builtin test_cmath 
test_concurrent_futures test_difflib test_distutils test_float test_format 
test_math test_optparse test_shutil test_statistics test_types test_unicode"
+# Exclude tests that are taking very long on qemux86
+PYTHON3_PROFILE_TASK_DEFAULT += "test_lib2to3 test_buffer test_pickle test_io 
test_threading test_asyncio test_urllib2_localnet test_itertools test_tuple 
test_trace test_tarfile test_unicodedata test_decimal"
+PYTHON3_PROFILE_TASK_DEFAULT += "test_long test_zipfile test_deque test_descr 
test_email test_venv test_bytes test_compileall test_ast test_multibytecodec"
 
 PYTHON3_PROFILE_TASK ?= "${PYTHON3_PROFILE_TASK_DEFAULT}"
 
-- 
2.12.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 08/20] python: add python-tools subpackage

2017-05-16 Thread Markus Lehtonen

Useful in developing Python, e.g. in benchmarking.

[YOCTO #9338]

Signed-off-by: Markus Lehtonen 
---
 meta/recipes-devtools/python/python-2.7-manifest.inc | 8 +---
 meta/recipes-devtools/python/python_2.7.13.bb| 5 +
 2 files changed, 10 insertions(+), 3 deletions(-)

diff --git a/meta/recipes-devtools/python/python-2.7-manifest.inc 
b/meta/recipes-devtools/python/python-2.7-manifest.inc
index 7ed254bbd6..33189e0f0b 100644
--- a/meta/recipes-devtools/python/python-2.7-manifest.inc
+++ b/meta/recipes-devtools/python/python-2.7-manifest.inc
@@ -4,9 +4,9 @@
 
  
 
-PROVIDES+="${PN}-2to3 ${PN}-argparse ${PN}-audio ${PN}-bsddb ${PN}-codecs 
${PN}-compile ${PN}-compiler ${PN}-compression ${PN}-contextlib ${PN}-core 
${PN}-crypt ${PN}-ctypes ${PN}-curses ${PN}-datetime ${PN}-db ${PN}-debugger 
${PN}-dev ${PN}-difflib ${PN}-distutils ${PN}-distutils-staticdev ${PN}-doctest 
${PN}-email ${PN}-fcntl ${PN}-gdbm ${PN}-hotshot ${PN}-html ${PN}-idle 
${PN}-image ${PN}-importlib ${PN}-io ${PN}-json ${PN}-lang ${PN}-logging 
${PN}-mailbox ${PN}-math ${PN}-mime ${PN}-mmap ${PN}-multiprocessing 
${PN}-netclient ${PN}-netserver ${PN}-numbers ${PN}-pickle ${PN}-pkgutil 
${PN}-plistlib ${PN}-pprint ${PN}-profile ${PN}-pydoc ${PN}-re ${PN}-readline 
${PN}-resource ${PN}-robotparser ${PN}-shell ${PN}-smtpd ${PN}-sqlite3 
${PN}-sqlite3-tests ${PN}-stringold ${PN}-subprocess ${PN}-syslog 
${PN}-terminal ${PN}-tests ${PN}-textutils ${PN}-threading ${PN}-tkinter 
${PN}-unittest ${PN}-unixadmin ${PN}-xml ${PN}-xmlrpc ${PN}-zlib "
+PROVIDES+="${PN}-2to3 ${PN}-argparse ${PN}-audio ${PN}-bsddb ${PN}-codecs 
${PN}-compile ${PN}-compiler ${PN}-compression ${PN}-contextlib ${PN}-core 
${PN}-crypt ${PN}-ctypes ${PN}-curses ${PN}-datetime ${PN}-db ${PN}-debugger 
${PN}-dev ${PN}-difflib ${PN}-distutils ${PN}-distutils-staticdev ${PN}-doctest 
${PN}-email ${PN}-fcntl ${PN}-gdbm ${PN}-hotshot ${PN}-html ${PN}-idle 
${PN}-image ${PN}-importlib ${PN}-io ${PN}-json ${PN}-lang ${PN}-logging 
${PN}-mailbox ${PN}-math ${PN}-mime ${PN}-mmap ${PN}-multiprocessing 
${PN}-netclient ${PN}-netserver ${PN}-numbers ${PN}-pickle ${PN}-pkgutil 
${PN}-plistlib ${PN}-pprint ${PN}-profile ${PN}-pydoc ${PN}-re ${PN}-readline 
${PN}-resource ${PN}-robotparser ${PN}-shell ${PN}-smtpd ${PN}-sqlite3 
${PN}-sqlite3-tests ${PN}-stringold ${PN}-subprocess ${PN}-syslog 
${PN}-terminal ${PN}-tests ${PN}-textutils ${PN}-threading ${PN}-tkinter 
${PN}-unittest ${PN}-unixadmin ${PN}-xml ${PN}-xmlrpc ${PN}-zlib ${PN}-tools"
 
-PACKAGES="${PN}-dbg ${PN}-2to3 ${PN}-argparse ${PN}-audio ${PN}-bsddb 
${PN}-codecs ${PN}-compile ${PN}-compiler ${PN}-compression ${PN}-contextlib 
${PN}-core ${PN}-crypt ${PN}-ctypes ${PN}-curses ${PN}-datetime ${PN}-db 
${PN}-debugger ${PN}-dev ${PN}-difflib ${PN}-distutils-staticdev 
${PN}-distutils ${PN}-doctest ${PN}-email ${PN}-fcntl ${PN}-gdbm ${PN}-hotshot 
${PN}-html ${PN}-idle ${PN}-image ${PN}-importlib ${PN}-io ${PN}-json 
${PN}-lang ${PN}-logging ${PN}-mailbox ${PN}-math ${PN}-mime ${PN}-mmap 
${PN}-multiprocessing ${PN}-netclient ${PN}-netserver ${PN}-numbers 
${PN}-pickle ${PN}-pkgutil ${PN}-plistlib ${PN}-pprint ${PN}-profile 
${PN}-pydoc ${PN}-re ${PN}-readline ${PN}-resource ${PN}-robotparser 
${PN}-shell ${PN}-smtpd ${PN}-sqlite3 ${PN}-sqlite3-tests ${PN}-stringold 
${PN}-subprocess ${PN}-syslog ${PN}-terminal ${PN}-tests ${PN}-textutils 
${PN}-threading ${PN}-tkinter ${PN}-unittest ${PN}-unixadmin ${PN}-xml 
${PN}-xmlrpc ${PN}-zlib ${PN}-modules"
+PACKAGES="${PN}-dbg ${PN}-2to3 ${PN}-argparse ${PN}-audio ${PN}-bsddb 
${PN}-codecs ${PN}-compile ${PN}-compiler ${PN}-compression ${PN}-contextlib 
${PN}-core ${PN}-crypt ${PN}-ctypes ${PN}-curses ${PN}-datetime ${PN}-db 
${PN}-debugger ${PN}-dev ${PN}-difflib ${PN}-distutils-staticdev 
${PN}-distutils ${PN}-doctest ${PN}-email ${PN}-fcntl ${PN}-gdbm ${PN}-hotshot 
${PN}-html ${PN}-idle ${PN}-image ${PN}-importlib ${PN}-io ${PN}-json 
${PN}-lang ${PN}-logging ${PN}-mailbox ${PN}-math ${PN}-mime ${PN}-mmap 
${PN}-multiprocessing ${PN}-netclient ${PN}-netserver ${PN}-numbers 
${PN}-pickle ${PN}-pkgutil ${PN}-plistlib ${PN}-pprint ${PN}-profile 
${PN}-pydoc ${PN}-re ${PN}-readline ${PN}-resource ${PN}-robotparser 
${PN}-shell ${PN}-smtpd ${PN}-sqlite3 ${PN}-sqlite3-tests ${PN}-stringold 
${PN}-subprocess ${PN}-syslog ${PN}-terminal ${PN}-tests ${PN}-textutils 
${PN}-threading ${PN}-tkinter ${PN}-unittest ${PN}-unixadmin ${PN}-xml 
${PN}-xmlrpc ${PN}-zlib ${PN}-modules ${PN}-tools"
 
 SUMMARY_${PN}-2to3="Python automated Python 2 to 3 code translator"
 RDEPENDS_${PN}-2to3="${PN}-core"
@@ -284,4 +284,6 @@ SUMMARY_${PN}-modules="All Python modules"
 RDEPENDS_${PN}-modules="${PN}-2to3 ${PN}-argparse ${PN}-audio ${PN}-bsddb 
${PN}-codecs ${PN}-compile ${PN}-compiler ${PN}-compression ${PN}-contextlib 
${PN}-core ${PN}-crypt ${PN}-ctypes ${PN}-curses ${PN}-datetime ${PN}-db 
${PN}-debugger ${PN}-difflib ${PN}-distutils ${PN}-doctest ${PN}-email

[OE-core] [PATCH v2 14/20] python-pgo-image: profiling for python3

2017-05-16 Thread Markus Lehtonen

Add a new 'do_profile3' task for python-pgo-image that runs profiling
task for python3 and retrieves the profile data, similarly to
'do_profile' for python2. Profile data will be copied into a directory
pointed by PYTHON3_PROFILE_DIR on the host system. The profile task may
be specified with PYTHON3_PROFILE_TASK.

[YOCTO #9338]

Signed-off-by: Markus Lehtonen 
---
 meta/recipes-devtools/images/python-pgo-image.bb | 57 +++-
 1 file changed, 45 insertions(+), 12 deletions(-)

diff --git a/meta/recipes-devtools/images/python-pgo-image.bb 
b/meta/recipes-devtools/images/python-pgo-image.bb
index 05252a1a46..285e403d32 100644
--- a/meta/recipes-devtools/images/python-pgo-image.bb
+++ b/meta/recipes-devtools/images/python-pgo-image.bb
@@ -3,6 +3,7 @@ SUMMARY = "Minimal image for doing Python profiling (for PGO)"
 IMAGE_FEATURES += "ssh-server-dropbear"
 IMAGE_INSTALL = "packagegroup-core-boot"
 IMAGE_INSTALL += "python-profile-opt python-profile-opt-tests 
python-profile-opt-tools"
+IMAGE_INSTALL += "python-profile-opt3 python-profile-opt3-tests"
 
 LICENSE = "MIT"
 
@@ -20,13 +21,18 @@ PROFILE_DATA_WORKDIR = "${WORKDIR}/profile-data"
 PYTHON_PROFILE_TASK_DEFAULT = 
"/opt/share/doc/python-profile-opt/Tools/pybench/pybench.py -n 2 --with-gc 
--with-syscheck"
 PYTHON_PROFILE_TASK ?= "${PYTHON_PROFILE_TASK_DEFAULT}"
 
+PYTHON3_PROFILE_DIR ?= "${TMPDIR}/work-shared/${MACHINE}/python3/pgo-data"
+PYTHON3_PROFILE_TASK_DEFAULT = "-m test.regrtest --pgo -w -x test_asyncore 
test_gdb test_multiprocessing_fork test_multiprocessing_forkserver 
test_multiprocessing_main_handling test_multiprocessing_spawn test_subprocess"
+
+PYTHON3_PROFILE_TASK ?= "${PYTHON3_PROFILE_TASK_DEFAULT}"
+
 # We need these because we're utilizing the runtime test helpers from oeqa
 TEST_TARGET ?= "qemu"
 TEST_QEMUBOOT_TIMEOUT ?= "1000"
 TEST_LOG_DIR ?= "${WORKDIR}/qemulogs"
 FIND_ROOTFS = "1"
 
-python do_profile() {
+def run_profile(d, profile_bin, profile_task, tgt_in_dir, host_out_dir):
 from oeqa.targetcontrol import get_target_controller
 from oe.utils import getstatusoutput
 
@@ -40,28 +46,34 @@ python do_profile() {
 target.start(extra_bootparams=bootparams)
 
 # Run profile task
-profile_cmd = 'LD_LIBRARY_PATH=/opt/lib /opt/bin/python %s' % 
d.getVar("PYTHON_PROFILE_TASK", True)
-ret, output = target.run(profile_cmd, timeout=7200)
+ret, output = target.run(profile_bin + ' ' + profile_task, 
timeout=7200)
 if ret:
 bb.fatal("Failed to run profile task on target: %s" % output)
-ret, output = target.run('tar czf pgo-data.tgz -C 
/home/root/python-pgo-profiles/ .')
+ret, output = target.run('tar czf pgo-data.tgz -C %s .' % tgt_in_dir)
 if ret:
 bb.fatal("Failed to archive profile data on target: %s" % output)
 
 # Retrieve and unpack profile data
-profile_dir = d.getVar("PROFILE_DATA_WORKDIR", True)
-target.copy_from('/home/root/pgo-data.tgz', profile_dir)
+target.copy_from('/home/root/pgo-data.tgz', host_out_dir)
 
-profile_tarball = os.path.join(profile_dir, 'pgo-data.tgz')
-ret, output = getstatusoutput('tar xf %s -C %s' % (profile_tarball, 
profile_dir))
+profile_tarball = os.path.join(host_out_dir, 'pgo-data.tgz')
+ret, output = getstatusoutput('tar xf %s -C %s' % (profile_tarball, 
host_out_dir))
 os.unlink(profile_tarball)
 if ret:
 bb.fatal("Failed to unpack python profile data: %s" % output)
 finally:
 target.stop()
 
+
+# Profile task for Python2
+python do_profile() {
+outdir = os.path.join(d.getVar("PROFILE_DATA_WORKDIR", True), 'python')
+run_profile(d, 'LD_LIBRARY_PATH=/opt/lib /opt/bin/python',
+d.getVar('PYTHON_PROFILE_TASK', True),
+'/home/root/python-pgo-profiles/',
+outdir)
 # Exclude files causing problems in cross-profiling
-excludes = [os.path.join(profile_dir, 'Modules', 'posixmodule.gcda')]
+excludes = [os.path.join(outdir, 'Modules', 'posixmodule.gcda')]
 for path in excludes:
 if os.path.exists(path):
 os.unlink(path)
@@ -69,14 +81,35 @@ python do_profile() {
 
 addtask profile after do_build
 do_profile[depends] += "qemu-native:do_populate_sysroot 
qemu-helper-native:do_populate_sysroot"
-do_profile[cleandirs] = "${PROFILE_DATA_WORKDIR}"
-
+do_profile[cleandirs] = "${PROFILE_DATA_WORKDIR}/python"
 
 python do_profile_setscene () {
 sstate_setscene(d)
 }
 
 SSTATETASKS += "do_profile"
-do_profile[sstate-inputdirs] = "${PROFILE_DATA_WORKDIR}"
+do_profile[sstate-inputdirs] = "${PROFILE_DATA_WORKDIR}/python"
 do_profile[sstate-outputdirs] = "${PYTHON_PROFILE_DIR}"
 addtask do_profile_setscene
+
+
+# Profile task for Python3
+python do_profile3() {
+run_profile(d, 'LD_LIBRARY_PATH=/opt/lib /opt/bin/python3',
+d.getVar('PYTHON3_PROFILE_TASK', True),
+

[OE-core] [PATCH v2 05/20] devtools/images: add python-pgo-image

2017-05-16 Thread Markus Lehtonen

This is a special image for profiling Python in order to utilize
profile-guided-optimization. Profile data can be obtained by by running
bitbake python-pgo-image -c profile. It will be located in the directory
pointed by ${PYTHON_PROFILE_DIR}. The profile task that is run can be
altered by specifying ${PYTHON_PROFILE_TASK}.

[YOCTO #9338]

Signed-off-by: Markus Lehtonen 
---
 meta/recipes-devtools/images/python-pgo-image.bb | 62 
 1 file changed, 62 insertions(+)
 create mode 100644 meta/recipes-devtools/images/python-pgo-image.bb

diff --git a/meta/recipes-devtools/images/python-pgo-image.bb 
b/meta/recipes-devtools/images/python-pgo-image.bb
new file mode 100644
index 00..af9da20338
--- /dev/null
+++ b/meta/recipes-devtools/images/python-pgo-image.bb
@@ -0,0 +1,62 @@
+SUMMARY = "Minimal image for doing Python profiling (for PGO)"
+
+IMAGE_FEATURES += "ssh-server-dropbear"
+IMAGE_INSTALL = "packagegroup-core-boot python-profile-opt 
python-profile-opt-tests"
+
+LICENSE = "MIT"
+
+inherit core-image
+
+PYTHON_PROFILE_DIR ?= "${TMPDIR}/work-shared/${MACHINE}/python/pgo-data"
+PYTHON_PROFILE_TASK ?= "-m test.regrtest --pgo -w -x test_asyncore test_gdb 
test_multiprocessing test_subprocess"
+
+# We need these because we're utilizing the runtime test helpers from oeqa
+TEST_TARGET ?= "qemu"
+TEST_QEMUBOOT_TIMEOUT ?= "1000"
+TEST_LOG_DIR ?= "${WORKDIR}/qemulogs"
+FIND_ROOTFS = "1"
+
+python do_profile() {
+from oeqa.targetcontrol import get_target_controller
+from oe.utils import getstatusoutput
+
+target = get_target_controller(d)
+target.deploy()
+try:
+# Boot target
+bootparams = None
+if d.getVar('VIRTUAL-RUNTIME_init_manager', True) == 'systemd':
+bootparams = 'systemd.log_level=debug systemd.log_target=console'
+target.start(extra_bootparams=bootparams)
+
+# Run profile task
+profile_cmd = 'LD_LIBRARY_PATH=/opt/lib /opt/bin/python %s' % 
d.getVar("PYTHON_PROFILE_TASK", True)
+ret, output = target.run(profile_cmd, timeout=7200)
+if ret:
+bb.fatal("Failed to run profile task on target: %s" % output)
+ret, output = target.run('tar czf pgo-data.tgz -C 
/home/root/python-pgo-profiles/ .')
+if ret:
+bb.fatal("Failed to archive profile data on target: %s" % output)
+
+# Retrieve and unpack profile data
+profile_dir = d.getVar("PYTHON_PROFILE_DIR", True)
+target.copy_from('/home/root/pgo-data.tgz', profile_dir)
+
+profile_tarball = os.path.join(profile_dir, 'pgo-data.tgz')
+ret, output = getstatusoutput('tar xf %s -C %s' % (profile_tarball, 
profile_dir))
+os.unlink(profile_tarball)
+if ret:
+bb.fatal("Failed to unpack python profile data: %s" % output)
+finally:
+target.stop()
+
+# Exclude files causing problems in cross-profiling
+excludes = [os.path.join(profile_dir, 'Modules', 'posixmodule.gcda')]
+for path in excludes:
+if os.path.exists(path):
+os.unlink(path)
+}
+
+addtask profile after do_build
+do_profile[depends] += "qemu-native:do_populate_sysroot 
qemu-helper-native:do_populate_sysroot"
+do_profile[cleandirs] = "${PYTHON_PROFILE_DIR}"
-- 
2.12.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 00/20] support profile-optimized build for Python

2017-05-16 Thread Markus Lehtonen

This patchset makes it possible to make a PGO (profile-guided-optimization)
build of python. This version of the patchset is almost identical to v1
submitted back in February, with these changes:
- rebased on top of latest oe-core master
- exclude profile data for Modules/posixmodule of Python 2.7 as it was not
  working correctly

[YOCTO #9338]

The following changes since commit 123962018251dfb1d6ca5aa5c0d02534007de3ab:

  build-appliance-image: Update to master head revision (2017-05-01 08:56:43 
+0100)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib marquiz/fixes-9338
  
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=marquiz/fixes-9338

Markus Lehtonen (20):
  python-native: support profile optimized build
  python: add python-profile-opt recipe
  python: remove path hack from setup.py
  python-profile-opt: rename libpython
  devtools/images: add python-pgo-image
  python: make profile-optimized build possible
  python-pgo-image: exclude tests from the default profile target
  python: add python-tools subpackage
  python-pgo-image: switch python default profile task to pybench
  python-pgo-image: enable sstate for do_profile
  python3-native: support profile optimized build
  python3: fix depends of python-tests
  python3: add python-profile-opt3 recipe
  python-pgo-image: profiling for python3
  python3: remove two setup.py cross-compile hacks
  python3: support profile optimized build
  python3: fix profile-optimized build of modules
  python-pgo-image: exclude tests from the python3 profile target
  python3: add python3-tools subpackage
  python-pgo-image: change python3 profile target to pybench

 meta/recipes-devtools/images/python-pgo-image.bb   | 121 +
 .../python/python-2.7-manifest.inc |   8 +-
 .../python/python-3.5-manifest.inc |  10 +-
 ...x-.so-loading-when-when-running-profile-t.patch |  26 +
 .../python/python-native_2.7.13.bb |  10 ++
 meta/recipes-devtools/python/python-profile-opt3   |   1 +
 .../python/python-profile-opt3_3.5.2.bb|  15 +++
 .../python/python-profile-opt_2.7.13.bb|  13 +++
 .../01-use-proper-tools-for-cross-build.patch  |  10 --
 ...rename-libpython-to-libpython-profile-opt.patch |  94 
 .../python/python3-native_3.5.2.bb |   9 ++
 .../python/python3/000-cross-compile.patch |   9 --
 ...efile-add-install_generate_profile-target.patch |  25 +
 ...-CFLAGS-for-extensions-when-cross-compili.patch |  56 ++
 .../python3-setup.py-no-host-headers-libs.patch|  33 --
 ...name-libpython3-to-libpython-profile-opt3.patch | 111 +++
 .../setup.py-find-libraries-in-staging-dirs.patch  |  30 -
 meta/recipes-devtools/python/python3_3.5.2.bb  |  74 +
 meta/recipes-devtools/python/python_2.7.13.bb  |  54 +++--
 19 files changed, 590 insertions(+), 119 deletions(-)
 create mode 100644 meta/recipes-devtools/images/python-pgo-image.bb
 create mode 100644 
meta/recipes-devtools/python/python-native/Makefile-fix-.so-loading-when-when-running-profile-t.patch
 create mode 12 meta/recipes-devtools/python/python-profile-opt3
 create mode 100644 meta/recipes-devtools/python/python-profile-opt3_3.5.2.bb
 create mode 100644 meta/recipes-devtools/python/python-profile-opt_2.7.13.bb
 create mode 100644 
meta/recipes-devtools/python/python/rename-libpython-to-libpython-profile-opt.patch
 create mode 100644 
meta/recipes-devtools/python/python3/Makefile-add-install_generate_profile-target.patch
 create mode 100644 
meta/recipes-devtools/python/python3/Use-correct-CFLAGS-for-extensions-when-cross-compili.patch
 delete mode 100644 
meta/recipes-devtools/python/python3/python3-setup.py-no-host-headers-libs.patch
 create mode 100644 
meta/recipes-devtools/python/python3/rename-libpython3-to-libpython-profile-opt3.patch
 delete mode 100644 
meta/recipes-devtools/python/python3/setup.py-find-libraries-in-staging-dirs.patch

-- 
2.12.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 01/20] python-native: support profile optimized build

2017-05-16 Thread Markus Lehtonen

Make it possible to build python-native with profile directed
optimization enabled.  The feature is enabled by setting
PYTHON_NATIVE_PROFILE_OPT = "1" in local.conf.

The profile task to run may be specified with PYTHON_NATIVE_PROFILE_TASK
variable in local.conf, e.g.
PYTHON_NATIVE_PROFILE_TASK = "${S}/Tools/pybench/pybench.py -n 2 --with-gc 
--with-syscheck"

[YOCTO #9338]

Signed-off-by: Markus Lehtonen 
---
 ...x-.so-loading-when-when-running-profile-t.patch | 26 ++
 .../python/python-native_2.7.13.bb | 10 +
 2 files changed, 36 insertions(+)
 create mode 100644 
meta/recipes-devtools/python/python-native/Makefile-fix-.so-loading-when-when-running-profile-t.patch

diff --git 
a/meta/recipes-devtools/python/python-native/Makefile-fix-.so-loading-when-when-running-profile-t.patch
 
b/meta/recipes-devtools/python/python-native/Makefile-fix-.so-loading-when-when-running-profile-t.patch
new file mode 100644
index 00..e4c89dba03
--- /dev/null
+++ 
b/meta/recipes-devtools/python/python-native/Makefile-fix-.so-loading-when-when-running-profile-t.patch
@@ -0,0 +1,26 @@
+Makefile: fix .so loading when when running profile task
+
+Use correct search path when doing profiling for profile-optimized
+build. This way we load the correct Python libraries, i.e. the ones from
+our build directory, and not the ones from the host os.
+
+Upstream-Status: Pending
+
+Signed-off-by: Markus Lehtonen 
+
+diff --git a/Makefile.pre.in b/Makefile.pre.in
+index 2affc41..4355e7f 100644
+--- a/Makefile.pre.in
 b/Makefile.pre.in
+@@ -458,7 +458,7 @@ build_all_generate_profile:
+ 
+ run_profile_task:
+   : # FIXME: can't run for a cross build
+-  $(LLVM_PROF_FILE) ./$(BUILDPYTHON) $(PROFILE_TASK) || true
++  $(LLVM_PROF_FILE) $(RUNSHARED) ./$(BUILDPYTHON) $(PROFILE_TASK) || true
+ 
+ build_all_merge_profile:
+   $(LLVM_PROF_MERGER)
+-- 
+2.6.6
+
diff --git a/meta/recipes-devtools/python/python-native_2.7.13.bb 
b/meta/recipes-devtools/python/python-native_2.7.13.bb
index 7edf153489..28fb007d32 100644
--- a/meta/recipes-devtools/python/python-native_2.7.13.bb
+++ b/meta/recipes-devtools/python/python-native_2.7.13.bb
@@ -17,6 +17,7 @@ SRC_URI += "\
 file://builddir.patch \
 file://parallel-makeinst-create-bindir.patch \
 file://revert_use_of_sysconfigdata.patch \
+file://Makefile-fix-.so-loading-when-when-running-profile-t.patch \
"
 
 S = "${WORKDIR}/Python-${PV}"
@@ -35,10 +36,19 @@ EXTRA_OEMAKE = '\
   STAGING_INCDIR=${STAGING_INCDIR_NATIVE} \
 '
 
+PYTHON_NATIVE_MAKE_TARGET ?= "${@'profile-opt' if 
d.getVar('PYTHON_NATIVE_PROFILE_OPT', True) == '1' else ''}"
+
 do_configure_append() {
autoreconf --verbose --install --force --exclude=autopoint 
../Python-${PV}/Modules/_ctypes/libffi
 }
 
+do_compile() {
+if [ -n "${PYTHON_NATIVE_PROFILE_TASK}" ]; then
+sed -i -e 
's,^PROFILE_TASK=.*,PROFILE_TASK=${PYTHON_NATIVE_PROFILE_TASK},g' Makefile
+fi
+oe_runmake ${PYTHON_NATIVE_MAKE_TARGET}
+}
+
 do_install() {
oe_runmake 'DESTDIR=${D}' install
install -d ${D}${bindir}/${PN}
-- 
2.12.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 03/20] python: remove path hack from setup.py

2017-05-16 Thread Markus Lehtonen

There should be no need to hack the library and include dirs in
setup.py. The script detects them from compiler output. We need to
remove this is because python-profile-opt recipe is built with a
non-standard prefix setting (i.e. /opt) and the hack breaks the
discovery of system libraries and headers.

[YOCTO #9338]

Signed-off-by: Markus Lehtonen 
---
 .../python/python/01-use-proper-tools-for-cross-build.patch| 10 --
 1 file changed, 10 deletions(-)

diff --git 
a/meta/recipes-devtools/python/python/01-use-proper-tools-for-cross-build.patch 
b/meta/recipes-devtools/python/python/01-use-proper-tools-for-cross-build.patch
index 366ce3e400..02656d23c5 100644
--- 
a/meta/recipes-devtools/python/python/01-use-proper-tools-for-cross-build.patch
+++ 
b/meta/recipes-devtools/python/python/01-use-proper-tools-for-cross-build.patch
@@ -85,13 +85,3 @@ Index: Python-2.7.13/setup.py
  assert not self.inplace
  basename, tail = os.path.splitext(ext_filename)
  newname = basename + "_failed" + tail
-@@ -574,6 +575,9 @@ class PyBuildExt(build_ext):
- 
- # XXX Omitted modules: gl, pure, dl, SGI-specific modules
- 
-+lib_dirs = [ os.getenv("STAGING_LIBDIR"),  
os.getenv("STAGING_BASELIBDIR") ]
-+inc_dirs = [ os.getenv("STAGING_INCDIR") ]
-+
- #
- # The following modules are all pretty straightforward, and compile
- # on pretty much any POSIXish platform.
-- 
2.12.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 06/20] python: make profile-optimized build possible

2017-05-16 Thread Markus Lehtonen

This patch makes it possible to build python with profile directed
optimization. That is, feed python build process with profile data to
guide optimization. This is the third (and the last) step in profile
directed optimization for Python.

In order to do a profile-optimized build you need to set
PYTHON_PROFILE_OPT = "1" in your local.conf. Be sure to get the profile
data for python, preferably by running
"bitbake python-pgo-image -c profile".

[YOCTO #9338]

Signed-off-by: Markus Lehtonen 
---
 meta/recipes-devtools/python/python_2.7.13.bb | 25 -
 1 file changed, 24 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/python/python_2.7.13.bb 
b/meta/recipes-devtools/python/python_2.7.13.bb
index efe02aa8a3..d9e458cf72 100644
--- a/meta/recipes-devtools/python/python_2.7.13.bb
+++ b/meta/recipes-devtools/python/python_2.7.13.bb
@@ -42,6 +42,17 @@ EXTRA_OECONF += "ac_cv_file__dev_ptmx=yes 
ac_cv_file__dev_ptc=no"
 STAGING_INCDIR_DEFAULT ?= "${STAGING_INCDIR}"
 STAGING_LIBDIR_DEFAULT ?= "${STAGING_LIBDIR}"
 
+# Automatic profile guided optimization
+PYTHON_MAKE_TARGET ?= "${@'build_all_use_profile' if 
d.getVar('PYTHON_PROFILE_OPT', True) == '1' else ''}"
+PYTHON_PROFILE_DIR ?= "${@'${TMPDIR}/work-shared/${MACHINE}/python/pgo-data' 
if d.getVar('PYTHON_PROFILE_OPT', True) == '1' else ''}"
+python () {
+if (d.getVar('PYTHON_PROFILE_OPT', True) == '1' and
+d.getVar('PYTHON_MAKE_TARGET', True) == 'build_all_use_profile'):
+profile_dir = d.getVar('PYTHON_PROFILE_DIR', True)
+bb.utils.mkdirhier(profile_dir)
+d.setVarFlag('do_compile', 'file-checksums', '%s:True' % profile_dir)
+}
+
 do_configure_append() {
rm -f ${S}/Makefile.orig
 autoreconf -Wcross --verbose --install --force --exclude=autopoint 
../Python-${PV}/Modules/_ctypes/libffi
@@ -79,9 +90,15 @@ do_compile() {
 
export CROSS_COMPILE="${TARGET_PREFIX}"
export PYTHONBUILDDIR="${B}"
-# This is only used in PGO profiling by python-profile-opt package
 if [ "${PYTHON_MAKE_TARGET}" = "build_all_generate_profile" ]; then
+# This is only used in PGO profiling by python-profile-opt package
 export EXTRA_CFLAGS="-fprofile-dir=./python-pgo-profiles/"
+else
+if [ -n "${PYTHON_PROFILE_DIR}" ]; then
+export EXTRA_CFLAGS="-fprofile-dir=${PYTHON_PROFILE_DIR}"
+# Remove non-optimized build artefacts
+oe_runmake clean
+fi
 fi
 
oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/python-native/pgen \
@@ -99,6 +116,11 @@ do_install() {
 
export CROSS_COMPILE="${TARGET_PREFIX}"
export PYTHONBUILDDIR="${B}"
+# This only has effect if we build with -fprofile-use, e.g. when make
+# target is build_all_use_profile
+if [ -n "${PYTHON_PROFILE_DIR}" ]; then
+export EXTRA_CFLAGS="-fprofile-dir=${PYTHON_PROFILE_DIR}"
+fi
 
# After swizzling the makefile, we need to run the build again.
# install can race with the build so we have to run this first, then 
install
@@ -146,6 +168,7 @@ py_package_preprocess () {
 python -m py_compile 
${PKGD}/${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata.py
 }
 
+
 require python-${PYTHON_MAJMIN}-manifest.inc
 
 # manual dependency additions
-- 
2.12.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 07/20] python-pgo-image: exclude tests from the default profile target

2017-05-16 Thread Markus Lehtonen

Exclue tests that are segfaulting or otherwise failing (on qemux86
target), and thus, causing the profile task to fail.

Also, disable tests that are taking very long to execute. In qemux86
target on my i7-3770K:
  test_lib2to3: 1908.8s
  test_io: 1278.4s
  test_itertools: 436.4s

[YOCTO #9338]

Signed-off-by: Markus Lehtonen 
---
 meta/recipes-devtools/images/python-pgo-image.bb | 9 -
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/images/python-pgo-image.bb 
b/meta/recipes-devtools/images/python-pgo-image.bb
index af9da20338..426d6acb45 100644
--- a/meta/recipes-devtools/images/python-pgo-image.bb
+++ b/meta/recipes-devtools/images/python-pgo-image.bb
@@ -8,7 +8,14 @@ LICENSE = "MIT"
 inherit core-image
 
 PYTHON_PROFILE_DIR ?= "${TMPDIR}/work-shared/${MACHINE}/python/pgo-data"
-PYTHON_PROFILE_TASK ?= "-m test.regrtest --pgo -w -x test_asyncore test_gdb 
test_multiprocessing test_subprocess"
+PYTHON_PROFILE_TASK_DEFAULT = "-m test.regrtest --pgo -w -x test_asyncore 
test_gdb test_multiprocessing test_subprocess"
+# Exclude tests that are segfaulting on qemux86 target
+PYTHON_PROFILE_TASK_DEFAULT += "test_bytes test_str test_string test_tuple 
test_unicode test_userstring test_xmlrpc"
+# Exclude tests that are failing on qemux86
+PYTHON_PROFILE_TASK_DEFAULT += "test_StringIO test_builtin test_calendar 
test_cmath test_ctypes test_distutils test_exceptions test_getargs test_gzip 
test_json test_math test_shutil test_socket test_sqlite test_sysconfig 
test_traceback test_warnings"
+# Exclude tests that are taking very long on qemux86
+PYTHON_PROFILE_TASK_DEFAULT += "test_io test_lib2to3 test_itertools"
+PYTHON_PROFILE_TASK ?= "${PYTHON_PROFILE_TASK_DEFAULT}"
 
 # We need these because we're utilizing the runtime test helpers from oeqa
 TEST_TARGET ?= "qemu"
-- 
2.12.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 02/20] python: add python-profile-opt recipe

2017-05-16 Thread Markus Lehtonen

This patch adds a new recipe that builds a special version of python
that produces profile data for profile guided optimization. It will
install under /opt in order to prevent clash with the "normal" python.
The new recipe directly includes the base python recipe so that they are
build in as similar way as possible and this hopefully decreases the
recipe maintenance burden, too.

Normally, python profile-guided-optimization is done simply by doing
"make profile-opt" which first builds python with profile
instrumentation enabled, then runs a profile task to get the profile
data, and last, re-builds python with profile data guiding the
optimization. However, in our cross-build environment this gets a lot
trickier. We need to split out the steps as building is done on the
build host but we need to run the second step (i.e. run the profile
task) on the target hardware. This patch enables the first step, i.e.
building python with profile instrumentation enabled

[YOCTO #9338]

Signed-off-by: Markus Lehtonen 
---
 .../python/python-profile-opt_2.7.13.bb| 17 ++
 meta/recipes-devtools/python/python_2.7.13.bb  | 26 +++---
 2 files changed, 35 insertions(+), 8 deletions(-)
 create mode 100644 meta/recipes-devtools/python/python-profile-opt_2.7.13.bb

diff --git a/meta/recipes-devtools/python/python-profile-opt_2.7.13.bb 
b/meta/recipes-devtools/python/python-profile-opt_2.7.13.bb
new file mode 100644
index 00..058cc63bc9
--- /dev/null
+++ b/meta/recipes-devtools/python/python-profile-opt_2.7.13.bb
@@ -0,0 +1,17 @@
+require python_${PV}.bb
+
+PROVIDES += "libpython2"
+RPROVIDES_${PN}-core += "libpython2"
+
+# Use special prefix in order to prevent clash with normal python
+STAGING_INCDIR_DEFAULT = "${STAGING_DIR_HOST}/usr/include"
+STAGING_LIBDIR_DEFAULT = "${STAGING_DIR_HOST}/usr/${baselib}"
+TARGET_CFLAGS += "-I${STAGING_INCDIR_DEFAULT}"
+TARGET_CPPFLAGS += "-I${STAGING_INCDIR_DEFAULT}"
+prefix = "/opt"
+exec_prefix = "/opt"
+
+PYTHON_MAKE_TARGET = "build_all_generate_profile"
+
+# Force pkgdata not to clash with python recipe
+PKG_libpython-profile-opt2 = "libpython-profile-opt2.7-1.0"
diff --git a/meta/recipes-devtools/python/python_2.7.13.bb 
b/meta/recipes-devtools/python/python_2.7.13.bb
index 4ef99523e0..efe02aa8a3 100644
--- a/meta/recipes-devtools/python/python_2.7.13.bb
+++ b/meta/recipes-devtools/python/python_2.7.13.bb
@@ -37,6 +37,11 @@ CONFIGUREOPTS += " --with-system-ffi "
 
 EXTRA_OECONF += "ac_cv_file__dev_ptmx=yes ac_cv_file__dev_ptc=no"
 
+# These are needed in order to build with modified prefix (python-prorile-opt
+# recipe)
+STAGING_INCDIR_DEFAULT ?= "${STAGING_INCDIR}"
+STAGING_LIBDIR_DEFAULT ?= "${STAGING_LIBDIR}"
+
 do_configure_append() {
rm -f ${S}/Makefile.orig
 autoreconf -Wcross --verbose --install --force --exclude=autopoint 
../Python-${PV}/Modules/_ctypes/libffi
@@ -45,12 +50,12 @@ do_configure_append() {
 do_compile() {
 # regenerate platform specific files, because they depend on system 
headers
 cd ${S}/Lib/plat-linux2
-include=${STAGING_INCDIR} 
${STAGING_BINDIR_NATIVE}/python-native/python \
+include=${STAGING_INCDIR_DEFAULT} 
${STAGING_BINDIR_NATIVE}/python-native/python \
 ${S}/Tools/scripts/h2py.py -i '(u_long)' \
-${STAGING_INCDIR}/dlfcn.h \
-${STAGING_INCDIR}/linux/cdrom.h \
-${STAGING_INCDIR}/netinet/in.h \
-${STAGING_INCDIR}/sys/types.h
+${STAGING_INCDIR_DEFAULT}/dlfcn.h \
+${STAGING_INCDIR_DEFAULT}/linux/cdrom.h \
+${STAGING_INCDIR_DEFAULT}/netinet/in.h \
+${STAGING_INCDIR_DEFAULT}/sys/types.h
 sed -e 's,${STAGING_DIR_HOST},,g' -i *.py
 cd -
 
@@ -60,7 +65,7 @@ do_compile() {
if [ ! -f Makefile.orig ]; then
install -m 0644 Makefile Makefile.orig
fi
-   sed -i -e 's#^LDFLAGS=.*#LDFLAGS=${LDFLAGS} -L. -L${STAGING_LIBDIR}#g' \
+   sed -i -e 's#^LDFLAGS=.*#LDFLAGS=${LDFLAGS} -L. 
-L${STAGING_LIBDIR_DEFAULT}#g' \
-e 's,libdir=${libdir},libdir=${STAGING_LIBDIR},g' \
-e 
's,libexecdir=${libexecdir},libexecdir=${STAGING_DIR_HOST}${libexecdir},g' \
-e 's,^LIBDIR=.*,LIBDIR=${STAGING_LIBDIR},g' \
@@ -74,13 +79,17 @@ do_compile() {
 
export CROSS_COMPILE="${TARGET_PREFIX}"
export PYTHONBUILDDIR="${B}"
+# This is only used in PGO profiling by python-profile-opt package
+if [ "${PYTHON_MAKE_TARGET}" = "build_all_generate_profile" ]; then
+export EXTRA_CFLAGS="-fprofile-dir=./python-pgo-profiles/"
+fi
 
oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/python-native/pgen \
HOSTPYTHON=${STAGING_BINDIR_NATIVE}/python-native/python \
STAGING_LIBDIR=${STAGING_LIBDIR} \
STAGING_INCDIR=${STAGING_INCDIR} \

[OE-core] [PATCH v2 04/20] python-profile-opt: rename libpython

2017-05-16 Thread Markus Lehtonen

The python library needs to be renamed so that the automatic dependency
generation/checking mechanism in bitbake does not get confused.
Otherwise python-profile-opt will depend on libpython from the "normal"
python package.

[YOCTO #9338]

Signed-off-by: Markus Lehtonen 
---
 .../python/python-profile-opt_2.7.13.bb|  6 +-
 ...rename-libpython-to-libpython-profile-opt.patch | 94 ++
 2 files changed, 95 insertions(+), 5 deletions(-)
 create mode 100644 
meta/recipes-devtools/python/python/rename-libpython-to-libpython-profile-opt.patch

diff --git a/meta/recipes-devtools/python/python-profile-opt_2.7.13.bb 
b/meta/recipes-devtools/python/python-profile-opt_2.7.13.bb
index 058cc63bc9..df5eee89c8 100644
--- a/meta/recipes-devtools/python/python-profile-opt_2.7.13.bb
+++ b/meta/recipes-devtools/python/python-profile-opt_2.7.13.bb
@@ -1,7 +1,6 @@
 require python_${PV}.bb
 
-PROVIDES += "libpython2"
-RPROVIDES_${PN}-core += "libpython2"
+SRC_URI += "file://rename-libpython-to-libpython-profile-opt.patch"
 
 # Use special prefix in order to prevent clash with normal python
 STAGING_INCDIR_DEFAULT = "${STAGING_DIR_HOST}/usr/include"
@@ -12,6 +11,3 @@ prefix = "/opt"
 exec_prefix = "/opt"
 
 PYTHON_MAKE_TARGET = "build_all_generate_profile"
-
-# Force pkgdata not to clash with python recipe
-PKG_libpython-profile-opt2 = "libpython-profile-opt2.7-1.0"
diff --git 
a/meta/recipes-devtools/python/python/rename-libpython-to-libpython-profile-opt.patch
 
b/meta/recipes-devtools/python/python/rename-libpython-to-libpython-profile-opt.patch
new file mode 100644
index 00..06bb4c02b3
--- /dev/null
+++ 
b/meta/recipes-devtools/python/python/rename-libpython-to-libpython-profile-opt.patch
@@ -0,0 +1,94 @@
+From bc4690886248d87c019cf3c955d5349327f6fdfc Mon Sep 17 00:00:00 2001
+From: Markus Lehtonen 
+Date: Mon, 17 Oct 2016 15:58:21 +0300
+Subject: [PATCH] rename libpython to libpython-profile-opt
+
+Upstream-Status: Inappropriate [oe-specific]
+
+Signed-off-by: Markus Lehtonen 
+---
+ Lib/distutils/command/build_ext.py | 2 +-
+ Makefile.pre.in| 2 +-
+ Misc/python-config.in  | 2 +-
+ Misc/python.pc.in  | 2 +-
+ configure.ac   | 6 +++---
+ 5 files changed, 7 insertions(+), 7 deletions(-)
+
+diff --git a/Lib/distutils/command/build_ext.py 
b/Lib/distutils/command/build_ext.py
+index 3a49454..56cabce 100644
+--- a/Lib/distutils/command/build_ext.py
 b/Lib/distutils/command/build_ext.py
+@@ -758,7 +758,7 @@ class build_ext (Command):
+ else:
+ from distutils import sysconfig
+ if sysconfig.get_config_var('Py_ENABLE_SHARED'):
+-template = "python%d.%d"
++template = "python-profile-opt%d.%d"
+ pythonlib = (template %
+  (sys.hexversion >> 24, (sys.hexversion >> 16) & 
0xff))
+ return ext.libraries + [pythonlib]
+diff --git a/Makefile.pre.in b/Makefile.pre.in
+index d4b81dc..43fe5fe 100644
+--- a/Makefile.pre.in
 b/Makefile.pre.in
+@@ -562,7 +562,7 @@ $(LIBRARY): $(LIBRARY_OBJS)
+   $(AR) $(ARFLAGS) $@ $(MODOBJS)
+   $(RANLIB) $@
+ 
+-libpython$(VERSION).so: $(LIBRARY_OBJS)
++libpython-profile-opt$(VERSION).so: $(LIBRARY_OBJS)
+   if test $(INSTSONAME) != $(LDLIBRARY); then \
+   $(BLDSHARED) -Wl,-h$(INSTSONAME) -o $(INSTSONAME) 
$(LIBRARY_OBJS) $(MODLIBS) $(SHLIBS) $(LIBC) $(LIBM) $(LDLAST); \
+   $(LN) -f $(INSTSONAME) $@; \
+diff --git a/Misc/python-config.in b/Misc/python-config.in
+index 9ee0fdf..9fb149d 100644
+--- a/Misc/python-config.in
 b/Misc/python-config.in
+@@ -44,7 +44,7 @@ for opt in opt_flags:
+ print ' '.join(flags)
+ 
+ elif opt in ('--libs', '--ldflags'):
+-libs = ['-lpython' + pyver]
++libs = ['-lpython-profile-opt' + pyver]
+ libs += getvar('LIBS').split()
+ libs += getvar('SYSLIBS').split()
+ # add the prefix/lib/pythonX.Y/config dir, but only if there is no
+diff --git a/Misc/python.pc.in b/Misc/python.pc.in
+index 08481a9..0822e54 100644
+--- a/Misc/python.pc.in
 b/Misc/python.pc.in
+@@ -8,6 +8,6 @@ Description: Python library
+ Requires: 
+ Version: @VERSION@
+ Libs.private: @LIBS@
+-Libs: -L${libdir} -lpython@VERSION@
++Libs: -L${libdir} -lpython-profile-opt@VERSION@
+ Cflags: -I${includedir}/python@VERSION@ 
+ 
+diff --git a/configure.ac b/configure.ac
+index 353b533..0b8e399 100644
+--- a/configure.ac
 b/configure.ac
+@@ -765,7 +765,7 @@ AC_SUBST(LIBRARY)
+ AC_MSG_CHECKING(LIBRARY)
+ if test -z "$LIBRARY"
+ then
+-  LIBRARY='libpython$(VERSION).a'
++  LIBRARY='libpython-profile-opt$(VERSION).a'
+ fi
+ AC_MSG_RESULT($LIBRARY)
+ 
+@@ -911,8 +911,8 @@ if test $enable_shared = "yes"; then
+ INSTSONAME="$LDLIBRARY".$SOVERSION
+   ;;
+

Re: [OE-core] [PATCH 0/2] Allow oe-pkgdata-util package-info to display more info

2017-05-16 Thread Richard Purdie

On Tue, 2017-05-16 at 12:56 +0200, Peter Kjellerstedt wrote:
> After a build in our autobuilder, we use `oe-pkgdata-util package-
> info
> -f package.manifest` to store a file with information about each
> installed package. This is typically used to compare builds later
> on. Since not every difference to a package is explained by its
> version, we have found it beneficial to also include the SRC_URI in
> the generated file.
> 
> This patch set adds SRC_URI to the pkgdata that is stored for each
> package, and adds a new option to oe-pkgdata-util package-info,
> -e , that can be used to display extra variables from the
> pkgdata.

I'm going to say no to this.

The reason is that pkgdata is not really about collecting up all build
information. If you want to know how two different builds differ, you'd
use the sigdata files. If I take this patch, more will follow where you
find some new difference you want to track and there are other
mechanisms I'd suggest (buildhistory and siginfo for starters). I don't
want to turn the pkgdata files into something they're not.

Cheers,

Richard
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] gcc-cross-canadian: Use ${target_includedir} for compatibility with meta-micro

2017-05-16 Thread Mike Crowe

meta-micro puts headers in /include rather than /usr/include in the
sysroot. ${target_includedir} means that the correct path will be used
automatically.

Signed-off-by: Mike Crowe 
Acked-by: Phil Blundell 
---
 meta/recipes-devtools/gcc/gcc-cross-canadian.inc | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/gcc/gcc-cross-canadian.inc 
b/meta/recipes-devtools/gcc/gcc-cross-canadian.inc
index ec1d281800..91a6fa6994 100644
--- a/meta/recipes-devtools/gcc/gcc-cross-canadian.inc
+++ b/meta/recipes-devtools/gcc/gcc-cross-canadian.inc
@@ -80,7 +80,7 @@ FILES_${PN} = "\
 ${includedir}/c++/${BINV} \
 ${prefix}/${TARGET_SYS}/bin/* \
 ${prefix}/${TARGET_SYS}/lib/* \
-${prefix}/${TARGET_SYS}/usr/include/* \
+${prefix}/${TARGET_SYS}${target_includedir}/* \
 "
 INSANE_SKIP_${PN} += "dev-so"
 
@@ -153,7 +153,7 @@ ELFUTILS = "nativesdk-elfutils"
 DEPENDS += "nativesdk-gmp nativesdk-mpfr nativesdk-libmpc ${ELFUTILS} 
nativesdk-zlib"
 RDEPENDS_${PN} += "nativesdk-mpfr nativesdk-libmpc ${ELFUTILS}"
 
-SYSTEMHEADERS = "/usr/include"
+SYSTEMHEADERS = "${target_includedir}/"
 SYSTEMLIBS = "${target_base_libdir}/"
 SYSTEMLIBS1 = "${target_libdir}/"
 
-- 
2.11.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [poky][PATCH 0/6] Add patches for gstreamer1.0-plugins-bad V1.10.4

2017-05-16 Thread Carol Zhu

Hi Jussi,

Thanks for your reply.

These six patches are all backports.
And they are just some enhancements and bug fix.

- 0002-qmlglsrc-some-enhancements-for-qmlglsrc.patch
   Add enhancement for qmlglsrc.

- 0003-player-Add-configuration-for-enabling-accurate-seeks.patch
- 0004-player-Add-get-track-number-media-info-API.patch
- 0005-player-Add-overlayvideorenderer-video-sink.patch
- 0006-gst-player-fix-gst-player-failed-to-load-external-su.patch
  And enhancement and bug fix for gst-player.

   - 0007-glupload-fix-memory-leak-when-use-a-GVal.patch
  Bug fix.

   Now that Maxin is working on upgrading the GStreamer stack to V1.12 in 
master, Could you please help to review them and put them into Pyro if they are 
qualified?
   Thanks~



B.R.
Carol

From: Jussi Kukkonen [mailto:jussi.kukko...@intel.com]
Sent: Monday, May 15, 2017 7:38 PM
To: Carol Zhu 
Cc: Patches and discussions about the oe-core layer 

Subject: Re: [OE-core] [poky][PATCH 0/6] Add patches for 
gstreamer1.0-plugins-bad V1.10.4

On 15 May 2017 at 11:52, Yuqing Zhu 
> wrote:
-Add some enhancements for qmlglsrc

-Add configuration for enabling accurate seeks

-Add get track number media info API

-Add overlayvideorenderer video sink

-Fix gst-player failed to load external subtitle uri

-Fix memory leak when use a GValue in glupload

These all seem to be backports from 1.12 or earlier. Maxin is currently working 
on upgrading the GStreamer stack to to 1.12 so these probably do not make sense 
for master.

The patches could be considered for Pyro (not too fond of backported new 
features though -- I don't think oe-core should normally do that).

Jussi

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] Qemu: update to 2.8.1.1

2017-05-16 Thread Sona Sarmadi

Fixed CVE:

 - CVE-2017-2620 (Severity = High)
display: cirrus: potential arbitrary code execution via cirrus_bitblt_cputovideo

Reference
https://lists.gnu.org/archive/html/qemu-devel/2017-02/msg04700.html

Removed patches (already in upstream):

- target-ppc-fix-user-mode.patch

Signed-off-by: Sona Sarmadi 
---
 meta/recipes-devtools/qemu/{qemu_2.8.0.bb => qemu_2.8.1.1.bb} | 7 ++-
 1 file changed, 2 insertions(+), 5 deletions(-)
 rename meta/recipes-devtools/qemu/{qemu_2.8.0.bb => qemu_2.8.1.1.bb} (87%)

diff --git a/meta/recipes-devtools/qemu/qemu_2.8.0.bb 
b/meta/recipes-devtools/qemu/qemu_2.8.1.1.bb
similarity index 87%
rename from meta/recipes-devtools/qemu/qemu_2.8.0.bb
rename to meta/recipes-devtools/qemu/qemu_2.8.1.1.bb
index 19d7e8f..1d5bcf9 100644
--- a/meta/recipes-devtools/qemu/qemu_2.8.0.bb
+++ b/meta/recipes-devtools/qemu/qemu_2.8.1.1.bb
@@ -19,7 +19,6 @@ SRC_URI += " \
 file://no-valgrind.patch \
 file://pathlimit.patch \
 file://qemu-2.5.0-cflags.patch \
-file://target-ppc-fix-user-mode.patch \
 file://glibc-2.25.patch \
 "
 
@@ -28,8 +27,6 @@ SRC_URI += " \
 
file://0002-Introduce-condition-to-notify-waiters-of-completed-c.patch \
 
file://0003-Introduce-condition-in-TPM-backend-for-notification.patch \
 file://0004-Add-support-for-VM-suspend-resume-for-TPM-TIS.patch \
-file://CVE-2016-9908.patch \
-file://CVE-2016-9912.patch \
 "
 
 SRC_URI_append_class-native = " \
@@ -39,8 +36,8 @@ SRC_URI_append_class-native = " \
 
 SRC_URI =+ "http://wiki.qemu-project.org/download/${BP}.tar.bz2;
 
-SRC_URI[md5sum] = "17940dce063b6ce450a12e719a6c9c43"
-SRC_URI[sha256sum] = 
"dafd5d7f649907b6b617b822692f4c82e60cf29bc0fc58bc2036219b591e5e62"
+SRC_URI[md5sum] = "a6a23a0c59fd0f8ec564b0fb89a79954"
+SRC_URI[sha256sum] = 
"f62ab18a1fb9ff5b4c81ed44becc945b11581eff777618141bdb787da55d3638"
 
 COMPATIBLE_HOST_mipsarchn32 = "null"
 COMPATIBLE_HOST_mipsarchn64 = "null"
-- 
1.9.1

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v3] libnl: 3.2.29 -> 3.3.0

2017-05-16 Thread Burton, Ross

On 15 May 2017 at 16:58, Khem Raj  wrote:

> >> +SRC_URI_append_libc-musl = "file://musl-header-conflict.patch"
> >
> >
> > Does the patched version not build with glibc or why is this only patched
> > for musl?
>
> this should be generic
>

Yes, please always apply patches which despite fixing build failures on a
specific libc are in fact safe for all libcs.

Ross
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [RFC][PATCH 0/6] development vs. production builds

2017-05-16 Thread Alexander Kanavin


On 05/16/2017 11:21 AM, Patrick Ohly wrote:


While the "development/production" switch may be great for some projects,
it'll make things only more complicated for others while gaining nothing above
what we have now.


What about the approach I outlined in my reponse to Richard, where we
just introduce the IMAGE_MODE mechanism in OE-core without defining
specific modes? Would you find that useful?


Please no. Global variables are a headache, and the less we have of them 
the better.


Here in particularly, what is wrong with defining:

patricks-awesome-image-developers.bb
(containing IMAGE_FEATURE = "no-password-for-anything")

patricks-awesome-image-production.bb
(containing IMAGE_FEATURE = "serious-security")

and the common bits can go to patricks-awesome-image.inc.

It's simple, direct and self-explanatory. At the end of the day, you 
*always* need to drill down to the code that does the work, and learn 
the details, to truly understand what is happening behind a 
configuration setting, and adding abstraction layers and scattering 
functionality over many different files does not help - quite the opposite.


Alex

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] autogen: fix autoopts script generation to handle shebang

2017-05-16 Thread Alexander Kanavin


On 05/16/2017 08:36 AM, Belal, Awais wrote:

The standard way to fix too long #! lines in oe is to patch
upstream code to use #!/usr/bin/env something (where something is
just the binary name).



Why not simply replace ${POSIX_SHELL} with /bin/sh? Where and how
is it set?


POSIX_SHELL is being set to "/usr/bin/env sh" already through the
recipe but it gets resolved/expanded to
/hosttools/ during the configuration process so
it's not usable for shebang in deep directory hierarchy scenarios. I
guess a better way would be simply to use the same technique that
we're using for perl. So for shell we'll have

#!/usr/bin/env sh

in mk-tpl-config.sh directly. I'll submit the change as v2 if you
think this is okay.


You should not patch out the use of POSIX_SHELL after the fact. Find 
where it is resolved/expanded in the source code in the first place, and 
patch it there. Also, please check why setting POSIX_SHELL in the recipe 
no longer has any effect - there is a patch called 
0001-config-libopts.m4-regenerate-it-from-config-libopts.patch which 
should do the trick, but does not.


Alex

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 7/9] gstreamer1.0-rtsp-server: upgrade to version 1.12.0

2017-05-16 Thread Carlos Rafael Giani

Signed-off-by: Carlos Rafael Giani 
---
 .../recipes-multimedia/gstreamer/gstreamer1.0-rtsp-server_1.10.4.bb | 6 --
 .../recipes-multimedia/gstreamer/gstreamer1.0-rtsp-server_1.12.0.bb | 6 ++
 2 files changed, 6 insertions(+), 6 deletions(-)
 delete mode 100644 
meta/recipes-multimedia/gstreamer/gstreamer1.0-rtsp-server_1.10.4.bb
 create mode 100644 
meta/recipes-multimedia/gstreamer/gstreamer1.0-rtsp-server_1.12.0.bb

diff --git 
a/meta/recipes-multimedia/gstreamer/gstreamer1.0-rtsp-server_1.10.4.bb 
b/meta/recipes-multimedia/gstreamer/gstreamer1.0-rtsp-server_1.10.4.bb
deleted file mode 100644
index 6aa9a53..000
--- a/meta/recipes-multimedia/gstreamer/gstreamer1.0-rtsp-server_1.10.4.bb
+++ /dev/null
@@ -1,6 +0,0 @@
-require gstreamer1.0-rtsp-server.inc
-
-SRC_URI[md5sum] = "ef587fa6393e09bc26f884510596e305"
-SRC_URI[sha256sum] = 
"2f6e12fd4e3568ee190dc24e57e4c3a878971c3a3fb6904a9674404fac256de6"
-
-LIC_FILES_CHKSUM = "file://COPYING;md5=6762ed442b3822387a51c92d928ead0d"
diff --git 
a/meta/recipes-multimedia/gstreamer/gstreamer1.0-rtsp-server_1.12.0.bb 
b/meta/recipes-multimedia/gstreamer/gstreamer1.0-rtsp-server_1.12.0.bb
new file mode 100644
index 000..7a208cc
--- /dev/null
+++ b/meta/recipes-multimedia/gstreamer/gstreamer1.0-rtsp-server_1.12.0.bb
@@ -0,0 +1,6 @@
+require gstreamer1.0-rtsp-server.inc
+
+SRC_URI[md5sum] = "70815a2ca055e1d8505965b17ac79aeb"
+SRC_URI[sha256sum] = 
"85ae6bbe173b365ddf4859967144f1999b436531ecbe09935914bfa9f6b37652"
+
+LIC_FILES_CHKSUM = "file://COPYING;md5=6762ed442b3822387a51c92d928ead0d"
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 9/9] gstreamer1.0-vaapi: upgrade to version 1.12.0

2017-05-16 Thread Carlos Rafael Giani

* Added PACKAGECONFIG_GL variable to make it possible for BSP layers to
  customize what should be the default, egl or glx
* Set virtual/egl instead of virtual/mesa as egl dependency in case
  platform specific drivers provide virtual/egl functionality
* Removed backported allocator patch since it was merged in the release
  and is no longer needed

Signed-off-by: Carlos Rafael Giani 
---
 ...ideobufferpool-create-allocator-if-needed.patch | 61 --
 .../gstreamer/gstreamer1.0-vaapi.inc   |  9 +++-
 .../gstreamer/gstreamer1.0-vaapi_1.10.4.bb |  7 ---
 .../gstreamer/gstreamer1.0-vaapi_1.12.0.bb |  5 ++
 4 files changed, 12 insertions(+), 70 deletions(-)
 delete mode 100644 
meta/recipes-multimedia/gstreamer/gstreamer-vaapi/vaapivideobufferpool-create-allocator-if-needed.patch
 delete mode 100644 
meta/recipes-multimedia/gstreamer/gstreamer1.0-vaapi_1.10.4.bb
 create mode 100644 
meta/recipes-multimedia/gstreamer/gstreamer1.0-vaapi_1.12.0.bb

diff --git 
a/meta/recipes-multimedia/gstreamer/gstreamer-vaapi/vaapivideobufferpool-create-allocator-if-needed.patch
 
b/meta/recipes-multimedia/gstreamer/gstreamer-vaapi/vaapivideobufferpool-create-allocator-if-needed.patch
deleted file mode 100644
index f666adc..000
--- 
a/meta/recipes-multimedia/gstreamer/gstreamer-vaapi/vaapivideobufferpool-create-allocator-if-needed.patch
+++ /dev/null
@@ -1,61 +0,0 @@
-Signed-off-by: Jussi Kukkonen 
-Upstream-Status: Backport [commit 59a731be6b in '1.10' branch]
-
-
-From 02a6002c3a80b3a5301c0943b1a1518bbdf439fc Mon Sep 17 00:00:00 2001
-From: =?UTF-8?q?V=C3=ADctor=20Manuel=20J=C3=A1quez=20Leal?=
- 
-Date: Fri, 21 Apr 2017 19:07:18 +0200
-Subject: [PATCH] vaapivideobufferpool: create allocator if needed
-
-Backport to branch 1.10
-
-Sometimes a video decoder could set different buffer pool
-configurations, because their frame size changes. In this case we
-did not reconfigure the allocator.
-
-This patch enables this use case, creating a new allocator inside
-the VAAPI buffer pool if the caps changed, if it is not dmabuf-based.
-If so, it is just reconfigured, since it doesn't have a surface pool.
-
-https://bugzilla.gnome.org/show_bug.cgi?id=781577

- gst/vaapi/gstvaapivideobufferpool.c | 21 +
- 1 file changed, 21 insertions(+)
-
-diff --git a/gst/vaapi/gstvaapivideobufferpool.c 
b/gst/vaapi/gstvaapivideobufferpool.c
-index a3b9223f..9a50614b 100644
 a/gst/vaapi/gstvaapivideobufferpool.c
-+++ b/gst/vaapi/gstvaapivideobufferpool.c
-@@ -159,6 +159,27 @@ gst_vaapi_video_buffer_pool_set_config (GstBufferPool * 
pool,
- gst_object_replace ((GstObject **) & priv->allocator, NULL);
-   priv->video_info = new_vip;
- 
-+  {
-+guint surface_alloc_flags;
-+gboolean vinfo_changed = FALSE;
-+
-+if (allocator) {
-+  const GstVideoInfo *alloc_vinfo =
-+  gst_allocator_get_vaapi_video_info (allocator, 
_alloc_flags);
-+  vinfo_changed = gst_video_info_changed (alloc_vinfo, _vip);
-+}
-+
-+if (vinfo_changed && allocator && priv->use_dmabuf_memory) {
-+  gst_allocator_set_vaapi_video_info (allocator, _vip,
-+  surface_alloc_flags);
-+} else if (!priv->use_dmabuf_memory && (vinfo_changed || !allocator)) {
-+  /* let's destroy the other allocator and create a new one */
-+  allocator = gst_vaapi_video_allocator_new (priv->display, _vip,
-+  surface_alloc_flags);
-+  gst_buffer_pool_config_set_allocator (config, allocator, NULL);
-+}
-+  }
-+
-   if (!gst_buffer_pool_config_has_option (config,
-   GST_BUFFER_POOL_OPTION_VAAPI_VIDEO_META))
- goto error_no_vaapi_video_meta_option;
--- 
-2.11.0
-
-
diff --git a/meta/recipes-multimedia/gstreamer/gstreamer1.0-vaapi.inc 
b/meta/recipes-multimedia/gstreamer/gstreamer1.0-vaapi.inc
index ef0734b..f0ff3c7 100644
--- a/meta/recipes-multimedia/gstreamer/gstreamer1.0-vaapi.inc
+++ b/meta/recipes-multimedia/gstreamer/gstreamer1.0-vaapi.inc
@@ -22,12 +22,17 @@ REQUIRED_DISTRO_FEATURES ?= "opengl"
 
 PACKAGES =+ "${PN}-tests"
 
+# opengl packageconfig factored out to make it easy for distros
+# and BSP layers to pick either glx, egl, or no GL
+PACKAGECONFIG_GL ?= "${@bb.utils.contains('DISTRO_FEATURES', 'opengl', 'egl', 
'', d)}"
+
 PACKAGECONFIG ??= "drm \
-   ${@bb.utils.contains('DISTRO_FEATURES', 'opengl x11', 
'glx', '', d)} \
+   ${PACKAGECONFIG_GL} \
${@bb.utils.filter('DISTRO_FEATURES', 'wayland x11', d)}"
 
 PACKAGECONFIG[drm] = "--enable-drm,--disable-drm,udev libdrm"
-PACKAGECONFIG[glx] = "--enable-glx,--disable-glx,virtual/mesa"
+PACKAGECONFIG[egl] = "--enable-egl,--disable-egl,virtual/egl"
+PACKAGECONFIG[glx] = "--enable-glx,--disable-glx,virtual/libgl"
 PACKAGECONFIG[wayland] = "--enable-wayland,--disable-wayland,wayland"
 PACKAGECONFIG[x11] = "--enable-x11,--disable-x11,virtual/libx11 libxrandr 
libxrender"

[OE-core] [PATCH 6/9] gstreamer1.0-libav: upgrade to version 1.12.0

2017-05-16 Thread Carlos Rafael Giani

* Updated version numbers in patches from 0.10.x to 1.12.0

Signed-off-by: Carlos Rafael Giani 
---
 .../gstreamer/gstreamer1.0-libav/mips64_cpu_detection.patch | 6 +++---
 .../{gstreamer1.0-libav_1.10.4.bb => gstreamer1.0-libav_1.12.0.bb}  | 4 ++--
 2 files changed, 5 insertions(+), 5 deletions(-)
 rename meta/recipes-multimedia/gstreamer/{gstreamer1.0-libav_1.10.4.bb => 
gstreamer1.0-libav_1.12.0.bb} (88%)

diff --git 
a/meta/recipes-multimedia/gstreamer/gstreamer1.0-libav/mips64_cpu_detection.patch
 
b/meta/recipes-multimedia/gstreamer/gstreamer1.0-libav/mips64_cpu_detection.patch
index 7a0b448..3a7f79d 100644
--- 
a/meta/recipes-multimedia/gstreamer/gstreamer1.0-libav/mips64_cpu_detection.patch
+++ 
b/meta/recipes-multimedia/gstreamer/gstreamer1.0-libav/mips64_cpu_detection.patch
@@ -13,10 +13,10 @@ failed to merge target specific data of file
 -Khem
 Upstream-Status: Inappropriate [OE-Specific]
 
-Index: gst-libav-1.10.1/gst-libs/ext/libav/configure
+Index: gst-libav-1.12.0/gst-libs/ext/libav/configure
 ===
 gst-libav-1.10.1.orig/gst-libs/ext/libav/configure
-+++ gst-libav-1.10.1/gst-libs/ext/libav/configure
+--- gst-libav-1.12.0.orig/gst-libs/ext/libav/configure
 gst-libav-1.12.0/gst-libs/ext/libav/configure
 @@ -5269,12 +5269,9 @@ elif enabled mips; then
  
  # Enable minimum ISA based on selected options
diff --git a/meta/recipes-multimedia/gstreamer/gstreamer1.0-libav_1.10.4.bb 
b/meta/recipes-multimedia/gstreamer/gstreamer1.0-libav_1.12.0.bb
similarity index 88%
rename from meta/recipes-multimedia/gstreamer/gstreamer1.0-libav_1.10.4.bb
rename to meta/recipes-multimedia/gstreamer/gstreamer1.0-libav_1.12.0.bb
index e4bb477..dd8a3ea 100644
--- a/meta/recipes-multimedia/gstreamer/gstreamer1.0-libav_1.10.4.bb
+++ b/meta/recipes-multimedia/gstreamer/gstreamer1.0-libav_1.12.0.bb
@@ -14,7 +14,7 @@ SRC_URI = 
"http://gstreamer.freedesktop.org/src/gst-libav/gst-libav-${PV}.tar.xz
file://mips64_cpu_detection.patch \
file://0001-configure-check-for-armv7ve-variant.patch \
"
-SRC_URI[md5sum] = "e2bdd9fde6ca3ff7efffb93df121f4fd"
-SRC_URI[sha256sum] = 
"6ca0feca75e3d48315e07f20ec37cf6260ed1e9dde58df355febd5016246268b"
+SRC_URI[md5sum] = "f9c4593947f8484b237c5d9782939ec3"
+SRC_URI[sha256sum] = 
"39d1477f642ee980b008e78d716b16801eec9a6e5958c5a6cdc0cb04ab0750c4"
 
 S = "${WORKDIR}/gst-libav-${PV}"
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 5/9] gstreamer1.0-plugins-ugly: upgrade to version 1.12.0

2017-05-16 Thread Carlos Rafael Giani

* libmad plugin was removed in 1.12.0, since mad is GPLed, unmaintained,
  and both gst-libav & the mpg123 plugin are fully functional alternatives

Signed-off-by: Carlos Rafael Giani 
---
 meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-ugly.inc   | 1 -
 ...1.0-plugins-ugly_1.10.4.bb => gstreamer1.0-plugins-ugly_1.12.0.bb} | 4 ++--
 2 files changed, 2 insertions(+), 3 deletions(-)
 rename meta/recipes-multimedia/gstreamer/{gstreamer1.0-plugins-ugly_1.10.4.bb 
=> gstreamer1.0-plugins-ugly_1.12.0.bb} (76%)

diff --git a/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-ugly.inc 
b/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-ugly.inc
index 708ad7a..60aa968 100644
--- a/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-ugly.inc
+++ b/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-ugly.inc
@@ -18,7 +18,6 @@ PACKAGECONFIG[amrwb]= 
"--enable-amrwb,--disable-amrwb,opencore-amr"
 PACKAGECONFIG[cdio] = "--enable-cdio,--disable-cdio,libcdio"
 PACKAGECONFIG[dvdread]  = "--enable-dvdread,--disable-dvdread,libdvdread"
 PACKAGECONFIG[lame] = "--enable-lame,--disable-lame,lame"
-PACKAGECONFIG[mad]  = "--enable-mad,--disable-mad,libmad"
 PACKAGECONFIG[mpeg2dec] = "--enable-mpeg2dec,--disable-mpeg2dec,mpeg2dec"
 PACKAGECONFIG[mpg123]   = "--enable-mpg123,--disable-mpg123,mpg123"
 PACKAGECONFIG[x264] = "--enable-x264,--disable-x264,x264"
diff --git 
a/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-ugly_1.10.4.bb 
b/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-ugly_1.12.0.bb
similarity index 76%
rename from 
meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-ugly_1.10.4.bb
rename to meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-ugly_1.12.0.bb
index 92a2caa..3bf9a9c 100644
--- a/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-ugly_1.10.4.bb
+++ b/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-ugly_1.12.0.bb
@@ -7,7 +7,7 @@ SRC_URI = " \
 
http://gstreamer.freedesktop.org/src/gst-plugins-ugly/gst-plugins-ugly-${PV}.tar.xz
 \
 file://0001-introspection.m4-prefix-pkgconfig-paths-with-PKG_CON.patch \
 "
-SRC_URI[md5sum] = "c68d0509c9980b0b70a4b836ff73fff1"
-SRC_URI[sha256sum] = 
"6386c77ca8459cba431ed0b63da780c7062c7cc48055d222024d8eaf198ffa59"
+SRC_URI[md5sum] = "30e815834819a4890eb1ef6ef67cf68d"
+SRC_URI[sha256sum] = 
"5e68ba5046e83ee87b17d7a13931e6091466fd771e1338c5b929ee0944d40ad6"
 
 S = "${WORKDIR}/gst-plugins-ugly-${PV}"
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 8/9] gstreamer1.0-omx: upgrade to version 1.12.0

2017-05-16 Thread Carlos Rafael Giani

Signed-off-by: Carlos Rafael Giani 
---
 .../{gstreamer1.0-omx_1.10.4.bb => gstreamer1.0-omx_1.12.0.bb}| 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
 rename meta/recipes-multimedia/gstreamer/{gstreamer1.0-omx_1.10.4.bb => 
gstreamer1.0-omx_1.12.0.bb} (69%)

diff --git a/meta/recipes-multimedia/gstreamer/gstreamer1.0-omx_1.10.4.bb 
b/meta/recipes-multimedia/gstreamer/gstreamer1.0-omx_1.12.0.bb
similarity index 69%
rename from meta/recipes-multimedia/gstreamer/gstreamer1.0-omx_1.10.4.bb
rename to meta/recipes-multimedia/gstreamer/gstreamer1.0-omx_1.12.0.bb
index dfeefa5..73c5adb 100644
--- a/meta/recipes-multimedia/gstreamer/gstreamer1.0-omx_1.10.4.bb
+++ b/meta/recipes-multimedia/gstreamer/gstreamer1.0-omx_1.12.0.bb
@@ -5,7 +5,7 @@ LIC_FILES_CHKSUM = 
"file://COPYING;md5=4fbd65380cdd255951079008b364516c \
 
 SRC_URI = "http://gstreamer.freedesktop.org/src/gst-omx/gst-omx-${PV}.tar.xz;
 
-SRC_URI[md5sum] = "cedb230f1c47d0cf4b575d70dff66ff2"
-SRC_URI[sha256sum] = 
"45072925cf262f0fd528fab78f0de52734e46a5a88aa802fae51c67c09c81aa2"
+SRC_URI[md5sum] = "eaf9505c7acb9b96215cb0ff775d278d"
+SRC_URI[sha256sum] = 
"acee8c8a88b995fb95601b7b43bf0c83f502d9026bca5705f9c0dc44a757997e"
 
 S = "${WORKDIR}/gst-omx-${PV}"
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 4/9] gstreamer1.0-plugins-bad: upgrade to version 1.12.0

2017-05-16 Thread Carlos Rafael Giani

* 0001-Makefile.am-don-t-hardcode-libtool-name-when-running.patch:
  Extended patch to include fix for libgstallocators
* 0001-Prepend-PKG_CONFIG_SYSROOT_DIR-to-pkg-config-output.patch,
  0001-gstreamer-gl.pc.in-don-t-append-GL_CFLAGS-to-CFLAGS.patch:
  Updated to apply to 1.12.0
* 0001-mssdemux-improved-live-playback-support.patch,
  0001-smoothstreaming-implement-adaptivedemux-s-get_live_s.patch,
  0001-smoothstreaming-use-the-duration-from-the-list-of-fr.patch:
  Removed because they have been merged and are now part of 1.12.0,
  so they are no longe needed
* Updated version numbers in patches from 0.10.x to 1.12.0
* tta plugin was removed, so the corresponding license checks had to go
* Introduced gtk3 support, conditionally enabled if either x11 or wayland
  distro features are enabled as well
* In 1.12.0, old unported plugins were removed, so the .inc file was
  cleaned up accordingly

Signed-off-by: Carlos Rafael Giani 
---
 .../gstreamer/gstreamer1.0-plugins-bad.inc |  35 +-
 ...-don-t-hardcode-libtool-name-when-running.patch |  46 +-
 ...G_CONFIG_SYSROOT_DIR-to-pkg-config-output.patch |  25 +-
 ...gl.pc.in-don-t-append-GL_CFLAGS-to-CFLAGS.patch |  12 +-
 ...1-mssdemux-improved-live-playback-support.patch | 929 -
 ...ming-implement-adaptivedemux-s-get_live_s.patch | 183 
 ...ming-use-the-duration-from-the-list-of-fr.patch |  62 --
 ...valid-sentinels-for-gst_structure_get-etc.patch |  21 +-
 10.4.bb => gstreamer1.0-plugins-bad_1.12.0.bb} |  11 +-
 9 files changed, 82 insertions(+), 1242 deletions(-)
 delete mode 100644 
meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0001-mssdemux-improved-live-playback-support.patch
 delete mode 100644 
meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0001-smoothstreaming-implement-adaptivedemux-s-get_live_s.patch
 delete mode 100644 
meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0001-smoothstreaming-use-the-duration-from-the-list-of-fr.patch
 rename meta/recipes-multimedia/gstreamer/{gstreamer1.0-plugins-bad_1.10.4.bb 
=> gstreamer1.0-plugins-bad_1.12.0.bb} (61%)

diff --git a/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad.inc 
b/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad.inc
index 0ccfc89..b16947e 100644
--- a/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad.inc
+++ b/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad.inc
@@ -14,14 +14,13 @@ SRC_URI_append = " \
 # and BSP layers to pick either (desktop) opengl, gles2, or no GL
 PACKAGECONFIG_GL ?= "${@bb.utils.contains('DISTRO_FEATURES', 'opengl', 'gles2 
egl', '', d)}"
 
-# gtk is not in the PACKAGECONFIG variable by default until
-# the transition to gtk+3 is finished
 PACKAGECONFIG ??= " \
 ${GSTREAMER_ORC} \
 ${PACKAGECONFIG_GL} \
 ${@bb.utils.contains('DISTRO_FEATURES', 'bluetooth', 'bluez', '', d)} \
 ${@bb.utils.filter('DISTRO_FEATURES', 'directfb', d)} \
-${@bb.utils.contains('DISTRO_FEATURES', 'wayland', 'wayland egl', '', d)} \
+${@bb.utils.contains('DISTRO_FEATURES', 'x11', 'gtk', '', d)} \
+${@bb.utils.contains('DISTRO_FEATURES', 'wayland', 'wayland egl gtk', '', 
d)} \
 bz2 curl dash dtls hls neon rsvg sbc smoothstreaming sndfile uvch264 webp \
 "
 
@@ -69,16 +68,11 @@ PACKAGECONFIG[voamrwbenc]  = 
"--enable-voamrwbenc,--disable-voamrwbenc,vo-am
 PACKAGECONFIG[wayland] = 
"--enable-wayland,--disable-wayland,wayland-native wayland wayland-protocols"
 PACKAGECONFIG[webp]= "--enable-webp,--disable-webp,libwebp"
 
-# these plugins have not been ported to 1.0 (yet):
-#   apexsink linsys nas timidity sdl xvid wininet
-#   sndio cdxaparse dccp faceoverlay hdvparse tta mve nuvdemux
-#   patchdetect sdi videomeasure
-
 # these plugins have no corresponding library in OE-core or meta-openembedded:
-#   openni2 winks direct3d directsound winscreencap acm apple_media
+#   openni2 winks direct3d directsound winscreencap acm apple_media iqa
 #   android_media avc bs2b chromaprint daala dts fdkaac gme gsm kate ladspa 
libde265
-#   lv2 mimic mpeg2enc mplex musepack nvenc ofa openh264 opensles pvr 
soundtouch spandsp
-#   spc teletextdec tinyalsa vdpau vulkan wasapi x265 zbar
+#   lv2 mpeg2enc mplex msdk musepack nvenc ofa openh264 opensles soundtouch 
spandsp
+#   spc teletextdec tinyalsa vdpau vulkan wasapi x265 zbar webrtcdsp
 
 # qt5 support is disabled, because it is not present in OE core, and requires 
more work than
 # just adding a packageconfig (it requires access to moc, uic, rcc, and qmake 
paths).
@@ -94,7 +88,6 @@ EXTRA_OECONF += " \
 --enable-vcd \
 --disable-acm \
 --disable-android_media \
---disable-apexsink \
 --disable-apple_media \
 --disable-avc \
 --disable-bs2b \
@@ -107,43 +100,35 @@ EXTRA_OECONF += " \
 --disable-fdk_aac \
 --disable-gme \
 --disable-gsm \
+--disable-iqa \
 --disable-kate \
 --disable-ladspa \
 --disable-libde265 \
-

[OE-core] [PATCH 2/9] gstreamer1.0-plugins-base: upgrade to version 1.12.0

2017-05-16 Thread Carlos Rafael Giani

* Removed encodebin patch since it has been merged

Signed-off-by: Carlos Rafael Giani 
---
 ...eed-more-buffers-in-output-queue-for-bett.patch | 32 --
 ...10.4.bb => gstreamer1.0-plugins-base_1.12.0.bb} |  5 ++--
 2 files changed, 2 insertions(+), 35 deletions(-)
 delete mode 100644 
meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-base/encodebin-Need-more-buffers-in-output-queue-for-bett.patch
 rename meta/recipes-multimedia/gstreamer/{gstreamer1.0-plugins-base_1.10.4.bb 
=> gstreamer1.0-plugins-base_1.12.0.bb} (78%)

diff --git 
a/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-base/encodebin-Need-more-buffers-in-output-queue-for-bett.patch
 
b/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-base/encodebin-Need-more-buffers-in-output-queue-for-bett.patch
deleted file mode 100644
index 5341e3c..000
--- 
a/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-base/encodebin-Need-more-buffers-in-output-queue-for-bett.patch
+++ /dev/null
@@ -1,32 +0,0 @@
-From 540e02c92c75e08b90326863dc787fa5cadf9da6 Mon Sep 17 00:00:00 2001
-From: Song Bing 
-Date: Fri, 13 Mar 2015 18:04:31 +0800
-Subject: [PATCH] encodebin: Need more buffers in output queue for better
- performance
-
-Need more buffers in output queue for better performance
-
-Upstream-Status: Submitted [https://bugzilla.gnome.org/show_bug.cgi?id=744191]
-
-Signed-off-by: Song Bing 

- gst/encoding/gstencodebin.c |3 +--
- 1 file changed, 1 insertion(+), 2 deletions(-)
-
-diff --git a/gst/encoding/gstencodebin.c b/gst/encoding/gstencodebin.c
-index 6728e58..32daae4 100644
 a/gst/encoding/gstencodebin.c
-+++ b/gst/encoding/gstencodebin.c
-@@ -1228,8 +1228,7 @@ _create_stream_group (GstEncodeBin * ebin, 
GstEncodingProfile * sprof,
-* We only use a 1buffer long queue here, the actual queueing will be done
-* in the input queue */
-   last = sgroup->outqueue = gst_element_factory_make ("queue", NULL);
--  g_object_set (sgroup->outqueue, "max-size-buffers", (guint32) 1,
--  "max-size-bytes", (guint32) 0, "max-size-time", (guint64) 0,
-+  g_object_set (sgroup->outqueue, "max-size-time", (guint64) 0,
-   "silent", TRUE, NULL);
- 
-   gst_bin_add (GST_BIN (ebin), sgroup->outqueue);
--- 
-1.7.9.5
-
diff --git 
a/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-base_1.10.4.bb 
b/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-base_1.12.0.bb
similarity index 78%
rename from 
meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-base_1.10.4.bb
rename to meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-base_1.12.0.bb
index 7c81670..637ea3e 100644
--- a/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-base_1.10.4.bb
+++ b/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-base_1.12.0.bb
@@ -9,11 +9,10 @@ SRC_URI = " \
 file://get-caps-from-src-pad-when-query-caps.patch \
 file://0003-ssaparse-enhance-SSA-text-lines-parsing.patch \
 file://0004-subparse-set-need_segment-after-sink-pad-received-GS.patch \
-file://encodebin-Need-more-buffers-in-output-queue-for-bett.patch \
 file://make-gio_unix_2_0-dependency-configurable.patch \
 file://0001-introspection.m4-prefix-pkgconfig-paths-with-PKG_CON.patch \
 "
-SRC_URI[md5sum] = "f6b46f8fac01eb773d556e3efc369e86"
-SRC_URI[sha256sum] = 
"f6d245b6b3d4cb733f81ebb021074c525ece83db0c10e932794b339b8d935eb7"
+SRC_URI[md5sum] = "f34fc5630f9db0db53172fcf3b955e01"
+SRC_URI[sha256sum] = 
"345fc6877f54b8b6e97aacf2996be37a51a0e369f53fc2cf83108af9f764364d"
 
 S = "${WORKDIR}/gst-plugins-base-${PV}"
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/9] gstreamer1.0: upgrade to version 1.12.0

2017-05-16 Thread Carlos Rafael Giani

* Version 1.12 introduces support for libdw (provided by elfutils)
  libdw adds source lines & numbers to backtraces
  new dw packageconfig added for enabling/disabling this feature
* Leftover docbook cruft was removed

Signed-off-by: Carlos Rafael Giani 
---
 meta/recipes-multimedia/gstreamer/gstreamer1.0.inc |  2 +-
 ...tection-of-libunwind-and-libdw-determinis.patch | 56 ++
 .../gstreamer1.0/deterministic-unwind.patch| 24 --
 ...treamer1.0_1.10.4.bb => gstreamer1.0_1.12.0.bb} |  6 +--
 4 files changed, 60 insertions(+), 28 deletions(-)
 create mode 100644 
meta/recipes-multimedia/gstreamer/gstreamer1.0/0001-Make-the-detection-of-libunwind-and-libdw-determinis.patch
 delete mode 100644 
meta/recipes-multimedia/gstreamer/gstreamer1.0/deterministic-unwind.patch
 rename meta/recipes-multimedia/gstreamer/{gstreamer1.0_1.10.4.bb => 
gstreamer1.0_1.12.0.bb} (59%)

diff --git a/meta/recipes-multimedia/gstreamer/gstreamer1.0.inc 
b/meta/recipes-multimedia/gstreamer/gstreamer1.0.inc
index 72d7ce6..3291934 100644
--- a/meta/recipes-multimedia/gstreamer/gstreamer1.0.inc
+++ b/meta/recipes-multimedia/gstreamer/gstreamer1.0.inc
@@ -25,10 +25,10 @@ PACKAGECONFIG[tests] = "--enable-tests,--disable-tests"
 PACKAGECONFIG[valgrind] = "--enable-valgrind,--disable-valgrind,valgrind,"
 PACKAGECONFIG[gst-tracer-hooks] = 
"--enable-gst-tracer-hooks,--disable-gst-tracer-hooks,"
 PACKAGECONFIG[unwind] = "--with-unwind,--without-unwind,libunwind"
+PACKAGECONFIG[dw] = "--with-dw,--without-dw,elfutils"
 
 EXTRA_OECONF = " \
 --disable-dependency-tracking \
---disable-docbook \
 --disable-examples \
 "
 
diff --git 
a/meta/recipes-multimedia/gstreamer/gstreamer1.0/0001-Make-the-detection-of-libunwind-and-libdw-determinis.patch
 
b/meta/recipes-multimedia/gstreamer/gstreamer1.0/0001-Make-the-detection-of-libunwind-and-libdw-determinis.patch
new file mode 100644
index 000..11ce5e9
--- /dev/null
+++ 
b/meta/recipes-multimedia/gstreamer/gstreamer1.0/0001-Make-the-detection-of-libunwind-and-libdw-determinis.patch
@@ -0,0 +1,56 @@
+From fa2344e1a0ffed7e648e36ac72c8871d817e40e0 Mon Sep 17 00:00:00 2001
+From: Carlos Rafael Giani 
+Date: Mon, 8 May 2017 01:38:57 +0200
+Subject: [PATCH] Make the detection of libunwind and libdw deterministic
+
+Upstream-Status: Pending
+
+Signed-off-by: Carlos Rafael Giani 
+---
+ configure.ac | 28 
+ 1 file changed, 20 insertions(+), 8 deletions(-)
+
+diff --git a/configure.ac b/configure.ac
+index b151208..b7922ac 100644
+--- a/configure.ac
 b/configure.ac
+@@ -821,16 +821,28 @@ fi
+ AM_CONDITIONAL(HAVE_GTK, test "x$HAVE_GTK" = "xyes")
+ 
+ dnl libunwind is optionally used by the leaks tracer
+-PKG_CHECK_MODULES(UNWIND, libunwind, HAVE_UNWIND=yes, HAVE_UNWIND=no)
+-if test "x$HAVE_UNWIND" = "xyes"; then
+-  AC_DEFINE(HAVE_UNWIND, 1, [libunwind available])
+-fi
++AC_ARG_WITH([unwind],[AS_HELP_STRING([--with-unwind],[use libunwind])],
++[], [with_unwind=yes])
++AS_IF([test "x$with_unwind" = xyes],
++  [PKG_CHECK_MODULES(UNWIND, libunwind)
++   HAVE_UNWIND=yes
++   AC_DEFINE(HAVE_UNWIND, 1, [libunwind available])
++  ], [
++   HAVE_UNWIND="no"
++  ]
++)
+ 
+ dnl libdw is optionally used to add source lines and numbers to backtraces
+-PKG_CHECK_MODULES(DW, libdw, HAVE_DW=yes, HAVE_DW=no)
+-if test "x$HAVE_DW" = "xyes"; then
+-  AC_DEFINE(HAVE_DW, 1, [libdw available])
+-fi
++AC_ARG_WITH([dw],[AS_HELP_STRING([--with-dw],[use libdw])],
++[], [with_dw=yes])
++AS_IF([test "x$with_dw" = xyes],
++  [PKG_CHECK_MODULES(DW, libdw)
++   HAVE_DW=yes
++   AC_DEFINE(HAVE_DW, 1, [libdw available])
++  ], [
++   HAVE_DW=no
++  ]
++)
+ 
+ dnl Check for backtrace() from libc
+ AC_CHECK_FUNC(backtrace, [
+-- 
+2.7.4
+
diff --git 
a/meta/recipes-multimedia/gstreamer/gstreamer1.0/deterministic-unwind.patch 
b/meta/recipes-multimedia/gstreamer/gstreamer1.0/deterministic-unwind.patch
deleted file mode 100644
index e39e6ca..000
--- a/meta/recipes-multimedia/gstreamer/gstreamer1.0/deterministic-unwind.patch
+++ /dev/null
@@ -1,24 +0,0 @@
-Make the detection of libunwind deterministic.
-
-Upstream-Status: Pending
-Signed-off-by: Ross Burton 
-
-diff --git a/configure.ac b/configure.ac
-index ac88fb2..182c19a 100644
 a/configure.ac
-+++ b/configure.ac
-@@ -829,3 +828,0 @@ AM_CONDITIONAL(HAVE_GTK, test "x$HAVE_GTK" = "xyes")
--dnl libunwind is optionally used by the leaks tracer
--PKG_CHECK_MODULES(UNWIND, libunwind, HAVE_UNWIND=yes, HAVE_UNWIND=no)
--
-@@ -839,3 +836,7 @@ AC_CHECK_FUNC(backtrace, [
--if test "x$HAVE_UNWIND" = "xyes"; then
--  AC_DEFINE(HAVE_UNWIND, 1, [libunwind available])
--fi
-+dnl libunwind is optionally used by the leaks tracer
-+AC_ARG_WITH([unwind],[AS_HELP_STRING([--with-unwind],[use libunwind])],
-+[], [with_unwind=yes])
-+AS_IF([test

[OE-core] [PATCH 3/9] gstreamer1.0-plugins-good: upgrade to version 1.12.0

2017-05-16 Thread Carlos Rafael Giani

* Removed backported v4l2object patch since it is no longer needed

Signed-off-by: Carlos Rafael Giani 
---
 ...Also-add-videometa-if-there-is-padding-to.patch | 35 --
 ...10.4.bb => gstreamer1.0-plugins-good_1.12.0.bb} |  5 ++--
 2 files changed, 2 insertions(+), 38 deletions(-)
 delete mode 100644 
meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-good/0001-v4l2object-Also-add-videometa-if-there-is-padding-to.patch
 rename meta/recipes-multimedia/gstreamer/{gstreamer1.0-plugins-good_1.10.4.bb 
=> gstreamer1.0-plugins-good_1.12.0.bb} (78%)

diff --git 
a/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-good/0001-v4l2object-Also-add-videometa-if-there-is-padding-to.patch
 
b/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-good/0001-v4l2object-Also-add-videometa-if-there-is-padding-to.patch
deleted file mode 100644
index 2a9a23e..000
--- 
a/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-good/0001-v4l2object-Also-add-videometa-if-there-is-padding-to.patch
+++ /dev/null
@@ -1,35 +0,0 @@
-From 22be02612adc757f6a43cefc6ee65ecaef68f0d9 Mon Sep 17 00:00:00 2001
-From: Carlos Rafael Giani 
-Date: Thu, 23 Mar 2017 22:13:05 +0100
-Subject: [PATCH] v4l2object: Also add videometa if there is padding to the
- right and bottom
-
-https://bugzilla.gnome.org/show_bug.cgi?id=780478
-
-Upstream-Status: Backport [1.10.5]
-
-Signed-off-by: Carlos Rafael Giani 

- sys/v4l2/gstv4l2object.c | 5 +++--
- 1 file changed, 3 insertions(+), 2 deletions(-)
-
-diff --git a/sys/v4l2/gstv4l2object.c b/sys/v4l2/gstv4l2object.c
-index 91c8ff0..ed4654e 100644
 a/sys/v4l2/gstv4l2object.c
-+++ b/sys/v4l2/gstv4l2object.c
-@@ -3070,9 +3070,10 @@ store_info:
-   GST_DEBUG_OBJECT (v4l2object->element, "Got sizeimage %" G_GSIZE_FORMAT,
-   info->size);
- 
--  /* to avoid copies we need video meta if top or left padding */
-+  /* to avoid copies we need video meta if there is padding */
-   v4l2object->need_video_meta =
--  ((align->padding_top + align->padding_left) != 0);
-+  ((align->padding_top + align->padding_left + align->padding_right +
-+  align->padding_bottom) != 0);
- 
-   /* ... or if stride is non "standard" */
-   if (!standard_stride)
--- 
-2.7.4
-
diff --git 
a/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-good_1.10.4.bb 
b/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-good_1.12.0.bb
similarity index 78%
rename from 
meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-good_1.10.4.bb
rename to meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-good_1.12.0.bb
index 57447bf..42dd706 100644
--- a/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-good_1.10.4.bb
+++ b/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-good_1.12.0.bb
@@ -10,9 +10,8 @@ SRC_URI = " \
 file://avoid-including-sys-poll.h-directly.patch \
 file://ensure-valid-sentinel-for-gst_structure_get.patch \
 file://0001-introspection.m4-prefix-pkgconfig-paths-with-PKG_CON.patch \
-file://0001-v4l2object-Also-add-videometa-if-there-is-padding-to.patch \
 "
-SRC_URI[md5sum] = "cc0cc13cdb07d4237600b6886b81f31d"
-SRC_URI[sha256sum] = 
"8a86c61434a8c44665365bd0b3557a040937d1f44bf69caee4e9ea816ce74d7e"
+SRC_URI[md5sum] = "9294b22ddab3bec373cbc5e84ff4c084"
+SRC_URI[sha256sum] = 
"8a1d734db7338e00c28b794a7f0a5a9e67d1c5c8b0074075b50638207d372ebc"
 
 S = "${WORKDIR}/gst-plugins-good-${PV}"
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 0/2] Allow oe-pkgdata-util package-info to display more info

2017-05-16 Thread Peter Kjellerstedt

After a build in our autobuilder, we use `oe-pkgdata-util package-info
-f package.manifest` to store a file with information about each
installed package. This is typically used to compare builds later
on. Since not every difference to a package is explained by its
version, we have found it beneficial to also include the SRC_URI in
the generated file.

This patch set adds SRC_URI to the pkgdata that is stored for each
package, and adds a new option to oe-pkgdata-util package-info,
-e , that can be used to display extra variables from the
pkgdata.

//Peter

The following changes since commit 0fcbb4d0ec26ef24db78190dcdc5568af3beaa14:

  GNU_MIRROR: switch from ftp to https (2017-05-11 16:56:00 +0100)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib pkj/oe-pkgdata-util_extra
  
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=pkj/oe-pkgdata-util_extra

Peter Kjellerstedt (2):
  package.bbclass: Add SRC_URI to pkgdata
  oe-pkgdata-util: package-info: Allow extra variables to be displayed

 meta/classes/package.bbclass |  1 +
 scripts/oe-pkgdata-util  | 11 +--
 2 files changed, 10 insertions(+), 2 deletions(-)

-- 
2.12.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/2] oe-pkgdata-util: package-info: Allow extra variables to be displayed

2017-05-16 Thread Peter Kjellerstedt

By specifying the -e  option one or multiple times, extra
variables available in the pkgdata can be displayed, e.,g,
`oe-pkgdata-util package-info -e SUMMARY -e SRC_URI ...`. The extra
variables displayed are quoted since some of them may contain
whitespace.

Signed-off-by: Peter Kjellerstedt 
---
 scripts/oe-pkgdata-util | 11 +--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/scripts/oe-pkgdata-util b/scripts/oe-pkgdata-util
index 6255662a4b..741bc7bf47 100755
--- a/scripts/oe-pkgdata-util
+++ b/scripts/oe-pkgdata-util
@@ -325,8 +325,14 @@ def package_info(args):
 recipe_version = recipe_version + "-" + mappings[pkg]['PR']
 pkg_size = mappings[pkg]['PKGSIZE']
 
-items.append("%s %s %s %s %s" %
- (pkg, pkg_version, recipe, recipe_version, pkg_size))
+line = "%s %s %s %s %s" % (pkg, pkg_version, recipe, recipe_version, 
pkg_size)
+
+for val in args.extra:
+src_uri = mappings[pkg][val].strip()
+src_uri = re.sub(r'\s+', ' ', src_uri)
+line += ' "%s"' % src_uri
+
+items.append(line)
 print('\n'.join(items))
 
 def get_recipe_pkgs(pkgdata_dir, recipe, unpackaged):
@@ -530,6 +536,7 @@ def main():
   description='Looks up the specified 
runtime package(s) and display information')
 parser_package_info.add_argument('pkg', nargs='*', help='Runtime package 
name to look up')
 parser_package_info.add_argument('-f', '--file', help='Read package names 
from the specified file (one per line, first field only)')
+parser_package_info.add_argument('-e', '--extra', help='Extra variables to 
display, e.g., SRC_URI (can be specified multiple times)', action='append')
 parser_package_info.set_defaults(func=package_info)
 
 parser_find_path = subparsers.add_parser('find-path',
-- 
2.12.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/2] package.bbclass: Add SRC_URI to pkgdata

2017-05-16 Thread Peter Kjellerstedt

Emit SRC_URI variable in pkgdata. Having this available is beneficial
when comparing the information generated for two different builds and
looking for changes not directly related to the package versions.

Signed-off-by: Peter Kjellerstedt 
---
 meta/classes/package.bbclass | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index cc466bd1b2..f6641f26e9 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -1370,6 +1370,7 @@ python emit_pkgdata() {
 write_if_exists(sf, pkg, 'LICENSE')
 write_if_exists(sf, pkg, 'DESCRIPTION')
 write_if_exists(sf, pkg, 'SUMMARY')
+write_if_exists(sf, pkg, 'SRC_URI')
 write_if_exists(sf, pkg, 'RDEPENDS')
 rprov = write_if_exists(sf, pkg, 'RPROVIDES')
 write_if_exists(sf, pkg, 'RRECOMMENDS')
-- 
2.12.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [for-krogoth] Fix pseudo issue in some hosts

2017-05-16 Thread Otavio Salvador

Hello guys,

I've been applying following change, on krogoth, and using without problems:

commit db2cf6ec905e2a8870d1adb6c069c08c36055b01 (HEAD -> krogoth)
Author: Richard Purdie 
Date:   Wed May 18 19:28:29 2016 +0100

pseudo: Work around issues with glibc 2.24

There are issues with a change made to RTLD_NEXT behaviour in glibc 2.24
and that change was also backported to older glibc versions in some distros
like Fedora 23. This adds a workaround whilst the pseudo maintainer fixes
various issues properly.

Signed-off-by: Richard Purdie 
(cherry picked from commit 21c38a091c4a1917f62a942c4751b0fd11dce340)

It would be great if this could be queued together of the pending
merges, for next updates.

-- 
Otavio Salvador O.S. Systems
http://www.ossystems.com.brhttp://code.ossystems.com.br
Mobile: +55 (53) 9981-7854Mobile: +1 (347) 903-9750
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [RFC][PATCH 0/6] development vs. production builds

2017-05-16 Thread Mike Looijmans

This seems to assume that every project is similar and that develop vs 
production is something well defined. Well, it's not.


I for one would welcome the demise of debug-tweaks. It's an option that 
enforces one of two policies that may both be wrong.


For example, if the target is a settop box where users tend to port-forward 
the SSH port to the internet, and which is open so that they're expected to 
log in and make changes, the policy we want is that there is initially no root 
password, and that SSH access requires a valid password. This makes the system 
secure against outside attacks. Setting debug-tweaks will allow blank SSH 
passwords, while not setting it would invalidate the root login. Neither is 
acceptable for this project, so one has to resort to overriding the image 
class methods zap_root_password and ssh_allow_empty_password to do nothing.


Instead of some obscure switch with implications hidden somewhere in various 
recipes, classes, distros and local configs, can't we just define options that 
actually do what they say, like for example a boolean option 
"ssh_allow_empty_password"?


While the "development/production" switch may be great for some projects, 
it'll make things only more complicated for others while gaining nothing above 
what we have now.




On 15-05-17 15:26, Patrick Ohly wrote:

At OEDAM [1] I took the AR to flesh out some of my ideas for
introducing global and per-image settings for switching between
development and production builds. The goal is partly to establish
common configure options that then can be used by different layers,
partly to have some actual useful functionality attached to them
already in OE-core.

"development" builds are what a developer does when trying out a
distro or working on his own personal device. "production" is what
device manufacturer put onto the actual end-user hardware.

At OEDAM we already concluded that per-image settings are more
useful. However, sometimes a component also has compile-time choices
that cannot be changed later on in an image, and indeed I found one
example for that (kmod) in OE-core.

Therefore I have included "debug-build" DISTRO_FEATURES support. It's a
bit similar to manpages.bbclass, but in contrast to that (currently)
is meant to be inherited globally - that's partly due to
misunderstanding how manpages.bbclass was meant to be used. This can
be changed, for now I just want to demonstrate that such a distro
feature is not entirely useless, and what effect it could have already
in OE-core.

In refkit, we also switch globally between development and production
builds via .inc files. However, I am in the process of replacing that
with the more flexible per-image IMAGE_MODE check. So from my
perspective, the IMAGE_MODE is more important and useful than the
"debug-build" DISTRO_FEATURE.

From a design perspective, the approach taken here is to let a
developer or image define what mode it wants, and default features can
be configured accordingly. That alternative would be to continue defining
features as before and use the mode to configure QA warnings or errors.

But I find that less flexible, and I suspect it would be harder to
keep track of what is meant to be usable in which mode. Developers
also would have a harder time overriding the defaults.

[1] https://www.openembedded.org/wiki/OEDAM_2017

Patrick Ohly (6):
  build-mode.bbclass: distro-wide debug-build mode
  basefiles: warn about non-production DISTRO_FEATURES in motd
  defaultsetup.conf: enable special "debug-build" DISTRO_FEATURES support
  image-mode.bbclass: per-image production/development/debug mode
  image.bbclass: include IMAGE_MODE support
  local.conf.sample: make debug-tweaks depend on IMAGE_MODE

 meta/classes/build-mode.bbclass   | 16 -
 meta/classes/image-mode.bbclass   | 62 -
 meta/classes/image.bbclass|  3 +-
 meta/conf/distro/defaultsetup.conf|  2 +-
 meta/conf/local.conf.sample   |  5 +-
 meta/recipes-core/base-files/base-files_3.0.14.bb | 11 +++-
 6 files changed, 96 insertions(+), 3 deletions(-)
 create mode 100644 meta/classes/build-mode.bbclass
 create mode 100644 meta/classes/image-mode.bbclass

base-commit: 9f9ebf2e1ba6eda48fb5e3f20d4ca5bbabe3dad4





Kind regards,

Mike Looijmans
System Expert

TOPIC Products
Materiaalweg 4, NL-5681 RJ Best
Postbus 440, NL-5680 AK Best
Telefoon: +31 (0) 499 33 69 79
E-mail: mike.looijm...@topicproducts.com
Website: www.topicproducts.com

Please consider the environment before printing this e-mail



Join our presentation at Electronics & Applications 2017:
FPGA for real-time data processing, subject “Hardware platform for industrial 
ultrasound steel plate Inspection” Topic Embedded Systems - Herman Kuster, 1st 
June 10 AM

Visit http://eabeurs.nl/author/612884/ for more information

--
___
Openembedded-core mailing list

Re: [OE-core] [RFC][PATCH 0/6] development vs. production builds

2017-05-16 Thread Patrick Ohly

On Tue, 2017-05-16 at 09:35 +0200, Mike Looijmans wrote:
> This seems to assume that every project is similar and that develop vs 
> production is something well defined. Well, it's not.
> 
> I for one would welcome the demise of debug-tweaks. It's an option that 
> enforces one of two policies that may both be wrong.

If that's an option, then I'm also for it, and I don't mind dropping the
"debug-build" distro feature.

> While the "development/production" switch may be great for some projects, 
> it'll make things only more complicated for others while gaining nothing 
> above 
> what we have now.

What about the approach I outlined in my reponse to Richard, where we
just introduce the IMAGE_MODE mechanism in OE-core without defining
specific modes? Would you find that useful?

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [RFC][PATCH 0/6] development vs. production builds

2017-05-16 Thread Patrick Ohly

On Tue, 2017-05-16 at 08:29 +0100, Richard Purdie wrote:
> I'm not really sure I like this change, it seems fairly invasive and
> complex for gains which don't seem to add up to me.
> 
> For example after your patchset, we end up with two different places
> motd may be generated based on various flags. I think it would be
> confusing for the user to find out where a message as coming from.

Fair enough. I'm not a big fan of the "debug-build" DISTRO_FEATURES
either, and just included it as reference for what could be done at the
distro level.

What about just patch 4, the one which introduces image-mode.bbclass? Is
that of enough interest to carry it in OE-core? With or without enabling
it by default for images (patch 5)?

That classes currently defines three modes in IMAGE_MODE_VALID, but
intentionally with the weakest default possible. Distros can override
that to match their intended usage modes, or (better, now that I think
about it) we make the default empty and when users try to set IMAGE_MODE
(because they might have seen it in some other distro) they get an error
(assuming that we inherit the class in image.bbclass).

My concern with not having this class in OE-core is that each distro
will have to come up with their own way of doing something like this,
with no consistency between distros. That is going to confuse some
users. If a distro does not want this feature, they can just ignore the
class, once we make IMAGE_MODE_VALID empty by default.

> One of the reasons I have grown to hate "debug-tweaks" is that when you
> enable it, you have no idea what you're really changing as the name
> doesn't tell you what it does.

I'm fine with eliminating "debug-tweaks" entirely. In that case,
local.conf.sample should just list the individual features directly?

>  I worry "debug-build" will become the
> same problem, it doesn't do one thing well but many things badly. If it
> really just changes debug and logging PACKAGECONFIG (if present), why
> doesn't the distro just do that?

The idea was that the distro or user can set a global flag without
having to know all these details. For example, consider the case where a
user adds a custom recipe to a distro. If the distro has a
"development.inc" and "production.inc" with PACKAGECONFIGs that tune
recipes to match the intended usage, then those lists won't configure
the custom recipe. 

> I also worry about the number of classes this introduces. Whilst
> classes are cheap and easy, I have heard concerns we have too many of
> them. In this case image-mode gets hardcoded into image.bbclass.

We could merge the content of image-mode.bbclass also into
image.bbclass, if there is consensus that this code should be active by
default.

I've done it differently here because I wanted to have the ability to
use the standalone image-mode.bbclass in refkit.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [RFC][PATCH 0/6] development vs. production builds

2017-05-16 Thread Richard Purdie

On Mon, 2017-05-15 at 15:26 +0200, Patrick Ohly wrote:
> At OEDAM [1] I took the AR to flesh out some of my ideas for
> introducing global and per-image settings for switching between
> development and production builds. The goal is partly to establish
> common configure options that then can be used by different layers,
> partly to have some actual useful functionality attached to them
> already in OE-core.
> 
> "development" builds are what a developer does when trying out a
> distro or working on his own personal device. "production" is what
> device manufacturer put onto the actual end-user hardware.
> 
> At OEDAM we already concluded that per-image settings are more
> useful. However, sometimes a component also has compile-time choices
> that cannot be changed later on in an image, and indeed I found one
> example for that (kmod) in OE-core.
> 
> Therefore I have included "debug-build" DISTRO_FEATURES support. It's
> a
> bit similar to manpages.bbclass, but in contrast to that (currently)
> is meant to be inherited globally - that's partly due to
> misunderstanding how manpages.bbclass was meant to be used. This can
> be changed, for now I just want to demonstrate that such a distro
> feature is not entirely useless, and what effect it could have
> already
> in OE-core.
> 
> In refkit, we also switch globally between development and production
> builds via .inc files. However, I am in the process of replacing that
> with the more flexible per-image IMAGE_MODE check. So from my
> perspective, the IMAGE_MODE is more important and useful than the
> "debug-build" DISTRO_FEATURE.
> 
> From a design perspective, the approach taken here is to let a
> developer or image define what mode it wants, and default features
> can
> be configured accordingly. That alternative would be to continue
> defining
> features as before and use the mode to configure QA warnings or
> errors.
> 
> But I find that less flexible, and I suspect it would be harder to
> keep track of what is meant to be usable in which mode. Developers
> also would have a harder time overriding the defaults.
> 
> [1] https://www.openembedded.org/wiki/OEDAM_2017

I'm not really sure I like this change, it seems fairly invasive and
complex for gains which don't seem to add up to me.

For example after your patchset, we end up with two different places
motd may be generated based on various flags. I think it would be
confusing for the user to find out where a message as coming from.

One of the reasons I have grown to hate "debug-tweaks" is that when you
enable it, you have no idea what you're really changing as the name
doesn't tell you what it does. I worry "debug-build" will become the
same problem, it doesn't do one thing well but many things badly. If it
really just changes debug and logging PACKAGECONFIG (if present), why
doesn't the distro just do that?

I also worry about the number of classes this introduces. Whilst
classes are cheap and easy, I have heard concerns we have too many of
them. In this case image-mode gets hardcoded into image.bbclass.

Also, have you seen bb.utils.filter()? It might be useful instead of
some of your .intersection() code?

I do think you're right that we need to make some changes in this area
but overall this patchset seems too complex to me somehow and I think
we probably can achieve something similar in a simpler way...

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [RFC][PATCH 6/6] local.conf.sample: make debug-tweaks depend on IMAGE_MODE

2017-05-16 Thread Patrick Ohly

On Mon, 2017-05-15 at 15:27 +0200, Patrick Ohly wrote:
> Enabling "debug-tweaks" unconditionally, even if it is only in the
> local.conf.sample file, runs the risk of that getting used in
> production images.
> 
> By checking the per-image IMAGE_MODE, the debug tweaks only get
> enabled for images not meant for production.
> 
> Signed-off-by: Patrick Ohly 
> ---
>  meta/conf/local.conf.sample | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/meta/conf/local.conf.sample b/meta/conf/local.conf.sample
> index 85c5e21..edadbb7 100644
> --- a/meta/conf/local.conf.sample
> +++ b/meta/conf/local.conf.sample
> @@ -114,8 +114,9 @@ PACKAGE_CLASSES ?= "package_ipk"
>  # e.g. ssh root access has a blank password
>  # There are other application targets that can be used here too, see
>  # meta/classes/image.bbclass and meta/classes/core-image.bbclass for more 
> details.
> -# We default to enabling the debugging tweaks.
> -EXTRA_IMAGE_FEATURES ?= "debug-tweaks"
> +# We default to enabling the debugging tweaks unless an image is explicitly
> +# requested to be built for production.

Would it help to add an additional comment here, like this?

# Note that this expression gets evaluated separately for each image
# and that IMAGE_MODE can get set differently for different images,
# even in the same build.

> +EXTRA_IMAGE_FEATURES ?= "${@ '' if 'production' == d.getVar('IMAGE_MODE') 
> else 'debug-tweaks'}"

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.



-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] ✗ patchtest: failure for [meta-oe,morty,1/1] openssl-native: Compile with -fPIC

2017-05-16 Thread Mirza Krak

2017-05-16 0:10 GMT+02:00 Denys Dmytriyenko :
> On Mon, May 15, 2017 at 09:05:23AM +0200, Mirza Krak wrote:
>> 2017-05-10 13:31 GMT+02:00 Patchwork :
>> > == Series Details ==
>> >
>> > Series: [meta-oe,morty,1/1] openssl-native: Compile with -fPIC
>> > Revision: 1
>> > URL   : https://patchwork.openembedded.org/series/6671/
>> > State : failure
>>
>> I seem to have messed up here.
>>
>> Why does the testing framework think that I was targeting master? Was
>> my subject prefix wrong?
>
> Why is there "meta-oe" in there? This is oe-core, not meta-oe...

Indeed that was an error on my end, using the wrong subject prefix.

-- 
Med Vänliga Hälsningar / Best Regards

Mirza Krak
Senior Embedded Engineer

Endian Technologies AB
Flöjelbergsgatan 11
SE-431 37 Mölndal, Sweden
mobile: +46 730 28 06 22
email: mirza.k...@endian.se
www.endian.se
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] ✗ patchtest: failure for [meta-oe,morty,1/1] openssl-native: Compile with -fPIC

2017-05-16 Thread Mirza Krak

Hi Paul.

2017-05-16 5:42 GMT+02:00 Paul Eggleton :
> Hi Mirza,
>
> On Monday, 15 May 2017 7:05:23 PM NZST Mirza Krak wrote:
>> 2017-05-10 13:31 GMT+02:00 Patchwork :
>> > == Series Details ==
>> >
>> > Series: [meta-oe,morty,1/1] openssl-native: Compile with -fPIC
>> > Revision: 1
>> > URL   : https://patchwork.openembedded.org/series/6671/
>> > State : failure
>>
>> I seem to have messed up here.
>>
>> Why does the testing framework think that I was targeting master? Was
>> my subject prefix wrong?
>
> It probably doesn't understand "," as a separator and thus didn't see "morty"
> as a branch name and assumed master as the default - if that's the case we
> should probably fix that.

I actually did not use "," as seperatator. Do not know where this came
from. My patch is [1].

I guess my error here was adding "meta-oe" as subject prefix in
oe-core mailing list, and I probably triggered some weird behavior in
patchwork with this?

[1]. 
http://lists.openembedded.org/pipermail/openembedded-core/2017-May/136489.html


-- 
Med Vänliga Hälsningar / Best Regards

Mirza Krak
Senior Embedded Engineer

Endian Technologies AB
Flöjelbergsgatan 11
SE-431 37 Mölndal, Sweden
mobile: +46 730 28 06 22
email: mirza.k...@endian.se
www.endian.se
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [RFC][PATCH 6/6] local.conf.sample: make debug-tweaks depend on IMAGE_MODE

2017-05-16 Thread Patrick Ohly

On Mon, 2017-05-15 at 13:25 -0700, Khem Raj wrote:
> On Mon, May 15, 2017 at 12:47 PM, Patrick Ohly  wrote:
> > The production image recipe "foobar-image-production.bb" can use
> > IMAGE_MODE_forcevariable = "production"
> > and the development image recipe "foobar-image-development.bb" can use
> > IMAGE_MODE_forcevariable = "development".

I'll probably automate the creation of these extra image variants via
BBCLASSEXTEND in an utility class. This seems to be of general
usefulness: base image with default IMAGE_MODE, one variant per valid
IMAGE_MODE where that mode is selected explicitly.

> > Then whatever the user might configure in local.conf is ignored in favor
> > of the fixed recipe values. If there's a concern about using
> > _forcevariable: that could be addressed by configuring a global
> > IMAGE_MODE_DEFAULT ??= "" and an IMAGE_MODE ??= "${IMAGE_MODE_DEFAULT}"
> > in image-mode.bbclass and changing IMAGE_MODE_DEFAULT in distro or local
> > conf. Then individual recipes can set IMAGE_MODE =
> > "development/production" without having to fall back to _forcevariable.
> >
> > Or do you mean that there's just one image .bb and traditionally
> > IMAGE_FEATURES were changed to switch back and forth? The same works
> > with IMAGE_MODE. The advantage over enabling or disabling dangerous
> > IMAGE_FEATURES is that users of a distro don't need to know about them.
> > They get the guarantee that (for a responsible distro) the dangerous
> > once will not get enabled by default for IMAGE_MODE=development.
> >
> 
> IMAGE_MODE is a distro settings not image setting is that correct ?

No, it is per-image. The code that checks the variable always runs in
individual image recipes.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.



-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

74 matches

Mail list logo