from:"Paulo Neves"

Re: [OE-core] [PATCH v3 1/2] ntirpc: Add ntirpc recipe

2022-09-15 Thread Paulo Neves

libntirpc is not compatible with libtirpc. Some work would make it so, 
but it is not a trivial change, i tried it :)


Not only the API is different, the pkgconfig presents itself as ntirpc 
whereas most dependencies of litbtirpc specifically try to detect 
libtirpc. A good summary of the differences and the feasibility is in 
this page [1].


I can break the extending of liburcu into another commit no problem.

I was able to break the circular dependency as well, with a patch that 
is already for review upstream.


[1] https://bugs.launchpad.net/ubuntu/+source/nfs-ganesha/+bug/1843403

On 9/15/22 12:50, Ross Burton wrote:

On 28 Aug 2022, at 18:43, Paulo Neves via lists.openembedded.org 
 wrote:

ntirpc is an actively maintained rpc implementation. We cannot completely
remove libtirpc due to arising circular dependencies where ntirpc
depends on libnsl2, which in turn depends on an RPC implementation.

Can you split this into a series of commits?  Extend liburcu, add ntirpc, 
update the recipes to use ntirpc.

Ross



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#170689): 
https://lists.openembedded.org/g/openembedded-core/message/170689
Mute This Topic: https://lists.openembedded.org/mt/93311083/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v3 2/2] nfs-ganesha: Replace unfs3 with nfs-ganesha

2022-09-15 Thread Paulo Neves


Hello Adrian,

Sorry for my late replies but i can only work on this effort 
sporadically. I was not aware of the issue of runqemu and given that i 
am progressing slowly i definitely will follow the breakdown you suggest.


It is taking me quite longer due to the fact that ntirpc is not 
compatible with tirpc. I spent sometime trying to make it so but it is 
huge increase in scope for the current delivery.


ntirpc is also requiring some adaptations with musl that i did not 
anticipate. Some upstream discussions are also taking a bit longer than 
expected.


Paulo Neves

On 9/14/22 23:19, Adrian Freihofer wrote:

Hi Paulo

Is it possible to do the transition from unfsd to nfs-ganesha step by
step?

1. Add nfs-ganesha
2. Refactor runqemu-export-rootfs to work with ganesha.nfsd instead
   of unfsd.
3. Remove unfs3

My conclusion so far is that just removing the unfs3 recipe would break
runqemu-export-rootfs and some features of runqemu. Therefore I would
suggest that you send a V4 which only adds nfs-ganesha as a first step.
Note: There is also a typo: The "-D" from USE_ACL_MAPPING=OFF is
missing.

Regards,
Adrian






-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#170681): 
https://lists.openembedded.org/g/openembedded-core/message/170681
Mute This Topic: https://lists.openembedded.org/mt/93311086/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 1/1] watchdog: Remove unneeded tirpc dependency

2022-09-12 Thread Paulo Neves

it should as there is no rpc code in the project as far as i scanned. I 
built it with musl successfully.


Paulo Neves

On 9/12/22 17:06, Khem Raj wrote:

On Mon, Sep 12, 2022 at 2:21 AM Paulo Neves  wrote:

watchdog code does not have any dependency on rpc code,
therefore the dependency and flags to try to use it are
removed.

Signed-off-by: Paulo Neves 
---
  meta/recipes-extended/watchdog/watchdog_5.16.bb | 4 
  1 file changed, 4 deletions(-)

diff --git a/meta/recipes-extended/watchdog/watchdog_5.16.bb 
b/meta/recipes-extended/watchdog/watchdog_5.16.bb
index 26fcc10487..6031dcaa16 100644
--- a/meta/recipes-extended/watchdog/watchdog_5.16.bb
+++ b/meta/recipes-extended/watchdog/watchdog_5.16.bb
@@ -28,10 +28,6 @@ UPSTREAM_CHECK_REGEX = "/watchdog/(?P(\d+[\.\-_]*)+)/"

  inherit autotools update-rc.d systemd pkgconfig

-DEPENDS += "libtirpc"
-CFLAGS += "-I${STAGING_INCDIR}/tirpc"
-LDFLAGS += "-ltirpc"
-

does musl build works too ?


  EXTRA_OECONF += " --disable-nfs "
  CACHED_CONFIGUREVARS += "ac_cv_path_PATH_SENDMAIL=${sbindir}/sendmail"

--
2.25.1







-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#170542): 
https://lists.openembedded.org/g/openembedded-core/message/170542
Mute This Topic: https://lists.openembedded.org/mt/93628579/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 1/1] watchdog: Remove unneeded tirpc dependency

2022-09-12 Thread Paulo Neves

watchdog code does not have any dependency on rpc code,
therefore the dependency and flags to try to use it are
removed.

Signed-off-by: Paulo Neves 
---
 meta/recipes-extended/watchdog/watchdog_5.16.bb | 4 
 1 file changed, 4 deletions(-)

diff --git a/meta/recipes-extended/watchdog/watchdog_5.16.bb 
b/meta/recipes-extended/watchdog/watchdog_5.16.bb
index 26fcc10487..6031dcaa16 100644
--- a/meta/recipes-extended/watchdog/watchdog_5.16.bb
+++ b/meta/recipes-extended/watchdog/watchdog_5.16.bb
@@ -28,10 +28,6 @@ UPSTREAM_CHECK_REGEX = "/watchdog/(?P(\d+[\.\-_]*)+)/"
 
 inherit autotools update-rc.d systemd pkgconfig
 
-DEPENDS += "libtirpc"
-CFLAGS += "-I${STAGING_INCDIR}/tirpc"
-LDFLAGS += "-ltirpc"
-
 EXTRA_OECONF += " --disable-nfs "
 CACHED_CONFIGUREVARS += "ac_cv_path_PATH_SENDMAIL=${sbindir}/sendmail"
 
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#170533): 
https://lists.openembedded.org/g/openembedded-core/message/170533
Mute This Topic: https://lists.openembedded.org/mt/93628579/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 1/2] ntirpc: Add ntirpc recipe

2022-08-29 Thread Paulo Neves

No. The rationale for ntirpc is that it is a dependency of nfs-ganesha, 
which in turn is the implementation not maintained.


Paulo Neves

On 8/29/22 20:01, Alexander Kanavin wrote:

I think the rationale is that the implementation it replaces is not
anymore maintained?

Alex

On Mon, 29 Aug 2022 at 19:48, Khem Raj  wrote:


On 8/28/22 10:40 AM, Paulo Neves wrote:

Hello Alexander,

Thank you for your review

I updated the commit message for v2 to contain the following text:

We cannot completely  remove libtirpc due to arising circular
dependencies where ntirpc depends on libnsl2, which  in turn depends
on an RPC implementation.


is it core enough to be included into core right away ?



Paulo Neves

On 8/27/22 20:07, Alexander Kanavin wrote:

Can you please explain where the circular dependencies are?

Alex

On Sat, 27 Aug 2022 at 19:46, Paulo Neves  wrote:

ntirpc is an actively maintained rpc implementation. We cannot
completely
remove libtirpc due to arising circular dependencies. libtirpc and
ntirpc
are not mutually exclusive.

Signed-off-by: Paulo Neves 
---
   meta/recipes-extended/ntirpc/ntirpc_4.0.bb| 30
+++
   meta/recipes-extended/quota/quota_4.06.bb |  2 +-
   .../recipes-extended/rpcbind/rpcbind_1.2.6.bb |  2 +-
   .../watchdog/watchdog_5.16.bb |  2 +-
   .../xorg-xserver/xserver-xorg.inc |  2 +-
   .../recipes-support/liburcu/liburcu_0.13.1.bb |  1 +
   6 files changed, 35 insertions(+), 4 deletions(-)
   create mode 100644 meta/recipes-extended/ntirpc/ntirpc_4.0.bb

diff --git a/meta/recipes-extended/ntirpc/ntirpc_4.0.bb
b/meta/recipes-extended/ntirpc/ntirpc_4.0.bb
new file mode 100644
index 00..edebcb8ca9
--- /dev/null
+++ b/meta/recipes-extended/ntirpc/ntirpc_4.0.bb
@@ -0,0 +1,30 @@
+LICENSE = "BSD"
+LIC_FILES_CHKSUM =
"file://COPYING;md5=f835cce8852481e4b2bbbdd23b5e47f3"
+
+SRC_URI =
"https://download.nfs-ganesha.org/4/${PV}/ntirpc-${PV}.tar.gz";
+SRC_URI[md5sum] = "17b0baada54936dcde80eba27bb6d88d"
+SRC_URI[sha256sum] =
"6a7921cd48d02077b394a02be42bf4f9b4678d4ebfd86a7ae4385bbcdce55462"
+
+DEPENDS = "liburcu libnsl2"
+
+inherit cmake pkgconfig
+
+PACKAGECONFIG[gss] = "-DUSE_GSS=true,-DUSE_GSS=false,krb5"
+PACKAGECONFIG[rdma] =
"-DUSE_RPC_RDMA=true,-DUSE_RPC_RDMA=false,rdma-core"
+PACKAGECONFIG[winbind] =
"-D_MSPAC_SUPPORT=true,-D_MSPAC_SUPPORT=false,samba"
+PACKAGECONFIG[profile] =
"-DUSE_PROFILE=true,-DUSE_PROFILE=false,gperftools"
+PACKAGECONFIG[lttng] =
"-DUSE_LTTNG=true,-DUSE_LTTNG=false,lttng-ust lttng-tools"
+
+# Sort out the brain-dead cmake file
+# (https://github.com/nfs-ganesha/ntirpc/issues/150)
+do_install:append() {
+if [ "${prefix}/lib64" != "${libdir}" -a -d ${D}${prefix}/lib64
]; then
+mv ${D}${prefix}/lib64 ${D}${libdir}
+fi
+}
+
+BBCLASSEXTEND = "native nativesdk"
+EXTRA_OECMAKE:armv5 += '-DSYSTEM_LIBRARIES="-latomic"'
+EXTRA_OECMAKE:riscv32 += '-DSYSTEM_LIBRARIES="-latomic"'
+EXTRA_OECMAKE:riscv64 += '-DSYSTEM_LIBRARIES="-latomic"'
+EXTRA_OECMAKE:powerpc += '-DSYSTEM_LIBRARIES="-latomic"'
diff --git a/meta/recipes-extended/quota/quota_4.06.bb
b/meta/recipes-extended/quota/quota_4.06.bb
index 0535d14c20..304477d72b 100644
--- a/meta/recipes-extended/quota/quota_4.06.bb
+++ b/meta/recipes-extended/quota/quota_4.06.bb
@@ -30,6 +30,6 @@ PACKAGECONFIG ??= "tcp-wrappers rpc bsd"
   PACKAGECONFIG:libc-musl = "tcp-wrappers rpc"

   PACKAGECONFIG[tcp-wrappers] =
"--enable-libwrap,--disable-libwrap,tcp-wrappers"
-PACKAGECONFIG[rpc] = "--enable-rpc,--disable-rpc,libtirpc"
+PACKAGECONFIG[rpc] = "--enable-rpc,--disable-rpc,ntirpc"
   PACKAGECONFIG[bsd] =
"--enable-bsd_behaviour=yes,--enable-bsd_behaviour=no,"
   PACKAGECONFIG[ldapmail] =
"--enable-ldapmail,--disable-ldapmail,openldap"
diff --git a/meta/recipes-extended/rpcbind/rpcbind_1.2.6.bb
b/meta/recipes-extended/rpcbind/rpcbind_1.2.6.bb
index dd89726afc..e30a23909c 100644
--- a/meta/recipes-extended/rpcbind/rpcbind_1.2.6.bb
+++ b/meta/recipes-extended/rpcbind/rpcbind_1.2.6.bb
@@ -4,7 +4,7 @@ DESCRIPTION = "The rpcbind utility is a server that
converts RPC \
   SECTION = "console/network"
   HOMEPAGE = "http://sourceforge.net/projects/rpcbind/";
   BUGTRACKER =
"http://sourceforge.net/tracker/?group_id=201237&atid=976751";
-DEPENDS = "libtirpc quota"
+DEPENDS = "ntirpc quota"

   LICENSE = "BSD-3-Clause"
   LIC_FILES_CHKSUM =
"file://COPYING;md5=b46486e4c4a416602693a711bb5bfa39 \
diff --git a/meta/recipes-extended/watchdog/watchdog_5.16.bb
b/meta/recipes-extended/watchdog/watchdog_5.16.bb
index 26fcc10487..5f5fd4dc87 100644
--- a/meta/recipes-extended/watchdog/watchdog_5.16.bb
+++ b/meta/rec

Re: [OE-core] [PATCH 2/2] nfs-ganesha: Replace unfs3 with nfs-ganesha

2022-08-28 Thread Paulo Neves


Hello Alexander,

Thank you for the review. Please see v3.

On 8/27/22 20:06, Alexander Kanavin wrote:

Thank for the replacement, but it seems like the patch isn't quite
complete? The upstream statuses are all incorrect, and the recipe
lacks summary/description/homepage, and has a TODO and an unknown
license in it.

Alex

On Sat, 27 Aug 2022 at 19:46, Paulo Neves  wrote:

nfs-ganesha is actively maintained in contrast with unfs3. glibc tests
pass with nfs-ganesha.

Signed-off-by: Paulo Neves 
---
  meta/lib/oeqa/utils/nfs.py|   2 +-
  ...ix-possible-null-pointer-dereference.patch |  40 +
  ...-Workaround-for-erroneous-gcc-Werror.patch |  33 
  ...ke-nfsv3_optype-guarded-by-_USE_NFS3.patch |  37 
  .../nfs-ganesha/nfs-ganesha/module.patch  |  27 +++
  .../nfs-ganesha/nfs-ganesha_4.0.bb|  42 +
  ...1-Add-listen-action-for-a-tcp-socket.patch |  54 --
  .../0001-attr-fix-utime-for-symlink.patch |  85 --
  ...0001-daemon.c-Libtirpc-porting-fixes.patch |  37 
  .../unfs3/unfs3/alternate_rpc_ports.patch | 158 --
  .../unfs3/unfs3/fix_compile_warning.patch |  25 ---
  ...fix_pid_race_parent_writes_child_pid.patch |  61 ---
  .../unfs3/unfs3/no-yywrap.patch   |  14 --
  .../unfs3/relative_max_socket_path_len.patch  |  74 
  .../unfs3/unfs3/rename_fh_cache.patch |  64 ---
  .../unfs3/unfs3/tcp_no_delay.patch|  56 ---
  .../unfs3/unfs3/unfs3_parallel_build.patch|  37 
  meta/recipes-devtools/unfs3/unfs3_git.bb  |  45 -
  18 files changed, 180 insertions(+), 711 deletions(-)
  create mode 100644 
meta/recipes-connectivity/nfs-ganesha/nfs-ganesha/0001-Fix-possible-null-pointer-dereference.patch
  create mode 100644 
meta/recipes-connectivity/nfs-ganesha/nfs-ganesha/0002-Workaround-for-erroneous-gcc-Werror.patch
  create mode 100644 
meta/recipes-connectivity/nfs-ganesha/nfs-ganesha/0003-Make-nfsv3_optype-guarded-by-_USE_NFS3.patch
  create mode 100644 
meta/recipes-connectivity/nfs-ganesha/nfs-ganesha/module.patch
  create mode 100644 meta/recipes-connectivity/nfs-ganesha/nfs-ganesha_4.0.bb
  delete mode 100644 
meta/recipes-devtools/unfs3/unfs3/0001-Add-listen-action-for-a-tcp-socket.patch
  delete mode 100644 
meta/recipes-devtools/unfs3/unfs3/0001-attr-fix-utime-for-symlink.patch
  delete mode 100644 
meta/recipes-devtools/unfs3/unfs3/0001-daemon.c-Libtirpc-porting-fixes.patch
  delete mode 100644 meta/recipes-devtools/unfs3/unfs3/alternate_rpc_ports.patch
  delete mode 100644 meta/recipes-devtools/unfs3/unfs3/fix_compile_warning.patch
  delete mode 100644 
meta/recipes-devtools/unfs3/unfs3/fix_pid_race_parent_writes_child_pid.patch
  delete mode 100644 meta/recipes-devtools/unfs3/unfs3/no-yywrap.patch
  delete mode 100644 
meta/recipes-devtools/unfs3/unfs3/relative_max_socket_path_len.patch
  delete mode 100644 meta/recipes-devtools/unfs3/unfs3/rename_fh_cache.patch
  delete mode 100644 meta/recipes-devtools/unfs3/unfs3/tcp_no_delay.patch
  delete mode 100644 
meta/recipes-devtools/unfs3/unfs3/unfs3_parallel_build.patch
  delete mode 100644 meta/recipes-devtools/unfs3/unfs3_git.bb

diff --git a/meta/lib/oeqa/utils/nfs.py b/meta/lib/oeqa/utils/nfs.py
index c1218656ce..bd947c0b88 100644
--- a/meta/lib/oeqa/utils/nfs.py
+++ b/meta/lib/oeqa/utils/nfs.py
@@ -16,7 +16,7 @@ def unfs_server(directory, logger = None):
  unfs_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE", "unfs3-native")
  if not os.path.exists(os.path.join(unfs_sysroot, "usr", "bin", "unfsd")):
  # build native tool
-bitbake("unfs3-native -c addto_recipe_sysroot")
+bitbake("nfs-ganesha-native -c addto_recipe_sysroot")

  exports = None
  cmd = None
diff --git 
a/meta/recipes-connectivity/nfs-ganesha/nfs-ganesha/0001-Fix-possible-null-pointer-dereference.patch
 
b/meta/recipes-connectivity/nfs-ganesha/nfs-ganesha/0001-Fix-possible-null-pointer-dereference.patch
new file mode 100644
index 00..b06fc8d07e
--- /dev/null
+++ 
b/meta/recipes-connectivity/nfs-ganesha/nfs-ganesha/0001-Fix-possible-null-pointer-dereference.patch
@@ -0,0 +1,40 @@
+From e70f112dc79263298baf7a1b92c19234b8f1e506 Mon Sep 17 00:00:00 2001
+From: Paulo Neves 
+Date: Tue, 23 Aug 2022 14:56:44 +0200
+Subject: [PATCH] Fix possible null pointer dereference
+
+If logging is enabled on nfs3_lookup, name is NULL initialized
+and used. Assign name to a valid location before it is used.
+
+Upstream-Status: Pending [1]
+
+[1] https://github.com/nfs-ganesha/nfs-ganesha/pull/846
+
+Change-Id: Ia1a922a1d34268a7d0b5ba97293e647ef07f8b3a
+Signed-off-by: Paulo Neves 
+
+---
+ src/Protocols/NFS/nfs3_lookup.c | 3 +--
+ 1 file changed, 1 insertion(+), 2 deletions(-)
+
+diff --git a/src/Protocols/NFS/nfs3_lookup.c b/src/Protocols/NFS/nfs3_lookup.c
+index da65769..97c9603 100644
+--- a/src/Protocols/NFS/nfs3_lookup.c
 b/s

[OE-core] [PATCH v3 2/2] nfs-ganesha: Replace unfs3 with nfs-ganesha

2022-08-28 Thread Paulo Neves

nfs-ganesha is actively maintained in contrast with unfs3. glibc tests
pass with nfs-ganesha.

Signed-off-by: Paulo Neves 
---
 meta/lib/oeqa/utils/nfs.py|   2 +-
 ...ix-possible-null-pointer-dereference.patch |  38 +
 ...-Workaround-for-erroneous-gcc-Werror.patch |  33 
 ...ke-nfsv3_optype-guarded-by-_USE_NFS3.patch |  35 
 .../nfs-ganesha/nfs-ganesha/module.patch  |  27 +++
 .../nfs-ganesha/nfs-ganesha_4.0.bb|  44 +
 ...1-Add-listen-action-for-a-tcp-socket.patch |  54 --
 .../0001-attr-fix-utime-for-symlink.patch |  85 --
 ...0001-daemon.c-Libtirpc-porting-fixes.patch |  37 
 .../unfs3/unfs3/alternate_rpc_ports.patch | 158 --
 .../unfs3/unfs3/fix_compile_warning.patch |  25 ---
 ...fix_pid_race_parent_writes_child_pid.patch |  61 ---
 .../unfs3/unfs3/no-yywrap.patch   |  14 --
 .../unfs3/relative_max_socket_path_len.patch  |  74 
 .../unfs3/unfs3/rename_fh_cache.patch |  64 ---
 .../unfs3/unfs3/tcp_no_delay.patch|  56 ---
 .../unfs3/unfs3/unfs3_parallel_build.patch|  37 
 meta/recipes-devtools/unfs3/unfs3_git.bb  |  45 -
 18 files changed, 178 insertions(+), 711 deletions(-)
 create mode 100644 
meta/recipes-connectivity/nfs-ganesha/nfs-ganesha/0001-Fix-possible-null-pointer-dereference.patch
 create mode 100644 
meta/recipes-connectivity/nfs-ganesha/nfs-ganesha/0002-Workaround-for-erroneous-gcc-Werror.patch
 create mode 100644 
meta/recipes-connectivity/nfs-ganesha/nfs-ganesha/0003-Make-nfsv3_optype-guarded-by-_USE_NFS3.patch
 create mode 100644 
meta/recipes-connectivity/nfs-ganesha/nfs-ganesha/module.patch
 create mode 100644 meta/recipes-connectivity/nfs-ganesha/nfs-ganesha_4.0.bb
 delete mode 100644 
meta/recipes-devtools/unfs3/unfs3/0001-Add-listen-action-for-a-tcp-socket.patch
 delete mode 100644 
meta/recipes-devtools/unfs3/unfs3/0001-attr-fix-utime-for-symlink.patch
 delete mode 100644 
meta/recipes-devtools/unfs3/unfs3/0001-daemon.c-Libtirpc-porting-fixes.patch
 delete mode 100644 meta/recipes-devtools/unfs3/unfs3/alternate_rpc_ports.patch
 delete mode 100644 meta/recipes-devtools/unfs3/unfs3/fix_compile_warning.patch
 delete mode 100644 
meta/recipes-devtools/unfs3/unfs3/fix_pid_race_parent_writes_child_pid.patch
 delete mode 100644 meta/recipes-devtools/unfs3/unfs3/no-yywrap.patch
 delete mode 100644 
meta/recipes-devtools/unfs3/unfs3/relative_max_socket_path_len.patch
 delete mode 100644 meta/recipes-devtools/unfs3/unfs3/rename_fh_cache.patch
 delete mode 100644 meta/recipes-devtools/unfs3/unfs3/tcp_no_delay.patch
 delete mode 100644 meta/recipes-devtools/unfs3/unfs3/unfs3_parallel_build.patch
 delete mode 100644 meta/recipes-devtools/unfs3/unfs3_git.bb

diff --git a/meta/lib/oeqa/utils/nfs.py b/meta/lib/oeqa/utils/nfs.py
index c1218656ce..bd947c0b88 100644
--- a/meta/lib/oeqa/utils/nfs.py
+++ b/meta/lib/oeqa/utils/nfs.py
@@ -16,7 +16,7 @@ def unfs_server(directory, logger = None):
 unfs_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE", "unfs3-native")
 if not os.path.exists(os.path.join(unfs_sysroot, "usr", "bin", "unfsd")):
 # build native tool
-bitbake("unfs3-native -c addto_recipe_sysroot")
+bitbake("nfs-ganesha-native -c addto_recipe_sysroot")
 
 exports = None
 cmd = None
diff --git 
a/meta/recipes-connectivity/nfs-ganesha/nfs-ganesha/0001-Fix-possible-null-pointer-dereference.patch
 
b/meta/recipes-connectivity/nfs-ganesha/nfs-ganesha/0001-Fix-possible-null-pointer-dereference.patch
new file mode 100644
index 00..6c98a8a59e
--- /dev/null
+++ 
b/meta/recipes-connectivity/nfs-ganesha/nfs-ganesha/0001-Fix-possible-null-pointer-dereference.patch
@@ -0,0 +1,38 @@
+From e70f112dc79263298baf7a1b92c19234b8f1e506 Mon Sep 17 00:00:00 2001
+From: Paulo Neves 
+Date: Tue, 23 Aug 2022 14:56:44 +0200
+Subject: [PATCH] Fix possible null pointer dereference
+
+If logging is enabled on nfs3_lookup, name is NULL initialized
+and used. Assign name to a valid location before it is used.
+
+Upstream-Status: Submitted 
[https://github.com/nfs-ganesha/nfs-ganesha/pull/846]
+
+Change-Id: Ia1a922a1d34268a7d0b5ba97293e647ef07f8b3a
+Signed-off-by: Paulo Neves 
+
+---
+ src/Protocols/NFS/nfs3_lookup.c | 3 +--
+ 1 file changed, 1 insertion(+), 2 deletions(-)
+
+diff --git a/src/Protocols/NFS/nfs3_lookup.c b/src/Protocols/NFS/nfs3_lookup.c
+index da65769..97c9603 100644
+--- a/src/Protocols/NFS/nfs3_lookup.c
 b/src/Protocols/NFS/nfs3_lookup.c
+@@ -74,6 +74,7 @@ int nfs3_lookup(nfs_arg_t *arg, struct svc_req *req, 
nfs_res_t *res)
+*/
+   fsal_prepare_attrs(&attrs, ATTRS_NFS3 | ATTR_RDATTR_ERR);
+ 
++  name = arg->arg_lookup3.what.name;
+   LogNFS3_Operation(COMPONENT_NFSPROTO, req, &arg->arg_lookup3.what.dir,
+ " name: %s", name);
+ 
+@@ -89,8 +90,6 @@

[OE-core] [PATCH v3 1/2] ntirpc: Add ntirpc recipe

2022-08-28 Thread Paulo Neves

ntirpc is an actively maintained rpc implementation. We cannot completely
remove libtirpc due to arising circular dependencies where ntirpc
depends on libnsl2, which in turn depends on an RPC implementation.

libtirpc and ntirpc are not mutually exclusive.

Signed-off-by: Paulo Neves 
---
 meta/recipes-extended/ntirpc/ntirpc_4.0.bb| 36 +++
 meta/recipes-extended/quota/quota_4.06.bb |  2 +-
 .../recipes-extended/rpcbind/rpcbind_1.2.6.bb |  2 +-
 .../watchdog/watchdog_5.16.bb |  2 +-
 .../xorg-xserver/xserver-xorg.inc |  2 +-
 .../recipes-support/liburcu/liburcu_0.13.1.bb |  1 +
 6 files changed, 41 insertions(+), 4 deletions(-)
 create mode 100644 meta/recipes-extended/ntirpc/ntirpc_4.0.bb

diff --git a/meta/recipes-extended/ntirpc/ntirpc_4.0.bb 
b/meta/recipes-extended/ntirpc/ntirpc_4.0.bb
new file mode 100644
index 00..eff9d36069
--- /dev/null
+++ b/meta/recipes-extended/ntirpc/ntirpc_4.0.bb
@@ -0,0 +1,36 @@
+SUMMARY = "Transport-Independent RPC library (nfs-ganesha fork)"
+DESCRIPTION = "ntirpc is a port of Suns Transport-Independent RPC library to 
Linux"
+SECTION = "libs/network"
+HOMEPAGE = "https://github.com/nfs-ganesha/ntirpc";
+BUGTRACKER = "https://github.com/nfs-ganesha/ntirpc/issues";
+LICENSE = "BSD-3-Clause"
+LIC_FILES_CHKSUM = "file://COPYING;md5=f835cce8852481e4b2bbbdd23b5e47f3 \
+
file://src/netname.c;beginline=1;endline=27;md5=f8a8cd2cb25ac5aa16767364fb0e3c24"
+
+SRC_URI = "https://download.nfs-ganesha.org/4/${PV}/ntirpc-${PV}.tar.gz";
+SRC_URI[md5sum] = "17b0baada54936dcde80eba27bb6d88d"
+SRC_URI[sha256sum] = 
"6a7921cd48d02077b394a02be42bf4f9b4678d4ebfd86a7ae4385bbcdce55462"
+
+DEPENDS = "liburcu libnsl2"
+
+inherit cmake pkgconfig
+
+PACKAGECONFIG[gss] = "-DUSE_GSS=true,-DUSE_GSS=false,krb5"
+PACKAGECONFIG[rdma] = "-DUSE_RPC_RDMA=true,-DUSE_RPC_RDMA=false,rdma-core"
+PACKAGECONFIG[winbind] = "-D_MSPAC_SUPPORT=true,-D_MSPAC_SUPPORT=false,samba"
+PACKAGECONFIG[profile] = "-DUSE_PROFILE=true,-DUSE_PROFILE=false,gperftools"
+PACKAGECONFIG[lttng] = "-DUSE_LTTNG=true,-DUSE_LTTNG=false,lttng-ust 
lttng-tools"
+
+# Sort out the brain-dead cmake file
+# (https://github.com/nfs-ganesha/ntirpc/issues/150)
+do_install:append() {
+if [ "${prefix}/lib64" != "${libdir}" -a -d ${D}${prefix}/lib64 ]; then
+mv ${D}${prefix}/lib64 ${D}${libdir}
+fi
+}
+
+BBCLASSEXTEND = "native nativesdk"
+EXTRA_OECMAKE:armv5 += '-DSYSTEM_LIBRARIES="-latomic"'
+EXTRA_OECMAKE:riscv32 += '-DSYSTEM_LIBRARIES="-latomic"'
+EXTRA_OECMAKE:riscv64 += '-DSYSTEM_LIBRARIES="-latomic"'
+EXTRA_OECMAKE:powerpc += '-DSYSTEM_LIBRARIES="-latomic"'
diff --git a/meta/recipes-extended/quota/quota_4.06.bb 
b/meta/recipes-extended/quota/quota_4.06.bb
index 0535d14c20..304477d72b 100644
--- a/meta/recipes-extended/quota/quota_4.06.bb
+++ b/meta/recipes-extended/quota/quota_4.06.bb
@@ -30,6 +30,6 @@ PACKAGECONFIG ??= "tcp-wrappers rpc bsd"
 PACKAGECONFIG:libc-musl = "tcp-wrappers rpc"
 
 PACKAGECONFIG[tcp-wrappers] = "--enable-libwrap,--disable-libwrap,tcp-wrappers"
-PACKAGECONFIG[rpc] = "--enable-rpc,--disable-rpc,libtirpc"
+PACKAGECONFIG[rpc] = "--enable-rpc,--disable-rpc,ntirpc"
 PACKAGECONFIG[bsd] = "--enable-bsd_behaviour=yes,--enable-bsd_behaviour=no,"
 PACKAGECONFIG[ldapmail] = "--enable-ldapmail,--disable-ldapmail,openldap"
diff --git a/meta/recipes-extended/rpcbind/rpcbind_1.2.6.bb 
b/meta/recipes-extended/rpcbind/rpcbind_1.2.6.bb
index dd89726afc..e30a23909c 100644
--- a/meta/recipes-extended/rpcbind/rpcbind_1.2.6.bb
+++ b/meta/recipes-extended/rpcbind/rpcbind_1.2.6.bb
@@ -4,7 +4,7 @@ DESCRIPTION = "The rpcbind utility is a server that converts 
RPC \
 SECTION = "console/network"
 HOMEPAGE = "http://sourceforge.net/projects/rpcbind/";
 BUGTRACKER = "http://sourceforge.net/tracker/?group_id=201237&atid=976751";
-DEPENDS = "libtirpc quota"
+DEPENDS = "ntirpc quota"
 
 LICENSE = "BSD-3-Clause"
 LIC_FILES_CHKSUM = "file://COPYING;md5=b46486e4c4a416602693a711bb5bfa39 \
diff --git a/meta/recipes-extended/watchdog/watchdog_5.16.bb 
b/meta/recipes-extended/watchdog/watchdog_5.16.bb
index 26fcc10487..5f5fd4dc87 100644
--- a/meta/recipes-extended/watchdog/watchdog_5.16.bb
+++ b/meta/recipes-extended/watchdog/watchdog_5.16.bb
@@ -28,7 +28,7 @@ UPSTREAM_CHECK_REGEX = "/watchdog/(?P(\d+[\.\-_]*)+)/"
 
 inherit autotools update-rc.d systemd pkgconfig
 
-DEPENDS += "libtirpc"
+DEPENDS += "ntirpc"
 CFLAGS += "-I${STAGING_INCDIR}/tirpc"
 LDFLAGS += "-ltirpc"
 
diff --git a/meta/recipes-graphics/xorg-xserver/xserver-x

Re: [OE-core] [PATCH 1/2] ntirpc: Add ntirpc recipe

2022-08-28 Thread Paulo Neves


Hello Alexander,

Thank you for your review

I updated the commit message for v2 to contain the following text:

We cannot completely  remove libtirpc due to arising circular 
dependencies where ntirpc depends on libnsl2, which  in turn depends on 
an RPC implementation.


Paulo Neves

On 8/27/22 20:07, Alexander Kanavin wrote:

Can you please explain where the circular dependencies are?

Alex

On Sat, 27 Aug 2022 at 19:46, Paulo Neves  wrote:

ntirpc is an actively maintained rpc implementation. We cannot completely
remove libtirpc due to arising circular dependencies. libtirpc and ntirpc
are not mutually exclusive.

Signed-off-by: Paulo Neves 
---
  meta/recipes-extended/ntirpc/ntirpc_4.0.bb| 30 +++
  meta/recipes-extended/quota/quota_4.06.bb |  2 +-
  .../recipes-extended/rpcbind/rpcbind_1.2.6.bb |  2 +-
  .../watchdog/watchdog_5.16.bb |  2 +-
  .../xorg-xserver/xserver-xorg.inc |  2 +-
  .../recipes-support/liburcu/liburcu_0.13.1.bb |  1 +
  6 files changed, 35 insertions(+), 4 deletions(-)
  create mode 100644 meta/recipes-extended/ntirpc/ntirpc_4.0.bb

diff --git a/meta/recipes-extended/ntirpc/ntirpc_4.0.bb 
b/meta/recipes-extended/ntirpc/ntirpc_4.0.bb
new file mode 100644
index 00..edebcb8ca9
--- /dev/null
+++ b/meta/recipes-extended/ntirpc/ntirpc_4.0.bb
@@ -0,0 +1,30 @@
+LICENSE = "BSD"
+LIC_FILES_CHKSUM = "file://COPYING;md5=f835cce8852481e4b2bbbdd23b5e47f3"
+
+SRC_URI = "https://download.nfs-ganesha.org/4/${PV}/ntirpc-${PV}.tar.gz";
+SRC_URI[md5sum] = "17b0baada54936dcde80eba27bb6d88d"
+SRC_URI[sha256sum] = 
"6a7921cd48d02077b394a02be42bf4f9b4678d4ebfd86a7ae4385bbcdce55462"
+
+DEPENDS = "liburcu libnsl2"
+
+inherit cmake pkgconfig
+
+PACKAGECONFIG[gss] = "-DUSE_GSS=true,-DUSE_GSS=false,krb5"
+PACKAGECONFIG[rdma] = "-DUSE_RPC_RDMA=true,-DUSE_RPC_RDMA=false,rdma-core"
+PACKAGECONFIG[winbind] = "-D_MSPAC_SUPPORT=true,-D_MSPAC_SUPPORT=false,samba"
+PACKAGECONFIG[profile] = "-DUSE_PROFILE=true,-DUSE_PROFILE=false,gperftools"
+PACKAGECONFIG[lttng] = "-DUSE_LTTNG=true,-DUSE_LTTNG=false,lttng-ust 
lttng-tools"
+
+# Sort out the brain-dead cmake file
+# (https://github.com/nfs-ganesha/ntirpc/issues/150)
+do_install:append() {
+if [ "${prefix}/lib64" != "${libdir}" -a -d ${D}${prefix}/lib64 ]; then
+mv ${D}${prefix}/lib64 ${D}${libdir}
+fi
+}
+
+BBCLASSEXTEND = "native nativesdk"
+EXTRA_OECMAKE:armv5 += '-DSYSTEM_LIBRARIES="-latomic"'
+EXTRA_OECMAKE:riscv32 += '-DSYSTEM_LIBRARIES="-latomic"'
+EXTRA_OECMAKE:riscv64 += '-DSYSTEM_LIBRARIES="-latomic"'
+EXTRA_OECMAKE:powerpc += '-DSYSTEM_LIBRARIES="-latomic"'
diff --git a/meta/recipes-extended/quota/quota_4.06.bb 
b/meta/recipes-extended/quota/quota_4.06.bb
index 0535d14c20..304477d72b 100644
--- a/meta/recipes-extended/quota/quota_4.06.bb
+++ b/meta/recipes-extended/quota/quota_4.06.bb
@@ -30,6 +30,6 @@ PACKAGECONFIG ??= "tcp-wrappers rpc bsd"
  PACKAGECONFIG:libc-musl = "tcp-wrappers rpc"

  PACKAGECONFIG[tcp-wrappers] = 
"--enable-libwrap,--disable-libwrap,tcp-wrappers"
-PACKAGECONFIG[rpc] = "--enable-rpc,--disable-rpc,libtirpc"
+PACKAGECONFIG[rpc] = "--enable-rpc,--disable-rpc,ntirpc"
  PACKAGECONFIG[bsd] = "--enable-bsd_behaviour=yes,--enable-bsd_behaviour=no,"
  PACKAGECONFIG[ldapmail] = "--enable-ldapmail,--disable-ldapmail,openldap"
diff --git a/meta/recipes-extended/rpcbind/rpcbind_1.2.6.bb 
b/meta/recipes-extended/rpcbind/rpcbind_1.2.6.bb
index dd89726afc..e30a23909c 100644
--- a/meta/recipes-extended/rpcbind/rpcbind_1.2.6.bb
+++ b/meta/recipes-extended/rpcbind/rpcbind_1.2.6.bb
@@ -4,7 +4,7 @@ DESCRIPTION = "The rpcbind utility is a server that converts 
RPC \
  SECTION = "console/network"
  HOMEPAGE = "http://sourceforge.net/projects/rpcbind/";
  BUGTRACKER = "http://sourceforge.net/tracker/?group_id=201237&atid=976751";
-DEPENDS = "libtirpc quota"
+DEPENDS = "ntirpc quota"

  LICENSE = "BSD-3-Clause"
  LIC_FILES_CHKSUM = "file://COPYING;md5=b46486e4c4a416602693a711bb5bfa39 \
diff --git a/meta/recipes-extended/watchdog/watchdog_5.16.bb 
b/meta/recipes-extended/watchdog/watchdog_5.16.bb
index 26fcc10487..5f5fd4dc87 100644
--- a/meta/recipes-extended/watchdog/watchdog_5.16.bb
+++ b/meta/recipes-extended/watchdog/watchdog_5.16.bb
@@ -28,7 +28,7 @@ UPSTREAM_CHECK_REGEX = "/watchdog/(?P(\d+[\.\-_]*)+)/"

  inherit autotools update-rc.d systemd pkgconfig

-DEPENDS += "libtirpc"
+DEPENDS += "ntirpc"
  CFLAGS += "-I${STAGING_INCDIR}/tirpc"
  LDFLAGS += "-ltirpc"

diff --git a/meta/recipes-graphics/xorg-xserver/xserver-xorg.inc 
b/meta/recipes-graphics/xorg-xserver/xserv

[OE-core] [PATCH v2 2/2] nfs-ganesha: Replace unfs3 with nfs-ganesha

2022-08-28 Thread Paulo Neves

nfs-ganesha is actively maintained in contrast with unfs3. glibc tests
pass with nfs-ganesha.

Signed-off-by: Paulo Neves 
---
 meta/lib/oeqa/utils/nfs.py|   2 +-
 ...ix-possible-null-pointer-dereference.patch |  38 +
 ...-Workaround-for-erroneous-gcc-Werror.patch |  33 
 ...ke-nfsv3_optype-guarded-by-_USE_NFS3.patch |  35 
 .../nfs-ganesha/nfs-ganesha/module.patch  |  27 +++
 .../nfs-ganesha/nfs-ganesha_4.0.bb|  41 +
 ...1-Add-listen-action-for-a-tcp-socket.patch |  54 --
 .../0001-attr-fix-utime-for-symlink.patch |  85 --
 ...0001-daemon.c-Libtirpc-porting-fixes.patch |  37 
 .../unfs3/unfs3/alternate_rpc_ports.patch | 158 --
 .../unfs3/unfs3/fix_compile_warning.patch |  25 ---
 ...fix_pid_race_parent_writes_child_pid.patch |  61 ---
 .../unfs3/unfs3/no-yywrap.patch   |  14 --
 .../unfs3/relative_max_socket_path_len.patch  |  74 
 .../unfs3/unfs3/rename_fh_cache.patch |  64 ---
 .../unfs3/unfs3/tcp_no_delay.patch|  56 ---
 .../unfs3/unfs3/unfs3_parallel_build.patch|  37 
 meta/recipes-devtools/unfs3/unfs3_git.bb  |  45 -
 18 files changed, 175 insertions(+), 711 deletions(-)
 create mode 100644 
meta/recipes-connectivity/nfs-ganesha/nfs-ganesha/0001-Fix-possible-null-pointer-dereference.patch
 create mode 100644 
meta/recipes-connectivity/nfs-ganesha/nfs-ganesha/0002-Workaround-for-erroneous-gcc-Werror.patch
 create mode 100644 
meta/recipes-connectivity/nfs-ganesha/nfs-ganesha/0003-Make-nfsv3_optype-guarded-by-_USE_NFS3.patch
 create mode 100644 
meta/recipes-connectivity/nfs-ganesha/nfs-ganesha/module.patch
 create mode 100644 meta/recipes-connectivity/nfs-ganesha/nfs-ganesha_4.0.bb
 delete mode 100644 
meta/recipes-devtools/unfs3/unfs3/0001-Add-listen-action-for-a-tcp-socket.patch
 delete mode 100644 
meta/recipes-devtools/unfs3/unfs3/0001-attr-fix-utime-for-symlink.patch
 delete mode 100644 
meta/recipes-devtools/unfs3/unfs3/0001-daemon.c-Libtirpc-porting-fixes.patch
 delete mode 100644 meta/recipes-devtools/unfs3/unfs3/alternate_rpc_ports.patch
 delete mode 100644 meta/recipes-devtools/unfs3/unfs3/fix_compile_warning.patch
 delete mode 100644 
meta/recipes-devtools/unfs3/unfs3/fix_pid_race_parent_writes_child_pid.patch
 delete mode 100644 meta/recipes-devtools/unfs3/unfs3/no-yywrap.patch
 delete mode 100644 
meta/recipes-devtools/unfs3/unfs3/relative_max_socket_path_len.patch
 delete mode 100644 meta/recipes-devtools/unfs3/unfs3/rename_fh_cache.patch
 delete mode 100644 meta/recipes-devtools/unfs3/unfs3/tcp_no_delay.patch
 delete mode 100644 meta/recipes-devtools/unfs3/unfs3/unfs3_parallel_build.patch
 delete mode 100644 meta/recipes-devtools/unfs3/unfs3_git.bb

diff --git a/meta/lib/oeqa/utils/nfs.py b/meta/lib/oeqa/utils/nfs.py
index c1218656ce..bd947c0b88 100644
--- a/meta/lib/oeqa/utils/nfs.py
+++ b/meta/lib/oeqa/utils/nfs.py
@@ -16,7 +16,7 @@ def unfs_server(directory, logger = None):
 unfs_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE", "unfs3-native")
 if not os.path.exists(os.path.join(unfs_sysroot, "usr", "bin", "unfsd")):
 # build native tool
-bitbake("unfs3-native -c addto_recipe_sysroot")
+bitbake("nfs-ganesha-native -c addto_recipe_sysroot")
 
 exports = None
 cmd = None
diff --git 
a/meta/recipes-connectivity/nfs-ganesha/nfs-ganesha/0001-Fix-possible-null-pointer-dereference.patch
 
b/meta/recipes-connectivity/nfs-ganesha/nfs-ganesha/0001-Fix-possible-null-pointer-dereference.patch
new file mode 100644
index 00..6c98a8a59e
--- /dev/null
+++ 
b/meta/recipes-connectivity/nfs-ganesha/nfs-ganesha/0001-Fix-possible-null-pointer-dereference.patch
@@ -0,0 +1,38 @@
+From e70f112dc79263298baf7a1b92c19234b8f1e506 Mon Sep 17 00:00:00 2001
+From: Paulo Neves 
+Date: Tue, 23 Aug 2022 14:56:44 +0200
+Subject: [PATCH] Fix possible null pointer dereference
+
+If logging is enabled on nfs3_lookup, name is NULL initialized
+and used. Assign name to a valid location before it is used.
+
+Upstream-Status: Submitted 
[https://github.com/nfs-ganesha/nfs-ganesha/pull/846]
+
+Change-Id: Ia1a922a1d34268a7d0b5ba97293e647ef07f8b3a
+Signed-off-by: Paulo Neves 
+
+---
+ src/Protocols/NFS/nfs3_lookup.c | 3 +--
+ 1 file changed, 1 insertion(+), 2 deletions(-)
+
+diff --git a/src/Protocols/NFS/nfs3_lookup.c b/src/Protocols/NFS/nfs3_lookup.c
+index da65769..97c9603 100644
+--- a/src/Protocols/NFS/nfs3_lookup.c
 b/src/Protocols/NFS/nfs3_lookup.c
+@@ -74,6 +74,7 @@ int nfs3_lookup(nfs_arg_t *arg, struct svc_req *req, 
nfs_res_t *res)
+*/
+   fsal_prepare_attrs(&attrs, ATTRS_NFS3 | ATTR_RDATTR_ERR);
+ 
++  name = arg->arg_lookup3.what.name;
+   LogNFS3_Operation(COMPONENT_NFSPROTO, req, &arg->arg_lookup3.what.dir,
+ " name: %s", name);
+ 
+@@ -89,8 +90,6 @@

[OE-core] [PATCH v2 1/2] ntirpc: Add ntirpc recipe

2022-08-28 Thread Paulo Neves

ntirpc is an actively maintained rpc implementation. We cannot completely
remove libtirpc due to arising circular dependencies where ntirpc
depends on libnsl2, which in turn depends on an RPC implementation.

libtirpc and ntirpc are not mutually exclusive.

Signed-off-by: Paulo Neves 
---
 meta/recipes-extended/ntirpc/ntirpc_4.0.bb| 36 +++
 meta/recipes-extended/quota/quota_4.06.bb |  2 +-
 .../recipes-extended/rpcbind/rpcbind_1.2.6.bb |  2 +-
 .../watchdog/watchdog_5.16.bb |  2 +-
 .../xorg-xserver/xserver-xorg.inc |  2 +-
 .../recipes-support/liburcu/liburcu_0.13.1.bb |  1 +
 6 files changed, 41 insertions(+), 4 deletions(-)
 create mode 100644 meta/recipes-extended/ntirpc/ntirpc_4.0.bb

diff --git a/meta/recipes-extended/ntirpc/ntirpc_4.0.bb 
b/meta/recipes-extended/ntirpc/ntirpc_4.0.bb
new file mode 100644
index 00..eff9d36069
--- /dev/null
+++ b/meta/recipes-extended/ntirpc/ntirpc_4.0.bb
@@ -0,0 +1,36 @@
+SUMMARY = "Transport-Independent RPC library (nfs-ganesha fork)"
+DESCRIPTION = "ntirpc is a port of Suns Transport-Independent RPC library to 
Linux"
+SECTION = "libs/network"
+HOMEPAGE = "https://github.com/nfs-ganesha/ntirpc";
+BUGTRACKER = "https://github.com/nfs-ganesha/ntirpc/issues";
+LICENSE = "BSD-3-Clause"
+LIC_FILES_CHKSUM = "file://COPYING;md5=f835cce8852481e4b2bbbdd23b5e47f3 \
+
file://src/netname.c;beginline=1;endline=27;md5=f8a8cd2cb25ac5aa16767364fb0e3c24"
+
+SRC_URI = "https://download.nfs-ganesha.org/4/${PV}/ntirpc-${PV}.tar.gz";
+SRC_URI[md5sum] = "17b0baada54936dcde80eba27bb6d88d"
+SRC_URI[sha256sum] = 
"6a7921cd48d02077b394a02be42bf4f9b4678d4ebfd86a7ae4385bbcdce55462"
+
+DEPENDS = "liburcu libnsl2"
+
+inherit cmake pkgconfig
+
+PACKAGECONFIG[gss] = "-DUSE_GSS=true,-DUSE_GSS=false,krb5"
+PACKAGECONFIG[rdma] = "-DUSE_RPC_RDMA=true,-DUSE_RPC_RDMA=false,rdma-core"
+PACKAGECONFIG[winbind] = "-D_MSPAC_SUPPORT=true,-D_MSPAC_SUPPORT=false,samba"
+PACKAGECONFIG[profile] = "-DUSE_PROFILE=true,-DUSE_PROFILE=false,gperftools"
+PACKAGECONFIG[lttng] = "-DUSE_LTTNG=true,-DUSE_LTTNG=false,lttng-ust 
lttng-tools"
+
+# Sort out the brain-dead cmake file
+# (https://github.com/nfs-ganesha/ntirpc/issues/150)
+do_install:append() {
+if [ "${prefix}/lib64" != "${libdir}" -a -d ${D}${prefix}/lib64 ]; then
+mv ${D}${prefix}/lib64 ${D}${libdir}
+fi
+}
+
+BBCLASSEXTEND = "native nativesdk"
+EXTRA_OECMAKE:armv5 += '-DSYSTEM_LIBRARIES="-latomic"'
+EXTRA_OECMAKE:riscv32 += '-DSYSTEM_LIBRARIES="-latomic"'
+EXTRA_OECMAKE:riscv64 += '-DSYSTEM_LIBRARIES="-latomic"'
+EXTRA_OECMAKE:powerpc += '-DSYSTEM_LIBRARIES="-latomic"'
diff --git a/meta/recipes-extended/quota/quota_4.06.bb 
b/meta/recipes-extended/quota/quota_4.06.bb
index 0535d14c20..304477d72b 100644
--- a/meta/recipes-extended/quota/quota_4.06.bb
+++ b/meta/recipes-extended/quota/quota_4.06.bb
@@ -30,6 +30,6 @@ PACKAGECONFIG ??= "tcp-wrappers rpc bsd"
 PACKAGECONFIG:libc-musl = "tcp-wrappers rpc"
 
 PACKAGECONFIG[tcp-wrappers] = "--enable-libwrap,--disable-libwrap,tcp-wrappers"
-PACKAGECONFIG[rpc] = "--enable-rpc,--disable-rpc,libtirpc"
+PACKAGECONFIG[rpc] = "--enable-rpc,--disable-rpc,ntirpc"
 PACKAGECONFIG[bsd] = "--enable-bsd_behaviour=yes,--enable-bsd_behaviour=no,"
 PACKAGECONFIG[ldapmail] = "--enable-ldapmail,--disable-ldapmail,openldap"
diff --git a/meta/recipes-extended/rpcbind/rpcbind_1.2.6.bb 
b/meta/recipes-extended/rpcbind/rpcbind_1.2.6.bb
index dd89726afc..e30a23909c 100644
--- a/meta/recipes-extended/rpcbind/rpcbind_1.2.6.bb
+++ b/meta/recipes-extended/rpcbind/rpcbind_1.2.6.bb
@@ -4,7 +4,7 @@ DESCRIPTION = "The rpcbind utility is a server that converts 
RPC \
 SECTION = "console/network"
 HOMEPAGE = "http://sourceforge.net/projects/rpcbind/";
 BUGTRACKER = "http://sourceforge.net/tracker/?group_id=201237&atid=976751";
-DEPENDS = "libtirpc quota"
+DEPENDS = "ntirpc quota"
 
 LICENSE = "BSD-3-Clause"
 LIC_FILES_CHKSUM = "file://COPYING;md5=b46486e4c4a416602693a711bb5bfa39 \
diff --git a/meta/recipes-extended/watchdog/watchdog_5.16.bb 
b/meta/recipes-extended/watchdog/watchdog_5.16.bb
index 26fcc10487..5f5fd4dc87 100644
--- a/meta/recipes-extended/watchdog/watchdog_5.16.bb
+++ b/meta/recipes-extended/watchdog/watchdog_5.16.bb
@@ -28,7 +28,7 @@ UPSTREAM_CHECK_REGEX = "/watchdog/(?P(\d+[\.\-_]*)+)/"
 
 inherit autotools update-rc.d systemd pkgconfig
 
-DEPENDS += "libtirpc"
+DEPENDS += "ntirpc"
 CFLAGS += "-I${STAGING_INCDIR}/tirpc"
 LDFLAGS += "-ltirpc"
 
diff --git a/meta/recipes-graphics/xorg-xserver/xserver-x

[OE-core] [PATCH 2/2] nfs-ganesha: Replace unfs3 with nfs-ganesha

2022-08-27 Thread Paulo Neves

nfs-ganesha is actively maintained in contrast with unfs3. glibc tests
pass with nfs-ganesha.

Signed-off-by: Paulo Neves 
---
 meta/lib/oeqa/utils/nfs.py|   2 +-
 ...ix-possible-null-pointer-dereference.patch |  40 +
 ...-Workaround-for-erroneous-gcc-Werror.patch |  33 
 ...ke-nfsv3_optype-guarded-by-_USE_NFS3.patch |  37 
 .../nfs-ganesha/nfs-ganesha/module.patch  |  27 +++
 .../nfs-ganesha/nfs-ganesha_4.0.bb|  42 +
 ...1-Add-listen-action-for-a-tcp-socket.patch |  54 --
 .../0001-attr-fix-utime-for-symlink.patch |  85 --
 ...0001-daemon.c-Libtirpc-porting-fixes.patch |  37 
 .../unfs3/unfs3/alternate_rpc_ports.patch | 158 --
 .../unfs3/unfs3/fix_compile_warning.patch |  25 ---
 ...fix_pid_race_parent_writes_child_pid.patch |  61 ---
 .../unfs3/unfs3/no-yywrap.patch   |  14 --
 .../unfs3/relative_max_socket_path_len.patch  |  74 
 .../unfs3/unfs3/rename_fh_cache.patch |  64 ---
 .../unfs3/unfs3/tcp_no_delay.patch|  56 ---
 .../unfs3/unfs3/unfs3_parallel_build.patch|  37 
 meta/recipes-devtools/unfs3/unfs3_git.bb  |  45 -
 18 files changed, 180 insertions(+), 711 deletions(-)
 create mode 100644 
meta/recipes-connectivity/nfs-ganesha/nfs-ganesha/0001-Fix-possible-null-pointer-dereference.patch
 create mode 100644 
meta/recipes-connectivity/nfs-ganesha/nfs-ganesha/0002-Workaround-for-erroneous-gcc-Werror.patch
 create mode 100644 
meta/recipes-connectivity/nfs-ganesha/nfs-ganesha/0003-Make-nfsv3_optype-guarded-by-_USE_NFS3.patch
 create mode 100644 
meta/recipes-connectivity/nfs-ganesha/nfs-ganesha/module.patch
 create mode 100644 meta/recipes-connectivity/nfs-ganesha/nfs-ganesha_4.0.bb
 delete mode 100644 
meta/recipes-devtools/unfs3/unfs3/0001-Add-listen-action-for-a-tcp-socket.patch
 delete mode 100644 
meta/recipes-devtools/unfs3/unfs3/0001-attr-fix-utime-for-symlink.patch
 delete mode 100644 
meta/recipes-devtools/unfs3/unfs3/0001-daemon.c-Libtirpc-porting-fixes.patch
 delete mode 100644 meta/recipes-devtools/unfs3/unfs3/alternate_rpc_ports.patch
 delete mode 100644 meta/recipes-devtools/unfs3/unfs3/fix_compile_warning.patch
 delete mode 100644 
meta/recipes-devtools/unfs3/unfs3/fix_pid_race_parent_writes_child_pid.patch
 delete mode 100644 meta/recipes-devtools/unfs3/unfs3/no-yywrap.patch
 delete mode 100644 
meta/recipes-devtools/unfs3/unfs3/relative_max_socket_path_len.patch
 delete mode 100644 meta/recipes-devtools/unfs3/unfs3/rename_fh_cache.patch
 delete mode 100644 meta/recipes-devtools/unfs3/unfs3/tcp_no_delay.patch
 delete mode 100644 meta/recipes-devtools/unfs3/unfs3/unfs3_parallel_build.patch
 delete mode 100644 meta/recipes-devtools/unfs3/unfs3_git.bb

diff --git a/meta/lib/oeqa/utils/nfs.py b/meta/lib/oeqa/utils/nfs.py
index c1218656ce..bd947c0b88 100644
--- a/meta/lib/oeqa/utils/nfs.py
+++ b/meta/lib/oeqa/utils/nfs.py
@@ -16,7 +16,7 @@ def unfs_server(directory, logger = None):
 unfs_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE", "unfs3-native")
 if not os.path.exists(os.path.join(unfs_sysroot, "usr", "bin", "unfsd")):
 # build native tool
-bitbake("unfs3-native -c addto_recipe_sysroot")
+bitbake("nfs-ganesha-native -c addto_recipe_sysroot")
 
 exports = None
 cmd = None
diff --git 
a/meta/recipes-connectivity/nfs-ganesha/nfs-ganesha/0001-Fix-possible-null-pointer-dereference.patch
 
b/meta/recipes-connectivity/nfs-ganesha/nfs-ganesha/0001-Fix-possible-null-pointer-dereference.patch
new file mode 100644
index 00..b06fc8d07e
--- /dev/null
+++ 
b/meta/recipes-connectivity/nfs-ganesha/nfs-ganesha/0001-Fix-possible-null-pointer-dereference.patch
@@ -0,0 +1,40 @@
+From e70f112dc79263298baf7a1b92c19234b8f1e506 Mon Sep 17 00:00:00 2001
+From: Paulo Neves 
+Date: Tue, 23 Aug 2022 14:56:44 +0200
+Subject: [PATCH] Fix possible null pointer dereference
+
+If logging is enabled on nfs3_lookup, name is NULL initialized
+and used. Assign name to a valid location before it is used.
+
+Upstream-Status: Pending [1]
+
+[1] https://github.com/nfs-ganesha/nfs-ganesha/pull/846
+
+Change-Id: Ia1a922a1d34268a7d0b5ba97293e647ef07f8b3a
+Signed-off-by: Paulo Neves 
+
+---
+ src/Protocols/NFS/nfs3_lookup.c | 3 +--
+ 1 file changed, 1 insertion(+), 2 deletions(-)
+
+diff --git a/src/Protocols/NFS/nfs3_lookup.c b/src/Protocols/NFS/nfs3_lookup.c
+index da65769..97c9603 100644
+--- a/src/Protocols/NFS/nfs3_lookup.c
 b/src/Protocols/NFS/nfs3_lookup.c
+@@ -74,6 +74,7 @@ int nfs3_lookup(nfs_arg_t *arg, struct svc_req *req, 
nfs_res_t *res)
+*/
+   fsal_prepare_attrs(&attrs, ATTRS_NFS3 | ATTR_RDATTR_ERR);
+ 
++  name = arg->arg_lookup3.what.name;
+   LogNFS3_Operation(COMPONENT_NFSPROTO, req, &arg->arg_lookup3.what.dir,
+ " name: %s", name);
+ 
+@@ -89,8 +90,6 @@

[OE-core] [PATCH 1/2] ntirpc: Add ntirpc recipe

2022-08-27 Thread Paulo Neves

ntirpc is an actively maintained rpc implementation. We cannot completely
remove libtirpc due to arising circular dependencies. libtirpc and ntirpc
are not mutually exclusive.

Signed-off-by: Paulo Neves 
---
 meta/recipes-extended/ntirpc/ntirpc_4.0.bb| 30 +++
 meta/recipes-extended/quota/quota_4.06.bb |  2 +-
 .../recipes-extended/rpcbind/rpcbind_1.2.6.bb |  2 +-
 .../watchdog/watchdog_5.16.bb |  2 +-
 .../xorg-xserver/xserver-xorg.inc |  2 +-
 .../recipes-support/liburcu/liburcu_0.13.1.bb |  1 +
 6 files changed, 35 insertions(+), 4 deletions(-)
 create mode 100644 meta/recipes-extended/ntirpc/ntirpc_4.0.bb

diff --git a/meta/recipes-extended/ntirpc/ntirpc_4.0.bb 
b/meta/recipes-extended/ntirpc/ntirpc_4.0.bb
new file mode 100644
index 00..edebcb8ca9
--- /dev/null
+++ b/meta/recipes-extended/ntirpc/ntirpc_4.0.bb
@@ -0,0 +1,30 @@
+LICENSE = "BSD"
+LIC_FILES_CHKSUM = "file://COPYING;md5=f835cce8852481e4b2bbbdd23b5e47f3"
+
+SRC_URI = "https://download.nfs-ganesha.org/4/${PV}/ntirpc-${PV}.tar.gz";
+SRC_URI[md5sum] = "17b0baada54936dcde80eba27bb6d88d"
+SRC_URI[sha256sum] = 
"6a7921cd48d02077b394a02be42bf4f9b4678d4ebfd86a7ae4385bbcdce55462"
+
+DEPENDS = "liburcu libnsl2"
+
+inherit cmake pkgconfig
+
+PACKAGECONFIG[gss] = "-DUSE_GSS=true,-DUSE_GSS=false,krb5"
+PACKAGECONFIG[rdma] = "-DUSE_RPC_RDMA=true,-DUSE_RPC_RDMA=false,rdma-core"
+PACKAGECONFIG[winbind] = "-D_MSPAC_SUPPORT=true,-D_MSPAC_SUPPORT=false,samba"
+PACKAGECONFIG[profile] = "-DUSE_PROFILE=true,-DUSE_PROFILE=false,gperftools"
+PACKAGECONFIG[lttng] = "-DUSE_LTTNG=true,-DUSE_LTTNG=false,lttng-ust 
lttng-tools"
+
+# Sort out the brain-dead cmake file
+# (https://github.com/nfs-ganesha/ntirpc/issues/150)
+do_install:append() {
+if [ "${prefix}/lib64" != "${libdir}" -a -d ${D}${prefix}/lib64 ]; then
+mv ${D}${prefix}/lib64 ${D}${libdir}
+fi
+}
+
+BBCLASSEXTEND = "native nativesdk"
+EXTRA_OECMAKE:armv5 += '-DSYSTEM_LIBRARIES="-latomic"'
+EXTRA_OECMAKE:riscv32 += '-DSYSTEM_LIBRARIES="-latomic"'
+EXTRA_OECMAKE:riscv64 += '-DSYSTEM_LIBRARIES="-latomic"'
+EXTRA_OECMAKE:powerpc += '-DSYSTEM_LIBRARIES="-latomic"'
diff --git a/meta/recipes-extended/quota/quota_4.06.bb 
b/meta/recipes-extended/quota/quota_4.06.bb
index 0535d14c20..304477d72b 100644
--- a/meta/recipes-extended/quota/quota_4.06.bb
+++ b/meta/recipes-extended/quota/quota_4.06.bb
@@ -30,6 +30,6 @@ PACKAGECONFIG ??= "tcp-wrappers rpc bsd"
 PACKAGECONFIG:libc-musl = "tcp-wrappers rpc"
 
 PACKAGECONFIG[tcp-wrappers] = "--enable-libwrap,--disable-libwrap,tcp-wrappers"
-PACKAGECONFIG[rpc] = "--enable-rpc,--disable-rpc,libtirpc"
+PACKAGECONFIG[rpc] = "--enable-rpc,--disable-rpc,ntirpc"
 PACKAGECONFIG[bsd] = "--enable-bsd_behaviour=yes,--enable-bsd_behaviour=no,"
 PACKAGECONFIG[ldapmail] = "--enable-ldapmail,--disable-ldapmail,openldap"
diff --git a/meta/recipes-extended/rpcbind/rpcbind_1.2.6.bb 
b/meta/recipes-extended/rpcbind/rpcbind_1.2.6.bb
index dd89726afc..e30a23909c 100644
--- a/meta/recipes-extended/rpcbind/rpcbind_1.2.6.bb
+++ b/meta/recipes-extended/rpcbind/rpcbind_1.2.6.bb
@@ -4,7 +4,7 @@ DESCRIPTION = "The rpcbind utility is a server that converts 
RPC \
 SECTION = "console/network"
 HOMEPAGE = "http://sourceforge.net/projects/rpcbind/";
 BUGTRACKER = "http://sourceforge.net/tracker/?group_id=201237&atid=976751";
-DEPENDS = "libtirpc quota"
+DEPENDS = "ntirpc quota"
 
 LICENSE = "BSD-3-Clause"
 LIC_FILES_CHKSUM = "file://COPYING;md5=b46486e4c4a416602693a711bb5bfa39 \
diff --git a/meta/recipes-extended/watchdog/watchdog_5.16.bb 
b/meta/recipes-extended/watchdog/watchdog_5.16.bb
index 26fcc10487..5f5fd4dc87 100644
--- a/meta/recipes-extended/watchdog/watchdog_5.16.bb
+++ b/meta/recipes-extended/watchdog/watchdog_5.16.bb
@@ -28,7 +28,7 @@ UPSTREAM_CHECK_REGEX = "/watchdog/(?P(\d+[\.\-_]*)+)/"
 
 inherit autotools update-rc.d systemd pkgconfig
 
-DEPENDS += "libtirpc"
+DEPENDS += "ntirpc"
 CFLAGS += "-I${STAGING_INCDIR}/tirpc"
 LDFLAGS += "-ltirpc"
 
diff --git a/meta/recipes-graphics/xorg-xserver/xserver-xorg.inc 
b/meta/recipes-graphics/xorg-xserver/xserver-xorg.inc
index 057a1ba6ad..2085f742d6 100644
--- a/meta/recipes-graphics/xorg-xserver/xserver-xorg.inc
+++ b/meta/recipes-graphics/xorg-xserver/xserver-xorg.inc
@@ -36,7 +36,7 @@ inherit meson pkgconfig
 inherit features_check
 REQUIRED_DISTRO_FEATURES = "x11"
 
-LIB_DEPS = "pixman libxfont2 xtrans libxau libxext libxdmcp libdrm libxkbfile 
libpciaccess libxcvt libtirpc"
+LIB_DEPS = "pixman libxfont2 xtrans libxau li

Re: [OE-core] [PATCH 1/1] lame: Remove hardcoded rpaths

2022-07-27 Thread Paulo Neves

I thought of that myself, but did not yet investigate. We have 
hardcode_into_libs disabling in 4 "recipes":


git grep hardcode_into_libs -- meta
recipes-devtools/binutils/binutils/0009-Use-libtool-2.4.patch: 
hardcode_into_libs=yes
recipes-devtools/binutils/binutils/0009-Use-libtool-2.4.patch: if test 
"$hardcode_into_libs" != yes || test "$build_old_libs" = yes; then
recipes-devtools/gcc/gcc-sanitizers.inc:    sed -i -e 
's/hardcode_into_libs=yes/hardcode_into_libs=no/' 
${B}/${TARGET_SYS}/libsanitizer/libtool
recipes-devtools/gcc/gcc-source.inc:    cmd = d.expand("sed -i -e 
's/hardcode_into_libs=yes/hardcode_into_libs=no/' ${S}/libcc1/configure")
recipes-devtools/gcc/libgfortran.inc:  sed -i -e 
's/hardcode_into_libs=yes/hardcode_into_libs=no/' 
${B}/${TARGET_SYS}/$target/libtool
recipes-support/db/db_5.3.28.bb:    sed -i -e 
's|hardcode_into_libs=yes|hardcode_into_libs=no|' ${B}/libtool


The hardcode_into_libs=no did not fix things for the lame case. We have 
a patch for libtools that looks like could be the culprit and need 
tightening [1]



[1] 
recipes-devtools/libtool/libtool/0004-ltmain.sh-Fix-sysroot-paths-being-encoded-into-RPATH.patch


Paulo Neves

On 7/26/22 23:34, Richard Purdie wrote:

On Tue, 2022-07-26 at 22:23 +0200, Paulo Neves wrote:

Due to hardcoded rpaths tmpdir contamination happened on lame
executable. This commit hot patches libtool to not hardcode
the .libs directory in the rpath. This solves [YOCTO #14863]

Signed-off-by: Paulo Neves 
---
  meta/recipes-multimedia/lame/lame_3.100.bb | 4 
  1 file changed, 4 insertions(+)

diff --git a/meta/recipes-multimedia/lame/lame_3.100.bb 
b/meta/recipes-multimedia/lame/lame_3.100.bb
index 5b42795f4e..2d9bd9d29f 100644
--- a/meta/recipes-multimedia/lame/lame_3.100.bb
+++ b/meta/recipes-multimedia/lame/lame_3.100.bb
@@ -24,3 +24,7 @@ FILES:${PN} = "${bindir}/lame"
  FILES:libmp3lame = "${libdir}/libmp3lame.so.*"
  FILES:libmp3lame-dev = "${includedir} ${libdir}/*"
  FILES:${PN}-dev = ""
+do_compile:prepend() {
+# Stop libtool adding RPATHs
+sed -i -e 's|hardcode_libdir_flag_spec=".*"|hardcode_libdir_flag_spec=""|' 
${B}/libtool
+}

Any idea why lame has this issue but we don't see it with anything else
using libtool? I'm wondering if there is an underlying cause we should
look at?

Cheers,

Richard



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#168549): 
https://lists.openembedded.org/g/openembedded-core/message/168549
Mute This Topic: https://lists.openembedded.org/mt/92636379/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 1/1] lame: Remove hardcoded rpaths

2022-07-26 Thread Paulo Neves

Due to hardcoded rpaths tmpdir contamination happened on lame
executable. This commit hot patches libtool to not hardcode
the .libs directory in the rpath. This solves [YOCTO #14863]

Signed-off-by: Paulo Neves 
---
 meta/recipes-multimedia/lame/lame_3.100.bb | 4 
 1 file changed, 4 insertions(+)

diff --git a/meta/recipes-multimedia/lame/lame_3.100.bb 
b/meta/recipes-multimedia/lame/lame_3.100.bb
index 5b42795f4e..2d9bd9d29f 100644
--- a/meta/recipes-multimedia/lame/lame_3.100.bb
+++ b/meta/recipes-multimedia/lame/lame_3.100.bb
@@ -24,3 +24,7 @@ FILES:${PN} = "${bindir}/lame"
 FILES:libmp3lame = "${libdir}/libmp3lame.so.*"
 FILES:libmp3lame-dev = "${includedir} ${libdir}/*"
 FILES:${PN}-dev = ""
+do_compile:prepend() {
+# Stop libtool adding RPATHs
+sed -i -e 's|hardcode_libdir_flag_spec=".*"|hardcode_libdir_flag_spec=""|' 
${B}/libtool
+}
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#168514): 
https://lists.openembedded.org/g/openembedded-core/message/168514
Mute This Topic: https://lists.openembedded.org/mt/92636379/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v2 1/1] oeqa: test_invalid_recipe_src_uri expect parse time error

2022-07-14 Thread Paulo Neves

Before this commit the test expected a runtime error but the logic of
the fetcher changed so that files not found for checksumming cause a
fatal interruption of the parse, thuse never reaching the runtime error
this test expected. This commit aligns with the bitbake changes to the
fetcher.

Signed-off-by: Paulo Neves 
---
 meta/lib/oeqa/selftest/cases/bbtests.py | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/bbtests.py 
b/meta/lib/oeqa/selftest/cases/bbtests.py
index b42bbb651d..89267c7145 100644
--- a/meta/lib/oeqa/selftest/cases/bbtests.py
+++ b/meta/lib/oeqa/selftest/cases/bbtests.py
@@ -145,12 +145,10 @@ INHERIT:remove = \"report-error\"
 """)
 self.track_for_cleanup(os.path.join(self.builddir, 
"download-selftest"))
 
-bitbake('-ccleanall man-db')
 result = bitbake('-c fetch man-db', ignore_status=True)
-bitbake('-ccleanall man-db')
 self.delete_recipeinc('man-db')
 self.assertEqual(result.status, 1, msg="Command succeded when it 
should have failed. bitbake output: %s" % result.output)
-self.assertIn('Fetcher failure: Unable to find file file://invalid 
anywhere. The paths that were searched were:', result.output)
+self.assertIn('Unable to get checksum for man-db SRC_URI entry 
invalid: file could not be found', result.output)
 
 def test_rename_downloaded_file(self):
 # TODO unique dldir instead of using cleanall
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#168032): 
https://lists.openembedded.org/g/openembedded-core/message/168032
Mute This Topic: https://lists.openembedded.org/mt/92376758/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 2/3] oeqa/sysroot.py: Check bitbake return status

2022-07-07 Thread Paulo Neves

After re-thinking this is the correct version of the patch. We need to 
ignore_status because we are asserting that it actually fails. Otherwise 
the bitbake() will fail the test on the desired failure.


Paulo Neves

On 7/4/22 16:30, Richard Purdie wrote:

On Mon, 2022-07-04 at 16:28 +0200, Paulo Neves wrote:

On 7/4/22 16:16, Richard Purdie wrote:

On Sun, 2022-07-03 at 13:41 +0200, Paulo Neves wrote:

bitbake ran but we incorrectly did not assert the exit status needs to
be non 0. Now all sysroot tests commands expected to fail are verified
to do so.

Signed-off-by: Paulo Neves 
---
   meta/lib/oeqa/selftest/cases/sysroot.py | 5 +
   1 file changed, 5 insertions(+)

diff --git a/meta/lib/oeqa/selftest/cases/sysroot.py 
b/meta/lib/oeqa/selftest/cases/sysroot.py
index 588fc8c713..294ba4a4a0 100644
--- a/meta/lib/oeqa/selftest/cases/sysroot.py
+++ b/meta/lib/oeqa/selftest/cases/sysroot.py
@@ -45,6 +45,7 @@ TESTSTRING:pn-sysroot-test-arch2 = "%s"
   expected = "maximum shebang size exceeded, the maximum size is 128. 
[shebang-size]"
   res = bitbake("sysroot-shebang-test-native -c populate_sysroot", 
ignore_status=True)
   self.assertTrue(expected in res.output, msg=res.output)
+self.assertTrue(res.status != 0)
   
   def test_sysroot_la(self):

   """

I did have a question on this patch. Wouldn't it be simpler to remove
the "ignore_status=True" from the bitbake() call?

Cheers,

Richard

You are right. I guess the assert is more explicit and with the removal
of the ignore_status it becomes implicit. Let me know if you want me to
remove the assert and the ignore status and will submit a v2.

Please, I think that would be cleaner.

Cheers,

Richard



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#167785): 
https://lists.openembedded.org/g/openembedded-core/message/167785
Mute This Topic: https://lists.openembedded.org/mt/92145212/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [poky] [PATCH] perl: Fix perms to avoid patch fails

2022-07-07 Thread Paulo Neves


On 7/7/22 03:26, Randy MacLeod wrote:

Hi K,

Add oe-core list and move poky to BCC to ensure follow to the main 
oe-core code review list.


On 2022-06-17 03:13, K. wrote:
Fix missing write permissions to avoid problems with applying patches 
in particular on compressed filesystems:


ERROR: perl-5.34.1-r0 do_patch: Applying patch 
'0001-Somehow-this-module-breaks-through-the-perl-wrapper-.patch' on 
target directory 
'/home/kayo/devel/elpitech/poc/openbmc/build/evb-ast2600/tmp/work/armv7ahf-vfpv4d16-openbmc-linux-gnueabi/perl/5.34.1-r0/perl-5.34.1'
CmdError('quilt --quiltrc 
/home/kayo/devel/elpitech/poc/openbmc/build/evb-ast2600/tmp/work/armv7ahf-vfpv4d16-openbmc-linux-gnueabi/perl/5.34.1-r0/recipe-sysroot-native/etc/quiltrc 
push', 0, 'stdout: Applying patch 
0001-Somehow-this-module-breaks-through-the-perl-wrapper-.patch
File cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_Unix.pm is read-only; 
trying to patch anyway

patching file cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_Unix.pm
Hunk #1 succeeded at 1147 (offset 37 lines).
Hunk #2 succeeded at 2086 (offset 77 lines).
patch: setting attribute btrfs.compression for btrfs.compression: 
Permission denied
Patch 0001-Somehow-this-module-breaks-through-the-perl-wrapper-.patch 
does not apply (enforce with -f)


Signed-off-by: K 
---
  meta/recipes-devtools/perl/perl_5.36.0.bb | 7 +++
  1 file changed, 7 insertions(+)

diff --git a/meta/recipes-devtools/perl/perl_5.36.0.bb 
b/meta/recipes-devtools/perl/perl_5.36.0.bb

index 4456cdbcfb..6374717ab0 100644
--- a/meta/recipes-devtools/perl/perl_5.36.0.bb
+++ b/meta/recipes-devtools/perl/perl_5.36.0.bb
@@ -45,6 +45,13 @@ PACKAGECONFIG[gdbm] = ",-Ui_gdbm,gdbm"
  # Don't generate comments in enc2xs output files. They are not 
reproducible

  export ENC2XS_NO_COMMENTS = "1"
  +# Fix missing write permissions to avoid troubles
+# with applying patches on compressed filesystems
+do_fix_perms() {
+    find ${S} -type f ! -perm -u=w -exec chmod u+w {} \;
+}


I've never done a build on btrfs let alone on a compressed btrfs volume.
This does seem like it should affect many patches and not just the 
perl ones.


Any idea why only perl was affected and why it only shows up on that 
FS type?

This seems like a work-around rather than a proper fix to me so far.

../Randy



+addtask do_fix_perms after do_unpack before do_patch
+
  do_configure:prepend() {
  cp -rfp ${STAGING_DATADIR_NATIVE}/perl-cross/* ${S}
  }



This patch is not needed as the issue has been fixed in open embedded 
already in [1]. I think it is not yet in poky though.


[1] 
https://github.com/openembedded/openembedded-core/commit/47973fd4f3fbb0af1a0d1bc2c39f2900a963177d

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#167781): 
https://lists.openembedded.org/g/openembedded-core/message/167781
Mute This Topic: https://lists.openembedded.org/mt/92220035/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v2 2/3] utils: create_cmdline_shebang_wrapper preserve permission and ownership

2022-07-04 Thread Paulo Neves

The .real command was not given the same permissions and ownership as
the original pre-wrap file and this is now fixed.

A situation where the original pre-wrap file did not have write
permissions would cause a failure in the wrapping is also fixed.

Test update also included.

Signed-off-by: Paulo Neves 
Signed-off-by: Richard Purdie 
---
 .../wrapper/cmdline-shebang-wrapper-test.bb   | 11 ++-
 meta/classes/utils.bbclass|  3 +++
 2 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb 
b/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb
index c4126a41fc..c3d3548d4a 100644
--- a/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb
+++ b/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb
@@ -8,7 +8,10 @@ SRC_URI += "file://test.awk"
 EXCLUDE_FROM_WORLD = "1"
 do_install() {
 install -d ${D}${bindir}
-install -m 0755 ${WORKDIR}/test.awk ${D}${bindir}/test
+# was not able to make ownership preservation check
+install -m 0400 ${WORKDIR}/test.awk ${D}${bindir}/test
+
+perm_old="$(stat --format='%a' ${D}${bindir}/test)"
 sed -i -e 's|@AWK_BIN@|${bindir}/awk|g' ${D}${bindir}/test
 create_cmdline_shebang_wrapper ${D}${bindir}/test
 if [ $(${D}${bindir}/test) != "Don't Panic!" ]; then
@@ -16,6 +19,12 @@ do_install() {
 else
 bbnote "Wrapper is good"
 fi
+
+perm_new="$(stat --format='%a' ${D}${bindir}/test.real)"
+
+if [ "$perm_new" != "$perm_old" ]; then
+bbfatal "Wrapper permissions for ${D}${bindir}/test.real not 
preserved. Found $perm_new but expected $perm_old"
+fi
 }
 
 BBCLASSEXTEND = "native"
diff --git a/meta/classes/utils.bbclass b/meta/classes/utils.bbclass
index b58c22771f..e6f7f95d80 100644
--- a/meta/classes/utils.bbclass
+++ b/meta/classes/utils.bbclass
@@ -202,6 +202,9 @@ create_cmdline_shebang_wrapper () {
argument="$(sed -ne 's/^#! *//p;q' $cmd)"
# strip the shebang from the real script as we do not want it to be 
usable anyway
tail -n +2 $cmd > $cmd.real
+   chown --reference=$cmd $cmd.real
+   chmod --reference=$cmd $cmd.real
+   rm -f $cmd
cmdname=$(basename $cmd)
dirname=$(dirname $cmd)
cmdoptions=$@
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#167620): 
https://lists.openembedded.org/g/openembedded-core/message/167620
Mute This Topic: https://lists.openembedded.org/mt/92166128/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v2 3/3] oeqa/sysroot.py: Check bitbake return status

2022-07-04 Thread Paulo Neves

bitbake ran but we incorrectly did not check the exit status needs to
be non 0. Now all sysroot tests commands expected to fail are verified
to do so.

Signed-off-by: Paulo Neves 
---
 meta/lib/oeqa/selftest/cases/sysroot.py | 11 ++-
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/sysroot.py 
b/meta/lib/oeqa/selftest/cases/sysroot.py
index 588fc8c713..6371a10074 100644
--- a/meta/lib/oeqa/selftest/cases/sysroot.py
+++ b/meta/lib/oeqa/selftest/cases/sysroot.py
@@ -43,7 +43,7 @@ TESTSTRING:pn-sysroot-test-arch2 = "%s"
 Author:Paulo Neves 
 """
 expected = "maximum shebang size exceeded, the maximum size is 128. 
[shebang-size]"
-res = bitbake("sysroot-shebang-test-native -c populate_sysroot", 
ignore_status=True)
+res = bitbake("sysroot-shebang-test-native -c populate_sysroot")
 self.assertTrue(expected in res.output, msg=res.output)
 
 def test_sysroot_la(self):
@@ -54,11 +54,11 @@ TESTSTRING:pn-sysroot-test-arch2 = "%s"
 """
 expected = "la-test.la failed sanity test (workdir) in path"
 
-res = bitbake("sysroot-la-test -c populate_sysroot", 
ignore_status=True)
+res = bitbake("sysroot-la-test -c populate_sysroot")
 self.assertTrue(expected in res.output, msg=res.output)
 self.assertTrue('[la]' in res.output, msg=res.output)
 
-res = bitbake("sysroot-la-test-native -c populate_sysroot", 
ignore_status=True)
+res = bitbake("sysroot-la-test-native -c populate_sysroot")
 self.assertTrue(expected in res.output, msg=res.output)
 self.assertTrue('[la]' in res.output, msg=res.output)
 
@@ -70,10 +70,11 @@ TESTSTRING:pn-sysroot-test-arch2 = "%s"
 """
 expected = "test.pc failed sanity test (tmpdir) in path"
 
-res = bitbake("sysroot-pc-test -c populate_sysroot", 
ignore_status=True)
+res = bitbake("sysroot-pc-test -c populate_sysroot")
 self.assertTrue('[pkgconfig]' in res.output, msg=res.output)
 self.assertTrue(expected in res.output, msg=res.output)
 
-res = bitbake("sysroot-pc-test-native -c populate_sysroot", 
ignore_status=True)
+res = bitbake("sysroot-pc-test-native -c populate_sysroot")
 self.assertTrue(expected in res.output, msg=res.output)
 self.assertTrue('[pkgconfig]' in res.output, msg=res.output)
+self.assertTrue(res.status != 0)
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#167621): 
https://lists.openembedded.org/g/openembedded-core/message/167621
Mute This Topic: https://lists.openembedded.org/mt/92166130/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v2 1/3] utils: create_cmdline_shebang_wrapper whitespace and sed refactor

2022-07-04 Thread Paulo Neves

Correct whitespace to match the rest of the code in utils.
Refactored sed expression with a simpler equivalent.

Signed-off-by: Paulo Neves 
Signed-off-by: Richard Purdie 
---
 meta/classes/utils.bbclass | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/meta/classes/utils.bbclass b/meta/classes/utils.bbclass
index b617632d9f..b58c22771f 100644
--- a/meta/classes/utils.bbclass
+++ b/meta/classes/utils.bbclass
@@ -188,7 +188,7 @@ create_cmdline_shebang_wrapper () {
# Create a wrapper script where commandline options are needed
#
# These are useful to work around shebang relocation issues, where 
shebangs are too
-  # long or have arguments in them, thus preventing them from using the 
/usr/bin/env
+   # long or have arguments in them, thus preventing them from using the 
/usr/bin/env
# shebang
#
# Usage: create_cmdline_wrapper FILENAME 
@@ -198,10 +198,10 @@ create_cmdline_shebang_wrapper () {
 
echo "Generating wrapper script for $cmd"
 
-  # Strip #! and get remaining interpreter + arg
-  argument="$(basename "$(head -n1 $cmd | sed -e 's|#![ ]*||g' )")"
-  # strip the shebang from the real script as we do not want it to be usable 
anyway
-  tail -n +2 $cmd > $cmd.real
+   # Strip #! and get remaining interpreter + arg
+   argument="$(sed -ne 's/^#! *//p;q' $cmd)"
+   # strip the shebang from the real script as we do not want it to be 
usable anyway
+   tail -n +2 $cmd > $cmd.real
cmdname=$(basename $cmd)
dirname=$(dirname $cmd)
cmdoptions=$@
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#167619): 
https://lists.openembedded.org/g/openembedded-core/message/167619
Mute This Topic: https://lists.openembedded.org/mt/92166127/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 2/3] oeqa/sysroot.py: Check bitbake return status

2022-07-04 Thread Paulo Neves


On 7/4/22 16:16, Richard Purdie wrote:

On Sun, 2022-07-03 at 13:41 +0200, Paulo Neves wrote:

bitbake ran but we incorrectly did not assert the exit status needs to
be non 0. Now all sysroot tests commands expected to fail are verified
to do so.

Signed-off-by: Paulo Neves 
---
  meta/lib/oeqa/selftest/cases/sysroot.py | 5 +
  1 file changed, 5 insertions(+)

diff --git a/meta/lib/oeqa/selftest/cases/sysroot.py 
b/meta/lib/oeqa/selftest/cases/sysroot.py
index 588fc8c713..294ba4a4a0 100644
--- a/meta/lib/oeqa/selftest/cases/sysroot.py
+++ b/meta/lib/oeqa/selftest/cases/sysroot.py
@@ -45,6 +45,7 @@ TESTSTRING:pn-sysroot-test-arch2 = "%s"
  expected = "maximum shebang size exceeded, the maximum size is 128. 
[shebang-size]"
  res = bitbake("sysroot-shebang-test-native -c populate_sysroot", 
ignore_status=True)
  self.assertTrue(expected in res.output, msg=res.output)
+self.assertTrue(res.status != 0)
  
  def test_sysroot_la(self):

  """

I did have a question on this patch. Wouldn't it be simpler to remove
the "ignore_status=True" from the bitbake() call?

Cheers,

Richard
You are right. I guess the assert is more explicit and with the removal 
of the ignore_status it becomes implicit. Let me know if you want me to 
remove the assert and the ignore status and will submit a v2.


Paulo Neves

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#167617): 
https://lists.openembedded.org/g/openembedded-core/message/167617
Mute This Topic: https://lists.openembedded.org/mt/92145212/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 2/3] oeqa/sysroot.py: Check bitbake return status

2022-07-03 Thread Paulo Neves

bitbake ran but we incorrectly did not assert the exit status needs to
be non 0. Now all sysroot tests commands expected to fail are verified
to do so.

Signed-off-by: Paulo Neves 
---
 meta/lib/oeqa/selftest/cases/sysroot.py | 5 +
 1 file changed, 5 insertions(+)

diff --git a/meta/lib/oeqa/selftest/cases/sysroot.py 
b/meta/lib/oeqa/selftest/cases/sysroot.py
index 588fc8c713..294ba4a4a0 100644
--- a/meta/lib/oeqa/selftest/cases/sysroot.py
+++ b/meta/lib/oeqa/selftest/cases/sysroot.py
@@ -45,6 +45,7 @@ TESTSTRING:pn-sysroot-test-arch2 = "%s"
 expected = "maximum shebang size exceeded, the maximum size is 128. 
[shebang-size]"
 res = bitbake("sysroot-shebang-test-native -c populate_sysroot", 
ignore_status=True)
 self.assertTrue(expected in res.output, msg=res.output)
+self.assertTrue(res.status != 0)
 
 def test_sysroot_la(self):
 """
@@ -57,10 +58,12 @@ TESTSTRING:pn-sysroot-test-arch2 = "%s"
 res = bitbake("sysroot-la-test -c populate_sysroot", 
ignore_status=True)
 self.assertTrue(expected in res.output, msg=res.output)
 self.assertTrue('[la]' in res.output, msg=res.output)
+self.assertTrue(res.status != 0)
 
 res = bitbake("sysroot-la-test-native -c populate_sysroot", 
ignore_status=True)
 self.assertTrue(expected in res.output, msg=res.output)
 self.assertTrue('[la]' in res.output, msg=res.output)
+self.assertTrue(res.status != 0)
 
 def test_sysroot_pkgconfig(self):
 """
@@ -73,7 +76,9 @@ TESTSTRING:pn-sysroot-test-arch2 = "%s"
 res = bitbake("sysroot-pc-test -c populate_sysroot", 
ignore_status=True)
 self.assertTrue('[pkgconfig]' in res.output, msg=res.output)
 self.assertTrue(expected in res.output, msg=res.output)
+self.assertTrue(res.status != 0)
 
 res = bitbake("sysroot-pc-test-native -c populate_sysroot", 
ignore_status=True)
 self.assertTrue(expected in res.output, msg=res.output)
 self.assertTrue('[pkgconfig]' in res.output, msg=res.output)
+self.assertTrue(res.status != 0)
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#167554): 
https://lists.openembedded.org/g/openembedded-core/message/167554
Mute This Topic: https://lists.openembedded.org/mt/92145212/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 3/3] utils: create_cmdline_shebang_wrapper preserve permission and ownership

2022-07-03 Thread Paulo Neves

The .real command was not given the same permissions and ownership as
the original pre-wrap file and this is now fixed.

A situation where the original pre-wrap file did not have write
permissions would cause a failure in the wrapping is also fixed.

Test update also included.

Signed-off-by: Paulo Neves 
---
 .../wrapper/cmdline-shebang-wrapper-test.bb   | 11 ++-
 meta/classes/utils.bbclass|  3 +++
 2 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb 
b/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb
index c4126a41fc..c3d3548d4a 100644
--- a/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb
+++ b/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb
@@ -8,7 +8,10 @@ SRC_URI += "file://test.awk"
 EXCLUDE_FROM_WORLD = "1"
 do_install() {
 install -d ${D}${bindir}
-install -m 0755 ${WORKDIR}/test.awk ${D}${bindir}/test
+# was not able to make ownership preservation check
+install -m 0400 ${WORKDIR}/test.awk ${D}${bindir}/test
+
+perm_old="$(stat --format='%a' ${D}${bindir}/test)"
 sed -i -e 's|@AWK_BIN@|${bindir}/awk|g' ${D}${bindir}/test
 create_cmdline_shebang_wrapper ${D}${bindir}/test
 if [ $(${D}${bindir}/test) != "Don't Panic!" ]; then
@@ -16,6 +19,12 @@ do_install() {
 else
 bbnote "Wrapper is good"
 fi
+
+perm_new="$(stat --format='%a' ${D}${bindir}/test.real)"
+
+if [ "$perm_new" != "$perm_old" ]; then
+bbfatal "Wrapper permissions for ${D}${bindir}/test.real not 
preserved. Found $perm_new but expected $perm_old"
+fi
 }
 
 BBCLASSEXTEND = "native"
diff --git a/meta/classes/utils.bbclass b/meta/classes/utils.bbclass
index b58c22771f..e6f7f95d80 100644
--- a/meta/classes/utils.bbclass
+++ b/meta/classes/utils.bbclass
@@ -202,6 +202,9 @@ create_cmdline_shebang_wrapper () {
argument="$(sed -ne 's/^#! *//p;q' $cmd)"
# strip the shebang from the real script as we do not want it to be 
usable anyway
tail -n +2 $cmd > $cmd.real
+   chown --reference=$cmd $cmd.real
+   chmod --reference=$cmd $cmd.real
+   rm -f $cmd
cmdname=$(basename $cmd)
dirname=$(dirname $cmd)
cmdoptions=$@
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#167555): 
https://lists.openembedded.org/g/openembedded-core/message/167555
Mute This Topic: https://lists.openembedded.org/mt/92145213/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 1/3] utils: create_cmdline_shebang_wrapper whitespace and sed refactor

2022-07-03 Thread Paulo Neves

Correct whitespace to match the rest of the code in utils.
Refactored sed expression with a simpler equivalent.

Signed-off-by: Paulo Neves 
---
 meta/classes/utils.bbclass | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/meta/classes/utils.bbclass b/meta/classes/utils.bbclass
index b617632d9f..b58c22771f 100644
--- a/meta/classes/utils.bbclass
+++ b/meta/classes/utils.bbclass
@@ -188,7 +188,7 @@ create_cmdline_shebang_wrapper () {
# Create a wrapper script where commandline options are needed
#
# These are useful to work around shebang relocation issues, where 
shebangs are too
-  # long or have arguments in them, thus preventing them from using the 
/usr/bin/env
+   # long or have arguments in them, thus preventing them from using the 
/usr/bin/env
# shebang
#
# Usage: create_cmdline_wrapper FILENAME 
@@ -198,10 +198,10 @@ create_cmdline_shebang_wrapper () {
 
echo "Generating wrapper script for $cmd"
 
-  # Strip #! and get remaining interpreter + arg
-  argument="$(basename "$(head -n1 $cmd | sed -e 's|#![ ]*||g' )")"
-  # strip the shebang from the real script as we do not want it to be usable 
anyway
-  tail -n +2 $cmd > $cmd.real
+   # Strip #! and get remaining interpreter + arg
+   argument="$(sed -ne 's/^#! *//p;q' $cmd)"
+   # strip the shebang from the real script as we do not want it to be 
usable anyway
+   tail -n +2 $cmd > $cmd.real
cmdname=$(basename $cmd)
dirname=$(dirname $cmd)
cmdoptions=$@
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#167553): 
https://lists.openembedded.org/g/openembedded-core/message/167553
Mute This Topic: https://lists.openembedded.org/mt/92145211/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v6 7/7] libcheck: Fix too long shebang for native case.

2022-06-19 Thread Paulo Neves

It requires a shebang wrapper due to the fact that awk interpreter
has an argument.
---
 meta/recipes-support/libcheck/libcheck_0.15.2.bb | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meta/recipes-support/libcheck/libcheck_0.15.2.bb 
b/meta/recipes-support/libcheck/libcheck_0.15.2.bb
index 188d689cc3..1393aa2a1c 100644
--- a/meta/recipes-support/libcheck/libcheck_0.15.2.bb
+++ b/meta/recipes-support/libcheck/libcheck_0.15.2.bb
@@ -23,6 +23,9 @@ CACHED_CONFIGUREVARS += "ac_cv_path_AWK_PATH=${bindir}/gawk"
 
 RREPLACES:${PN} = "check (<= 0.9.5)"
 
+do_install:append:class-native() {
+create_cmdline_shebang_wrapper ${D}${bindir}/checkmk
+}
 BBCLASSEXTEND = "native nativesdk"
 
 PACKAGES =+ "checkmk"
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#167045): 
https://lists.openembedded.org/g/openembedded-core/message/167045
Mute This Topic: https://lists.openembedded.org/mt/91863581/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v6 4/7] oeqa/selftest: Add test for shebang overflow

2022-06-19 Thread Paulo Neves

Make sure we do not stage any executable with a bigger shebang
than 128. Fixes [1]

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11053

Signed-off-by: Paulo Neves 
---
 .../sysroot-test/sysroot-shebang-test_1.0.bb | 12 
 meta/lib/oeqa/selftest/cases/sysroot.py  | 11 +++
 2 files changed, 23 insertions(+)
 create mode 100644 
meta-selftest/recipes-test/sysroot-test/sysroot-shebang-test_1.0.bb

diff --git 
a/meta-selftest/recipes-test/sysroot-test/sysroot-shebang-test_1.0.bb 
b/meta-selftest/recipes-test/sysroot-test/sysroot-shebang-test_1.0.bb
new file mode 100644
index 00..6c834be897
--- /dev/null
+++ b/meta-selftest/recipes-test/sysroot-test/sysroot-shebang-test_1.0.bb
@@ -0,0 +1,12 @@
+SUMMARY = "Check that shebang does not exceed 128 characters"
+LICENSE = "CLOSED"
+INHIBIT_DEFAULT_DEPS = "1"
+
+EXCLUDE_FROM_WORLD = "1"
+do_install() {
+install -d ${D}${bindir}
+echo 
'#!BiM3cnVd1Amtv6PG+FynrQiVMbZnX5ELgF21q3EkuB+44JEGWtq8TvBJ7EGidfVs3eR3wVOUbLnjYDlKUWcm7YC/ute7f+KDHbwxziRUSUBZAUqgjiQdfQ0HnxajI0ozbM863E9JV9k13yZKYfh9/zR77Y6Dl4Dd3zOWS75LSpkAXV'
 > ${D}${bindir}/max-shebang
+chmod 755 ${D}${bindir}/max-shebang
+}
+
+BBCLASSEXTEND = "native"
diff --git a/meta/lib/oeqa/selftest/cases/sysroot.py 
b/meta/lib/oeqa/selftest/cases/sysroot.py
index 315d1a61c2..a17fd1818a 100644
--- a/meta/lib/oeqa/selftest/cases/sysroot.py
+++ b/meta/lib/oeqa/selftest/cases/sysroot.py
@@ -35,3 +35,14 @@ TESTSTRING:pn-sysroot-test-arch1 = "%s"
 TESTSTRING:pn-sysroot-test-arch2 = "%s"
 """ % (uuid1, uuid2))
 bitbake("sysroot-test")
+
+def test_sysroot_max_shebang(self):
+"""
+Summary:   Check max shebang triggers. To confirm [YOCTO #11053] is 
closed.
+Expected:  Fail when a shebang bigger than the max shebang-size is 
reached.
+Author:Paulo Neves 
+"""
+expected = "maximum shebang size exceeded, the maximum size is 128. 
[shebang-size]"
+res = bitbake("sysroot-shebang-test-native -c populate_sysroot", 
ignore_status=True)
+self.assertTrue(expected in res.output, msg=res.output)
+self.assertTrue(res.status != 0)
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#167042): 
https://lists.openembedded.org/g/openembedded-core/message/167042
Mute This Topic: https://lists.openembedded.org/mt/91863577/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v6 6/7] utils: Add cmdline_shebang_wrapper util.

2022-06-19 Thread Paulo Neves

Useful to work around shebang relocation issues, where
shebangs are too long or have arguments in them, thus preventing them
from using the /usr/bin/env shebang.
---
 .../wrapper/cmdline-shebang-wrapper-test.bb   | 21 
 .../recipes-test/wrapper/files/test.awk   |  2 ++
 meta/classes/utils.bbclass| 34 +++
 meta/lib/oeqa/selftest/cases/wrapper.py   | 11 ++
 4 files changed, 68 insertions(+)
 create mode 100644 
meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb
 create mode 100644 meta-selftest/recipes-test/wrapper/files/test.awk
 create mode 100644 meta/lib/oeqa/selftest/cases/wrapper.py

diff --git a/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb 
b/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb
new file mode 100644
index 00..c4126a41fc
--- /dev/null
+++ b/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb
@@ -0,0 +1,21 @@
+SUMMARY = "Check that create_cmdline_shebang works"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = 
"file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
+INHIBIT_DEFAULT_DEPS = "1"
+
+SRC_URI += "file://test.awk"
+
+EXCLUDE_FROM_WORLD = "1"
+do_install() {
+install -d ${D}${bindir}
+install -m 0755 ${WORKDIR}/test.awk ${D}${bindir}/test
+sed -i -e 's|@AWK_BIN@|${bindir}/awk|g' ${D}${bindir}/test
+create_cmdline_shebang_wrapper ${D}${bindir}/test
+if [ $(${D}${bindir}/test) != "Don't Panic!" ]; then
+bbfatal "Wrapper is broken"
+else
+bbnote "Wrapper is good"
+fi
+}
+
+BBCLASSEXTEND = "native"
diff --git a/meta-selftest/recipes-test/wrapper/files/test.awk 
b/meta-selftest/recipes-test/wrapper/files/test.awk
new file mode 100644
index 00..91429197b1
--- /dev/null
+++ b/meta-selftest/recipes-test/wrapper/files/test.awk
@@ -0,0 +1,2 @@
+#! @AWK_BIN@ -f
+BEGIN { print "Don't Panic!" }
diff --git a/meta/classes/utils.bbclass b/meta/classes/utils.bbclass
index b4eb3d38ab..b58c22771f 100644
--- a/meta/classes/utils.bbclass
+++ b/meta/classes/utils.bbclass
@@ -184,6 +184,40 @@ END
chmod +x $cmd
 }
 
+create_cmdline_shebang_wrapper () {
+   # Create a wrapper script where commandline options are needed
+   #
+   # These are useful to work around shebang relocation issues, where 
shebangs are too
+   # long or have arguments in them, thus preventing them from using the 
/usr/bin/env
+   # shebang
+   #
+   # Usage: create_cmdline_wrapper FILENAME 
+
+   cmd=$1
+   shift
+
+   echo "Generating wrapper script for $cmd"
+
+   # Strip #! and get remaining interpreter + arg
+   argument="$(sed -ne 's/^#! *//p;q' $cmd)"
+   # strip the shebang from the real script as we do not want it to be 
usable anyway
+   tail -n +2 $cmd > $cmd.real
+   cmdname=$(basename $cmd)
+   dirname=$(dirname $cmd)
+   cmdoptions=$@
+   if [ "${base_prefix}" != "" ]; then
+   relpath=`python3 -c "import os; 
print(os.path.relpath('${D}${base_prefix}', '$dirname'))"`
+   cmdoptions=`echo $@ | sed -e 
"s:${base_prefix}:\\$realdir/$relpath:g"`
+   fi
+   cat <$cmd
+#!/usr/bin/env bash
+realpath=\`readlink -fn \$0\`
+realdir=\`dirname \$realpath\`
+exec -a \$realdir/$cmdname $argument \$realdir/$cmdname.real $cmdoptions "\$@"
+END
+   chmod +x $cmd
+}
+
 create_wrapper () {
# Create a wrapper script where extra environment variables are needed
#
diff --git a/meta/lib/oeqa/selftest/cases/wrapper.py 
b/meta/lib/oeqa/selftest/cases/wrapper.py
new file mode 100644
index 00..6de63310c0
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/wrapper.py
@@ -0,0 +1,11 @@
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import bitbake
+
+class WrapperTests(OESelftestTestCase):
+def test_shebang_wrapper(self):
+"""
+Summary:   Build a recipe which will fail if the 
cmdline_shebang_wrapper function is defective.
+Expected:  Exit status to be 0.
+Author:Paulo Neves 
+"""
+res = bitbake("cmdline-shebang-wrapper-test -c install", 
ignore_status=False)
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#167044): 
https://lists.openembedded.org/g/openembedded-core/message/167044
Mute This Topic: https://lists.openembedded.org/mt/91863579/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v6 5/7] oeqa/selftest: Test staged .la and .pc files

2022-06-19 Thread Paulo Neves

These files are checked by qa_check_staged but there was no
test cases for whether the tests actually worked. Now there
are.

Signed-off-by: Paulo Neves 
---
 .../sysroot-test/sysroot-la-test_1.0.bb   | 16 +
 .../sysroot-test/sysroot-pc-test_1.0.bb   | 12 +++
 meta/lib/oeqa/selftest/cases/sysroot.py   | 36 +++
 3 files changed, 64 insertions(+)
 create mode 100644 
meta-selftest/recipes-test/sysroot-test/sysroot-la-test_1.0.bb
 create mode 100644 
meta-selftest/recipes-test/sysroot-test/sysroot-pc-test_1.0.bb

diff --git a/meta-selftest/recipes-test/sysroot-test/sysroot-la-test_1.0.bb 
b/meta-selftest/recipes-test/sysroot-test/sysroot-la-test_1.0.bb
new file mode 100644
index 00..21f06782fb
--- /dev/null
+++ b/meta-selftest/recipes-test/sysroot-test/sysroot-la-test_1.0.bb
@@ -0,0 +1,16 @@
+SUMMARY = "Produce a broken la file"
+LICENSE = "CLOSED"
+INHIBIT_DEFAULT_DEPS = "1"
+
+EXCLUDE_FROM_WORLD = "1"
+
+# remove-libtool.bbclass is inherited by default and removes all
+# .la files which for this test we specifically do not want.
+REMOVE_LIBTOOL_LA = "0"
+
+do_install() {
+install -d ${D}${libdir}/test/
+echo '${WORKDIR}' > ${D}${libdir}/test/la-test.la
+}
+
+BBCLASSEXTEND += "native"
diff --git a/meta-selftest/recipes-test/sysroot-test/sysroot-pc-test_1.0.bb 
b/meta-selftest/recipes-test/sysroot-test/sysroot-pc-test_1.0.bb
new file mode 100644
index 00..e748310fc4
--- /dev/null
+++ b/meta-selftest/recipes-test/sysroot-test/sysroot-pc-test_1.0.bb
@@ -0,0 +1,12 @@
+SUMMARY = "Produce a broken pc file"
+LICENSE = "CLOSED"
+INHIBIT_DEFAULT_DEPS = "1"
+
+EXCLUDE_FROM_WORLD = "1"
+
+do_install() {
+install -d ${D}${libdir}/test/
+echo '${WORKDIR}' > ${D}${libdir}/test/test.pc
+}
+
+BBCLASSEXTEND += "native"
diff --git a/meta/lib/oeqa/selftest/cases/sysroot.py 
b/meta/lib/oeqa/selftest/cases/sysroot.py
index a17fd1818a..294ba4a4a0 100644
--- a/meta/lib/oeqa/selftest/cases/sysroot.py
+++ b/meta/lib/oeqa/selftest/cases/sysroot.py
@@ -46,3 +46,39 @@ TESTSTRING:pn-sysroot-test-arch2 = "%s"
 res = bitbake("sysroot-shebang-test-native -c populate_sysroot", 
ignore_status=True)
 self.assertTrue(expected in res.output, msg=res.output)
 self.assertTrue(res.status != 0)
+
+def test_sysroot_la(self):
+"""
+    Summary:   Check that workdir paths are not contained in .la files.
+Expected:  Fail when a workdir path is found in the file content.
+Author:Paulo Neves 
+"""
+expected = "la-test.la failed sanity test (workdir) in path"
+
+res = bitbake("sysroot-la-test -c populate_sysroot", 
ignore_status=True)
+self.assertTrue(expected in res.output, msg=res.output)
+self.assertTrue('[la]' in res.output, msg=res.output)
+self.assertTrue(res.status != 0)
+
+res = bitbake("sysroot-la-test-native -c populate_sysroot", 
ignore_status=True)
+self.assertTrue(expected in res.output, msg=res.output)
+self.assertTrue('[la]' in res.output, msg=res.output)
+self.assertTrue(res.status != 0)
+
+def test_sysroot_pkgconfig(self):
+"""
+Summary:   Check that tmpdir paths are not contained in .pc files.
+Expected:  Fail when a tmpdir path is found in the file content.
+Author:Paulo Neves 
+"""
+expected = "test.pc failed sanity test (tmpdir) in path"
+
+res = bitbake("sysroot-pc-test -c populate_sysroot", 
ignore_status=True)
+self.assertTrue('[pkgconfig]' in res.output, msg=res.output)
+self.assertTrue(expected in res.output, msg=res.output)
+self.assertTrue(res.status != 0)
+
+res = bitbake("sysroot-pc-test-native -c populate_sysroot", 
ignore_status=True)
+self.assertTrue(expected in res.output, msg=res.output)
+self.assertTrue('[pkgconfig]' in res.output, msg=res.output)
+self.assertTrue(res.status != 0)
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#167043): 
https://lists.openembedded.org/g/openembedded-core/message/167043
Mute This Topic: https://lists.openembedded.org/mt/91863578/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v6 3/7] insane.bbclass: Make do_qa_staging check shebangs

2022-06-19 Thread Paulo Neves

As reported in the bug report [1], there was no check for shebang
sizes on native scripts and now this is fixed.

The path scope of the qa_staging was increased from just checking
libdir to all the relevant SYSROOT_DIRS.

It is possible to skip this check through INSANE_SKIP.

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11053

Signed-off-by: Paulo Neves 
---
 meta/classes/insane.bbclass | 16 +++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index 9ca84bace9..b2951a48fe 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -630,6 +630,11 @@ def qa_check_staged(path,d):
 bb.note("Recipe %s skipping qa checking: pkgconfig" % d.getVar('PN'))
 skip_pkgconfig = True
 
+skip_shebang_size = False
+if 'shebang-size' in skip:
+bb.note("Recipe %s skipping qa checkking: shebang-size" % 
d.getVar('PN'))
+skip_shebang_size = True
+
 # find all .la and .pc files
 # read the content
 # and check for stuff that looks wrong
@@ -651,6 +656,13 @@ def qa_check_staged(path,d):
 error_msg = "%s failed sanity test (tmpdir) in path 
%s" % (file,root)
 oe.qa.handle_error("pkgconfig", error_msg, d)
 
+if not skip_shebang_size:
+errors = {}
+package_qa_check_shebang_size(path, "", d, None, errors)
+for e in errors:
+oe.qa.handle_error(e, errors[e], d)
+
+
 # Run all package-wide warnfuncs and errorfuncs
 def package_qa_package(warnfuncs, errorfuncs, package, d):
 warnings = {}
@@ -1139,7 +1151,9 @@ addtask do_package_qa_setscene
 
 python do_qa_staging() {
 bb.note("QA checking staging")
-qa_check_staged(d.expand('${SYSROOT_DESTDIR}${libdir}'), d)
+sysroot_destdir = d.expand('${SYSROOT_DESTDIR}')
+for sysroot_dir in d.expand('${SYSROOT_DIRS}').split():
+qa_check_staged(sysroot_destdir + sysroot_dir, d)
 oe.qa.exit_with_message_if_errors("QA staging was broken by the package 
built above", d)
 }
 
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#167041): 
https://lists.openembedded.org/g/openembedded-core/message/167041
Mute This Topic: https://lists.openembedded.org/mt/91863576/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v6 1/7] python: Avoid shebang overflow on python-config.py

2022-06-19 Thread Paulo Neves

The native path may be too big, leading to shebang
overflow. Just use the #!/usr/bin/env python3.

Signed-off-by: Paulo Neves 
---
 ...shebang-overflow-on-python-config.py.patch | 33 +++
 .../recipes-devtools/python/python3_3.10.4.bb |  1 +
 2 files changed, 34 insertions(+)
 create mode 100644 
meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch

diff --git 
a/meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch
 
b/meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch
new file mode 100644
index 00..921da8de7c
--- /dev/null
+++ 
b/meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch
@@ -0,0 +1,33 @@
+From f0c9dec63d452a7cd1e15ea653f4aced281f021c Mon Sep 17 00:00:00 2001
+From: Paulo Neves 
+Date: Tue, 7 Jun 2022 16:16:41 +0200
+Subject: [PATCH 1/1] Avoid shebang overflow on python-config.py
+
+The whole native path may be too big, leading to shebang
+overflow. Let's just use the env shebang.
+
+Denial reason: [1]
+
+Upstream-Status: Denied [distribution]
+
+[1] https://github.com/python/cpython/pull/93760#pullrequestreview-1005365737
+---
+ Makefile.pre.in | 2 ++
+ 1 file changed, 2 insertions(+)
+
+diff --git a/Makefile.pre.in b/Makefile.pre.in
+index 
f0aedb76cb58999427804255da56fa53284d7032..dd88e43114730f7681715777cc76dabb31113176
 100644
+--- a/Makefile.pre.in
 b/Makefile.pre.in
+@@ -1638,6 +1638,8 @@ python-config: $(srcdir)/Misc/python-config.in 
Misc/python-config.sh
+   @ # Substitution happens here, as the completely-expanded BINDIR
+   @ # is not available in configure
+   sed -e "s,@EXENAME@,$(BINDIR)/python$(LDVERSION)$(EXE)," < 
$(srcdir)/Misc/python-config.in >python-config.py
++  @ # Otherwise we might get huge shebangs with native paths
++  sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' python-config.py
+   @ # Replace makefile compat. variable references with shell script 
compat. ones; $(VAR) -> ${VAR}
+   LC_ALL=C sed -e 's,\$$(\([A-Za-z0-9_]*\)),\$$\{\1\},g' < 
Misc/python-config.sh >python-config
+   @  # In OpenEmbedded, always use the python version of the script, the 
shell
+-- 
+2.25.1
+
diff --git a/meta/recipes-devtools/python/python3_3.10.4.bb 
b/meta/recipes-devtools/python/python3_3.10.4.bb
index 6bd3a6aba8..357025f856 100644
--- a/meta/recipes-devtools/python/python3_3.10.4.bb
+++ b/meta/recipes-devtools/python/python3_3.10.4.bb
@@ -34,6 +34,7 @@ SRC_URI = 
"http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \

file://0001-Lib-pty.py-handle-stdin-I-O-errors-same-way-as-maste.patch \

file://0001-setup.py-Do-not-detect-multiarch-paths-when-cross-co.patch \
file://deterministic_imports.patch \
+   file://0001-Avoid-shebang-overflow-on-python-config.py.patch \
"
 
 SRC_URI:append:class-native = " \
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#167039): 
https://lists.openembedded.org/g/openembedded-core/message/167039
Mute This Topic: https://lists.openembedded.org/mt/91863574/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v6 2/7] gtk-doc: Fix potential shebang overflow on gtkdoc-mkhtml2

2022-06-19 Thread Paulo Neves

A native path can cause a shebang overflow on gtkdoc-mkhtml.
Replace it with /usr/bin/env.

Signed-off-by: Paulo Neves 
---
 meta/recipes-gnome/gtk-doc/gtk-doc_1.33.2.bb | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-gnome/gtk-doc/gtk-doc_1.33.2.bb 
b/meta/recipes-gnome/gtk-doc/gtk-doc_1.33.2.bb
index 150d2c0b23..392913fcc6 100644
--- a/meta/recipes-gnome/gtk-doc/gtk-doc_1.33.2.bb
+++ b/meta/recipes-gnome/gtk-doc/gtk-doc_1.33.2.bb
@@ -43,6 +43,7 @@ do_install:append () {
 ${datadir}/gtk-doc/python/gtkdoc/config.py; do
 sed -e 
's,${RECIPE_SYSROOT_NATIVE}/usr/bin/pkg-config,${bindir}/pkg-config,' \
 -e 's,${HOSTTOOLS_DIR}/python3,${bindir}/python3,' \
+-e '1s|^#!.*|#!/usr/bin/env python3|' \
 -i ${D}$fn
 done
 }
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#167040): 
https://lists.openembedded.org/g/openembedded-core/message/167040
Mute This Topic: https://lists.openembedded.org/mt/91863575/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v5 6/7] utils: Add cmdline_shebang_wrapper util.

2022-06-19 Thread Paulo Neves


You are right never mind.

Paulo Neves

On 6/16/22 00:16, Peter Kjellerstedt wrote:

-Original Message-
From: Paulo Neves 
Sent: den 15 juni 2022 15:51
To: Peter Kjellerstedt ; Ross Burton

Cc: openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH v5 6/7] utils: Add cmdline_shebang_wrapper
util.

On 6/15/22 15:11, Peter Kjellerstedt wrote:

-Original Message-
From: openembedded-core@lists.openembedded.org  On Behalf Of Ross Burton
Sent: den 15 juni 2022 12:52
To: ptsne...@gmail.com
Cc: openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH v5 6/7] utils: Add

cmdline_shebang_wrapper

util.

On 14 Jun 2022, at 16:11, Paulo Neves via lists.openembedded.org
 wrote:

+create_cmdline_shebang_wrapper () {

The indentation in this function is a mix of tabs and spaces, please
unify.


+  argument="$(basename "$(head -n1 $cmd | sed -e 's|#![ ]*||g' )”)”

[ ]* seems overly, complex, no need for the brackets. I’d anchor to the
beginning of the line, and remove the g as there’s only one expected.

This should work:

argument=$(basename "$(sed -ne 's/^#! *//p;q')" $cmd)

However, if the first line is something like "#!/usr/bin/env python3", then
$argument will be "env python3" and somehow I assume that is not exactly
what the rest of the code expects.

The use case is that whatever is in shebang should be extracted and ran
verbatim, so truncating like your suggestion can work but has slightly
different semantic.

Umm, my suggested line above should give exactly the same output as yours,
only with fewer commands...


Might also be wise to assert that you found something here, so running
this on an ELF fails with an obvious error.

Ross

//Peter


Paulo Neves

//Peter




-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#167038): 
https://lists.openembedded.org/g/openembedded-core/message/167038
Mute This Topic: https://lists.openembedded.org/mt/91751270/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v5 3/7] insane.bbclass: Make do_qa_staging check shebangs

2022-06-19 Thread Paulo Neves

Yes, when I was making the change I also realized the scope of the check
was quite bigger than before. I am going to deliver the final changes
requested and leave it up to you to decide on the merge. Let me know your
decision so I can close the bug as well.

Paulo Neves

On Fri, Jun 17, 2022, 18:50 Richard Purdie <
richard.pur...@linuxfoundation.org> wrote:

> On Tue, 2022-06-14 at 17:11 +0200, Paulo Neves wrote:
> > As reported in the bug report [1], there was no check for shebang
> > sizes on native scripts and now this is fixed.
> >
> > The path scope of the qa_staging was increased from just checking
> > libdir to all the relevant SYSROOT_DIRS.
> >
> > It is possible to skip this check through INSANE_SKIP.
> >
> > [1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11053
> >
> > Signed-off-by: Paulo Neves 
> > ---
> >  meta/classes/insane.bbclass | 16 +++-
> >  1 file changed, 15 insertions(+), 1 deletion(-)
> >
> > diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
> > index 9ca84bace9..b2951a48fe 100644
> > --- a/meta/classes/insane.bbclass
> > +++ b/meta/classes/insane.bbclass
> > @@ -630,6 +630,11 @@ def qa_check_staged(path,d):
> >  bb.note("Recipe %s skipping qa checking: pkgconfig" %
> d.getVar('PN'))
> >  skip_pkgconfig = True
> >
> > +skip_shebang_size = False
> > +if 'shebang-size' in skip:
> > +bb.note("Recipe %s skipping qa checkking: shebang-size" %
> d.getVar('PN'))
> > +skip_shebang_size = True
> > +
> >  # find all .la and .pc files
> >  # read the content
> >  # and check for stuff that looks wrong
> > @@ -651,6 +656,13 @@ def qa_check_staged(path,d):
> >  error_msg = "%s failed sanity test (tmpdir) in
> path %s" % (file,root)
> >  oe.qa.handle_error("pkgconfig", error_msg, d)
> >
> > +if not skip_shebang_size:
> > +errors = {}
> > +package_qa_check_shebang_size(path, "", d, None, errors)
> > +for e in errors:
> > +oe.qa.handle_error(e, errors[e], d)
> > +
> > +
> >  # Run all package-wide warnfuncs and errorfuncs
> >  def package_qa_package(warnfuncs, errorfuncs, package, d):
> >  warnings = {}
> > @@ -1139,7 +1151,9 @@ addtask do_package_qa_setscene
> >
> >  python do_qa_staging() {
> >  bb.note("QA checking staging")
> > -qa_check_staged(d.expand('${SYSROOT_DESTDIR}${libdir}'), d)
> > +sysroot_destdir = d.expand('${SYSROOT_DESTDIR}')
> > +for sysroot_dir in d.expand('${SYSROOT_DIRS}').split():
> > +qa_check_staged(sysroot_destdir + sysroot_dir, d)
> >  oe.qa.exit_with_message_if_errors("QA staging was broken by the
> package built above", d)
> >  }
>
>
> I'm a little worried about the performance implications of this, we're
> going from scanning files in libdir to scanning nearly all files in the
> sysroots, reading from many of them. In isolation that doesn't seem
> much but I suspect the IO will impact builds overall.
>
> That leaves me a little torn on this change :/
>
> Cheers,
>
> Richard
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#167033): 
https://lists.openembedded.org/g/openembedded-core/message/167033
Mute This Topic: https://lists.openembedded.org/mt/91751267/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v5 6/7] utils: Add cmdline_shebang_wrapper util.

2022-06-15 Thread Paulo Neves

On 6/15/22 15:11, Peter Kjellerstedt wrote:

-Original Message-
From: openembedded-core@lists.openembedded.org  On Behalf Of Ross Burton
Sent: den 15 juni 2022 12:52
To: ptsne...@gmail.com
Cc: openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH v5 6/7] utils: Add cmdline_shebang_wrapper
util.

On 14 Jun 2022, at 16:11, Paulo Neves via lists.openembedded.org
 wrote:

+create_cmdline_shebang_wrapper () {

The indentation in this function is a mix of tabs and spaces, please
unify.

+  argument="$(basename "$(head -n1 $cmd | sed -e 's|#![ ]*||g' )”)”

[ ]* seems overly, complex, no need for the brackets. I’d anchor to the
beginning of the line, and remove the g as there’s only one expected.

This should work:

argument=$(basename "$(sed -ne 's/^#! *//p;q')" $cmd)

However, if the first line is something like "#!/usr/bin/env python3", then
$argument will be "env python3" and somehow I assume that is not exactly
what the rest of the code expects.

The use case is that whatever is in shebang should be extracted and ran 
verbatim, so truncating like your suggestion can work but has slightly 
different semantic.

Might also be wise to assert that you found something here, so running
this on an ELF fails with an obvious error.

Ross

//Peter

Paulo Neves

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166988): 
https://lists.openembedded.org/g/openembedded-core/message/166988
Mute This Topic: https://lists.openembedded.org/mt/91751270/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 1/3] insane.bbclass: Make do_qa_staging check shebangs

2022-06-15 Thread Paulo Neves


:)

The function should probably be refactored so it does not imply it is 
only useful for packaging.


Paulo Neves

On 6/15/22 13:08, Ross Burton wrote:

Never mind, can’t read :)


On 15 Jun 2022, at 12:07, Ross Burton  wrote:


+if not skip_shebang_size:
+errors = {}
+package_qa_check_shebang_size(path, "", d, None, errors)
+for e in errors:
+oe.qa.handle_error(e, errors[e], d)

Where is package_qa_check_shebang_size() defined?

Ross

IMPORTANT NOTICE: The contents of this email and any attachments are 
confidential and may also be privileged. If you are not the intended recipient, 
please notify the sender immediately and do not disclose the contents to any 
other person, use it for any purpose, or store or copy the information in any 
medium. Thank you.



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166934): 
https://lists.openembedded.org/g/openembedded-core/message/166934
Mute This Topic: https://lists.openembedded.org/mt/91580233/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v4 6/7] utils: Add cmdline_shebang_wrapper util.

2022-06-14 Thread Paulo Neves

Thanks for the great tips. Yeah the habit of using the FILESPATH is that 
mostly I work on bbappends. Today i learned.


I sent a new v5 with the corrections you mention.

Paulo Neves

On 6/14/22 16:17, Martin Jansa wrote:
You can use bitbake-getvar to see the default FILESPATH, but "files" 
next to recipe is included by default (I prefer to use BPN directory 
for slightly faster lookup - unless the files are shared between 
different recipes). FILESEXTRAPATHS is usually only needed from 
bbappends which add new files to SRC_URI.


For LICENSE you can use MIT and
LIC_FILES_CHKSUM = 
"file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"

like many "pure-metadata" recipes in oe-core.

On Tue, Jun 14, 2022 at 3:46 PM Paulo Neves  wrote:

Regarding the FILESEXTRAPATHS:prepend  i was convinced that
${THISDIR}/files was not automatically added and was needed for
the test.awk file. I guess I have been cargo culting for some time
on this. Will remove it.

This is a recipe for a test and if i add a license different than
CLOSED than i need to add a license file, which seems a bit
overkill for a one line. Am I wrong? What license should it even
be? Dont Panic is a common string used in awk docs.

Paulo Neves

On 6/14/22 15:39, Martin Jansa wrote:

FILESEXTRAPATHS:prepend doesn't seem to be needed and LICENSE is
really CLOSED?

    On Tue, Jun 14, 2022 at 3:17 PM Paulo Neves 
wrote:

Useful to work around shebang relocation issues, where
shebangs are too long or have arguments in them, thus
preventing them
from using the /usr/bin/env shebang.
---
 .../wrapper/cmdline-shebang-wrapper-test.bb
<http://cmdline-shebang-wrapper-test.bb>  | 21 
 .../recipes-test/wrapper/files/test.awk       |  2 ++
 meta/classes/utils.bbclass                    | 34
+++
 meta/lib/oeqa/selftest/cases/wrapper.py       | 11 ++
 4 files changed, 68 insertions(+)
 create mode 100644
meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb
<http://cmdline-shebang-wrapper-test.bb>
 create mode 100644
meta-selftest/recipes-test/wrapper/files/test.awk
 create mode 100644 meta/lib/oeqa/selftest/cases/wrapper.py

diff --git
a/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb
<http://cmdline-shebang-wrapper-test.bb>
b/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb
<http://cmdline-shebang-wrapper-test.bb>
new file mode 100644
index 00..302eea8901
--- /dev/null
+++
b/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb
<http://cmdline-shebang-wrapper-test.bb>
@@ -0,0 +1,21 @@
+FILESEXTRAPATHS:prepend := "${THISDIR}/files:"
+SUMMARY = "Check that create_cmdline_shebang works"
+LICENSE = "CLOSED"
+INHIBIT_DEFAULT_DEPS = "1"
+
+SRC_URI += "file://test.awk"
+
+EXCLUDE_FROM_WORLD = "1"
+do_install() {
+    install -d ${D}${bindir}
+    install -m 0755 ${WORKDIR}/test.awk ${D}${bindir}/test
+    sed -i -e 's|@AWK_BIN@|${bindir}/awk|g' ${D}${bindir}/test
+    create_cmdline_shebang_wrapper ${D}${bindir}/test
+    if [ $(${D}${bindir}/test) != "Don't Panic!" ]; then
+        bbfatal "Wrapper is broken"
+    else
+        bbnote "Wrapper is good"
+    fi
+}
+
+BBCLASSEXTEND = "native"
diff --git
a/meta-selftest/recipes-test/wrapper/files/test.awk
b/meta-selftest/recipes-test/wrapper/files/test.awk
new file mode 100644
index 00..91429197b1
--- /dev/null
+++ b/meta-selftest/recipes-test/wrapper/files/test.awk
@@ -0,0 +1,2 @@
+#! @AWK_BIN@ -f
+BEGIN { print "Don't Panic!" }
diff --git a/meta/classes/utils.bbclass
b/meta/classes/utils.bbclass
index b4eb3d38ab..b617632d9f 100644
--- a/meta/classes/utils.bbclass
+++ b/meta/classes/utils.bbclass
@@ -184,6 +184,40 @@ END
        chmod +x $cmd
 }

+create_cmdline_shebang_wrapper () {
+       # Create a wrapper script where commandline options
are needed
+       #
+       # These are useful to work around shebang relocation
issues, where shebangs are too
+  # long or have arguments in them, thus preventing them
from using the /usr/bin/env
+       # shebang
+       #
+       # Usage: create_cmdline_wrapper FILEN

[OE-core] [PATCH v5 6/7] utils: Add cmdline_shebang_wrapper util.

2022-06-14 Thread Paulo Neves

Useful to work around shebang relocation issues, where
shebangs are too long or have arguments in them, thus preventing them
from using the /usr/bin/env shebang.
---
 .../wrapper/cmdline-shebang-wrapper-test.bb   | 21 
 .../recipes-test/wrapper/files/test.awk   |  2 ++
 meta/classes/utils.bbclass| 34 +++
 meta/lib/oeqa/selftest/cases/wrapper.py   | 11 ++
 4 files changed, 68 insertions(+)
 create mode 100644 
meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb
 create mode 100644 meta-selftest/recipes-test/wrapper/files/test.awk
 create mode 100644 meta/lib/oeqa/selftest/cases/wrapper.py

diff --git a/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb 
b/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb
new file mode 100644
index 00..c4126a41fc
--- /dev/null
+++ b/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb
@@ -0,0 +1,21 @@
+SUMMARY = "Check that create_cmdline_shebang works"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = 
"file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
+INHIBIT_DEFAULT_DEPS = "1"
+
+SRC_URI += "file://test.awk"
+
+EXCLUDE_FROM_WORLD = "1"
+do_install() {
+install -d ${D}${bindir}
+install -m 0755 ${WORKDIR}/test.awk ${D}${bindir}/test
+sed -i -e 's|@AWK_BIN@|${bindir}/awk|g' ${D}${bindir}/test
+create_cmdline_shebang_wrapper ${D}${bindir}/test
+if [ $(${D}${bindir}/test) != "Don't Panic!" ]; then
+bbfatal "Wrapper is broken"
+else
+bbnote "Wrapper is good"
+fi
+}
+
+BBCLASSEXTEND = "native"
diff --git a/meta-selftest/recipes-test/wrapper/files/test.awk 
b/meta-selftest/recipes-test/wrapper/files/test.awk
new file mode 100644
index 00..91429197b1
--- /dev/null
+++ b/meta-selftest/recipes-test/wrapper/files/test.awk
@@ -0,0 +1,2 @@
+#! @AWK_BIN@ -f
+BEGIN { print "Don't Panic!" }
diff --git a/meta/classes/utils.bbclass b/meta/classes/utils.bbclass
index b4eb3d38ab..b617632d9f 100644
--- a/meta/classes/utils.bbclass
+++ b/meta/classes/utils.bbclass
@@ -184,6 +184,40 @@ END
chmod +x $cmd
 }
 
+create_cmdline_shebang_wrapper () {
+   # Create a wrapper script where commandline options are needed
+   #
+   # These are useful to work around shebang relocation issues, where 
shebangs are too
+  # long or have arguments in them, thus preventing them from using the 
/usr/bin/env
+   # shebang
+   #
+   # Usage: create_cmdline_wrapper FILENAME 
+
+   cmd=$1
+   shift
+
+   echo "Generating wrapper script for $cmd"
+
+  # Strip #! and get remaining interpreter + arg
+  argument="$(basename "$(head -n1 $cmd | sed -e 's|#![ ]*||g' )")"
+  # strip the shebang from the real script as we do not want it to be usable 
anyway
+  tail -n +2 $cmd > $cmd.real
+   cmdname=$(basename $cmd)
+   dirname=$(dirname $cmd)
+   cmdoptions=$@
+   if [ "${base_prefix}" != "" ]; then
+   relpath=`python3 -c "import os; 
print(os.path.relpath('${D}${base_prefix}', '$dirname'))"`
+   cmdoptions=`echo $@ | sed -e 
"s:${base_prefix}:\\$realdir/$relpath:g"`
+   fi
+   cat <$cmd
+#!/usr/bin/env bash
+realpath=\`readlink -fn \$0\`
+realdir=\`dirname \$realpath\`
+exec -a \$realdir/$cmdname $argument \$realdir/$cmdname.real $cmdoptions "\$@"
+END
+   chmod +x $cmd
+}
+
 create_wrapper () {
# Create a wrapper script where extra environment variables are needed
#
diff --git a/meta/lib/oeqa/selftest/cases/wrapper.py 
b/meta/lib/oeqa/selftest/cases/wrapper.py
new file mode 100644
index 00..6de63310c0
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/wrapper.py
@@ -0,0 +1,11 @@
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import bitbake
+
+class WrapperTests(OESelftestTestCase):
+def test_shebang_wrapper(self):
+"""
+Summary:   Build a recipe which will fail if the 
cmdline_shebang_wrapper function is defective.
+Expected:  Exit status to be 0.
+Author:Paulo Neves 
+"""
+res = bitbake("cmdline-shebang-wrapper-test -c install", 
ignore_status=False)
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166900): 
https://lists.openembedded.org/g/openembedded-core/message/166900
Mute This Topic: https://lists.openembedded.org/mt/91751270/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v5 7/7] libcheck: Fix too long shebang for native case.

2022-06-14 Thread Paulo Neves

It requires a shebang wrapper due to the fact that awk interpreter
has an argument.
---
 meta/recipes-support/libcheck/libcheck_0.15.2.bb | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meta/recipes-support/libcheck/libcheck_0.15.2.bb 
b/meta/recipes-support/libcheck/libcheck_0.15.2.bb
index 188d689cc3..1393aa2a1c 100644
--- a/meta/recipes-support/libcheck/libcheck_0.15.2.bb
+++ b/meta/recipes-support/libcheck/libcheck_0.15.2.bb
@@ -23,6 +23,9 @@ CACHED_CONFIGUREVARS += "ac_cv_path_AWK_PATH=${bindir}/gawk"
 
 RREPLACES:${PN} = "check (<= 0.9.5)"
 
+do_install:append:class-native() {
+create_cmdline_shebang_wrapper ${D}${bindir}/checkmk
+}
 BBCLASSEXTEND = "native nativesdk"
 
 PACKAGES =+ "checkmk"
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166901): 
https://lists.openembedded.org/g/openembedded-core/message/166901
Mute This Topic: https://lists.openembedded.org/mt/91751271/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v5 5/7] oeqa/selftest: Test staged .la and .pc files

2022-06-14 Thread Paulo Neves

These files are checked by qa_check_staged but there was no
test cases for whether the tests actually worked. Now there
are.

Signed-off-by: Paulo Neves 
---
 .../sysroot-test/sysroot-la-test_1.0.bb   | 16 ++
 .../sysroot-test/sysroot-pc-test_1.0.bb   | 12 +++
 meta/lib/oeqa/selftest/cases/sysroot.py   | 32 +++
 3 files changed, 60 insertions(+)
 create mode 100644 
meta-selftest/recipes-test/sysroot-test/sysroot-la-test_1.0.bb
 create mode 100644 
meta-selftest/recipes-test/sysroot-test/sysroot-pc-test_1.0.bb

diff --git a/meta-selftest/recipes-test/sysroot-test/sysroot-la-test_1.0.bb 
b/meta-selftest/recipes-test/sysroot-test/sysroot-la-test_1.0.bb
new file mode 100644
index 00..21f06782fb
--- /dev/null
+++ b/meta-selftest/recipes-test/sysroot-test/sysroot-la-test_1.0.bb
@@ -0,0 +1,16 @@
+SUMMARY = "Produce a broken la file"
+LICENSE = "CLOSED"
+INHIBIT_DEFAULT_DEPS = "1"
+
+EXCLUDE_FROM_WORLD = "1"
+
+# remove-libtool.bbclass is inherited by default and removes all
+# .la files which for this test we specifically do not want.
+REMOVE_LIBTOOL_LA = "0"
+
+do_install() {
+install -d ${D}${libdir}/test/
+echo '${WORKDIR}' > ${D}${libdir}/test/la-test.la
+}
+
+BBCLASSEXTEND += "native"
diff --git a/meta-selftest/recipes-test/sysroot-test/sysroot-pc-test_1.0.bb 
b/meta-selftest/recipes-test/sysroot-test/sysroot-pc-test_1.0.bb
new file mode 100644
index 00..e748310fc4
--- /dev/null
+++ b/meta-selftest/recipes-test/sysroot-test/sysroot-pc-test_1.0.bb
@@ -0,0 +1,12 @@
+SUMMARY = "Produce a broken pc file"
+LICENSE = "CLOSED"
+INHIBIT_DEFAULT_DEPS = "1"
+
+EXCLUDE_FROM_WORLD = "1"
+
+do_install() {
+install -d ${D}${libdir}/test/
+echo '${WORKDIR}' > ${D}${libdir}/test/test.pc
+}
+
+BBCLASSEXTEND += "native"
diff --git a/meta/lib/oeqa/selftest/cases/sysroot.py 
b/meta/lib/oeqa/selftest/cases/sysroot.py
index 9457f1e3ac..588fc8c713 100644
--- a/meta/lib/oeqa/selftest/cases/sysroot.py
+++ b/meta/lib/oeqa/selftest/cases/sysroot.py
@@ -45,3 +45,35 @@ TESTSTRING:pn-sysroot-test-arch2 = "%s"
 expected = "maximum shebang size exceeded, the maximum size is 128. 
[shebang-size]"
 res = bitbake("sysroot-shebang-test-native -c populate_sysroot", 
ignore_status=True)
 self.assertTrue(expected in res.output, msg=res.output)
+
+def test_sysroot_la(self):
+"""
+    Summary:   Check that workdir paths are not contained in .la files.
+Expected:  Fail when a workdir path is found in the file content.
+Author:Paulo Neves 
+"""
+expected = "la-test.la failed sanity test (workdir) in path"
+
+res = bitbake("sysroot-la-test -c populate_sysroot", 
ignore_status=True)
+self.assertTrue(expected in res.output, msg=res.output)
+self.assertTrue('[la]' in res.output, msg=res.output)
+
+res = bitbake("sysroot-la-test-native -c populate_sysroot", 
ignore_status=True)
+self.assertTrue(expected in res.output, msg=res.output)
+self.assertTrue('[la]' in res.output, msg=res.output)
+
+def test_sysroot_pkgconfig(self):
+"""
+Summary:   Check that tmpdir paths are not contained in .pc files.
+Expected:  Fail when a tmpdir path is found in the file content.
+Author:Paulo Neves 
+"""
+expected = "test.pc failed sanity test (tmpdir) in path"
+
+res = bitbake("sysroot-pc-test -c populate_sysroot", 
ignore_status=True)
+self.assertTrue('[pkgconfig]' in res.output, msg=res.output)
+self.assertTrue(expected in res.output, msg=res.output)
+
+res = bitbake("sysroot-pc-test-native -c populate_sysroot", 
ignore_status=True)
+self.assertTrue(expected in res.output, msg=res.output)
+self.assertTrue('[pkgconfig]' in res.output, msg=res.output)
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166899): 
https://lists.openembedded.org/g/openembedded-core/message/166899
Mute This Topic: https://lists.openembedded.org/mt/91751269/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v5 4/7] oeqa/selftest: Add test for shebang overflow

2022-06-14 Thread Paulo Neves

Make sure we do not stage any executable with a bigger shebang
than 128. Fixes [1]

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11053

Signed-off-by: Paulo Neves 
---
 .../sysroot-test/sysroot-shebang-test_1.0.bb | 12 
 meta/lib/oeqa/selftest/cases/sysroot.py  | 10 ++
 2 files changed, 22 insertions(+)
 create mode 100644 
meta-selftest/recipes-test/sysroot-test/sysroot-shebang-test_1.0.bb

diff --git 
a/meta-selftest/recipes-test/sysroot-test/sysroot-shebang-test_1.0.bb 
b/meta-selftest/recipes-test/sysroot-test/sysroot-shebang-test_1.0.bb
new file mode 100644
index 00..6c834be897
--- /dev/null
+++ b/meta-selftest/recipes-test/sysroot-test/sysroot-shebang-test_1.0.bb
@@ -0,0 +1,12 @@
+SUMMARY = "Check that shebang does not exceed 128 characters"
+LICENSE = "CLOSED"
+INHIBIT_DEFAULT_DEPS = "1"
+
+EXCLUDE_FROM_WORLD = "1"
+do_install() {
+install -d ${D}${bindir}
+echo 
'#!BiM3cnVd1Amtv6PG+FynrQiVMbZnX5ELgF21q3EkuB+44JEGWtq8TvBJ7EGidfVs3eR3wVOUbLnjYDlKUWcm7YC/ute7f+KDHbwxziRUSUBZAUqgjiQdfQ0HnxajI0ozbM863E9JV9k13yZKYfh9/zR77Y6Dl4Dd3zOWS75LSpkAXV'
 > ${D}${bindir}/max-shebang
+chmod 755 ${D}${bindir}/max-shebang
+}
+
+BBCLASSEXTEND = "native"
diff --git a/meta/lib/oeqa/selftest/cases/sysroot.py 
b/meta/lib/oeqa/selftest/cases/sysroot.py
index 315d1a61c2..9457f1e3ac 100644
--- a/meta/lib/oeqa/selftest/cases/sysroot.py
+++ b/meta/lib/oeqa/selftest/cases/sysroot.py
@@ -35,3 +35,13 @@ TESTSTRING:pn-sysroot-test-arch1 = "%s"
 TESTSTRING:pn-sysroot-test-arch2 = "%s"
 """ % (uuid1, uuid2))
 bitbake("sysroot-test")
+
+def test_sysroot_max_shebang(self):
+"""
+Summary:   Check max shebang triggers. To confirm [YOCTO #11053] is 
closed.
+Expected:  Fail when a shebang bigger than the max shebang-size is 
reached.
+Author:Paulo Neves 
+"""
+expected = "maximum shebang size exceeded, the maximum size is 128. 
[shebang-size]"
+res = bitbake("sysroot-shebang-test-native -c populate_sysroot", 
ignore_status=True)
+self.assertTrue(expected in res.output, msg=res.output)
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166898): 
https://lists.openembedded.org/g/openembedded-core/message/166898
Mute This Topic: https://lists.openembedded.org/mt/91751268/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v5 1/7] python: Avoid shebang overflow on python-config.py

2022-06-14 Thread Paulo Neves

The native path may be too big, leading to shebang
overflow. Just use the #!/usr/bin/env python3.

Signed-off-by: Paulo Neves 
---
 ...shebang-overflow-on-python-config.py.patch | 33 +++
 .../recipes-devtools/python/python3_3.10.4.bb |  1 +
 2 files changed, 34 insertions(+)
 create mode 100644 
meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch

diff --git 
a/meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch
 
b/meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch
new file mode 100644
index 00..921da8de7c
--- /dev/null
+++ 
b/meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch
@@ -0,0 +1,33 @@
+From f0c9dec63d452a7cd1e15ea653f4aced281f021c Mon Sep 17 00:00:00 2001
+From: Paulo Neves 
+Date: Tue, 7 Jun 2022 16:16:41 +0200
+Subject: [PATCH 1/1] Avoid shebang overflow on python-config.py
+
+The whole native path may be too big, leading to shebang
+overflow. Let's just use the env shebang.
+
+Denial reason: [1]
+
+Upstream-Status: Denied [distribution]
+
+[1] https://github.com/python/cpython/pull/93760#pullrequestreview-1005365737
+---
+ Makefile.pre.in | 2 ++
+ 1 file changed, 2 insertions(+)
+
+diff --git a/Makefile.pre.in b/Makefile.pre.in
+index 
f0aedb76cb58999427804255da56fa53284d7032..dd88e43114730f7681715777cc76dabb31113176
 100644
+--- a/Makefile.pre.in
 b/Makefile.pre.in
+@@ -1638,6 +1638,8 @@ python-config: $(srcdir)/Misc/python-config.in 
Misc/python-config.sh
+   @ # Substitution happens here, as the completely-expanded BINDIR
+   @ # is not available in configure
+   sed -e "s,@EXENAME@,$(BINDIR)/python$(LDVERSION)$(EXE)," < 
$(srcdir)/Misc/python-config.in >python-config.py
++  @ # Otherwise we might get huge shebangs with native paths
++  sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' python-config.py
+   @ # Replace makefile compat. variable references with shell script 
compat. ones; $(VAR) -> ${VAR}
+   LC_ALL=C sed -e 's,\$$(\([A-Za-z0-9_]*\)),\$$\{\1\},g' < 
Misc/python-config.sh >python-config
+   @  # In OpenEmbedded, always use the python version of the script, the 
shell
+-- 
+2.25.1
+
diff --git a/meta/recipes-devtools/python/python3_3.10.4.bb 
b/meta/recipes-devtools/python/python3_3.10.4.bb
index 6bd3a6aba8..357025f856 100644
--- a/meta/recipes-devtools/python/python3_3.10.4.bb
+++ b/meta/recipes-devtools/python/python3_3.10.4.bb
@@ -34,6 +34,7 @@ SRC_URI = 
"http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \

file://0001-Lib-pty.py-handle-stdin-I-O-errors-same-way-as-maste.patch \

file://0001-setup.py-Do-not-detect-multiarch-paths-when-cross-co.patch \
file://deterministic_imports.patch \
+   file://0001-Avoid-shebang-overflow-on-python-config.py.patch \
"
 
 SRC_URI:append:class-native = " \
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166895): 
https://lists.openembedded.org/g/openembedded-core/message/166895
Mute This Topic: https://lists.openembedded.org/mt/91751265/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v5 3/7] insane.bbclass: Make do_qa_staging check shebangs

2022-06-14 Thread Paulo Neves

As reported in the bug report [1], there was no check for shebang
sizes on native scripts and now this is fixed.

The path scope of the qa_staging was increased from just checking
libdir to all the relevant SYSROOT_DIRS.

It is possible to skip this check through INSANE_SKIP.

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11053

Signed-off-by: Paulo Neves 
---
 meta/classes/insane.bbclass | 16 +++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index 9ca84bace9..b2951a48fe 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -630,6 +630,11 @@ def qa_check_staged(path,d):
 bb.note("Recipe %s skipping qa checking: pkgconfig" % d.getVar('PN'))
 skip_pkgconfig = True
 
+skip_shebang_size = False
+if 'shebang-size' in skip:
+bb.note("Recipe %s skipping qa checkking: shebang-size" % 
d.getVar('PN'))
+skip_shebang_size = True
+
 # find all .la and .pc files
 # read the content
 # and check for stuff that looks wrong
@@ -651,6 +656,13 @@ def qa_check_staged(path,d):
 error_msg = "%s failed sanity test (tmpdir) in path 
%s" % (file,root)
 oe.qa.handle_error("pkgconfig", error_msg, d)
 
+if not skip_shebang_size:
+errors = {}
+package_qa_check_shebang_size(path, "", d, None, errors)
+for e in errors:
+oe.qa.handle_error(e, errors[e], d)
+
+
 # Run all package-wide warnfuncs and errorfuncs
 def package_qa_package(warnfuncs, errorfuncs, package, d):
 warnings = {}
@@ -1139,7 +1151,9 @@ addtask do_package_qa_setscene
 
 python do_qa_staging() {
 bb.note("QA checking staging")
-qa_check_staged(d.expand('${SYSROOT_DESTDIR}${libdir}'), d)
+sysroot_destdir = d.expand('${SYSROOT_DESTDIR}')
+for sysroot_dir in d.expand('${SYSROOT_DIRS}').split():
+qa_check_staged(sysroot_destdir + sysroot_dir, d)
 oe.qa.exit_with_message_if_errors("QA staging was broken by the package 
built above", d)
 }
 
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166897): 
https://lists.openembedded.org/g/openembedded-core/message/166897
Mute This Topic: https://lists.openembedded.org/mt/91751267/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v5 2/7] gtk-doc: Fix potential shebang overflow on gtkdoc-mkhtml2

2022-06-14 Thread Paulo Neves

A native path can cause a shebang overflow on gtkdoc-mkhtml.
Replace it with /usr/bin/env.

Signed-off-by: Paulo Neves 
---
 meta/recipes-gnome/gtk-doc/gtk-doc_1.33.2.bb | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-gnome/gtk-doc/gtk-doc_1.33.2.bb 
b/meta/recipes-gnome/gtk-doc/gtk-doc_1.33.2.bb
index 150d2c0b23..392913fcc6 100644
--- a/meta/recipes-gnome/gtk-doc/gtk-doc_1.33.2.bb
+++ b/meta/recipes-gnome/gtk-doc/gtk-doc_1.33.2.bb
@@ -43,6 +43,7 @@ do_install:append () {
 ${datadir}/gtk-doc/python/gtkdoc/config.py; do
 sed -e 
's,${RECIPE_SYSROOT_NATIVE}/usr/bin/pkg-config,${bindir}/pkg-config,' \
 -e 's,${HOSTTOOLS_DIR}/python3,${bindir}/python3,' \
+-e '1s|^#!.*|#!/usr/bin/env python3|' \
 -i ${D}$fn
 done
 }
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166896): 
https://lists.openembedded.org/g/openembedded-core/message/166896
Mute This Topic: https://lists.openembedded.org/mt/91751266/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v4 6/7] utils: Add cmdline_shebang_wrapper util.

2022-06-14 Thread Paulo Neves

Regarding the FILESEXTRAPATHS:prepend  i was convinced that 
${THISDIR}/files was not automatically added and was needed for the 
test.awk file. I guess I have been cargo culting for some time on this. 
Will remove it.


This is a recipe for a test and if i add a license different than CLOSED 
than i need to add a license file, which seems a bit overkill for a one 
line. Am I wrong? What license should it even be? Dont Panic is a common 
string used in awk docs.


Paulo Neves

On 6/14/22 15:39, Martin Jansa wrote:
FILESEXTRAPATHS:prepend doesn't seem to be needed and LICENSE is 
really CLOSED?


On Tue, Jun 14, 2022 at 3:17 PM Paulo Neves  wrote:

Useful to work around shebang relocation issues, where
shebangs are too long or have arguments in them, thus preventing them
from using the /usr/bin/env shebang.
---
 .../wrapper/cmdline-shebang-wrapper-test.bb
<http://cmdline-shebang-wrapper-test.bb>  | 21 
 .../recipes-test/wrapper/files/test.awk       |  2 ++
 meta/classes/utils.bbclass                    | 34
+++
 meta/lib/oeqa/selftest/cases/wrapper.py       | 11 ++
 4 files changed, 68 insertions(+)
 create mode 100644
meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb
<http://cmdline-shebang-wrapper-test.bb>
 create mode 100644 meta-selftest/recipes-test/wrapper/files/test.awk
 create mode 100644 meta/lib/oeqa/selftest/cases/wrapper.py

diff --git
a/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb
<http://cmdline-shebang-wrapper-test.bb>
b/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb
<http://cmdline-shebang-wrapper-test.bb>
new file mode 100644
index 00..302eea8901
--- /dev/null
+++
b/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb
<http://cmdline-shebang-wrapper-test.bb>
@@ -0,0 +1,21 @@
+FILESEXTRAPATHS:prepend := "${THISDIR}/files:"
+SUMMARY = "Check that create_cmdline_shebang works"
+LICENSE = "CLOSED"
+INHIBIT_DEFAULT_DEPS = "1"
+
+SRC_URI += "file://test.awk"
+
+EXCLUDE_FROM_WORLD = "1"
+do_install() {
+    install -d ${D}${bindir}
+    install -m 0755 ${WORKDIR}/test.awk ${D}${bindir}/test
+    sed -i -e 's|@AWK_BIN@|${bindir}/awk|g' ${D}${bindir}/test
+    create_cmdline_shebang_wrapper ${D}${bindir}/test
+    if [ $(${D}${bindir}/test) != "Don't Panic!" ]; then
+        bbfatal "Wrapper is broken"
+    else
+        bbnote "Wrapper is good"
+    fi
+}
+
+BBCLASSEXTEND = "native"
diff --git a/meta-selftest/recipes-test/wrapper/files/test.awk
b/meta-selftest/recipes-test/wrapper/files/test.awk
new file mode 100644
index 00..91429197b1
--- /dev/null
+++ b/meta-selftest/recipes-test/wrapper/files/test.awk
@@ -0,0 +1,2 @@
+#! @AWK_BIN@ -f
+BEGIN { print "Don't Panic!" }
diff --git a/meta/classes/utils.bbclass b/meta/classes/utils.bbclass
index b4eb3d38ab..b617632d9f 100644
--- a/meta/classes/utils.bbclass
+++ b/meta/classes/utils.bbclass
@@ -184,6 +184,40 @@ END
        chmod +x $cmd
 }

+create_cmdline_shebang_wrapper () {
+       # Create a wrapper script where commandline options are needed
+       #
+       # These are useful to work around shebang relocation
issues, where shebangs are too
+  # long or have arguments in them, thus preventing them from
using the /usr/bin/env
+       # shebang
+       #
+       # Usage: create_cmdline_wrapper FILENAME 
+
+       cmd=$1
+       shift
+
+       echo "Generating wrapper script for $cmd"
+
+  # Strip #! and get remaining interpreter + arg
+  argument="$(basename "$(head -n1 $cmd | sed -e 's|#![ ]*||g' )")"
+  # strip the shebang from the real script as we do not want it
to be usable anyway
+  tail -n +2 $cmd > $cmd.real
+       cmdname=$(basename $cmd)
+       dirname=$(dirname $cmd)
+       cmdoptions=$@
+       if [ "${base_prefix}" != "" ]; then
+               relpath=`python3 -c "import os;
print(os.path.relpath('${D}${base_prefix}', '$dirname'))"`
+               cmdoptions=`echo $@ | sed -e
"s:${base_prefix}:\\$realdir/$relpath:g"`
+       fi
+       cat <$cmd
+#!/usr/bin/env bash
+realpath=\`readlink -fn \$0\`
+realdir=\`dirname \$realpath\`
+exec -a \$realdir/$cmdname $argument \$realdir/$cmdname.real
$cmdoptions "\$@"
+END
+       chmod +x $cmd
+}
+
 create_wrapper () {
        # Create a wrapper script where extra environmen

[OE-core] [PATCH v4 7/7] libcheck: Fix too long shebang for native case.

2022-06-14 Thread Paulo Neves

It requires a shebang wrapper due to the fact that awk interpreter
has an argument.
---
 meta/recipes-support/libcheck/libcheck_0.15.2.bb | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meta/recipes-support/libcheck/libcheck_0.15.2.bb 
b/meta/recipes-support/libcheck/libcheck_0.15.2.bb
index 188d689cc3..1393aa2a1c 100644
--- a/meta/recipes-support/libcheck/libcheck_0.15.2.bb
+++ b/meta/recipes-support/libcheck/libcheck_0.15.2.bb
@@ -23,6 +23,9 @@ CACHED_CONFIGUREVARS += "ac_cv_path_AWK_PATH=${bindir}/gawk"
 
 RREPLACES:${PN} = "check (<= 0.9.5)"
 
+do_install:append:class-native() {
+create_cmdline_shebang_wrapper ${D}${bindir}/checkmk
+}
 BBCLASSEXTEND = "native nativesdk"
 
 PACKAGES =+ "checkmk"
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166888): 
https://lists.openembedded.org/g/openembedded-core/message/166888
Mute This Topic: https://lists.openembedded.org/mt/91748693/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v4 5/7] oeqa/selftest: Test staged .la and .pc files

2022-06-14 Thread Paulo Neves

These files are checked by qa_check_staged but there was no
test cases for whether the tests actually worked. Now there
are.

Signed-off-by: Paulo Neves 
---
 .../sysroot-test/sysroot-la-test_1.0.bb   | 16 ++
 .../sysroot-test/sysroot-pc-test_1.0.bb   | 12 +++
 meta/lib/oeqa/selftest/cases/sysroot.py   | 32 +++
 3 files changed, 60 insertions(+)
 create mode 100644 
meta-selftest/recipes-test/sysroot-test/sysroot-la-test_1.0.bb
 create mode 100644 
meta-selftest/recipes-test/sysroot-test/sysroot-pc-test_1.0.bb

diff --git a/meta-selftest/recipes-test/sysroot-test/sysroot-la-test_1.0.bb 
b/meta-selftest/recipes-test/sysroot-test/sysroot-la-test_1.0.bb
new file mode 100644
index 00..21f06782fb
--- /dev/null
+++ b/meta-selftest/recipes-test/sysroot-test/sysroot-la-test_1.0.bb
@@ -0,0 +1,16 @@
+SUMMARY = "Produce a broken la file"
+LICENSE = "CLOSED"
+INHIBIT_DEFAULT_DEPS = "1"
+
+EXCLUDE_FROM_WORLD = "1"
+
+# remove-libtool.bbclass is inherited by default and removes all
+# .la files which for this test we specifically do not want.
+REMOVE_LIBTOOL_LA = "0"
+
+do_install() {
+install -d ${D}${libdir}/test/
+echo '${WORKDIR}' > ${D}${libdir}/test/la-test.la
+}
+
+BBCLASSEXTEND += "native"
diff --git a/meta-selftest/recipes-test/sysroot-test/sysroot-pc-test_1.0.bb 
b/meta-selftest/recipes-test/sysroot-test/sysroot-pc-test_1.0.bb
new file mode 100644
index 00..e748310fc4
--- /dev/null
+++ b/meta-selftest/recipes-test/sysroot-test/sysroot-pc-test_1.0.bb
@@ -0,0 +1,12 @@
+SUMMARY = "Produce a broken pc file"
+LICENSE = "CLOSED"
+INHIBIT_DEFAULT_DEPS = "1"
+
+EXCLUDE_FROM_WORLD = "1"
+
+do_install() {
+install -d ${D}${libdir}/test/
+echo '${WORKDIR}' > ${D}${libdir}/test/test.pc
+}
+
+BBCLASSEXTEND += "native"
diff --git a/meta/lib/oeqa/selftest/cases/sysroot.py 
b/meta/lib/oeqa/selftest/cases/sysroot.py
index 9457f1e3ac..588fc8c713 100644
--- a/meta/lib/oeqa/selftest/cases/sysroot.py
+++ b/meta/lib/oeqa/selftest/cases/sysroot.py
@@ -45,3 +45,35 @@ TESTSTRING:pn-sysroot-test-arch2 = "%s"
 expected = "maximum shebang size exceeded, the maximum size is 128. 
[shebang-size]"
 res = bitbake("sysroot-shebang-test-native -c populate_sysroot", 
ignore_status=True)
 self.assertTrue(expected in res.output, msg=res.output)
+
+def test_sysroot_la(self):
+"""
+    Summary:   Check that workdir paths are not contained in .la files.
+Expected:  Fail when a workdir path is found in the file content.
+Author:Paulo Neves 
+"""
+expected = "la-test.la failed sanity test (workdir) in path"
+
+res = bitbake("sysroot-la-test -c populate_sysroot", 
ignore_status=True)
+self.assertTrue(expected in res.output, msg=res.output)
+self.assertTrue('[la]' in res.output, msg=res.output)
+
+res = bitbake("sysroot-la-test-native -c populate_sysroot", 
ignore_status=True)
+self.assertTrue(expected in res.output, msg=res.output)
+self.assertTrue('[la]' in res.output, msg=res.output)
+
+def test_sysroot_pkgconfig(self):
+"""
+Summary:   Check that tmpdir paths are not contained in .pc files.
+Expected:  Fail when a tmpdir path is found in the file content.
+Author:Paulo Neves 
+"""
+expected = "test.pc failed sanity test (tmpdir) in path"
+
+res = bitbake("sysroot-pc-test -c populate_sysroot", 
ignore_status=True)
+self.assertTrue('[pkgconfig]' in res.output, msg=res.output)
+self.assertTrue(expected in res.output, msg=res.output)
+
+res = bitbake("sysroot-pc-test-native -c populate_sysroot", 
ignore_status=True)
+self.assertTrue(expected in res.output, msg=res.output)
+self.assertTrue('[pkgconfig]' in res.output, msg=res.output)
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166886): 
https://lists.openembedded.org/g/openembedded-core/message/166886
Mute This Topic: https://lists.openembedded.org/mt/91748691/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v4 6/7] utils: Add cmdline_shebang_wrapper util.

2022-06-14 Thread Paulo Neves

Useful to work around shebang relocation issues, where
shebangs are too long or have arguments in them, thus preventing them
from using the /usr/bin/env shebang.
---
 .../wrapper/cmdline-shebang-wrapper-test.bb   | 21 
 .../recipes-test/wrapper/files/test.awk   |  2 ++
 meta/classes/utils.bbclass| 34 +++
 meta/lib/oeqa/selftest/cases/wrapper.py   | 11 ++
 4 files changed, 68 insertions(+)
 create mode 100644 
meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb
 create mode 100644 meta-selftest/recipes-test/wrapper/files/test.awk
 create mode 100644 meta/lib/oeqa/selftest/cases/wrapper.py

diff --git a/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb 
b/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb
new file mode 100644
index 00..302eea8901
--- /dev/null
+++ b/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb
@@ -0,0 +1,21 @@
+FILESEXTRAPATHS:prepend := "${THISDIR}/files:"
+SUMMARY = "Check that create_cmdline_shebang works"
+LICENSE = "CLOSED"
+INHIBIT_DEFAULT_DEPS = "1"
+
+SRC_URI += "file://test.awk"
+
+EXCLUDE_FROM_WORLD = "1"
+do_install() {
+install -d ${D}${bindir}
+install -m 0755 ${WORKDIR}/test.awk ${D}${bindir}/test
+sed -i -e 's|@AWK_BIN@|${bindir}/awk|g' ${D}${bindir}/test
+create_cmdline_shebang_wrapper ${D}${bindir}/test
+if [ $(${D}${bindir}/test) != "Don't Panic!" ]; then
+bbfatal "Wrapper is broken"
+else
+bbnote "Wrapper is good"
+fi
+}
+
+BBCLASSEXTEND = "native"
diff --git a/meta-selftest/recipes-test/wrapper/files/test.awk 
b/meta-selftest/recipes-test/wrapper/files/test.awk
new file mode 100644
index 00..91429197b1
--- /dev/null
+++ b/meta-selftest/recipes-test/wrapper/files/test.awk
@@ -0,0 +1,2 @@
+#! @AWK_BIN@ -f
+BEGIN { print "Don't Panic!" }
diff --git a/meta/classes/utils.bbclass b/meta/classes/utils.bbclass
index b4eb3d38ab..b617632d9f 100644
--- a/meta/classes/utils.bbclass
+++ b/meta/classes/utils.bbclass
@@ -184,6 +184,40 @@ END
chmod +x $cmd
 }
 
+create_cmdline_shebang_wrapper () {
+   # Create a wrapper script where commandline options are needed
+   #
+   # These are useful to work around shebang relocation issues, where 
shebangs are too
+  # long or have arguments in them, thus preventing them from using the 
/usr/bin/env
+   # shebang
+   #
+   # Usage: create_cmdline_wrapper FILENAME 
+
+   cmd=$1
+   shift
+
+   echo "Generating wrapper script for $cmd"
+
+  # Strip #! and get remaining interpreter + arg
+  argument="$(basename "$(head -n1 $cmd | sed -e 's|#![ ]*||g' )")"
+  # strip the shebang from the real script as we do not want it to be usable 
anyway
+  tail -n +2 $cmd > $cmd.real
+   cmdname=$(basename $cmd)
+   dirname=$(dirname $cmd)
+   cmdoptions=$@
+   if [ "${base_prefix}" != "" ]; then
+   relpath=`python3 -c "import os; 
print(os.path.relpath('${D}${base_prefix}', '$dirname'))"`
+   cmdoptions=`echo $@ | sed -e 
"s:${base_prefix}:\\$realdir/$relpath:g"`
+   fi
+   cat <$cmd
+#!/usr/bin/env bash
+realpath=\`readlink -fn \$0\`
+realdir=\`dirname \$realpath\`
+exec -a \$realdir/$cmdname $argument \$realdir/$cmdname.real $cmdoptions "\$@"
+END
+   chmod +x $cmd
+}
+
 create_wrapper () {
# Create a wrapper script where extra environment variables are needed
#
diff --git a/meta/lib/oeqa/selftest/cases/wrapper.py 
b/meta/lib/oeqa/selftest/cases/wrapper.py
new file mode 100644
index 00..6de63310c0
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/wrapper.py
@@ -0,0 +1,11 @@
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import bitbake
+
+class WrapperTests(OESelftestTestCase):
+def test_shebang_wrapper(self):
+"""
+Summary:   Build a recipe which will fail if the 
cmdline_shebang_wrapper function is defective.
+Expected:  Exit status to be 0.
+Author:Paulo Neves 
+"""
+res = bitbake("cmdline-shebang-wrapper-test -c install", 
ignore_status=False)
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166887): 
https://lists.openembedded.org/g/openembedded-core/message/166887
Mute This Topic: https://lists.openembedded.org/mt/91748692/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v4 4/7] oeqa/selftest: Add test for shebang overflow

2022-06-14 Thread Paulo Neves

Make sure we do not stage any executable with a bigger shebang
than 128. Fixes [1]

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11053

Signed-off-by: Paulo Neves 
---
 .../sysroot-test/sysroot-shebang-test_1.0.bb | 12 
 meta/lib/oeqa/selftest/cases/sysroot.py  | 10 ++
 2 files changed, 22 insertions(+)
 create mode 100644 
meta-selftest/recipes-test/sysroot-test/sysroot-shebang-test_1.0.bb

diff --git 
a/meta-selftest/recipes-test/sysroot-test/sysroot-shebang-test_1.0.bb 
b/meta-selftest/recipes-test/sysroot-test/sysroot-shebang-test_1.0.bb
new file mode 100644
index 00..6c834be897
--- /dev/null
+++ b/meta-selftest/recipes-test/sysroot-test/sysroot-shebang-test_1.0.bb
@@ -0,0 +1,12 @@
+SUMMARY = "Check that shebang does not exceed 128 characters"
+LICENSE = "CLOSED"
+INHIBIT_DEFAULT_DEPS = "1"
+
+EXCLUDE_FROM_WORLD = "1"
+do_install() {
+install -d ${D}${bindir}
+echo 
'#!BiM3cnVd1Amtv6PG+FynrQiVMbZnX5ELgF21q3EkuB+44JEGWtq8TvBJ7EGidfVs3eR3wVOUbLnjYDlKUWcm7YC/ute7f+KDHbwxziRUSUBZAUqgjiQdfQ0HnxajI0ozbM863E9JV9k13yZKYfh9/zR77Y6Dl4Dd3zOWS75LSpkAXV'
 > ${D}${bindir}/max-shebang
+chmod 755 ${D}${bindir}/max-shebang
+}
+
+BBCLASSEXTEND = "native"
diff --git a/meta/lib/oeqa/selftest/cases/sysroot.py 
b/meta/lib/oeqa/selftest/cases/sysroot.py
index 315d1a61c2..9457f1e3ac 100644
--- a/meta/lib/oeqa/selftest/cases/sysroot.py
+++ b/meta/lib/oeqa/selftest/cases/sysroot.py
@@ -35,3 +35,13 @@ TESTSTRING:pn-sysroot-test-arch1 = "%s"
 TESTSTRING:pn-sysroot-test-arch2 = "%s"
 """ % (uuid1, uuid2))
 bitbake("sysroot-test")
+
+def test_sysroot_max_shebang(self):
+"""
+Summary:   Check max shebang triggers. To confirm [YOCTO #11053] is 
closed.
+Expected:  Fail when a shebang bigger than the max shebang-size is 
reached.
+Author:Paulo Neves 
+"""
+expected = "maximum shebang size exceeded, the maximum size is 128. 
[shebang-size]"
+res = bitbake("sysroot-shebang-test-native -c populate_sysroot", 
ignore_status=True)
+self.assertTrue(expected in res.output, msg=res.output)
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166885): 
https://lists.openembedded.org/g/openembedded-core/message/166885
Mute This Topic: https://lists.openembedded.org/mt/91748690/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v4 3/7] insane.bbclass: Make do_qa_staging check shebangs

2022-06-14 Thread Paulo Neves

As reported in the bug report [1], there was no check for shebang
sizes on native scripts and now this is fixed.

The path scope of the qa_staging was increased from just checking
libdir to all the relevant SYSROOT_DIRS.

It is possible to skip this check through INSANE_SKIP.

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11053

Signed-off-by: Paulo Neves 
---
 meta/classes/insane.bbclass | 16 +++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index 9ca84bace9..b2951a48fe 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -630,6 +630,11 @@ def qa_check_staged(path,d):
 bb.note("Recipe %s skipping qa checking: pkgconfig" % d.getVar('PN'))
 skip_pkgconfig = True
 
+skip_shebang_size = False
+if 'shebang-size' in skip:
+bb.note("Recipe %s skipping qa checkking: shebang-size" % 
d.getVar('PN'))
+skip_shebang_size = True
+
 # find all .la and .pc files
 # read the content
 # and check for stuff that looks wrong
@@ -651,6 +656,13 @@ def qa_check_staged(path,d):
 error_msg = "%s failed sanity test (tmpdir) in path 
%s" % (file,root)
 oe.qa.handle_error("pkgconfig", error_msg, d)
 
+if not skip_shebang_size:
+errors = {}
+package_qa_check_shebang_size(path, "", d, None, errors)
+for e in errors:
+oe.qa.handle_error(e, errors[e], d)
+
+
 # Run all package-wide warnfuncs and errorfuncs
 def package_qa_package(warnfuncs, errorfuncs, package, d):
 warnings = {}
@@ -1139,7 +1151,9 @@ addtask do_package_qa_setscene
 
 python do_qa_staging() {
 bb.note("QA checking staging")
-qa_check_staged(d.expand('${SYSROOT_DESTDIR}${libdir}'), d)
+sysroot_destdir = d.expand('${SYSROOT_DESTDIR}')
+for sysroot_dir in d.expand('${SYSROOT_DIRS}').split():
+qa_check_staged(sysroot_destdir + sysroot_dir, d)
 oe.qa.exit_with_message_if_errors("QA staging was broken by the package 
built above", d)
 }
 
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166884): 
https://lists.openembedded.org/g/openembedded-core/message/166884
Mute This Topic: https://lists.openembedded.org/mt/91748689/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v4 1/7] python: Avoid shebang overflow on python-config.py

2022-06-14 Thread Paulo Neves

The native path may be too big, leading to shebang
overflow. Just use the #!/usr/bin/env python3.

Signed-off-by: Paulo Neves 
---
 ...shebang-overflow-on-python-config.py.patch | 33 +++
 .../recipes-devtools/python/python3_3.10.4.bb |  1 +
 2 files changed, 34 insertions(+)
 create mode 100644 
meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch

diff --git 
a/meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch
 
b/meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch
new file mode 100644
index 00..921da8de7c
--- /dev/null
+++ 
b/meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch
@@ -0,0 +1,33 @@
+From f0c9dec63d452a7cd1e15ea653f4aced281f021c Mon Sep 17 00:00:00 2001
+From: Paulo Neves 
+Date: Tue, 7 Jun 2022 16:16:41 +0200
+Subject: [PATCH 1/1] Avoid shebang overflow on python-config.py
+
+The whole native path may be too big, leading to shebang
+overflow. Let's just use the env shebang.
+
+Denial reason: [1]
+
+Upstream-Status: Denied [distribution]
+
+[1] https://github.com/python/cpython/pull/93760#pullrequestreview-1005365737
+---
+ Makefile.pre.in | 2 ++
+ 1 file changed, 2 insertions(+)
+
+diff --git a/Makefile.pre.in b/Makefile.pre.in
+index 
f0aedb76cb58999427804255da56fa53284d7032..dd88e43114730f7681715777cc76dabb31113176
 100644
+--- a/Makefile.pre.in
 b/Makefile.pre.in
+@@ -1638,6 +1638,8 @@ python-config: $(srcdir)/Misc/python-config.in 
Misc/python-config.sh
+   @ # Substitution happens here, as the completely-expanded BINDIR
+   @ # is not available in configure
+   sed -e "s,@EXENAME@,$(BINDIR)/python$(LDVERSION)$(EXE)," < 
$(srcdir)/Misc/python-config.in >python-config.py
++  @ # Otherwise we might get huge shebangs with native paths
++  sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' python-config.py
+   @ # Replace makefile compat. variable references with shell script 
compat. ones; $(VAR) -> ${VAR}
+   LC_ALL=C sed -e 's,\$$(\([A-Za-z0-9_]*\)),\$$\{\1\},g' < 
Misc/python-config.sh >python-config
+   @  # In OpenEmbedded, always use the python version of the script, the 
shell
+-- 
+2.25.1
+
diff --git a/meta/recipes-devtools/python/python3_3.10.4.bb 
b/meta/recipes-devtools/python/python3_3.10.4.bb
index 6bd3a6aba8..357025f856 100644
--- a/meta/recipes-devtools/python/python3_3.10.4.bb
+++ b/meta/recipes-devtools/python/python3_3.10.4.bb
@@ -34,6 +34,7 @@ SRC_URI = 
"http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \

file://0001-Lib-pty.py-handle-stdin-I-O-errors-same-way-as-maste.patch \

file://0001-setup.py-Do-not-detect-multiarch-paths-when-cross-co.patch \
file://deterministic_imports.patch \
+   file://0001-Avoid-shebang-overflow-on-python-config.py.patch \
"
 
 SRC_URI:append:class-native = " \
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166882): 
https://lists.openembedded.org/g/openembedded-core/message/166882
Mute This Topic: https://lists.openembedded.org/mt/91748687/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v4 2/7] gtk-doc: Fix potential shebang overflow on gtkdoc-mkhtml2

2022-06-14 Thread Paulo Neves

A native path can cause a shebang overflow on gtkdoc-mkhtml.
Replace it with /usr/bin/env.

Signed-off-by: Paulo Neves 
---
 meta/recipes-gnome/gtk-doc/gtk-doc_1.33.2.bb | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-gnome/gtk-doc/gtk-doc_1.33.2.bb 
b/meta/recipes-gnome/gtk-doc/gtk-doc_1.33.2.bb
index 150d2c0b23..392913fcc6 100644
--- a/meta/recipes-gnome/gtk-doc/gtk-doc_1.33.2.bb
+++ b/meta/recipes-gnome/gtk-doc/gtk-doc_1.33.2.bb
@@ -43,6 +43,7 @@ do_install:append () {
 ${datadir}/gtk-doc/python/gtkdoc/config.py; do
 sed -e 
's,${RECIPE_SYSROOT_NATIVE}/usr/bin/pkg-config,${bindir}/pkg-config,' \
 -e 's,${HOSTTOOLS_DIR}/python3,${bindir}/python3,' \
+-e '1s|^#!.*|#!/usr/bin/env python3|' \
 -i ${D}$fn
 done
 }
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166883): 
https://lists.openembedded.org/g/openembedded-core/message/166883
Mute This Topic: https://lists.openembedded.org/mt/91748688/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v2 4/5] oeqa/selftest: Add test for shebang overflow

2022-06-13 Thread Paulo Neves


Hey Luca,

I sent a new patchset and from my tests it all should be good now.

Paulo Neves

On 6/8/22 17:06, Luca Ceresoli wrote:

Hi Paulo,

On Wed, 8 Jun 2022 14:53:05 +0200
"Luca Ceresoli via lists.openembedded.org"
 wrote:


Hi Paulo,

On Tue,  7 Jun 2022 17:11:22 +0200
"Paulo Neves"  wrote:


Make sure we do not stage any executable with a bigger shebang
than 128. Fixes [1]

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11053

Signed-off-by: Paulo Neves 

This check seems to be working very well!! It triggered a huge amount
of build failures on the autobuilders due to libcheck having a shebang
too long in the checkmk script, e.g.:

#! 
/home/pokybuild/yocto-worker/genericx86-64-alt/build/build/tmp/work/x86_64-linux/libcheck-native/0.15.2-r0/recipe-sysroot-native/usr/bin/gawk
 -f

Here are a few logs:

https://autobuilder.yoctoproject.org/typhoon/#/builders/65/builds/5367/steps/14/logs/errors
https://autobuilder.yoctoproject.org/typhoon/#/builders/65/builds/5367/steps/11/logs/errors
https://autobuilder.yoctoproject.org/typhoon/#/builders/20/builds/5689/steps/11/logs/errors
https://autobuilder.yoctoproject.org/typhoon/#/builders/37/builds/5340/steps/12/logs/errors

It would be great if you could add another patch to your series to fix
libcheck, and also to do 'bitbake world' to test as many packages as
possible before discovering from the autobuilders.

Here are more failures:

https://autobuilder.yoctoproject.org/typhoon/#/builders/20/builds/5689/steps/12/logs/errors
https://autobuilder.yoctoproject.org/typhoon/#/builders/20/builds/5689/steps/12/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/62/builds/5340/steps/12/logs/errors
https://autobuilder.yoctoproject.org/typhoon/#/builders/62/builds/5340/steps/12/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/37/builds/5340/steps/13/logs/errors
https://autobuilder.yoctoproject.org/typhoon/#/builders/37/builds/5340/steps/13/logs/stdio

This time the error is:

stdio: ERROR: core-image-sato-1.0-r0 do_testsdk: The toolchain <...> is not built. 
Build it before running the tests: 'bitbake  -c populate_sdk' .

I'm not sure exactly how your code triggers such error, but it appeared
when testing on the autobuilders with this patch series and disappeared
when I removed only these 5 patches, thus it seems related.




-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166837): 
https://lists.openembedded.org/g/openembedded-core/message/166837
Mute This Topic: https://lists.openembedded.org/mt/91602895/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v3 1/7] python: Avoid shebang overflow on python-config.py

2022-06-13 Thread Paulo Neves


As requested
https://github.com/python/cpython/pull/93760

Paulo Neves

On 6/13/22 09:30, Alexander Kanavin wrote:

As requested, please submit upstream.

Alex

On Fri, 10 Jun 2022 at 23:43, Paulo Neves  wrote:

The native path may be too big, leading to shebang
overflow. Just use the #!/usr/bin/env python3.

Signed-off-by: Paulo Neves 
---
  ...shebang-overflow-on-python-config.py.patch | 29 +++
  .../recipes-devtools/python/python3_3.10.4.bb |  1 +
  2 files changed, 30 insertions(+)
  create mode 100644 
meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch

diff --git 
a/meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch
 
b/meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch
new file mode 100644
index 00..9f23278a30
--- /dev/null
+++ 
b/meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch
@@ -0,0 +1,29 @@
+From f0c9dec63d452a7cd1e15ea653f4aced281f021c Mon Sep 17 00:00:00 2001
+From: Paulo Neves 
+Date: Tue, 7 Jun 2022 16:16:41 +0200
+Subject: [PATCH 1/1] Avoid shebang overflow on python-config.py
+
+The whole native path may be too big, leading to shebang
+overflow. Let's just use the env shebang.
+
+Upstream-Status: Inappropriate [distribution]
+---
+ Makefile.pre.in | 2 ++
+ 1 file changed, 2 insertions(+)
+
+diff --git a/Makefile.pre.in b/Makefile.pre.in
+index 
f0aedb76cb58999427804255da56fa53284d7032..dd88e43114730f7681715777cc76dabb31113176
 100644
+--- a/Makefile.pre.in
 b/Makefile.pre.in
+@@ -1638,6 +1638,8 @@ python-config: $(srcdir)/Misc/python-config.in 
Misc/python-config.sh
+   @ # Substitution happens here, as the completely-expanded BINDIR
+   @ # is not available in configure
+   sed -e "s,@EXENAME@,$(BINDIR)/python$(LDVERSION)$(EXE)," < 
$(srcdir)/Misc/python-config.in >python-config.py
++  @ # Otherwise we might get huge shebangs with native paths
++  sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' python-config.py
+   @ # Replace makefile compat. variable references with shell script compat. 
ones; $(VAR) -> ${VAR}
+   LC_ALL=C sed -e 's,\$$(\([A-Za-z0-9_]*\)),\$$\{\1\},g' < 
Misc/python-config.sh >python-config
+   @  # In OpenEmbedded, always use the python version of the script, the 
shell
+--
+2.25.1
+
diff --git a/meta/recipes-devtools/python/python3_3.10.4.bb 
b/meta/recipes-devtools/python/python3_3.10.4.bb
index 6bd3a6aba8..357025f856 100644
--- a/meta/recipes-devtools/python/python3_3.10.4.bb
+++ b/meta/recipes-devtools/python/python3_3.10.4.bb
@@ -34,6 +34,7 @@ SRC_URI = 
"http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \
 
file://0001-Lib-pty.py-handle-stdin-I-O-errors-same-way-as-maste.patch \
 
file://0001-setup.py-Do-not-detect-multiarch-paths-when-cross-co.patch \
 file://deterministic_imports.patch \
+   file://0001-Avoid-shebang-overflow-on-python-config.py.patch \
 "

  SRC_URI:append:class-native = " \
--
2.25.1







-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166836): 
https://lists.openembedded.org/g/openembedded-core/message/166836
Mute This Topic: https://lists.openembedded.org/mt/91678611/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v3 7/7] libcheck: Fix too long shebang for native case.

2022-06-10 Thread Paulo Neves

It requires a shebang wrapper due to the fact that awk interpreter
has an argument.
---
 meta/recipes-support/libcheck/libcheck_0.15.2.bb | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meta/recipes-support/libcheck/libcheck_0.15.2.bb 
b/meta/recipes-support/libcheck/libcheck_0.15.2.bb
index 188d689cc3..1393aa2a1c 100644
--- a/meta/recipes-support/libcheck/libcheck_0.15.2.bb
+++ b/meta/recipes-support/libcheck/libcheck_0.15.2.bb
@@ -23,6 +23,9 @@ CACHED_CONFIGUREVARS += "ac_cv_path_AWK_PATH=${bindir}/gawk"
 
 RREPLACES:${PN} = "check (<= 0.9.5)"
 
+do_install:append:class-native() {
+create_cmdline_shebang_wrapper ${D}${bindir}/checkmk
+}
 BBCLASSEXTEND = "native nativesdk"
 
 PACKAGES =+ "checkmk"
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166819): 
https://lists.openembedded.org/g/openembedded-core/message/166819
Mute This Topic: https://lists.openembedded.org/mt/91678620/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v3 6/7] utils: Add cmdline_shebang_wrapper util.

2022-06-10 Thread Paulo Neves

Useful to work around shebang relocation issues, where
shebangs are too long or have arguments in them, thus preventing them
from using the /usr/bin/env shebang.
---
 .../wrapper/cmdline-shebang-wrapper-test.bb   | 21 
 .../recipes-test/wrapper/files/test.awk   |  2 ++
 meta/classes/utils.bbclass| 34 +++
 meta/lib/oeqa/selftest/cases/wrapper.py   | 11 ++
 4 files changed, 68 insertions(+)
 create mode 100644 
meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb
 create mode 100644 meta-selftest/recipes-test/wrapper/files/test.awk
 create mode 100644 meta/lib/oeqa/selftest/cases/wrapper.py

diff --git a/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb 
b/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb
new file mode 100644
index 00..302eea8901
--- /dev/null
+++ b/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb
@@ -0,0 +1,21 @@
+FILESEXTRAPATHS:prepend := "${THISDIR}/files:"
+SUMMARY = "Check that create_cmdline_shebang works"
+LICENSE = "CLOSED"
+INHIBIT_DEFAULT_DEPS = "1"
+
+SRC_URI += "file://test.awk"
+
+EXCLUDE_FROM_WORLD = "1"
+do_install() {
+install -d ${D}${bindir}
+install -m 0755 ${WORKDIR}/test.awk ${D}${bindir}/test
+sed -i -e 's|@AWK_BIN@|${bindir}/awk|g' ${D}${bindir}/test
+create_cmdline_shebang_wrapper ${D}${bindir}/test
+if [ $(${D}${bindir}/test) != "Don't Panic!" ]; then
+bbfatal "Wrapper is broken"
+else
+bbnote "Wrapper is good"
+fi
+}
+
+BBCLASSEXTEND = "native"
diff --git a/meta-selftest/recipes-test/wrapper/files/test.awk 
b/meta-selftest/recipes-test/wrapper/files/test.awk
new file mode 100644
index 00..91429197b1
--- /dev/null
+++ b/meta-selftest/recipes-test/wrapper/files/test.awk
@@ -0,0 +1,2 @@
+#! @AWK_BIN@ -f
+BEGIN { print "Don't Panic!" }
diff --git a/meta/classes/utils.bbclass b/meta/classes/utils.bbclass
index b4eb3d38ab..b617632d9f 100644
--- a/meta/classes/utils.bbclass
+++ b/meta/classes/utils.bbclass
@@ -184,6 +184,40 @@ END
chmod +x $cmd
 }
 
+create_cmdline_shebang_wrapper () {
+   # Create a wrapper script where commandline options are needed
+   #
+   # These are useful to work around shebang relocation issues, where 
shebangs are too
+  # long or have arguments in them, thus preventing them from using the 
/usr/bin/env
+   # shebang
+   #
+   # Usage: create_cmdline_wrapper FILENAME 
+
+   cmd=$1
+   shift
+
+   echo "Generating wrapper script for $cmd"
+
+  # Strip #! and get remaining interpreter + arg
+  argument="$(basename "$(head -n1 $cmd | sed -e 's|#![ ]*||g' )")"
+  # strip the shebang from the real script as we do not want it to be usable 
anyway
+  tail -n +2 $cmd > $cmd.real
+   cmdname=$(basename $cmd)
+   dirname=$(dirname $cmd)
+   cmdoptions=$@
+   if [ "${base_prefix}" != "" ]; then
+   relpath=`python3 -c "import os; 
print(os.path.relpath('${D}${base_prefix}', '$dirname'))"`
+   cmdoptions=`echo $@ | sed -e 
"s:${base_prefix}:\\$realdir/$relpath:g"`
+   fi
+   cat <$cmd
+#!/usr/bin/env bash
+realpath=\`readlink -fn \$0\`
+realdir=\`dirname \$realpath\`
+exec -a \$realdir/$cmdname $argument \$realdir/$cmdname.real $cmdoptions "\$@"
+END
+   chmod +x $cmd
+}
+
 create_wrapper () {
# Create a wrapper script where extra environment variables are needed
#
diff --git a/meta/lib/oeqa/selftest/cases/wrapper.py 
b/meta/lib/oeqa/selftest/cases/wrapper.py
new file mode 100644
index 00..6de63310c0
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/wrapper.py
@@ -0,0 +1,11 @@
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import bitbake
+
+class WrapperTests(OESelftestTestCase):
+def test_shebang_wrapper(self):
+"""
+Summary:   Build a recipe which will fail if the 
cmdline_shebang_wrapper function is defective.
+Expected:  Exit status to be 0.
+Author:Paulo Neves 
+"""
+res = bitbake("cmdline-shebang-wrapper-test -c install", 
ignore_status=False)
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166818): 
https://lists.openembedded.org/g/openembedded-core/message/166818
Mute This Topic: https://lists.openembedded.org/mt/91678619/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v3 5/7] oeqa/selftest: Test staged .la and .pc files

2022-06-10 Thread Paulo Neves

These files are checked by qa_check_staged but there was no
test cases for whether the tests actually worked. Now there
are.

Signed-off-by: Paulo Neves 
---
 .../sysroot-test/sysroot-la-test_1.0.bb   | 16 ++
 .../sysroot-test/sysroot-pc-test_1.0.bb   | 12 +++
 meta/lib/oeqa/selftest/cases/sysroot.py   | 32 +++
 3 files changed, 60 insertions(+)
 create mode 100644 
meta-selftest/recipes-test/sysroot-test/sysroot-la-test_1.0.bb
 create mode 100644 
meta-selftest/recipes-test/sysroot-test/sysroot-pc-test_1.0.bb

diff --git a/meta-selftest/recipes-test/sysroot-test/sysroot-la-test_1.0.bb 
b/meta-selftest/recipes-test/sysroot-test/sysroot-la-test_1.0.bb
new file mode 100644
index 00..21f06782fb
--- /dev/null
+++ b/meta-selftest/recipes-test/sysroot-test/sysroot-la-test_1.0.bb
@@ -0,0 +1,16 @@
+SUMMARY = "Produce a broken la file"
+LICENSE = "CLOSED"
+INHIBIT_DEFAULT_DEPS = "1"
+
+EXCLUDE_FROM_WORLD = "1"
+
+# remove-libtool.bbclass is inherited by default and removes all
+# .la files which for this test we specifically do not want.
+REMOVE_LIBTOOL_LA = "0"
+
+do_install() {
+install -d ${D}${libdir}/test/
+echo '${WORKDIR}' > ${D}${libdir}/test/la-test.la
+}
+
+BBCLASSEXTEND += "native"
diff --git a/meta-selftest/recipes-test/sysroot-test/sysroot-pc-test_1.0.bb 
b/meta-selftest/recipes-test/sysroot-test/sysroot-pc-test_1.0.bb
new file mode 100644
index 00..e748310fc4
--- /dev/null
+++ b/meta-selftest/recipes-test/sysroot-test/sysroot-pc-test_1.0.bb
@@ -0,0 +1,12 @@
+SUMMARY = "Produce a broken pc file"
+LICENSE = "CLOSED"
+INHIBIT_DEFAULT_DEPS = "1"
+
+EXCLUDE_FROM_WORLD = "1"
+
+do_install() {
+install -d ${D}${libdir}/test/
+echo '${WORKDIR}' > ${D}${libdir}/test/test.pc
+}
+
+BBCLASSEXTEND += "native"
diff --git a/meta/lib/oeqa/selftest/cases/sysroot.py 
b/meta/lib/oeqa/selftest/cases/sysroot.py
index 9457f1e3ac..588fc8c713 100644
--- a/meta/lib/oeqa/selftest/cases/sysroot.py
+++ b/meta/lib/oeqa/selftest/cases/sysroot.py
@@ -45,3 +45,35 @@ TESTSTRING:pn-sysroot-test-arch2 = "%s"
 expected = "maximum shebang size exceeded, the maximum size is 128. 
[shebang-size]"
 res = bitbake("sysroot-shebang-test-native -c populate_sysroot", 
ignore_status=True)
 self.assertTrue(expected in res.output, msg=res.output)
+
+def test_sysroot_la(self):
+"""
+    Summary:   Check that workdir paths are not contained in .la files.
+Expected:  Fail when a workdir path is found in the file content.
+Author:Paulo Neves 
+"""
+expected = "la-test.la failed sanity test (workdir) in path"
+
+res = bitbake("sysroot-la-test -c populate_sysroot", 
ignore_status=True)
+self.assertTrue(expected in res.output, msg=res.output)
+self.assertTrue('[la]' in res.output, msg=res.output)
+
+res = bitbake("sysroot-la-test-native -c populate_sysroot", 
ignore_status=True)
+self.assertTrue(expected in res.output, msg=res.output)
+self.assertTrue('[la]' in res.output, msg=res.output)
+
+def test_sysroot_pkgconfig(self):
+"""
+Summary:   Check that tmpdir paths are not contained in .pc files.
+Expected:  Fail when a tmpdir path is found in the file content.
+Author:Paulo Neves 
+"""
+expected = "test.pc failed sanity test (tmpdir) in path"
+
+res = bitbake("sysroot-pc-test -c populate_sysroot", 
ignore_status=True)
+self.assertTrue('[pkgconfig]' in res.output, msg=res.output)
+self.assertTrue(expected in res.output, msg=res.output)
+
+res = bitbake("sysroot-pc-test-native -c populate_sysroot", 
ignore_status=True)
+self.assertTrue(expected in res.output, msg=res.output)
+self.assertTrue('[pkgconfig]' in res.output, msg=res.output)
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166817): 
https://lists.openembedded.org/g/openembedded-core/message/166817
Mute This Topic: https://lists.openembedded.org/mt/91678618/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v3 4/7] oeqa/selftest: Add test for shebang overflow

2022-06-10 Thread Paulo Neves

Make sure we do not stage any executable with a bigger shebang
than 128. Fixes [1]

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11053

Signed-off-by: Paulo Neves 
---
 .../sysroot-test/sysroot-shebang-test_1.0.bb | 12 
 meta/lib/oeqa/selftest/cases/sysroot.py  | 10 ++
 2 files changed, 22 insertions(+)
 create mode 100644 
meta-selftest/recipes-test/sysroot-test/sysroot-shebang-test_1.0.bb

diff --git 
a/meta-selftest/recipes-test/sysroot-test/sysroot-shebang-test_1.0.bb 
b/meta-selftest/recipes-test/sysroot-test/sysroot-shebang-test_1.0.bb
new file mode 100644
index 00..6c834be897
--- /dev/null
+++ b/meta-selftest/recipes-test/sysroot-test/sysroot-shebang-test_1.0.bb
@@ -0,0 +1,12 @@
+SUMMARY = "Check that shebang does not exceed 128 characters"
+LICENSE = "CLOSED"
+INHIBIT_DEFAULT_DEPS = "1"
+
+EXCLUDE_FROM_WORLD = "1"
+do_install() {
+install -d ${D}${bindir}
+echo 
'#!BiM3cnVd1Amtv6PG+FynrQiVMbZnX5ELgF21q3EkuB+44JEGWtq8TvBJ7EGidfVs3eR3wVOUbLnjYDlKUWcm7YC/ute7f+KDHbwxziRUSUBZAUqgjiQdfQ0HnxajI0ozbM863E9JV9k13yZKYfh9/zR77Y6Dl4Dd3zOWS75LSpkAXV'
 > ${D}${bindir}/max-shebang
+chmod 755 ${D}${bindir}/max-shebang
+}
+
+BBCLASSEXTEND = "native"
diff --git a/meta/lib/oeqa/selftest/cases/sysroot.py 
b/meta/lib/oeqa/selftest/cases/sysroot.py
index 315d1a61c2..9457f1e3ac 100644
--- a/meta/lib/oeqa/selftest/cases/sysroot.py
+++ b/meta/lib/oeqa/selftest/cases/sysroot.py
@@ -35,3 +35,13 @@ TESTSTRING:pn-sysroot-test-arch1 = "%s"
 TESTSTRING:pn-sysroot-test-arch2 = "%s"
 """ % (uuid1, uuid2))
 bitbake("sysroot-test")
+
+def test_sysroot_max_shebang(self):
+"""
+Summary:   Check max shebang triggers. To confirm [YOCTO #11053] is 
closed.
+Expected:  Fail when a shebang bigger than the max shebang-size is 
reached.
+Author:Paulo Neves 
+"""
+expected = "maximum shebang size exceeded, the maximum size is 128. 
[shebang-size]"
+res = bitbake("sysroot-shebang-test-native -c populate_sysroot", 
ignore_status=True)
+self.assertTrue(expected in res.output, msg=res.output)
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166816): 
https://lists.openembedded.org/g/openembedded-core/message/166816
Mute This Topic: https://lists.openembedded.org/mt/91678616/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v3 3/7] insane.bbclass: Make do_qa_staging check shebangs

2022-06-10 Thread Paulo Neves

As reported in the bug report [1], there was no check for shebang
sizes on native scripts and now this is fixed.

The path scope of the qa_staging was increased from just checking
libdir to all the relevant SYSROOT_DIRS.

It is possible to skip this check through INSANE_SKIP.

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11053

Signed-off-by: Paulo Neves 
---
 meta/classes/insane.bbclass | 16 +++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index 9ca84bace9..b2951a48fe 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -630,6 +630,11 @@ def qa_check_staged(path,d):
 bb.note("Recipe %s skipping qa checking: pkgconfig" % d.getVar('PN'))
 skip_pkgconfig = True
 
+skip_shebang_size = False
+if 'shebang-size' in skip:
+bb.note("Recipe %s skipping qa checkking: shebang-size" % 
d.getVar('PN'))
+skip_shebang_size = True
+
 # find all .la and .pc files
 # read the content
 # and check for stuff that looks wrong
@@ -651,6 +656,13 @@ def qa_check_staged(path,d):
 error_msg = "%s failed sanity test (tmpdir) in path 
%s" % (file,root)
 oe.qa.handle_error("pkgconfig", error_msg, d)
 
+if not skip_shebang_size:
+errors = {}
+package_qa_check_shebang_size(path, "", d, None, errors)
+for e in errors:
+oe.qa.handle_error(e, errors[e], d)
+
+
 # Run all package-wide warnfuncs and errorfuncs
 def package_qa_package(warnfuncs, errorfuncs, package, d):
 warnings = {}
@@ -1139,7 +1151,9 @@ addtask do_package_qa_setscene
 
 python do_qa_staging() {
 bb.note("QA checking staging")
-qa_check_staged(d.expand('${SYSROOT_DESTDIR}${libdir}'), d)
+sysroot_destdir = d.expand('${SYSROOT_DESTDIR}')
+for sysroot_dir in d.expand('${SYSROOT_DIRS}').split():
+qa_check_staged(sysroot_destdir + sysroot_dir, d)
 oe.qa.exit_with_message_if_errors("QA staging was broken by the package 
built above", d)
 }
 
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166815): 
https://lists.openembedded.org/g/openembedded-core/message/166815
Mute This Topic: https://lists.openembedded.org/mt/91678615/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v3 1/7] python: Avoid shebang overflow on python-config.py

2022-06-10 Thread Paulo Neves

The native path may be too big, leading to shebang
overflow. Just use the #!/usr/bin/env python3.

Signed-off-by: Paulo Neves 
---
 ...shebang-overflow-on-python-config.py.patch | 29 +++
 .../recipes-devtools/python/python3_3.10.4.bb |  1 +
 2 files changed, 30 insertions(+)
 create mode 100644 
meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch

diff --git 
a/meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch
 
b/meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch
new file mode 100644
index 00..9f23278a30
--- /dev/null
+++ 
b/meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch
@@ -0,0 +1,29 @@
+From f0c9dec63d452a7cd1e15ea653f4aced281f021c Mon Sep 17 00:00:00 2001
+From: Paulo Neves 
+Date: Tue, 7 Jun 2022 16:16:41 +0200
+Subject: [PATCH 1/1] Avoid shebang overflow on python-config.py
+
+The whole native path may be too big, leading to shebang
+overflow. Let's just use the env shebang.
+
+Upstream-Status: Inappropriate [distribution]
+---
+ Makefile.pre.in | 2 ++
+ 1 file changed, 2 insertions(+)
+
+diff --git a/Makefile.pre.in b/Makefile.pre.in
+index 
f0aedb76cb58999427804255da56fa53284d7032..dd88e43114730f7681715777cc76dabb31113176
 100644
+--- a/Makefile.pre.in
 b/Makefile.pre.in
+@@ -1638,6 +1638,8 @@ python-config: $(srcdir)/Misc/python-config.in 
Misc/python-config.sh
+   @ # Substitution happens here, as the completely-expanded BINDIR
+   @ # is not available in configure
+   sed -e "s,@EXENAME@,$(BINDIR)/python$(LDVERSION)$(EXE)," < 
$(srcdir)/Misc/python-config.in >python-config.py
++  @ # Otherwise we might get huge shebangs with native paths
++  sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' python-config.py
+   @ # Replace makefile compat. variable references with shell script 
compat. ones; $(VAR) -> ${VAR}
+   LC_ALL=C sed -e 's,\$$(\([A-Za-z0-9_]*\)),\$$\{\1\},g' < 
Misc/python-config.sh >python-config
+   @  # In OpenEmbedded, always use the python version of the script, the 
shell
+-- 
+2.25.1
+
diff --git a/meta/recipes-devtools/python/python3_3.10.4.bb 
b/meta/recipes-devtools/python/python3_3.10.4.bb
index 6bd3a6aba8..357025f856 100644
--- a/meta/recipes-devtools/python/python3_3.10.4.bb
+++ b/meta/recipes-devtools/python/python3_3.10.4.bb
@@ -34,6 +34,7 @@ SRC_URI = 
"http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \

file://0001-Lib-pty.py-handle-stdin-I-O-errors-same-way-as-maste.patch \

file://0001-setup.py-Do-not-detect-multiarch-paths-when-cross-co.patch \
file://deterministic_imports.patch \
+   file://0001-Avoid-shebang-overflow-on-python-config.py.patch \
"
 
 SRC_URI:append:class-native = " \
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166813): 
https://lists.openembedded.org/g/openembedded-core/message/166813
Mute This Topic: https://lists.openembedded.org/mt/91678611/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v3 2/7] gtk-doc: Fix potential shebang overflow on gtkdoc-mkhtml2

2022-06-10 Thread Paulo Neves

A native path can cause a shebang overflow on gtkdoc-mkhtml.
Replace it with /usr/bin/env.

Signed-off-by: Paulo Neves 
---
 meta/recipes-gnome/gtk-doc/gtk-doc_1.33.2.bb | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-gnome/gtk-doc/gtk-doc_1.33.2.bb 
b/meta/recipes-gnome/gtk-doc/gtk-doc_1.33.2.bb
index 150d2c0b23..392913fcc6 100644
--- a/meta/recipes-gnome/gtk-doc/gtk-doc_1.33.2.bb
+++ b/meta/recipes-gnome/gtk-doc/gtk-doc_1.33.2.bb
@@ -43,6 +43,7 @@ do_install:append () {
 ${datadir}/gtk-doc/python/gtkdoc/config.py; do
 sed -e 
's,${RECIPE_SYSROOT_NATIVE}/usr/bin/pkg-config,${bindir}/pkg-config,' \
 -e 's,${HOSTTOOLS_DIR}/python3,${bindir}/python3,' \
+-e '1s|^#!.*|#!/usr/bin/env python3|' \
 -i ${D}$fn
 done
 }
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166814): 
https://lists.openembedded.org/g/openembedded-core/message/166814
Mute This Topic: https://lists.openembedded.org/mt/91678612/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] e2fsprogs: add alternatives handling of lsattr as well

2022-06-08 Thread Paulo Neves

Looks good to me.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166756): 
https://lists.openembedded.org/g/openembedded-core/message/166756
Mute This Topic: https://lists.openembedded.org/mt/91622646/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v2 4/5] oeqa/selftest: Add test for shebang overflow

2022-06-08 Thread Paulo Neves

On 6/8/22 14:53, Luca Ceresoli wrote:

Hi Paulo,

On Tue, 7 Jun 2022 17:11:22 +0200
"Paulo Neves" wrote:

Make sure we do not stage any executable with a bigger shebang
than 128. Fixes [1]

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11053

Signed-off-by: Paulo Neves

This check seems to be working very well!! It triggered a huge amount
of build failures on the autobuilders due to libcheck having a shebang
too long in the checkmk script, e.g.:

#!
/home/pokybuild/yocto-worker/genericx86-64-alt/build/build/tmp/work/x86_64-linux/libcheck-native/0.15.2-r0/recipe-sysroot-native/usr/bin/gawk
-f

Here are a few logs:

https://autobuilder.yoctoproject.org/typhoon/#/builders/65/builds/5367/steps/14/logs/errors
https://autobuilder.yoctoproject.org/typhoon/#/builders/65/builds/5367/steps/11/logs/errors
https://autobuilder.yoctoproject.org/typhoon/#/builders/20/builds/5689/steps/11/logs/errors
https://autobuilder.yoctoproject.org/typhoon/#/builders/37/builds/5340/steps/12/logs/errors

I am not seeing an immediate way to fix this optimally. The go-to
solution to this class of issues is to just make
the #!/usr/bin/env . The issue is that there is an extra
-f, which with /usr/bin/env, will not work. The awk manual also implies
this is not optional for standalone scripts. I think we can create a
wrapper, or maybe we already have such a wrapper?

It would be great if you could add another patch to your series to fix
libcheck, and also to do 'bitbake world' to test as many packages as
possible before discovering from the autobuilders.
It takes quite a while on my computer and often i get out of disk space.
I tried asking the linaro guys for a tuxsuite token but no answer yet.
If you have some way of getting resources to make builds let me know.

Thank you!

Thank you
Paulo Neves

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166734):
https://lists.openembedded.org/g/openembedded-core/message/166734
Mute This Topic: https://lists.openembedded.org/mt/91602895/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] vim: put xxd in its own package

2022-06-08 Thread Paulo Neves

Looks good to me then ;) Had a look at the code [1] and indeed it is a 
very minimal standalone binary.


Grateful for your patience
Paulo Neves

[1] https://github.com/vim/vim/blob/master/src/xxd/xxd.c

On 6/8/22 14:36, Rasmus Villemoes wrote:

On 08/06/2022 14.30, Paulo Neves wrote:

On 6/8/22 14:13, Rasmus Villemoes wrote:


On 08/06/2022 13.58, Paulo Neves via lists.openembedded.org wrote:

I mean that your patch allows for a new standalone package ${PN}-xxd.
Were somebody to use this package standalone, meaning without installing vim,
the ncurses-terminfo-base would not be installed, possibly breaking xxd.

Before your patch this risk did not occur due to xxd being include in the ${PN}
which had an RDEPENDS to ncurses-terminfo-base.

Thanks, now I understand what you were saying. And no, xxd does not in
any way depend on those terminfo files; it's a simple utility for
converting binary files to hex (or the reverse, which is what I happen
to need, taking input in the form of ascii hex chars and producing
binary output). Yes, I've tested it on target as well.

Rasmus



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166708): 
https://lists.openembedded.org/g/openembedded-core/message/166708
Mute This Topic: https://lists.openembedded.org/mt/91618936/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] vim: put xxd in its own package

2022-06-08 Thread Paulo Neves


On 6/8/22 14:13, Rasmus Villemoes wrote:


On 08/06/2022 13.58, Paulo Neves via lists.openembedded.org wrote:

Forgive me if it is a stupid question, but does xxd not rdepend on
ncurses-terminfo-base as well?

No, it does not, it's a trivial standalone utility whose only dynamic
dependency is libc.


The original RDEPENDS on ncurses-terminfo-base is not about dynamic
libraries but about terminfo files that are used for describing capabilities
of terminals. They are usually in /etc/terminfo directory.
There are programs that load these files and without them will
display broken output our refuse to run. vim seemed to be one of those programs.
Have you tested that xxd works without these files?


I ask because before your patch it could
was implied it that it did, s part of PN.

I don't understand what you're trying to say. I just put the extra
RDEPENDS from vim to vim-xxd into the existing RDEPENDS:vim line. That
doesn't imply any dependency either way between the two items listed on
the RHS.


I mean that your patch allows for a new standalone package ${PN}-xxd.
Were somebody to use this package standalone, meaning without installing vim,
the ncurses-terminfo-base would not be installed, possibly breaking xxd.

Before your patch this risk did not occur due to xxd being include in the ${PN}
which had an RDEPENDS to ncurses-terminfo-base.


Rasmus


Paulo Neves

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166704): 
https://lists.openembedded.org/g/openembedded-core/message/166704
Mute This Topic: https://lists.openembedded.org/mt/91618936/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] vim: put xxd in its own package

2022-06-08 Thread Paulo Neves

Forgive me if it is a stupid question, but does xxd not rdepend on 
ncurses-terminfo-base as well? I ask because before your patch it could was 
implied it that it did, s part of PN.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166701): 
https://lists.openembedded.org/g/openembedded-core/message/166701
Mute This Topic: https://lists.openembedded.org/mt/91618936/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v2 1/5] python: Avoid shebang overflow on python-config.py

2022-06-07 Thread Paulo Neves

I think this patch will never be accepted because it hard-codes and depends
on env. I recall that red hat for example does not accept the use of env,
possibly for security reasons.

Paulo Neves

On Tue, Jun 7, 2022, 18:53 Alexander Kanavin  wrote:

> I'm not sure this is Inappropriate. Can you send this to upstream
> nevertheless?
>
> Alex
>
> On Tue, 7 Jun 2022 at 17:11, Paulo Neves  wrote:
> >
> > The native path may be too big, leading to shebang
> > overflow. Just use the #!/usr/bin/env python3.
> >
> > Signed-off-by: Paulo Neves 
> > ---
> >  ...shebang-overflow-on-python-config.py.patch | 29 +++
> >  .../recipes-devtools/python/python3_3.10.4.bb |  1 +
> >  2 files changed, 30 insertions(+)
> >  create mode 100644
> meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch
> >
> > diff --git
> a/meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch
> b/meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch
> > new file mode 100644
> > index 00..9f23278a30
> > --- /dev/null
> > +++
> b/meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch
> > @@ -0,0 +1,29 @@
> > +From f0c9dec63d452a7cd1e15ea653f4aced281f021c Mon Sep 17 00:00:00 2001
> > +From: Paulo Neves 
> > +Date: Tue, 7 Jun 2022 16:16:41 +0200
> > +Subject: [PATCH 1/1] Avoid shebang overflow on python-config.py
> > +
> > +The whole native path may be too big, leading to shebang
> > +overflow. Let's just use the env shebang.
> > +
> > +Upstream-Status: Inappropriate [distribution]
> > +---
> > + Makefile.pre.in | 2 ++
> > + 1 file changed, 2 insertions(+)
> > +
> > +diff --git a/Makefile.pre.in b/Makefile.pre.in
> > +index
> f0aedb76cb58999427804255da56fa53284d7032..dd88e43114730f7681715777cc76dabb31113176
> 100644
> > +--- a/Makefile.pre.in
> >  b/Makefile.pre.in
> > +@@ -1638,6 +1638,8 @@ python-config: $(srcdir)/Misc/python-config.in
> Misc/python-config.sh
> > +   @ # Substitution happens here, as the completely-expanded BINDIR
> > +   @ # is not available in configure
> > +   sed -e "s,@EXENAME@,$(BINDIR)/python$(LDVERSION)$(EXE)," <
> $(srcdir)/Misc/python-config.in >python-config.py
> > ++  @ # Otherwise we might get huge shebangs with native paths
> > ++  sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' python-config.py
> > +   @ # Replace makefile compat. variable references with shell
> script compat. ones; $(VAR) -> ${VAR}
> > +   LC_ALL=C sed -e 's,\$$(\([A-Za-z0-9_]*\)),\$$\{\1\},g' <
> Misc/python-config.sh >python-config
> > +   @  # In OpenEmbedded, always use the python version of the
> script, the shell
> > +--
> > +2.25.1
> > +
> > diff --git a/meta/recipes-devtools/python/python3_3.10.4.bb
> b/meta/recipes-devtools/python/python3_3.10.4.bb
> > index 6bd3a6aba8..357025f856 100644
> > --- a/meta/recipes-devtools/python/python3_3.10.4.bb
> > +++ b/meta/recipes-devtools/python/python3_3.10.4.bb
> > @@ -34,6 +34,7 @@ SRC_URI = "
> http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \
> >
>  file://0001-Lib-pty.py-handle-stdin-I-O-errors-same-way-as-maste.patch \
> >
>  file://0001-setup.py-Do-not-detect-multiarch-paths-when-cross-co.patch \
> > file://deterministic_imports.patch \
> > +   file://0001-Avoid-shebang-overflow-on-python-config.py.patch
> \
> > "
> >
> >  SRC_URI:append:class-native = " \
> > --
> > 2.25.1
> >
> >
> > 
> >
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166685): 
https://lists.openembedded.org/g/openembedded-core/message/166685
Mute This Topic: https://lists.openembedded.org/mt/91602889/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v2 5/5] oeqa/selftest: Test staged .la and .pc files

2022-06-07 Thread Paulo Neves

These files are checked by qa_check_staged but there was no
test cases for whether the tests actually worked. Now there
are.

Signed-off-by: Paulo Neves 
---
 .../sysroot-test/sysroot-la-test_1.0.bb   | 16 ++
 .../sysroot-test/sysroot-pc-test_1.0.bb   | 12 +++
 meta/lib/oeqa/selftest/cases/sysroot.py   | 32 +++
 3 files changed, 60 insertions(+)
 create mode 100644 
meta-selftest/recipes-test/sysroot-test/sysroot-la-test_1.0.bb
 create mode 100644 
meta-selftest/recipes-test/sysroot-test/sysroot-pc-test_1.0.bb

diff --git a/meta-selftest/recipes-test/sysroot-test/sysroot-la-test_1.0.bb 
b/meta-selftest/recipes-test/sysroot-test/sysroot-la-test_1.0.bb
new file mode 100644
index 00..21f06782fb
--- /dev/null
+++ b/meta-selftest/recipes-test/sysroot-test/sysroot-la-test_1.0.bb
@@ -0,0 +1,16 @@
+SUMMARY = "Produce a broken la file"
+LICENSE = "CLOSED"
+INHIBIT_DEFAULT_DEPS = "1"
+
+EXCLUDE_FROM_WORLD = "1"
+
+# remove-libtool.bbclass is inherited by default and removes all
+# .la files which for this test we specifically do not want.
+REMOVE_LIBTOOL_LA = "0"
+
+do_install() {
+install -d ${D}${libdir}/test/
+echo '${WORKDIR}' > ${D}${libdir}/test/la-test.la
+}
+
+BBCLASSEXTEND += "native"
diff --git a/meta-selftest/recipes-test/sysroot-test/sysroot-pc-test_1.0.bb 
b/meta-selftest/recipes-test/sysroot-test/sysroot-pc-test_1.0.bb
new file mode 100644
index 00..e748310fc4
--- /dev/null
+++ b/meta-selftest/recipes-test/sysroot-test/sysroot-pc-test_1.0.bb
@@ -0,0 +1,12 @@
+SUMMARY = "Produce a broken pc file"
+LICENSE = "CLOSED"
+INHIBIT_DEFAULT_DEPS = "1"
+
+EXCLUDE_FROM_WORLD = "1"
+
+do_install() {
+install -d ${D}${libdir}/test/
+echo '${WORKDIR}' > ${D}${libdir}/test/test.pc
+}
+
+BBCLASSEXTEND += "native"
diff --git a/meta/lib/oeqa/selftest/cases/sysroot.py 
b/meta/lib/oeqa/selftest/cases/sysroot.py
index 9457f1e3ac..588fc8c713 100644
--- a/meta/lib/oeqa/selftest/cases/sysroot.py
+++ b/meta/lib/oeqa/selftest/cases/sysroot.py
@@ -45,3 +45,35 @@ TESTSTRING:pn-sysroot-test-arch2 = "%s"
 expected = "maximum shebang size exceeded, the maximum size is 128. 
[shebang-size]"
 res = bitbake("sysroot-shebang-test-native -c populate_sysroot", 
ignore_status=True)
 self.assertTrue(expected in res.output, msg=res.output)
+
+def test_sysroot_la(self):
+"""
+    Summary:   Check that workdir paths are not contained in .la files.
+Expected:  Fail when a workdir path is found in the file content.
+Author:Paulo Neves 
+"""
+expected = "la-test.la failed sanity test (workdir) in path"
+
+res = bitbake("sysroot-la-test -c populate_sysroot", 
ignore_status=True)
+self.assertTrue(expected in res.output, msg=res.output)
+self.assertTrue('[la]' in res.output, msg=res.output)
+
+res = bitbake("sysroot-la-test-native -c populate_sysroot", 
ignore_status=True)
+self.assertTrue(expected in res.output, msg=res.output)
+self.assertTrue('[la]' in res.output, msg=res.output)
+
+def test_sysroot_pkgconfig(self):
+"""
+Summary:   Check that tmpdir paths are not contained in .pc files.
+Expected:  Fail when a tmpdir path is found in the file content.
+Author:Paulo Neves 
+"""
+expected = "test.pc failed sanity test (tmpdir) in path"
+
+res = bitbake("sysroot-pc-test -c populate_sysroot", 
ignore_status=True)
+self.assertTrue('[pkgconfig]' in res.output, msg=res.output)
+self.assertTrue(expected in res.output, msg=res.output)
+
+res = bitbake("sysroot-pc-test-native -c populate_sysroot", 
ignore_status=True)
+self.assertTrue(expected in res.output, msg=res.output)
+self.assertTrue('[pkgconfig]' in res.output, msg=res.output)
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166674): 
https://lists.openembedded.org/g/openembedded-core/message/166674
Mute This Topic: https://lists.openembedded.org/mt/91602896/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v2 3/5] insane.bbclass: Make do_qa_staging check shebangs

2022-06-07 Thread Paulo Neves

As reported in the bug report [1], there was no check for shebang
sizes on native scripts and now this is fixed.

The path scope of the qa_staging was increased from just checking
libdir to all the relevant SYSROOT_DIRS.

It is possible to skip this check through INSANE_SKIP.

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11053

Signed-off-by: Paulo Neves 
---
 meta/classes/insane.bbclass | 16 +++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index 9ca84bace9..b2951a48fe 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -630,6 +630,11 @@ def qa_check_staged(path,d):
 bb.note("Recipe %s skipping qa checking: pkgconfig" % d.getVar('PN'))
 skip_pkgconfig = True
 
+skip_shebang_size = False
+if 'shebang-size' in skip:
+bb.note("Recipe %s skipping qa checkking: shebang-size" % 
d.getVar('PN'))
+skip_shebang_size = True
+
 # find all .la and .pc files
 # read the content
 # and check for stuff that looks wrong
@@ -651,6 +656,13 @@ def qa_check_staged(path,d):
 error_msg = "%s failed sanity test (tmpdir) in path 
%s" % (file,root)
 oe.qa.handle_error("pkgconfig", error_msg, d)
 
+if not skip_shebang_size:
+errors = {}
+package_qa_check_shebang_size(path, "", d, None, errors)
+for e in errors:
+oe.qa.handle_error(e, errors[e], d)
+
+
 # Run all package-wide warnfuncs and errorfuncs
 def package_qa_package(warnfuncs, errorfuncs, package, d):
 warnings = {}
@@ -1139,7 +1151,9 @@ addtask do_package_qa_setscene
 
 python do_qa_staging() {
 bb.note("QA checking staging")
-qa_check_staged(d.expand('${SYSROOT_DESTDIR}${libdir}'), d)
+sysroot_destdir = d.expand('${SYSROOT_DESTDIR}')
+for sysroot_dir in d.expand('${SYSROOT_DIRS}').split():
+qa_check_staged(sysroot_destdir + sysroot_dir, d)
 oe.qa.exit_with_message_if_errors("QA staging was broken by the package 
built above", d)
 }
 
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166672): 
https://lists.openembedded.org/g/openembedded-core/message/166672
Mute This Topic: https://lists.openembedded.org/mt/91602894/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v2 4/5] oeqa/selftest: Add test for shebang overflow

2022-06-07 Thread Paulo Neves

Make sure we do not stage any executable with a bigger shebang
than 128. Fixes [1]

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11053

Signed-off-by: Paulo Neves 
---
 .../sysroot-test/sysroot-shebang-test_1.0.bb | 12 
 meta/lib/oeqa/selftest/cases/sysroot.py  | 10 ++
 2 files changed, 22 insertions(+)
 create mode 100644 
meta-selftest/recipes-test/sysroot-test/sysroot-shebang-test_1.0.bb

diff --git 
a/meta-selftest/recipes-test/sysroot-test/sysroot-shebang-test_1.0.bb 
b/meta-selftest/recipes-test/sysroot-test/sysroot-shebang-test_1.0.bb
new file mode 100644
index 00..6c834be897
--- /dev/null
+++ b/meta-selftest/recipes-test/sysroot-test/sysroot-shebang-test_1.0.bb
@@ -0,0 +1,12 @@
+SUMMARY = "Check that shebang does not exceed 128 characters"
+LICENSE = "CLOSED"
+INHIBIT_DEFAULT_DEPS = "1"
+
+EXCLUDE_FROM_WORLD = "1"
+do_install() {
+install -d ${D}${bindir}
+echo 
'#!BiM3cnVd1Amtv6PG+FynrQiVMbZnX5ELgF21q3EkuB+44JEGWtq8TvBJ7EGidfVs3eR3wVOUbLnjYDlKUWcm7YC/ute7f+KDHbwxziRUSUBZAUqgjiQdfQ0HnxajI0ozbM863E9JV9k13yZKYfh9/zR77Y6Dl4Dd3zOWS75LSpkAXV'
 > ${D}${bindir}/max-shebang
+chmod 755 ${D}${bindir}/max-shebang
+}
+
+BBCLASSEXTEND = "native"
diff --git a/meta/lib/oeqa/selftest/cases/sysroot.py 
b/meta/lib/oeqa/selftest/cases/sysroot.py
index 315d1a61c2..9457f1e3ac 100644
--- a/meta/lib/oeqa/selftest/cases/sysroot.py
+++ b/meta/lib/oeqa/selftest/cases/sysroot.py
@@ -35,3 +35,13 @@ TESTSTRING:pn-sysroot-test-arch1 = "%s"
 TESTSTRING:pn-sysroot-test-arch2 = "%s"
 """ % (uuid1, uuid2))
 bitbake("sysroot-test")
+
+def test_sysroot_max_shebang(self):
+"""
+Summary:   Check max shebang triggers. To confirm [YOCTO #11053] is 
closed.
+Expected:  Fail when a shebang bigger than the max shebang-size is 
reached.
+Author:Paulo Neves 
+"""
+expected = "maximum shebang size exceeded, the maximum size is 128. 
[shebang-size]"
+res = bitbake("sysroot-shebang-test-native -c populate_sysroot", 
ignore_status=True)
+self.assertTrue(expected in res.output, msg=res.output)
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166673): 
https://lists.openembedded.org/g/openembedded-core/message/166673
Mute This Topic: https://lists.openembedded.org/mt/91602895/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v2 1/5] python: Avoid shebang overflow on python-config.py

2022-06-07 Thread Paulo Neves

The native path may be too big, leading to shebang
overflow. Just use the #!/usr/bin/env python3.

Signed-off-by: Paulo Neves 
---
 ...shebang-overflow-on-python-config.py.patch | 29 +++
 .../recipes-devtools/python/python3_3.10.4.bb |  1 +
 2 files changed, 30 insertions(+)
 create mode 100644 
meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch

diff --git 
a/meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch
 
b/meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch
new file mode 100644
index 00..9f23278a30
--- /dev/null
+++ 
b/meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch
@@ -0,0 +1,29 @@
+From f0c9dec63d452a7cd1e15ea653f4aced281f021c Mon Sep 17 00:00:00 2001
+From: Paulo Neves 
+Date: Tue, 7 Jun 2022 16:16:41 +0200
+Subject: [PATCH 1/1] Avoid shebang overflow on python-config.py
+
+The whole native path may be too big, leading to shebang
+overflow. Let's just use the env shebang.
+
+Upstream-Status: Inappropriate [distribution]
+---
+ Makefile.pre.in | 2 ++
+ 1 file changed, 2 insertions(+)
+
+diff --git a/Makefile.pre.in b/Makefile.pre.in
+index 
f0aedb76cb58999427804255da56fa53284d7032..dd88e43114730f7681715777cc76dabb31113176
 100644
+--- a/Makefile.pre.in
 b/Makefile.pre.in
+@@ -1638,6 +1638,8 @@ python-config: $(srcdir)/Misc/python-config.in 
Misc/python-config.sh
+   @ # Substitution happens here, as the completely-expanded BINDIR
+   @ # is not available in configure
+   sed -e "s,@EXENAME@,$(BINDIR)/python$(LDVERSION)$(EXE)," < 
$(srcdir)/Misc/python-config.in >python-config.py
++  @ # Otherwise we might get huge shebangs with native paths
++  sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' python-config.py
+   @ # Replace makefile compat. variable references with shell script 
compat. ones; $(VAR) -> ${VAR}
+   LC_ALL=C sed -e 's,\$$(\([A-Za-z0-9_]*\)),\$$\{\1\},g' < 
Misc/python-config.sh >python-config
+   @  # In OpenEmbedded, always use the python version of the script, the 
shell
+-- 
+2.25.1
+
diff --git a/meta/recipes-devtools/python/python3_3.10.4.bb 
b/meta/recipes-devtools/python/python3_3.10.4.bb
index 6bd3a6aba8..357025f856 100644
--- a/meta/recipes-devtools/python/python3_3.10.4.bb
+++ b/meta/recipes-devtools/python/python3_3.10.4.bb
@@ -34,6 +34,7 @@ SRC_URI = 
"http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \

file://0001-Lib-pty.py-handle-stdin-I-O-errors-same-way-as-maste.patch \

file://0001-setup.py-Do-not-detect-multiarch-paths-when-cross-co.patch \
file://deterministic_imports.patch \
+   file://0001-Avoid-shebang-overflow-on-python-config.py.patch \
"
 
 SRC_URI:append:class-native = " \
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166670): 
https://lists.openembedded.org/g/openembedded-core/message/166670
Mute This Topic: https://lists.openembedded.org/mt/91602889/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v2 2/5] gtk-doc: Fix potential shebang overflow on gtkdoc-mkhtml2

2022-06-07 Thread Paulo Neves

A native path can cause a shebang overflow on gtkdoc-mkhtml.
Replace it with /usr/bin/env.

Signed-off-by: Paulo Neves 
---
 meta/recipes-gnome/gtk-doc/gtk-doc_1.33.2.bb | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-gnome/gtk-doc/gtk-doc_1.33.2.bb 
b/meta/recipes-gnome/gtk-doc/gtk-doc_1.33.2.bb
index 150d2c0b23..392913fcc6 100644
--- a/meta/recipes-gnome/gtk-doc/gtk-doc_1.33.2.bb
+++ b/meta/recipes-gnome/gtk-doc/gtk-doc_1.33.2.bb
@@ -43,6 +43,7 @@ do_install:append () {
 ${datadir}/gtk-doc/python/gtkdoc/config.py; do
 sed -e 
's,${RECIPE_SYSROOT_NATIVE}/usr/bin/pkg-config,${bindir}/pkg-config,' \
 -e 's,${HOSTTOOLS_DIR}/python3,${bindir}/python3,' \
+-e '1s|^#!.*|#!/usr/bin/env python3|' \
 -i ${D}$fn
 done
 }
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166671): 
https://lists.openembedded.org/g/openembedded-core/message/166671
Mute This Topic: https://lists.openembedded.org/mt/91602893/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 1/3] insane.bbclass: Make do_qa_staging check shebangs

2022-06-06 Thread Paulo Neves

As reported in the bug report [1], there was no check for shebang sizes
on native scripts and now this is fixed.

The path scope of qa_staging needed to be increased from just
checking libdir to all the relevant SYSROOT_DIRS.

It is possible to skip this check through INSANE_SKIP.

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11053
---
 meta/classes/insane.bbclass | 16 +++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index 9ca84bace9..dd0cf2a769 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -630,6 +630,11 @@ def qa_check_staged(path,d):
 bb.note("Recipe %s skipping qa checking: pkgconfig" % d.getVar('PN'))
 skip_pkgconfig = True
 
+skip_shebang_size = False
+if 'shebang-size' in skip:
+bb.note("Recipe %s skipping qa checkking: shebang-size" % 
d.getVar('PN'))
+skip_shebang_size = True
+
 # find all .la and .pc files
 # read the content
 # and check for stuff that looks wrong
@@ -651,6 +656,13 @@ def qa_check_staged(path,d):
 error_msg = "%s failed sanity test (tmpdir) in path 
%s" % (file,root)
 oe.qa.handle_error("pkgconfig", error_msg, d)
 
+if not skip_shebang_size:
+errors = {}
+package_qa_check_shebang_size(path, "", d, None, errors)
+for e in errors:
+oe.qa.handle_error(e, errors[e], d)
+
+
 # Run all package-wide warnfuncs and errorfuncs
 def package_qa_package(warnfuncs, errorfuncs, package, d):
 warnings = {}
@@ -1139,7 +1151,9 @@ addtask do_package_qa_setscene
 
 python do_qa_staging() {
 bb.note("QA checking staging")
-qa_check_staged(d.expand('${SYSROOT_DESTDIR}${libdir}'), d)
+sysroot_destdir = d.expand('${SYSROOT_DESTDIR}')
+for sysroot_dir in d.expand('${SYSROOT_DIRS}').split():
+qa_check_staged(os.path.join(sysroot_destdir, sysroot_dir), d)
 oe.qa.exit_with_message_if_errors("QA staging was broken by the package 
built above", d)
 }
 
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166651): 
https://lists.openembedded.org/g/openembedded-core/message/166651
Mute This Topic: https://lists.openembedded.org/mt/91580233/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 3/3] oeqa/selftest: Test staged .la and .pc files

2022-06-06 Thread Paulo Neves

These files are checked by qa_check_staged but there was no
test cases for whether the tests actually worked. Now there
are.
---
 .../sysroot-test/sysroot-la-test_1.0.bb   | 16 ++
 .../sysroot-test/sysroot-pc-test_1.0.bb   | 12 +++
 meta/lib/oeqa/selftest/cases/sysroot.py   | 32 +++
 3 files changed, 60 insertions(+)
 create mode 100644 
meta-selftest/recipes-test/sysroot-test/sysroot-la-test_1.0.bb
 create mode 100644 
meta-selftest/recipes-test/sysroot-test/sysroot-pc-test_1.0.bb

diff --git a/meta-selftest/recipes-test/sysroot-test/sysroot-la-test_1.0.bb 
b/meta-selftest/recipes-test/sysroot-test/sysroot-la-test_1.0.bb
new file mode 100644
index 00..21f06782fb
--- /dev/null
+++ b/meta-selftest/recipes-test/sysroot-test/sysroot-la-test_1.0.bb
@@ -0,0 +1,16 @@
+SUMMARY = "Produce a broken la file"
+LICENSE = "CLOSED"
+INHIBIT_DEFAULT_DEPS = "1"
+
+EXCLUDE_FROM_WORLD = "1"
+
+# remove-libtool.bbclass is inherited by default and removes all
+# .la files which for this test we specifically do not want.
+REMOVE_LIBTOOL_LA = "0"
+
+do_install() {
+install -d ${D}${libdir}/test/
+echo '${WORKDIR}' > ${D}${libdir}/test/la-test.la
+}
+
+BBCLASSEXTEND += "native"
diff --git a/meta-selftest/recipes-test/sysroot-test/sysroot-pc-test_1.0.bb 
b/meta-selftest/recipes-test/sysroot-test/sysroot-pc-test_1.0.bb
new file mode 100644
index 00..e748310fc4
--- /dev/null
+++ b/meta-selftest/recipes-test/sysroot-test/sysroot-pc-test_1.0.bb
@@ -0,0 +1,12 @@
+SUMMARY = "Produce a broken pc file"
+LICENSE = "CLOSED"
+INHIBIT_DEFAULT_DEPS = "1"
+
+EXCLUDE_FROM_WORLD = "1"
+
+do_install() {
+install -d ${D}${libdir}/test/
+echo '${WORKDIR}' > ${D}${libdir}/test/test.pc
+}
+
+BBCLASSEXTEND += "native"
diff --git a/meta/lib/oeqa/selftest/cases/sysroot.py 
b/meta/lib/oeqa/selftest/cases/sysroot.py
index 9457f1e3ac..588fc8c713 100644
--- a/meta/lib/oeqa/selftest/cases/sysroot.py
+++ b/meta/lib/oeqa/selftest/cases/sysroot.py
@@ -45,3 +45,35 @@ TESTSTRING:pn-sysroot-test-arch2 = "%s"
 expected = "maximum shebang size exceeded, the maximum size is 128. 
[shebang-size]"
 res = bitbake("sysroot-shebang-test-native -c populate_sysroot", 
ignore_status=True)
 self.assertTrue(expected in res.output, msg=res.output)
+
+def test_sysroot_la(self):
+"""
+    Summary:   Check that workdir paths are not contained in .la files.
+Expected:  Fail when a workdir path is found in the file content.
+Author:Paulo Neves 
+"""
+expected = "la-test.la failed sanity test (workdir) in path"
+
+res = bitbake("sysroot-la-test -c populate_sysroot", 
ignore_status=True)
+self.assertTrue(expected in res.output, msg=res.output)
+self.assertTrue('[la]' in res.output, msg=res.output)
+
+res = bitbake("sysroot-la-test-native -c populate_sysroot", 
ignore_status=True)
+self.assertTrue(expected in res.output, msg=res.output)
+self.assertTrue('[la]' in res.output, msg=res.output)
+
+def test_sysroot_pkgconfig(self):
+"""
+Summary:   Check that tmpdir paths are not contained in .pc files.
+Expected:  Fail when a tmpdir path is found in the file content.
+Author:Paulo Neves 
+"""
+expected = "test.pc failed sanity test (tmpdir) in path"
+
+res = bitbake("sysroot-pc-test -c populate_sysroot", 
ignore_status=True)
+self.assertTrue('[pkgconfig]' in res.output, msg=res.output)
+self.assertTrue(expected in res.output, msg=res.output)
+
+res = bitbake("sysroot-pc-test-native -c populate_sysroot", 
ignore_status=True)
+self.assertTrue(expected in res.output, msg=res.output)
+self.assertTrue('[pkgconfig]' in res.output, msg=res.output)
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166653): 
https://lists.openembedded.org/g/openembedded-core/message/166653
Mute This Topic: https://lists.openembedded.org/mt/91580236/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 2/3] oeqa/selftest: Add test for shebang overflow

2022-06-06 Thread Paulo Neves

Make sure we do not stage any executable with a bigger shebang
than 128. Fixes [1]

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11053
---
 .../sysroot-test/sysroot-shebang-test_1.0.bb | 12 
 meta/lib/oeqa/selftest/cases/sysroot.py  | 10 ++
 2 files changed, 22 insertions(+)
 create mode 100644 
meta-selftest/recipes-test/sysroot-test/sysroot-shebang-test_1.0.bb

diff --git 
a/meta-selftest/recipes-test/sysroot-test/sysroot-shebang-test_1.0.bb 
b/meta-selftest/recipes-test/sysroot-test/sysroot-shebang-test_1.0.bb
new file mode 100644
index 00..6c834be897
--- /dev/null
+++ b/meta-selftest/recipes-test/sysroot-test/sysroot-shebang-test_1.0.bb
@@ -0,0 +1,12 @@
+SUMMARY = "Check that shebang does not exceed 128 characters"
+LICENSE = "CLOSED"
+INHIBIT_DEFAULT_DEPS = "1"
+
+EXCLUDE_FROM_WORLD = "1"
+do_install() {
+install -d ${D}${bindir}
+echo 
'#!BiM3cnVd1Amtv6PG+FynrQiVMbZnX5ELgF21q3EkuB+44JEGWtq8TvBJ7EGidfVs3eR3wVOUbLnjYDlKUWcm7YC/ute7f+KDHbwxziRUSUBZAUqgjiQdfQ0HnxajI0ozbM863E9JV9k13yZKYfh9/zR77Y6Dl4Dd3zOWS75LSpkAXV'
 > ${D}${bindir}/max-shebang
+chmod 755 ${D}${bindir}/max-shebang
+}
+
+BBCLASSEXTEND = "native"
diff --git a/meta/lib/oeqa/selftest/cases/sysroot.py 
b/meta/lib/oeqa/selftest/cases/sysroot.py
index 315d1a61c2..9457f1e3ac 100644
--- a/meta/lib/oeqa/selftest/cases/sysroot.py
+++ b/meta/lib/oeqa/selftest/cases/sysroot.py
@@ -35,3 +35,13 @@ TESTSTRING:pn-sysroot-test-arch1 = "%s"
 TESTSTRING:pn-sysroot-test-arch2 = "%s"
 """ % (uuid1, uuid2))
 bitbake("sysroot-test")
+
+def test_sysroot_max_shebang(self):
+"""
+Summary:   Check max shebang triggers. To confirm [YOCTO #11053] is 
closed.
+Expected:  Fail when a shebang bigger than the max shebang-size is 
reached.
+Author:Paulo Neves 
+"""
+expected = "maximum shebang size exceeded, the maximum size is 128. 
[shebang-size]"
+res = bitbake("sysroot-shebang-test-native -c populate_sysroot", 
ignore_status=True)
+self.assertTrue(expected in res.output, msg=res.output)
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166652): 
https://lists.openembedded.org/g/openembedded-core/message/166652
Mute This Topic: https://lists.openembedded.org/mt/91580235/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] wget: update to 1.20.3

2019-05-08 Thread Paulo Neves

This patch is useful to go around CVE-2019-5953 but other versions of
poky, like the thud do not have the correction.

Should this version bump be backported or the patch
http://git.savannah.gnu.org/cgit/wget.git/commit/?id=692d5c5215de0db482c252492a92fc424cc6a97c
be applied in older versions?

On Wed, Apr 24, 2019 at 8:56 AM Oleksandr Kravchuk
 wrote:
>
> Signed-off-by: Oleksandr Kravchuk 
> ---
>  meta/recipes-extended/wget/wget_1.20.2.bb | 8 
>  meta/recipes-extended/wget/wget_1.20.3.bb | 8 
>  2 files changed, 8 insertions(+), 8 deletions(-)
>  delete mode 100644 meta/recipes-extended/wget/wget_1.20.2.bb
>  create mode 100644 meta/recipes-extended/wget/wget_1.20.3.bb
>
> diff --git a/meta/recipes-extended/wget/wget_1.20.2.bb 
> b/meta/recipes-extended/wget/wget_1.20.2.bb
> deleted file mode 100644
> index cd20785259..00
> --- a/meta/recipes-extended/wget/wget_1.20.2.bb
> +++ /dev/null
> @@ -1,8 +0,0 @@
> -SRC_URI = "${GNU_MIRROR}/wget/wget-${PV}.tar.gz \
> -   file://0002-improve-reproducibility.patch \
> -  "
> -
> -SRC_URI[md5sum] = "2692f6678e93601441306b5c1fc6a77a"
> -SRC_URI[sha256sum] = 
> "7e43b98cb5e10234836ebef6faf24c4d96c0ae7a480e49ff658117cc4793d166"
> -
> -require wget.inc
> diff --git a/meta/recipes-extended/wget/wget_1.20.3.bb 
> b/meta/recipes-extended/wget/wget_1.20.3.bb
> new file mode 100644
> index 00..4fa273d093
> --- /dev/null
> +++ b/meta/recipes-extended/wget/wget_1.20.3.bb
> @@ -0,0 +1,8 @@
> +SRC_URI = "${GNU_MIRROR}/wget/wget-${PV}.tar.gz \
> +   file://0002-improve-reproducibility.patch \
> +  "
> +
> +SRC_URI[md5sum] = "db4e6dc7977cbddcd543b240079a4899"
> +SRC_URI[sha256sum] = 
> "31cccfc6630528db1c8e3a06f6decf2a370060b982841cfab2b8677400a5092e"
> +
> +require wget.inc
> --
> 2.17.1
>
> --
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 1/2] util-linux: disable systemd

2019-01-04 Thread Paulo Neves

Hello all,

I have the need to enable this services from system. What is the
current status of this circular dependency?
There was talk of coming back to this topic after the milesone but 4
years passed and there was no changes to this topic? Is it still
applicable?

Paulo Neves
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [meta-oe][PATCH v2 4/4] testimage: Moved write_image_test_data to testimage bbclass.

2018-10-02 Thread Paulo Neves

Hello Richard,

Would assure you if it generated the test data on image creation *and*
on test task?
If not please describe more or less how I could collect which
variables write_image_test_data depends upon. Just a high level
description would do.

I have really been in a good spirit to get this topic merged but I do
not see a clear guidance for me to fix what was clearly broken.
Actually the current code put there by intel should should be reverted
in my opinion, as it broke all the controllers besides simpleremote
and qemu.

Paulo Neves
On Tue, Sep 4, 2018 at 12:30 PM Richard Purdie
 wrote:
>
> On Thu, 2018-08-30 at 19:49 +0200, Paulo Neves wrote:
> > Previously the write_image_test_data was a rootfs post
> > process command. This function ran only when the rootfs
> > task was ran. Due to this if a variable was changed
> > or added to the datastore that would not trigger the do_rootfs
> > task, the variable would never be written into the json
> > file. Consequently the do_testimage task was potentially
> > unreproduceable and could fail if for some reason this
> > variable was then used.
> >
> > In this commit we move the recording of the datastore to the
> > start of the do_testimage task. The do_testimage
> > task then reads this freshly generated json and passes it into
> > the normal test machinery. This approach allows for the test
> > machinery to still be loosely coupled to bitbake, and thus
> > still allows for the tests to be exported and used independently
> > from bitbake. The caveat is, if there are datastore changes
> > then the bitbake testimage task should be run before the tests
> > can be ran independently again.
>
> I'm not sure I agree with this. The test data really should be written
> at either rootfs or image generation, not just before the tests are
> run.
>
> If you wanted to run the tests independently, after this change you
> have no way to generate the data to do that without first running the
> tests at least once. That doesn't make sense.
>
> The correct thing to do here is to ensure the write_image_test_data()
> function indicates which variables it depends upon and hence reruns
> correctly.
>
> Cheers,
>
> Richard
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [meta-oe][PATCH v2 4/4] testimage: Moved write_image_test_data to testimage bbclass.

2018-08-30 Thread Paulo Neves

Previously the write_image_test_data was a rootfs post
process command. This function ran only when the rootfs
task was ran. Due to this if a variable was changed
or added to the datastore that would not trigger the do_rootfs
task, the variable would never be written into the json
file. Consequently the do_testimage task was potentially
unreproduceable and could fail if for some reason this
variable was then used.

In this commit we move the recording of the datastore to the
start of the do_testimage task. The do_testimage
task then reads this freshly generated json and passes it into
the normal test machinery. This approach allows for the test
machinery to still be loosely coupled to bitbake, and thus
still allows for the tests to be exported and used independently
from bitbake. The caveat is, if there are datastore changes
then the bitbake testimage task should be run before the tests
can be ran independently again.

Signed-off-by: Paulo Neves 
---
 meta/classes/rootfs-postcommands.bbclass | 22 --
 meta/classes/testimage.bbclass   | 19 ++-
 2 files changed, 18 insertions(+), 23 deletions(-)

diff --git a/meta/classes/rootfs-postcommands.bbclass 
b/meta/classes/rootfs-postcommands.bbclass
index e816824..a396c04 100644
--- a/meta/classes/rootfs-postcommands.bbclass
+++ b/meta/classes/rootfs-postcommands.bbclass
@@ -25,9 +25,6 @@ ROOTFS_POSTPROCESS_COMMAND += 
'${@bb.utils.contains("IMAGE_FEATURES", "read-only
 # and we don't want to disable such a default that by setting a value here.
 APPEND_append = '${@bb.utils.contains("IMAGE_FEATURES", "read-only-rootfs", " 
ro", "", d)}'
 
-# Generates test data file with data store variables expanded in json format
-ROOTFS_POSTPROCESS_COMMAND += "write_image_test_data ; "
-
 # Write manifest
 IMAGE_MANIFEST = "${IMGDEPLOYDIR}/${IMAGE_NAME}.rootfs.manifest"
 ROOTFS_POSTUNINSTALL_COMMAND =+ "write_image_manifest ; "
@@ -312,25 +309,6 @@ rootfs_sysroot_relativelinks () {
sysroot-relativelinks.py ${SDK_OUTPUT}/${SDKTARGETSYSROOT}
 }
 
-# Generated test data json file
-python write_image_test_data() {
-from oe.data import export2json
-
-deploy_dir = d.getVar('IMGDEPLOYDIR')
-link_name = d.getVar('IMAGE_LINK_NAME')
-testdata_name = os.path.join(deploy_dir, "%s.testdata.json" % 
d.getVar('IMAGE_NAME'))
-
-searchString = "%s/"%(d.getVar("TOPDIR")).replace("//","/")
-export2json(d, testdata_name, searchString=searchString, replaceString="")
-
-if os.path.exists(testdata_name):
-testdata_link = os.path.join(deploy_dir, "%s.testdata.json" % 
link_name)
-if os.path.lexists(testdata_link):
-os.remove(testdata_link)
-os.symlink(os.path.basename(testdata_name), testdata_link)
-}
-write_image_test_data[vardepsexclude] += "TOPDIR"
-
 # Check for unsatisfied recommendations (RRECOMMENDS)
 python rootfs_log_check_recommends() {
 log_path = d.expand("${T}/log.do_rootfs")
diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index f58d11d..192febd 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -137,6 +137,21 @@ def testimage_sanity(d):
 bb.fatal('When TEST_TARGET is set to "simpleremote" '
  'TEST_TARGET_IP and TEST_SERVER_IP are needed too.')
 
+# Generated test data json file
+def write_image_test_data(d):
+from oe.data import export2json
+
+testdata = "%s/%s.testdata.json" % (d.getVar('DEPLOY_DIR_IMAGE'), 
d.getVar('IMAGE_NAME'))
+testdata_link = "%s/%s.testdata.json" % (d.getVar('DEPLOY_DIR_IMAGE'), 
d.getVar('IMAGE_LINK_NAME'))
+
+bb.utils.mkdirhier(os.path.dirname(testdata))
+searchString = "%s/"%(d.getVar("TOPDIR")).replace("//","/")
+export2json(d, testdata,searchString=searchString,replaceString="")
+if testdata_link != testdata:
+if os.path.lexists(testdata_link):
+   os.remove(testdata_link)
+os.symlink(os.path.basename(testdata), testdata_link)
+
 def testimage_main(d):
 import os
 import json
@@ -170,12 +185,14 @@ def testimage_main(d):
 image_name = ("%s/%s" % (d.getVar('DEPLOY_DIR_IMAGE'),
  d.getVar('IMAGE_LINK_NAME')))
 
+
+write_image_test_data(d)
+
 tdname = "%s.testdata.json" % image_name
 try:
 td = json.load(open(tdname, "r"))
 except (FileNotFoundError) as err:
  bb.fatal('File %s Not Found. Have you built the image with 
INHERIT+="testimage" in the conf/local.conf?' % tdname)
-
 # Some variables need to be updates (mostly paths) with the
 # ones of the current environment because some tests require them.
 updateTestData(d, td, d.getVar('TESTIMAGE_UPDATE_VARS').split())
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [meta-oe][PATCH v2 3/4] masterimage: Check for rootfs path instead of file.

2018-08-30 Thread Paulo Neves

The rootfs might be a directory path or a file path. A
case in point is if the test wants to deploy a rootfs
directly by rsyncing it to the target or even use it
as part of a tftp server.

Signed-off-by: Paulo Neves 
---
 meta/lib/oeqa/controllers/masterimage.py | 6 +-
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/meta/lib/oeqa/controllers/masterimage.py 
b/meta/lib/oeqa/controllers/masterimage.py
index 26e3870..fb8ab50 100644
--- a/meta/lib/oeqa/controllers/masterimage.py
+++ b/meta/lib/oeqa/controllers/masterimage.py
@@ -51,11 +51,7 @@ class 
MasterImageHardwareTarget(oeqa.targetcontrol.BaseTarget, metaclass=ABCMeta
 self.image_fstype = self.get_image_fstype(td)
 self.rootfs = os.path.join(td['DEPLOY_DIR_IMAGE'], 
td['IMAGE_LINK_NAME'] + '.' + self.image_fstype)
 self.kernel = os.path.join(td['DEPLOY_DIR_IMAGE'], 
td["KERNEL_IMAGETYPE"] + '-' + td['MACHINE'] + '.bin')
-if not os.path.isfile(self.rootfs):
-# we could've checked that IMAGE_FSTYPES contains tar.gz but the 
config for running testimage might not be
-# the same as the config with which the image was build, ie
-# you bitbake core-image-sato with IMAGE_FSTYPES += "tar.gz"
-# and your autobuilder overwrites the config, adds the test bits 
and runs bitbake core-image-sato -c testimage
+if not os.path.exists(self.rootfs):
 bb.fatal("No rootfs found. Did you build the image ?\n"  \
   "If yes, did you build it with IMAGE_FSTYPES += \"{}\" ?"  \
   "\nExpected path: {}".format(" 
".join(self.supported_image_fstypes), self.rootfs))
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [meta-oe][PATCH v2 2/4] testimage: target.start exceptions not masked.

2018-08-30 Thread Paulo Neves

Previously the target.start exceptions were being
masked by the catch clause of RuntimeError and
BlockingIOError which are very broad. We decoupled
the start method try clause from the runTests try
clause which requires catching the BlockingIOError and
RuntimeError.

Signed-off-by: Paulo Neves 
---
 meta/classes/testimage.bbclass | 36 +---
 1 file changed, 21 insertions(+), 15 deletions(-)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 2aa59b0..f58d11d 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -269,24 +269,30 @@ def testimage_main(d):
 # Add systemd.log_level=debug to enable systemd debug logging
 bootparams = 'systemd.log_target=console'
 
+
 tc.target.deploy()
-results = None
-orig_sigterm_handler = signal.signal(signal.SIGTERM, sigterm_exception)
+
+#We do not want the runTests to mask exeptions of target.start but we want 
a failed start
+#to trigger a stop.
 try:
-# We need to check if runqemu ends unexpectedly
-# or if the worker send us a SIGTERM
-tc.target.start(extra_bootparams=bootparams)
-results = tc.runTests()
-except (RuntimeError, BlockingIOError) as err:
-if isinstance(err, RuntimeError):
-bb.error('testimage received SIGTERM, shutting down...')
-else:
-bb.error('runqemu failed, shutting down...')
-if results:
-results.stop()
-results = None
+tc.target.start(extra_bootparams=bootparams) 
+results = None
+orig_sigterm_handler = signal.signal(signal.SIGTERM, sigterm_exception)
+try:
+# We need to check if runqemu ends unexpectedly
+# or if the worker send us a SIGTERM
+results = tc.runTests()
+except (RuntimeError, BlockingIOError) as err:
+if isinstance(err, RuntimeError):
+bb.error('testimage received SIGTERM, shutting down...')
+else:
+bb.error('runqemu failed, shutting down...')
+if results:
+results.stop()
+results = None
+finally:
+signal.signal(signal.SIGTERM, orig_sigterm_handler)
 finally:
-signal.signal(signal.SIGTERM, orig_sigterm_handler)
 tc.target.stop()
 
 # Show results (if we have them)
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [meta-oe][PATCH v2 1/4] testimage: Refactoring and fixing.

2018-08-30 Thread Paulo Neves

The testimage.bbclass invocation of controllers besides
simpletarget and qemutarget was broken.  Setting TEST_TARGET
to class names did not work, only hardcoded values set in
context.py were allowed.  On the other hand the other
TEST_TARGETs available were supposedly available through the
class name convetion but none of them were working. See:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=12842

In this commit qemuwrapper and simpletarget TEST_TARGETS are
now not available through these names, but through the
common convention of class names.

The code for layer defined controllers was outdated and
broken because the example controllers in meta-yocto-bsp as
well as the controllers available in masterimage were
dependent only on the datastore d being available for
__init__ contruction, when the testimage.bbclass and
context.py, which initialize test controllers never passed
the datastore. This commit fixes all the openembedded
controllers to not use the datastore, but the testdata json
file that is loaded by the testimage.bbclass. This way the
tests can still be exported to be ran outside bitbake. The
kwargs and logger is still passed in the constructor and
there maybe some redundancies between the kwargs and the td
test datstore, but this is not addressed in this commit.

In masterimage.py there was a duplicate SystemdbootTarget
which was removed. It is my opinion that all targets should
go to meta-yocto-bsp or meta/ and should not get special
treatment whether they come from poky or other layer.

In this commit functionality that was lost in masterimage
about retrieving a fresh BB_ORIGENV. The test data json file
does not store this variable, so the build-time enviromental
variables cannot be retrieved later on, when running the
tests.

The deploy() method invocation of the test cases was removed
in the refactoring and this commit re-adds it in the same
logical place. The deploy method was added as an abstract
method so that all Test controller classes have this method.
This method had been lost and was not used in the
simpleremote and qemuwrapper, because these controllers do
not need the functionality, but almost any real board
requires certain steps to deploy the new images, before the
tests can be ran. An example of this need was visible in the
master image and subclasses in meta-yocto-bsp.

A dependency on images fstypes of qemu was removed from
testimage.  The allowed file system types were not relevant
for other controllers, and made it so that there was an
import of the OEQemuTarget python class applicable to all
possible controllers, including non qemu.

The SimpleRemoteTarget, OEQemuTarget and a custom
controller, developed according to the manual were
successfully tested.

Signed-off-by: Paulo Neves 
---
 meta/classes/testimage.bbclass |  18 +---
 meta/lib/oeqa/controllers/masterimage.py   | 107 ++---
 .../target/qemu.py => controllers/qemutarget.py}   |  30 --
 meta/lib/oeqa/controllers/simpleremote.py  |  33 +++
 .../target/ssh.py => controllers/sshtarget.py} |   9 +-
 meta/lib/oeqa/core/target/__init__.py  |   4 +
 meta/lib/oeqa/runtime/context.py   |  32 +-
 meta/lib/oeqa/targetcontrol.py |  97 ++-
 8 files changed, 137 insertions(+), 193 deletions(-)
 rename meta/lib/oeqa/{core/target/qemu.py => controllers/qemutarget.py} (58%)
 create mode 100644 meta/lib/oeqa/controllers/simpleremote.py
 rename meta/lib/oeqa/{core/target/ssh.py => controllers/sshtarget.py} (98%)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 6fa901b..2aa59b0 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -75,7 +75,7 @@ DEFAULT_TEST_SUITES_remove_qemumips64 = "${MIPSREMOVE}"
 TEST_SUITES ?= "${DEFAULT_TEST_SUITES}"
 
 TEST_QEMUBOOT_TIMEOUT ?= "1000"
-TEST_TARGET ?= "qemu"
+TEST_TARGET ?= "OEQemuTarget"
 
 TESTIMAGEDEPENDS = ""
 TESTIMAGEDEPENDS_qemuall = "qemu-native:do_populate_sysroot 
qemu-helper-native:do_populate_sysroot 
qemu-helper-native:do_addto_recipe_sysroot"
@@ -147,7 +147,6 @@ def testimage_main(d):
 from oeqa.core.utils.misc import updateTestData
 from oeqa.runtime.context import OERuntimeTestContext
 from oeqa.runtime.context import OERuntimeTestContextExecutor
-from oeqa.core.target.qemu import supported_fstypes
 from oeqa.core.utils.test import getSuiteCases
 from oeqa.utils import make_logger_bitbake_compatible
 
@@ -189,15 +188,6 @@ def testimage_main(d):
 # Get machine
 machine = d.getVar("MACHINE")
 
-# Get rootfs
-fstypes = [fs for fs in d.getVar('IMAGE_FSTYPES').split(' ')
-  if fs in supported_fstypes]
-if not fstypes:
-bb.fatal('Unsupported image type built. Add a comptible image to '
-

Re: [OE-core] [meta-oe][PATCH 1/3] testimage: Refactoring and fixing.

2018-08-18 Thread Paulo Neves

Ping.

Does anybody have any comments? Also I noticed that my commit once was
in the master-next but it was removed. Without it being in master I
cannot submit my changes to poky mailing list with the fixes for the
remaining targets.

Paulo Neves
On Sun, Aug 12, 2018 at 9:49 PM Paulo Neves  wrote:
>
> I found a bug in my code where changing a variable, for example in
> layer.conf does not re-trigger the generation of the json file
> containing the data store at the time of image creation. The json file
> is generated in
> ROOTFS_POSTPROCESS_COMMAND += "write_image_test_data ; " @
> meta/classes/rootfs-postcommands.bbclass.
>
> Can anybody tell me if changing a variable in the datastore and
> accessing it in the postprocess_command will never trigger the
> postprocess_command? It happens that in this case this variable is not
> important for the rootfs task itself but only for the post-process.
>
> Paulo Neves
> On Fri, Aug 10, 2018 at 9:38 PM Paulo Neves  wrote:
> >
> > The testimage.bbclass invocation of controllers besides
> > simpletarget and qemutarget was broken.  Setting TEST_TARGET
> > to class names did not work, only hardcoded values set in
> > context.py were allowed.  On the other hand the other
> > TEST_TARGETs available were supposedly available through the
> > class name convetion but none of them were working. See:
> > https://bugzilla.yoctoproject.org/show_bug.cgi?id=12842
> >
> > In this commit qemuwrapper and simpletarget TEST_TARGETS are
> > now not available through these names, but through the
> > common convention of class names.
> >
> > The code for layer defined controllers was outdated and
> > broken because the example controllers in meta-yocto-bsp as
> > well as the controllers available in masterimage were
> > dependent only on the datastore d being available for
> > __init__ contruction, when the testimage.bbclass and
> > context.py, which initialize test controllers never passed
> > the datastore. This commit fixes all the openembedded
> > controllers to not use the datastore, but the testdata json
> > file that is loaded by the testimage.bbclass. This way the
> > tests can still be exported to be ran outside bitbake. The
> > kwargs and logger is still passed in the constructor and
> > there maybe some redundancies between the kwargs and the td
> > test datstore, but this is not addressed in this commit.
> >
> > In masterimage.py there was a duplicate SystemdbootTarget
> > which was removed. It is my opinion that all targets should
> > go to meta-yocto-bsp or meta/ and should not get special
> > treatment whether they come from poky or other layer.
> >
> > In this commit functionality that was lost in masterimage
> > about retrieving a fresh BB_ORIGENV. The test data json file
> > does not store this variable, so the build-time enviromental
> > variables cannot be retrieved later on, when running the
> > tests.
> >
> > The deploy() method invocation of the test cases was removed
> > in the refactoring and this commit re-adds it in the same
> > logical place. The deploy method was added as an abstract
> > method so that all Test controller classes have this method.
> > This method had been lost and was not used in the
> > simpleremote and qemuwrapper, because these controllers do
> > not need the functionality, but almost any real board
> > requires certain steps to deploy the new images, before the
> > tests can be ran. An example of this need was visible in the
> > master image and subclasses in meta-yocto-bsp.
> >
> > A dependency on images fstypes of qemu was removed from
> > testimage.  The allowed file system types were not relevant
> > for other controllers, and made it so that there was an
> > import of the OEQemuTarget python class applicable to all
> > possible controllers, including non qemu.
> >
> > The SimpleRemoteTarget, OEQemuTarget and a custom
> > controller, developed according to the manual were
> > successfully tested.
> >
> > Signed-off-by: Paulo Neves 
> > ---
> >  meta/classes/testimage.bbclass |  18 +---
> >  meta/lib/oeqa/controllers/masterimage.py   | 107 
> > ++---
> >  .../target/qemu.py => controllers/qemutarget.py}   |  30 --
> >  meta/lib/oeqa/controllers/simpleremote.py  |  33 +++
> >  .../target/ssh.py => controllers/sshtarget.py} |   9 +-
> >  meta/lib/oeqa/core/target/__init__.py  |   4 +
> >  meta/lib/oeqa/runtime/context.py   |  32 +-
> >  meta/lib/oeqa/targetcontrol.py

Re: [OE-core] [meta-oe][PATCH 1/3] testimage: Refactoring and fixing.

2018-08-12 Thread Paulo Neves

I found a bug in my code where changing a variable, for example in
layer.conf does not re-trigger the generation of the json file
containing the data store at the time of image creation. The json file
is generated in
ROOTFS_POSTPROCESS_COMMAND += "write_image_test_data ; " @
meta/classes/rootfs-postcommands.bbclass.

Can anybody tell me if changing a variable in the datastore and
accessing it in the postprocess_command will never trigger the
postprocess_command? It happens that in this case this variable is not
important for the rootfs task itself but only for the post-process.

Paulo Neves
On Fri, Aug 10, 2018 at 9:38 PM Paulo Neves  wrote:
>
> The testimage.bbclass invocation of controllers besides
> simpletarget and qemutarget was broken.  Setting TEST_TARGET
> to class names did not work, only hardcoded values set in
> context.py were allowed.  On the other hand the other
> TEST_TARGETs available were supposedly available through the
> class name convetion but none of them were working. See:
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=12842
>
> In this commit qemuwrapper and simpletarget TEST_TARGETS are
> now not available through these names, but through the
> common convention of class names.
>
> The code for layer defined controllers was outdated and
> broken because the example controllers in meta-yocto-bsp as
> well as the controllers available in masterimage were
> dependent only on the datastore d being available for
> __init__ contruction, when the testimage.bbclass and
> context.py, which initialize test controllers never passed
> the datastore. This commit fixes all the openembedded
> controllers to not use the datastore, but the testdata json
> file that is loaded by the testimage.bbclass. This way the
> tests can still be exported to be ran outside bitbake. The
> kwargs and logger is still passed in the constructor and
> there maybe some redundancies between the kwargs and the td
> test datstore, but this is not addressed in this commit.
>
> In masterimage.py there was a duplicate SystemdbootTarget
> which was removed. It is my opinion that all targets should
> go to meta-yocto-bsp or meta/ and should not get special
> treatment whether they come from poky or other layer.
>
> In this commit functionality that was lost in masterimage
> about retrieving a fresh BB_ORIGENV. The test data json file
> does not store this variable, so the build-time enviromental
> variables cannot be retrieved later on, when running the
> tests.
>
> The deploy() method invocation of the test cases was removed
> in the refactoring and this commit re-adds it in the same
> logical place. The deploy method was added as an abstract
> method so that all Test controller classes have this method.
> This method had been lost and was not used in the
> simpleremote and qemuwrapper, because these controllers do
> not need the functionality, but almost any real board
> requires certain steps to deploy the new images, before the
> tests can be ran. An example of this need was visible in the
> master image and subclasses in meta-yocto-bsp.
>
> A dependency on images fstypes of qemu was removed from
> testimage.  The allowed file system types were not relevant
> for other controllers, and made it so that there was an
> import of the OEQemuTarget python class applicable to all
> possible controllers, including non qemu.
>
> The SimpleRemoteTarget, OEQemuTarget and a custom
> controller, developed according to the manual were
> successfully tested.
>
> Signed-off-by: Paulo Neves 
> ---
>  meta/classes/testimage.bbclass |  18 +---
>  meta/lib/oeqa/controllers/masterimage.py   | 107 
> ++---
>  .../target/qemu.py => controllers/qemutarget.py}   |  30 --
>  meta/lib/oeqa/controllers/simpleremote.py  |  33 +++
>  .../target/ssh.py => controllers/sshtarget.py} |   9 +-
>  meta/lib/oeqa/core/target/__init__.py  |   4 +
>  meta/lib/oeqa/runtime/context.py   |  32 +-
>  meta/lib/oeqa/targetcontrol.py |  97 ++-
>  8 files changed, 137 insertions(+), 193 deletions(-)
>  rename meta/lib/oeqa/{core/target/qemu.py => controllers/qemutarget.py} (58%)
>  create mode 100644 meta/lib/oeqa/controllers/simpleremote.py
>  rename meta/lib/oeqa/{core/target/ssh.py => controllers/sshtarget.py} (98%)
>
> diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
> index 6fa901b..2aa59b0 100644
> --- a/meta/classes/testimage.bbclass
> +++ b/meta/classes/testimage.bbclass
> @@ -75,7 +75,7 @@ DEFAULT_TEST_SUITES_remove_qemumips64 = "${MIPSREMOVE}"
>  TEST_SUITES ?= "${DEFAULT_TEST_SUITES}"
>
>  TEST_QEMUBOOT_TIMEOU

[OE-core] [meta-oe][PATCH 2/3] testimage: target.start exceptions not masked.

2018-08-10 Thread Paulo Neves

Previously the target.start exceptions were being
masked by the catch clause of RuntimeError and
BlockingIOError which are very broad. We decoupled
the start method try clause from the runTests try
clause which requires catching the BlockingIOError and
RuntimeError.

Signed-off-by: Paulo Neves 
---
 meta/classes/testimage.bbclass | 36 +---
 1 file changed, 21 insertions(+), 15 deletions(-)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 2aa59b0..f58d11d 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -269,24 +269,30 @@ def testimage_main(d):
 # Add systemd.log_level=debug to enable systemd debug logging
 bootparams = 'systemd.log_target=console'
 
+
 tc.target.deploy()
-results = None
-orig_sigterm_handler = signal.signal(signal.SIGTERM, sigterm_exception)
+
+#We do not want the runTests to mask exeptions of target.start but we want 
a failed start
+#to trigger a stop.
 try:
-# We need to check if runqemu ends unexpectedly
-# or if the worker send us a SIGTERM
-tc.target.start(extra_bootparams=bootparams)
-results = tc.runTests()
-except (RuntimeError, BlockingIOError) as err:
-if isinstance(err, RuntimeError):
-bb.error('testimage received SIGTERM, shutting down...')
-else:
-bb.error('runqemu failed, shutting down...')
-if results:
-results.stop()
-results = None
+tc.target.start(extra_bootparams=bootparams) 
+results = None
+orig_sigterm_handler = signal.signal(signal.SIGTERM, sigterm_exception)
+try:
+# We need to check if runqemu ends unexpectedly
+# or if the worker send us a SIGTERM
+results = tc.runTests()
+except (RuntimeError, BlockingIOError) as err:
+if isinstance(err, RuntimeError):
+bb.error('testimage received SIGTERM, shutting down...')
+else:
+bb.error('runqemu failed, shutting down...')
+if results:
+results.stop()
+results = None
+finally:
+signal.signal(signal.SIGTERM, orig_sigterm_handler)
 finally:
-signal.signal(signal.SIGTERM, orig_sigterm_handler)
 tc.target.stop()
 
 # Show results (if we have them)
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [meta-oe][PATCH 1/3] testimage: Refactoring and fixing.

2018-08-10 Thread Paulo Neves

The testimage.bbclass invocation of controllers besides
simpletarget and qemutarget was broken.  Setting TEST_TARGET
to class names did not work, only hardcoded values set in
context.py were allowed.  On the other hand the other
TEST_TARGETs available were supposedly available through the
class name convetion but none of them were working. See:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=12842

In this commit qemuwrapper and simpletarget TEST_TARGETS are
now not available through these names, but through the
common convention of class names.

The code for layer defined controllers was outdated and
broken because the example controllers in meta-yocto-bsp as
well as the controllers available in masterimage were
dependent only on the datastore d being available for
__init__ contruction, when the testimage.bbclass and
context.py, which initialize test controllers never passed
the datastore. This commit fixes all the openembedded
controllers to not use the datastore, but the testdata json
file that is loaded by the testimage.bbclass. This way the
tests can still be exported to be ran outside bitbake. The
kwargs and logger is still passed in the constructor and
there maybe some redundancies between the kwargs and the td
test datstore, but this is not addressed in this commit.

In masterimage.py there was a duplicate SystemdbootTarget
which was removed. It is my opinion that all targets should
go to meta-yocto-bsp or meta/ and should not get special
treatment whether they come from poky or other layer.

In this commit functionality that was lost in masterimage
about retrieving a fresh BB_ORIGENV. The test data json file
does not store this variable, so the build-time enviromental
variables cannot be retrieved later on, when running the
tests.

The deploy() method invocation of the test cases was removed
in the refactoring and this commit re-adds it in the same
logical place. The deploy method was added as an abstract
method so that all Test controller classes have this method.
This method had been lost and was not used in the
simpleremote and qemuwrapper, because these controllers do
not need the functionality, but almost any real board
requires certain steps to deploy the new images, before the
tests can be ran. An example of this need was visible in the
master image and subclasses in meta-yocto-bsp.

A dependency on images fstypes of qemu was removed from
testimage.  The allowed file system types were not relevant
for other controllers, and made it so that there was an
import of the OEQemuTarget python class applicable to all
possible controllers, including non qemu.

The SimpleRemoteTarget, OEQemuTarget and a custom
controller, developed according to the manual were
successfully tested.

Signed-off-by: Paulo Neves 
---
 meta/classes/testimage.bbclass |  18 +---
 meta/lib/oeqa/controllers/masterimage.py   | 107 ++---
 .../target/qemu.py => controllers/qemutarget.py}   |  30 --
 meta/lib/oeqa/controllers/simpleremote.py  |  33 +++
 .../target/ssh.py => controllers/sshtarget.py} |   9 +-
 meta/lib/oeqa/core/target/__init__.py  |   4 +
 meta/lib/oeqa/runtime/context.py   |  32 +-
 meta/lib/oeqa/targetcontrol.py |  97 ++-
 8 files changed, 137 insertions(+), 193 deletions(-)
 rename meta/lib/oeqa/{core/target/qemu.py => controllers/qemutarget.py} (58%)
 create mode 100644 meta/lib/oeqa/controllers/simpleremote.py
 rename meta/lib/oeqa/{core/target/ssh.py => controllers/sshtarget.py} (98%)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 6fa901b..2aa59b0 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -75,7 +75,7 @@ DEFAULT_TEST_SUITES_remove_qemumips64 = "${MIPSREMOVE}"
 TEST_SUITES ?= "${DEFAULT_TEST_SUITES}"
 
 TEST_QEMUBOOT_TIMEOUT ?= "1000"
-TEST_TARGET ?= "qemu"
+TEST_TARGET ?= "OEQemuTarget"
 
 TESTIMAGEDEPENDS = ""
 TESTIMAGEDEPENDS_qemuall = "qemu-native:do_populate_sysroot 
qemu-helper-native:do_populate_sysroot 
qemu-helper-native:do_addto_recipe_sysroot"
@@ -147,7 +147,6 @@ def testimage_main(d):
 from oeqa.core.utils.misc import updateTestData
 from oeqa.runtime.context import OERuntimeTestContext
 from oeqa.runtime.context import OERuntimeTestContextExecutor
-from oeqa.core.target.qemu import supported_fstypes
 from oeqa.core.utils.test import getSuiteCases
 from oeqa.utils import make_logger_bitbake_compatible
 
@@ -189,15 +188,6 @@ def testimage_main(d):
 # Get machine
 machine = d.getVar("MACHINE")
 
-# Get rootfs
-fstypes = [fs for fs in d.getVar('IMAGE_FSTYPES').split(' ')
-  if fs in supported_fstypes]
-if not fstypes:
-bb.fatal('Unsupported image type built. Add a comptible image to '
-

[OE-core] [meta-oe][PATCH 3/3] masterimage: Check for rootfs path instead of file.

2018-08-10 Thread Paulo Neves

The rootfs might be a directory path or a file path. A
case in point is if the test wants to deploy a rootfs
directly by rsyncing it to the target or even use it
as part of a tftp server.

Signed-off-by: Paulo Neves 
---
 meta/lib/oeqa/controllers/masterimage.py | 6 +-
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/meta/lib/oeqa/controllers/masterimage.py 
b/meta/lib/oeqa/controllers/masterimage.py
index 26e3870..fb8ab50 100644
--- a/meta/lib/oeqa/controllers/masterimage.py
+++ b/meta/lib/oeqa/controllers/masterimage.py
@@ -51,11 +51,7 @@ class 
MasterImageHardwareTarget(oeqa.targetcontrol.BaseTarget, metaclass=ABCMeta
 self.image_fstype = self.get_image_fstype(td)
 self.rootfs = os.path.join(td['DEPLOY_DIR_IMAGE'], 
td['IMAGE_LINK_NAME'] + '.' + self.image_fstype)
 self.kernel = os.path.join(td['DEPLOY_DIR_IMAGE'], 
td["KERNEL_IMAGETYPE"] + '-' + td['MACHINE'] + '.bin')
-if not os.path.isfile(self.rootfs):
-# we could've checked that IMAGE_FSTYPES contains tar.gz but the 
config for running testimage might not be
-# the same as the config with which the image was build, ie
-# you bitbake core-image-sato with IMAGE_FSTYPES += "tar.gz"
-# and your autobuilder overwrites the config, adds the test bits 
and runs bitbake core-image-sato -c testimage
+if not os.path.exists(self.rootfs):
 bb.fatal("No rootfs found. Did you build the image ?\n"  \
   "If yes, did you build it with IMAGE_FSTYPES += \"{}\" ?"  \
   "\nExpected path: {}".format(" 
".join(self.supported_image_fstypes), self.rootfs))
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] classes/reproducible_build: Avoid dereferencing symlinks

2018-08-08 Thread Paulo Neves

minor nitpick. Your patch changed the whitespace indentation ;)

On Mon, Aug 6, 2018 at 5:25 PM, Joshua Watt  wrote:
> Using os.path.getmtime() will dereference symbolic links in an attempt
> to get the last modified time. This can cause errors if the target
> doesn't exist, or worse map to some absolute build host path which would
> make a build not reproducible.
>
> Signed-off-by: Joshua Watt 
> ---
>  meta/classes/reproducible_build.bbclass | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/meta/classes/reproducible_build.bbclass 
> b/meta/classes/reproducible_build.bbclass
> index 2df805330aa..268b5fb8f19 100644
> --- a/meta/classes/reproducible_build.bbclass
> +++ b/meta/classes/reproducible_build.bbclass
> @@ -56,7 +56,7 @@ def get_source_date_epoch_known_files(d, path):
>  for file in known_files:
>  filepath = os.path.join(path,file)
>  if os.path.isfile(filepath):
> -mtime = int(os.path.getmtime(filepath))
> +mtime = int(os.lstat(filepath).st_mtime)
>  # There may be more than one "known_file" present, if so, use 
> the youngest one
>  if mtime > source_date_epoch:
>  source_date_epoch = mtime
> @@ -114,7 +114,7 @@ python do_create_source_date_epoch_stamp() {
>  for fname in files:
>  filename = os.path.join(root, fname)
>  try:
> -mtime = int(os.path.getmtime(filename))
> +mtime = int(os.lstat(filename).st_mtime)
>  except ValueError:
>  mtime = 0
>  if mtime > source_date_epoch:
> --
> 2.17.1
>
> --
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] ✗ patchtest: failure for "[meta-oe,v2] context.py: Do no..." and 3 more

2018-08-06 Thread Paulo Neves

Can anybody help me sort out this changeset?
As this was my second attempt at submitting my changes with errors on
not using the correct mailing list I checked the patchtest-oe and I
saw that I would need to split my series into 3 mailing lists
[oe-core, doc, poky]. The splitting would need the commit to be broken
and it would also lead to quite strange singular commit patches in the
different mailing lists. For example the context code[oe-core] which
invokes controllers in meta-yocto-bsp would not work. Are some of the
changes I made forbidden?

On Mon, Aug 6, 2018 at 1:32 PM, Patchwork
 wrote:
> == Series Details ==
>
> Series: "[meta-oe,v2] context.py: Do no..." and 3 more
> Revision: 1
> URL   : https://patchwork.openembedded.org/series/13399/
> State : failure
>
> == Summary ==
>
>
> Thank you for submitting this patch series to OpenEmbedded Core. This is
> an automated response. Several tests have been executed on the proposed
> series by patchtest resulting in the following failures:
>
>
>
> * Patch[meta-oe,v2,1/4] context.py: Do not mask exceptions 
> needlessly.
>  Issue Series sent to the wrong mailing list 
> [test_target_mailing_list]
>   Suggested fixCheck the project's README (meta-oe,v2,1/4) and send the 
> patch to the indicated list
>
> * Issue Series does not apply on top of target branch 
> [test_series_merge_on_head]
>   Suggested fixRebase your series on top of targeted branch
>   Targeted branch  master (currently at 1c4d9f46d5)
>
>
>
> If you believe any of these test results are incorrect, please reply to the
> mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
> Otherwise we would appreciate you correcting the issues and submitting a new
> version of the patchset if applicable. Please ensure you add/increment the
> version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
> [PATCH v3] -> ...).
>
> ---
> Guidelines: 
> https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
> Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
> Test suite: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe
>
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [meta-oe][PATCH v2 4/4] masterimage: Check for rootfs path instead of file.

2018-08-06 Thread Paulo Neves

The rootfs might be a directory path or a file path. A
case in point is if the test wants to deploy a rootfs
directly by rsyncing it to the target or even use it
as part of a tftp server.

Signed-off-by: Paulo Neves 
---
 meta/lib/oeqa/controllers/masterimage.py | 6 +-
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/meta/lib/oeqa/controllers/masterimage.py 
b/meta/lib/oeqa/controllers/masterimage.py
index 26e3870..fb8ab50 100644
--- a/meta/lib/oeqa/controllers/masterimage.py
+++ b/meta/lib/oeqa/controllers/masterimage.py
@@ -51,11 +51,7 @@ class 
MasterImageHardwareTarget(oeqa.targetcontrol.BaseTarget, metaclass=ABCMeta
 self.image_fstype = self.get_image_fstype(td)
 self.rootfs = os.path.join(td['DEPLOY_DIR_IMAGE'], 
td['IMAGE_LINK_NAME'] + '.' + self.image_fstype)
 self.kernel = os.path.join(td['DEPLOY_DIR_IMAGE'], 
td["KERNEL_IMAGETYPE"] + '-' + td['MACHINE'] + '.bin')
-if not os.path.isfile(self.rootfs):
-# we could've checked that IMAGE_FSTYPES contains tar.gz but the 
config for running testimage might not be
-# the same as the config with which the image was build, ie
-# you bitbake core-image-sato with IMAGE_FSTYPES += "tar.gz"
-# and your autobuilder overwrites the config, adds the test bits 
and runs bitbake core-image-sato -c testimage
+if not os.path.exists(self.rootfs):
 bb.fatal("No rootfs found. Did you build the image ?\n"  \
   "If yes, did you build it with IMAGE_FSTYPES += \"{}\" ?"  \
   "\nExpected path: {}".format(" 
".join(self.supported_image_fstypes), self.rootfs))
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [meta-oe][PATCH v2 2/4] testimage: Refactoring and fixing.

2018-08-06 Thread Paulo Neves

The testimage.bbclass invocation of controllers besides
simpletarget and qemutarget was broken.
Setting TEST_TARGET to
class names did not work, only hardcoded values set in context.py were allowed.
On the other hand the other TEST_TARGETs available were
supposedly available through the class name convetion but
none of them were working. See:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=12842

In this commit qemuwrapper and simpletarget TEST_TARGETS are now
not available through these names, but through the common
convention of class names.

The code for layer defined controllers was outdated and broken
because the example controllers in meta-yocto-bsp as well as
the controllers available in masterimage were dependent only
on the datastore d being available for __init__ contruction, when
the testimage.bbclass and context.py, which initialize
test controllers never passed the datastore. This commit fixes
all the controllers to not use the datastore, but the testdata
json file that is loaded by the testimage.bbclass. This way
the tests can still be exported to be ran outside bitbake. The
kwargs and logger is still passed in the constructor and there
maybe some redundancies between the kwargs and the td test
datstore, but this is not addressed in this commit.

In masterimage.py there was a duplicate SystemdbootTarget which
was removed. It is my opinion that all targets should go to
meta-yocto-bsp or meta/ and should not get special treatment whether they come
from poky or other layer.

There was a functionality that was lost in masterimage about
retrieving a fresh BB_ORIGENV. The test data json file does not
store this, so the build-time enviromental variables cannot
be retrieved later on, when running the tests.

The deploy() method invocation of the test cases was removed in
the refactoring and this commit re-adds it in the same logical
place. The deploy method was added as an abstract method so that
all Test controller classes have this method. This method had been
lost and was not used in the simpleremote and qemuwrapper, because
these controllers do not need the functionality, but almost any
real board requires certain steps to deploy the new images, before
the tests can be ran. An example of this need was visible in the
master image and subclasses in meta-yocto-bsp.

A dependency on images fstypes of qemu was removed from testimage.
The allowed file system types were not relevant for other controllers,
and made it so that there was an import of the OEQemuTarget python
class applicable to all possible controllers, including non qemu.

The SimpleRemoteTarget, OEQemuTarget and a custom controller,
developed according to the manual were successfully tested.

Signed-off-by: Paulo Neves 
---
 .../lib/oeqa/controllers/beaglebonetarget.py   |   6 +-
 .../lib/oeqa/controllers/edgeroutertarget.py   |   6 +-
 meta-yocto-bsp/lib/oeqa/controllers/grubtarget.py  |   4 +-
 meta/classes/testimage.bbclass |  18 +---
 meta/lib/oeqa/controllers/masterimage.py   | 107 ++---
 .../target/qemu.py => controllers/qemutarget.py}   |  30 --
 meta/lib/oeqa/controllers/simpleremote.py  |  33 +++
 .../target/ssh.py => controllers/sshtarget.py} |   9 +-
 meta/lib/oeqa/core/target/__init__.py  |   4 +
 meta/lib/oeqa/runtime/context.py   |  32 +-
 meta/lib/oeqa/targetcontrol.py |  97 ++-
 11 files changed, 145 insertions(+), 201 deletions(-)
 rename meta/lib/oeqa/{core/target/qemu.py => controllers/qemutarget.py} (58%)
 create mode 100644 meta/lib/oeqa/controllers/simpleremote.py
 rename meta/lib/oeqa/{core/target/ssh.py => controllers/sshtarget.py} (98%)

diff --git a/meta-yocto-bsp/lib/oeqa/controllers/beaglebonetarget.py 
b/meta-yocto-bsp/lib/oeqa/controllers/beaglebonetarget.py
index 0f1aeb3..440a8f8 100644
--- a/meta-yocto-bsp/lib/oeqa/controllers/beaglebonetarget.py
+++ b/meta-yocto-bsp/lib/oeqa/controllers/beaglebonetarget.py
@@ -36,10 +36,10 @@ class BeagleBoneTarget(MasterImageHardwareTarget):
 def get_extra_files(self):
 return list(self.dtbs.keys())
 
-def __init__(self, d):
-super(BeagleBoneTarget, self).__init__(d)
+def __init__(self, td, logger, **kwargs):
+super(BeagleBoneTarget, self).__init__(td, logger, **kwargs)
 
-self.image_fstype = self.get_image_fstype(d)
+self.image_fstype = self.get_image_fstype(td)
 self.deploy_cmds = [
 'mkdir -p /mnt/testrootfs',
 'mount -L testrootfs /mnt/testrootfs',
diff --git a/meta-yocto-bsp/lib/oeqa/controllers/edgeroutertarget.py 
b/meta-yocto-bsp/lib/oeqa/controllers/edgeroutertarget.py
index 9c47b5b..3c5484c 100644
--- a/meta-yocto-bsp/lib/oeqa/controllers/edgeroutertarget.py
+++ b/meta-yocto-bsp/lib/oeqa/controllers/edgeroutertarget.py
@@ -41,10 +41,10 @@ from oeqa.controllers.masterimage import 
MasterImageHardware

[OE-core] [meta-oe][PATCH v2 1/4] context.py: Do not mask exceptions needlessly.

2018-08-06 Thread Paulo Neves

There were a lot of assumptions in the controller import
code of context.py which were not true anymore.  These
assumptions reflected themselves by catching exceptions and
re-raising them with unhelpful error messages in other parts
of the code.

This commit does not fix the classes controller classes that
became broken after the refactor but at least it allows for
the exceptions to be thrown where the imports fail, thus
actually showing what exactly went wrong with the import.

An example of such an improvement is that before if the
controller class failed during it's init contructor the
controller would just be skipped and the task would just
complain it could not find the controller.  Now for example,
if there is a NamerError due to a variable not being
declared, the user will get that report.

Signed-off-by: Paulo Neves 
---
 meta/lib/oeqa/runtime/context.py | 14 --
 1 file changed, 4 insertions(+), 10 deletions(-)

diff --git a/meta/lib/oeqa/runtime/context.py b/meta/lib/oeqa/runtime/context.py
index 0294003..a7f3823 100644
--- a/meta/lib/oeqa/runtime/context.py
+++ b/meta/lib/oeqa/runtime/context.py
@@ -112,12 +112,9 @@ class OERuntimeTestContextExecutor(OETestContextExecutor):
 # XXX: Don't base your targets on this code it will be refactored
 # in the near future.
 # Custom target module loading
-try:
-target_modules_path = kwargs.get('target_modules_path', '')
-controller = 
OERuntimeTestContextExecutor.getControllerModule(target_type, 
target_modules_path)
-target = controller(logger, target_ip, server_ip, **kwargs)
-except ImportError as e:
-raise TypeError("Failed to import %s from available controller 
modules" % target_type)
+target_modules_path = kwargs.get('target_modules_path', '')
+controller = 
OERuntimeTestContextExecutor.getControllerModule(target_type, 
target_modules_path)
+target = controller(logger, target_ip, server_ip, **kwargs)
 
 return target
 
@@ -173,10 +170,7 @@ class OERuntimeTestContextExecutor(OETestContextExecutor):
 def _loadControllerFromModule(target, modulename):
 obj = None
 # import module, allowing it to raise import exception
-try:
-module = __import__(modulename, globals(), locals(), [target])
-except Exception as e:
-return obj
+module = __import__(modulename, globals(), locals(), [target])
 # look for target class in the module, catching any exceptions as it
 # is valid that a module may not have the target class.
 try:
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [meta-oe][PATCH v2 3/4] testimage: target.start exceptions not masked.

2018-08-06 Thread Paulo Neves

Previously the target.start exceptions were being
masked by the catch clause of RuntimeError and
BlockingIOError which are very broad. We decoupled
the start method try clause from the runTests try
clause which requires catching the BlockingIOError and
RuntimeError.

Signed-off-by: Paulo Neves 
---
 meta/classes/testimage.bbclass | 36 +---
 1 file changed, 21 insertions(+), 15 deletions(-)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 2aa59b0..f58d11d 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -269,24 +269,30 @@ def testimage_main(d):
 # Add systemd.log_level=debug to enable systemd debug logging
 bootparams = 'systemd.log_target=console'
 
+
 tc.target.deploy()
-results = None
-orig_sigterm_handler = signal.signal(signal.SIGTERM, sigterm_exception)
+
+#We do not want the runTests to mask exeptions of target.start but we want 
a failed start
+#to trigger a stop.
 try:
-# We need to check if runqemu ends unexpectedly
-# or if the worker send us a SIGTERM
-tc.target.start(extra_bootparams=bootparams)
-results = tc.runTests()
-except (RuntimeError, BlockingIOError) as err:
-if isinstance(err, RuntimeError):
-bb.error('testimage received SIGTERM, shutting down...')
-else:
-bb.error('runqemu failed, shutting down...')
-if results:
-results.stop()
-results = None
+tc.target.start(extra_bootparams=bootparams) 
+results = None
+orig_sigterm_handler = signal.signal(signal.SIGTERM, sigterm_exception)
+try:
+# We need to check if runqemu ends unexpectedly
+# or if the worker send us a SIGTERM
+results = tc.runTests()
+except (RuntimeError, BlockingIOError) as err:
+if isinstance(err, RuntimeError):
+bb.error('testimage received SIGTERM, shutting down...')
+else:
+bb.error('runqemu failed, shutting down...')
+if results:
+results.stop()
+results = None
+finally:
+signal.signal(signal.SIGTERM, orig_sigterm_handler)
 finally:
-signal.signal(signal.SIGTERM, orig_sigterm_handler)
 tc.target.stop()
 
 # Show results (if we have them)
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 3/4] testimage: target.start exceptions not masked.

2018-08-05 Thread Paulo Neves

Previously the target.start exceptions were being
masked by the catch clause of RuntimeError and
BlockingIOError which are very broad. We decoupled
the start method try clause from the runTests try
clause which requires catching the BlockingIOError and
RuntimeError.

Signed-off-by: Paulo Neves 
---
 meta/classes/testimage.bbclass | 36 +---
 1 file changed, 21 insertions(+), 15 deletions(-)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 1b33f72..8e42294 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -264,24 +264,30 @@ def testimage_main(d):
 # Add systemd.log_level=debug to enable systemd debug logging
 bootparams = 'systemd.log_target=console'
 
+
 tc.target.deploy()
-results = None
-orig_sigterm_handler = signal.signal(signal.SIGTERM, sigterm_exception)
+
+#We do not want the runTests to mask exeptions of target.start but we want 
a failed start
+#to trigger a stop.
 try:
-# We need to check if runqemu ends unexpectedly
-# or if the worker send us a SIGTERM
-tc.target.start(extra_bootparams=bootparams)
-results = tc.runTests()
-except (RuntimeError, BlockingIOError) as err:
-if isinstance(err, RuntimeError):
-bb.error('testimage received SIGTERM, shutting down...')
-else:
-bb.error('runqemu failed, shutting down...')
-if results:
-results.stop()
-results = None
+tc.target.start(extra_bootparams=bootparams) 
+results = None
+orig_sigterm_handler = signal.signal(signal.SIGTERM, sigterm_exception)
+try:
+# We need to check if runqemu ends unexpectedly
+# or if the worker send us a SIGTERM
+results = tc.runTests()
+except (RuntimeError, BlockingIOError) as err:
+if isinstance(err, RuntimeError):
+bb.error('testimage received SIGTERM, shutting down...')
+else:
+bb.error('runqemu failed, shutting down...')
+if results:
+results.stop()
+results = None
+finally:
+signal.signal(signal.SIGTERM, orig_sigterm_handler)
 finally:
-signal.signal(signal.SIGTERM, orig_sigterm_handler)
 tc.target.stop()
 
 # Show results (if we have them)
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 4/4] masterimage: Check for rootfs path instead of file.

2018-08-05 Thread Paulo Neves

The rootfs might be a directory path or a file path. A
case in point is if the test wants to deploy a rootfs
directly by rsyncing it to the target or even use it
as part of a tftp server.

Signed-off-by: Paulo Neves 
---
 meta/lib/oeqa/controllers/masterimage.py | 6 +-
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/meta/lib/oeqa/controllers/masterimage.py 
b/meta/lib/oeqa/controllers/masterimage.py
index 26e3870..fb8ab50 100644
--- a/meta/lib/oeqa/controllers/masterimage.py
+++ b/meta/lib/oeqa/controllers/masterimage.py
@@ -51,11 +51,7 @@ class 
MasterImageHardwareTarget(oeqa.targetcontrol.BaseTarget, metaclass=ABCMeta
 self.image_fstype = self.get_image_fstype(td)
 self.rootfs = os.path.join(td['DEPLOY_DIR_IMAGE'], 
td['IMAGE_LINK_NAME'] + '.' + self.image_fstype)
 self.kernel = os.path.join(td['DEPLOY_DIR_IMAGE'], 
td["KERNEL_IMAGETYPE"] + '-' + td['MACHINE'] + '.bin')
-if not os.path.isfile(self.rootfs):
-# we could've checked that IMAGE_FSTYPES contains tar.gz but the 
config for running testimage might not be
-# the same as the config with which the image was build, ie
-# you bitbake core-image-sato with IMAGE_FSTYPES += "tar.gz"
-# and your autobuilder overwrites the config, adds the test bits 
and runs bitbake core-image-sato -c testimage
+if not os.path.exists(self.rootfs):
 bb.fatal("No rootfs found. Did you build the image ?\n"  \
   "If yes, did you build it with IMAGE_FSTYPES += \"{}\" ?"  \
   "\nExpected path: {}".format(" 
".join(self.supported_image_fstypes), self.rootfs))
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/4] testimage: Refactoring and fixing.

2018-08-05 Thread Paulo Neves

The testimage.bbclass invocation of controllers besides
simpletarget and qemutarget was broken.
Setting TEST_TARGET to
class names did not work, only hardcoded values set in context.py were allowed.
On the other hand the other TEST_TARGETs available were
supposedly available through the class name convetion but
none of them were working. See:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=12842

In this commit qemuwrapper and simpletarget TEST_TARGETS are now
not available through these names, but through the common
convention of class names.

The code for layer defined controllers was outdated and broken
because the example controllers in meta-yocto-bsp as well as
the controllers available in masterimage were dependent only
on the datastore d being available for __init__ contruction, when
the testimage.bbclass and context.py, which initialize
test controllers never passed the datastore. This commit fixes
all the controllers to not use the datastore, but the testdata
json file that is loaded by the testimage.bbclass. This way
the tests can still be exported to be ran outside bitbake. The
kwargs and logger is still passed in the constructor and there
maybe some redundancies between the kwargs and the td test
datstore, but this is not addressed in this commit.

In masterimage.py there was a duplicate SystemdbootTarget which
was removed. It is my opinion that all targets should go to
meta-yocto-bsp or meta/ and should not get special treatment whether they come
from poky or other layer.

There was a functionality that was lost in masterimage about
retrieving a fresh BB_ORIGENV. The test data json file does not
store this, so the build-time enviromental variables cannot
be retrieved later on, when running the tests.

The deploy() method invocation of the test cases was removed in
the refactoring and this commit re-adds it in the same logical
place. The deploy method was added as an abstract method so that
all Test controller classes have this method. This method had been
lost and was not used in the simpleremote and qemuwrapper, because
these controllers do not need the functionality, but almost any
real board requires certain steps to deploy the new images, before
the tests can be ran. An example of this need was visible in the
master image and subclasses in meta-yocto-bsp.

A dependency on images fstypes of qemu was removed from testimage.
The allowed file system types were not relevant for other controllers,
and made it so that there was an import of the OEQemuTarget python
class applicable to all possible controllers, including non qemu.

The manual was changed wherever I saw it was appropriate but,
some improvements may still be needed.

The SimpleRemoteTarget, OEQemuTarget and a custom controller,
developed according to the manual were successfully tested.

Signed-off-by: Paulo Neves 
---
 .../dev-manual/dev-manual-common-tasks.xml |   8 +-
 documentation/ref-manual/ref-variables.xml |  19 +---
 .../lib/oeqa/controllers/beaglebonetarget.py   |   6 +-
 .../lib/oeqa/controllers/edgeroutertarget.py   |   6 +-
 meta-yocto-bsp/lib/oeqa/controllers/grubtarget.py  |   4 +-
 meta/classes/testimage.bbclass |  18 +---
 meta/lib/oeqa/controllers/masterimage.py   | 107 ++---
 .../target/qemu.py => controllers/qemutarget.py}   |  30 --
 meta/lib/oeqa/controllers/simpleremote.py  |  33 +++
 .../target/ssh.py => controllers/sshtarget.py} |   9 +-
 meta/lib/oeqa/core/target/__init__.py  |   4 +
 meta/lib/oeqa/runtime/context.py   |  32 +-
 meta/lib/oeqa/targetcontrol.py |  97 ++-
 13 files changed, 153 insertions(+), 220 deletions(-)
 rename meta/lib/oeqa/{core/target/qemu.py => controllers/qemutarget.py} (58%)
 create mode 100644 meta/lib/oeqa/controllers/simpleremote.py
 rename meta/lib/oeqa/{core/target/ssh.py => controllers/sshtarget.py} (98%)

diff --git a/documentation/dev-manual/dev-manual-common-tasks.xml 
b/documentation/dev-manual/dev-manual-common-tasks.xml
index fe1bfba..95aa533 100644
--- a/documentation/dev-manual/dev-manual-common-tasks.xml
+++ b/documentation/dev-manual/dev-manual-common-tasks.xml
@@ -11400,7 +11400,7 @@
 within a separately started QEMU or any
 other virtual machine manager.
 
-
"Systemd-bootTarget":
+
"SystemdbootTarget":
 Choose "Systemd-bootTarget" if your hardware is
 an EFI-based machine with
 systemd-boot as bootloader and
@@ -11459,10 +11459,10 @@
 
 
 
-Selecting Systemd-bootTarget
+Selecting SystemdbootTarget
 
 
-If you did not set TEST_TARGET to
+

[OE-core] [PATCH 1/4] context.py: Do not mask exceptions needlessly.

2018-08-05 Thread Paulo Neves

There were a lot of assumptions in the controller import
code of context.py which were not true anymore.  These
assumptions reflected themselves by catching exceptions and
re-raising them with unhelpful error messages in other parts
of the code.

This commit does not fix the classes controller classes that
became broken after the refactor but at least it allows for
the exceptions to be thrown where the imports fail, thus
actually showing what exactly went wrong with the import.

An example of such an improvement is that before if the
controller class failed during it's init contructor the
controller would just be skipped and the task would just
complain it could not find the controller.  Now for example,
if there is a NamerError due to a variable not being
declared, the user will get that report.

Signed-off-by: Paulo Neves 
---
 meta/lib/oeqa/runtime/context.py | 14 --
 1 file changed, 4 insertions(+), 10 deletions(-)

diff --git a/meta/lib/oeqa/runtime/context.py b/meta/lib/oeqa/runtime/context.py
index 0294003..a7f3823 100644
--- a/meta/lib/oeqa/runtime/context.py
+++ b/meta/lib/oeqa/runtime/context.py
@@ -112,12 +112,9 @@ class OERuntimeTestContextExecutor(OETestContextExecutor):
 # XXX: Don't base your targets on this code it will be refactored
 # in the near future.
 # Custom target module loading
-try:
-target_modules_path = kwargs.get('target_modules_path', '')
-controller = 
OERuntimeTestContextExecutor.getControllerModule(target_type, 
target_modules_path)
-target = controller(logger, target_ip, server_ip, **kwargs)
-except ImportError as e:
-raise TypeError("Failed to import %s from available controller 
modules" % target_type)
+target_modules_path = kwargs.get('target_modules_path', '')
+controller = 
OERuntimeTestContextExecutor.getControllerModule(target_type, 
target_modules_path)
+target = controller(logger, target_ip, server_ip, **kwargs)
 
 return target
 
@@ -173,10 +170,7 @@ class OERuntimeTestContextExecutor(OETestContextExecutor):
 def _loadControllerFromModule(target, modulename):
 obj = None
 # import module, allowing it to raise import exception
-try:
-module = __import__(modulename, globals(), locals(), [target])
-except Exception as e:
-return obj
+module = __import__(modulename, globals(), locals(), [target])
 # look for target class in the module, catching any exceptions as it
 # is valid that a module may not have the target class.
 try:
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] u-boot crash on network code hardkernel v2017.05

2018-07-23 Thread Paulo Neves

@Alex: Thanks for that commit id. I tried to manually add the
attribute but just to the net include[1], but the problem had not been
solved. I see in this commit more packed attributes changes, so I will
give it a try and let you all know.

>can you see if latest u-boot also have similar issues ?
>For me in past hardkernel uboot required gcc 4.9 and moment
>i switched to newer gcc it borked.
@Khem I saw your recipe commit, and was ready to commit to the same
solution you did in meta-odroid, of just downloading the prebuilt
linaro toolchain and use it to build u-boot. Even so this raises an
issue: Is there not a better way to depend on a version of GCC for a
specific recipe? Solving issues like these can be way beyond most
people's abilities (at least my own), and having a mechanism in yocto
to choose another gcc version might be desirable.

Paulo Neves

[1] https://lists.denx.de/pipermail/u-boot//2017-July/298986.html

On Sat, Jul 21, 2018 at 9:52 PM, Alex Kiernan  wrote:
> On Sat, Jul 21, 2018 at 11:09 AM Paulo Neves  wrote:
>>
>> Hello,
>>
>> I use a modified version of meta-odroid so that I can build u-boot
>> from hardkernel v2017.05
>> (https://github.com/hardkernel/u-boot.git@88af53fbcef8386cb4d5f04c19f4b2bcb69e90ca)
>>
>> Recently i noticed my u-boot crashes in any network operation (even
>> ping) if it is compiled by the yocto toolchain. To abstract away any
>> recipe differences and narrow down my changes to the toolchain, I
>> cloned the hardkernel repository and made
>>
>> With Ubuntu 16.04.4 LTS and gcc-arm-linux-gnueabihf package [1]
>>
>> make CROSS_COMPILE=arm-linux-gnueabihf- ARCH=arm odroid-xu4_defconfig
>> make CROSS_COMPILE=arm-linux-gnueabihf- ARCH=arm
>> cd ./sdfuse
>> ./sdfuse.sh /dev/mmcblk0
>>
>> The Odroid successfully boots
>>
>> On the other hand the exact same commands on the same workspace but
>> with the PATH set to the yocto recipe-sysroot (sumo) I am able to boot
>> but upon using a network resource it always hangs as below [2]. This
>> is exactly the same crash behavior as the one produced by the normal
>> yocto build.With u-boot built with the yocto toolchain, if i try to
>> boot from the sd card everything works fine. The only way that the
>> yocto toolchain seems to impact is in network related operations in
>> u-boot. I have tried to correct some of the minor warnings that pop up
>> in both versions. I also tried to build with -O0 but the problem
>> persists.
>>
>> [1]
>> ~/Projects/u-boot$ arm-linux-gnueabihf-gcc -v
>> Using built-in specs.
>> COLLECT_GCC=arm-linux-gnueabihf-gcc
>> COLLECT_LTO_WRAPPER=/usr/lib/gcc-cross/arm-linux-gnueabihf/5/lto-wrapper
>> Target: arm-linux-gnueabihf
>> Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro
>> 5.4.0-6ubuntu1~16.04.9'
>> --with-bugurl=file:///usr/share/doc/gcc-5/README.Bugs
>> --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++
>> --prefix=/usr --program-suffix=-5 --enable-shared
>> --enable-linker-build-id --libexecdir=/usr/lib
>> --without-included-gettext --enable-threads=posix --libdir=/usr/lib
>> --enable-nls --with-sysroot=/ --enable-clocale=gnu
>> --enable-libstdcxx-debug --enable-libstdcxx-time=yes
>> --with-default-libstdcxx-abi=new --enable-gnu-unique-object
>> --disable-libitm --disable-libquadmath --enable-plugin
>> --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk
>> --enable-gtk-cairo
>> --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-5-armhf-cross/jre
>> --enable-java-home
>> --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-5-armhf-cross
>> --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-5-armhf-cross
>> --with-arch-directory=arm
>> --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --disable-libgcj
>> --enable-objc-gc --enable-multiarch --enable-multilib
>> --disable-sjlj-exceptions --with-arch=armv7-a --with-fpu=vfpv3-d16
>> --with-float=hard --with-mode=thumb --disable-werror --enable-multilib
>> --enable-checking=release --build=x86_64-linux-gnu
>> --host=x86_64-linux-gnu --target=arm-linux-gnueabihf
>> --program-prefix=arm-linux-gnueabihf-
>> --includedir=/usr/arm-linux-gnueabihf/include
>> Thread model: posix
>> gcc version 5.4.0 20160609 (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.9
>>
>> [2] TFTP from server 192.168.3.123; our IP address is 192.168.3.71
>> Filename 'zImage'.
>> Load address: 0x40008000
>> Loading: *
>>
>>
>> Let me know if you have any suggestions.
>> Paulo Neves
>
> Try cherry-picking 704f3acfcf55343043bbed01c5fb0a0094a68e8a:
>
> http://git.denx.de/?p=u-boot.git;a=commit;h=704f3acfcf55343043bbed01c5fb0a0094a68e8a
>
> --
> Alex Kiernan
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] u-boot crash on network code hardkernel v2017.05

2018-07-21 Thread Paulo Neves

Replying to myself in the hope that it further helps:

I narrowed down the bug to a gcc 7 problem. Using the prebuilt gcc
from linaro, i am able to use networking functionalities in gccv6 but
not gccv7. gcc7 is the default gcc version of poky/sumo.

Paulo Neves

On Sat, Jul 21, 2018 at 12:08 PM, Paulo Neves  wrote:
> Hello,
>
> I use a modified version of meta-odroid so that I can build u-boot
> from hardkernel v2017.05
> (https://github.com/hardkernel/u-boot.git@88af53fbcef8386cb4d5f04c19f4b2bcb69e90ca)
>
> Recently i noticed my u-boot crashes in any network operation (even
> ping) if it is compiled by the yocto toolchain. To abstract away any
> recipe differences and narrow down my changes to the toolchain, I
> cloned the hardkernel repository and made
>
> With Ubuntu 16.04.4 LTS and gcc-arm-linux-gnueabihf package [1]
>
> make CROSS_COMPILE=arm-linux-gnueabihf- ARCH=arm odroid-xu4_defconfig
> make CROSS_COMPILE=arm-linux-gnueabihf- ARCH=arm
> cd ./sdfuse
> ./sdfuse.sh /dev/mmcblk0
>
> The Odroid successfully boots
>
> On the other hand the exact same commands on the same workspace but
> with the PATH set to the yocto recipe-sysroot (sumo) I am able to boot
> but upon using a network resource it always hangs as below [2]. This
> is exactly the same crash behavior as the one produced by the normal
> yocto build.With u-boot built with the yocto toolchain, if i try to
> boot from the sd card everything works fine. The only way that the
> yocto toolchain seems to impact is in network related operations in
> u-boot. I have tried to correct some of the minor warnings that pop up
> in both versions. I also tried to build with -O0 but the problem
> persists.
>
> [1]
> ~/Projects/u-boot$ arm-linux-gnueabihf-gcc -v
> Using built-in specs.
> COLLECT_GCC=arm-linux-gnueabihf-gcc
> COLLECT_LTO_WRAPPER=/usr/lib/gcc-cross/arm-linux-gnueabihf/5/lto-wrapper
> Target: arm-linux-gnueabihf
> Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro
> 5.4.0-6ubuntu1~16.04.9'
> --with-bugurl=file:///usr/share/doc/gcc-5/README.Bugs
> --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++
> --prefix=/usr --program-suffix=-5 --enable-shared
> --enable-linker-build-id --libexecdir=/usr/lib
> --without-included-gettext --enable-threads=posix --libdir=/usr/lib
> --enable-nls --with-sysroot=/ --enable-clocale=gnu
> --enable-libstdcxx-debug --enable-libstdcxx-time=yes
> --with-default-libstdcxx-abi=new --enable-gnu-unique-object
> --disable-libitm --disable-libquadmath --enable-plugin
> --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk
> --enable-gtk-cairo
> --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-5-armhf-cross/jre
> --enable-java-home
> --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-5-armhf-cross
> --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-5-armhf-cross
> --with-arch-directory=arm
> --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --disable-libgcj
> --enable-objc-gc --enable-multiarch --enable-multilib
> --disable-sjlj-exceptions --with-arch=armv7-a --with-fpu=vfpv3-d16
> --with-float=hard --with-mode=thumb --disable-werror --enable-multilib
> --enable-checking=release --build=x86_64-linux-gnu
> --host=x86_64-linux-gnu --target=arm-linux-gnueabihf
> --program-prefix=arm-linux-gnueabihf-
> --includedir=/usr/arm-linux-gnueabihf/include
> Thread model: posix
> gcc version 5.4.0 20160609 (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.9
>
> [2] TFTP from server 192.168.3.123; our IP address is 192.168.3.71
> Filename 'zImage'.
> Load address: 0x40008000
> Loading: *
>
>
> Let me know if you have any suggestions.
> Paulo Neves
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] u-boot crash on network code hardkernel v2017.05

2018-07-21 Thread Paulo Neves

Hello,

I use a modified version of meta-odroid so that I can build u-boot
from hardkernel v2017.05
(https://github.com/hardkernel/u-boot.git@88af53fbcef8386cb4d5f04c19f4b2bcb69e90ca)

Recently i noticed my u-boot crashes in any network operation (even
ping) if it is compiled by the yocto toolchain. To abstract away any
recipe differences and narrow down my changes to the toolchain, I
cloned the hardkernel repository and made

With Ubuntu 16.04.4 LTS and gcc-arm-linux-gnueabihf package [1]

make CROSS_COMPILE=arm-linux-gnueabihf- ARCH=arm odroid-xu4_defconfig
make CROSS_COMPILE=arm-linux-gnueabihf- ARCH=arm
cd ./sdfuse
./sdfuse.sh /dev/mmcblk0

The Odroid successfully boots

On the other hand the exact same commands on the same workspace but
with the PATH set to the yocto recipe-sysroot (sumo) I am able to boot
but upon using a network resource it always hangs as below [2]. This
is exactly the same crash behavior as the one produced by the normal
yocto build.With u-boot built with the yocto toolchain, if i try to
boot from the sd card everything works fine. The only way that the
yocto toolchain seems to impact is in network related operations in
u-boot. I have tried to correct some of the minor warnings that pop up
in both versions. I also tried to build with -O0 but the problem
persists.

[1]
~/Projects/u-boot$ arm-linux-gnueabihf-gcc -v
Using built-in specs.
COLLECT_GCC=arm-linux-gnueabihf-gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc-cross/arm-linux-gnueabihf/5/lto-wrapper
Target: arm-linux-gnueabihf
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro
5.4.0-6ubuntu1~16.04.9'
--with-bugurl=file:///usr/share/doc/gcc-5/README.Bugs
--enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++
--prefix=/usr --program-suffix=-5 --enable-shared
--enable-linker-build-id --libexecdir=/usr/lib
--without-included-gettext --enable-threads=posix --libdir=/usr/lib
--enable-nls --with-sysroot=/ --enable-clocale=gnu
--enable-libstdcxx-debug --enable-libstdcxx-time=yes
--with-default-libstdcxx-abi=new --enable-gnu-unique-object
--disable-libitm --disable-libquadmath --enable-plugin
--with-system-zlib --disable-browser-plugin --enable-java-awt=gtk
--enable-gtk-cairo
--with-java-home=/usr/lib/jvm/java-1.5.0-gcj-5-armhf-cross/jre
--enable-java-home
--with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-5-armhf-cross
--with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-5-armhf-cross
--with-arch-directory=arm
--with-ecj-jar=/usr/share/java/eclipse-ecj.jar --disable-libgcj
--enable-objc-gc --enable-multiarch --enable-multilib
--disable-sjlj-exceptions --with-arch=armv7-a --with-fpu=vfpv3-d16
--with-float=hard --with-mode=thumb --disable-werror --enable-multilib
--enable-checking=release --build=x86_64-linux-gnu
--host=x86_64-linux-gnu --target=arm-linux-gnueabihf
--program-prefix=arm-linux-gnueabihf-
--includedir=/usr/arm-linux-gnueabihf/include
Thread model: posix
gcc version 5.4.0 20160609 (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.9

[2] TFTP from server 192.168.3.123; our IP address is 192.168.3.71
Filename 'zImage'.
Load address: 0x40008000
Loading: *


Let me know if you have any suggestions.
Paulo Neves
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

1 2 >

1 - 100 of 117 matches

Mail list logo