Re: [OE-core] [PATCH v3] perf: Correct the substitution of python shebangs
On 2/21/20 2:23 PM, Bruce Ashfield wrote: On Fri, Feb 21, 2020 at 5:16 PM Khem Raj wrote: On 2/21/20 1:43 PM, Joshua Watt wrote: On Thu, Feb 13, 2020 at 9:58 PM wrote: From: He Zhe To make the native python3 is always used, - Use sed one-liner instead - Add substitution for ${S}/scripts/bpf_helpers_doc.py to fix the following warning. File "/usr/lib/python3.6/sysconfig.py", line 421, in _init_posix _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0) ModuleNotFoundError: No module named '_sysconfigdata' This issue is first reported by Joel Stanley The sed one-liner is credited to Anuj Mittal Signed-off-by: He Zhe --- meta/recipes-kernel/perf/perf.bb | 7 ++- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb index 6d1b06693d..a6fb51d3db 100644 --- a/meta/recipes-kernel/perf/perf.bb +++ b/meta/recipes-kernel/perf/perf.bb @@ -239,11 +239,8 @@ do_configure_prepend () { fi # use /usr/bin/env instead of version specific python -for s in `find ${S}/tools/perf/ -name '*.py'`; do -sed -i 's,/usr/bin/python,/usr/bin/env python3,' "${s}" -sed -i 's,/usr/bin/python2,/usr/bin/env python3,' "${s}" -sed -i 's,/usr/bin/env python2,/usr/bin/env python3,' "${s}" -sed -i 's,/usr/bin/python3,/usr/bin/env python3,' "${s}" +for s in `find ${S}/tools/perf/ -name '*.py'` ${S}/scripts/bpf_helpers_doc.py; do Do we have to hard code bpf_helpers_doc.py? It doesn't exist in older versions (e.g. 4.14) and causes do_configure to fail. Probably not, I also stumbled into this problem. Yah, We can either just do the entire directory (but during the review, we were talking about the fact that since we aren't using them, we don't actually know if they are py3 safe, so blindly converting the shebang isnt' a great idea either). But the solution we use in the devsrc recipe would work easily enough here, just do $(find . -name ${S}/scripts/bpf_helpers_doc.py) and let it return nothing for those older kernels. then perhaps its better to callout each script by name and check if that file exists, so older kernels can ignore that Bruce +sed -i -e "s,#!.*python.*,#!${USRBINPATH}/env python3," ${s} done # unistd.h can be out of sync between libc-headers and the captured version in the perf source -- 2.24.1 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH v3] perf: Correct the substitution of python shebangs
On Fri, Feb 21, 2020 at 5:16 PM Khem Raj wrote: > > > > On 2/21/20 1:43 PM, Joshua Watt wrote: > > On Thu, Feb 13, 2020 at 9:58 PM wrote: > >> > >> From: He Zhe > >> > >> To make the native python3 is always used, > >> > >> - Use sed one-liner instead > >> - Add substitution for ${S}/scripts/bpf_helpers_doc.py to fix the > >>following warning. > >> > >> File "/usr/lib/python3.6/sysconfig.py", line 421, in _init_posix > >> _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0) > >> ModuleNotFoundError: No module named '_sysconfigdata' > >> > >> This issue is first reported by Joel Stanley > >> The sed one-liner is credited to Anuj Mittal > >> > >> Signed-off-by: He Zhe > >> --- > >> meta/recipes-kernel/perf/perf.bb | 7 ++- > >> 1 file changed, 2 insertions(+), 5 deletions(-) > >> > >> diff --git a/meta/recipes-kernel/perf/perf.bb > >> b/meta/recipes-kernel/perf/perf.bb > >> index 6d1b06693d..a6fb51d3db 100644 > >> --- a/meta/recipes-kernel/perf/perf.bb > >> +++ b/meta/recipes-kernel/perf/perf.bb > >> @@ -239,11 +239,8 @@ do_configure_prepend () { > >> fi > >> > >> # use /usr/bin/env instead of version specific python > >> -for s in `find ${S}/tools/perf/ -name '*.py'`; do > >> -sed -i 's,/usr/bin/python,/usr/bin/env python3,' "${s}" > >> -sed -i 's,/usr/bin/python2,/usr/bin/env python3,' "${s}" > >> -sed -i 's,/usr/bin/env python2,/usr/bin/env python3,' "${s}" > >> -sed -i 's,/usr/bin/python3,/usr/bin/env python3,' "${s}" > >> +for s in `find ${S}/tools/perf/ -name '*.py'` > >> ${S}/scripts/bpf_helpers_doc.py; do > > > > Do we have to hard code bpf_helpers_doc.py? It doesn't exist in older > > versions (e.g. 4.14) and causes do_configure to fail. > > Probably not, I also stumbled into this problem. Yah, We can either just do the entire directory (but during the review, we were talking about the fact that since we aren't using them, we don't actually know if they are py3 safe, so blindly converting the shebang isnt' a great idea either). But the solution we use in the devsrc recipe would work easily enough here, just do $(find . -name ${S}/scripts/bpf_helpers_doc.py) and let it return nothing for those older kernels. Bruce > > > > >> +sed -i -e "s,#!.*python.*,#!${USRBINPATH}/env python3," ${s} > >> done > >> > >> # unistd.h can be out of sync between libc-headers and the captured > >> version in the perf source > >> -- > >> 2.24.1 > >> > >> -- > >> ___ > >> Openembedded-core mailing list > >> Openembedded-core@lists.openembedded.org > >> http://lists.openembedded.org/mailman/listinfo/openembedded-core > -- > ___ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.openembedded.org/mailman/listinfo/openembedded-core -- - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH v3] perf: Correct the substitution of python shebangs
On 2/21/20 1:43 PM, Joshua Watt wrote: On Thu, Feb 13, 2020 at 9:58 PM wrote: From: He Zhe To make the native python3 is always used, - Use sed one-liner instead - Add substitution for ${S}/scripts/bpf_helpers_doc.py to fix the following warning. File "/usr/lib/python3.6/sysconfig.py", line 421, in _init_posix _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0) ModuleNotFoundError: No module named '_sysconfigdata' This issue is first reported by Joel Stanley The sed one-liner is credited to Anuj Mittal Signed-off-by: He Zhe --- meta/recipes-kernel/perf/perf.bb | 7 ++- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb index 6d1b06693d..a6fb51d3db 100644 --- a/meta/recipes-kernel/perf/perf.bb +++ b/meta/recipes-kernel/perf/perf.bb @@ -239,11 +239,8 @@ do_configure_prepend () { fi # use /usr/bin/env instead of version specific python -for s in `find ${S}/tools/perf/ -name '*.py'`; do -sed -i 's,/usr/bin/python,/usr/bin/env python3,' "${s}" -sed -i 's,/usr/bin/python2,/usr/bin/env python3,' "${s}" -sed -i 's,/usr/bin/env python2,/usr/bin/env python3,' "${s}" -sed -i 's,/usr/bin/python3,/usr/bin/env python3,' "${s}" +for s in `find ${S}/tools/perf/ -name '*.py'` ${S}/scripts/bpf_helpers_doc.py; do Do we have to hard code bpf_helpers_doc.py? It doesn't exist in older versions (e.g. 4.14) and causes do_configure to fail. Probably not, I also stumbled into this problem. +sed -i -e "s,#!.*python.*,#!${USRBINPATH}/env python3," ${s} done # unistd.h can be out of sync between libc-headers and the captured version in the perf source -- 2.24.1 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH v3] perf: Correct the substitution of python shebangs
On Thu, Feb 13, 2020 at 9:58 PM wrote: > > From: He Zhe > > To make the native python3 is always used, > > - Use sed one-liner instead > - Add substitution for ${S}/scripts/bpf_helpers_doc.py to fix the > following warning. > > File "/usr/lib/python3.6/sysconfig.py", line 421, in _init_posix > _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0) > ModuleNotFoundError: No module named '_sysconfigdata' > > This issue is first reported by Joel Stanley > The sed one-liner is credited to Anuj Mittal > > Signed-off-by: He Zhe > --- > meta/recipes-kernel/perf/perf.bb | 7 ++- > 1 file changed, 2 insertions(+), 5 deletions(-) > > diff --git a/meta/recipes-kernel/perf/perf.bb > b/meta/recipes-kernel/perf/perf.bb > index 6d1b06693d..a6fb51d3db 100644 > --- a/meta/recipes-kernel/perf/perf.bb > +++ b/meta/recipes-kernel/perf/perf.bb > @@ -239,11 +239,8 @@ do_configure_prepend () { > fi > > # use /usr/bin/env instead of version specific python > -for s in `find ${S}/tools/perf/ -name '*.py'`; do > -sed -i 's,/usr/bin/python,/usr/bin/env python3,' "${s}" > -sed -i 's,/usr/bin/python2,/usr/bin/env python3,' "${s}" > -sed -i 's,/usr/bin/env python2,/usr/bin/env python3,' "${s}" > -sed -i 's,/usr/bin/python3,/usr/bin/env python3,' "${s}" > +for s in `find ${S}/tools/perf/ -name '*.py'` > ${S}/scripts/bpf_helpers_doc.py; do Do we have to hard code bpf_helpers_doc.py? It doesn't exist in older versions (e.g. 4.14) and causes do_configure to fail. > +sed -i -e "s,#!.*python.*,#!${USRBINPATH}/env python3," ${s} > done > > # unistd.h can be out of sync between libc-headers and the captured > version in the perf source > -- > 2.24.1 > > -- > ___ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.openembedded.org/mailman/listinfo/openembedded-core -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH v3] perf: Correct the substitution of python shebangs
On Fri, 2020-02-14 at 08:18 -0500, Bruce Ashfield wrote: > On Thu, Feb 13, 2020 at 10:58 PM wrote: > > From: He Zhe > > > > To make the native python3 is always used, > > > > - Use sed one-liner instead > > - Add substitution for ${S}/scripts/bpf_helpers_doc.py to fix the > > following warning. > > > > File "/usr/lib/python3.6/sysconfig.py", line 421, in _init_posix > > _temp = __import__(name, globals(), locals(), ['build_time_vars'], > > 0) > > ModuleNotFoundError: No module named '_sysconfigdata' > > > > This issue is first reported by Joel Stanley > > The sed one-liner is credited to Anuj Mittal > > > > Thanks! > > This looks good to me. I'm not sure how but with this applied we see breaking kernel module tests: https://autobuilder.yoctoproject.org/typhoon/#/builders/60/builds/1555 https://autobuilder.yoctoproject.org/typhoon/#/builders/102/builds/275 https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/1559 (mips/sysvinit, mips/systemd and mips64) I'll remove this patch from -next and retest see if I can confirm it is this change somehow. Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH v3] perf: Correct the substitution of python shebangs
On Thu, Feb 13, 2020 at 10:58 PM wrote: > > From: He Zhe > > To make the native python3 is always used, > > - Use sed one-liner instead > - Add substitution for ${S}/scripts/bpf_helpers_doc.py to fix the > following warning. > > File "/usr/lib/python3.6/sysconfig.py", line 421, in _init_posix > _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0) > ModuleNotFoundError: No module named '_sysconfigdata' > > This issue is first reported by Joel Stanley > The sed one-liner is credited to Anuj Mittal Thanks! This looks good to me. Cheers, Bruce > > Signed-off-by: He Zhe > --- > meta/recipes-kernel/perf/perf.bb | 7 ++- > 1 file changed, 2 insertions(+), 5 deletions(-) > > diff --git a/meta/recipes-kernel/perf/perf.bb > b/meta/recipes-kernel/perf/perf.bb > index 6d1b06693d..a6fb51d3db 100644 > --- a/meta/recipes-kernel/perf/perf.bb > +++ b/meta/recipes-kernel/perf/perf.bb > @@ -239,11 +239,8 @@ do_configure_prepend () { > fi > > # use /usr/bin/env instead of version specific python > -for s in `find ${S}/tools/perf/ -name '*.py'`; do > -sed -i 's,/usr/bin/python,/usr/bin/env python3,' "${s}" > -sed -i 's,/usr/bin/python2,/usr/bin/env python3,' "${s}" > -sed -i 's,/usr/bin/env python2,/usr/bin/env python3,' "${s}" > -sed -i 's,/usr/bin/python3,/usr/bin/env python3,' "${s}" > +for s in `find ${S}/tools/perf/ -name '*.py'` > ${S}/scripts/bpf_helpers_doc.py; do > +sed -i -e "s,#!.*python.*,#!${USRBINPATH}/env python3," ${s} > done > > # unistd.h can be out of sync between libc-headers and the captured > version in the perf source > -- > 2.24.1 > > -- > ___ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.openembedded.org/mailman/listinfo/openembedded-core -- - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH v3] perf: Correct the substitution of python shebangs
TA From: openembedded-core-boun...@lists.openembedded.org [openembedded-core-boun...@lists.openembedded.org] on behalf of zhe...@windriver.com [zhe...@windriver.com] Sent: Friday, February 14, 2020 11:57 AM To: openembedded-core@lists.openembedded.org Subject: [OE-core] [PATCH v3] perf: Correct the substitution of python shebangs From: He Zhe To make the native python3 is always used, - Use sed one-liner instead - Add substitution for ${S}/scripts/bpf_helpers_doc.py to fix the following warning. File "/usr/lib/python3.6/sysconfig.py", line 421, in _init_posix _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0) ModuleNotFoundError: No module named '_sysconfigdata' This issue is first reported by Joel Stanley The sed one-liner is credited to Anuj Mittal Signed-off-by: He Zhe --- meta/recipes-kernel/perf/perf.bb | 7 ++- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb index 6d1b06693d..a6fb51d3db 100644 --- a/meta/recipes-kernel/perf/perf.bb +++ b/meta/recipes-kernel/perf/perf.bb @@ -239,11 +239,8 @@ do_configure_prepend () { fi # use /usr/bin/env instead of version specific python -for s in `find ${S}/tools/perf/ -name '*.py'`; do -sed -i 's,/usr/bin/python,/usr/bin/env python3,' "${s}" -sed -i 's,/usr/bin/python2,/usr/bin/env python3,' "${s}" -sed -i 's,/usr/bin/env python2,/usr/bin/env python3,' "${s}" -sed -i 's,/usr/bin/python3,/usr/bin/env python3,' "${s}" +for s in `find ${S}/tools/perf/ -name '*.py'` ${S}/scripts/bpf_helpers_doc.py; do +sed -i -e "s,#!.*python.*,#!${USRBINPATH}/env python3," ${s} done # unistd.h can be out of sync between libc-headers and the captured version in the perf source -- 2.24.1 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v3] perf: Correct the substitution of python shebangs
From: He Zhe To make the native python3 is always used, - Use sed one-liner instead - Add substitution for ${S}/scripts/bpf_helpers_doc.py to fix the following warning. File "/usr/lib/python3.6/sysconfig.py", line 421, in _init_posix _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0) ModuleNotFoundError: No module named '_sysconfigdata' This issue is first reported by Joel Stanley The sed one-liner is credited to Anuj Mittal Signed-off-by: He Zhe --- meta/recipes-kernel/perf/perf.bb | 7 ++- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb index 6d1b06693d..a6fb51d3db 100644 --- a/meta/recipes-kernel/perf/perf.bb +++ b/meta/recipes-kernel/perf/perf.bb @@ -239,11 +239,8 @@ do_configure_prepend () { fi # use /usr/bin/env instead of version specific python -for s in `find ${S}/tools/perf/ -name '*.py'`; do -sed -i 's,/usr/bin/python,/usr/bin/env python3,' "${s}" -sed -i 's,/usr/bin/python2,/usr/bin/env python3,' "${s}" -sed -i 's,/usr/bin/env python2,/usr/bin/env python3,' "${s}" -sed -i 's,/usr/bin/python3,/usr/bin/env python3,' "${s}" +for s in `find ${S}/tools/perf/ -name '*.py'` ${S}/scripts/bpf_helpers_doc.py; do +sed -i -e "s,#!.*python.*,#!${USRBINPATH}/env python3," ${s} done # unistd.h can be out of sync between libc-headers and the captured version in the perf source -- 2.24.1 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core