Re: [OE-core] [PATCH 3/3] glibc/glibc-locale: Fix do_stash_locale to work with usrmerge and multilibs
On 7/6/19 2:31 PM, Adrian Bunk wrote: On Thu, Jul 04, 2019 at 06:57:50AM -0500, Jason Wessel wrote: On 7/4/19 5:57 AM, Adrian Bunk wrote: On Thu, Jul 04, 2019 at 11:13:18AM +0200, Martin Jansa wrote: I don't see how this change (or the previous glibc-locale one) would cause that, but since the oe-core upgrade yesterday I'm seeing following glibc-locale.do_package failure: http://errors.yoctoproject.org/Errors/Details/250557/ DEBUG: Executing shell function do_prep_locale_tree tar: i18n: Cannot stat: No such file or directory tar: Exiting with failure status due to previous errors gzip: TOPDIR/tmp-glibc/work/core2-64-oe-linux/glibc-locale/2.29-r0/locale-tree//usr/share/i18n/charmaps/*gz.gz: No such file or directory WARNING: TOPDIR/tmp-glibc/work/core2-64-oe-linux/glibc-locale/2.29-r0/temp/run.do_prep_locale_tree.23303:1 exit 1 from 'gunzip $i' anyone else seeing this? Yes. I don't know that it was specifically this change that causes the problem but rather any change to this function. If that is not the case can either of you send me your local.conf file, and I'll try to reproduce it. If the following doesn't work around the problem, there is clearly another corner case that is not well understood. bitbake -c cleansstate glibc glibc-locale ... bitbake -c cleansstate glibc glibc-locale nativesdk-glibc nativesdk-glibc-locale This seems to workaround the problem in the cases I have seen so far, so what you said about the cause looks plausible to me. I chatted with Richard to figure out how to fix the original implementation, so this will not continue to happen. I am no fan of incremental or sstate based builds failing. The pending fix is below. https://patchwork.openembedded.org/patch/162886/ Cheers, Jason. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 3/3] glibc/glibc-locale: Fix do_stash_locale to work with usrmerge and multilibs
On Thu, Jul 04, 2019 at 06:57:50AM -0500, Jason Wessel wrote: > On 7/4/19 5:57 AM, Adrian Bunk wrote: > > On Thu, Jul 04, 2019 at 11:13:18AM +0200, Martin Jansa wrote: > > > I don't see how this change (or the previous glibc-locale one) would cause > > > that, but since the oe-core upgrade yesterday I'm seeing following > > > glibc-locale.do_package failure: > > > http://errors.yoctoproject.org/Errors/Details/250557/ > > > > > > DEBUG: Executing shell function do_prep_locale_tree > > > tar: i18n: Cannot stat: No such file or directory > > > tar: Exiting with failure status due to previous errors > > > gzip: > > > TOPDIR/tmp-glibc/work/core2-64-oe-linux/glibc-locale/2.29-r0/locale-tree//usr/share/i18n/charmaps/*gz.gz: > > > No such file or directory > > > WARNING: > > > TOPDIR/tmp-glibc/work/core2-64-oe-linux/glibc-locale/2.29-r0/temp/run.do_prep_locale_tree.23303:1 > > > exit 1 from 'gunzip $i' > > > > > > anyone else seeing this? > > > > Yes. > > > I don't know that it was specifically this change that causes the problem but > rather any change to this function. If that is not the case can either of > you send me your local.conf file, and I'll try to reproduce it. If the > following doesn't work around the problem, there is clearly another corner > case that is not well understood. > > bitbake -c cleansstate glibc glibc-locale >... bitbake -c cleansstate glibc glibc-locale nativesdk-glibc nativesdk-glibc-locale This seems to workaround the problem in the cases I have seen so far, so what you said about the cause looks plausible to me. > Cheers, > > Jason. cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 3/3] glibc/glibc-locale: Fix do_stash_locale to work with usrmerge and multilibs
On 7/4/19 5:57 AM, Adrian Bunk wrote: On Thu, Jul 04, 2019 at 11:13:18AM +0200, Martin Jansa wrote: I don't see how this change (or the previous glibc-locale one) would cause that, but since the oe-core upgrade yesterday I'm seeing following glibc-locale.do_package failure: http://errors.yoctoproject.org/Errors/Details/250557/ DEBUG: Executing shell function do_prep_locale_tree tar: i18n: Cannot stat: No such file or directory tar: Exiting with failure status due to previous errors gzip: TOPDIR/tmp-glibc/work/core2-64-oe-linux/glibc-locale/2.29-r0/locale-tree//usr/share/i18n/charmaps/*gz.gz: No such file or directory WARNING: TOPDIR/tmp-glibc/work/core2-64-oe-linux/glibc-locale/2.29-r0/temp/run.do_prep_locale_tree.23303:1 exit 1 from 'gunzip $i' anyone else seeing this? Yes. I don't know that it was specifically this change that causes the problem but rather any change to this function. If that is not the case can either of you send me your local.conf file, and I'll try to reproduce it. If the following doesn't work around the problem, there is clearly another corner case that is not well understood. bitbake -c cleansstate glibc glibc-locale I believe that any change to the .inc file which has locale stash function will cause the problem due to the fact it performs and "mv" operation to move the files. When it runs a second time, there are no more files to move and you will not have them for use later on. When I was experimenting earlier, I tried running the glibc temp/run.do_install script to put the files back, but then hit a problem with localdef being missing, because it comes from the temp/run.do_compile functions. My conclusion thus far has been is that the ties between the glibc and glibc-locale are very subtle and break easily. This is a separate problem than what I was attempting to fix, where the usrmerge + multilibs didn't even get the correct files in the first place. Cheers, Jason. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 3/3] glibc/glibc-locale: Fix do_stash_locale to work with usrmerge and multilibs
On Thu, Jul 04, 2019 at 11:13:18AM +0200, Martin Jansa wrote: > I don't see how this change (or the previous glibc-locale one) would cause > that, but since the oe-core upgrade yesterday I'm seeing following > glibc-locale.do_package failure: > http://errors.yoctoproject.org/Errors/Details/250557/ > > DEBUG: Executing shell function do_prep_locale_tree > tar: i18n: Cannot stat: No such file or directory > tar: Exiting with failure status due to previous errors > gzip: > TOPDIR/tmp-glibc/work/core2-64-oe-linux/glibc-locale/2.29-r0/locale-tree//usr/share/i18n/charmaps/*gz.gz: > No such file or directory > WARNING: > TOPDIR/tmp-glibc/work/core2-64-oe-linux/glibc-locale/2.29-r0/temp/run.do_prep_locale_tree.23303:1 > exit 1 from 'gunzip $i' > > anyone else seeing this? Yes. cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 3/3] glibc/glibc-locale: Fix do_stash_locale to work with usrmerge and multilibs
I don't see how this change (or the previous glibc-locale one) would cause
that, but since the oe-core upgrade yesterday I'm seeing following
glibc-locale.do_package failure:
http://errors.yoctoproject.org/Errors/Details/250557/
DEBUG: Executing shell function do_prep_locale_tree
tar: i18n: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
gzip:
TOPDIR/tmp-glibc/work/core2-64-oe-linux/glibc-locale/2.29-r0/locale-tree//usr/share/i18n/charmaps/*gz.gz:
No such file or directory
WARNING:
TOPDIR/tmp-glibc/work/core2-64-oe-linux/glibc-locale/2.29-r0/temp/run.do_prep_locale_tree.23303:1
exit 1 from 'gunzip $i'
anyone else seeing this?
On Tue, Jul 2, 2019 at 10:16 PM Jason Wessel
wrote:
> The do_stash_locale was not working consistently across the 4 build
> configurations and the multilib, usrmerge configuration would fail
> entirely with the obscure message:
>
> | DEBUG: Executing shell function do_prep_locale_tree
> | tar: i18n: Cannot stat: No such file or directory
> | tar: Exiting with failure status due to previous errors
> | gzip:
> /poky/build/tmp/work/core2-64-poky-linux/glibc-locale/2.29-r0/locale-tree//usr/share/i18n/charmaps/*gz.gz:
> No such file or directory
> | WARNING:
> /poky/build/tmp/work/core2-64-poky-linux/glibc-locale/2.29-r0/temp/run.do_prep_locale_tree.124690:1
> exit 1 from 'gunzip $i'
>
> Here is the 4 build configurations without the patch applied:
>
> A) x86-64 no multilibs, no usrmerge
> find ./tmp/work/*/glibc/2.29-r0/stashed-locale -type f |grep -v
> nscd.service |wc -l
> 909
> B) x86-64 no multilibs, usrmerge
> find ./tmp/work/*/glibc/2.29-r0/stashed-locale -type f |grep -v
> nscd.service |wc -l
> 909
> C) x86-64 multilibs, no usrmerge
> find ./tmp/work/*/glibc/2.29-r0/stashed-locale -type f |grep -v
> nscd.service |wc -l
> 885
> D) x86-64 multilibs, usrmerge
> find ./tmp/work/*/glibc/2.29-r0/stashed-locale -type f |grep -v
> nscd.service |wc -l
> 864
>
> The issue here is that all the moves should be processed first, then a
> copy should be made of the lib directories, but only in the case they
> are different when using the usrmerge feature. Even though the build
> worked for the multilib configuration without usrmerge, the content
> was not the same.
>
> After applying the patch the same number of files are in all the
> configurations. The list of files was also diffed, after normalizing
> the directory names to ensure all the correct files were copied.
>
> Ultimately there are probably additional files that should be pruned
> from what is copied to the stated_locale, but the purpose of this
> patch is make it 100% consistent between the build types and fix the
> builds.
>
> Signed-off-by: Jason Wessel
> ---
> meta/recipes-core/glibc/glibc-package.inc | 19 +--
> 1 file changed, 13 insertions(+), 6 deletions(-)
>
> diff --git a/meta/recipes-core/glibc/glibc-package.inc
> b/meta/recipes-core/glibc/glibc-package.inc
> index a1d79b3075..ff17a193c3 100644
> --- a/meta/recipes-core/glibc/glibc-package.inc
> +++ b/meta/recipes-core/glibc/glibc-package.inc
> @@ -162,21 +162,28 @@ bashscripts = "mtrace sotruss xtrace"
> do_stash_locale () {
> dest=${LOCALESTASH}
> install -d ${dest}${base_libdir} ${dest}${bindir} ${dest}${libdir}
> ${dest}${datadir}
> - if [ "${base_libdir}" != "${libdir}" ]; then
> - cp -fpPR ${D}${base_libdir}/* ${dest}${base_libdir}
> - fi
> + # Hide away the locale data from the deployment
> if [ -e ${D}${bindir}/localedef ]; then
> mv -f ${D}${bindir}/localedef ${dest}${bindir}
> fi
> if [ -e ${D}${libdir}/gconv ]; then
> mv -f ${D}${libdir}/gconv ${dest}${libdir}
> fi
> - if [ -e ${D}${exec_prefix}/lib ]; then
> - cp -fpPR ${D}${exec_prefix}/lib ${dest}${exec_prefix}
> - fi
> if [ -e ${D}${datadir}/i18n ]; then
> mv ${D}${datadir}/i18n ${dest}${datadir}
> fi
> +
> + # Make a copy of all the libraries into the locale stash
> + cp -fpPR ${D}${libdir}/* ${dest}${libdir}
> + if [ "${base_libdir}" != "${libdir}" ]; then
> + cp -fpPR ${D}${base_libdir}/* ${dest}${base_libdir}
> + fi
> + if [ -e ${D}${exec_prefix}/lib ]; then
> + if [ ${exec_prefix}/lib != ${base_libdir} ] && [
> ${exec_prefix}/lib != ${libdir} ]; then
> + cp -fpPR ${D}${exec_prefix}/lib
> ${dest}${exec_prefix}
> + fi
> + fi
> +
> cp -fpPR ${D}${datadir}/* ${dest}${datadir}
> rm -rf ${D}${datadir}/locale/
> cp -fpPR ${WORKDIR}/SUPPORTED ${dest}
> --
> 2.21.0
>
> --
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
--
___
Openembedded-core mailing list
[OE-core] [PATCH 3/3] glibc/glibc-locale: Fix do_stash_locale to work with usrmerge and multilibs
The do_stash_locale was not working consistently across the 4 build
configurations and the multilib, usrmerge configuration would fail
entirely with the obscure message:
| DEBUG: Executing shell function do_prep_locale_tree
| tar: i18n: Cannot stat: No such file or directory
| tar: Exiting with failure status due to previous errors
| gzip:
/poky/build/tmp/work/core2-64-poky-linux/glibc-locale/2.29-r0/locale-tree//usr/share/i18n/charmaps/*gz.gz:
No such file or directory
| WARNING:
/poky/build/tmp/work/core2-64-poky-linux/glibc-locale/2.29-r0/temp/run.do_prep_locale_tree.124690:1
exit 1 from 'gunzip $i'
Here is the 4 build configurations without the patch applied:
A) x86-64 no multilibs, no usrmerge
find ./tmp/work/*/glibc/2.29-r0/stashed-locale -type f |grep -v nscd.service
|wc -l
909
B) x86-64 no multilibs, usrmerge
find ./tmp/work/*/glibc/2.29-r0/stashed-locale -type f |grep -v nscd.service
|wc -l
909
C) x86-64 multilibs, no usrmerge
find ./tmp/work/*/glibc/2.29-r0/stashed-locale -type f |grep -v nscd.service
|wc -l
885
D) x86-64 multilibs, usrmerge
find ./tmp/work/*/glibc/2.29-r0/stashed-locale -type f |grep -v nscd.service
|wc -l
864
The issue here is that all the moves should be processed first, then a
copy should be made of the lib directories, but only in the case they
are different when using the usrmerge feature. Even though the build
worked for the multilib configuration without usrmerge, the content
was not the same.
After applying the patch the same number of files are in all the
configurations. The list of files was also diffed, after normalizing
the directory names to ensure all the correct files were copied.
Ultimately there are probably additional files that should be pruned
from what is copied to the stated_locale, but the purpose of this
patch is make it 100% consistent between the build types and fix the
builds.
Signed-off-by: Jason Wessel
---
meta/recipes-core/glibc/glibc-package.inc | 19 +--
1 file changed, 13 insertions(+), 6 deletions(-)
diff --git a/meta/recipes-core/glibc/glibc-package.inc
b/meta/recipes-core/glibc/glibc-package.inc
index a1d79b3075..ff17a193c3 100644
--- a/meta/recipes-core/glibc/glibc-package.inc
+++ b/meta/recipes-core/glibc/glibc-package.inc
@@ -162,21 +162,28 @@ bashscripts = "mtrace sotruss xtrace"
do_stash_locale () {
dest=${LOCALESTASH}
install -d ${dest}${base_libdir} ${dest}${bindir} ${dest}${libdir}
${dest}${datadir}
- if [ "${base_libdir}" != "${libdir}" ]; then
- cp -fpPR ${D}${base_libdir}/* ${dest}${base_libdir}
- fi
+ # Hide away the locale data from the deployment
if [ -e ${D}${bindir}/localedef ]; then
mv -f ${D}${bindir}/localedef ${dest}${bindir}
fi
if [ -e ${D}${libdir}/gconv ]; then
mv -f ${D}${libdir}/gconv ${dest}${libdir}
fi
- if [ -e ${D}${exec_prefix}/lib ]; then
- cp -fpPR ${D}${exec_prefix}/lib ${dest}${exec_prefix}
- fi
if [ -e ${D}${datadir}/i18n ]; then
mv ${D}${datadir}/i18n ${dest}${datadir}
fi
+
+ # Make a copy of all the libraries into the locale stash
+ cp -fpPR ${D}${libdir}/* ${dest}${libdir}
+ if [ "${base_libdir}" != "${libdir}" ]; then
+ cp -fpPR ${D}${base_libdir}/* ${dest}${base_libdir}
+ fi
+ if [ -e ${D}${exec_prefix}/lib ]; then
+ if [ ${exec_prefix}/lib != ${base_libdir} ] && [
${exec_prefix}/lib != ${libdir} ]; then
+ cp -fpPR ${D}${exec_prefix}/lib ${dest}${exec_prefix}
+ fi
+ fi
+
cp -fpPR ${D}${datadir}/* ${dest}${datadir}
rm -rf ${D}${datadir}/locale/
cp -fpPR ${WORKDIR}/SUPPORTED ${dest}
--
2.21.0
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
