Re: [OE-core] [PATCH 2/2] image-live.bbclass: remove MLPREFIX from syslinux
On 01/04/2018 07:13 PM, Robert Yang wrote: Hi RP, Thanks for the reply. On 01/03/2018 10:19 PM, Richard Purdie wrote: On Wed, 2018-01-03 at 21:56 +0800, Robert Yang wrote: Also, there are some things which never make sense as a multilib, the kernel is one example and I'm starting to wonder if syslinux would be another. In the kernel (and kernel module) case we'd provide all the libX variants from the same recipe, we may want to do that for syslinux. I think that syslinux is different from kernel, 64bit kernel can provide 32bit kernel via kernel config, but syslinux can't (except these non arch files). Think about this differently. The system can ever only boot one single kernel. The image can have several different multilibs running at once in different userspace processes but there can only ever be one running kernel. How that kernel is configured is obviously important but the key thing is there can only be one running. The bootloader is similar in that you likely only ever want one and I suspect syslinux is similar. Makes sense, I will work on it. Also, don't confuse this with multi-boot or multi partition systems where there could be a "main" and a "backup" kernel. In those cases there would only ever be one running at once. It may be we can't avoid the multiple compiler issue and the current codebase may not do so, I think currently we can avoid multiple glibc though. I'm not sure about a problem on this, should lib32-image can run 64bit programs or not ? If yes, then kernel should be 64bit, and we can't avoid building 64bit compilers. And I'm leaning to yes since we call it multilib, the pure 32bit image which can't run 64bit programs can't be called multilib. That is a configuration issue for the multilib you select. In general, yes you'd want a 64 bit kernel. I did a rough search on why glibc is built, when bitbake lib32-image, it is because: 64 bit kernel -> gcc-cross-x86_64 -> virtual/${TARGET_PREFIX}libc- for-gcc, and the TARGET_PREFIX is x86_64 when building gcc-cross-x86_64, While the virtual/x86_64-poky-linux-libc-for-gcc is provided by glibc. I tried to remove the depends, both "bitbake gcc-cross-x86_64" and "bitbake linux-yocto" can be built, but other recipes such as quilt would be failed: I find it interesting that gcc-cross-x86_64 would build without glibc. If that really is the case we may be able to speed up our compiler bootstrap so that could be worth investigating further. Its not surprising that libgcc won't build without glibc though. Perhaps gcc-cross works since we split out the build of libgcc? collect2: error: ld returned 1 exit status Makefile:978: recipe for target 'libgcc_s.so' failed And move virtual/${TARGET_PREFIX}libc-for-gcc to BASE_DEFAULT_DEPS can't make it work either, so it seems that we can't avoid building glibc. No, I'd forgotten that gcc-cross depends on glibc. Based on the above there may still be some optimisation we can make here. In the back of my mind, I'm concious that one x86 cross compiler should be able to work for 32 and 64 bit too, its just the compiler options and libgcc etc. would need to be generated for both... Build one toolchain for both x86 32 and 64 is possible, I'd like to work on it if it is worth. Add Mark in the loop, here is his comments: """ We had a single toolchain built outside of the Yocto Project environment with a custom spec file. The only way to do this inside of the Yocto Project would be to generate a custom spec file for the cross-toolchain based on a specific configuration. In the past when this was explored, it was deemed to not be a 'generic' solution. So using the stock GCC spec file(s) and/or multiple toolchains was the better generic approach. """ So I'm leaning to keep the current status, multiple toolchains for multilib. // Robert // Robert Regardless, I do think this needs a little more thought, we also need better multiple test cases as we're not catching issues like this. think this needs to be revisited along with your outstanding multilib patch series which I haven't found the time to review yet (sorry). That's all right, I'm very glad to make mutilib work well, including adding test cases for them. It is nearly broken after changed from smart + rpm5 to dnf + rpm4, those patches fixed the problem. Agreed, its important and on my list of things to review, I've just had to focus on getting build testing working properly and now I can try and clear some of the patch backlog. Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/2] image-live.bbclass: remove MLPREFIX from syslinux
Hi RP, On 01/04/2018 07:13 PM, Robert Yang wrote: Hi RP, Thanks for the reply. On 01/03/2018 10:19 PM, Richard Purdie wrote: On Wed, 2018-01-03 at 21:56 +0800, Robert Yang wrote: Also, there are some things which never make sense as a multilib, the kernel is one example and I'm starting to wonder if syslinux would be another. In the kernel (and kernel module) case we'd provide all the libX variants from the same recipe, we may want to do that for syslinux. I think that syslinux is different from kernel, 64bit kernel can provide 32bit kernel via kernel config, but syslinux can't (except these non arch files). Think about this differently. The system can ever only boot one single kernel. The image can have several different multilibs running at once in different userspace processes but there can only ever be one running kernel. How that kernel is configured is obviously important but the key thing is there can only be one running. The bootloader is similar in that you likely only ever want one and I suspect syslinux is similar. Makes sense, I will work on it. I tried syslinux, we can make syslinux provide lib32-syslinux, if we treat it as a bootloader, it is OK since we use syslinux-native to install the bootloader, but if we treat it as an application, e.g.: IMAGE_INSTALL += "syslinux" And then: bitbake lib32-core-image-minimal I think that we need a 32bit syslinux to be installed rather than 64bit, so I'm afraid that we can't let syslinux provide lib32-syslinux. // Robert Also, don't confuse this with multi-boot or multi partition systems where there could be a "main" and a "backup" kernel. In those cases there would only ever be one running at once. It may be we can't avoid the multiple compiler issue and the current codebase may not do so, I think currently we can avoid multiple glibc though. I'm not sure about a problem on this, should lib32-image can run 64bit programs or not ? If yes, then kernel should be 64bit, and we can't avoid building 64bit compilers. And I'm leaning to yes since we call it multilib, the pure 32bit image which can't run 64bit programs can't be called multilib. That is a configuration issue for the multilib you select. In general, yes you'd want a 64 bit kernel. I did a rough search on why glibc is built, when bitbake lib32-image, it is because: 64 bit kernel -> gcc-cross-x86_64 -> virtual/${TARGET_PREFIX}libc- for-gcc, and the TARGET_PREFIX is x86_64 when building gcc-cross-x86_64, While the virtual/x86_64-poky-linux-libc-for-gcc is provided by glibc. I tried to remove the depends, both "bitbake gcc-cross-x86_64" and "bitbake linux-yocto" can be built, but other recipes such as quilt would be failed: I find it interesting that gcc-cross-x86_64 would build without glibc. If that really is the case we may be able to speed up our compiler bootstrap so that could be worth investigating further. Its not surprising that libgcc won't build without glibc though. Perhaps gcc-cross works since we split out the build of libgcc? collect2: error: ld returned 1 exit status Makefile:978: recipe for target 'libgcc_s.so' failed And move virtual/${TARGET_PREFIX}libc-for-gcc to BASE_DEFAULT_DEPS can't make it work either, so it seems that we can't avoid building glibc. No, I'd forgotten that gcc-cross depends on glibc. Based on the above there may still be some optimisation we can make here. In the back of my mind, I'm concious that one x86 cross compiler should be able to work for 32 and 64 bit too, its just the compiler options and libgcc etc. would need to be generated for both... Build one toolchain for both x86 32 and 64 is possible, I'd like to work on it if it is worth. // Robert Regardless, I do think this needs a little more thought, we also need better multiple test cases as we're not catching issues like this. think this needs to be revisited along with your outstanding multilib patch series which I haven't found the time to review yet (sorry). That's all right, I'm very glad to make mutilib work well, including adding test cases for them. It is nearly broken after changed from smart + rpm5 to dnf + rpm4, those patches fixed the problem. Agreed, its important and on my list of things to review, I've just had to focus on getting build testing working properly and now I can try and clear some of the patch backlog. Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/2] image-live.bbclass: remove MLPREFIX from syslinux
Hi RP, Thanks for the reply. On 01/03/2018 10:19 PM, Richard Purdie wrote: On Wed, 2018-01-03 at 21:56 +0800, Robert Yang wrote: Also, there are some things which never make sense as a multilib, the kernel is one example and I'm starting to wonder if syslinux would be another. In the kernel (and kernel module) case we'd provide all the libX variants from the same recipe, we may want to do that for syslinux. I think that syslinux is different from kernel, 64bit kernel can provide 32bit kernel via kernel config, but syslinux can't (except these non arch files). Think about this differently. The system can ever only boot one single kernel. The image can have several different multilibs running at once in different userspace processes but there can only ever be one running kernel. How that kernel is configured is obviously important but the key thing is there can only be one running. The bootloader is similar in that you likely only ever want one and I suspect syslinux is similar. Makes sense, I will work on it. Also, don't confuse this with multi-boot or multi partition systems where there could be a "main" and a "backup" kernel. In those cases there would only ever be one running at once. It may be we can't avoid the multiple compiler issue and the current codebase may not do so, I think currently we can avoid multiple glibc though. I'm not sure about a problem on this, should lib32-image can run 64bit programs or not ? If yes, then kernel should be 64bit, and we can't avoid building 64bit compilers. And I'm leaning to yes since we call it multilib, the pure 32bit image which can't run 64bit programs can't be called multilib. That is a configuration issue for the multilib you select. In general, yes you'd want a 64 bit kernel. I did a rough search on why glibc is built, when bitbake lib32-image, it is because: 64 bit kernel -> gcc-cross-x86_64 -> virtual/${TARGET_PREFIX}libc- for-gcc, and the TARGET_PREFIX is x86_64 when building gcc-cross-x86_64, While the virtual/x86_64-poky-linux-libc-for-gcc is provided by glibc. I tried to remove the depends, both "bitbake gcc-cross-x86_64" and "bitbake linux-yocto" can be built, but other recipes such as quilt would be failed: I find it interesting that gcc-cross-x86_64 would build without glibc. If that really is the case we may be able to speed up our compiler bootstrap so that could be worth investigating further. Its not surprising that libgcc won't build without glibc though. Perhaps gcc-cross works since we split out the build of libgcc? collect2: error: ld returned 1 exit status Makefile:978: recipe for target 'libgcc_s.so' failed And move virtual/${TARGET_PREFIX}libc-for-gcc to BASE_DEFAULT_DEPS can't make it work either, so it seems that we can't avoid building glibc. No, I'd forgotten that gcc-cross depends on glibc. Based on the above there may still be some optimisation we can make here. In the back of my mind, I'm concious that one x86 cross compiler should be able to work for 32 and 64 bit too, its just the compiler options and libgcc etc. would need to be generated for both... Build one toolchain for both x86 32 and 64 is possible, I'd like to work on it if it is worth. // Robert Regardless, I do think this needs a little more thought, we also need better multiple test cases as we're not catching issues like this. think this needs to be revisited along with your outstanding multilib patch series which I haven't found the time to review yet (sorry). That's all right, I'm very glad to make mutilib work well, including adding test cases for them. It is nearly broken after changed from smart + rpm5 to dnf + rpm4, those patches fixed the problem. Agreed, its important and on my list of things to review, I've just had to focus on getting build testing working properly and now I can try and clear some of the patch backlog. Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/2] image-live.bbclass: remove MLPREFIX from syslinux
On Wed, 2018-01-03 at 21:56 +0800, Robert Yang wrote: > > Also, there are some things which never make sense as a multilib, > > the > > kernel is one example and I'm starting to wonder if syslinux would > > be > > another. In the kernel (and kernel module) case we'd provide all > > the > > libX variants from the same recipe, we may want to do that for > > syslinux. > I think that syslinux is different from kernel, 64bit kernel can > provide 32bit kernel via kernel config, but syslinux can't (except > these non arch files). Think about this differently. The system can ever only boot one single kernel. The image can have several different multilibs running at once in different userspace processes but there can only ever be one running kernel. How that kernel is configured is obviously important but the key thing is there can only be one running. The bootloader is similar in that you likely only ever want one and I suspect syslinux is similar. Also, don't confuse this with multi-boot or multi partition systems where there could be a "main" and a "backup" kernel. In those cases there would only ever be one running at once. > > It may be we can't avoid the multiple compiler issue and the > > current > > codebase may not do so, I think currently we can avoid multiple > > glibc > > though. > I'm not sure about a problem on this, should lib32-image can run > 64bit programs or not ? If yes, then kernel should be 64bit, and we > can't avoid building 64bit compilers. And I'm leaning to yes since we > call it multilib, the pure 32bit image which can't run 64bit programs > can't be called multilib. That is a configuration issue for the multilib you select. In general, yes you'd want a 64 bit kernel. > I did a rough search on why glibc is built, when bitbake lib32-image, > it is because: > > 64 bit kernel -> gcc-cross-x86_64 -> virtual/${TARGET_PREFIX}libc- > for-gcc, > and the TARGET_PREFIX is x86_64 when building gcc-cross-x86_64, > While the virtual/x86_64-poky-linux-libc-for-gcc is provided by > glibc. > > I tried to remove the depends, both "bitbake gcc-cross-x86_64" and > "bitbake linux-yocto" can be built, but other recipes such as quilt > would be failed: I find it interesting that gcc-cross-x86_64 would build without glibc. If that really is the case we may be able to speed up our compiler bootstrap so that could be worth investigating further. Its not surprising that libgcc won't build without glibc though. Perhaps gcc-cross works since we split out the build of libgcc? > > collect2: error: ld returned 1 exit status > > Makefile:978: recipe for target 'libgcc_s.so' failed > And move virtual/${TARGET_PREFIX}libc-for-gcc to BASE_DEFAULT_DEPS > can't make it work either, so it seems that we can't avoid building > glibc. No, I'd forgotten that gcc-cross depends on glibc. Based on the above there may still be some optimisation we can make here. In the back of my mind, I'm concious that one x86 cross compiler should be able to work for 32 and 64 bit too, its just the compiler options and libgcc etc. would need to be generated for both... > > Regardless, I do think this needs a little more thought, we also > > need > > better multiple test cases as we're not catching issues like this. > > think this needs to be revisited along with your outstanding > > multilib > > patch series which I haven't found the time to review yet (sorry). > That's all right, I'm very glad to make mutilib work well, including > adding test cases for them. It is nearly broken after changed from > smart + rpm5 to dnf + rpm4, those patches fixed the problem. Agreed, its important and on my list of things to review, I've just had to focus on getting build testing working properly and now I can try and clear some of the patch backlog. Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/2] image-live.bbclass: remove MLPREFIX from syslinux
On 01/03/2018 08:43 PM, Richard Purdie wrote: On Wed, 2017-12-13 at 10:45 +0800, Robert Yang wrote: Fixed: MACHINE = "qemux86-64" require conf/multilib.conf MULTILIBS = "multilib:lib32" DEFAULTTUNE_virtclass-multilib-lib32 = "x86" IMAGE_FSTYPES += "iso" $ bitbake lib32-core-image-minimal ERROR: lib32-core-image-minimal-1.0-r0 do_bootimg: The file /usr/include/printf.h is installed by both glibc and lib32-glibc, aborting This was because: lib32-syslinux -> lib32-glibc virtual/kernel -> glibc We can build 64bit syslinux (only build, not install) to fix the problem, the do_bootimg only needs several data files of syslinux such as vesamenu.c32, these files are not arch related. Hi Robert, Hi RP, Thanks for the reply. I've been thinking about this one and I'm not 100% convinced this is the right thing to do. When we build "lib32-core-image-minimal", one of the things we want to avoid is building two different toolchains, there should only be one needed for this image. If there is a dependency on "syslinux", that will need the non-multilib toolchain. I suspect that is why libX-syslinux was used as a dependency. Also, there are some things which never make sense as a multilib, the kernel is one example and I'm starting to wonder if syslinux would be another. In the kernel (and kernel module) case we'd provide all the libX variants from the same recipe, we may want to do that for syslinux. I think that syslinux is different from kernel, 64bit kernel can provide 32bit kernel via kernel config, but syslinux can't (except these non arch files). It may be we can't avoid the multiple compiler issue and the current codebase may not do so, I think currently we can avoid multiple glibc though. I'm not sure about a problem on this, should lib32-image can run 64bit programs or not ? If yes, then kernel should be 64bit, and we can't avoid building 64bit compilers. And I'm leaning to yes since we call it multilib, the pure 32bit image which can't run 64bit programs can't be called multilib. I did a rough search on why glibc is built, when bitbake lib32-image, it is because: 64 bit kernel -> gcc-cross-x86_64 -> virtual/${TARGET_PREFIX}libc-for-gcc, and the TARGET_PREFIX is x86_64 when building gcc-cross-x86_64, While the virtual/x86_64-poky-linux-libc-for-gcc is provided by glibc. I tried to remove the depends, both "bitbake gcc-cross-x86_64" and "bitbake linux-yocto" can be built, but other recipes such as quilt would be failed: | /workspace1/lyang1/test_1/tmp/work/core2-64-poky-linux/libgcc/7.2.0-r0/recipe-sysroot-native/usr/bin/x86_64-poky-linux/../../libexec/x86_64-poky-linux/gcc/x86_64-poky-linux/7.2.0/ld: cannot find crti.o: No such file or directory | /workspace1/lyang1/test_1/tmp/work/core2-64-poky-linux/libgcc/7.2.0-r0/recipe-sysroot-native/usr/bin/x86_64-poky-linux/../../libexec/x86_64-poky-linux/gcc/x86_64-poky-linux/7.2.0/ld: cannot find -lc | /workspace1/lyang1/test_1/tmp/work/core2-64-poky-linux/libgcc/7.2.0-r0/recipe-sysroot-native/usr/bin/x86_64-poky-linux/../../libexec/x86_64-poky-linux/gcc/x86_64-poky-linux/7.2.0/ld: cannot find crtn.o: No such file or directory | collect2: error: ld returned 1 exit status | Makefile:978: recipe for target 'libgcc_s.so' failed And move virtual/${TARGET_PREFIX}libc-for-gcc to BASE_DEFAULT_DEPS can't make it work either, so it seems that we can't avoid building glibc. Regardless, I do think this needs a little more thought, we also need better multiple test cases as we're not catching issues like this. think this needs to be revisited along with your outstanding multilib patch series which I haven't found the time to review yet (sorry). That's all right, I'm very glad to make mutilib work well, including adding test cases for them. It is nearly broken after changed from smart + rpm5 to dnf + rpm4, those patches fixed the problem. // Robert Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/2] image-live.bbclass: remove MLPREFIX from syslinux
On Wed, 2017-12-13 at 10:45 +0800, Robert Yang wrote: > Fixed: > MACHINE = "qemux86-64" > require conf/multilib.conf > MULTILIBS = "multilib:lib32" > DEFAULTTUNE_virtclass-multilib-lib32 = "x86" > IMAGE_FSTYPES += "iso" > > $ bitbake lib32-core-image-minimal > ERROR: lib32-core-image-minimal-1.0-r0 do_bootimg: The file > /usr/include/printf.h is installed by both glibc and lib32-glibc, > aborting > > This was because: > lib32-syslinux -> lib32-glibc > virtual/kernel -> glibc > > We can build 64bit syslinux (only build, not install) to fix the > problem, the > do_bootimg only needs several data files of syslinux such as > vesamenu.c32, > these files are not arch related. Hi Robert, I've been thinking about this one and I'm not 100% convinced this is the right thing to do. When we build "lib32-core-image-minimal", one of the things we want to avoid is building two different toolchains, there should only be one needed for this image. If there is a dependency on "syslinux", that will need the non-multilib toolchain. I suspect that is why libX-syslinux was used as a dependency. Also, there are some things which never make sense as a multilib, the kernel is one example and I'm starting to wonder if syslinux would be another. In the kernel (and kernel module) case we'd provide all the libX variants from the same recipe, we may want to do that for syslinux. It may be we can't avoid the multiple compiler issue and the current codebase may not do so, I think currently we can avoid multiple glibc though. Regardless, I do think this needs a little more thought, we also need better multiple test cases as we're not catching issues like this. I think this needs to be revisited along with your outstanding multilib patch series which I haven't found the time to review yet (sorry). Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 2/2] image-live.bbclass: remove MLPREFIX from syslinux
Fixed: MACHINE = "qemux86-64" require conf/multilib.conf MULTILIBS = "multilib:lib32" DEFAULTTUNE_virtclass-multilib-lib32 = "x86" IMAGE_FSTYPES += "iso" $ bitbake lib32-core-image-minimal ERROR: lib32-core-image-minimal-1.0-r0 do_bootimg: The file /usr/include/printf.h is installed by both glibc and lib32-glibc, aborting This was because: lib32-syslinux -> lib32-glibc virtual/kernel -> glibc We can build 64bit syslinux (only build, not install) to fix the problem, the do_bootimg only needs several data files of syslinux such as vesamenu.c32, these files are not arch related. Signed-off-by: Robert Yang--- meta/classes/image-live.bbclass | 2 +- meta/classes/syslinux.bbclass | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/meta/classes/image-live.bbclass b/meta/classes/image-live.bbclass index 9f8ff79..8872195 100644 --- a/meta/classes/image-live.bbclass +++ b/meta/classes/image-live.bbclass @@ -31,7 +31,7 @@ do_bootimg[depends] += "dosfstools-native:do_populate_sysroot \ mtools-native:do_populate_sysroot \ cdrtools-native:do_populate_sysroot \ virtual/kernel:do_deploy \ -${MLPREFIX}syslinux:do_populate_sysroot \ +syslinux:do_populate_sysroot \ syslinux-native:do_populate_sysroot \ ${@oe.utils.ifelse(d.getVar('COMPRESSISO', False),'zisofs-tools-native:do_populate_sysroot','')} \ ${PN}:do_image_${@d.getVar('LIVE_ROOTFS_TYPE').replace('-', '_')} \ diff --git a/meta/classes/syslinux.bbclass b/meta/classes/syslinux.bbclass index d6f8824..98939ba 100644 --- a/meta/classes/syslinux.bbclass +++ b/meta/classes/syslinux.bbclass @@ -17,7 +17,7 @@ # ${SYSLINUX_SERIAL_TTY} - Set alternate console=tty... kernel boot argument # ${SYSLINUX_KERNEL_ARGS} - Add additional kernel arguments -do_bootimg[depends] += "${MLPREFIX}syslinux:do_populate_sysroot \ +do_bootimg[depends] += "syslinux:do_populate_sysroot \ syslinux-native:do_populate_sysroot" ISOLINUXDIR ?= "/isolinux" -- 2.7.4 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core