Re: [lfs-dev] Modified build of LFS using pure cross-compilation and sysroot
On Sat, 2020-05-02 at 11:19 +1000, James B via lfs-dev wrote: > > > > > [1] http://www.linuxfromscratch.org/~pierre/lfs-modified/index.html > > If you can produce a single HTML version of this, that would be good > because it would be easier to archive for future reference. This idea > has its merits regardless its inclusion into the main LFS book. > nochunk and sysd versions added. Thanks for comments and support Pierre -- http://lists.linuxfromscratch.org/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: [lfs-dev] Modified build of LFS using pure cross-compilation and sysroot
On Sat, 2020-05-02 at 10:55 +0800, Xi Ruoyao via lfs-dev wrote: > On 2020-05-01 20:51 -0500, Douglas R. Reno via lfs-dev wrote: > > On 5/1/20 8:53 AM, Pierre Labastie via lfs-dev wrote: > > >cons: chroot is entered in the middle of chapter 5 (maybe > > > chapter 5 > > > should be split), the debug sections of several packages > > > reference > > > x86_64-lfs-linux-gnu instead of x86_64-pc-linux-gnu, binutils- > > > pass2 > > > needs "enable-shared". > > > > The debug information being broken is a major dealbreaker for me. I > > use > > the debug information from these packages semi-frequently and not > > having > > that at my disposal would make things a lot tougher for me. > > Hopefully we can fix the debug info with some -ffile-prefix-map > parameter in > $CC. Yes, This can be done, but it is messy: LFS_DIR=/usr/lib/gcc/$(uname -m)-lfs-linux-gnu DIR=/usr/lib/gcc/$(../scripts/config.guess) ../configure --prefix=/usr\ CC="gcc -ffile-prefix-map=$LFS_DIR=$DIR" blah But actually, the fact that the debug sections reference a wrong source location does not prevent using gdb in this case: the sources referenced are private gcc headers, which don't generate code, so are not used by gdb. Pierre -- http://lists.linuxfromscratch.org/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: [lfs-dev] Modified build of LFS using pure cross-compilation and sysroot
On 2020-05-01 20:51 -0500, Douglas R. Reno via lfs-dev wrote: > On 5/1/20 8:53 AM, Pierre Labastie via lfs-dev wrote: > >cons: chroot is entered in the middle of chapter 5 (maybe chapter 5 > > should be split), the debug sections of several packages reference > > x86_64-lfs-linux-gnu instead of x86_64-pc-linux-gnu, binutils-pass2 > > needs "enable-shared". > > The debug information being broken is a major dealbreaker for me. I use > the debug information from these packages semi-frequently and not having > that at my disposal would make things a lot tougher for me. Hopefully we can fix the debug info with some -ffile-prefix-map parameter in $CC. -- Xi Ruoyao School of Aerospace Science and Technology, Xidian University -- http://lists.linuxfromscratch.org/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: [lfs-dev] Modified build of LFS using pure cross-compilation and sysroot
On 5/1/20 8:53 AM, Pierre Labastie via lfs-dev wrote: Hi, I propose a new way to build LFS, which removes the need for the /tools symlink, and decreases the number of tweaks needed when building gcc. The current build builds a cross-compiler in pass1, and uses it as a native compiler in pass2. This needs to use a non standard directory (/tools) to host the toolchain and insulate it from the build machine. The modified build uses the cross compiler to cross compile packages that need themselves to be rebuilt, thus insulating them automatically from the host, without the need for a non standard directory layout. Chroot is entered as soon as possible, and the remaining chapter 5 packages are built in chroot. This is WIP: the text must be improved at several places, bison and flex may be moved to after chroot (to be tested). But the commands seem to produce an acceptable system, with almost clean ICA. You can view it at [1], only for sys V since I have not tested systemd yet (I do not expect many changes). There are pros and cons compared to the current method: pros: no /tools symlink, no need to tweak gcc sources, no need to build twice ld in binutils-pass2, no need to readjust the toolchain after chapter 6 glibc, no need to tweak the gcc specs, no need to reinstall kernel headers in chapter 6. These are some nice pros! :) This is very interesting cons: chroot is entered in the middle of chapter 5 (maybe chapter 5 should be split), the debug sections of several packages reference x86_64-lfs-linux-gnu instead of x86_64-pc-linux-gnu, binutils-pass2 needs "enable-shared". The debug information being broken is a major dealbreaker for me. I use the debug information from these packages semi-frequently and not having that at my disposal would make things a lot tougher for me. I think Chapter 5 would have to end after "Changing Ownership". Otherwise we'd run the risk of causing confusion and potentially breaking people's host systems (as Ken mentioned) Another pro, not tried, is that some simple packages built in chapter 5 may be built only once if testing them is not required. Comments and suggestions for improvement (there's a lot of room for improvement) welcome. Regards Pierre [1] http://www.linuxfromscratch.org/~pierre/lfs-modified/index.html -- http://lists.linuxfromscratch.org/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: [lfs-dev] Modified build of LFS using pure cross-compilation and sysroot
On Fri, 01 May 2020 15:53:55 +0200 Pierre Labastie via lfs-dev wrote: > Hi, > > I propose a new way to build LFS, which removes the need for the /tools > symlink, and decreases the number of tweaks needed when building gcc. This is very interesting! > > The current build builds a cross-compiler in pass1, and uses it as a > native compiler in pass2. This needs to use a non standard directory > (/tools) to host the toolchain and insulate it from the build machine. > > The modified build uses the cross compiler to cross compile packages > that need themselves to be rebuilt, thus insulating them automatically > from the host, without the need for a non standard directory layout. > Chroot is entered as soon as possible, and the remaining chapter 5 > packages are built in chroot. I have always wondered why it was not done like this. Cross-LFS - which originally came from LFS - add another layer on top of this; so in the end gcc was built thrice (cross-compiler, the "/tools" compiler, and then the final native compiler). With your method, cross-lfs could have done it in two-pass - exactly as the same as LFS. > > This is WIP: the text must be improved at several places, bison and > flex may be moved to after chroot (to be tested). But the commands seem > to produce an acceptable system, with almost clean ICA. > Sorry, I'm not familiar with the term. What does "ICA" stands for? > You can view it at [1], only for sys V since I have not tested systemd > yet (I do not expect many changes). I use SysV so this is good enough for me. Not familiar enough with systemd to give any intelligent comment whether more extensive tweaks are needed (especially when tests are involved). > > There are pros and cons compared to the current method: > > pros: no /tools symlink, no need to tweak gcc sources, no need to > build twice ld in binutils-pass2, no need to readjust the toolchain > after chapter 6 glibc, no need to tweak the gcc specs, no need to > reinstall kernel headers in chapter 6. In my view this is a big pro, especially the "no tweak" part, since it means less breakage when upstream updates gcc. The reduction of the builds is also welcome, though. > > cons: chroot is entered in the middle of chapter 5 (maybe chapter 5 > should be split), Either it it split (Chapter 5A and 5B), or move those sections _after_ the chroot to Chapter 6. Alternatively split and renumber (5A stays as 5, 5B becomes 6, 6 becomes 7) but that's probably an unacceptably big change. > the debug sections of several packages reference > x86_64-lfs-linux-gnu instead of x86_64-pc-linux-gnu, binutils-pass2 > needs "enable-shared". The debug stuff is a let-down - though like Ken, I seldom use it; however it is very handy for the time when I _really_ need to it (and that has happened a couple of times). Which packages are you talking about? Hopefully not glibc/libgcc/libstdc++? > > Another pro, not tried, is that some simple packages built in chapter 5 > may be built only once if testing them is not required. It's nice to have them tested during development builds (=when the book is being updated). But it's also nice to have options to skip these for the "brave" end-user :) > > Comments and suggestions for improvement (there's a lot of room for > improvement) welcome. > > Regards > Pierre > > > [1] http://www.linuxfromscratch.org/~pierre/lfs-modified/index.html If you can produce a single HTML version of this, that would be good because it would be easier to archive for future reference. This idea has its merits regardless its inclusion into the main LFS book. Thanks! -- James B -- http://lists.linuxfromscratch.org/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: [lfs-dev] Modified build of LFS using pure cross-compilation and sysroot
As a user of LFS, I appreciate the cleanliness of building the “/tools” directory in Chapter 5, for this simple reason: I can pause and log out midway through executing chapter 5, and come back - without worrying about re-establishing a “context” like a chroot in chapter 5 would require. If I forget to enter chroot, the mistake can be costly. This is VERY important: if I come back and continue without remembering to restart that chroot (if chroot were used in chapter 5) - I could mess up my current, running Linux system with a build step in chapter 5 that assumes I’m in a chroot environment. Whereas, if I forget the chroot step in Getting back to chapter 6 the next day, rarely does anything bad happen, because the build steps in chapter 6 are mostly such that they are exactly what I’d execute when installing a new version of one of the packages. (Yes, a build of glibc outside of the chroot could cause trouble...) In short, forgetting to enter chroot In chapter 6 has far less possible downside than it would if I forgot in chapter 5. /tools approach in chapter 5 is harder to forget because the use of /tools is part of every build instruction just where it needs to be. There’s fewer assumptions about the current environment. Joel Sent from my iPhone > On May 1, 2020, at 3:58 PM, Ken Moffat via lfs-dev > wrote: > > On Fri, May 01, 2020 at 03:53:55PM +0200, Pierre Labastie via lfs-dev wrote: >> Hi, >> >> I propose a new way to build LFS, which removes the need for the /tools >> symlink, and decreases the number of tweaks needed when building gcc. >> >> The current build builds a cross-compiler in pass1, and uses it as a >> native compiler in pass2. This needs to use a non standard directory >> (/tools) to host the toolchain and insulate it from the build machine. >> >> The modified build uses the cross compiler to cross compile packages >> that need themselves to be rebuilt, thus insulating them automatically >> from the host, without the need for a non standard directory layout. >> Chroot is entered as soon as possible, and the remaining chapter 5 >> packages are built in chroot. >> > > If that happens, entering chroot ought to be after chapter 5. At > the moment, we believe that up to the end of chapter 5 (apart from > when creating the filesystem and setting up the lfs user) you will > not trash the host system. I'm worried about how people will > misinterpret what they should be doing if they come from our present > builds, particularly if they have created their own scripts. > >> This is WIP: the text must be improved at several places, bison and >> flex may be moved to after chroot (to be tested). But the commands seem >> to produce an acceptable system, with almost clean ICA. >> >> You can view it at [1], only for sys V since I have not tested systemd >> yet (I do not expect many changes). >> >> There are pros and cons compared to the current method: >> >> pros: no /tools symlink, no need to tweak gcc sources, no need to >> build twice ld in binutils-pass2, no need to readjust the toolchain >> after chapter 6 glibc, no need to tweak the gcc specs, no need to >> reinstall kernel headers in chapter 6. >> >> cons: chroot is entered in the middle of chapter 5 (maybe chapter 5 >> should be split), the debug sections of several packages reference >> x86_64-lfs-linux-gnu instead of x86_64-pc-linux-gnu, binutils-pass2 >> needs "enable-shared". >> > > Not keen on that in the debug data, although I almost never use it. > >> Another pro, not tried, is that some simple packages built in chapter 5 >> may be built only once if testing them is not required. >> > > Based on experience from Pure LFS (5.0, where tests were introduced) > I don't regard that as a pro :) > >> Comments and suggestions for improvement (there's a lot of room for >> improvement) welcome. >> >> Regards >> Pierre >> >> >> [1] http://www.linuxfromscratch.org/~pierre/lfs-modified/index.html >> > > My main concern with making a change like this is not that we might > break the reliabilty, or the purity (ICA), but that we now > understand most of the things which can go wrong, and we try to > ameliorate them. > > With a revised process, we might lose most of our past knowledge > about how breakage occurs (in the sense of "I saw this, I recall > that I'd done/omitted ...), and therefore how to fix it. > > ĸen > -- > See You Later, Holy Poppadom! >-- Red Dwarf, The Promised Land > -- > http://lists.linuxfromscratch.org/listinfo/lfs-dev > FAQ: http://www.linuxfromscratch.org/faq/ > Unsubscribe: See the above information page -- http://lists.linuxfromscratch.org/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: [lfs-dev] Modified build of LFS using pure cross-compilation and sysroot
On Fri, May 01, 2020 at 03:53:55PM +0200, Pierre Labastie via lfs-dev wrote: > Hi, > > I propose a new way to build LFS, which removes the need for the /tools > symlink, and decreases the number of tweaks needed when building gcc. > > The current build builds a cross-compiler in pass1, and uses it as a > native compiler in pass2. This needs to use a non standard directory > (/tools) to host the toolchain and insulate it from the build machine. > > The modified build uses the cross compiler to cross compile packages > that need themselves to be rebuilt, thus insulating them automatically > from the host, without the need for a non standard directory layout. > Chroot is entered as soon as possible, and the remaining chapter 5 > packages are built in chroot. > If that happens, entering chroot ought to be after chapter 5. At the moment, we believe that up to the end of chapter 5 (apart from when creating the filesystem and setting up the lfs user) you will not trash the host system. I'm worried about how people will misinterpret what they should be doing if they come from our present builds, particularly if they have created their own scripts. > This is WIP: the text must be improved at several places, bison and > flex may be moved to after chroot (to be tested). But the commands seem > to produce an acceptable system, with almost clean ICA. > > You can view it at [1], only for sys V since I have not tested systemd > yet (I do not expect many changes). > > There are pros and cons compared to the current method: > > pros: no /tools symlink, no need to tweak gcc sources, no need to > build twice ld in binutils-pass2, no need to readjust the toolchain > after chapter 6 glibc, no need to tweak the gcc specs, no need to > reinstall kernel headers in chapter 6. > > cons: chroot is entered in the middle of chapter 5 (maybe chapter 5 > should be split), the debug sections of several packages reference > x86_64-lfs-linux-gnu instead of x86_64-pc-linux-gnu, binutils-pass2 > needs "enable-shared". > Not keen on that in the debug data, although I almost never use it. > Another pro, not tried, is that some simple packages built in chapter 5 > may be built only once if testing them is not required. > Based on experience from Pure LFS (5.0, where tests were introduced) I don't regard that as a pro :) > Comments and suggestions for improvement (there's a lot of room for > improvement) welcome. > > Regards > Pierre > > > [1] http://www.linuxfromscratch.org/~pierre/lfs-modified/index.html > My main concern with making a change like this is not that we might break the reliabilty, or the purity (ICA), but that we now understand most of the things which can go wrong, and we try to ameliorate them. With a revised process, we might lose most of our past knowledge about how breakage occurs (in the sense of "I saw this, I recall that I'd done/omitted ...), and therefore how to fix it. ĸen -- See You Later, Holy Poppadom! -- Red Dwarf, The Promised Land -- http://lists.linuxfromscratch.org/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: [lfs-dev] Modified build of LFS using pure cross-compilation and sysroot
On 5/1/20 8:53 AM, Pierre Labastie via lfs-dev wrote: Hi, I propose a new way to build LFS, which removes the need for the /tools symlink, and decreases the number of tweaks needed when building gcc. The current build builds a cross-compiler in pass1, and uses it as a native compiler in pass2. This needs to use a non standard directory (/tools) to host the toolchain and insulate it from the build machine. The modified build uses the cross compiler to cross compile packages that need themselves to be rebuilt, thus insulating them automatically from the host, without the need for a non standard directory layout. Chroot is entered as soon as possible, and the remaining chapter 5 packages are built in chroot. This is WIP: the text must be improved at several places, bison and flex may be moved to after chroot (to be tested). But the commands seem to produce an acceptable system, with almost clean ICA. You can view it at [1], only for sys V since I have not tested systemd yet (I do not expect many changes). There are pros and cons compared to the current method: pros: no /tools symlink, no need to tweak gcc sources, no need to build twice ld in binutils-pass2, no need to readjust the toolchain after chapter 6 glibc, no need to tweak the gcc specs, no need to reinstall kernel headers in chapter 6. cons: chroot is entered in the middle of chapter 5 (maybe chapter 5 should be split), the debug sections of several packages reference x86_64-lfs-linux-gnu instead of x86_64-pc-linux-gnu, binutils-pass2 needs "enable-shared". Another pro, not tried, is that some simple packages built in chapter 5 may be built only once if testing them is not required. Comments and suggestions for improvement (there's a lot of room for improvement) welcome. Regards Pierre [1] http://www.linuxfromscratch.org/~pierre/lfs-modified/index.html It's an interesting idea, but generally I'm in the "If it's not broke, don't fix it." camp. What we have is a very well tested system. The current way we isolate /tools as user lfs in Chapter 5 and then go into chroot in Chapter 6 seems to encapsulate each chapter well. There are always multiple ways to accomplish a complex goal like LFS. To me the advantages of the proposal are relatively minor and my initial reaction is to leave things the way they are. -- Bruce -- http://lists.linuxfromscratch.org/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
[lfs-dev] Modified build of LFS using pure cross-compilation and sysroot
Hi, I propose a new way to build LFS, which removes the need for the /tools symlink, and decreases the number of tweaks needed when building gcc. The current build builds a cross-compiler in pass1, and uses it as a native compiler in pass2. This needs to use a non standard directory (/tools) to host the toolchain and insulate it from the build machine. The modified build uses the cross compiler to cross compile packages that need themselves to be rebuilt, thus insulating them automatically from the host, without the need for a non standard directory layout. Chroot is entered as soon as possible, and the remaining chapter 5 packages are built in chroot. This is WIP: the text must be improved at several places, bison and flex may be moved to after chroot (to be tested). But the commands seem to produce an acceptable system, with almost clean ICA. You can view it at [1], only for sys V since I have not tested systemd yet (I do not expect many changes). There are pros and cons compared to the current method: pros: no /tools symlink, no need to tweak gcc sources, no need to build twice ld in binutils-pass2, no need to readjust the toolchain after chapter 6 glibc, no need to tweak the gcc specs, no need to reinstall kernel headers in chapter 6. cons: chroot is entered in the middle of chapter 5 (maybe chapter 5 should be split), the debug sections of several packages reference x86_64-lfs-linux-gnu instead of x86_64-pc-linux-gnu, binutils-pass2 needs "enable-shared". Another pro, not tried, is that some simple packages built in chapter 5 may be built only once if testing them is not required. Comments and suggestions for improvement (there's a lot of room for improvement) welcome. Regards Pierre [1] http://www.linuxfromscratch.org/~pierre/lfs-modified/index.html -- http://lists.linuxfromscratch.org/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
