Re: [lfs-dev] Modified build of LFS using pure cross-compilation and sysroot
On Sat, 2020-05-02 at 19:02 +0200, Thomas Trepl via lfs-dev wrote: > Am Freitag, den 01.05.2020, 15:53 +0200 schrieb Pierre Labastie via > lfs-dev: > > Hi, > > > > I propose a new way to build LFS, which removes the need for the > > /tools > > symlink, and decreases the number of tweaks needed when building > > gcc. > > > > The current build builds a cross-compiler in pass1, and uses it as > > a > > native compiler in pass2. This needs to use a non standard > > directory > > (/tools) to host the toolchain and insulate it from the build > > machine. > > > > The modified build uses the cross compiler to cross compile > > packages > > that need themselves to be rebuilt, thus insulating them > > automatically > > from the host, without the need for a non standard directory > > layout. > > Chroot is entered as soon as possible, and the remaining chapter 5 > > packages are built in chroot. > > Wow, quite interesting! I'd vote to have a deeper look to that > mechanism. > > > This is WIP: the text must be improved at several places, bison and > > flex may be moved to after chroot (to be tested). But the commands > > seem > > to produce an acceptable system, with almost clean ICA. > > > > You can view it at [1], only for sys V since I have not tested > > systemd > > yet (I do not expect many changes). > > > > There are pros and cons compared to the current method: > > > > pros: no /tools symlink, no need to tweak gcc sources, no need to > > build twice ld in binutils-pass2, no need to readjust the toolchain > > after chapter 6 glibc, no need to tweak the gcc specs, no need to > > reinstall kernel headers in chapter 6. > > The less we have to tweak core packages (and gcc is for sure one of > them), the better. I would see that as a great benefit. > > > cons: chroot is entered in the middle of chapter 5 (maybe chapter > > 5 > > should be split), the debug sections of several packages reference > > x86_64-lfs-linux-gnu instead of x86_64-pc-linux-gnu, binutils-pass2 > > needs "enable-shared". > Ok, here is the "traditional" separation of Chapter 5 (as run with > host tools) and the Chapter 6 (running previously built tools in > chroot) in danger. But at the end, its just a naming issue. > > With your method we have kinda Chapter 5a building core tools, a > Chapter 5b building toolchain for full build using 5a in chroot and > finally the well-known Chapter 6. > > For the cost of entering the chroot in the middle of chap5, we got > rid > of the adjusting which sometimes causes trouble for newbies. This > knot > is simply untied. > > > Another pro, not tried, is that some simple packages built in > > chapter 5 > > may be built only once if testing them is not required. > > > > Comments and suggestions for improvement (there's a lot of room for > > improvement) welcome. > > Its great! I'd see that as a big evolution step in the LFS > ecosystem. > > I see a restart to continue not more complicated as it is now. When > continuing an interrupted build in chap5 now, user has also to > prepare > the environment carefully. Same in new chapter 5a so no change here. > Continuing in chap 5b, only need to reenter the chroot, this not any > different to want currently has to be done while it comes a few steps > earlier. > > Maybe the new Chapter 5 should end with leaving the chroot > environment > with unmounting the virtFS. It could be a proper end to chap5 as at > this stage, a backup of the system might be recommended (as a hint > for > the user). > The new chapter 6 should then reenter the chroot env again. IMHO this > would make the differentiation of chap5 and 6 a bit easier. > Thanks to all who answered. The book source is now available at: svn co svn://svn.linuxfromscratch.org/LFS/branches/cross-chap5 Some fixes have been done, but there is still a problem with ncurses: the terminfo database is not built because it needs tic from the build host, and we do not require it. Then using the terminal in chroot is painful (no backspace nor line editing capability). One simple fix would be to require tic on the build host with version greater than or equal to 6.0, and remove the --disable-db-install switch. Another fix would be to first build ncurses for the build host (no cross-compile), and use the just built tic (there are configure switches for that). Another fix is to cheat and use the cross-compiled tic on the build host (but it's not pure cross compilation anymore). Pierre -- http://lists.linuxfromscratch.org/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: [lfs-dev] Modified build of LFS using pure cross-compilation and sysroot
On Sat, 2020-05-02 at 11:19 +1000, James B via lfs-dev wrote: > > > > > [1] http://www.linuxfromscratch.org/~pierre/lfs-modified/index.html > > If you can produce a single HTML version of this, that would be good > because it would be easier to archive for future reference. This idea > has its merits regardless its inclusion into the main LFS book. > nochunk and sysd versions added. Thanks for comments and support Pierre -- http://lists.linuxfromscratch.org/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: [lfs-dev] Modified build of LFS using pure cross-compilation and sysroot
On Sat, 2020-05-02 at 10:55 +0800, Xi Ruoyao via lfs-dev wrote: > On 2020-05-01 20:51 -0500, Douglas R. Reno via lfs-dev wrote: > > On 5/1/20 8:53 AM, Pierre Labastie via lfs-dev wrote: > > >cons: chroot is entered in the middle of chapter 5 (maybe > > > chapter 5 > > > should be split), the debug sections of several packages > > > reference > > > x86_64-lfs-linux-gnu instead of x86_64-pc-linux-gnu, binutils- > > > pass2 > > > needs "enable-shared". > > > > The debug information being broken is a major dealbreaker for me. I > > use > > the debug information from these packages semi-frequently and not > > having > > that at my disposal would make things a lot tougher for me. > > Hopefully we can fix the debug info with some -ffile-prefix-map > parameter in > $CC. Yes, This can be done, but it is messy: LFS_DIR=/usr/lib/gcc/$(uname -m)-lfs-linux-gnu DIR=/usr/lib/gcc/$(../scripts/config.guess) ../configure --prefix=/usr\ CC="gcc -ffile-prefix-map=$LFS_DIR=$DIR" blah But actually, the fact that the debug sections reference a wrong source location does not prevent using gdb in this case: the sources referenced are private gcc headers, which don't generate code, so are not used by gdb. Pierre -- http://lists.linuxfromscratch.org/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: [lfs-dev] Modified build of LFS using pure cross-compilation and sysroot
On 2020-05-01 20:51 -0500, Douglas R. Reno via lfs-dev wrote: > On 5/1/20 8:53 AM, Pierre Labastie via lfs-dev wrote: > >cons: chroot is entered in the middle of chapter 5 (maybe chapter 5 > > should be split), the debug sections of several packages reference > > x86_64-lfs-linux-gnu instead of x86_64-pc-linux-gnu, binutils-pass2 > > needs "enable-shared". > > The debug information being broken is a major dealbreaker for me. I use > the debug information from these packages semi-frequently and not having > that at my disposal would make things a lot tougher for me. Hopefully we can fix the debug info with some -ffile-prefix-map parameter in $CC. -- Xi Ruoyao School of Aerospace Science and Technology, Xidian University -- http://lists.linuxfromscratch.org/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: [lfs-dev] Modified build of LFS using pure cross-compilation and sysroot
On 5/1/20 8:53 AM, Pierre Labastie via lfs-dev wrote: Hi, I propose a new way to build LFS, which removes the need for the /tools symlink, and decreases the number of tweaks needed when building gcc. The current build builds a cross-compiler in pass1, and uses it as a native compiler in pass2. This needs to use a non standard directory (/tools) to host the toolchain and insulate it from the build machine. The modified build uses the cross compiler to cross compile packages that need themselves to be rebuilt, thus insulating them automatically from the host, without the need for a non standard directory layout. Chroot is entered as soon as possible, and the remaining chapter 5 packages are built in chroot. This is WIP: the text must be improved at several places, bison and flex may be moved to after chroot (to be tested). But the commands seem to produce an acceptable system, with almost clean ICA. You can view it at [1], only for sys V since I have not tested systemd yet (I do not expect many changes). There are pros and cons compared to the current method: pros: no /tools symlink, no need to tweak gcc sources, no need to build twice ld in binutils-pass2, no need to readjust the toolchain after chapter 6 glibc, no need to tweak the gcc specs, no need to reinstall kernel headers in chapter 6. These are some nice pros! :) This is very interesting cons: chroot is entered in the middle of chapter 5 (maybe chapter 5 should be split), the debug sections of several packages reference x86_64-lfs-linux-gnu instead of x86_64-pc-linux-gnu, binutils-pass2 needs "enable-shared". The debug information being broken is a major dealbreaker for me. I use the debug information from these packages semi-frequently and not having that at my disposal would make things a lot tougher for me. I think Chapter 5 would have to end after "Changing Ownership". Otherwise we'd run the risk of causing confusion and potentially breaking people's host systems (as Ken mentioned) Another pro, not tried, is that some simple packages built in chapter 5 may be built only once if testing them is not required. Comments and suggestions for improvement (there's a lot of room for improvement) welcome. Regards Pierre [1] http://www.linuxfromscratch.org/~pierre/lfs-modified/index.html -- http://lists.linuxfromscratch.org/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: [lfs-dev] Modified build of LFS using pure cross-compilation and sysroot
On Fri, 01 May 2020 15:53:55 +0200 Pierre Labastie via lfs-dev wrote: > Hi, > > I propose a new way to build LFS, which removes the need for the /tools > symlink, and decreases the number of tweaks needed when building gcc. This is very interesting! > > The current build builds a cross-compiler in pass1, and uses it as a > native compiler in pass2. This needs to use a non standard directory > (/tools) to host the toolchain and insulate it from the build machine. > > The modified build uses the cross compiler to cross compile packages > that need themselves to be rebuilt, thus insulating them automatically > from the host, without the need for a non standard directory layout. > Chroot is entered as soon as possible, and the remaining chapter 5 > packages are built in chroot. I have always wondered why it was not done like this. Cross-LFS - which originally came from LFS - add another layer on top of this; so in the end gcc was built thrice (cross-compiler, the "/tools" compiler, and then the final native compiler). With your method, cross-lfs could have done it in two-pass - exactly as the same as LFS. > > This is WIP: the text must be improved at several places, bison and > flex may be moved to after chroot (to be tested). But the commands seem > to produce an acceptable system, with almost clean ICA. > Sorry, I'm not familiar with the term. What does "ICA" stands for? > You can view it at [1], only for sys V since I have not tested systemd > yet (I do not expect many changes). I use SysV so this is good enough for me. Not familiar enough with systemd to give any intelligent comment whether more extensive tweaks are needed (especially when tests are involved). > > There are pros and cons compared to the current method: > > pros: no /tools symlink, no need to tweak gcc sources, no need to > build twice ld in binutils-pass2, no need to readjust the toolchain > after chapter 6 glibc, no need to tweak the gcc specs, no need to > reinstall kernel headers in chapter 6. In my view this is a big pro, especially the "no tweak" part, since it means less breakage when upstream updates gcc. The reduction of the builds is also welcome, though. > > cons: chroot is entered in the middle of chapter 5 (maybe chapter 5 > should be split), Either it it split (Chapter 5A and 5B), or move those sections _after_ the chroot to Chapter 6. Alternatively split and renumber (5A stays as 5, 5B becomes 6, 6 becomes 7) but that's probably an unacceptably big change. > the debug sections of several packages reference > x86_64-lfs-linux-gnu instead of x86_64-pc-linux-gnu, binutils-pass2 > needs "enable-shared". The debug stuff is a let-down - though like Ken, I seldom use it; however it is very handy for the time when I _really_ need to it (and that has happened a couple of times). Which packages are you talking about? Hopefully not glibc/libgcc/libstdc++? > > Another pro, not tried, is that some simple packages built in chapter 5 > may be built only once if testing them is not required. It's nice to have them tested during development builds (=when the book is being updated). But it's also nice to have options to skip these for the "brave" end-user :) > > Comments and suggestions for improvement (there's a lot of room for > improvement) welcome. > > Regards > Pierre > > > [1] http://www.linuxfromscratch.org/~pierre/lfs-modified/index.html If you can produce a single HTML version of this, that would be good because it would be easier to archive for future reference. This idea has its merits regardless its inclusion into the main LFS book. Thanks! -- James B -- http://lists.linuxfromscratch.org/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: [lfs-dev] Modified build of LFS using pure cross-compilation and sysroot
As a user of LFS, I appreciate the cleanliness of building the “/tools” directory in Chapter 5, for this simple reason: I can pause and log out midway through executing chapter 5, and come back - without worrying about re-establishing a “context” like a chroot in chapter 5 would require. If I forget to enter chroot, the mistake can be costly. This is VERY important: if I come back and continue without remembering to restart that chroot (if chroot were used in chapter 5) - I could mess up my current, running Linux system with a build step in chapter 5 that assumes I’m in a chroot environment. Whereas, if I forget the chroot step in Getting back to chapter 6 the next day, rarely does anything bad happen, because the build steps in chapter 6 are mostly such that they are exactly what I’d execute when installing a new version of one of the packages. (Yes, a build of glibc outside of the chroot could cause trouble...) In short, forgetting to enter chroot In chapter 6 has far less possible downside than it would if I forgot in chapter 5. /tools approach in chapter 5 is harder to forget because the use of /tools is part of every build instruction just where it needs to be. There’s fewer assumptions about the current environment. Joel Sent from my iPhone > On May 1, 2020, at 3:58 PM, Ken Moffat via lfs-dev > wrote: > > On Fri, May 01, 2020 at 03:53:55PM +0200, Pierre Labastie via lfs-dev wrote: >> Hi, >> >> I propose a new way to build LFS, which removes the need for the /tools >> symlink, and decreases the number of tweaks needed when building gcc. >> >> The current build builds a cross-compiler in pass1, and uses it as a >> native compiler in pass2. This needs to use a non standard directory >> (/tools) to host the toolchain and insulate it from the build machine. >> >> The modified build uses the cross compiler to cross compile packages >> that need themselves to be rebuilt, thus insulating them automatically >> from the host, without the need for a non standard directory layout. >> Chroot is entered as soon as possible, and the remaining chapter 5 >> packages are built in chroot. >> > > If that happens, entering chroot ought to be after chapter 5. At > the moment, we believe that up to the end of chapter 5 (apart from > when creating the filesystem and setting up the lfs user) you will > not trash the host system. I'm worried about how people will > misinterpret what they should be doing if they come from our present > builds, particularly if they have created their own scripts. > >> This is WIP: the text must be improved at several places, bison and >> flex may be moved to after chroot (to be tested). But the commands seem >> to produce an acceptable system, with almost clean ICA. >> >> You can view it at [1], only for sys V since I have not tested systemd >> yet (I do not expect many changes). >> >> There are pros and cons compared to the current method: >> >> pros: no /tools symlink, no need to tweak gcc sources, no need to >> build twice ld in binutils-pass2, no need to readjust the toolchain >> after chapter 6 glibc, no need to tweak the gcc specs, no need to >> reinstall kernel headers in chapter 6. >> >> cons: chroot is entered in the middle of chapter 5 (maybe chapter 5 >> should be split), the debug sections of several packages reference >> x86_64-lfs-linux-gnu instead of x86_64-pc-linux-gnu, binutils-pass2 >> needs "enable-shared". >> > > Not keen on that in the debug data, although I almost never use it. > >> Another pro, not tried, is that some simple packages built in chapter 5 >> may be built only once if testing them is not required. >> > > Based on experience from Pure LFS (5.0, where tests were introduced) > I don't regard that as a pro :) > >> Comments and suggestions for improvement (there's a lot of room for >> improvement) welcome. >> >> Regards >> Pierre >> >> >> [1] http://www.linuxfromscratch.org/~pierre/lfs-modified/index.html >> > > My main concern with making a change like this is not that we might > break the reliabilty, or the purity (ICA), but that we now > understand most of the things which can go wrong, and we try to > ameliorate them. > > With a revised process, we might lose most of our past knowledge > about how breakage occurs (in the sense of "I saw this, I recall > that I'd done/omitted ...), and therefore how to fix it. > > ĸen > -- > See You Later, Holy Poppadom! >-- Red Dwarf, The Promised Land > -- > http://lists.linuxfromscratch.org/listinfo/lfs-dev > FAQ: http://www.linuxfromscratch.org/faq/ > Unsubscribe: See the above information page -- http://lists.linuxfromscratch.org/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: [lfs-dev] Modified build of LFS using pure cross-compilation and sysroot
On Fri, May 01, 2020 at 03:53:55PM +0200, Pierre Labastie via lfs-dev wrote: > Hi, > > I propose a new way to build LFS, which removes the need for the /tools > symlink, and decreases the number of tweaks needed when building gcc. > > The current build builds a cross-compiler in pass1, and uses it as a > native compiler in pass2. This needs to use a non standard directory > (/tools) to host the toolchain and insulate it from the build machine. > > The modified build uses the cross compiler to cross compile packages > that need themselves to be rebuilt, thus insulating them automatically > from the host, without the need for a non standard directory layout. > Chroot is entered as soon as possible, and the remaining chapter 5 > packages are built in chroot. > If that happens, entering chroot ought to be after chapter 5. At the moment, we believe that up to the end of chapter 5 (apart from when creating the filesystem and setting up the lfs user) you will not trash the host system. I'm worried about how people will misinterpret what they should be doing if they come from our present builds, particularly if they have created their own scripts. > This is WIP: the text must be improved at several places, bison and > flex may be moved to after chroot (to be tested). But the commands seem > to produce an acceptable system, with almost clean ICA. > > You can view it at [1], only for sys V since I have not tested systemd > yet (I do not expect many changes). > > There are pros and cons compared to the current method: > > pros: no /tools symlink, no need to tweak gcc sources, no need to > build twice ld in binutils-pass2, no need to readjust the toolchain > after chapter 6 glibc, no need to tweak the gcc specs, no need to > reinstall kernel headers in chapter 6. > > cons: chroot is entered in the middle of chapter 5 (maybe chapter 5 > should be split), the debug sections of several packages reference > x86_64-lfs-linux-gnu instead of x86_64-pc-linux-gnu, binutils-pass2 > needs "enable-shared". > Not keen on that in the debug data, although I almost never use it. > Another pro, not tried, is that some simple packages built in chapter 5 > may be built only once if testing them is not required. > Based on experience from Pure LFS (5.0, where tests were introduced) I don't regard that as a pro :) > Comments and suggestions for improvement (there's a lot of room for > improvement) welcome. > > Regards > Pierre > > > [1] http://www.linuxfromscratch.org/~pierre/lfs-modified/index.html > My main concern with making a change like this is not that we might break the reliabilty, or the purity (ICA), but that we now understand most of the things which can go wrong, and we try to ameliorate them. With a revised process, we might lose most of our past knowledge about how breakage occurs (in the sense of "I saw this, I recall that I'd done/omitted ...), and therefore how to fix it. ĸen -- See You Later, Holy Poppadom! -- Red Dwarf, The Promised Land -- http://lists.linuxfromscratch.org/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
Re: [lfs-dev] Modified build of LFS using pure cross-compilation and sysroot
On 5/1/20 8:53 AM, Pierre Labastie via lfs-dev wrote: Hi, I propose a new way to build LFS, which removes the need for the /tools symlink, and decreases the number of tweaks needed when building gcc. The current build builds a cross-compiler in pass1, and uses it as a native compiler in pass2. This needs to use a non standard directory (/tools) to host the toolchain and insulate it from the build machine. The modified build uses the cross compiler to cross compile packages that need themselves to be rebuilt, thus insulating them automatically from the host, without the need for a non standard directory layout. Chroot is entered as soon as possible, and the remaining chapter 5 packages are built in chroot. This is WIP: the text must be improved at several places, bison and flex may be moved to after chroot (to be tested). But the commands seem to produce an acceptable system, with almost clean ICA. You can view it at [1], only for sys V since I have not tested systemd yet (I do not expect many changes). There are pros and cons compared to the current method: pros: no /tools symlink, no need to tweak gcc sources, no need to build twice ld in binutils-pass2, no need to readjust the toolchain after chapter 6 glibc, no need to tweak the gcc specs, no need to reinstall kernel headers in chapter 6. cons: chroot is entered in the middle of chapter 5 (maybe chapter 5 should be split), the debug sections of several packages reference x86_64-lfs-linux-gnu instead of x86_64-pc-linux-gnu, binutils-pass2 needs "enable-shared". Another pro, not tried, is that some simple packages built in chapter 5 may be built only once if testing them is not required. Comments and suggestions for improvement (there's a lot of room for improvement) welcome. Regards Pierre [1] http://www.linuxfromscratch.org/~pierre/lfs-modified/index.html It's an interesting idea, but generally I'm in the "If it's not broke, don't fix it." camp. What we have is a very well tested system. The current way we isolate /tools as user lfs in Chapter 5 and then go into chroot in Chapter 6 seems to encapsulate each chapter well. There are always multiple ways to accomplish a complex goal like LFS. To me the advantages of the proposal are relatively minor and my initial reaction is to leave things the way they are. -- Bruce -- http://lists.linuxfromscratch.org/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
[lfs-dev] Modified build of LFS using pure cross-compilation and sysroot
Hi, I propose a new way to build LFS, which removes the need for the /tools symlink, and decreases the number of tweaks needed when building gcc. The current build builds a cross-compiler in pass1, and uses it as a native compiler in pass2. This needs to use a non standard directory (/tools) to host the toolchain and insulate it from the build machine. The modified build uses the cross compiler to cross compile packages that need themselves to be rebuilt, thus insulating them automatically from the host, without the need for a non standard directory layout. Chroot is entered as soon as possible, and the remaining chapter 5 packages are built in chroot. This is WIP: the text must be improved at several places, bison and flex may be moved to after chroot (to be tested). But the commands seem to produce an acceptable system, with almost clean ICA. You can view it at [1], only for sys V since I have not tested systemd yet (I do not expect many changes). There are pros and cons compared to the current method: pros: no /tools symlink, no need to tweak gcc sources, no need to build twice ld in binutils-pass2, no need to readjust the toolchain after chapter 6 glibc, no need to tweak the gcc specs, no need to reinstall kernel headers in chapter 6. cons: chroot is entered in the middle of chapter 5 (maybe chapter 5 should be split), the debug sections of several packages reference x86_64-lfs-linux-gnu instead of x86_64-pc-linux-gnu, binutils-pass2 needs "enable-shared". Another pro, not tried, is that some simple packages built in chapter 5 may be built only once if testing them is not required. Comments and suggestions for improvement (there's a lot of room for improvement) welcome. Regards Pierre [1] http://www.linuxfromscratch.org/~pierre/lfs-modified/index.html -- http://lists.linuxfromscratch.org/listinfo/lfs-dev FAQ: http://www.linuxfromscratch.org/faq/ Unsubscribe: See the above information page
