Re: What happend to bootstrap-lean? (was: What happened to bubblestrap?)
Hi Paolo, since we are approaching the GCC 4.2 release, I thought I'd point out the question of bootstrap-lean again, which is still documented and which I found rather useful in some settings. On Sat, 17 Dec 2005, Gerald Pfeifer wrote: On Fri, 16 Dec 2005, Paolo Bonzini wrote: Yes. make bubblestrap is now called simply make. Okay, how is make bootstrap-lean called these days? ;-) In fact, bootstrap-lean is still documented in install.texi and makefile.texi, but it no longer seems to be present in the Makefile machinery. Could we get this back? Also, would you mind adding some notes to gcc-4.2/changes.html, as far as changes are user visible? Thanks, Gerald
Re: What happend to bootstrap-lean? (was: What happened to bubblestrap?)
Because the whole point of this process is to remove all the bootstrap logic from the gcc subdirectory, which is exactly where it doesn't belong. This will let us take major steps forward in our build process How does *removing* something take major steps forward? The whole bootstrap logic you are talking about is 30 lines, if that. If we want a better process (and I certainly agree we do with respect to libraries), indeed adding a top-level bootstrap process is appropriate. By why delete what's there and works fine? If it ain't broke, don't fix it! is very prudent engineering and very much applies here. I feel very strongly that: (1) If all I want to do is a bootstrap, I should be able to do that bootstrap without having to reconfigure, maintain separate directories, rebuild files I've already built, or any other such kludge. (2) It should be straightforward to write a build script that works with all versions of GCC. It's important to keep in mind that many of us, especially those supporting customers that use GCC, have to maintain many build directories, for multiple targets and versions of GCC. Having vastly different procedures to use in different directories is a real pain. At a minimum, there needs to be some compatibility mode to support the older procedures.
Re: What happend to bootstrap-lean? (was: What happened to bubblestrap?)
On Sun, Dec 18, 2005 at 08:28:13AM -0500, Richard Kenner wrote: Because the whole point of this process is to remove all the bootstrap logic from the gcc subdirectory, which is exactly where it doesn't belong. This will let us take major steps forward in our build process How does *removing* something take major steps forward? The whole bootstrap logic you are talking about is 30 lines, if that. This shows pretty clearly that you haven't looked at it - the basic bootstrap targets are about 10% of gcc/Makefile.in, which is rather more than 30 lines. If we want a better process (and I certainly agree we do with respect to libraries), indeed adding a top-level bootstrap process is appropriate. By why delete what's there and works fine? If it ain't broke, don't fix it! is very prudent engineering and very much applies here. It's not prudent engineering when you suddenly discover you have to maintain two fragile ways of doing something instead of just one. The GCC build process is ridiculously rich and has a ridiculous number of vaguely user-tunable knobs, e.g. about six ways to specify what assembler the compiler is going to use. The more complicated your needs, the less likely they are to continue to work. ./configure make bootstrap is going to continue working, presumably forever. It's important to keep in mind that many of us, especially those supporting customers that use GCC, have to maintain many build directories, for multiple targets and versions of GCC. Having vastly different procedures to use in different directories is a real pain. At a minimum, there needs to be some compatibility mode to support the older procedures. Any such build script - we've got plenty of our own here, thanks - grows little widgets over time to handle particular configurations and particular versions of GCC, in my experience. Paolo's already explained how to handle both old and new versions of bootstrapping. We're investigating losing the configure option. But if you insist that you must continue to run 'make' in the gcc subdirectory, you won't get a bootstrap, just a rebuild of the current stage. -- Daniel Jacobowitz CodeSourcery, LLC
Re: What happend to bootstrap-lean? (was: What happened to bubblestrap?)
On Sun, Dec 18, 2005 at 12:12:17PM -0500, Richard Kenner wrote: Backwards compatibility is indeed expensive, but is critical. All vendors do it and we need to as well. You can be certain that if there were six ways of specifying something in VMS on a VAX in 1979, all six will still work today on all VMS targets. There's a reason for that: the users demand and expect it. I am perfectly willing to maintain backwards compatibility for _users_ of GCC. Most users of GCC don't build GCC, and most builders of GCC really don't use the full expressiveness of the build system. I disagree with the assertion that we need to maintain compatibility in our Makefiles. Obviously we want to in as much as it is practical; this isn't. And don't you think that talking about compatibility expected by our users is just a little bit disingenuous, when you're talking about running make inside the gcc subdirectory? Users don't do that! Only developers of GCC do. It's only useful for incremental builds; a full build of GCC always starts and ends in the top level. ./configure make bootstrap is going to continue working, presumably forever. I thought the whole point was that bootstrap wasn't a target at the top level, but just in the gcc/ directory, so that the above did *not* work. Huh? Are you talking about before, or after, the changes? bootstrap has always been a target at the top level. It used to be implemented by building a bunch of other stuff and then forwarding to a bootstrap target in the gcc subdirectory and then building another bunch of stuff, but is no longer. Any such build script - we've got plenty of our own here, thanks - grows little widgets over time to handle particular configurations and particular versions of GCC, in my experience. Sure, but the issue is the *size* of the differences. Are we talking about something subtle or a completely different sequence of make targets? Paolo already answered this question, in a message directly to you: http://gcc.gnu.org/ml/gcc/2005-12/msg00490.html But if you insist that you must continue to run 'make' in the gcc subdirectory, you won't get a bootstrap, just a rebuild of the current stage. You lost me. make in the gcc subdirectory would always build into that directory using the system compiler (i.e., what you'd normally use to build the stage1 compiler), not do a bootstrap. It was make bootstrap that would continue the bootstrap from the current location. I was referring to make $(anything), i.e. the command /usr/bin/make. In my experience, the four most common targets in the gcc subdirectory were: empty, as above bootstrap, as above compare, the obvious unstage1, to undo a make bootstrap I think those four targets need to keep working because they are also the oldest targets. I certainly agree that it's reasonable to change the way that the lesser-used targets are executed, so long as there's a clearly-documented mapping. However, I also don't see why we can't simply put that target into gcc/Makefile.in to simply recursively execute the make with the equivalent target. That does not have the maintenance burden you suggest above. Because it would have to recurse to the parent directory, which is then going to rename your current directory and do bits elsewhere, in other directories; it's likely to leave you far away from the results of your make. Do you really think that'll leave you any less confused? I'd be baffled! I hate it when things rename my $PWD. -- Daniel Jacobowitz CodeSourcery, LLC
Re: What happend to bootstrap-lean? (was: What happened to bubblestrap?)
And don't you think that talking about compatibility expected by our users is just a little bit disingenuous, when you're talking about running make inside the gcc subdirectory? Users don't do that! Only developers of GCC do. It's only useful for incremental builds; a full build of GCC always starts and ends in the top level. Sorry: my comment on that topic was directed at your comment about there being six ways to specify the assembler to use, not about building inside the gcc subdirctory. My point was that whatever we did, we needed to make sure we retained those six ways. You can always simplify if you're willing to sacrifice backwards compatibility, but I don't think we should do that. Because it would have to recurse to the parent directory, Why do you have to recurse to the parent directory to bootstrap GCC? If the desire was to make pieces elsewhere, the command would have been issued from elsewhere. I'm talking about bootstrapping the compiler, nothing else. Sure, there's a value in doing a bootstrap including the libraries, but there's also a value in *not* doing it. which is then going to rename your current directory Why would a makefile rename a directory in any situation? That seems like trouble waiting to happen to me.
Re: What happend to bootstrap-lean? (was: What happened to bubblestrap?)
On Sun, Dec 18, 2005 at 01:01:11PM -0500, Richard Kenner wrote: Because it would have to recurse to the parent directory, Why do you have to recurse to the parent directory to bootstrap GCC? If the desire was to make pieces elsewhere, the command would have been issued from elsewhere. I'm talking about bootstrapping the compiler, nothing else. Sure, there's a value in doing a bootstrap including the libraries, but there's also a value in *not* doing it. which is then going to rename your current directory Why would a makefile rename a directory in any situation? That seems like trouble waiting to happen to me. The answer to both of these questions is the same. Toplevel bootstrap deliberately - as a design decision, and in my opinion, a very good one - puts every stage in its own directory. Instead of trying to move all the object files we're staging into a subdirectory, each stage of gcc is built in its own directory. This lets us not have to guess which files need to be moved and which (e.g. generated by configure) need to be left alone. For GCC we already knew what this list is, by exhaustive trial and error. For other directories, however, we don't want to have to work it out. It lets the bootstrap mechanism be more modularly independent of the components being bootstrapped. Therefore, you get obj/stage1-gcc and obj/stage2-gcc. And via either symlinks or directory renames (and it looks like we'll have to switch back to only via directory renames) one of these is obj/gcc and another is obj/prev-gcc at any given time. -- Daniel Jacobowitz CodeSourcery, LLC
Re: What happend to bootstrap-lean? (was: What happened to bubblestrap?)
The answer to both of these questions is the same. Toplevel bootstrap deliberately - as a design decision, and in my opinion, a very good one - puts every stage in its own directory. Of course: we've always had each stage living in a different directory. You're not going to get any disagreement there. My question was why you have to go *up*. Unless you're rebuilding a library outside the gcc/ directory, all the stages can be in subdirectories of that directory. Instead of trying to move all the object files we're staging into a subdirectory, each stage of gcc is built in its own directory. I'm not sure I see the difference there, but OK, as long as it doesn't complicate anything else. This lets us not have to guess which files need to be moved and which (e.g. generated by configure) need to be left alone. For GCC we already knew what this list is, by exhaustive trial and error. As you say, we know it, so it ain't broken. That list is only very rarely changed, but if there's a concern about maintaining it, I think the straightforward approach is to take the very few things that *are* to be left alone and put them in a directory of their own. Then you can copy everything in the build directory into a stage directory. So you have ., where things are built, ./invar, which are the things that are kept the same through the process, and ./stage[123], which is where the built stages are put. For other directories, however, we don't want to have to work it out. It lets the bootstrap mechanism be more modularly independent of the components being bootstrapped. Here you've lost me completely. When you say bootstrap, I interpret that as meaning using X to build another copy of X from sources enough times so that you can compare successive versions and expect them to be the same. But what else is there in the gcc tree that can be used to build itself? So what's to be made independent? Therefore, you get obj/stage1-gcc and obj/stage2-gcc. And via either symlinks or directory renames (and it looks like we'll have to switch back to only via directory renames) one of these is obj/gcc and another is obj/prev-gcc at any given time. What's wrong with the present obj/gcc/stage1 and obj/gcc/stage2? As I said, I think renaming directories is going to cause no end of problems (having directories be renamed under people is just one example). What's the benefit? Before we made make bootstrap restart in the beginning, the Makefile was able to keep track internally of what it was up to. More recently, we used a file that stays where we are. What's so wrong with that approach that justifies renaming directories? Can't those two states (current and previous) just be variables in the Makefile that it imports from some status file?
Re: What happend to bootstrap-lean? (was: What happened to bubblestrap?)
On Sun, Dec 18, 2005 at 01:25:36PM -0500, Richard Kenner wrote: The answer to both of these questions is the same. Toplevel bootstrap deliberately - as a design decision, and in my opinion, a very good one - puts every stage in its own directory. Of course: we've always had each stage living in a different directory. You're not going to get any disagreement there. My question was why you have to go *up*. Unless you're rebuilding a library outside the gcc/ directory, all the stages can be in subdirectories of that directory. No, that's not at all what we do today. We build everything in gcc/ and shuffle the files off to other directories depending on the current stage. In the revised model, the stages are completely independent; the directories have nothing in common. For other directories, however, we don't want to have to work it out. It lets the bootstrap mechanism be more modularly independent of the components being bootstrapped. Here you've lost me completely. When you say bootstrap, I interpret that as meaning using X to build another copy of X from sources enough times so that you can compare successive versions and expect them to be the same. But what else is there in the gcc tree that can be used to build itself? So what's to be made independent? No! This is the old model of bootstrap: rebuild gcc a bunch of times with itself. The top level bootstrap model is to rebuild all the useful bits of the entire tree as a group; and repeat that as many times as necessary to be able to compare them. We bootstrap everything used in the build process. Some example consequences of this: We used to have some workarounds in the libcpp-to-gcc interface to work around the fact that we built libcpp once, with the system compiler, and then linked it to each stage of the bootstrap. Darwin had a system compiler that disagreed with the FSF GCC on the size of _Bool, I believe. Now we build the stage1 libcpp with the system compiler, build the stage1 gcc with the system compiler, link them together, and go on to build the stage2 libcpp with the stage1 gcc. We can bootstrap the assembler in a combined tree. The first stage's gcc will invoke a stage1 assembler, the second stage's gcc will invoke a stage2 assembler. This doesn't have any fundamental benefits except for thoroughness; it's an even better sanity check. Eventually I expect we'll have at least one frontend written in C++. We'll be able to build that frontend using a bootstrapped copy of libstdc++. What's wrong with the present obj/gcc/stage1 and obj/gcc/stage2? As I said, I think renaming directories is going to cause no end of problems (having directories be renamed under people is just one example). What's the benefit? Before we made make bootstrap restart in the beginning, the Makefile was able to keep track internally of what it was up to. More recently, we used a file that stays where we are. What's so wrong with that approach that justifies renaming directories? Can't those two states (current and previous) just be variables in the Makefile that it imports from some status file? It was an implementation decision to use obj/stage1-gcc instead of obj/gcc/stage1, so that (A) we didn't have to move every individual file around, and (B) we knew, robustly, that each stage's build would be completely separate from the previous stage's (no way to leave stray files behind, since the directory is created from scratch). Personally, I think this layout (which I did not pick) is superior. It makes it far more difficult to accidentally mix stages. We could change that decision, and move all the files around instead. That wouldn't change the need to hand off to the top level in order to do bootstraps though; the routines in gcc would be just for convenience. A bootstrap would need to build top level versions of helper tools and libraries. -- Daniel Jacobowitz CodeSourcery, LLC
Re: What happend to bootstrap-lean? (was: What happened to bubblestrap?)
On Dec 18, 2005, at 1:40 PM, Daniel Jacobowitz wrote: We used to have some workarounds in the libcpp-to-gcc interface to work around the fact that we built libcpp once, with the system compiler, and then linked it to each stage of the bootstrap. Darwin had a system compiler that disagreed with the FSF GCC on the size of _Bool, I believe. Now we build the stage1 libcpp with the system compiler, build the stage1 gcc with the system compiler, link them together, and go on to build the stage2 libcpp with the stage1 gcc. No it was not Darwin (this is one of the places where the FSF GCC is correct with respect of Darwin), but instead using 2.95.3 to bootstrap. When you started with 2.95.3, bool (as stdbool.h was included with 2.95.3) was an int. -- Pinski
Re: What happend to bootstrap-lean? (was: What happened to bubblestrap?)
Daniel Jacobowitz [EMAIL PROTECTED] writes: [...] | We can bootstrap the assembler in a combined tree. The first stage's | gcc will invoke a stage1 assembler, the second stage's gcc will invoke | a stage2 assembler. This doesn't have any fundamental benefits except | for thoroughness; it's an even better sanity check. And that is a good thing to have. -- Gaby
Re: What happend to bootstrap-lean? (was: What happened to bubblestrap?)
The top level bootstrap model is to rebuild all the useful bits of the entire tree as a group; and repeat that as many times as necessary to be able to compare them. Please define useful. I'm very concerned if we're doing more builds than before and don't have a way to restrict the builds to what we were doing previously since it slows down the development process, especially for those people still on slower machines. A bootstrap has a very precise purpose: you want to ensure that when you install a new compiler, you won't lose the ability to recompile it. So we ensure that everything that *has* to be compiled in order to build the compiler is. As we extend that to ancilary pieces we're rapidly getting into a process of diminishing returns and when I hear that we're making it *easy* to keep adding pieces, I worry even more about bootstrap speeds in the future. We can bootstrap the assembler in a combined tree. The first stage's gcc will invoke a stage1 assembler, the second stage's gcc will invoke a stage2 assembler. This doesn't have any fundamental benefits except for thoroughness; it's an even better sanity check. Sure, but I think it should be viewed as an *additional* check, much like we currently have two tests: bootstrap and make check. So we'd have three: a simple bootstrap, a complete bootstrap, and make check. The latter two would be a requirement for check-ins, but the former would exist to be used as part of the rough testing process. Eventually I expect we'll have at least one frontend written in C++. We'll be able to build that frontend using a bootstrapped copy of libstdc++. Sure, but I don't see what the major gain of such a thing would be over the current set of tests for libstdc++. Aren't those tests much more exhaustive a test than the usage of libstdc++ by such a front-end? Personally, I think this layout (which I did not pick) is superior. It makes it far more difficult to accidentally mix stages. I'm not sure I understand the logic of that conclusion, though I agree that the order in which the directories are named isn't a critical issue so long as the current *functionality* isn't lost. We could change that decision, and move all the files around instead. That wouldn't change the need to hand off to the top level in order to do bootstraps though; the routines in gcc would be just for convenience. A bootstrap would need to build top level versions of helper tools and libraries. Right, but it would avoid the need to rename directories and, as I said, I think that's a problem waiting to happen.
Re: What happend to bootstrap-lean? (was: What happened to bubblestrap?)
On Fri, 16 Dec 2005, Paolo Bonzini wrote: Yes. make bubblestrap is now called simply make. Okay, how is make bootstrap-lean called these days? ;-) In fact, bootstrap-lean is still documented in install.texi and makefile.texi, but it no longer seems to be present in the Makefile machinery. Could we get this back? bootstrap-lean is done by doing the following (which I feel is the wrong way): Configure with --enable-bootstrap=lean and then do a make bootstrap And yes this causes to use different scripts for different versions of gcc. Thanks, Andrew Pinski
Re: What happend to bootstrap-lean? (was: What happened to bubblestrap?)
Andrew Pinski [EMAIL PROTECTED] writes: | | On Fri, 16 Dec 2005, Paolo Bonzini wrote: | Yes. make bubblestrap is now called simply make. | | Okay, how is make bootstrap-lean called these days? ;-) | | In fact, bootstrap-lean is still documented in install.texi and | makefile.texi, but it no longer seems to be present in the Makefile | machinery. Could we get this back? | | bootstrap-lean is done by doing the following (which I feel is the wrong way): | Configure with --enable-bootstrap=lean | and then do a make bootstrap Hmm, does that mean that I would have to reconfigure GCC if I wanted to do make bootstrap-lean after a previous configuration and build? I think the answer must be no, but I'm not sure. -- Gaby
Re: What happend to bootstrap-lean? (was: What happened to bubblestrap?)
bootstrap-lean is done by doing the following (which I feel is the wrong way): Configure with --enable-bootstrap=lean and then do a make bootstrap I agree with you. Why not just keep this as a make target? Why go back and have to reconfigure? And yes this causes to use different scripts for different versions of gcc. I think that's a serious problem. I see no argument against upwards compatibility here: it's all well and good to add new options to the toplevel, but why remove ones from the gcc/ Makefile that have worked for decades and people still use?
Re: What happend to bootstrap-lean? (was: What happened to bubblestrap?)
On Sat, Dec 17, 2005 at 09:41:16PM -0500, Richard Kenner wrote: bootstrap-lean is done by doing the following (which I feel is the wrong way): Configure with --enable-bootstrap=lean and then do a make bootstrap I agree with you. Why not just keep this as a make target? Why go back and have to reconfigure? This is a known shortcoming. We've been talking about how to change it. And yes this causes to use different scripts for different versions of gcc. I think that's a serious problem. I see no argument against upwards compatibility here: it's all well and good to add new options to the toplevel, but why remove ones from the gcc/ Makefile that have worked for decades and people still use? Because the whole point of this process is to remove all the bootstrap logic from the gcc subdirectory, which is exactly where it doesn't belong. This will let us take major steps forward in our build process; one it's already taken is letting host libraries and tools (like libcpp) be bootstrapped along with the compiler. Another one it will take, soon, is letting target libraries be bootstrapped instead of the huge amount of cruft we have inside the gcc directory to handle libgcc. They've continued working for decades because up until now, no one's been brave enough to try to rework them :-) -- Daniel Jacobowitz CodeSourcery, LLC
